vBulletin to Discourse conversion.

I’m in the process of trying to move from vBulletin on a forum to Discourse and I’m having a bit of a hard time. There are no instructions on running the importing script so its taking a bit longer than it should. I’ll work to document what I’m trying in this post to help others who might have a difficult time.

Discourse looks great and works really well. Plus combined with docker its very simple to setup from scratch, upgrade, backup, and restore. Plus its free and vBulletin hasn’t really added any great features and Discourse is kicking its ass. The forum in question is currently at: technobadger.com, which used to be the old forums at psx-dude.net — my first domain from long ago.

I first started by following the docker instructions for discourse to get it setup:

I made a backup of my vBulletin MySQL database and copied it onto my test machine into a database called `technobadger_production`. Don’t forget to create a user that can access remotely.

[code]
mysql> GRANT ALL ON technobadger_production.* TO ‘technobadger_r’@’%’
IDENTIFIED BY ‘sX55ot6FJ#q#3%Ms’ WITH GRANT PRIVILEGES;
mysql> FLUSH PRIVILEGES;
[/code]

You’ll also need to let the mysql server listen for remote connections. You can edit the /etc/mysql/my.cnf and comment out the following line. Don’t forget to restart mysql for the changes to take effect.
[code]# bind-address 127.0.0.1[/code]
I copied the attachments folder into `/home/chris/attachments`. I copied the attachments into the container:
[code]
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b7764d41d245 local_discourse/app:latest "/sbin/boot" 29 hours ago Up 29 hours 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:2222->22/tcp app
[/code]
I copied changed into the root of the container:
[code]
cd /var/lib/docker/aufs/mnt/b7764d41d245394cf5d7f7e44cdd2c7bd2866fe2a42dd5572357e4fb760a3c7a
mkdir attachments && cd attachments
cp -R /home/chris/vb-attachments/attachments/* ./
[/code]
Change into the discourse directory: `cd /var/discourse` and launch into the docker:
[code]
./launcher enter app
cd /var/www/discourse
apt-get update && apt-get install libmysqlclient-dev
gem install mysql2
[/code]
I edited the script/import_scripts/vbulletin.rb file and changed the top lines to represent the database i installed on the container host (outside of the container) and added an entry for password. 10.0.0.128 is the ip of my main server that has mysql on it.

 

[code]
class ImportScripts::VBulletin < ImportScripts::Base
BATCH_SIZE = 1000

# CHANGE THESE BEFORE RUNNING THE IMPORTER
DATABASE = "technobadger_production"
TIMEZONE = "America/Los_Angeles"
ATTACHMENT_DIR = ‘/attachments’

def initialize
super

@old_username_to_new_usernames = {}

@tz = TZInfo::Timezone.get(TIMEZONE)

@htmlentities = HTMLEntities.new

@client = Mysql2::Client.new(
host: "10.0.0.128",
username: "technobadger_r",
password: "sX55ot6FJ#q#3%Ms",
database: DATABASE
)
end
[/code]

I edited the Gemfile in `/var/www/discourse` and added the following line to the end:
[code]gem `mysql2`[/code]
and then ran…
[code]bundle install –no-deployment[/code]
I modified the Postgres config file (/etc/postgresql/9.3/main/pg_hba.conf) to change everything to trust for the import. Obviously don’t use this on your production server as that could be bad.
[code]
# DO NOT DISABLE!
# If you change this first entry you will need to make sure that the
# database superuser can access the database using some other method.
# Noninteractive access to all databases is required during automatic
# maintenance (custom daily cronjobs, replication, and similar tasks).
#
# Database administrative login by Unix domain socket
# local all postgres peer # Chris: Replaced with line below
local all postgres trust

# TYPE DATABASE USER ADDRESS METHOD

# "local" is for Unix domain socket connections only
#local all all peer # Chris: Replaced with line below
local all all trust
# IPv4 local connections:
host all all 0.0.0.0/0 md5
# IPv6 local connections:
host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication postgres peer
#host replication postgres 127.0.0.1/32 md5
#host replication postgres ::1/128 md5
[/code]
I had to modify the importer script because some of my avatars didn’t have a file extension or were empty. Below is a diff of my changes for this file:

[code]
@@ -110,7 +111,11 @@ class ImportScripts::VBulletin < ImportScripts::Base
file.write(picture["filedata"].encode("ASCII-8BIT").force_encoding("UTF-8"))
file.rewind

– upload = Upload.create_for(imported_user.id, file, picture["filename"], file.size)
+ if picture[‘filename’].nil?
+ upload = Upload.create_for(imported_user.id, file, "anyfile.jpg", file.size)
+ else
+ upload = Upload.create_for(imported_user.id, file, picture["filename"], file.size)
+ end

return if !upload.persisted?

[/code]

I also had some forums that didn’t have parents in them. I had to update them manually:

[code]
mysql> SELECT forumid, title, description, displayorder, parentid FROM forum WHERE parentid NOT IN ( SELECT forumid FROM forum ) AND parentid <> -1 ORDER BY forumid;
+———+——————-+———————————————————————————————————————————————————————–+————–+———-+
| forumid | title | description | displayorder | parentid |
+———+——————-+———————————————————————————————————————————————————————–+————–+———-+
| 15 | Sub-forum 1 | Sub-forum1 description | 1 | 14 |
| 16 | Sub-forum 2 | Description | 1 | 14 |
| 19 | Talk about TDRM | | 1 | 18 |
| 20 | Editing | | 1 | 18 |
| 21 | Clans | | 1 | 18 |
| 23 | Newbie Discussion | | 1 | 22 |
| 26 | My Scripts | Having a problem with one of my scripts, want to see other people using it, or have a comment or request for a feature. Or still even have a code hack, post it here. | 1 | 25 |
+———+——————-+———————————————————————————————————————————————————————–+————–+———-+
7 rows in set (0.00 sec)

mysql> UPDATE forum SET parentid = 1 WHERE forumid IN (15,16,19,20,21,23,26);
Query OK, 7 rows affected (0.00 sec)
Rows matched: 7 Changed: 7 Warnings: 0
[/code]

Then I ran the following under the /var/www/discourse
[code] # rails runner script/import_scripts/vbulletin.rb[/code]