How To Migrate A Massive WordPress Site

A client of mine called me and told me their WordPress website was growing too big for the server it was on, even though it was a VPS, that the WordPress site was going offline periodically.  They asked me if I could help them migrate this gigantic site.

When I refer to gigantic and WordPress in the same sentence, to me that simply means that instead of megabytes of data in the database they are dealing with gigabytes of data in the database.  And instead of thousands of files, there are let’s say hundreds of thousands of files.

Small Biz Now Has Enterprise Level Issues

Luckily I have dealt with both the small and large servers over the years.  The real question has always been, can WordPress handle this high volume of DB and File issues.  Well, it can, but the issues have to be thought out and a problem like this turns from dealing with minor WordPress hick-ups to corporate-level Unix issues.

Why They Had To Migrate

Their server was just not able to handle what they were doing in terms of server speed, hard drive space, etc.  And this was causing the website to be unmanageable.  So, it was time to migrate. However, this was not your run of the mill WordPress migration.  It had all kinds of potential problems associated with it, like thousands of uploaded files, hundreds of thousands of records in some tables.

Why My Normal Tools Won’t Work

For instance, the database was too big to download using PHPMyAdmin.  Not that this is a big deal, it’s just if you have been a WordPress person for the last 5 to 10 years like me, you may not have all the tools in the tool chest to deal with this. I typically use a desktop ftp program to do this stuff, but not for this size. Using local ftp clients means downloading to your Mac or PC. It is not recommended at this level. For instance, if you are not on a fast enough download client, you are screwed…

The Tools

The reason I am writing this blog entry is so I can grab all those tools again in the following order in order to do this kind of migration again.

Here Are The Steps I took:

  1. Sized out the proper new server for the client. (My preference is StormOnDemand by Liquid Web, WHM/Cpanel cloud)
  2. Had the client buy the new server and handed over to me the login credentials.
  3. Received all the login credentials I need from the original server.
  4. Created a new website in WHM/Cpanel, generating a new IP.
  5. Created a temporary domain name for the new site and pointed the DNS to the IP.
  6. Created a Database Name, Database User in CPanel with the same name as the old DB Name and User.
  7. From the Shell command line (I use putty.exe) I used the tar command to zip up all the files beneath public_html.
  8. From the Shell command line I ran a mysqldump of the old database, creating a command line .sql file.
  9. From the Shell command line I ran FTP as new account name (not root) on new server and Put the tar file (like files.tar.gz) onto the new server.  (it was in the multigigs)
  10. From the Shell command line I ran FTP and Put the .sql file onto the new server. (it was over a gig)
  11. On the new server from the shell I unzipped the tar file.  Remember to position this right above where you want it to land.
  12. On the new server I switched over to root user (su root) to be able to import mysql.
  13. On the new server from the shell I ran a mysql import command which updated the database.
  14. Finally, while in test mode, there were two lines wp_options which need to point to the right domain.
  15. Make sure all the previous plugins are activated.
  16. We tested the new site with the temp domain name.
  17. Once we were ready, we switch the old IP at the DNS record to the new IP.
  18. Final thing is to rename the site name from the temp domain name to the actual domain name.
  19. Everything went relative smoothly and the site migrated.

That is all the actions I took, now here are the commands I used specifically:

he tar command to zip up the files to a .gz from right above the public_html directory:

The tar command to unzip the files from a .gz right above the new public_html directory:

The FTP commands I used to connect from Linux server to Linux Server:

The mysql command I used to dump the entire mysql database at the command line on the old server:

The mysql command I used import the entire mysql database at the command line on the new server:

Conclusion

Hopefully if you are reading this, you will use this to reduce the amount of time I took looking around the web for the right commands to do this. You may have to look up those other websites, because each of these commands have dozens of options. For instance, if you need to move more than one file at a time, using FTP, you would have to use mput and not put. So, keep up the good work. I hopefully will use this guide for myself next time.

Setting Up * Wildcard Domains On WHM/Cpanel

Are you looking to set up a wildcard domain like x.example.com and  y.example.com and have every other iteration land right at the same home directory in your web hosting account?  Or let me be more specific, are you trying to set up a new sub domain that will appear no matter what you put in.  For instance you have set up the whatever.example.com and now you want to always have any new “text” in front of the domain but have them all go to a single directory like test.example.com?  And finally do you use WHM/Cpanel?

If you have landed on my blog to figure this mystery out, I am going to reference some other discussion boards and blogs that give you specific instructions, but I will take you step by step what I did to set this up for myself and clarify some of the smaller items.

As far as WHM Cpanel is concerned, I believe you need to manage WHM directly.  So if you are using Cpanel, you are not going to be able to do this without the hosting company or administrator getting involved.  And then you have to use either a unix command line or ftp to make the final changes typically as root access.  That’s because there will be changes necessary to the apache configuration and the Cpanel user does not have this level of access.  While I say that, there still may be some way of using .htaccess to do this as a hack, because .htaccess is an extension of apache httpd.conf.  But, it would be a total hack, and I don’t believe it can be done because the wildcard implementation requires a Virtual Host entry change that you can’t do yourself.

Here are the steps:

1. Configuring Apache for the wildcard

This is the most difficult part of setting up a wildcard implementation on WHM/Panel.  It took me a few attempts to get this right and figure out what appears to be some confusing and not so helpful information.  Start by getting to the apache configuration and reviewing the current httpd.conf file.  It will be located at /usr/local/apache/conf/ directory.  I use the Linux command line or shell to move around, but you could ftp in as root and move over to this directory and view the httpd.conf file.

You may or may need to set up the basic domain name as full Cpanel account first.  This will create the directory and account you want the visitors to see as your root directory, plus WHM Cpanel configures things nicely. So if your domain name was example.com you would create a new account for this domain with a new user.  This will at least get you ready and get you a directory with files.

This instructional link gave me some insight into what you can and can not do in WHM/Cpanel.

Specifically he mentioned that you can not edit the httpd.conf file directly.  If you do, and rebuild the apache config file, you WILL LOSE your changes.  Below is from the apache documentation on setting up virtual hosts.  Notice this line:

<VirtualHost *:80>
    # This first-listed virtual host is also the default for *:80
    ServerName example.com
    ServerAlias *.example.com 
    DocumentRoot /www/domain
</VirtualHost>

# DO NOT EDIT. AUTOMATICALLY GENERATED.  IF YOU NEED TO MAKE A CHANGE PLEASE USE THE INCLUDE FILES.

I of course ignored the warning on writing to apache httpd.conf directly and did 
write it to test things out. So, if you are looking to add an additional alias 
like a *.example.com you can test things in the original file, but be aware you 
will lose your changes.  

There are 2 situations I ran into for wildcard domain setups. The first is setting 
up a wildcard for a domain that you have already configured.  For the situation where 
the wildcard is for an existing domain you have to write to 

/usr/local/apache/conf/userdata/std/2/userexample/example.com/*.conf

You can add 

ServerAlias *.example.com 

into a file.  They recommended calling the file extra.conf, but basically all files in this 
../user/example/example.com/ directory will get included when apache is restarted.

For the user name or "userexample" that becomes the "Account" user name.  example.com 
becomes the directory.  Yes, you have to create this domain tree and directories.  
Use mkdir to do this.  I believe only ../userdata/ existed. I had to create the rest 
of the directories manually.

The second situation is where you want to add a completely new wildcard domain and point it to
a specific directory.

Notice these lines at the bottom of the httpd.conf:

Include “/usr/local/apache/conf/includes/post_virtualhost_global.conf”
Include “/usr/local/apache/conf/includes/post_virtualhost_2.conf”

You can write lines in these files like (match the IP address of the hosting account you are point to):

<VirtualHost xx.yy.zz.aa:80>
    ServerName www.example.com
    ServerAlias example.com 
    DocumentRoot /www/domain
</VirtualHost>

Either way, you have to think a little to do this type of wildcard implementation, especially
on WHM/Cpanel.  Restart apache on Whm once you have pointed the proper A record on DNS.

2. The DNS entry
So let's talk about the DSN record set up.  Let's say you use Godaddy for your DNS.  I 
don't, but that's not important. You go under the A record under DNS and make sure there
is a wildcard or "*" placed as the domain name, instead of an example.com or www.example.com.  
In fact you can just remove thee www.example.com if you are going to use a star.  If it 
is already set up, you are good.  If you need to set it up, just drop in a star instead
of the domain name.

Restart apache under WHM and good luck.

 

 

Follow Dan Gudema on Google+!