Ubuntu Server – How to Sync Files Between Multiple Web Servers

Another installment in our How to Set up an Ubuntu Server Series:

Synchronizing multiple servers - You have more than one web server in your host configuration. What’s the easiest way to synch files among the multiple web servers?
Solution - rsync does the trick efficiently and easily.

This tutorial assumes you are comfortable on the linux command line.

Steps to Set up rsync for multiple servers synchronization:

Step One

First make sure rsync is installed on each server that you’ll be working with.  By typing rsync from a command prompt, you should get the rsync response asking you for parameters.  If rsync isn’t found, then install it.

sudo apt-get install rsync

Step Two

Test synchronizing files.  From a command prompt on your 2nd server, type in an rsync command to retrieve files from the 1st server.  Let’s assume you have a directory /var/www/ set up on both servers and you wish to synchronize the files from the first server to the 2nd. The first server acts as the master.  Any new files or changes to existing files on the master will overwrite existing files on the 2nd server.  Type in from the command prompt on server 2

sudo rsync -ra user@FirstServerIP:/var/www/ /var/www/

The first server in this example sends all the files in /var/www/ along with the (-r) recursive subdirectories. You can also sync only specific files, etc…
If this works, you are ready to move on to step 3. If not, check your connectivity with the FirstServer. Generally, if you can ssh to another server, you should be able to rsync with it as well. AWS EC2 Users If you are using AWS Amazon’s ec2 servers, which require a passkey, the rsync command will look like this:

rsync -rave “ssh -i /home/ubuntu/.ssh/passkey.pem” ubuntu@EC2ServerIP:/var/www/ /var/www/

Note that for AWS EC2, you will first need to copy your passkey over to your .ssh directory

Step Three

Next, let’s set this up to synchronize all the time automatically. We accomplish this with a cron job. Here again, we’re assuming you are familiar with how to set up and edit crons. From your root user (sudo su) edit your crontab

crontab -e

Then, add a line to the crontab that tells it to run the rsync command every minute (or any frequency that suits you).

* * * * * rsync -ar FirstServerIP:/var/www/* /var/www/

Test your configuration!  Go to your first server and make a change or add a file to the synchronized directory. Then go back to the 2nd server and see if the file is there. You can replicate this process to as many servers as you need, but keep in mind that frequency of sync and amount of data passing on your internal network can get costly and cause a hit on performance.

I hope this little tutorial was helpful.  Please drop me a note if you have any questions or comments.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>