Tuesday, December 12, 2006

Using Rsync for Remote Backup

I've been using Rsync for a while now for keeping a backup of my stuff on a remote server. Rsync is an open source program for fast incremental file transfer. It can be used to synchronize two remote directories and to synchronize a local folder with a remote folder (which is what I use it for). This also allows me to work on different machines on the same stuff: I download (or update) the stuff from the remote server, work on it, and the commit the changes back on the server.

Notice that rsync can only synchronize in one way (a source with a target, or the other way round, but not in both directions simulataneously); if you need a two way file synchronizer you might want to take a look at Unison.

Anyway, this is how I update my local version of a directory with the version on the server:

rsync -e ssh -avzu --delete -c -C \
--exclude-from=exclude-files \
USER@REMOTEMACHINE:REMOTEDIR LOCALDIR
where
  • -e ssh instructs rsync to use ssh as a remote shell (thus authentication and transfer will use ssh)
  • -a is archive mode (refer to documentation): basically recurs into directories and keeps permissions
  • -v increments verbosity
  • -z compresses the transferred data (very useful when using slow connections)
  • -u skips files that are newer on the receiver
  • --delete deletes files that are in the local folder but not in the remote folder
  • -c uses checksums to determine which files need to be synchronized
  • -C auto-ignores files in the same way CVS does
  • --exclude-from excludes all the files (using also regular expressions) specified in the file exclude-files.
  • USER is my user name on the REMOTEMACHINE
  • REMOTEDIR is the complete path of the folder on the remote machine and LOCALDIR is the path of the local folder (IMPORTANT: do not append the final slash on these paths otherwise the semantics changes, see the documentation)
Another useful option is -n or --dry-run that only simulates the transfer (so that you can see which files would be transferred).

And exclude files contains these regular expressions:
*~
*.dvi
*.aux
*.log
*.bbl
*.blg
*.toc
*.bak
*.idx
*.ilg
*.ind
*.class
*.out
core
But of course you can modify it as you see fit.

When I want to commit my local changes to the remote server I basically run the above command inverting source and destination:
rsync -e ssh -avzu --delete -c -C \
--exclude-from=exclude-files \
LOCALDIR USER@REMOTEMACHINE:REMOTEDIR
This is only one use of rsync. I refer to the documentation for more advanced usage. Hope you enjoy rsync as much as I do :-)

No comments: