The Rsync command

Rsync can be a great tool.I use it when I want to maintainidentical copies of files on two different computers (or media).I have used it for backups when I have two computers connected by a reasonably fast link.I have also used it with removable media (such as a big external USB hard drive)in much the same way. The BIG advantage of using rsync is that isfigures out how to make the two copies the same in a minimal and highlyefficient way (i.e. without just doing a brute force copy).

Rsync, however is terribly overfeatured and has more options and modesthan anyone can keep straight. These are simply my notes about how to use the few things that Ifind a regular need to use. I rarely use it across the network from one machine to another,although it is good to know that it can do such things and that it even supports its own protocolfor this kind of thing (when I have done this, I just use an ssh connection).

My typical use of rsync looks like one of the following commands:

rsync -av /home/wally /archive
rsync -av /home/wally/ /archive/wally

The difference between these two commands has bitten me many timesand is one of my main motivations for writing this little note.
Let's look at these one at a time:

1. rsync -av /home/wally /archive
   

   The above command will create (if it does not exist already) the directory /archive/wallyand recursively copy everything into there. This is just the thing if you are doing a first time copy.This is usually not what I want and makes me angry by producing a new directory.

2. rsync -av /home/wally/ /archive/wally
   

   This command will NOT create the wally directory inside /archive, but expects thesource and destination directories to be "equivalent".This is just the thing if you want to make a prior copy up to date,and usually matches how I think about things.

Also take note of the -av switch. The v option just says to be verbose andtell about every file that gets transferred. The a option is a composite optionthat is the same as -rlptgoD which is in general "just what I want".When I forget the -av switch, I get the confusing message:

You also may want to consider adding the -u (update) switch.This tells rsync to skip files that have newer timestamps at the destination.

skipping directory wally

Useful options

Things may go faster (though I am not overwhelmed) if you give the --size-only option.This simply uses file sizes to decide if a file has changed or not. This avoids calculatingcryptographic checksums, which would be useful to detect files whose contents had changed, butwhose sizes had not. Certainly calculating such checksums in computationally intensive, buta large rsync is dominated by IO times, and for a small rsync, who cares?This switch could also be useful if timestamps between two machines could simply not be trusted.

--bwlimit=2000 says to bandwidth limit what goes on in unitsof kb/second, which is nice for gigantic transfers over shared links.

--delete says to delete files on the receiver that are not onthe source. I do not routinely do this, but it is the thing to do whenyou do something like rearrange the directory structure on the source.

Consider the -u option (and thus using -auv instead of -av).This says to avoid overwriting a destination file witha newer modification time than the source, the idea being that thefiles are different, but that the destination file is more recent and we don't want tooverwrite it with an old copy from another directory or machine. In general the right thing.As an example, I use the following command to keep a backup copy of all my photos on agood sized external USB hard drive:

rsync -auv /home/Camera/photos /media/disk

Another useful switch is the --exclude-from file option. This allows you to specify files(and/or directories) relative to the source path that should be excluded from the copy.You use your favorite editor to create a file (pathnames relative to the source directory)that should be excluded from the sync.

For example if you were doing the following:

    rsync -av --exclude-from=exfile /backuproot/ /

Then the exfile might look like (along with other things)

dev
proc

Remote rsync

First, set up ssh keys so you can use ssh to go from one machine to the other without a password(or be prepared to supply a password when you are prompted for one).The game can be played in either of the following ways (fetching or putting):

rsync -auv -e ssh remotehost:/home/wally /local/archive

rsync -auv -e ssh /local/archive/wally remotehost:/home

All the same screwy business of the trailing slash and creating new directories mentioned above still applies.