Tag Archive for 'rsync'

Force rsync to use delta transfer to fix corrupt remote file

We host our Korora Project ISO images on SourceForge and I (naturally) use rsync to move them there (slowly, at 100kb/sec). Sometimes though the connection drops off and that’s OK because rsync picks up where it left off.

However occasionally the ISO ends up with the wrong checksum, so something went wrong in the transfer. No amount of re-rsyncing seems to fix this up because by default it uses file size and timestamps to check whether it should skip existing files.

Fortunately, I don’t need to re-send the whole file again as rsync can perform a delta transfer instead and only send the small difference. Yay!

The way I do this is by passing a combination of options to rsync, such as –checksum (to enable transfer of the file), –in-place (to transfer the file in place as rsync normally writes a temporary file, then moves) and –no-whole-file (which tells rsync to not copy the whole file, but use deltas instead).

This becomes something like:
rsync -Pa --checksum --inplace --no-whole-file local.file remote.server:

Here’s a real example:
chris@x220 ~ $ rsync -Pa --checksum --inplace --no-whole-file -e ssh korora-20-i386-cinnamon-live.iso csmart,kororaproject@frs.sourceforge.net:/home/frs/project/k/ko/kororaproject/20/
sending incremental file list
  1,715,470,336 100% 220.87MB/s 0:00:07 (xfr#1, to-chk=0/1)

So rsync just saved me 4 hours of uploading the ISO again. Thanks rsync.

Non-standard SSH port and rsync

I use rsync to move most of my data around the place (who doesn’t!?), but sometimes I have to copy files to a server where SSH is running on a port other than 22. How do I tell rsync to use a different port for SSH? Like this:

rsync -e "ssh -p [remote-ssh-port]" ~/local-files/ user@remote-server:remote-files/

Handy indeed. Thanks rsync!

I also like to pass “-P” to the command so that I get a nicer progress than verbose mode, so:

rsync -Pae "ssh -p [remote-ssh-port]" ~/local-files/ user@remote-server:remote-files/

I’m sure that most users will already know this, but it’s handy to put on my blog as a reference for my ageing brain!