Wednesday, September 7, 2011

Incremental backups with btrfs

btrfs is teh win.  No, seriously.  It's not the only file system that can do this, but it's the first one I've had installed, and it's beautiful, man.

I got myself a little 2TB external USB 3.0 hard drive ($99!  Past Sean, be very jealous) and wrote a backup script; my first version had all of this complex Towers of Hanoi rotation scheme, but then I realized I didn't need any of it if I used btrfs's snapshots.  Now, my backup script consists mainly of:

sg_start --start /dev/sdh   # Tell the drive to spin up
mount /dev/sdh1 /mnt/backups
btrfsctl -s /mnt/backups/backup-$todays_date \
            /mnt/backups/backup-$yesterdays_date
rsync -va --numeric-ids --delete-before --ignore-errors \
      --partial --inplace $backup_paths \
      /mnt/backups/backup-$todays_date
sync
umount /mnt/backups
sg_start --stop /dev/sdh

It does a little more than that (error checking, logging, etc), but not much more.  After the first backup, the disk usage was 15GB; ten days later, I have ten incremental backups, and it's still 15GB.

Of course, the problem with this is that snapshots are, essentially, COW hard links; this means that if there's a corruption on the disk for a file, it'll affect all child snapshots.  My mitigation is to add another backup disk: they're only $99 (cripes, I can't get over that price), and each backup is taking about 8 minutes to run (most of that time is spent in rsync, detecting changes) -- I can easily affordable to run two backups a night.  It's not as safe as a rotation backup, but it's safe enough.

No comments: