The goal in this article is to use rsync to back up multiple source directories to a single destination. There are better guides and documentation on the Internet than this article.

There are a few requirements for this task:

  • Delete or add source directories in the future
  • Addition or deletion of a source directory must sync to the destination
  • Exclude files/directories as needed

For the sake of an example pretend that /etc (excluding /etc/default), /usr/local/bin, and /home/will/Downloads are being backed up to some destination at /tmp/backup. 

/etc (but not /etc/default)  ===> \
/home/will/Downloads         ===>  > /tmp/backup
/usr/local/bin               ===> /

Using Filters

rsync filters are a helpful mechanism for defining a canonical list of source paths to sync (or not) to a destination. There is a little verbosity in the configuration of filters, but overall they are fairly clean and robust.

Somehow, define a filters configuration file. This is what a configuration file may look like for the goal of this article’s example. 

+ /etc/
- /etc/default/**
+ /home
+ /home/will
+ /home/will/Downloads
+ /home/will/Downloads/**
+ /usr/local/bin/**
- /**

Note a few interesting things here.

  • A build-up is necessary to sync subdirectories. + /home, and + /home/will and + /home/will/Downloads
  • An explicit exclude of - /** at the end prevents unlisted unwanted data from being synced
  • An exclude of - /etc/default/** allows syncing /etc without the /etc/default subdirectory

When running rsync the definitions in filters.txt are merged. The source directory is / and the destination is /tmp/backup. 

sudo rsync \
  --verbose \
  --progress \
  --recursive \
  --archive \
  --delete \
  --delete-excluded \
  --filter="merge /path/to/configuration/filters.txt" \
  / \
  /tmp/backup/

This handles all the requirements listed above. The filters can be updated to remove or add directories, subdirectory excludes can be defined, and the destination will be updated accoringly when the configuration changes.

Non-Filters Alternative: Multiple Sources

Another possible mechanism for syncing is to use multiple source directories. 

sudo rsync \
  --verbose \
  --progress \
  --recursive \
  --relative \
  --exclude /etc/default \
  --archive \
  --delete \
  --delete-excluded \
  /etc /home/will/Downloads /usr/local/bin \
  /tmp/backup/

This works initially, but fails one of the requirements listed above. If /usr/local/bin was removed as a source path it would not automatically be deleted from /tmp/backup. Manual intervention would be required.

Non-Filters Alternative: Exclude and Include Rules

Another possible mechanism for syncing is to use a single / source path like with filters in the example above, but using --include and --exclude rules rather than any filters. 

sudo rsync \
  --verbose \
  --progress \
  --recursive \
  --include "/etc" \
  --exclude "/etc/default" \
  --include "/etc/*" \
  --include "/etc/**/*" \
  --include "/usr" \
  --include "/home" \
  --include "/home/will" \
  --include "/home/will/Downloads" \
  --include "/home/will/Downloads/*" \
  --include "/home/will/Downloads/**/*" \
  --include "/usr/local" \
  --include "/usr/local/bin" \
  --include "/usr/local/bin/*" \
  --include "/usr/local/bin/**/*" \
  --exclude="*" \
  --archive \
  --delete \
  --delete-excluded \
  / \
  /tmp/backup/

With this approach a source path can be removed from the config and that change will sync to the destination, but it is more verbose than using filters.

There may be abbreviated ways to accomplish what is done in this approach. Functionally, this is very similar to using filters, but filters seem better suited to the task.