Off-Site Backup with Borgmatic
As the saying goes “RAID is not a backup”. So I needed a backup solution for my new NAS. Some of my files are already synced with my Google drive1) but I needed proper backup for the rest2).
For my personal server setup, I had decided on Borg and borgmatic. So that's what I did for my NAS as well. Here's how.
Hetzner Storage Box
The first question is where to store the backup data. I picked Hetzner's storage box which provides good value for a low price and has all the sync options you'd want.
To use it for borg backups, you need two things:
- a directory where the borg repo will reside
- a SSH key that can connect password-less
My borg backups are to be stored in /home/borg/<hostname>
. So I created a directory for my NAS:
ssh -p 23 uXXXXXX@uXXXXXX.your-storagebox.de 'mkdir -p /home/borg/nas'
I will run borgmatic via docker in /data/docker/borgmatic
. The ssh key will be mounted as a volume. I decided to create a dedicated key just for the backup:
mkdir -p /data/docker/borgmatic/conf/ssh ssh-keygen -t ed25519 -C 'borgmatic@nas' -f /data/docker/borgmatic/conf/ssh/id_ed25519
To add it to the authorized_keys entry in the storage box, we can use ssh-copy-id with the -s option to use the SFTP protocol:
ssh-copy-id -i /data/docker/borgmatic/conf/ssh/id_ed25519 -p 23 -s uXXXXXX@uXXXXXX.your-storagebox.de
Borgmatic Docker Compose Setup
I am using the borgmatic-collective Docker image with Docker Compose. The compose.yaml is relatively straight forward. Besides a few mounts for borg to store its caches and such, you need to mount the ssh key directory and a directory where the configuration will reside. And of course the container needs access to all data it should back up.
Simply mount the directories to back up into the /mnt/source/
dir in the container.
- borgmatic/compose.yaml
services: borgmatic: image: ghcr.io/borgmatic-collective/borgmatic:2 container_name: borgmatic restart: unless-stopped volumes: # mount folders to backup into /mnt/source - /data/docker:/mnt/source/docker:ro - /data/photo:/mnt/source/photo:ro - /home/andi:/mnt/source/andi:ro # config - ./conf/borgmatic:/etc/borgmatic.d/ - ./conf/ssh:/root/.ssh # state - ./volumes/borg:/root/.config/borg - ./volumes/cache:/root/.cache/borg environment: TZ: Europe/Berlin BORG_PASSPHRASE: ${BORG_PASSPHRASE} HEALTHCHECKS_PINGURL: ${HEALTHCHECKS_PINGURL}
In conf/borgmatic/
are two files: a crontab.txt to define when the backup should be run and a borgmatic configuration file.
- borgmatic/conf/borgmatic/config.yaml
source_directories: - /mnt/source/ repositories: - path: ssh://u408911@u408911.your-storagebox.de:23/home/borg/nas/repo label: hetzner # we disable this to not have to specify all the mounts individually: one_file_system: false # Passphase is set in varibable $BORG_PASSPHRASE # encryption_passphrase: "DoNotForgetToChangeYourPassphrase" compression: zlib archive_name_format: 'backup-{now}' exclude_patterns: - '*/log/*' - '*/logs/*' - '*.log' - '*/cache/*' - '*.cache' # Exclude directories that contain a CACHEDIR.TAG file. See # http://www.brynosaurus.com/cachedir/spec.html for details. Defaults # to false. exclude_caches: true # Exclude directories that contain a file with the given filenames. # Defaults to not set. exclude_if_present: - .nobackup # what to keep keep_daily: 1 keep_weekly: 1 keep_monthly: 1 checks: - name: repository frequency: 2 weeks - name: archives frequency: always - name: extract frequency: 2 weeks - name: data frequency: 1 month apprise: states: - start - finish - fail services: - url: mailto://192.168.1.33?to=splitbrain@gmail.com&from=borgmatic@69b.place label: mail start: title: "⚙️ Backup nas: started" body: Starting backup process. finish: title: "✅ Backup nas: SUCCESS" body: Backups successfully made. fail: title: "❌ Backup nas: FAILED" body: Your backups have failed. healthchecks: ping_url: ${HEALTHCHECKS_PINGURL}
Most of the stuff is self explanatory, but you can always reference the manual.
But let me point out a couple of choices I made:
I set one_file_system: false
. Because we mount different directories into /mnt/source
, each of them is technically a different file system. You could specify each one in the source_directories
array, but I prefer to only specify what I want to backup once – in the compose.yaml
.
I exclude a couple of typical cache and log file directories and also make use of the .nobackup
marker. The latter is useful to exclude a few things in my home directory.
Finally I use two different notification mechanisms to know when the backups fail.
The apprise
config sends mails. Currently I let it send mails for all occasions, just to keep an eye on the backup. Once I am sure it works as I want it, I will remove the start
and finish
states.
The healthchecks
entry makes use of the healthchecks.io Service. The nice thing about that service is that it will also send you an alert when no data is received for a while. This will let you know when your cron job is not running for any reason (eg. your container was accidentally stopped). The URL itself is defined as an environment variable.
The very last bit of configuration you need, is an .env
containing your backup passphrase. Something like this:
- borgmatic/.env
BORG_PASSPHRASE='My secrete passphrase!!1!1' HEALTHCHECKS_PINGURL='https://hc-ping.com/xxx-xxx-xxx-xxx'
Finally start the container:
cd /data/docker/borgmatic docker compose up -d
Initialize the Repo
Before the first backup, you need to initialize the borg repo. This is simple.
You need to open a shell within the container and run the init command:
docker compose exec borgmatic /bin/bash borgmatic init --encryption repokey exit
The above command will initialize the configured repository using the repokey method. This means the actual encryption key is stored in the repository (at Hetzner). So all you need to decrypt the data is your passphrase. Borg supports a different mode where the encryption key is only stored on the local machine, but then you need to make sure you do backup the key separately. repokey seems simpler to me.
Finally you may want to execute your first backup manually, to check that everything works. Again, do it from a shell within the container:
docker compose up -d borgmatic --stats -v 1 --files exit
Verify the Backup
A backup is only useful if you know how to access and restore data from it. Of course you can use the borgmatic container or the borg command line tool to access the data. However to me it was easiest to check if I could access the backup via the Vorta GUI client on my local machine.
yay -S vorta
In Vorta, click the + button in the repository tab, select “existing repo” and add the repo you configured in your borgmatic config.
Eg. ssh://uXXXXXX@uXXXXXX.your-storagebox.de:23/home/borg/nas/repo
. Give your passphrase in the “password” field.
You should now see your backups in the “Archives” tab. Select one and use the “Mount…” button to mount it to a local directory. You now can browse that directory and copy files from it. That is verification enough for me that the backup is working.