Self-Hosted: Backups and Restores

This article covers how to back up and restore a self-hosted Hudu instance. A complete backup has two separate parts: your PostgreSQL database and your uploaded files. Both are covered below, along with how to verify that your backups are actually usable and how to restore from them.

Backups of Hudu

Manual backup of Hudu Postgres

• Log in to the server you want to back up.
• Run cd ~/hudu2
• Make sure the server is up and running.
• Run: sudo docker compose exec -T db pg_dump -U postgres hudu_production > NAME-OF-DUMP.sql
• This file can now be moved onto a new server to be used. Make sure to keep the same .env variables.

Auto-backup script to S3 (Hudu Software - Doubletake)

The first thing to automate is your database. There are multiple ways to automatically back up Postgres databases. One simple way is to add the Doubletake service to the end of your docker-compose.yml. On the schedule you set, it dumps your database, verifies and optionally compresses or encrypts the dump, and uploads it to your S3-compatible bucket. It can also download, verify, and restore that dump when you need it.

postgres3:
  image: hudusoftware/doubletake:latest
  restart: unless-stopped
  links:
    - db
  environment:
    S3_HOST_BASE: 's3.us-west-1.wasabisys.com'
    S3_REGION: 's3.us-west-1'
    S3_BUCKET: 'bucketname'
    S3_FOLDER: 'foldername'
    S3_ACCESS_KEY_ID: 'XXXX'
    S3_SECRET_ACCESS_KEY: 'XXXXXXXX'
    CRON_SCHEDULE: '0 */6 * * *'
    DB_NAME: 'hudu_production'
    DB_USER: 'postgres'
    POSTGRES_PASSWORD:
    DB_HOST: 'db'
    DB_BACKUP_VERIFY: '1'
    DB_BACKUP_COMPRESS: '0' # Optional, default is 0
    DB_BACKUP_COMPRESS_LEVEL: # Optional, default is 4
    POSTGRES_EXTRA_OPTS: '--schema=public --blobs'

A few of the key options:

• CRON_SCHEDULE — how often the backup runs. The example above (0 */6 * * *) runs every six hours.
• DB_BACKUP_VERIFY — set to 1 to enable Doubletake's built-in integrity check (see the verification section below).
• DB_BACKUP_COMPRESS — set to 1 to compress dumps before upload. Compressed dumps are written as .xz files.

To run a backup on demand rather than waiting for the schedule:

sudo docker compose exec -T postgres3 /backup.sh

To view the log from the most recent run (useful for confirming a backup succeeded or troubleshooting a failure), replace postgres3 with your container name if you changed it:

sudo docker compose exec -T postgres3 /bin/sh -c "cat /doubletake.log"

Verifying the integrity of your backups

There are two layers of verification, and it's important to understand what each one does and does not prove.

1. Automated file verification (built in to Doubletake)

When DB_BACKUP_VERIFY is set to 1, Doubletake calculates a BLAKE2 hash and byte size for each dump and stores them in a small .meta file next to the dump in your bucket. When you download or restore that dump, Doubletake recalculates the hash and compares it to the .meta file to confirm the dump has not changed or become corrupted since it was written. On a successful restore or download you'll see confirmation in the output, for example:

metadata downloaded to /dump/backup.meta
Metadata is the same.
Metadata is identical.
Download successful.

2. Manual verification (recommended on a schedule)

The only way to be confident a backup is usable is to periodically restore it into a separate, non-production environment and confirm that Hudu comes up and your data is present. We recommend doing this on a recurring basis rather than only when you need it.

Between full test restores, you can run these quick sanity checks on a dump file:

• Confirm the file exists and isn't empty. A backup that failed part-way can leave a zero-byte or truncated file:

  ls -lh NAME-OF-DUMP.sql

• For an uncompressed plain SQL dump, confirm it completed. A complete pg_dump ends with a completion marker on the last line. If it's missing, the dump was cut short:

  tail -n 1 NAME-OF-DUMP.sql
  # expected: -- PostgreSQL database dump complete

• For a compressed dump, test the archive. This confirms the compressed file isn't corrupted:

  xz -t backup.sql.xz

To run a full test restore, restore the dump into a scratch database or a throwaway Hudu instance using the steps in Restoring a database backup below, then log in and spot-check that your records, passwords, and attachments are present. If the restore completes without errors and the data looks correct, you have a verified, usable backup.

Restoring a database backup

Once you have a database backup, you can restore your Hudu instance to the old backup by following these steps:

• Make sure you have an up-to-date backup of your documentation before you begin.
• Move the .sql database dump file into the ~/hudu2 directory. Typically, the easiest way to move files is via SCP or SFTP.
• Run sudo docker compose down to bring your instance down.
• Run sudo docker compose up -d db
• Run the command: sudo docker compose exec db dropdb hudu_production -U postgres
• Run the command: sudo docker compose exec db createdb hudu_production -U postgres
• Run the command: cat NAME-OF-DUMP.sql | sudo docker compose exec -T db psql -d hudu_production -U postgres
• Run sudo docker compose down
• Run sudo docker compose up -d to get your instance back up and running.

Backing up and restoring files

Your uploaded files (photos, attachments, document files, and logos) are stored separately from the database. Where they live depends on your file storage configuration: either on the local Docker volume or in an S3-compatible object storage bucket. Back up whichever one you use, on a schedule, alongside your database backups.

Local storage

When using local storage, files are located here: /var/lib/docker/volumes/hudu2_app_data/_data/. To restore, move this directory from the old server to the new server, keeping the same path.

S3 bucket storage

If your files live in an S3-compatible bucket (for example Wasabi or AWS S3), your backup job needs to copy the contents of that bucket somewhere else, on a schedule. Restoring the database alone will not bring your files back, so this step is required for a complete backup. You can use the AWS CLI, s3cmd, rclone, or a similar tool.

Sync the bucket to your local system with the AWS CLI

aws s3 sync s3://your-bucket-name /home/ubuntu/s3/your-bucket-name/

If your provider is not AWS (for example Wasabi), add the --endpoint-url flag pointing at your provider:

aws s3 sync s3://your-bucket-name /home/ubuntu/s3/your-bucket-name/ --endpoint-url=https://s3.wasabisys.com

Sync the bucket with s3cmd

s3cmd sync s3://your-bucket-name /home/ubuntu/s3/your-bucket-name/

For an offsite copy, you can also sync one bucket to another (ideally with a different provider or region) rather than pulling everything to local disk. Whichever approach you use, automate it with a cron job so file backups run as regularly as your database backups, and keep the copy separate from your primary bucket.

We also recommend using lifecycle rules to move older S3 backups to lower-cost cold storage such as Amazon Glacier.

Moving from object to local or vice versa

Both local and object storage use the same directory structure. When moving to or from local storage, move the files from /var/lib/docker/volumes/hudu2_app_data/_data/.

FAQ