Backup Nextcloud (or anything) to Proton Drive using restic, rclone and systemd

My Problem

Some of us will have heard of the 3-2-1 rule for backups - three copies, two different media types, one copy minimum offsite. I was in violation of this rule until recently. I had three copies on two different types of media, but they were all local. In this how-to, I'll explain how I set up automated, encrypted, and versioned backups for my self-hosted cloud.

Prerequisites

Check that you have the following:

• A Proton account with enough storage
• Basic knowledge of the command line
• Systemd installed (Linux)

Overview

While rclone can sync data, it doesn't provide versioning. This is where restic comes in; restic will be responsible for the backups, and rclone will provide the connection. Rclone has built-in support for restic, making this process simple. Systemd will be used to create a socket and service for rclone and a timer and service for restic.

Configure rclone

Download rclone

On Fedora: sudo dnf install rclone

Set up the remote (Proton Drive)

1. Open the configuration: rclone config
2. Select n for a new remote
3. Name the remote proton
4. Select the type of storage by index - in my case, 42

1. Enter your Proton username
2. Enter your Proton password (selecting y will prompt you to enter it)
3. Enter your 2-factor authentication TOTP code (e.g., 000000)

1. Confirm and quit
2. Test the remote with rclone lsd proton: This should return all directories in your Proton Drive.

Configure restic

Download restic

1. Open a new terminal, leaving the Rclone server running
2. On Fedora: sudo dnf install restic

Initialise the repository

1. Export ENVs for the Rclone server and restic password and then init:

$ export RESTIC_REPOSITORY=rest:http://localhost:8080/
$ export RESTIC_PASSWORD=createpassword
$ restic init

This will take a moment. When complete, it will output something like this:

# created restic backend xxxrepohashxxx at rest:http://localhost:8080/
# Please note that knowledge of your password is required to access
# the repository. Losing your password means that your data is
# irrecoverably lost.

Initial Backup

My Nextcloud install uses nextcloud-aio, which uses Docker. The Nextcloud data directory is configured as an external mount. My approach is to backup the data directory, and then backup /var/lib/docker/ - This should be plug-and-play on restore, but as always, a backup is only valid if you test it.

I recommend doing the initial backup from the console rather than using the systemd units we'll set up later. This can be a very long-running command (days), so use tmux or screen to keep the session active even when closing the terminal window

1. Install tmux if needed: sudo dnf install tmux
2. Create a new tmux session: tmux
3. Serve the endpoint: rclone serve restic proton:backup
4. Create a new window in tmux: CTRL+B followed by C

1. Start the backup: restic backup /mnt/ncdata and then restic backup /var/lib/docker

1. View the backup using restic snapshots
2. Verify the backup using restic check

Setting up the systemd units

Rclone

Protondrive API can go into a failed state, so I've built some error handling to recover on errors - if you have a better way, please let me know.

1. Create the service: sudo nano /etc/systemd/system/rclone-serve-restic.service

[Unit]
Description=Rclone Serve Restic Service
Wants=rclone-watchdog.service
After=rclone-serve-restic.socket


[Service]
User=youruser # set to your user or create a specific user for this
ExecStart=/usr/bin/rclone serve restic proton:backup

KillSignal=SIGTERM
SuccessExitStatus=143  # allow a sigterm exit to be considered a success - otherwise unit status will result in failed
Restart=on-failure

StandardOutput=truncate:/var/log/rclone_servelog
StandardError=truncate:/var/log/rclone_serve.log

[Install]
WantedBy=multi-user.target

2. Create the socket: sudo nano /etc/systemd/system/rclone-serve-restic.socket

[Unit]
Description=Rclone Serve Restic Socket

[Socket]
ListenStream=127.0.0.1:8080 # will listen for requests here and trigger rclone-restic-serve.service

[Install]
WantedBy=sockets.target

3. Create the watchdog: sudo nano /etc/systemd/system/rclone-watchdog.service

[Unit]
Description=Rclone Watchdog Service
After=rclone-serve-restic.service
BindsTo=rclone-serve-restic.service
OnFailure=rclone-handlefailure.service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/rclone_health.sh
SuccessExitStatus=TERM

[Install]
WantedBy=multi-user.target

4. Create the failure handler: sudo nano /etc/systemd/system/rclone-handlefailure.service

[Unit]
Description=Handle Failure Service
After=rclone-serve-restic.service

[Service]
Type=oneshot
ExecStart=/bin/systemctl kill --signal SIGINT rclone-serve-restic.service

[Install]
WantedBy=default.target

5. Create the script: sudo nano /usr/local/bin/rclone_health.sh

#!/bin/bash

# Log file path
LOG_FILE=/var/log/rclone_serve.log

# Error message to look for
# change if needed to match your error condition
ERROR_MESSAGE="received no response from API" 

while true; do
    # Check if the error message is present in the log file
    if grep -q "$ERROR_MESSAGE" $LOG_FILE; then
        exit 1
    fi
    # Wait for 1 second before checking again
    sleep 1
done

6. make the script executable sudo chmod +x /usr/local/bin/rclone_health.sh

Restic

1. Create the service: sudo nano /etc/systemd/system/restic-backup.service

[Unit]
Description=Restic Backup Service

[Service]
Type=oneshot
User=youruser # set to your user or create a specific user for this

Environment="RESTIC_CACHE_DIR=/home/youruser/.cache/restic" # update user
Environment="RESTIC_REPOSITORY=rest:http://localhost:8080/"
Environment="RESTIC_PASSWORD=your_restic_password"

ExecStart=/usr/bin/restic backup /mnt/ncdata/
ExecStart=/usr/bin/restic backup /var/lib/docker
ExecStart=/usr/bin/restic backup /home/youruser/docker # update user

ExecStartPost=/bin/systemctl stop rclone-serve-restic.service

ReadOnlyPaths=/mnt/ncdata /var/lib/docker /home/youruser/docker # mount the required dirs as read-only

[Install]
WantedBy=multi-user.target

2. Create the timer: sudo nano /etc/systemd/system/restic-backup.timer

[Unit]
Description=Restic Backup Timer

[Timer]
OnCalendar=*-*-* 03:00 # every day at 03 hours 
Persistent=true # persist across reboots

[Install]
WantedBy=timers.target

Enable

$ sudo systemctl daemon-reload
$ sudo systemctl enable restic-backup.timer rclone-serve-restic.socket
$ sudo systemctl start restic-backup.timer rclone-serve-restic.socket

Everything should now be set up. Verify correct operation by using sudo systemctl status unit.name.

Troubleshooting

• Verify no failed systemd units: sudo systemctl --failed
• Verify that the Rclone is connected: rclone lsd proton:backup
• Read the docs:
  • Restic
  • Rclone/Restic
  • Rclone/Proton
• Post a comment, and I can attempt to help.

To Do

While this is a good start, it's missing some features. I'll update this write-up as I implement them:

• Pruning (see Pruning). Add this into the restic-backup.service unit.
• Automated checks (see Checks). Add this as a separate service/timer and make it conflict with restic-backup.service to prevent concurrency issues.
• Notification of some sort on error. I haven't looked into this yet - this is critical.
• Do a test restore - both of an individual file, and an entire snapshot.