My server has /opt and /usr/share/nginx two directories, used to store website content. Data is invaluable, to avoid losing precious data, I decided to run rsnapshot on the Raspberry Pi to perform regular backups of the website content.
Why Choose rsnapshot?
-
rsnapshot is an open-source software based on rsync, with a simple principle, no backdoors, no mandatory encryption, and the backed-up data is exactly what you see. -
rsnapshot manages files through hard links, with the same file in different folders occupying only one storage space, saving disk space. -
rsnapshot defaults to incremental backups, saving bandwidth. -
rsnapshot has long-term maintenance (maintained since 2015), stable functionality, and has received 2.9k stars on GitHub’s open-source repository https://github.com/rsnapshot/rsnapshot, widely praised.
Installing rsnapshot
sudo apt install rsnapshot
Configuring Raspberry Pi for Password-less Login to Cloud Server
cd ~/.ssh
ssh-keygen
# Set key permissions
# Public key 644
sudo chmod 644 ~/.ssh/fangyuanxiaozhan.com.pub
# Private key 600
sudo chmod 600 ~/.ssh/fangyuanxiaozhan.com
Sending Public Key to Remote Host
ssh-copy-id -i ~/.ssh/fangyuanxiaozhan.com.pub remote_user@remote_ip_or_domain
Creating ~/.ssh/config on Raspberry Pi and Setting Permissions to 600
# If ~/.ssh/config already exists, no need to create
.touch ~/.ssh/config
chmod 600 ~/.ssh/config
Fill in the following content in ~/.ssh/config
Host remote_ip_or_domain
HostName remote_ip_or_domain
User root
IdentityFile ~/.ssh/fangyuanxiaozhan.com
SSH can log in without a password, and rsnapshot based on rsync can also transfer files from the server without a password.
Modify Configuration File as Needed
-
Create rsnapshot backup folder on Raspberry Pi
# Create root directory for rsnapshot storage
mkdir /opt/rsnapshot
# Create directory to store my Tianyi cloud server data
mkdir /opt/rsnapshot/CTYun
-
Set root directory
# Modify rsnapshot configuration file /etc/rsnapshot.conf, change the field corresponding to snapshot_root to the root directory created on Raspberry Pisnapshot_root /opt/rsnapshot/
Use Tabs to fill in between configuration lines, spaces will cause errors, for example, snapshot_root (use Tab to fill in) /opt/rsnapshot/
-
Comment out local backup configuration
At the bottom of /etc/rsnapshot.conf, we can see many lines starting with backup, among them the default few lines are for backing up several directories of the Raspberry Pi itself, we can add # in front to comment them out.
-
Remove the comment from the absolute path of ssh
Since rsnapshot relies on rsync for backing up data from the server, and rsync relies on ssh, we need to tell rsnapshot the absolute path of ssh, which means removing the # from the line where cmd_ssh is located.
cmd_ssh /usr/bin/ssh
-
Change the location of the lock file
The lockfile is used to prevent two rsnapshot instances from running simultaneously, acting as a lock. The location of this lock can be defined by ourselves. The default location for the lockfile is /var/run/rsnapshot.pid, if I want to run the rsnapshot instance as the user ubuntu, I need to configure the lock file location to a place where the ubuntu user has permission to modify, my approach is to create /home/ubuntu/.rsnapshot folder.
mkdir /home/ubuntu/.rsnapshot
Then change the lockfile to
lockfile /home/ubuntu/.rsnapshot/rsnapshot.pid
-
Backup folder configuration
Add the following at the end of the configuration file
# CTYun
backup [email protected]:/etc/nginx ./
backup [email protected]:/opt ./ exclude=/opt/before,exclude=/opt/EasyTypora/node_modules
backup [email protected]:/usr/share/nginx/fangyuanxiaozhan.com ./
Test if the configuration file format is correct
rsnapshot configtest
My final configuration file (for reference)
#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
# #
# PLEASE BE AWARE OF THE FOLLOWING RULE: #
# #
# This file requires tabs between elements #
# #
#################################################
#######################
# CONFIG FILE VERSION #
#######################
config_version 1.2
###########################
# SNAPSHOT ROOT DIRECTORY #
###########################
# All snapshots will be stored under this root directory.
#
snapshot_root /opt/rsnapshot/CTYun/
# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root 1
#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################
# LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp /bin/cp
# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm /bin/rm
# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync /usr/bin/rsync
# Uncomment this to enable remote ssh backups over rsync.
#
cmd_ssh /usr/bin/ssh
# Comment this out to disable syslog support.
#
cmd_logger /usr/bin/logger
# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
#cmd_du /usr/bin/du
# Uncomment this to specify the path to rsnapshot-diff.
#
#cmd_rsnapshot_diff /usr/bin/rsnapshot-diff
# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec /path/to/preexec/script
# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec /path/to/postexec/script
# Paths to lvcreate, lvremove, mount and umount commands, for use with
# Linux LVMs.
#
#linux_lvm_cmd_lvcreate /sbin/lvcreate
#linux_lvm_cmd_lvremove /sbin/lvremove
#linux_lvm_cmd_mount /bin/mount
#linux_lvm_cmd_umount /bin/umount
#########################################
# BACKUP LEVELS / INTERVALS #
# Must be unique and in ascending order #
# e.g. alpha, beta, gamma, etc. #
#########################################
retain alpha 6
retain beta 7
retain gamma 4
#retain delta 3
############################################
# GLOBAL OPTIONS #
# All are optional, with sensible defaults #
############################################
# Verbose level, 1 through 5.
# 1 Quiet Print fatal errors only
# 2 Default Print errors and warnings only
# 3 Verbose Show equivalent shell commands being executed
# 4 Extra Verbose Show extra verbose information
# 5 Debug mode Everything
#
verbose 2
# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
# If you want the rsync output, you have to set it to 4
#
loglevel 3
# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
#logfile /var/log/rsnapshot.log
# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile /home/ubuntu/.rsnapshot/rsnapshot.pid
# By default, rsnapshot check lockfile, check if PID is running
# and if not, consider lockfile as stale, then start
# Enabling this stop rsnapshot if PID in lockfile is not running
#
#stop_on_stale_lockfile 0
# Default rsync args. All rsync commands have at least these options set.
#
#rsync_short_args -a
#rsync_long_args --delete --numeric-ids --relative --delete-excluded
# ssh has no args passed by default, but you can specify some here.
#
#ssh_args -p 22
# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args -csh
# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs 0
# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns.
#
#include ???
#include ???
#exclude ???
#exclude ???
# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file /path/to/include/file
#exclude_file /path/to/exclude/file
# If your version of rsync supports --link-dest, consider enabling this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest 0
# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot alpha"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first 0
# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes 0
# Number of rsync re-tries. If you experience any network problems or
# network card issues that tend to cause ssh to fail with errors like
# "Corrupted MAC on input", for example, set this to a non-zero value
# to have the rsync operation re-tried.
#
#rsync_numtries 0
# LVM parameters. Used to backup with creating lvm snapshot before backup
# and removing it after. This should ensure consistency of data in some special
# cases
#
# LVM snapshot(s) size (lvcreate --size option).
#
#linux_lvm_snapshotsize 100M
# Name to be used when creating the LVM logical volume snapshot(s).
#
#linux_lvm_snapshotname rsnapshot
# Path to the LVM Volume Groups.
#
#linux_lvm_vgpath /dev
# Mount point to use to temporarily mount the snapshot(s).
#
#linux_lvm_mountpath /path/to/mount/lvm/snapshot/during/backup
###############################
### BACKUP POINTS / SCRIPTS ###
###############################
# LOCALHOST
#backup /home/ localhost/
#backup /etc/ localhost/
#backup /usr/local/ localhost/
#backup /var/log/rsnapshot localhost/
#backup /etc/passwd localhost/
#backup /home/foo/My Documents/ localhost/
#backup /foo/bar/ localhost/ one_fs=1,rsync_short_args=-urltvpog
#backup_script /usr/local/bin/backup_pgsql.sh localhost/postgres/
# You must set linux_lvm_* parameters below before using lvm snapshots
#backup lvm://vg0/xen-home/ lvm-vg0/xen-home/
# EXAMPLE.COM
#backup_exec /bin/date "+ backup of example.com started at %c"
#backup [email protected]:/home/ example.com/ +rsync_long_args=--bwlimit=16,exclude=core
#backup [email protected]:/etc/ example.com/ exclude=mtab,exclude=core
#backup_exec ssh [email protected] "mysqldump -A > /var/db/dump/mysql.sql"
#backup [email protected]:/var/db/dump/ example.com/
#backup_exec /bin/date "+ backup of example.com ended at %c"
# CVS.SOURCEFORGE.NET
#backup_script /usr/local/bin/backup_rsnapshot_cvsroot.sh rsnapshot.cvs.sourceforge.net/
# RSYNC.SAMBA.ORG
#backup rsync://rsync.samba.org/rsyncftp/ rsync.samba.org/rsyncftp/
# CTYun
backup [email protected]:/etc/nginx ./
backup [email protected]:/opt ./ exclude=/opt/before,exclude=/opt/EasyTypora/node_modules
backup [email protected]:/usr/share/nginx/fangyuanxiaozhan.com ./
Ensure that the empty parts in the configuration line are filled with Tabs, not spaces
-
Explanation of the backup quantity limit
In the above configuration file, there are the following lines
retain alpha 6
retain beta 7
retain gamma 4
For example, retain alpha 6 means a maximum of six backups, such as alpha.0 alpha.1 alpha.2 alpha.3 alpha.4 alpha.5, where alpha.0 is the latest backup, and alpha.5 is the oldest backup. If more than six backups are made, the oldest one will be cleared from the disk. After the first backup command is run, the alpha.0 folder will be generated in the root directory, which is /opt/rsnapshot/CTYun/alpha.0, and the /opt/rsnapshot/CTYun/alpha.0 folder contains the backed-up file directory.
-
Manual Backup
rsnapshot -c /etc/rsnapshot.conf alpha
In the above command summary, we use the alpha strategy for backup, specifying the location of the configuration file.
After execution, check the directory structure
You can see that alpha.0 has become the real root directory.
If we run rsnapshot -c /etc/rsnapshot.conf alpha multiple times, we can see several parallel alpha.* folders under /opt/rsnapshot/CTYun
Since retain alpha 6 is configured, no matter how many times we run it, alpha.* will not exceed six.
Automated Backup
crontab is a tool that can create scheduled tasks. We can add a task in crontab to run rsnapshot -c /etc/rsnapshot.conf alpha every four hours, which is equivalent to performing six backups daily.
In ubuntu, crontab is installed by default for all users, so we can create tasks without sudo permissions. The operation method is as follows, open the cron table
crontab -e
The first time you open it, you can choose the editor. I choose vim to open, and add the configuration to run the command once every four hours
0 */4 * * * rsnapshot -c /etc/rsnapshot.conf alpha
After writing the configuration, exit the editor;
Optimization: Create a Daily Task to Retain Backups for 31 Days
I hope to generate a new backup at 4 AM every day and retain it for 31 days, but I do not want it to conflict with the backup every four hours (two tasks sharing the logfile will affect each other).
Thus, I open a new configuration file /home/ubuntu/.rsnapshot/mouthly.conf, add retain monthly 31, and change the file lock to lockfile /home/ubuntu/.rsnapshot/monthly.pid
The monthly here can be fully customized and does not have special significance; it can be monthly001 or monthly002, as long as the command run corresponds.
The content is
#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
# #
# PLEASE BE AWARE OF THE FOLLOWING RULE: #
# #
# This file requires tabs between elements #
# #
#################################################
#######################
# CONFIG FILE VERSION #
#######################
config_version 1.2
###########################
# SNAPSHOT ROOT DIRECTORY #
###########################
# All snapshots will be stored under this root directory.
#
snapshot_root /opt/rsnapshot/CTYun/
# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root 1
#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################
# LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp /bin/cp
# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm /bin/rm
# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync /usr/bin/rsync
# Uncomment this to enable remote ssh backups over rsync.
#
cmd_ssh /usr/bin/ssh
# Comment this out to disable syslog support.
#
cmd_logger /usr/bin/logger
# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
#cmd_du /usr/bin/du
# Uncomment this to specify the path to rsnapshot-diff.
#
#cmd_rsnapshot_diff /usr/bin/rsnapshot-diff
# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec /path/to/preexec/script
# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec /path/to/postexec/script
# Paths to lvcreate, lvremove, mount and umount commands, for use with
# Linux LVMs.
#
#linux_lvm_cmd_lvcreate /sbin/lvcreate
#linux_lvm_cmd_lvremove /sbin/lvremove
#linux_lvm_cmd_mount /bin/mount
#linux_lvm_cmd_umount /bin/umount
#########################################
# BACKUP LEVELS / INTERVALS #
# Must be unique and in ascending order #
# e.g. alpha, beta, gamma, etc. #
#########################################
retain monthly 31
retain alpha 6
retain beta 7
retain gamma 4
#retain delta 3
############################################
# GLOBAL OPTIONS #
# All are optional, with sensible defaults #
############################################
# Verbose level, 1 through 5.
# 1 Quiet Print fatal errors only
# 2 Default Print errors and warnings only
# 3 Verbose Show equivalent shell commands being executed
# 4 Extra Verbose Show extra verbose information
# 5 Debug mode Everything
#
verbose 2
# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
# If you want the rsync output, you have to set it to 4
#
loglevel 3
# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
#logfile /var/log/rsnapshot.log
# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile /home/ubuntu/.rsnapshot/monthly.pid
# By default, rsnapshot check lockfile, check if PID is running
# and if not, consider lockfile as stale, then start
# Enabling this stop rsnapshot if PID in lockfile is not running
#
#stop_on_stale_lockfile 0
# Default rsync args. All rsync commands have at least these options set.
#
#rsync_short_args -a
#rsync_long_args --delete --numeric-ids --relative --delete-excluded
# ssh has no args passed by default, but you can specify some here.
#
#ssh_args -p 22
# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args -csh
# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs 0
# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns.
#
#include ???
#include ???
#exclude ???
#exclude ???
# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file /path/to/include/file
#exclude_file /path/to/exclude/file
# If your version of rsync supports --link-dest, consider enabling this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest 0
# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot alpha"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first 0
# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes 0
# Number of rsync re-tries. If you experience any network problems or
# network card issues that tend to cause ssh to fail with errors like
# "Corrupted MAC on input", for example, set this to a non-zero value
# to have the rsync operation re-tried.
#
#rsync_numtries 0
# LVM parameters. Used to backup with creating lvm snapshot before backup
# and removing it after. This should ensure consistency of data in some special
# cases
#
# LVM snapshot(s) size (lvcreate --size option).
#
#linux_lvm_snapshotsize 100M
# Name to be used when creating the LVM logical volume snapshot(s).
#
#linux_lvm_snapshotname rsnapshot
# Path to the LVM Volume Groups.
#
#linux_lvm_vgpath /dev
# Mount point to use to temporarily mount the snapshot(s).
#
#linux_lvm_mountpath /path/to/mount/lvm/snapshot/during/backup
###############################
### BACKUP POINTS / SCRIPTS ###
###############################
# LOCALHOST
#backup /home/ localhost/
#backup /etc/ localhost/
#backup /usr/local/ localhost/
#backup /var/log/rsnapshot localhost/
#backup /etc/passwd localhost/
#backup /home/foo/My Documents/ localhost/
#backup /foo/bar/ localhost/ one_fs=1,rsync_short_args=-urltvpog
#backup_script /usr/local/bin/backup_pgsql.sh localhost/postgres/
# You must set linux_lvm_* parameters below before using lvm snapshots
#backup lvm://vg0/xen-home/ lvm-vg0/xen-home/
# EXAMPLE.COM
#backup_exec /bin/date "+ backup of example.com started at %c"
#backup [email protected]:/home/ example.com/ +rsync_long_args=--bwlimit=16,exclude=core
#backup [email protected]:/etc/ example.com/ exclude=mtab,exclude=core
#backup_exec ssh [email protected] "mysqldump -A > /var/db/dump/mysql.sql"
#backup [email protected]:/var/db/dump/ example.com/
#backup_exec /bin/date "+ backup of example.com ended at %c"
# CVS.SOURCEFORGE.NET
#backup_script /usr/local/bin/backup_rsnapshot_cvsroot.sh rsnapshot.cvs.sourceforge.net/
# RSYNC.SAMBA.ORG
#backup rsync://rsync.samba.org/rsyncftp/ rsync.samba.org/rsyncftp/
# CTYun
backup [email protected]:/etc/nginx ./
backup [email protected]:/opt ./ exclude=/opt/before,exclude=/opt/EasyTypora/node_modules
backup [email protected]:/usr/share/nginx/fangyuanxiaozhan.com ./
Test the monthly backup configuration file
rsnapshot -c /home/ubuntu/.rsnapshot/monthly.conf configtest
Run crontab -e, add the following configuration as a scheduled task to run daily at 4 AM
0 4 * * * rsnapshot -c /home/ubuntu/.rsnapshot/monthly.conf monthly
The monthly task will produce folders starting with monthly after backup
rsnapshot -c /home/ubuntu/.rsnapshot/monthly.conf monthly
Conclusion
The hard drives from cloud service providers are quite expensive. For personal developers, saving some money is always good.
In 2023, mechanical hard drives are quite cheap. You can buy a brand new 16TB drive for a thousand yuan, plug it into the Raspberry Pi, and regularly perform incremental backups of server data at a cost much lower than that of cloud service providers.
If you are an office worker, deploying a set of rsnapshot backups in the office and at home can also support distributed disaster recovery of data.
Why back up server data locally? Currently, the ICP filing is hot, and after buying the server, due to the filing restrictions, you cannot immediately use ports such as 80 and 443. Even if the server is filed, the parsed domain name will also be restricted. If one day, due to some irresistible force, all the data associated with personal filings on the server is sealed, it is not impossible. Therefore, doing a good job of local data backup is always a good idea, and the cost is not high.
This article belongs to the twentieth issue of the series “Raspberry Pi Does Not Gather Dust”. The open-source address for the series of tutorials is github.com/zhaoolee/pi
This article’s permanent update address (welcome to read and leave comments):
This article may be quite boring for most people, and I couldn’t even find a suitable cover image. I flipped through my phone and used a picture as the cover image.
Cover image: “All In” 拼一次富三代.jpg shot at Hainan University Danzhou Campus (Zhaoolee, taken in 2015)
