The dyadic snapshot scheduler |
dss creates hardlink-based snapshots of a given directory on a remote or local host using rsync’s link-dest feature.
dss is admin friendly: It is easy to configure and needs little attention once configured to run in daemon mode. It keeps track of the available disk space and removes snapshots if disk space becomes sparse or snapshots become older than the specified time. Also, due to the hardlink-based approach, there is only one type of backup. Hence no full, incremental or differential backups need to be configured, and there is no database to maintain.
dss is also user-friendly because users can browse the snapshot directories without admin intervention and see the contents of the file system at the various times a snapshot was taken. Each snapshot looks like a full backup, so users can easily restore accidentally removed files by using their favorite file browser to simply copy files from the snapshot directory back to the live system.
dss gives your data an additional level of security besides the usual tape-based backups: If the file server goes down and all data is lost you can simply use the most recent snapshot as an immediate replacement – no need for a restore from tape that takes days to complete.
Snapshot pruning takes place in a dyadic fashion: Many recent snapshots are available, but the number of snapshots per time interval decreases exponentially. For example, one can configure dss so that it keeps 16 snapshots not older than one week, 8 snapshots between one and two weeks old, 4 snapshots between two and three weeks old, and so on.
Several fixes and improvements have accumulated over the years, so here is a new release. There are also some new features:
Support for multiple source directories.
The prune subcommand has been tweaked to use the same algorithm as the run subcommand to decide which snapshot should be pruned. The prune subcommand also gained the new –disk-space option.
When dss runs in daemon mode, it no longer sets the umask to zero.
The logo has been replaced by a slightly smaller one in svg format.
After almost ten years of development, dss has finally reached version 1.0.0. Many new features have been implemented for this release. The list below contains the user-visible changes since the previous 0.1.7 release. For the complete changelog consult the git log.
The biggest user-visible change is that the former modes of operation have been replaced by subcommands. For example, the former -R option to start dss in “run” mode corresponds to the “run” subcommand of version 1.0.0. Each subcommand has its own set of options. Global options and options for subcommands are split with the usual “–” separator. For example, to specify the global –loglevel option and the run-specific –daemon option, use dss –loglevel 2 – run –daemon.
Although the new version has different syntax rules, the on-disk snapshot format remains the same. No conversion is required to continue existing snapshot directories created with older dss versions.
The 0.1.x series will still be maintained for a while as the “maint” branch of the git repository, but this branch will only receive bug fixes.
The gengetopt option parser has been replaced by the lopsub library. Hence lopsub must be installed to compile this package. help2man is no longer required since lopsub has built-in roff support.
New subcommand “configtest” to check the command line options and the configuration file for syntactic correctness.
New option: –mountpoint. If this option is given, dss aborts if no file system is mounted on the destination directory.
New option –checksum to let rsync compute checksums occasionally.
The kill subcommand gained the new –wait option which instructs dss to wait until the signalled process has terminated.
The –no-resume option has been removed.
On exit, the run subcommand now waits for any previously spawned rsync or rm processes to terminate.
The ls subcommand now shows the age of incomplete snapshots rather than 0:00.
In run mode, dss no longer exits successfully if another instance is already running.
The command specified as the argument to –exit-hook is now subject to word splitting. Previously, the string was executed as-is.
Improved error diagnostics for the kill subcommand.
For all subcommands other than “run”, timestamps and function names are omitted from the log output.
“make install” will install the executable and the man page.
CFLAGS, CPPFLAGS and LDFLAGS can now be used to override the flags of the build system.
Source code files are now annotated with the SPDX GPL-2.0 license identifier.
The dss logo is now created with ImageMagick rather than dia.
Documentation improvements.
Improved error diagnostics.
dss no longer refuses to run in daemon mode without a log file.
Minor tweaks to the snapshot pruning algorithm.
A fix for create mode when rsync exists unsuccessfully.
NEWS and README have been converted to markdown format.
We now compile with -Wunused-parameter and -Wshadow.
New option –min-complete to specify the minimal number of snapshots to keep.
Improved handling of rsync errors. The new –max-rsync-errors option tells dss to terminate after the given number of rsync failures.
New home page URL and email address.
Reworked rsync restart logic, new –max-rsync-errors option.
Avoidance of “Errors with program diagnostics” error from rsync.
Improved signal handling.
snapshot recycling: outdated, redundant and orphaned snapshots are reused as the basis for new snapshots.
New subcommands: –kill, –reload.
The semaphore-based locking system prevents to start multiple instances of dss.
When dss receives SIGHUP it now writes the internal state and the configuration to the log file
First steps to make the source code ANSI C conform (thanks to Daniel Richard G).
dss writes log messages to stderr rather than to the logfile unless running in daemon mode.
Default hooks work also on systems where “/bin/true” does not exist, e.g. Mac OS.
This version of dss contains some new features, many improvements of existing features and several bug fixes.
Support for resuming snapshot creation after restart has been added and snapshot removal is deferred until at least one current snapshot exists. The internal handling of the various hooks has been simplified and many error messages have been clarified. On SIGHUP, dss now writes its configuration and internal state to the log file.
You can now specify pre-rm/post-rm-hooks that are executed whenever a snapshot is deleted. This release adds better compatibility with rsync 3.0, and avoids busy loops when automatically restarting the rsync process.
This release includes the reason why a snapshot gets removed in the log message. It will never remove the snapshot that is currently being created. It will really pass the full path to the last complete snapshot in the post_create_hook.
This release prevents busy loops on rsync exit code 13. It ignores any snapshot directory with creation time >= completion time. It opens /dev/null for reading and writing when executing rsync. It shows human readable snapshot creation duration when listing snapshots. It restarts the rsync process if it returned with exit code 13.
Initial public release.
Only the source code is available for download. Use git to clone the dss repository by executing
or grab the tarball of the current master branch. If you prefer to download the tarball of the latest release, select the corresponding snapshot link on the dss gitweb page
dss is known to compile on Linux, FreeBSD and NetBSD. However, it is run-tested only on Linux.
Note that lopsub is required to compile dss.
Type
make
in the dss source directory to build the dss executable and the man
page. If lopsub is installed in a non-standard path, you may need to
run make
as follows:
make CPPFLAGS=-I$HOME/lopsub/include LDFLAGS=-L$HOME/lopsub/lib
Then type
sudo make install
to install in /usr/local, or
make install PREFIX=/somewhere/else
to install in /somewhere/else.
Also make sure that rsync is installed on your system. Version 2.6.1 or newer is required.
Suppose you’d like to create snapshots of the existing directory
/foo/bar
in the directory
/baz/qux.
Create the config file
~/.dssrc
that contains the values for the source and the destination directories as follows:
echo 'source-dir "/foo/bar"' > ~/.dssrc
echo 'dest-dir "/baz/qux"' >> ~/.dssrc
Then execute the commands
mkdir /baz/qux
dss run
In order to print the list of all snapshots created so far, use
dss ls
Yes, it’s really that easy.
The second example involves a slightly more sophisticated config file. It instructs dss to exclude everything which matches at least one pattern of the given exclude file, prevents rsync from crossing file system boundaries and increases the number of snapshots.
source-dir "/foo/bar"
dest-dir "/baz/qux"
# exclude files matching patterns in /etc/dss.exclude
rsync-option "--exclude-from=/etc/dss.exclude"
# don't cross filesystem boundaries
rsync-option "--one-file-system"
# maintain 2^6 - 1 = 63 snapshots
num-intervals "6"
The /etc/dss.exclude file could look like this (see rsync(1) for more examples)
- /proc
- /**/tmp/
Note that dss supports many more features and config options such as taking snapshots from remote hosts and several hooks that are executed on certain events, for example whenever a snapshot was created successfully. Try
dss -h
for an overview of all supported command line options or
dss --detailed-help
for the full help text.
dss is open source software, licensed under the GNU General Public License, Version 2.
Email: André Noll, maan@tuebingen.mpg.de, Homepage: http://people.tuebingen.mpg.de/maan/
Comments and bug reports are welcome. Please provide enough info such as the version of dss you are using and relevant parts of the logs. Including the string [dss] in the subject line is also a good idea.