misma

NAME

misma - the minimal snapshot manager

SYNOPSIS

misma [global-options...] [--] [<subcommand> [subcommand-options...]]

DESCRIPTION

misma is an open source application which maintains snapshots of one or more thin provisioned logical volumes on Linux systems.

Snapshots are created and removed automatically according to the configured schedule. Old snapshots are replaced so that the time between two consecutive snapshots doubles at each step. To prevent data or metadata space exhaustion, the available space of the underlying thin pools is monitored periodically and snapshots are removed early when space gets tight.

Besides the run subcommand which implements snapshot scheduling and free space monitoring, misma supports additional subcommands to list existing snapshots and the utilization of the thin pools, or to create/remove snapshots manually.

OPTIONS

General options
-h, --help

print help and exit

--detailed-help

print help, including all details, and exit

-V, --version

print version and exit

-l, --loglevel=<severity>

control amount of logging

values: debug, info, notice, warning, error, crit, emerg

Log only messages with severity greater or equal than the given value. Possible values:

debug: produces really noisy output. info: still noisy, but won’t fill up the disk quickly. notice: indicates normal, but significant event. warning: unexpected events that can be handled. error: unhandled error condition. crit: system might be unreliable. emerg: last message before exit.

-c, --config-file=<path>

use alternative config file (default: ~/.mismarc)

Options may be given at the command line or in the configuration file. As usual, if an option is given both at the command line and in the configuration file, the command line option takes precedence.

The config file may contain global options as well as options for any subcommand, but subcommand specific options must be placed in a separate section. See the Examples section of the man page.

LVM options
--origin=<vg/tlv>

the VG and the thin LV to snapshot

The named volume group must exist and it must contain the named thin logical volume. This option may be given multiple times where each instance corresponds to one origin to snapshot.

SUBCOMMANDS

Misma supports the subcommands described below. If no subcommand is given, the list of available subcommands is shown and the program terminates successfully without performing any further action.

run - create and prune snapshots, discard unused blocks
Usage: run [--daemon] [--logfile=<path>] [--create-interval=<[lvmspec:]timespec>] [--max-age=<[lvmspec:]timespec>] [--check-interval=<timespec>] [--threshold=<[lvmspec:]data_threshold,meta_threshold>] [--trim-interval=<[lvmspec:]timespec>] [--exit-hook=<command>] [--suppress-lvm-warnings]

This is the main mode of operation. Snapshots are created and pruned periodically, the thin pool utilization is monitored and filesystem trims are scheduled as configured. The subcommand terminates only on fatal errors or after a terminating signal was received.
-d, --daemon

run as background daemon

If this option is given, the process detaches from the console and continues to run in the background.

-l, --logfile=<path>

where to write log output

default: /dev/null

This option is only honored if --daemon is given, in which case log messages go to the given file. Otherwise the option is silently ignored and log output is written to stderr.

--create-interval=<[lvmspec:]timespec>

Time span between two subsequent snapshots

default: 6h

The lvm specifier determines to which origins this instance of the option applies. If no specifier is given, the option applies to all origins. Otherwise the specifier may be in one of the following forms: <vg>: applies to all origins in VG vg, <vg|pool>: applies to all origins in thin pool <pool> of VG vg, or <vg/tlv>: applies to origin tlv of vg only. If more than one specifier match a particular origin, the narrowest scoped one applies. The order of precedence is therefore <vg/tlv>, <vg|pool>, <vg>, <global>.

The time specifier is an unsigned integer which is followed by a time unit, a single character of the set {s,m,h,d,y} for seconds, minutes, hours, days, and years.

--max-age=<[lvmspec:]timespec>

age of the oldest snapshot to keep

default: 1y

See --create-interval for the format of the lvm and time specifiers.

--check-interval=<timespec>

the time period between two utilization checks

default: 1m

The utilization of all thin pools which contain at least one thin logical volume specified as an argument to --origin are checked periodically. See --create-interval for the format of the time specifier.

--threshold=<[lvmspec:]data_threshold,meta_threshold>

high watermarks for snapshot removal (1-99)

default: 95,95

The threshold part of the argument is a comma-separated pair of percentages between 1 and 99, inclusively. If the percentage of used space in the data/metadata logical volume of the thin pool exceeds the corresponding threshold value, forced snapshot removal kicks in to bring back the utilization below the thresholds.

The format of the lvm specifier is described in the help text of --create-interval. However, since the utilization is a property of the pool, arguments of the form <vg/tlv> make no sense and are therefore rejected.

--trim-interval=<[lvmspec:]timespec>

discard unused blocks periodically

default: 0

The argument specifies the duration between two successive trims. The default value of zero deactivates this feature.

Trimming is performed in the same way as for the trim subcommand. Errors related to trimming are logged but are otherwise ignored.

See --create-interval for the format of the specifiers.

--exit-hook=<command>

command to be executed before exit

default: true

One possible application for this hook is to inform system manager that no more snapshots are going to be created.

A (quoted) string which describes the error that caused the termination is appended to the given command and the resulting string is passed as a single argument to /bin/sh -c.

--suppress-lvm-warnings

quieten lvcreate(8) and lvremove(8)

suppress

create - create a snapshot of each matching origin
Usage: create [--dry-run] [--] [<lvmspec>]...

This creates one snapshot of each origin which matches the given lvm specifier, ignoring creation intervals, maximal age and utilization thresholds. If no specifiers are given, all origins are regarded as matching so that one snapshot of each configured origin is created.

The subcommand fails if another "run", "create", or "remove" command is currently running.
-n, --dry-run

just print which snapshot would be created

rm - remove one snapshot of each matching origin
Usage: rm [--dry-run] [--] [<lvmspec>]...

The remarks stated in the description of the "create" subcommand apply for this subcommand as well.
-n, --dry-run

just print which snapshot would get removed

ls - print the snapshot list of each origin
Usage: ls [--long] [--] [<lvmspec>]...

The list is sorted by snapshot creation date.
-l, --long

use long listing format

The default output mode lists only the sequence number and the age of each snapshot as human readable text. This option adds additional output.

kill - signal another misma process
Usage: kill [--signal=<signal_number>] [--wait]

This sends a signal to the misma "run" process.
-s, --signal=<signal_number>

send the given signal rather than SIGTERM

default: 15

The standard Unix semantics apply if the specified signal number is zero. That is, no signal is actually sent, and the subcommand exits successfully only if a misma "run" process exists.

-w, --wait

wait until the signalled process has terminated

This option is handy for system shutdown scripts which would like to terminate the misma daemon process.

Without --wait the misma process which executes the kill subcommand exits right after the kill(2) system call returns. At this point the signalled process might still be alive (even if SIGKILL was sent). If --wait is given, the process waits until the signalled process has terminated or the timeout expires.

If --wait is not given, the kill subcommand exits successfully if and only if the signal was sent (i.e., if there exists another misma process to receive the signal). With --wait it exits successfully if, additionally, the signalled process has terminated before the timeout expires.

It makes only sense to use the option for signals which terminate the misma process.

trim - discard unused blocks of origin LVs
Usage: trim [--dry-run] [--] [<lvmspec>]...

Each matching origin LV is expected to contain a mounted and writable filesystem. The subcommand is equivalent to running fstrim(8) on the mountpoints of these filesystems. The full block range of each origin LV is taken into account and the default minimal block size for discards is used. This corresponds to the default values of fstrim(8).
-n, --dry-run

print the mount points, but do not trim

In dry-run mode the mount points are determined as usual, but the command exits without starting any trim operation.

help - list available subcommands or print subcommand-specific help
Usage: help [--long] [--] [subcommand]

Without any arguments, help prints the list of available subcommands. When called with a subcommand name argument, it prints the help text of the given subcommand.
-l, --long

show the long help text

If the optional argument is supplied, the long help text contains the synopsis, the purpose and the description of the specified subcommand, followed by the option list including summary and help text of each option. Without --long, the short help is shown instead. This omits the description of the subcommand and the option help.

If no subcommand is supplied but --long is given, the list contains the purpose of each subcommand.

utilization - show thin pool utilization
Usage: utilization

This prints the percentage of used blocks in the data and metadata logical volumes of each pool.

configtest - run a configuration file syntax test
Usage: configtest

This subcommand checks the command line options and the configuration file for syntactic correctness. It either reports "Syntax Ok" and exits successfully or prints information about the first syntax error detected and terminates with exit code 1.

NOTES

Naming
Snapshots created by misma are named misma-origin.seq, where origin is the name of the thin logical volume (i.e., the second component of the argument to --origin) and seq is a sequence number.

Snapshot Replacement Strategy
Assume that the arguments to --create-interval and --max-age correspond to d minutes and m days, respectively. These two quantities determine the length n of a sequence of snapshots such that

At startup, misma maps each existing snapshot to a slot in an array of length n. When a new snapshot has to be created and not all slots are mapped yet, the new snapshot is mapped to an unmapped slot. If all slots are mapped, an existing snapshot is removed first and its slot is reused. The slot number of the snapshot to be replaced is computed as ffz(seq % (2^n - 1)), where seq is the sequence number of the new snapshot, and ffz(x) is the first zero in the binary representation of x. By properties of the ffz() function, the frequency at which a slot gets reused halves at each step: the snapshot in slot 0 gets reused (roughly) every second time, the snapshot in slot one every fourth time, and so on.

Forced Snapshot Removal
In addition to the normal snapshot removal which takes place when a slot gets reused as described above, snapshots are force-removed when the utilization of a thin pool exceeds its configured thresholds. One snapshot is removed from each affected origin until the utilization drops below the thresholds. If the utilization still exceeds the thresholds after all snapshots have been removed, snapshot creation is suspended.

Forced removal reliably prevents data and metadata exhaustion if the pool is not overbooked. That is, if the sum of the (virtual) sizes of the non-snapshot logical volumes is smaller than the pool size.

Trimming
The trim operation instructs a mounted filesystem to identify blocks which are currently not in use and to pass this information to the underlying block device driver. For a configured misma origin, this driver is dm-thin, which keeps track of the used and unused blocks of each thin pool. The blocks which are freed by the trim operation become available for subsequent snapshots.

A one-shot trim operation is started by invoking the trim subcommand while periodic trims may be configured via the --trim-interval option of the run subcommand.

Trimming is implemented by issuing the FITRIM ioctl on the mount point, which is identical to how the fstrim(8) command works. The mount point is determined from the major and minor device numbers of the block special of the origin by parsing /proc/self/mountinfo.

Activating and Mounting Snapshots
Since thin provisioned snapshots have the activation-skip flag set, one must first activate the snapshot logical volume to create the corresponding device node.

Moreover, the XFS filesystem driver refuses to mount a block device which contains a UUID that is identical to the UUID of an already mounted filesystem. To mount a snapshot of an XFS filesystem, one must therefore tell XFS to skip the UUID check.

See the examples below for suitable command line options for lvchange(8) and mount(8).

Since logical volumes which contain a mounted filesystem cannot be removed, a thin pool which is not overbooked may still run out of space when one of its snapshot logical volumes is still mounted. It is therefore good practice to activate and mount snapshots only for as long as necessary.

EXAMPLES

lvcreate -Zn --type thin-pool -L 1T --poolmetadatasize 15G -n tp vg

lvcreate --thin -n tlv --virtualsize 100G --thinpool vg/tp

misma --origin vg/tlv run

misma --origin vg/tlv --create-interval 1h -- run -d

misma --origin vg/tlv -- ls -l

vg=vg; o=tlv
lvs -o ’lv_path,lv_attr,lv_time,origin’ \  
-S "vg_name = $vg && origin = $o" \  
--config "report/time_format=’%F %R’"

lvchange --ignoreactivationskip --activate y vg/misma-tlv.42

mount /dev/vg/misma-tlv.42 -o nouuid /mnt

misma --origin vg/tlv kill

# global options
origin vg/tlv
loglevel info
# an option for the "run" subcommand
[run]
logfile /var/log/misma.log

COPYRIGHT

Written by Andre Noll
Copyright (C) 2024 Andre Noll
License: GPL-2.0+
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Web page: https://people.tuebingen.mpg.de/maan/misma/
Git clone URL: https://git.tuebingen.mpg.de/misma
Gitweb: https://git.tuebingen.mpg.de/misma.git
Author’s home page: https://people.tuebingen.mpg.de/maan/
Report bugs to Andre Noll

SEE ALSO

lvm(8), fstrim(8), lvmthin(7), dss(1)