Profiles, schedule and hooks

Each endpoint has a profile. The profile sets the defaults for the schedule and for the alert limit. You can change every setting per endpoint afterwards: open the endpoint, then the Settings tab.

The two profiles

	Server	Client
Meant for	Machines that run all the time	Laptops and desktops that are often off or offline
Default schedule	Daily at 22:00, in the time zone of the tenant (Europe/Berlin if none is set)	On connect: when the instance is reachable, at most once every 4 hours
Alert when	No contact for more than 2 hours	No good backup for 7 days. A client never counts as silent, because a laptop is off at night.
Only on AC power	Off	Off

Both profiles start with these defaults for folders and exclusions. You can change them.

System	Folders that are backed up
Linux	/etc, /home, /root, /srv, /var/www
macOS	/Users

The default exclusions leave out caches, temporary files, the trash, node_modules, *.tmp and the restic cache.

“Only on AC power” is off by default for both profiles, because a backup that never runs is worse than one that runs on battery.

Schedule

There are three kinds of schedule.

Kind	Behavior
Daily at a fixed time	Runs at the chosen time in the chosen time zone (daylight saving is handled). A missed slot, for example because the machine was off, is caught up once when the machine is back. A fresh enrollment waits for the next slot.
At a fixed interval	Every N minutes, at least 5. The first run starts right after enrollment.
Whenever the machine is online (on_connect)	Starts as soon as the instance is reachable, at most once every 4 hours by default. You can change the 4 hours.

• Random delay. Each endpoint starts 0 to 10 minutes late, derived from its id, so that many servers do not hit your instance in the same second.
• Retries. After a failed run the agent retries after 5, 15, 30, 60 and 60 minutes, then waits for the next regular slot. A failure that is caused by a lost connection counts as an interruption, not as a failure.
• Resume after an interruption. A backup that was interrupted (agent stopped, machine went to sleep, connection lost) resumes as soon as the instance is reachable, without waiting for the next slot.
• Contact. The agent contacts the instance when it starts and then every 5 minutes, plus or minus 60 seconds. While the instance is unreachable it retries after 30 seconds, doubling up to 5 minutes, and a system suspend triggers an immediate check, so a laptop that just got network is noticed quickly. It fetches its configuration at start, hourly, on request and before every backup.
• One at a time. A machine runs one backup, restore or restore test at a time. Further tasks queue.

Settings

A change is saved in Restow at once. The machine applies it with its next contact, usually within a few minutes. If an agent runs an older configuration than the server holds, it receives the update by itself.

Setting	What it does
Name	Shown in the lists instead of the host name.
Folders to back up	Absolute paths on the machine. Everything below a folder is included.
Exclude patterns	One pattern per line, for example *.tmp or node_modules. Matching files are left out.
Schedule	Kind, time of day, interval in minutes and time zone.
Upload limit (kbit/s)	Limits the upload speed. Empty means unlimited.
Only back up on AC power	A laptop on battery skips the scheduled backup until it is plugged in.
Commands before and after the backup	The hooks, see below.
Retention	How many daily, weekly and monthly snapshots are kept.
Alerts	Hours without contact (server) and days without a good backup (client).

Folders and exclusions

• A configured folder that does not exist is skipped and logged. If none of them exists, the run fails and says so.
• A path that is a symbolic link is backed up through its target. On macOS, /etc and /var are links.
• The agent always leaves out its own cache and temporary folders and every folder named Restow-Restore-*, so earlier restores are not backed up again.
• Snapshots carry the host name recorded at enrollment and the tag restow-agent.

Upload limit

The limit is in kilobits per second, as its name says. 1000 kbit/s is about 1 Mbit/s. The agent converts it to the --limit-upload option of restic in KiB/s, rounded up.

Only on AC power

The check happens before a scheduled start. If the device is on battery, the backup waits. Two limits apply:

• Back up now from the web interface is not held back, and a backup that is already running is not stopped when the charger is unplugged.
• Power detection is best effort. Linux reads /sys/class/power_supply. macOS reads pmset -g batt, and a UPS counts as battery. When the state cannot be determined, the agent assumes AC power, so that a failing detection never blocks backups, and logs that once a day. Systems without battery information count as AC.

Commands before and after the backup (hooks)

A hook is a shell command that the agent runs around each backup. A typical use is a database dump, so that the dump file is what gets backed up instead of a database file that changes while it is read. The web interface suggests, as an example for the command before the backup:

pg_dumpall -U postgres > /var/backups/all.sql

Make sure the folder that receives the dump is one of the folders that are backed up. Other generic uses are stopping a service for the duration of the backup, or freezing a file system with fsfreeze and thawing it afterwards. A freeze blocks writes for as long as it lasts, which is the whole backup.

Hooks run as root

The agent runs hooks as root on the machine, with /bin/sh -c. Whoever may change an endpoint’s hooks can run commands as root on that machine. Hooks are an administrator function: tenant administrators, and provider administrators from the role Administrator. Enter only commands you trust and understand. See Security model.

How hooks behave:

• Environment. A hook gets a short list of variables, not the environment of the service, plus RESTOW_ENDPOINT_ID, RESTOW_RUN_ID and RESTOW_HOOK. The command after the backup also gets RESTOW_BACKUP_STATUS.
• Timeouts. 60 minutes for the command before the backup and 30 minutes for the command after it. On a timeout, the whole process group is stopped.
• A failing command before the backup stops the backup, because the data may be inconsistent. The command after the backup still runs.
• A failing command after the backup turns a successful backup into partial.
• Output. What a hook prints goes into the run log. A hook must not print secrets. The agent masks its own secrets and common secret patterns, and nothing more.
• Length. Each command can be up to 4096 characters.
• Audit log. When hooks are changed, the audit log records that they changed and a fingerprint of them, not their text, because a hook sometimes contains credentials.

There are no snapshots yet

Restow does not yet use LVM, ZFS, btrfs or APFS snapshots. A file that changes while it is read can end up inconsistent in the snapshot. For databases and similar data, use a hook that produces a consistent dump or stops the writer. See Overview.

Run limits and results

• 72 hours. A backup is limited to 72 hours. A stuck restic is then stopped and the run fails.
• A run ends as one of: Succeeded, Partial, Failed or Interrupted.

Partial means the backup finished and a snapshot exists, but some files could not be read, for example because they were open, locked or not readable. Those files are missing from that snapshot. The run lists which. A partial backup also results from a failing command after the backup, and on macOS from missing Full Disk Access. A restore test of a partial backup that passes rates the endpoint Attention instead of Ready.

Interrupted means the agent was restarted during the run (an update, a reboot, a stopped service) and continues by itself. It is not a failure, raises no alert and does not move the time of the last backup.

Open a run under Runs on the overview to read its result, its errors (at most 100) and the end of its log, with secrets masked. If the instance cannot be reached when a run ends, the agent keeps the report and delivers it with a later contact.

Exit codes of backup-now

restow-agent backup-now runs one backup in the foreground, for example to try a new hook or to test the connection.

Exit code	Meaning
0	The backup succeeded.
3	The backup is partial.
1	The backup failed.

Retention

Retention runs on the server once a day for each active endpoint. It keeps the last 30 daily, 12 weekly and 12 monthly snapshots by default. You set the three numbers per endpoint under Retention.

• Only the server removes snapshots. The agent can add backups, never delete them.
• Lowering a number permanently removes older snapshots at the next retention run.
• Retention does not group by host or paths. A repository belongs to one machine, and a change of the backed-up folders does not start a second series of snapshots that is kept forever.
• If the repository is locked because a backup is running, the retention job waits and is retried. That is not a finding.
• A revoked endpoint is no longer pruned.

On the overview of an endpoint, Repository shows the size in storage, the snapshots kept and the last retention run.

Alerts

Under Alerts you set when Restow tells you that an endpoint needs attention:

• Hours without contact for a server. Default: 2.
• Days without a good backup for a client. Default: 7.

The other alert events and how rules and recipients work are described in Alerts and reports.