Endpoint backup overview

Endpoint backup protects servers and client machines, such as laptops and desktops, file by file. A small program, the Restow agent, runs on the machine, drives restic and sends the backup to your Restow instance over HTTPS. Backups end up in the storage target of the tenant, like every other backup in Restow.

restic is the well-known open-source backup program, and Restow ships its unmodified release (verified against pinned checksums; the agent’s own release signature covers it too). Each endpoint’s backup is therefore a regular restic repository, which restic itself can open with the repository password, also without Restow (see Restore). restic is an independent open-source project under the BSD-2-Clause license (restic.net); Restow is not affiliated with it. The licenses of restic and of the 79 Go modules compiled into it are installed with the agent, in THIRD_PARTY_NOTICES.txt next to its binaries, and listed in THIRD_PARTY_NOTICES.md of the Restow source code.

This section uses the word endpoint for such a machine. The web interface says “machine” in places. An endpoint has one of two profiles: Server or Client.

What you get

File-based backups with restic 0.19.1. Deduplicated, compressed and encrypted. Single files and folders are backed up and restored.
One repository per endpoint. Each endpoint has its own restic repository with its own password. The password is generated by the server and stored there, encrypted with the tenant key.
Append-only. The agent can add backups. It can never delete or overwrite one. The instance enforces this, not the agent. Retention, pruning and integrity checks run on the Restow server only, and the server decides from its own records which snapshots retention removes.
Outbound only. The agent opens no port. It talks to your Restow instance over HTTPS and to nothing else.
Signed releases, root-owned installation. Every agent release is signed by the maintainer. The install script and the agent’s self-update check the signature before anything runs, and the agent lives in a folder only root can change. See Install.
Restores never overwrite. A restore goes into a new folder on the machine, or you download files as a ZIP from the web interface.
Restore tests. Restow reads sample files of the newest backup back and compares their SHA-256 hashes. The result feeds Recovery readiness like mailboxes and OneDrives do.
Agent policies (since 0.4.0). A backup window, catching up missed backups, an upload limit, AC power and retries, set for the installation, a tenant or a backup job. See Agent policies.
Data folders a job misses (since 0.4.0). The agent reports which common Linux data folders exist, and the job editor and the machine’s page offer to add the ones a job leaves out. See Profiles, schedule and hooks.
Ransomware signs and immutable backups (since 0.4.0). Every new restore point is judged for signs of ransomware, and the storage can lock restic repositories. See Ransomware detection and Immutable backups.
File restore for the restore role (since 0.4.0). Restore operators browse, download and restore files of machines. See Restore.
Storage budgets per machine and per tenant, so that one machine cannot fill the storage target. See Storage budgets.
Alerts for silent servers, clients without a recent good backup, failed backups, failed restores, failed restore tests or repository checks, suspicious snapshots, storage budgets and repositories that stay locked.
Pre and post commands (hooks) around each backup, for example to dump a database. They are off until root on the machine allows them. See Profiles, schedule and hooks.

Supported systems

System	Supported in 0.4.7
Linux	With systemd, on x86_64 and aarch64. The binaries are static and run on any distribution with systemd.
macOS	macOS 13 (Ventura) or newer, on Intel and Apple Silicon.
Windows	Planned, not part of 0.4.7. There is no Windows build, installer or service. The web interface shows Windows as “Planned” and does not let you choose it. See the roadmap.

How it fits together

In the web interface you create a server or client. Restow creates a one-time install command.
You run the command on the machine and paste the one-time token when it asks. The script downloads the agent and restic from your Restow instance, checks the maintainer’s signature and the checksums, installs them root-owned with a service and enrolls the machine. See Install on Linux and macOS.
The agent contacts your instance every 5 minutes, every minute while a backup runs since 0.4.2 (a “heartbeat”), and receives tasks, fetches its configuration when it needs it, and starts backups on its schedule. The instance never connects to the machine: tasks such as “back up now” or “restore” travel in the answer to the heartbeat.
The backup data goes to a restic repository that your instance exposes to that one agent in append-only mode. The repository lies in the tenant’s primary storage target, below endpoints/<endpoint id>/.
The server applies retention, checks the repository and runs restore tests.

Where endpoints appear

The section Servers & clients in the sidebar: Inventory lists every endpoint, with the chips All, Servers and Clients, and opens a machine’s page; File restore restores or downloads files from a machine’s restore point. Tenant administrators and provider administrators see both entries. Since 0.4.0 restore operators see File restore, and nothing else of this section. Since 0.4.0 revoked machines are hidden in the inventory until Show revoked machines, and File restore lists them in the group Revoked machines. Since 0.4.5 a click on a row of Inventory opens a detail panel of the machine (?machine=<id> in the address, so a link opens it): every state, the full error and warning texts of the newest backup in a copyable block (for a machine also the end of the log), what needs attention, the restore checks, the key facts, the newest runs and the actions, including Back up now, Stop, Revoke now and, for a revoked machine, Remove from Restow. Inventory keeps a fixed layout, with one compact State badge per row. Below the machines it lists the removed machines whose backups are still stored (Uninstall). The section’s entry Jobs holds the backup jobs for servers and clients (from 0.2.0). Since 0.2.1 a machine is backed up only while it is in a job: Inventory marks every machine without one as Without backup and offers Create a job and Add to job…, and a machine can be assigned to a person of the directory (Assigned to).
The Recovery readiness overview, in a section Servers and clients. Endpoints count in the tenant summary too.
The integration API: GET /api/v1/endpoints lists them, and the recovery readiness data carries an endpoints field. A failed endpoint run also triggers the job.failed webhook. See Integration API.

Alerts

Endpoint backup uses the same alert rules, recipients and channels (bell, email, webhook) as every other job. See Alerts and reports.

Event	When it fires
Server or client not reporting (`endpoint.stale`)	A server has not been in contact for more than 2 hours. A client has had no good backup for 7 days. Clients never count as silent, because a laptop is off at night. Both limits can be set per endpoint.
Backup failed (`backup.failed`)	A backup run failed. This includes a run whose agent disappeared and that was closed after 6 hours without progress.
Backup overdue (`backup.overdue`, since 0.3.1)	A server or client has had no successful backup for longer than its backup jobs plan, or longer than the rule’s own deadline (24 to 720 hours). Once per machine, until a backup succeeds again.
Restore failed (`restore.failed`)	A restore run on the endpoint failed.
Restore check failed (`verify.red`)	A restore test found proof that the backup is damaged (a hash that differs, data missing or damaged in the repository). A test that could not complete raises nothing. `verify.recovered` follows when a later test passes.
Storage damage found (`scrub.corrupt`)	The repository check found damage.
Suspicious snapshot of a server or client (`endpoint.suspicious_snapshot`)	Retention found a snapshot that no backup run reported, or one dated in the future. Retention keeps it, and the machine may be compromised. Once per snapshot.
Suspicious restore point (`backup.suspicious`, since 0.4.0)	Ransomware detection found signs of an attack in a new restore point of the machine. Once per attack episode, and retention of that machine is paused. See Ransomware detection.
Storage budget of servers and clients (`endpoint.storage_quota`)	A repository reached 90 percent of its storage budget, the budget is used up, or an upload was refused. Also, at most once a day, when all servers and clients of a tenant together reach 90 percent of the tenant budget.
Repository of a server or client stays locked (`endpoint.repository_locked`)	Retention and checks found the repository locked six times in a row, over at least 12 hours.

An existing rule for backup failed and restore check failed therefore covers endpoints without a change. A run that was interrupted because the agent restarted (an update, a reboot) is not a failure: it continues by itself and raises no alert and no job.failed webhook. Neither is a restore test on the machine that could not complete, also one the server closed after six hours without a report: it raises no alert and no job.failed and is offered again after 1 to 24 hours (see Verification in detail).

Retention and repository checks

Retention runs on the server once a day for every active endpoint. The default keeps 30 daily, 12 weekly and 12 monthly snapshots. You can change the numbers per endpoint. Lowering a number permanently removes older snapshots at the next retention run. A revoked endpoint is no longer pruned. A machine that was removed from Restow with the choice to keep its backups gets an end date instead (the newest backup plus the longest rule of its retention), after which they are deleted (Uninstall).
The server decides what retention removes, from facts it records itself: only snapshots that a backup run reported count, each dated by when the storage received its file or when the reporting run ended, whichever is earlier. The time written inside a snapshot decides nothing. Snapshots that no run reported, or that are dated in the future, are never deleted, are marked in the restore points and raise the alert Suspicious snapshot. See Profiles, schedule and hooks.
Repository check runs once a week for every endpoint. It runs restic check on one twentieth of the stored data, in a fixed order, so the whole repository is read over roughly five months. Damage turns the endpoint Not restorable and raises scrub.corrupt.
Restore tests run after every new good backup. See Restore.

Ransomware detection and immutable backups

Both came with 0.4.0 and apply to machines like to every other kind of backup.

Ransomware detection judges every new restore point of a machine from data the backup already has: the amount of change (new and changed files, bytes added) against the machine’s own history. The agent 0.4.0 adds an activity report (deletions, renamed extensions, known ransomware extensions and ransom notes, the entropy of sampled new files) and the data for the compression signal. An older agent sends neither, and a signal without data says “not available”, never clean. A suspicious restore point gets a red badge, and retention keeps the machine’s restore points until an administrator resolves it. It is no virus scanner and never blocks a backup. See Ransomware detection.
Immutable backups lock the objects of the machine’s restic repository (data/, snapshots/, keys/ and config) with S3 Object Lock, when the tenant’s repository has immutability on. Restic’s locks/ and index/ files are not locked. A machine’s restore points show Immutable until <date>, green only for a compliance lock that the storage itself reported. See Immutable backups.

Storage budgets

An agent can only add data, but adding is enough to fill a storage target that also holds the mailbox backups. Every endpoint repository therefore has a budget, and all endpoint repositories of a tenant share another one.

Per machine: RESTOW_ENDPOINT_QUOTA_GIB, default 2048 (2 TiB). You can set another value for one machine under Storage budget in its settings. Empty means the installation’s default.
Per tenant: RESTOW_ENDPOINT_TENANT_QUOTA_GIB, default 20480 (20 TiB), for all servers and clients of a tenant together.
0 switches a budget off. Both variables are read by the api and the worker role. See Environment variables.
An upload that does not fit is refused, so restic stops and the backup fails. Lock files do not count against the budget, so restores keep working when the budget is used up. Uploads that run at the same time can go a few packs beyond it.
The Backup data card on the machine’s page shows the usage against both budgets, which budget applies, the last refused upload and, from 90 percent, a note. The alert Storage budget of servers and clients warns at 90 percent and when uploads are refused.
Since 0.4.1 the Stored figures of the installation (Repositories → Usage, the dashboard tile, the statistics and the API) include the restic repositories of machines, file shares and Proxmox VE containers. Before, they counted the chunk store only. A repository’s size has no history, so the growth figures still come from the chunk store alone.

Agent updates

The agent checks for a newer version a minute after it starts and then every 6 hours, and installs only releases signed by the maintainer, only while no run is active. The switch Automatic agent updates on the Inventory page pauses updates for every machine of the tenant, also for machines added later. It appears once the tenant has at least one machine. Several settings of an agent policy, the data folders and the machine facts need the agent 0.4.0, so a tenant that paused updates keeps old agents without them.

Since 0.4.2 the agents follow a server update, and you can ask for an update yourself:

Agents follow the server. After the server has been updated, new runs stay paused while the agents (and the Proxmox VE nodes) install the new release, for 15 minutes at the longest. Every agent behind the release is told with its next heartbeat to update now, and installs it as soon as it is idle. An agent whose tenant or machine paused updates is not waited for. The Updates tab shows the progress and names the command for agents that were not reached. See Nodes and agents follow the update.
Update to X.Y.Z on the page of a machine (the button shows whenever a newer release is offered, the version in its label) asks the agent to install it with its next contact, as soon as no backup or restore runs on the machine, and to restart. It is audited as endpoint.update_requested. While updates are paused for the machine or its tenant, the page says so.
sudo restow-agent update on the machine installs the offered release at once, through the same signed download as the automatic update, restarts the service and says what it did (“No newer release is offered” when there is none). While a backup, a restore or a restore test runs, it refuses (“A backup is running; the update waits”, exit code 1) and the agent installs the update itself as soon as it is idle. Agents before 0.4.2 do not know the command: restart their service (sudo systemctl restart restow-agent, on macOS sudo launchctl kickstart -k system/com.restowbackup.agent) and they look for the update a minute later. See Commands.

Stopping a backup

Since 0.4.2 a running backup of a server or client can be stopped in the web interface, and a requested one withdrawn:

Stop backup is on the page of the machine and in the run’s detail (History). It asks first. Whoever may start a backup may stop one, and the audit log records endpoint.backup.cancelled with the person who asked.
From Inventory and for a whole job (since 0.4.5). Stop is also in the row of Inventory, its … menu and the machine’s detail panel, and Stop in Servers & clients → Jobs stops every queued or running backup of a job’s machines in one request (Backup jobs); it is audited as backup_job.run_cancelled on the job and endpoint.backup.cancelled per machine.
A backup that has not started (the request waits on the server) is withdrawn at once (“Withdraw backup”): the agent never receives it and the task shows as withdrawn.
A running backup gets a cancel request, and the page says “Cancelling…”. The agent 0.4.2 or newer receives it with its next heartbeat (every minute while a backup runs, every 5 minutes otherwise) and ends the run: restic gets an interrupt and stops cleanly, so no snapshot and no lock is left behind, and the command after the backup runs as it always does. Expect it within about a minute.
The end is “Cancelled”, not a failure. The run shows Cancelled (the error code is cancelled, “Cancelled in the web app”, with “Cancelled by” and the name of the person). It raises no alert and no job.failed webhook, the time of the last backup and the count of failed backups stay as they were, and the agent does not resume the run: the next scheduled backup starts as usual.
Older agents. An agent before 0.4.2 does not know the request, so its running backup runs to the end. If it starts a task that was withdrawn, the server refuses the run (409) and a current agent does not try again.

A backup of a Proxmox VE guest and a run of a file share have their own Stop (Proxmox VE, file shares).

What it is not

Not a disk image. There are no images and no bare-metal restore. Restoring a whole machine means reinstalling the operating system and restoring the files.
Not for Windows yet. Windows is planned.
No snapshots of the file system yet. LVM, ZFS, btrfs and APFS snapshots are not used. Consistency comes from optional hooks.

Known limits

No mTLS yet. Each agent authenticates with its own secret over HTTPS. See Security model.
No LVM, ZFS or btrfs snapshots, and no APFS integration. Files that change while they are read can end up inconsistent in a snapshot.
File-based only: no disk images, no bare-metal restore.
No Windows in 0.4.7.
The first installation trusts the install script that your instance serves, as any curl | sh does. Compare the release key fingerprint the script prints with the published one (SHA256:2LB7RPpIS0wqbcZ2NA8MjuQN7ZfFPqBX3CwrGFSu76o). See Install.
RHEL, Rocky Linux and AlmaLinux 8 and Amazon Linux 2 have no tool the install script can check the signature with. There you check it on another machine and pass the result to the script.
Hooks with the policy any run any command that the endpoint’s settings in Restow name, as root. Prefer scripts.
Linux without systemd is not supported by the installer and the service commands.
Whether the machine is on battery is detected on a best-effort basis, and a running backup is not stopped when the charger is unplugged.
A machine runs one backup, restore or restore test at a time. Further tasks queue.
Revoking an endpoint deletes nothing: its data stays in storage. Since 0.4.5 a revoked machine can be removed from Restow, with its backups deleted at once or when they expire; before 0.4.5 there was no deletion of an endpoint’s repository.
While a Restow update waits or runs, agents older than 0.4.0 do not hold their own schedules: a backup such a machine starts by itself during that time runs, and may fail if the api restarts under it. See Agent policies.
Agent policies, ransomware detection and immutable backups were tested against simulated services and simulated attacks only, not against real systems.
Changing a tenant’s primary storage target is refused while endpoints are active (HTTP 409, storage-active-endpoints), because an active agent keeps writing to the old primary while it is copied. To change it, revoke or uninstall the machines, change the primary target with a move, then enroll the machines again: they become new endpoints with new repositories in the new primary target. Since 0.4.2 that move copies the restic repositories of the revoked machines (and of containers and file shares) to the new primary, checked by size and SHA-256, so their backups can still be browsed and downloaded from there and the old target can be removed afterwards. With “keep existing backups where they are” they stay on the old target instead, which stays attached read-only. See Replacing the primary repository and Backing up Restow itself.
As of the 0.2.1 beta, the project’s tests ran the agent on Linux (Debian, arm64, in Docker) and natively on macOS 26 (Apple Silicon). The Linux install script ran end to end in a Linux container, and the macOS install script ran natively on macOS 26 as a normal user below a relocated root. The systemd unit and its hardening ran under a real systemd (Debian 12, in a privileged container). The amd64 builds were cross-compiled and checked but not run, and the LaunchDaemon was never started by a real launchd. The release smoke run installed the real install script only in a Linux container (check 10); the macOS installer and agent were not part of the release checks. Do a first install on a test machine, then run restow-agent status, restow-agent backup-now and a restore of a folder before you roll it out.