Security model

This page describes what protects the backups of an endpoint, and where the protection stops. It follows the design of the agent and the server. Read it before you roll the agent out widely, especially the section on root rights.

Outbound only

The agent opens no listening port. It connects out to your Restow instance over HTTPS, and to nothing else. The instance never connects to the endpoint. Tasks such as “back up now”, “restore” or “uninstall” travel in the answer to the agent’s regular contact with the instance.

Append-only, enforced by the instance

The agent writes to a restic repository that your instance exposes. The instance decides what the agent may do, not the agent and not restic:

The agent authenticates with its own secret. The repository endpoint refuses every delete and every overwrite, except for the lock files the agent wrote itself.
The agent also never calls forget, prune or any delete operation itself.
Retention, pruning and integrity checks run only on the Restow server.
The integration tests run the real agent against restic’s append-only REST server and assert that restic forget --prune with the agent’s credentials fails and leaves the snapshots in place.

Further protections of the repository endpoint:

Object names must be a SHA-256 in hex, so no path can leave the repository. The server checks on upload that the content matches its name and stores nothing if it does not.
Bodies over 128 MiB are refused. Requests are limited per endpoint, and wrong credentials are limited per address.
Every attempt by an agent to do something forbidden is written to the audit log (endpoint.repository.denied, at most one entry per endpoint and kind every 10 minutes). A machine that tries to delete its backups is a case for an administrator.
A revoked endpoint is refused and no longer backs up. A tenant that is suspended is refused too.

The agent never holds storage credentials

The agent knows its own agent secret and the repository password of its own repository. It never receives the credentials of the storage target, such as S3 keys or mount paths. It has no access to other endpoints or to the storage target itself.

The agent sends its secret to the repository, so it accepts a repository only on the host of your Restow instance (same scheme and host name), whatever the enrollment answer or its own state.json says. A repository address on any other host is refused, and the credentials are never sent there.

Retention decided by the server

Retention and pruning run on the server, and the server decides what they remove from facts it controls, not from what a machine writes. The time inside a restic snapshot comes from the machine, so it decides nothing:

The server lists the snapshot files in storage and counts only snapshots that a backup run reported.
It dates each one by the earlier of two times: when the storage received its file (the agent can neither set nor change that, because an existing object is never overwritten) and when the reporting run ended, as the server recorded it.
It applies the retention rules (30 daily, 12 weekly, 12 monthly by default) to these dates in the tenant’s time zone and tells restic to forget exactly the snapshots it chose, then prunes.
Snapshots that no run reported, or that are dated in the future, are never deleted. They are marked in the restore points and announced once with the alert endpoint.suspicious_snapshot.

A compromised machine therefore cannot make retention delete real backups by writing snapshots with times in the future. A machine that keeps backing up and reporting junk every day pushes the real daily snapshots out after 30 days. The weekly and monthly ones stay, and the restore tests turn red. That is the same limit as for any machine that backs up junk.

Since 0.4.0 two more layers sit on top. Ransomware detection judges every new restore point of a machine, and while a machine has an unresolved suspicious restore point its retention is paused: nothing is forgotten or pruned until an administrator marks the finding as a false alarm or releases the hold. It is a heuristic and judges what the backup saw change, so a machine that backs up slowly or in a way that looks ordinary can pass it. On a repository with immutable backups, the storage also refuses to delete locked objects, whoever asks.

Storage budgets and locks

Storage budgets. Each endpoint repository has a budget (RESTOW_ENDPOINT_QUOTA_GIB, default 2 TiB, adjustable per machine), and all endpoints of a tenant share another one (RESTOW_ENDPOINT_TENANT_QUOTA_GIB, default 20 TiB). An upload that does not fit is refused, so a machine fills the storage target at most up to its budget. Lock files do not count, so restores keep working. See Overview.
Lock ownership. The server records every lock file an agent writes and lets the agent delete only those. Any other lock, such as the exclusive lock the server holds while it prunes, stays, and the attempt is written to the audit log (endpoint.repository.denied, reason foreign_lock).
Stale locks. Before retention and the repository check, the server removes lock files that the storage received more than 30 minutes ago, whatever they say inside. restic writes a new lock file every five minutes for a lock that is in use.
One maintenance at a time. The server’s own work on a repository never overlaps: retention and the check hold it exclusively, restore tests, snapshot lists, browsing and downloads share it. A machine can still hold maintenance up by keeping a run open and renewing its locks, as a very long backup does, but not unnoticed: six locked attempts in a row over at least 12 hours raise endpoint.repository_locked, and a run without progress is closed after six hours.

One repository and one password per endpoint

The server generates the repository password, stores it encrypted with the tenant key and hands it to the agent at enrollment. Because the server keeps it, an administrator can restore even if the endpoint no longer exists. The server also keeps the password next to the repository, sealed with the tenant key, so the master key and the storage are enough to restore without the database. See Restore.

How the server reaches the repository

Retention, repository checks, restore tests, browsing, downloads and the creation of the repository need more than append-only access. They run as restic processes on the server, through a listener on 127.0.0.1 that lives for exactly one operation. Its credentials are random for that one listener and are passed to restic through its environment. There is no publicly reachable maintenance access and no stored maintenance password. These restic processes receive only PATH, TMPDIR and the RESTIC_* variables, not the environment of the server, so they see neither the database URL nor the master key.

Secrets

Secret	Where it is	How
Enrollment token	Shown once, next to the install command, not in it	Valid 24 hours, single use. Restow stores only a SHA-256 hash. The install script reads it with hidden input or from a root-only file and hands it to `restow-agent enroll` in the environment of that one process. It never appears on a command line, in a URL or in a log. The agent removes it from the environment at once.
Agent secret	`/etc/restow-agent/state.json` on the endpoint	Stored on the server only as a SHA-256 hash.
Repository password	`/etc/restow-agent/state.json` on the endpoint	On the server, encrypted with the tenant key, and next to the repository in the storage target, sealed with the tenant key.

At rest on the endpoint. /etc/restow-agent/state.json holds the agent secret and the repository password in plain text. The file has mode 0600 in a directory with mode 0700, owned by root. The agent corrects looser permissions and refuses a file owned by someone else. Nothing else secret is stored. status.json is readable by everyone and contains no secrets.
In use. restic receives the repository password and the REST credentials only through environment variables, never on a command line. Child processes (restic and hooks) inherit only a short list of variables.
In logs. Everything the agent logs, the log tail sent to the server and error messages pass a redactor. It masks the exact secret values, tokens, Authorization headers, credentials inside URLs and NAME=value pairs whose name suggests a secret. A hook that prints a secret of its own is not covered, see Profiles, schedule and hooks.
On the server. None of these secrets appear in logs or in audit details, and error messages from restic are redacted. The server does not rely on the agent’s redaction: before it stores a run report, it redacts the error messages and the log tail again, including bare values with the prefixes rset_, rsea_ and rsk_ and variables whose name suggests a credential, such as PGPASSWORD=. A hook or restic can print anything.

Integrity of what is installed

Signed releases. Every agent release (the agent, restic and the license notices THIRD_PARTY_NOTICES.txt, for every platform) is signed by the maintainer with an Ed25519 key that never leaves the maintainer’s machine. The signature is in the OpenSSH format (ssh-keygen -Y sign, namespace restow-agent-release) and covers the release’s SHA256SUMS, which lists the SHA-256 of every file. The public key (fingerprint SHA256:2LB7RPpIS0wqbcZ2NA8MjuQN7ZfFPqBX3CwrGFSu76o) is compiled into the agent and written into the install scripts, so a compromised Restow instance cannot make an installed agent run a binary of its own.

The install script checks the signature with ssh-keygen -Y verify (OpenSSH 8.1 or newer) or OpenSSL 3, then the SHA-256 of both binaries and of the license notices, before it runs anything. It prints the fingerprint of the release key it checked against. See Install.
The self-update. The agent checks for a newer version every 6 hours. It first downloads the release’s SHA256SUMS and SHA256SUMS.sig from your instance and checks the signature against the key compiled into it. Nothing of an unsigned or wrongly signed release is used. Then it downloads the license notices, restic (when the release pins another one) and the agent from the same host, in that order, checks each file against its signed SHA-256, stages it under a random name in the root-owned folder, runs the new agent’s version only after these checks and only then replaces the current binary. The old one stays as restow-agent.prev. Updates are applied only while no run is active, and a tenant can pause them with Automatic agent updates on the Inventory page (Servers & clients → Inventory).
restic is pinned to version 0.19.1 with checksums that were verified against the GPG-signed SHA256SUMS of the official release.

Root-owned installation. The binaries live in /opt/restow-agent/bin on Linux and /Library/Application Support/Restow/bin on macOS, the license notices one folder up (/opt/restow-agent/THIRD_PARTY_NOTICES.txt, /Library/Application Support/Restow/THIRD_PARTY_NOTICES.txt, readable by all). The install script and the agent check every folder on the path: each must belong to root and must not be writable by the group or by others. The script refuses to install otherwise. /usr/local/bin/restow-agent is only a link for administrators, and only where /usr/local/bin belongs to root, because a link in a folder a user controls could be swapped and then run with sudo.

The agent does nothing until its installation is safe. Before anything else, the agent checks that its binary, restic and every folder above them belong to root and are not writable by others. If not, it runs no backup, no task and no hook, writes the reason to its log every 30 minutes and tries again every 5 minutes. Running the install command again repairs the installation.

Limit: the first installation. The first installation trusts the install script your instance serves, as any curl | sh does. To rule out a compromised instance at that moment, compare the key fingerprint the script prints with SHA256:2LB7RPpIS0wqbcZ2NA8MjuQN7ZfFPqBX3CwrGFSu76o (also published in the release notes), or check the release by hand. See Install.

The agent runs as root

The agent must read every file it backs up and write restores anywhere, so it runs as root and the file system stays fully visible and writable to it.

Hooks need the machine’s consent. Hooks are commands set in Restow that the agent runs as root before and after a backup. Whether a machine runs them is decided on the machine, by root, and Restow cannot change it:

off (the default): no hook from Restow runs. A hook set anyway is skipped, and the backup is partial.
scripts: only scripts that root put into /etc/restow-agent/hooks.d, named in the hook, run directly without a shell and without arguments.
any: any command runs through /bin/sh -c. With this policy, whoever can change the endpoint’s settings in Restow can run commands as root on that machine.

Restow shows each machine’s policy, accepts only script names for a scripts machine and refuses hook settings the machine would not run. Setting or changing a hook also needs a sign-in from the last 10 minutes. That check adds to the machine’s consent, it does not replace it. See Profiles, schedule and hooks.

Service hardening. The systemd unit adds what a backup agent that reads and restores everything still allows: NoNewPrivileges, ProtectKernelTunables, ProtectKernelLogs, ProtectControlGroups, ProtectHostname, no loading of kernel modules, RestrictNamespaces, RestrictRealtime, LockPersonality, native system calls only and UMask=0077. On macOS the LaunchDaemon sets the umask 077. ProtectSystem, ProtectHome and PrivateTmp are not used on purpose, because backups of any path and restores into it need the whole file system. Hooks inherit the hardening: a hook that needs namespaces (rootless containers) or loads kernel modules does not run.

The service runs with low CPU and I/O priority and is restarted by systemd or launchd if it stops.

Who may do what

Most endpoint functions in the web interface and the API need a tenant administrator or a provider administrator. Since 0.4.0 one part is open to the restore role as well: restore operators (membership role “restore”) open Servers & clients → File restore, list the tenant’s machines, read a machine’s restore points and whether they are immutable, browse them, download files as a ZIP and restore files onto the machine. Backing up (it answers 403), settings, tokens, jobs, restore tests, revoking, uninstalling and the repository password stay with administrators, and every read, download and restore is audited with the operator as actor. See What a restore operator reaches. Provider team roles are split like this:

Provider role	May
Read only	See the list, the details, the runs and the restore points.
Technician	Back up now, restore, restore test, browse and download backed-up files.
Administrator	Create and revoke install commands, change the configuration and hooks, revoke and uninstall endpoints, remove revoked machines from Restow (with a recent sign-in), show the repository password.

See Team and roles for the roles themselves.

Agent policies (since 0.4.0: a backup window, catching up, an upload limit, AC power and retries) are set at three levels, see Agent policies. They change when and how fast an agent backs up, not what it may do to the repository: the append-only rules above stay as they are. Every change is audited (settings.agent_policy.updated, tenant.agent_policy.updated, endpoint.config.changed).
Hook texts are shown only to those who may change the endpoint’s settings. Read only and Technician see whether a hook is set and a short fingerprint.
A recent sign-in is needed to set or change a hook and to show the repository password: a sign-in from the last 10 minutes with a passkey, a password with an authenticator code, or single sign-on (OIDC). Otherwise Restow asks you to confirm it is you first (recent-sign-in-required). Removing all hooks of a machine needs no confirmation.

What a compromised endpoint can and cannot do

The endpoint holds the password and the credentials of its own repository, because it has to write and to read for restores. An attacker with root on the endpoint therefore can:

add new backups, including junk (restore tests and repository checks rate that, and retention cleans up),
read its own repository, and so decrypt it,
not delete or overwrite a backup, and not write a new repository configuration. The instance enforces this,
not make retention delete real backups, because the server decides from its own records, not from the times inside the snapshots,
not lift the server’s locks, and not hold maintenance up unnoticed,
not fill the storage target beyond the machine’s storage budget,
not reach other endpoints, other tenants or the storage target.

A stolen agent secret gives the same: writing new backups to that endpoint’s repository and reading it, but not deleting anything. To cut it off, revoke the endpoint in Restow.

Audit log

Every read of backed-up files and every change is recorded in the audit log, in every edition. Viewing the log (list, filter, details, chain verification) belongs to Business and Service Provider and is available, as is the audit log export (CSV, JSON with hashes) since 0.3.1. See Audit log.

Action	Recorded when
`endpoint.token.created`, `endpoint.token.revoked`	An install command is created or revoked.
`endpoint.enrolled`	A machine enrolls (host name, system, architecture, profile, token id, never a secret).
`endpoint.config.changed`	Settings change. For hooks, a fingerprint, not the text.
`endpoint.backup.requested`, `endpoint.restore_test.requested`	Someone starts a backup or a restore test.
`endpoint.backup.cancelled`	Someone stops a running backup or withdraws a requested one (since 0.4.2).
`endpoint.update_requested`	Someone chooses Update to X.Y.Z for a machine, or the server update asks it to update (since 0.4.2).
`endpoint.snapshot.browsed`, `endpoint.snapshot.downloaded`	Backed-up files are browsed or downloaded.
`endpoint.restore.requested`, `endpoint.restore.finished`	A restore onto the machine is requested and when it finishes.
`endpoint.repository.password.revealed`	Someone shows the repository password.
`endpoint.uninstall.requested`, `endpoint.revoked`	An uninstall is requested or an endpoint is revoked.
`endpoint.removed`, `endpoint.removal.held`, `endpoint.removal.deferred`, `endpoint.removal.completed`, `endpoint.removal.failed`	A revoked machine is removed from Restow (since 0.4.5), and what happens to its backups: held by a legal hold or a suspicion, deferred by a storage lock, deleted, or the deletion failed.
`endpoint.repository.denied`	An agent tried something the append-only rules forbid, including deleting a lock it did not write (`foreign_lock`).
`endpoint.updates.paused`, `endpoint.updates.resumed`	Automatic agent updates are paused or resumed for the tenant.

Known limits

No mTLS yet. Version 1 authenticates each agent with a per-agent secret over HTTPS. A stolen secret (root on the endpoint) allows writing new backups to that endpoint’s repository and reading it. It does not allow deleting anything. mTLS per agent is planned for a later release.
The first installation trusts the install script. The script comes from your instance. Compare the release key fingerprint it prints with the published one (SHA256:2LB7RPpIS0wqbcZ2NA8MjuQN7ZfFPqBX3CwrGFSu76o). After that, signatures protect the binaries and every update.
Root and hooks. The agent and its hooks run as root. With the hook policy any, whoever may change the endpoint’s settings in Restow can run commands as root on the machine. Prefer scripts.
Metadata the agent adds. restic prune trusts the index files of the repository, and the agent may add new ones. A compromised agent could add index or snapshot files that restic cannot read. Retention or the repository check then fail, and the weekly check rates the repository damaged (scrub.corrupt). Nothing is deleted that the server did not choose, and a failed retention run writes a report with the error.
The repository password decrypts the whole backup of an endpoint. Anyone who has it and can read the storage target can read every backed-up file.
Append-only is enforced by the instance, not by the storage, unless you lock the storage. Someone who can reach the storage target directly can delete data. Since 0.4.0 a repository can have immutable backups: on an S3 bucket with Object Lock, the storage then refuses to delete or overwrite the data/, snapshots/, keys/ and config objects of every machine’s repository before their date, also for someone who holds Restow’s storage key. It is off until an administrator turns it on, and it was tested against a fake S3 service, not yet against a real one. restic’s locks/ and index/ files are not locked (restic repair index rebuilds the index from the locked packs). A directory gets only a deletion delay that Restow keeps itself, which is not immutability.