Restore

There are three ways to get files back from an endpoint backup. None of them overwrites anything on the machine.

Way	Use it when
Restore onto the machine	The machine is running and its agent is connected. The files land in a new folder.
Browse and download as ZIP	You want the files on your own computer, or the machine is gone.
Restore without Restow	Restow itself is not available.

All three start from the Snapshots tab of the endpoint (“Restore points”): open the machine under Servers & clients → Inventory, pick a snapshot, and click through its folders. Servers & clients → File restore offers the same browser and restore dialog without opening the machine first: choose a machine, then a restore point. Select files or folders in the list. A selected folder includes everything inside it. Every successful backup creates a snapshot. Restore operators work in File restore only.

A restore point carries two marks that come from other features:

Immutable until <date> appears when the storage itself confirms a compliance lock on the restore point. See Immutable backups.
Suspicious appears when ransomware detection found signs of an attack in it. When the newest restore point is suspicious, the restore views offer the newest one without findings.

The file browser says which folders a restore point contains (This restore point contains:), with a link to the job’s folders, and a notice when a data folder of the machine is not in it. See Data folders a job misses.

Restore onto the machine

Select the files and folders, then choose Restore to the machine.
Optionally enter a target folder: an absolute path on the machine.
Confirm with Request restore.

The machine starts the restore with its next contact, usually within a few minutes. If it is off or offline, the request waits until it is back. You can follow the run under Runs on the overview of the machine, which shows the folder it restored into.

Default target. Without a target folder, the agent creates a new folder named Restow-Restore-<yyyyMMdd-HHmmss> inside the first backed-up folder that only root can change. If there is none, it uses / on Linux and /Users on macOS.
The target is a plain absolute path, without .., ., double or trailing slashes. Otherwise the run fails with the code invalid_task.
Every folder above the target must be changeable by root only. Its parent must exist. A folder that anyone may write to with the sticky bit set, such as /tmp, is fine. A target inside a user’s home folder is refused with the code target_unusable, because a user who owns a folder on the way could swap the new folder for a link while root restores into it.
The agent creates the target with mode 0700, so only root can enter it. The restored files keep their own modes and owners.
The target must not exist or must be an empty folder owned by root. Otherwise the run fails with the code target_not_empty, and nothing is touched. The agent also tells restic --overwrite never and --verify. The files that are on the machine now stay as they are.
File names with *, ? or [ are matched exactly.
You can select up to 200 items per restore.
The agent leaves folders named Restow-Restore-* out of later backups, so restored files are not backed up a second time.
A revoked endpoint takes no restore, because it takes no tasks. Download the files as a ZIP instead. Revoked machines are hidden in the inventory until Show revoked machines, and File restore lists them in their own group, Revoked machines. A machine that was removed from Restow can still be browsed and downloaded while its backups are stored (since 0.4.6), but never restored onto: the list of removed machines on Inventory links Restore files (Uninstall).

The request and the end of the restore are recorded in the audit log (endpoint.restore.requested, endpoint.restore.finished). If a restore fails, see Troubleshooting.

Browse and download as ZIP

Browsing and downloading work on the server. The machine does not have to be online, and a revoked endpoint can still be browsed and downloaded, and so can a removed machine until its backups are deleted (since 0.4.6).

Select the files and folders, then choose Download as ZIP.
Your browser downloads the ZIP.

How it behaves:

Up to 10,000 paths per ZIP. The server reads every file or folder you select on its own with a separate restic process, so thousands of single files take a while. A parent folder is much faster, and the interface points that out from 50 single files.
Browsing lists one folder at a time, page by page. The interface loads the first page when you open a folder and more with Load more. A page holds up to 5,000 entries (1,000 by default). Select the folder itself to include everything inside it.
The ZIP shows the folder you selected (docs/a.txt, not home/anna/docs/a.txt).
Entry names are safe to unpack anywhere. Backslashes, colons and control characters in names become _, and parts that consist only of dots are dropped. A Linux file name such as ..\..\x therefore stays a file name when the ZIP is unpacked on Windows.
Symbolic links and special files carry no content and are not in the ZIP. The interface marks them.
Restow checks first that every selected path exists in the snapshot. If one does not, the request fails instead of breaking off in the middle of a download.
The ZIP is built while it is sent and nothing is cached on disk. If reading fails along the way, the connection is broken off, so you never get a ZIP that looks complete and is not.
Large folders over very slow connections: the server keeps the connection open as long as restic delivers. A proxy with a short idle timeout in front of Restow can cut it.
The server runs a limited number of restic processes at the same time (6 in total, 3 per tenant). If the limit is reached, the interface asks you to try again in a moment.
While the server runs retention or a repository check on the repository, browsing and downloading wait up to two seconds and then answer The backup data is busy (endpoint-repository-locked). Try again in a minute. A prepared download that was turned away this way can still be started.

Browsing and downloading are recorded in the audit log (endpoint.snapshot.browsed, endpoint.snapshot.downloaded).

File restore for restore operators (since 0.4.0)

People with the tenant role Restore operator (the membership role “restore”, tenant_restore) open Servers & clients → File restore. Before 0.4.0 they could restore mailboxes and OneDrives but not machines. In File restore they can:

list the tenant’s machines, with the revoked ones in the group Revoked machines,
read a machine’s restore points and whether they are immutable,
browse the files of a restore point,
download files and folders as a ZIP,
restore files onto the machine, into a new folder as described above (not onto a revoked machine).

What stays with administrators: a machine’s settings, install commands, jobs, backing up (Back up now answers 403 for a restore operator), restore checks, the run log, agent updates, revoking or uninstalling a machine, and the repository password. The web app leaves out the links to pages a restore operator may not open, so there is no Inventory and no machine page for them. The machine’s agent policy is read-only for everybody, and the route that serves it is open to restore operators, but the page that shows it is the administrators’.

Every read, download and restore is recorded in the audit log with the operator as actor (endpoint.snapshot.browsed, endpoint.snapshot.downloaded, endpoint.restore.requested). See Audit log.

For Proxmox VE, the same role lists guests with a restore point in File restore and browses and downloads the files of container and VM restore points, but does not put a whole guest back, back it up or boot-test it. See Files of a guest. More on the roles: Team and roles.

Restore without Restow

The backup of an endpoint is a plain restic repository. With the repository password and access to the storage target, you can restore it with restic alone, even if your Restow server is gone.

Open the machine under Servers & clients → Inventory, the Settings tab, and the card Restore without Restow.
Choose Show repository password and confirm. If you signed in more than 10 minutes ago, Restow first asks you to confirm it is you (with a passkey, or by signing in again). Restow then shows the password. It is not stored in the browser and is hidden again when you close the window.
Keep the password in a password manager.

Showing the password is recorded in the audit log (endpoint.repository.password.revealed).

On a local storage target, the repository is the folder endpoints/<endpoint id> inside the storage path. The command then reads:

restic -r /path/to/storage/endpoints/<endpoint id> restore latest --target /restore

Replace /path/to/storage with the folder of your storage target. restic asks for the repository password. The command restores the newest snapshot into /restore. For an S3 storage target, open the bucket with restic directly (-r s3:..., with endpoints/<endpoint id> as the prefix), as long as the repository lies at that path.

With the master key only

The server also keeps every repository password next to its repository, sealed with the tenant key: endpoints/<endpoint id>/restow-repository-password.json. It writes the file when a machine enrolls, and every retention or check run replaces a damaged or outdated one. restic ignores the file, and the agent can neither read nor replace it. The master key (KEK) and the storage are therefore enough, even without the database and without a running Restow server. The standalone tool restow-restore (see Backing up Restow itself) recovers the password, and plain restic restores the files:

restow-restore endpoint-password --storage /path/to/storage --key <KEK file or variable> \
  --endpoint <endpoint id> --out /root/endpoint-password
restic -r /path/to/storage/endpoints/<endpoint id> --password-file /root/endpoint-password \
  restore latest --target /srv/restore

--key takes the master key (RESTOW_MASTER_KEY) as a file path or as the name of an environment variable that holds it. --out writes the password into a new file with mode 0600 and refuses to overwrite an existing one. Without it, the password goes to standard output and nothing else does. The endpoint id is the folder name below endpoints/, and the web interface shows it on the machine’s page. Delete the password file when the restore is done.

Restore tests

A backup that nobody has read back is unproven. After every new good backup, Restow proves that files can be read back.

How it works

After each successful backup, the agent picks up to 20 random files from the snapshot and reports their path, SHA-256 and size. The files are regular, non-empty, at most 256 MiB each and at most 1 GiB in total. The hash is taken from the file on disk, but only for files that are provably unchanged since the snapshot (same size and modification time before and after hashing). A later mismatch therefore points at the backup, not at an edit made afterwards.
The server reads these files out of the repository with restic dump, computes their SHA-256 and compares it with the reported value. The result is green only if all hashes match and at least one file was tested. Red needs proof: a hash that differs, or restic reporting that the file is not in the snapshot or that the data it needs is missing or damaged. A read that failed for any other reason (the repository busy, restic unable to start or to reach the repository) rates nothing, and the test is offered again.
Additionally, the server gives the agent a verify_sample task with the same files. The agent restores them with one restic restore into a temporary folder below its data directory, hashes each file, deletes the copy and reports what it found. The server judges that report by the same rules: a differing hash, a file missing after a restore that ended without error, or restic’s own findings rate it red; anything else (a full disk, an agent stopped during the test, an unreachable repository) rates nothing. Such a test shows as Not completed, will be retried and is offered again after 1, 2, 4, 8, 16 and 24 hours while the backup is the machine’s newest. A judged result becomes a second report for the same snapshot.
If either of the two is red, the endpoint is red. Details: Verification in detail.

You can start a test yourself with Start restore check on the machine’s page. It needs a successful backup first. The results are listed under Reports, with the differences if there are any. The reports say whether a result was “Read back by the server” or “Restored on the machine”. The server keeps the samples for 90 days.

Readiness states. The rating belongs to the backup that was checked. A green test of an older backup does not count for a newer one.

State	Meaning
No backup yet (`no_backup`)	There is no good backup yet.
Unverified (`unverified`)	A backup exists, but no restore test has checked exactly this backup.
Ready (`green`)	The restore test of the newest backup passed.
Attention (`yellow`)	The test passed, but the backup was partial: some files could not be read.
Not restorable (`red`)	A restore test of the newest backup failed, or a newer repository check found damage.

A rating that is older than 8 days is shown as Overdue. Endpoints appear in the Recovery readiness overview next to mailboxes and OneDrives. See How Restow checks that a backup can be restored for the same idea applied to mailboxes.

A restore test reads back and verifies: it proves that the sampled files come back intact. It does not prove every file, it restores nothing onto the machine, and it does not test a full restore of the whole machine.

Repository check and retention

Two more jobs run on the server, without the machine.

Repository check: once a week, restic check reads one twentieth of the stored data, in a fixed order, so the whole repository is read over roughly five months. It results in green or red. Damage raises the alert “Storage damage found” and turns the endpoint Not restorable.
Retention: once a day, the server removes snapshots beyond the kept numbers (30 daily, 12 weekly and 12 monthly by default). See Profiles, schedule and hooks.

Both results are listed under Reports of the machine.