How Restow checks that a backup can be restored

Restow treats a backup nobody has read back as unproven, not as “probably fine.” This page explains exactly what the automated check does, on what schedule, and, just as importantly, what it still does not prove.

What runs, and when

Two separate, automated jobs check that a backup can be restored.

Recovery-readiness check (verify)

For each protected object (a mailbox, a OneDrive, an IMAP account), the check:

loads the latest committed snapshot and its manifest,
draws a random sample (by default 20 mails and 20 files, with calendar events and contacts sampled at a quarter of that, 5 each), or, for a full check, every eligible object in the snapshot instead of a sample,
reads each sampled item back through the same code path a real restore uses: the chunk index, pack fetch with a fallback to a copy, AES-256-GCM decryption,
compares what came back with the manifest, and
rates the object Ready, Attention, Not restorable, Not verified or No backup yet (see below).

It runs:

weekly, on the recommended default schedule (Sunday, 03:00, the installation’s default time zone: Europe/Berlin unless configured otherwise, stored on the schedule itself),
after every backup of an object, automatically, once a tenant has an active verification schedule, so a newly taken backup gets checked, not only the oldest one, unless a check is already queued or running for that object,
on demand, from Recovery readiness (“Check now” for one object, “Check all now” for the tenant).

Without an active verification schedule, recovery readiness stays permanently unproven even while backups themselves keep succeeding. Restow says so plainly rather than defaulting to a silently green state.

Storage integrity check (scrub)

Separately, a scrub job checks the raw storage the chunks live in, independent of any one backup:

a sampled check weekly (SHA-256 over a random selection of packs, on every storage target that holds a copy),
a full check monthly (every pack),
automatic repair of a damaged copy from an intact copy on another target, where one exists,
a pack that comes back corrupt on every target is marked damaged: later backups stop deduplicating against it and rewrite its content intact instead, but only the content the source (the mailbox, OneDrive or IMAP account) still holds; content no longer present at the source cannot be recovered this way.

A scrub finding of unrepaired corruption is not just noted for the next weekly check. It is turned directly into a Not restorable rating for every protected object whose snapshots actually use the damaged pack.

What “read back” actually checks

The read-back path is not a shortcut copy of the restore code. It is the restore code: the same chunk-index lookup, the same pack fetch with a fallback to a copy, the same AES-256-GCM decryption a real restore performs. Each chunk is re-addressed twice: the id sealed into its own header and the id recomputed from its decrypted content must both match what the manifest names, which catches a chunk that decrypts cleanly but is actually the wrong one, not only a chunk that fails to decrypt at all. Finally, where the manifest records a whole-object SHA-256, the object’s size and a freshly computed SHA-256 are compared against it; older entries from before this field existed return not_recorded and get the chunk-level checks above only, without this extra whole-object comparison. An item comes back as verified, mismatch (the bytes differ from the manifest), missing (its chunks are not in the chunk index), or unreadable (a pack could not be fetched, parsed or decrypted).

Every check is recorded against the exact snapshot it read, not against the object in general. That has one direct consequence worth stating plainly: a backup taken after a passing check is not itself verified until a check reads back that specific snapshot, whatever an earlier backup scored.

Reading the result

Rating	Meaning
Ready	Every sampled item of the latest backup came back byte-exact, and that backup is current.
Attention	Restorable, but something needs a look: the latest backup is 48 hours or more old, the snapshot had nothing eligible to check, or a test restore ran but the target did not confirm it (once test restores are connected).
Not restorable	A restore of this object would fail or be incomplete today: the manifest cannot be read, sampled items came back missing, unreadable or mismatched, a scrub found the underlying storage corrupt and unrepaired, a test restore failed outright (once test restores are connected), or the latest backup is so old (7 days or more) that recent data could not be recovered.
Not verified	A backup exists, but no check has read back this backup yet.
No backup yet	Nothing exists to check.

Each rating carries its specific reasons (which items, how many, how old the backup is): open the object’s report to see exactly what was checked and what, if anything, went wrong, rather than just the colour.

Servers and clients

Servers and clients backed up by the Restow agent (Endpoint backup) feed the same overview and the tenant summary, in a section Servers and clients, with the same ratings. The rule is the same as for mailboxes: a rating belongs to the backup it checked.

No backup yet: no good backup exists yet.
Not verified: a backup exists, but no restore test has read back exactly that backup. A passing test of an older backup does not count.
Ready: the restore test of the newest backup passed.
Attention: the test passed, but the backup was partial (some files could not be read).
Not restorable: a restore test of the newest backup failed, or a newer repository check found damage.

After every successful backup the agent picks up to 20 random regular files (none larger than 256 MiB) and reports their path, size and SHA-256 hash. The hash must be that of the content inside the snapshot, not of the live file, so a file that changes between the backup and the hash cannot turn the test red by mistake. The server then reads those files back from the repository with restic, hashes them and compares. A rating is green only when every hash matches and at least one file was checked. The agent also restores the same files into a temporary folder on the machine and compares them again; if either side fails, the machine is red. A rating older than 8 days counts as overdue. A weekly repository check reads one twentieth of the stored data and turns the machine red when it finds damage.

What this does not prove yet

The recovery-readiness check confirms that a random sample of the latest restore point reads back intact and decrypts correctly through the real restore path; the separate scrub job checks pack-level hashes across all stored data, sampled or in full, independent of any one backup. It does not yet prove that writing them into a live mailbox, OneDrive or IMAP account today actually succeeds: permissions, API changes and throttling on the Microsoft or IMAP side are not part of the automated check.

A test-restore probe, restoring the verified sample into a real test mailbox or drive (in the same non-destructive, rename-safe mode as any other restore into another account) and checking that the target confirms each item, exists as a design in Restow’s restore-verification code, but is not wired up in the current release: nothing yet supplies a live target for it to use, so a check today performs the internal read-back only. Connecting it to a real target is planned; until then, run a manual restore yourself from time to time as the closest thing to an end-to-end proof available today.

Where to read more

Backup and restore: what gets backed up and how restore itself works.
Backups and schedules: putting verification and storage checks on a schedule.
Backup vs. archive: restore points, retention, and how backup differs from the archive.