Verifying a release

Restow is software that holds your backups and your keys, so you should be able to check that what you run is what the project built, and to see what was tested before it was published. This page shows how, and says plainly where the checks stop.

What is in a release

Every release is a GitHub release in restow-backup/restow, and the images live in the GitHub container registry. A release contains:

Two images, each for linux/amd64 and linux/arm64: ghcr.io/restow-backup/restow (api, worker, scheduler, restic, the agent downloads and the standalone restore tool restow-restore) and ghcr.io/restow-backup/restow-web (the Caddy edge and the web interface). Both carry the same tag and are signed with cosign.
The smoke reports: smoke-report.md (amd64) and smoke-report-arm64.md, the result of the release smoke run described below. The release notes summarise them in their Verification section.
The agent binaries for endpoint backup: restow-agent_<version>_<os>-<arch> for Linux and macOS on amd64 and arm64.
The SBOMs: restow-<version>-linux-<arch>.spdx.json, one per architecture, in SPDX format.
The release stack: docker-compose.yml and env.example, the files from deploy/release/ that the smoke run tests against.
SHA256SUMS (the SHA-256 of every other file of the release) and SHA256SUMS.sigstore.json (the cosign signature of that list).

The release notes also list the digests of the two multi-architecture manifests, one line each (restow: sha256:... and restow-web: sha256:...). The optional updater checks the images it pulls against these two lines.

Docker tags

0.x.y (beta): :<version> and :beta. There is no :latest yet.
From 1.0.0, a stable release: :<version>, :MAJOR.MINOR, :MAJOR and :latest.
A pre-release (-rc.N): only :<version>.

For anything you care about, name the exact version in .env (RESTOW_IMAGE, RESTOW_WEB_IMAGE) instead of a moving tag, so an update happens when you decide. See Updates.

Verify an image signature

The images are signed with cosign in keyless mode. There is no long-lived signing key that could leak: the release workflow on GitHub Actions proves its identity to Sigstore through GitHub’s OIDC token and receives a short-lived certificate. Verifying checks that the signature was made by exactly that workflow, in the restow-backup/restow repository, on a version tag.

cosign verify ghcr.io/restow-backup/restow:0.1.0 \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Run the same command for ghcr.io/restow-backup/restow-web:0.1.0. A signature that is missing, made by another workflow, or made for other content makes cosign exit with an error. The release pipeline signs with cosign 3.1.3, so use a current cosign.

To verify exactly the manifest named in the release notes, put its digest instead of the tag: ghcr.io/restow-backup/restow@sha256:<digest> with the same two options. The signature is made over that manifest.

Verify the SBOM

Each architecture has a software bill of materials in SPDX format, generated from the image with syft. You get it twice:

as the file restow-<version>-linux-<arch>.spdx.json on the GitHub release, for reading, scanning and archiving, and
as a cosign attestation of type spdxjson attached to the image of that architecture, so it is tied to the image by a signature and not only by a file name.

The attestation is attached to the image of each architecture (its own digest), not to the multi-architecture tag. docker buildx imagetools inspect ghcr.io/restow-backup/restow:0.1.0 lists the digest of each platform. The flags below come from the release workflow (--type spdxjson is the type it attests with, the identity options are the ones used for the signature). The command itself has not been run against a published release, so check the cosign documentation for verify-attestation if it behaves differently in your version:

cosign verify-attestation --type spdxjson \
  ghcr.io/restow-backup/restow@sha256:<digest of the linux/amd64 or linux/arm64 image> \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Verify the checksums

SHA256SUMS lists the SHA-256 of every file of the release, including the agent binaries, the SBOMs, the smoke reports, docker-compose.yml and env.example. It is signed keyless with cosign, and SHA256SUMS.sigstore.json is that signature as a Sigstore bundle. Download both files next to the files you want to check, then verify the signature of the list and the files against the list. The --bundle option is the counterpart of the --bundle the release workflow signs with, and the identity options are the same as above:

cosign verify-blob SHA256SUMS \
  --bundle SHA256SUMS.sigstore.json \
  --certificate-identity-regexp '^https://github.com/restow-backup/restow/\.github/workflows/release\.yml@refs/tags/v' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

sha256sum -c --ignore-missing SHA256SUMS

--ignore-missing skips the files you did not download. The agent binaries in the release are the same ones that sit inside the smoke-tested image, so this is also how you can check an agent binary independently of your own Restow instance (see the last section).

What the release smoke run checks

Before anything is published, a script starts the exact images that are about to be released from the release stack (deploy/release/docker-compose.yml) on a fresh runner, once on amd64 and once on arm64, and runs the checks below. The release is gated on it: if a check fails, nothing is published. The same script runs nightly on main without publishing anything, and you can run it yourself (pnpm smoke in the repository). The report of each run is attached to the release. The checks:

Install. The image carries its metadata (version, revision, platform), restic, restow-restore and the agent downloads with their checksums. The release compose file starts on an empty database with every migration applied. When an earlier release exists, a stack of the previous images with a tenant and a backup is switched to the new images. At the end the api is restarted on the populated database and changes nothing.
Health. /healthz and /readyz answer, the edge answers over HTTPS, worker and scheduler have registered, and no container keeps restarting.
Passkey sign-in. First-run setup, then in a real browser: emergency sign-in, registering a passkey on a virtual authenticator, signing out and in again with the passkey, creating a tenant in the wizard, and every screen in English and German without a missing translation.
Microsoft 365 against a developer tenant. It runs only when credentials for a Microsoft 365 developer tenant are configured as repository secrets, never against a customer tenant, and otherwise says “skipped” in plain words.
IMAP backup and restore. A mailbox with 24 messages in two folders on a real IMAP server, a backup, three more messages and an incremental backup, a restore next to the originals with every restored message compared by SHA-256, and the restore check green.
Journal receipt and chain check. A synthetic Exchange journal report over SMTP, the archive item present, the hash chain verified and continued by a second report, an unknown journal address refused, and the archive export (an EML ZIP with MANIFEST.csv and SHA256SUMS) holding the originals byte for byte.
Standalone restore. restow-restore from the image, with the server stopped and no network in its container, restores the storage of check 5 and every file matches. A copy with one damaged byte is refused.
Storage targets. S3 (Garage), a local path and a bind-mounted directory standing in for an NFS or SMB share each get a write (a backup), a read (the restore check) and a scrub.
Scans. Trivy on the image and pnpm audit for high and critical advisories. Critical findings that have a fix fail the run. Critical findings without a fix are listed in the report, not hidden.
Endpoint backup and restore. The real install script installs the agent in a Linux container from the running stack, enrolls it with a one-time token, backs up a folder, and a restore task puts it into a new directory, where every file matches by SHA-256.
Mail import and export. EML files and an MBOX in the server-side import folder and an MBOX uploaded in pieces become one imported mailbox. It is exported as an EML ZIP and as MBOX files. The EML files come back byte for byte, every message by Message-ID, and both checksum lists agree with the files. See Mail file import and export.

What the smoke run does not do

The report states each of these, and none is ever shown as a pass:

Microsoft 365 against a real tenant. Check 4 has not been run against a real tenant yet. The code path is implemented from the API and the documented Graph behaviour, and details are expected to need fixing on its first real run. This is the main reason to run the beta alongside your existing backups.
The upgrade path. There is no earlier public release for 0.1.0, so that step of check 1 reports “not run”. It runs from the next release on.
Journal items are not compared with their source. The journal token of a tenant is set in the database by the check because no API hands it out yet, and no API serves the raw bytes of an archived item, so the export is the witness that the originals come back unchanged.
Critical findings without a fix. The scan cannot act on criticals the distribution has not fixed (for example in the Debian base image). They are listed, not counted as failures.
Windows. There is no Windows agent in 0.1.0, and the endpoint check runs on Linux only.

What CI checks on every pull request

Every pull request and every push runs, and a release re-runs all of it on the tagged commit:

lint, and a guard that the core never imports from the ee/ modules,
type checking, unit tests and the tests against a real PostgreSQL, with a guard that none of the PostgreSQL suites was skipped,
i18n completeness: every key of the English translation exists in German and the other way round,
the full build, and a Docker build of both images with a look inside (restic, restow-restore, the agent checksums, the image labels),
pnpm audit for high and critical advisories,
dependency licenses against an allowlist, and third-party notices that are up to date,
a secret scan over the history (gitleaks),
the Go agent: formatting, go vet, race-detector tests, ShellCheck, integration tests against the real restic and an append-only REST server, the install script, and the cross-builds for Linux and macOS,
ShellCheck for every shell script, and a linter for the workflows.

The pipeline uses only the GITHUB_TOKEN of the run: no personal access token and no long-lived signing key. Every third-party action is pinned to a commit, and every container image a step runs is pinned by digest.

Contributor checks

Pull requests from outside also need two things, checked automatically: a signed Contributor License Agreement (a comment on the pull request, recorded in the repository) and a Developer Certificate of Origin sign-off on every commit (git commit -s). See License and editions for how the editions and the source are licensed.

What is not signed

Being honest about the gaps matters more than a long list of green checks:

The agent binaries are not code-signed. They carry no code signature, and updates of the agent are not signed either. The install scripts and the agent’s self-update verify SHA-256 checksums that come from your own Restow instance, the same instance that serves the binaries. That protects against corrupted or truncated downloads, not against a compromised instance. Signed agent releases are a later step. To check a binary independently, compare it with the signed SHA256SUMS of the release, as shown above.
Restow does not verify its own signature. Checking the cosign signature is something you do by hand. The optional updater checks the pulled images against the digests in the release notes, it does not check the signature, and the update check only reads the release list.
A signature is not a review. It proves which workflow built an image from which tag. It does not replace reading the release notes and testing a restore of your own data.