Updates

Restow ships as Docker images, in two builds per release (the full images and the Community images, see Get started), with database migrations applied automatically. There are two ways to update:

• By hand, with docker compose. This is the default and always works. The next sections start there.
• From the web interface, under Installation → Updates. Restow tells you when a newer version exists and, if you opt in to the updater service, installs it for you, now or at a date and time you choose (since 0.4.2): a countdown for everyone who is signed in, a database backup first, and an automatic rollback when the new version does not start, as long as its database migrations have not run. After a migration, which cannot be undone, the updater stops and tells you how to restore the dump (see When something goes wrong).

Nothing on this page happens unless you switch it on. A fresh installation does not look for updates, does not contact any server for that purpose, and does not run the updater.

Before every update

1. Read the release notes of every version between yours and the new one, especially Breaking changes and Update effort. Each release states its update effort (see the three kinds below), with the expected downtime and how to roll back.

2. Back up the database. It holds metadata, the job queue and the audit log. Backed-up content lives in the chunk store and is not touched by an update.

   docker compose exec -T postgres pg_dump -U restow -Fc restow > restow-$(date +%F).dump

   The updater does this itself (see below).

3. Check that RESTOW_MASTER_KEY is backed up offline. An update never changes it, but without it no backup can be read. See Backing up Restow itself.

4. Check the free space (since 0.4.0): the update needs room for a database dump, the new images and the migrations. Installation → Database shows the estimate; the updater refuses an update that does not fit (Disk space).

5. Think about runs in progress. A backup that is running when you update is stopped cleanly or waited for, as you choose in the install dialog (Runs in progress during an update). An update by hand does neither. Since 0.4.2 you can also stop a running backup of a guest, a machine or a file share yourself before you start, see Proxmox VE, servers and clients and file shares.

6. Check that your license key covers the release (full images with a Business or Service Provider key, since 0.4.5). Every key issued so far is a beta license and covers every 0.x and 1.x release. A key with an update period covers the releases made up to its end: the Updates tab marks the others, and a release that you start by hand and that the key does not cover does not start (Releases your key does not cover).

The three kinds of update

1. Configuration only

Only values in .env change (a new optional variable, a changed setting).

docker compose up -d

up -d recreates the containers whose configuration changed. docker compose restart is not an update: it restarts the running containers with their old image and their old environment, and never picks up a new image or a changed .env.

2. New image, with automatic migrations

The usual case. Restow applies database migrations itself: the api container runs them as the database owner (DATABASE_MIGRATION_URL) before it serves any request, and refuses to start if one fails. The release notes say whether there are migrations and roughly how long they take.

Release stack (the published images, deploy/release/docker-compose.yml). Name the version in .env and pull:

.env

RESTOW_IMAGE=ghcr.io/restow-backup/restow:X.Y.Z
RESTOW_WEB_IMAGE=ghcr.io/restow-backup/restow-web:X.Y.Z

An installation on the Community build updates the same way with its own two images, ghcr.io/restow-backup/restow-community and ghcr.io/restow-backup/restow-web-community. RESTOW_UPDATER_IMAGE is yours alone; an update never changes it (see The opt-in updater).

docker compose pull
docker compose up -d

Then watch the migrations and the start:

docker compose logs -f api

The log shows restow: applying database migrations, then restow: starting role 'api'.

3. Manual steps

A release that needs anything beyond the commands above (a new required variable, a changed Compose file, a one-off command) lists every step, in order, under Update effort and Breaking changes. Do them in that order, before or after docker compose up -d exactly as written. The updater cannot know about such steps: it applies the image and restarts the services. Read the notes first, and update by hand when they list anything else.

After the update

curl -fsS http://127.0.0.1:3000/healthz
curl -fsS http://127.0.0.1:3000/readyz
docker compose ps

Then check the version in the sidebar footer of the web interface, and run Verify now for one tenant: a restore check that passes proves the new version can still read what the old one wrote.

If you back up Proxmox VE nodes (a preview), servers or clients, their helpers follow the server update by themselves since 0.4.2: new runs stay paused while the nodes and agents install the new release, for 15 minutes at the longest. See Nodes and agents follow the update. When you updated by hand, or the step did not run, Update nodes and agents now starts it from the Updates tab (since 0.4.3). For a node that was not reached, run restow-pve update as root on it, see Update and remove nodes; for an agent, sudo restow-agent update.

Upgrading to 0.4.7

0.4.7 is a new image with four database migrations (0066 to 0069). They add columns, two check constraints and a trigger and rewrite no table; the only data change is that 0069 updates the rows that are already orphaned. It adds no required environment variable, and the docker-compose.yml of 0.4.0 stays: there is nothing to replace. The integration API contract moves to 1.7.0, additively, with one change of meaning that you must read first (the first item). What to know:

• Breaking: status: "orphaned" now also means “still at the source, but no longer backed up”. Before, orphaned meant removed from the source. Since 0.4.7 it also covers an object whose account is disabled, that has no Exchange Online license, or that Microsoft locked, and the new statusReason (account_disabled, no_license or locked; null for an object removed from the source) tells them apart. This applies in GET /api/v1/objects, GET /api/v1/users, the object.status of GET /api/v1/jobs and objects.orphaned of GET /api/v1/status. An integration that treats orphaned as a deleted account must check statusReason before it acts, for example before it offboards a user in a PSA. See Contract 1.7.0.

• The migrations, in this order. They run when the api starts:

  Migration	What it does
  0066_pve_guest_exclusion	Two nullable columns on pve_guests (excluded_at, excluded_by).
  0067_pricing	The column settings.pricing (jsonb, default {}: no price set).
  0068_app_secret_alerts	The column settings.app_secret_alerts (jsonb, default {}): the last reminder before an app credential expires.
  0069_object_status_reason	Five nullable columns on protected_objects (status_reason, inactive_since, inactive_notified_at, status_confirmed_at, status_confirmed_by), two check constraints and a trigger that keeps them in step with the object’s status. It updates only the rows that are already orphaned.

• Objects that are no longer backed up wait amber for a confirmation. A mailbox of a disabled or unlicensed account, a OneDrive or SharePoint site that Microsoft locked, and an object removed from the source read Not backed up any more, please confirm until an administrator answers Yes, intended; then they read grey. That is no failure, but readiness stays at least yellow until then. Objects that were already orphaned before the update count as confirmed (migration) and raise no notice, so no long amber list appears. Disabled and unlicensed mailboxes and locked OneDrives and sites that failed every night are recognised by the first directory sync or backup after the update (a locked site by its first backup). Expect one notice for each of them (object.inactive; an unlicensed user often has two objects, mailbox and OneDrive), or one warning for a tenant with many (object.inactive_many), and confirm them under protected objects once they are intended. See Objects that are no longer backed up.

• Backups that Microsoft Graph or the storage held back no longer count as failed. A backup that ends, after all its retries, with graph.throttled, graph.service_unavailable or storage.rate_limited reads Held back by the service, retried in amber. It leaves the failed counts, does not make a job failing and raises no backup.failed alert and no job.failed webhook; the overdue rules speak up when no backup gets through. The counting reads the cause stored with each run, so earlier runs with these causes read the same way after the update and leave the failed counts of past periods in the statistics and the dashboard. The same holds for a backup that ends neutral for an object that is no longer backed up (graph.mailbox_inactive, graph.object_locked, config.object_inactive). GET /api/v1/jobs keeps the status failed for both: read failure.code (and the object’s statusReason) and treat them as the object’s state, not as a failure. See Failure explanations.

• Webhooks and notification recipients get the new events only once they are ticked. The events object.inactive, object.inactive_many, source.app_secret_expiring and installation.app_secret_expiring reach a webhook or a recipient only when the notification matrix has them on; a bell column that was never changed shows them. A restore that completed without restoring anything while items failed now raises restore.failed and the job.failed webhook (data.job.status stays completed), and a job that the reconciler ends for good raises job.failed like any other. See Notifications and reports.

• A permission you grant to the app takes effect without a new client secret. Restow kept the token it fetched, with the permissions the app had then, for up to about an hour; a permission granted in between was refused until then or until a new secret was entered. Now a 401 or 403 from Graph makes Restow fetch a new token once (at most once a minute per tenant or source) and send the request again. Notification mail through a Microsoft 365 app is the exception, see the known issues below. See First steps.

• One queued backup can no longer block the backup queue of every tenant. A queued backup of an object whose backup was still running (the next scheduled run, Back up now) used to hold up the backups of all tenants until the running one ended. It now waits for its object and lets every other backup pass. The fix is a patch to pg-boss 10.4.2; building from source applies it through pnpm.patchedDependencies (patches/pg-boss@10.4.2.patch, applied by pnpm install), so keep the patches folder. The images carry the patch. See Build from source.

• Reminders before the client secret or certificate of an app registration expires. Restow warns 30 days before, again 7 days before and once it has expired, for the central app and for a source with an app of its own. It needs the date, because it does not read it from Microsoft Graph (that would need Application.Read.All). For the central app with its secret in .env, add the new optional ENTRA_CLIENT_SECRET_EXPIRES_AT (YYYY-MM-DD, as Entra shows it) and recreate the api. A certificate needs none, and an unreadable value counts as unknown. A source that was connected through its own app before 0.4.7 has no date, so no reminder: for a client secret, note the date on the source’s page under Expires; a certificate gets its date when it is entered again. The expiry badge turns amber 30 days before the end instead of 60, and red from 7 days. The release stack’s env.example does not list the variable (the repository’s .env.example does); env_file hands it to the api all the same (Environment variables). See Reminders before an app credential expires.

• Partial failures cost an area or an item, not the whole run. A mailbox backup keeps the mail of the run when calendars or contacts are refused (403), resumes after a 401 in the middle of a run, resyncs fully when Graph rejects a stored delta link, and stores a message Exchange cannot convert to MIME as JSON. A SharePoint library or list that Graph will not give is an item failure and the rest of the site gets its snapshot. A OneDrive whose drive Microsoft no longer finds fails as graph.onedrive_unavailable, permanent and not retried. An IMAP backup treats a login or a read that the server refuses for a limit as imap.server_limit (transient, retried) instead of a wrong password or a full mailbox. In the IMAP archive capture, one message that the server refuses no longer holds back its folder. A restore over IMAP whose connection drops is retried on a fresh connection. Nothing for you to do; the run pages and the failure explanations say what happened.

• Restores that restored nothing say so. A completed restore with failed items and none restored reads Nothing restored (red); one that skipped every item reads Everything skipped (amber) and stays restore.completed. A restore into the original mailbox of an account that is disabled or unlicensed is refused with the reason. See Backup and restore.

• Recovery readiness rates an object removed from its source without the age of its last backup. A mailbox deleted in Microsoft 365, for example, no longer turns red because its last backup grows old: nothing backs it up any more, so it is rated by whether that backup restores. The table says Removed from source, backups kept. This applies from the update on, without a new check; an object whose newest backup was never checked gets one more check from the next planned check. See Objects that are no longer backed up.

• Proxmox VE guests can be excluded from backup on purpose (a preview). The update excludes none: guests in no job keep counting as No backup job until an administrator excludes them. restow-pve needs no update in this release; the server withdraws the backups of an excluded guest before a node gets them. See Excluding a guest from backup.

• The recommended job switches the mail archive on. Where the installation offers capture from the mailbox (archive.mailboxCapture) and the tenant has mailboxes or accounts an archive can capture, Create recommended job and Protect everything without a job make the new mail job back up and archive, or switch archiving on in an existing mail job over all objects (its schedules stay). Existing mail jobs change only when an administrator applies the recommended job, never by the update. See Backup jobs.

• Prices are empty after the update. The new Installation → Prices (monthly price per mailbox and per GB or TB, set by the owner) shows the card Price per month on a tenant’s overview only once the owner has entered a price. Restow bills nothing and limits nothing. See Multi-tenancy.

• Smaller changes. A background job honours a Retry-After of up to 5 minutes from Microsoft Graph (the interface keeps one minute); a mailbox member with a schedule of its own is judged by that schedule for the overdue alert, and a backup that is still running (up to 21 days) is not overdue; the time limit of a job counts from when the queue handed it out; the Proxmox VE inventory shows the size of the first backup and the average follow-up backup as text in the row (Inventory); and the licence page no longer lists sign-in with Microsoft among what Business unlocks (it is still experimental and off by default).

• Security and dependencies. ip-address 10.7.5 and seroval 1.7.5 resolve advisories in transitive dependencies, and zod 4.6.7 in the copy that better-auth uses. An advisory about memory use when validating a large array (GHSA-mrhx-c7xc-m3w5) also names zod 3, which Restow’s own request validation uses and for which no fixed 3.x exists; the audit of CI ignores this one advisory until Restow moves to zod 4, and the request body limit of the api (1 MB on most routes) bounds it. Several other dependencies are updated (better-auth, mailparser, nodemailer, smtp-server, msgreader, Vite, React and others).

• Rolling back to 0.4.6 after the migrations was not tested. Take a database backup before the update and restore it to go back (Rolling back by hand).

• Known issues in 0.4.7:

  • The detection of disabled, unlicensed and locked objects was tested against a simulated Graph API, not against a real Microsoft 365 tenant, and the guest exclusion was not run against a real Proxmox VE node. An answer of Microsoft that Restow does not recognise as “no mailbox” or as a lock stays a counted failure, as before.
  • GET /api/v1/jobs has no field that sets a held-back or neutral backup apart from a failed one; read failure.code.
  • Notification mail sent through a Microsoft 365 app registration does not fetch a new token after a refusal: a Mail.Send permission granted later takes effect once the cached token expires.
  • The expiry date of a client secret is not read from Microsoft: without a date entered (under Installation, in ENTRA_CLIENT_SECRET_EXPIRES_AT or on the source) there is no reminder. A source with an older own secret without an own app (shared client ID) has no date in Restow; switch it to an own app or the central app.
  • On the Recovery readiness reports list, a filter on the rating matches the stored rating: a report stored before the update, or while the object was still at its source, counts the backup’s age, so such a report of an object removed from its source can be listed under red while it reads green. Checks from 0.4.7 on store the rating without the age. The readiness history of earlier days does not show the yellow of unconfirmed objects, because only the current confirmation is stored, not when it was given.
  • A message the IMAP archive capture gives up on (after three passes), messages the backfill could not read and a well-known mail folder that Graph refused to resolve are named in the worker log only, not in the interface.
  • Excluding a selection of Proxmox VE guests in the inventory sends one request per guest; when one fails, the guests before it stay excluded and the list refreshes.
  • Group calendars and conversations were tested against a simulated Graph API, never against a real Microsoft 365 tenant (as in 0.4.5), and IMAP sources with OAuth2 still cannot be created in the API or the interface.
  • The known issues of 0.4.4 still apply.

Upgrading to 0.4.6

0.4.6 is a new image with no database migration. It adds no required environment variable and no new setting, and the docker-compose.yml of 0.4.0 stays: there is nothing to replace. Nothing is removed from the API: the integration API contract stays at 1.6.0 (see below). What to know:

• Update restow-pve on every Proxmox VE node (a preview) before you rely on container restores or boot tests. Current Proxmox VE hands the storage ID to the backup provider’s container restore before the info hash, and the plugin of earlier releases failed on that with Can't use string ("restow") as a HASH ref. Every restore and every boot test of a container failed (Did not boot, Restore as a new guest of a container); VMs were not affected. The plugin of restow-pve now accepts both forms. The server offers restow-pve after its own update, nodes update themselves when idle, and the update reloads the Proxmox VE daemons (Nodes and agents follow the update); on a node that was not reached, run restow-pve update as root. Until a node is updated, container restores on it keep failing as before. See Restore.
• Container restore points that read “Did not boot”, or a failed restore check, from before the node was updated need one new run: choose Boot test now or Check now once the node runs the new restow-pve (Boot tests, Restore checks).
• A new backup chain of a container is no longer flagged as a ransomware attack. The first backup of a container’s new chain (its repository was missing, or reset) was judged against the old chain and flagged as suspicious, since every file is new in an empty repository. The baseline now begins with the newest chain, and that backup is called First backup, not Incremental. A restore point that was flagged before the update stays flagged: mark it as a false alarm (Ransomware detection).
• Removed machines keep their backups browsable. Until their backups are deleted, the page of a machine that was removed from Restow offers their restore, and the restore points, browsing, the ZIP download and the repository password keep working for them. The list of removed machines on Inventory links Restore files. Restoring onto the machine and every other action stay refused. See Uninstall and re-enroll.
• The restore check says it is queued. After Check now on a VM or container, the row says Check queued under the last check, and the guest’s page above the earlier result, until the new result is in; both refresh every few seconds meanwhile.
• Error texts open in a card. An error text behind a badge or a cut-off line (the boot test, the restore check of a VM or container on its page and in the recovery readiness table) opens on hover or keyboard focus in a card with the whole text, which you can select and copy with a Copy button, instead of a browser tooltip. A boot test that did not boot shows the node’s message under its badge.
• No migration to undo. Nothing in the database changed, so going back is the first case under Rolling back by hand; the release notes do not list a rollback test.
• Known issues in 0.4.6:
  • The container restore fix was tested with the Perl tests of the Proxmox VE plugin (the container restore with and without the storage ID), not against a real Proxmox VE node before the tag.
  • The known issues up to 0.4.5 still apply, except the one about removed machines (see below): since 0.4.6 a removed machine’s backups can be browsed and downloaded.

Upgrading to 0.4.5

0.4.5 is a new image with three database migrations (0063 to 0065). They add columns, an index and an enum value and rewrite no table, so they run in seconds whatever the size of the installation. It adds no required environment variable, and the docker-compose.yml of 0.4.0 stays: there is nothing to replace. The integration API contract moves to 1.6.0, additively (see below). What to know:

• The migrations, in this order. They run when the api starts:

  Migration	What it does
  0063_license_update_terms	Three nullable columns on the license table: the key’s series, its updates_until and the id of its signing key.
  0064_endpoint_removal	The removal state of a machine on endpoints (three nullable columns and a partial index over the removals that are due).
  0065_group_mailboxes	The protected object kind group (an enum value only).

• Installed license keys keep working unchanged. Every key issued before 0.4.5 is a beta license and covers every 0.x and 1.x release. From now on, though, a release that you start by hand and that the installed Business or Service Provider key does not cover stops before the database migrations and says so in its log; RESTOW_RUN_AS_COMMUNITY=1 starts it as Community instead. Nothing needs to be restored to go back. See License and editions and Releases your key does not cover.

• Keys with the new optional members (series, updates_until, kid) verify only on 0.4.5 and later. A key with them that is installed on an installation from before 0.4.5 is refused as invalid_payload: update to 0.4.5 first, with the key you have.

• Microsoft 365 groups are opt-in. After the update no group is backed up until the protection rules switch Protect all Microsoft 365 groups on, or single groups are included in the list of protected objects. The first directory sync after the update reads the groups of every Microsoft 365 source once in full. No new Microsoft permission and no new admin consent is needed: Group.Read.All and Calendars.ReadWrite were always required. See Connecting sources.

• Machines can be removed from Restow. A revoked machine gets Remove from Restow in the section Stop backing up this machine, with or without its backups. Nothing is deleted by the update, and uninstalling or revoking still deletes nothing. A machine without contact for more than a configurable number of days (30 by default) now suggests revoking it, and an uninstall that waits for a machine that is off offers Revoke now. See Uninstall and re-enroll.

• Both job lists look different. Run now is called Start now, and every row of Servers & clients → Jobs and Mail & SaaS → Jobs has Start now, which becomes Stop while the job’s backups run or wait. A click on a row opens a detail panel, like the one of VMs & containers. Inventory and the job lists keep a fixed layout, with one compact state badge per row. Nothing is lost: the same actions are there. See Backup jobs.

• A repository on the server’s system disk is warned about. The Repositories page, the line “Backups land in” of the job editors and the dashboard’s storage tile warn when a directory repository lies on the disk under the operating system, Docker and the database, and the api now also watches that disk for low disk space. Nothing is blocked (Where backups land).

• The setup wizard is stricter about the Entra tenant. For Graph mail it accepts only a tenant ID (a GUID) or a verified domain, as Settings already did; anything else is refused with 422.

• Integration API contract 1.6.0 (additive): ObjectKind includes group, the status reports lastSuccess.groups and version.releasedAt. A client that handles object kinds exhaustively should accept the new value (Integration API).

• New optional settings. None for a normal installation except RESTOW_RUN_AS_COMMUNITY (empty by default; see Environment variables). RESTOW_RELEASED_AT is written into the image like RESTOW_VERSION; do not set it. The public demo has settings of its own.

• Rolling back to 0.4.4 was not tested. Take a database backup before the update and restore it to go back (Rolling back by hand).

• Known issues in 0.4.5:

  • Group calendars and conversations were tested against a simulated Graph API, never against a real Microsoft 365 tenant; which calendar reads Graph allows an app on a group can differ by tenant (the backup falls back to calendarView and reports what it could not read).
  • Machines removed from Restow could not be restored from in the interface while their backups were kept until expiry. Fixed from 0.4.6 on.
  • IMAP sources with OAuth2 still cannot be created in the API or the interface.
  • The known issues up to 0.4.4 still apply.

Upgrading to 0.4.4

0.4.4 is a new image with four database migrations (0059 to 0062). It adds no required environment variable, and the docker-compose.yml of 0.4.0 stays: the release file has changed since 0.4.3 only in the example image tags in the messages of RESTOW_IMAGE and RESTOW_WEB_IMAGE, so there is nothing to replace (see the Release stack reference). Nothing is removed from the API: the integration API contract stays at 1.5.0 (see below). What to know:

• The migrations, in this order. They run when the api starts:

  Migration	What it does
  0059_archive_capture_enum	Adds the capture way smtp_forward (a copy that a mail provider forwards) to the archive.
  0060_archive_capture	The tables of the archive capture (a stream per archived mailbox and way, mail that Graph announced and that is not fetched yet, Graph subscriptions, the archive addresses of IMAP accounts, further captures of an archived mail), new columns on archive_items and the capture ways of a mail job (backup_jobs.archive_capture), and five indexes on archive_items.
  0061_explorer_parent_index	Two indexes on manifest_versions for the restore explorer, which now lists the folders of a large mailbox or OneDrive in milliseconds.
  0062_archive_capture_existing	A data migration: the capture ways of the mail jobs that already archive (see the next item).

• Plan the update of a large installation. The indexes are built inside the migration, where writes to the table wait: expect about a minute per ten million rows on SSD storage. 0060 builds five indexes on archive_items (one row per archived mail; three of them read every row, the two partial ones start empty). 0061 builds two on manifest_versions, usually the largest table of an installation (one row per mail, file or event, and per change of one); during that build, backups of Microsoft 365 wait to record their restore point. Count both beforehand and update outside the backup window when they hold tens of millions of rows. A quick estimate that does not read the tables (Row Level Security does not apply to it):

  docker compose exec -T postgres psql -U restow -d restow -c "SELECT relname, reltuples::bigint AS approx_rows FROM pg_class WHERE relname IN ('archive_items', 'manifest_versions');"

• Nothing new starts by itself. Mail jobs that archive keep doing exactly what they did before the update. Migration 0062 gives each of them Exchange journaling only for Microsoft 365 mailboxes and no capture way yet for IMAP and Google Workspace accounts (marked as waiting for a choice), so no capture from a mailbox and no takeover of existing mail begins. You switch Graph or IMAP archiving on, per job, when you want it:

  • With a Business or Service Provider license, the Archive page shows a notice that you can hide, New: capture by Microsoft Graph and IMAP available, with the jobs where Graph is off or no way is chosen yet, each with Open job. The capture status lists the same jobs under Waiting for a capture way.
  • In the job editor, the card of the IMAP accounts says that no capture way is chosen yet. Choosing one starts it with a one-time takeover of the mail that is already in the mailboxes, marked as captured afterwards. Edits that do not touch the ways (name, schedule) save as before.
  • Jobs created after the update, and a job that switches archiving on, start with the defaults: Graph for Microsoft 365, IMAP archiving for IMAP and Google Workspace, each with that takeover. See How mail reaches the archive.
  • The Community build keeps the capture service idle; the ways are locked and name the edition.

• Capture from the mailbox needs the capability archive.mailboxCapture, which follows the license’s edition: every Business and Service Provider license includes it without a new key.

• Graph capture is notified of new mail only when Microsoft can reach the installation. The public URL must be https:// on port 443 and reachable from Microsoft, and a reverse proxy of your own must pass /api/v1/hooks/graph/ through (at most 1 MiB per request). Otherwise every mailbox is checked every 15 minutes (ARCHIVE_GRAPH_POLL_MINUTES) and the Archive page says why. No new Microsoft permission and no new admin consent is needed. See Connecting sources and Network and ports.

• IMAP archiving keeps one connection per account open (IDLE) where the server offers it, so a provider with a tight connection limit may need the source set to Check at intervals only (archiveIdle: "off"). See Archiving IMAP and Google Workspace accounts.

• New optional settings. None is required; set them in .env, which the api and the worker read. The release stack’s env.example does not list them (the repository’s .env.example does): JOURNAL_RATE_PER_IP (now 600 connections per minute per sending address, was a fixed 60) and JOURNAL_RATE_PER_ADDRESS (300 deliveries per minute per archive address), and for the capture ARCHIVE_CAPTURE_CONCURRENCY (16), ARCHIVE_CAPTURE_TENANT_CONCURRENCY (4), ARCHIVE_BACKFILL_CONCURRENCY (4), ARCHIVE_GRAPH_CATCHUP_MINUTES (360), ARCHIVE_GRAPH_POLL_MINUTES (15), ARCHIVE_IMAP_IDLE_MAX (300), ARCHIVE_IMAP_POLL_MINUTES (5) and ARCHIVE_IMAP_LOGIN_POLL_MINUTES (10). See Environment variables.

• What behaves differently. Pausing a mail job stops only its scheduled backups; archiving goes on, and the pause question says so. Switching archiving or a capture way off in the editor asks first. The journal receiver delivers one SMTP transaction into one tenant’s archive (a recipient of a second tenant gets 452), and mail to the journal address that is not a journal report is archived as a forwarded copy instead of as an incomplete report. The archive search hides captures that a stronger way replaced unless you ask for them, and the export of the current search follows it. Opening the job editor of a job with IMAP accounts only no longer creates the tenant’s journal address.

• The restore explorer opens large mailboxes and OneDrives in milliseconds. The folder lists come from the new indexes, and a mail preview or attachment download reads exactly its bytes from storage. See Backup and restore. Restore points of 0.3.4 and earlier that the worker has not converted yet are listed as before.

• Integration API. Contract 1.5.0 is unchanged: its archive status and report count forwarded copies in their totals but do not list them as a channel of their own yet. The session API gains archiveCapture on mail jobs, captureKind on their members, mailSourceKinds in the job defaults and includeSuperseded on the archive search. See Integration API.

• New alerts. The event Archive sync failed (archive.failed) also reports a mailbox whose capture has failed for more than an hour, forwarded copies that stopped for 24 hours and a journal address without reports for 24 hours (Notifications and reports).

• Rolling back to 0.4.3 was not tested. Take a database backup before the update and restore it to go back (Rolling back by hand).

• Known issues in 0.4.4:

  • The capture ways were tested against simulations: Graph capture against a simulated Graph API, IMAP archiving against a simulated IMAP server and the Dovecot of the release smoke, forwarding against the receiver itself. None ran against a real Microsoft 365 or Google Workspace tenant, Postfix, mailcow or hoster.
  • Holds in a Microsoft 365 tenant are not detected, because Graph v1.0 cannot read them; an administrator confirms them per source. Of the Recoverable Items only the deletions folder is read, so mail purged before Restow fetched it can be missing, and Graph cannot export mail over about 150 MB (it is logged, not archived).
  • The journal receiver does not check the sender (SPF), and its limits count per api process.
  • IMAP sources with OAuth2 still cannot be created in the API or the interface, so a Google Workspace account is recognised by its host or folder list only.
  • The integration API’s archive status and report do not list forwarded copies as a channel of their own.
  • The known issues up to 0.4.3 still apply (see below and the roadmap).

Upgrading to 0.4.3

0.4.3 is a new image with no database migration. It adds no required environment variable, and the docker-compose.yml of 0.4.0 stays: there is nothing to replace. Nothing is removed from the API (contract 1.5.0 stays; a guest’s lastVerify.state can now be yellow, and a node carries offeredVersion and outdated, both additive). What to know:

• Proxmox VE nodes should run restow-pve 0.4.3. It waits for the server’s repository lock during a container backup and restarts the Proxmox VE daemons (pvedaemon, pveproxy, pvestatd, pvescheduler) after it updates itself. Nodes that run restow-pve from 0.4.2 on install it with the step after the server update, or with Update nodes and agents now (Nodes and agents follow the update).

• Update nodes and agents now is new on the Updates tab (since 0.4.3). It starts the step in which nodes and agents follow the server when no update led into it: the server was updated by hand, or the answer to an announced update was lost. Nodes and agents that are behind show as Update available on the card, and an administrator with every tenant starts the step from there.

• Nodes on restow-pve 0.4.1 need one update by hand. They ignore “update now” and the step after the server update. They update with their own check (two minutes after the service starts, then every six hours), or at once by hand, as root on the node, once no backup or restore runs on it:

  /opt/restow-pve/bin/restow-pve update
  systemctl restart restow-pve pvedaemon pvestatd pveproxy pvescheduler

  From 0.4.3 on restow-pve restarts the daemons itself after every update it installs. A node that an older release updated does it once by itself after the new release has started.

• Restore checks that a locked repository failed before the update stay red until the next check of the guest. Choose Check now on the guest, or on a restore point of its page, to read them again (Restore checks). From 0.4.3 on, a check that a lock keeps out is tried again and, if the lock stays, reads Could not check: repository locked in amber, never red.

• VMs & containers looks different. It has three tabs now (Guests, Backup jobs, Nodes), a search, filters and a detail panel for a guest. Nothing is lost: the same actions are there, and a bookmark of the page opens the Guests tab (Inventory, guests and restore points).

• The false “The updater is not running” is fixed. Announcing an update could report it although the updater accepted it a few seconds later; the update then ran without the pause of new runs and without the step for nodes and agents. See When something goes wrong.

• Rolling back to 0.4.2 works. There is no migration to undo, so the older image starts on the same database. It shows a restore point that could not be checked as unchecked.

• Known issues in 0.4.3:

  • The new parts were tested against a simulated Proxmox VE API and a simulated S3 store only, never against a real Proxmox VE cluster or S3 provider.
  • The redesigned VMs & containers page was tested with DOM tests, not visually in a browser with thousands of guests.
  • Healing a file share whose repository went missing after a storage change is still planned for a later release; a container’s is healed since 0.4.2.
  • The known issues up to 0.4.2 still apply (see below and the roadmap).

Upgrading to 0.4.2

0.4.2 is a new image with six database migrations (0053 to 0058). They only add a table, columns, indexes and enum values and widen a check, and they run in seconds whatever the size of the installation. It adds no required environment variable, and the docker-compose.yml of 0.4.0 stays: there is nothing to replace. Nothing is removed from the API (contract 1.5.0 is additive). What to know:

• The migrations, in this order:

  Migration	What it adds
  0053_pve_repository_chains	A container’s new backup chains (pve_guests.repository_chains) and pve_snapshots.unavailable_at.
  0054_pve_runs_042	The Restow server’s own VM (pve_guests.restow_server_detected, restow_server_flagged, self_backup_confirmed_at), pve_snapshots.verify_requested_at and the installation’s Proxmox VE settings (settings.pve_settings).
  0055_pack_read_failures	A new, empty table for packs the storage check could not read.
  0056_cancel_and_fleet_update	Cancel requests and the state cancelled for tasks and runs, the paused queue of a node, “update now” for nodes and machines, the phase fleet of the pause of new runs and settings.planned_update.
  0057_explorer_people	An index on the protected objects of a person in a tenant, for the list of people in the Restore explorer.
  0058_installation_time_zone	settings.time_zone.

• Proxmox VE nodes should run restow-pve 0.4.2. It reports the Restow server’s own VM with the inventory, keeps its heartbeat going while a backup runs (needed for Stop and for the progress) and no longer reports a backup that ends with “TASK WARNINGS” as failed. The fleet update after the server update installs it on every node that reports in; the Updates tab names the command for the others (Nodes and agents follow the update).

• Stopping a running backup needs restow-pve 0.4.2 on the node and agent 0.4.2 on the machine. An older node ends a cancelled backup at its next upload (the server refuses it), an older agent finishes the backup. Both update themselves with this release.

• “Keep existing backups where they are” is refused while containers or file shares have restic repositories (storage-switch-blocked-by-repositories), and so is promoting a copy to primary. Change the primary repository with Move existing backups, which now takes the restic repositories along. See Replacing the primary repository.

• Storage check reports from before 0.4.2 are read again. Packs that an earlier report listed as unreadable now count as temporarily unreadable, not damaged, and the next storage check reads them first; no full check is needed. An object rated Not restorable only because of such a pack is rated Attention until that check (Storage integrity).

• Integration API contract 1.5.0 (additive): storageIntegrity.state can be unreadable. A client that handles the values exhaustively should accept it (Integration API).

• No time zone of its own after the update. An existing installation uses RESTOW_TIME_ZONE, else TZ, else Europe/Berlin until an owner chooses one under Installation → Server & operation → Server (The installation’s time zone).

• Rolling back to 0.4.1 works. The migrations only add a table, columns, indexes and enum values and widen a check, so 0.4.1 starts with them in place. Runs and tasks in the state cancelled are unknown to 0.4.1.

• Known issues in 0.4.2:

  • A file share whose repository went missing after a storage change is not yet healed the way a container’s is (with a new backup chain). This is planned.
  • The fix for the maintenance and update banner covering the top of pages that size themselves to the window was not verified visually in a browser.
  • The new parts were tested against a simulated Proxmox VE API, S3 store and Microsoft Graph API only, never against a real Proxmox VE cluster, S3 provider or Microsoft 365 tenant.
  • The Restore explorer’s list of people groups the tenant’s protected objects in the database for every page it serves. That is fine up to about 10,000 people per tenant and slower beyond.
  • The known issues of 0.4.1 and 0.4.0 still apply (see below and the roadmap).

Upgrading to 0.4.1

0.4.1 is a new image with one database migration, 0052_pve_guest_repository_bytes (two columns on pve_guests), which runs in seconds. It adds no environment variable, and the docker-compose.yml of 0.4.0 stays: there is nothing to replace. What to know:

• Stored bytes change. Repositories → Usage, the Stored tile of the dashboard, the statistics and physicalBytes of GET /api/v1/status and GET /api/v1/storage now include the restic repositories of machines, file shares and Proxmox VE containers, not the chunk store alone (the fields and their types stay). The repository of a container counts under Stored from its next backup on, of a machine or file share as before (from its next backup or retention run); until then it counts as zero.
• Proxmox VE nodes pick up restow-pve 0.4.1 by themselves: it now asks for its update two minutes after it starts, then every six hours, and installs it once no backup or restore runs on the node.
• Rolling back to 0.4.0 works with the new migration in place: it only adds two columns.
• The known issues of 0.4.0 still apply (see below).

Upgrading to 0.4.0

0.4.0 is a new image, seventeen database migrations (0035 to 0051) and a new docker-compose.yml for file shares and the database move. Nothing stops working on update: existing compose files, webhooks, notification recipients, API clients and agents keep working. Do these in order:

1. Update as usual, with the opt-in updater or docker compose pull && docker compose up -d. The api applies the migrations as it starts. With the updater, the install dialog now asks what to do with runs in progress; the default stops them cleanly (Runs in progress during an update).
2. Run the 0.4.0 install.sh again in the installation directory (Running it again). It replaces docker-compose.yml with the file of 0.4.0 (checked against the signed SHA256SUMS), keeps the previous one as docker-compose.yml.bak-<timestamp> and leaves .env as it is. Without it everything else works, but file share runs cannot start (the mounter reports runner_network_missing) and Installation → Database cannot move the database. A compose file you keep by hand needs the changes of the release file: the database volume read from .env, the internal runners network on the api, and RESTOW_MOUNTER_URL, the runner limits and the restow-mounter-shared volume on the worker (see the Release stack reference).
3. Mounter default. New installations now start the mounter (Compose profile mounts) by default, because file share backup runs through it; scripted installations that do not want it pass --no-mounter (or --without-mounter). Running the installer again on an existing installation keeps that installation’s choice. For file shares on an existing installation, start the mounter if it is not running: Enable network shares in the web interface (with the opt-in updater) or docker compose --profile mounts up -d mounter. The mounter mounts the Docker socket, which is root access to the host: read Mounter and NFS storage. The Docker host needs the cifs module for SMB and the NFS client for NFS (File shares).
4. Proxmox VE nodes: restow-pve 0.4.0 updates itself (it checks every six hours and updates while idle; since 0.4.1 also two minutes after it starts), or run restow-pve update on the node. Older helpers keep working without progress, without the five-minute retry window and without the list of active runs.
5. Servers and clients: the endpoint agent 0.4.0 updates itself unless the tenant paused agent updates. Older agents keep working without data folders, machine facts, the backup window, catch-up rules and retry count of an agent policy, and they do not hold their own schedules during an update pause.
6. Plain-HTTP S3 endpoints. Since 0.4.0 an S3 endpoint saved or changed must use https://, except to localhost, 127.0.0.1 and [::1]. A repository or default repository that uses http:// to another host (and S3_ENDPOINT in .env) keeps working, so no backup stops, but its card, Installation → Default repository and every connection test say in red that it should move to HTTPS, and changing its location needs https://. A test setup with an object store on the compose network sets S3_ALLOW_INSECURE=true in .env (api and worker). See Immutable backups.
7. SharePoint Online. The backup’s app registration needs the application permission Sites.ReadWrite.All (Sites.Read.All is enough to back up, not to restore). Add it to the app registration, then the customer’s administrator opens the source’s consent link again (SharePoint Online, First steps). Until then mail and OneDrive keep working, and the directory sync warns that sites are neither found nor backed up (sites_unavailable).
8. Immutable backups with S3 Object Lock need a bucket that was created with Object Lock (and versioning); it cannot be added to an existing bucket. Nothing is locked until an administrator turns it on. The access key needs permissions of its own: see Immutable backups.
9. Ransomware detection is on after the update and judges backups of the last 72 hours. Its retention hold is on as well: a suspicious restore point pauses retention of its object until an administrator resolves it. Thresholds and both switches are under Installation → Ransomware detection (Ransomware detection).
10. Building from source needs Node.js 22.12 or newer (engines in package.json; commander 15 requires it). The Docker images run on node:22-slim as before, so nothing changes for Docker installations. See Build from source.

What behaves differently (the Breaking Changes of the release notes): notifications follow the matrix, where every channel keeps the events it gets and the webhook dialog has no event picker any more; the four flags of notification recipients are replaced by an event list (the tenant API still accepts and returns categories); new installations start the mounter; once the database was moved, install.sh refuses (exit code 8, nothing changed) a docker-compose.yml that does not read RESTOW_POSTGRES_DATA (Database).

The migrations, in this order: 0035_file_shares, 0036_backup_job_kinds_share_copy, 0037_backup_job_file_shares, 0038_notification_matrix (it translates the recipient flags and webhook event lists of every tenant and drops the four flag columns), 0039_pve_run_progress, 0040_recommended_jobs, 0041_pve_downloads, 0042_job_start_pause (the update pause), 0043_disk_alerts, 0044_pve_boot_tests, 0045_ransomware_detection, 0046_sharepoint_sites, 0047_immutable_backups, 0048_suspicion_retention_hold, 0049_perf_indexes, 0050_agent_policies (changes no machine’s configuration) and 0051_agent_waiting. They change the schema and backfill only the notification recipients and webhooks, a few rows per tenant, so they should take seconds on any installation, with one exception. 0049_perf_indexes adds six partial indexes (jobs, endpoint_runs, file_share_runs, pve_runs, pve_tasks); each build reads its table once while writes to it wait, which takes seconds on most installations and about a minute per ten million rows on large ones. 0035 creates the pg_trgm extension for the file share search when the database role may; an external PostgreSQL without it searches more slowly, with the same results. They ran on a 0.3.6 schema in the Postgres test suites; how long they take on a very large database was not measured.

New optional environment variables: RESTOW_POSTGRES_DATA_TYPE and RESTOW_POSTGRES_DATA (where the database lives; empty means the volume pgdata; the updater writes them when it moves the database), RESTOW_MOUNTER_MAX_RUNNERS (hard cap of file share runners, default 8) and RESTOW_MOUNTER_RUNNER_MAX_MEMORY_MIB (default 16384). No new required variable. See Environment variables.

Downtime and rollback. The containers restart and the migrations run, as for 0.3.6. The migrations are not reversible: go back to 0.3.6 only with the database dump taken before the update (Rolling back by hand). After a database move, keep a compose file that reads RESTOW_POSTGRES_DATA: the 0.3.6 file does not, and PostgreSQL then refuses to start on the old copy.

Upgrading to 0.3.6

0.3.6 is a new image with one database migration (0034, a small table for restart notices), which runs in seconds on any database size. It adds no required environment variable. The release notes of 0.3.6 list everything else. Four things to know before you start:

• The Restart now button needs the updater of 0.3.6. An updater from before 0.3.6 shows the command to run on the server instead, until it runs the 0.3.6 image. After an update from the web interface that passed the signature check, the updater moves itself to the new release (The updater updates itself); if you update by hand, or it does not follow, set RESTOW_UPDATER_IMAGE to the image in RESTOW_IMAGE and run docker compose --profile updater up -d updater. See Restart required.
• Installations whose docker-compose.yml has no mounter service: run the installer again. An installation keeps the compose file it was installed with, and an update changes only .env, so an installation from before the 0.3 line (made before 0.3.1; 0.3.0 was never published) has no mounter. Run the install script of 0.3.6 again on it: it replaces docker-compose.yml with the checked file of the release named in .env, keeps the old one as docker-compose.yml.bak-<time> and leaves .env as it is. See Running it again and no such service: mounter.
• A 0.3.5 index conversion that is still running needs nothing. The conversion goes on as before, and once it has finished, the api gives the space of the old backup index back by itself within about 15 minutes. The api restart that 0.3.5 asked for is no longer needed. How long the conversion takes depends on the database; the figures in the 0.3.5 section are from testing, and the first minutes are slower.
• Rolling back to 0.3.5 works only with the database dump taken before the update. The migration is not reversible, so the old version must not be started on the migrated database. See Rolling back by hand.

Upgrading to 0.3.5

0.3.5 is a new image with one database migration (0033), which only creates a table and drops two indexes and takes seconds on any database size. It adds no required environment variable; DATABASE_AUTH_URL is optional (see Database). The release notes of 0.3.5 list everything else. Four things to know before you start:

• Free disk space first. 0.3.5 stores the backup index (which items each backup contains) as item versions instead of a full copy per backup. After the start, the worker converts the existing index in the background, and it writes the new index before the old one is released. The updater also writes a database dump before it updates. On a nearly full disk, free space first.

• Compose files kept by hand need stop_grace_period: 60s on the worker service. It gives running jobs 60 seconds to save a checkpoint when the worker stops. Without the line, Docker stops the worker after 10 seconds, as before. Copy the line from the docker-compose.yml of the release, see the worker in the release stack reference. From 0.3.6 the install script can bring that file of an installation up to date for you (Running it again).

• The index conversion runs in the background. The worker converts the backups one at a time (in testing about 19,000 index rows per second, so roughly 15 minutes for 14 million rows; on a production database of 25 GB this is an estimate, not a measurement). Backups, browsing and restores keep working meanwhile. Follow it in the worker’s log:

  docker compose logs worker | grep "manifest index conversion"

  When it says manifest index conversion finished, the old index is empty, but the file system has not got its space back yet. From 0.3.6 the api does that by itself within about 15 minutes, and no restart is needed: it checks the size of the old table every 15 minutes (which reads no row), empties it as soon as nothing is left in it and then stops checking. Its log says the old manifest table was emptied and its disk space given back. It does this as the database owner (DATABASE_MIGRATION_URL, the connection of its migration step), with one short connection per attempt; no other role gets a new right. On 0.3.5 itself, restart the api once instead, which has the same effect:

  docker compose restart api

• Rolling back to 0.3.4 works only with the database dump taken before the update. The migration is not reversible, so the old version must not be started on the migrated database. See Rolling back by hand.

Step by step: an installation made by install.sh

The install script puts the stack into /opt/restow, owned by root, and never updates it (Running it again). From 0.3.6 it does bring docker-compose.yml up to the release named in .env when you run it again, but it still never changes the images or .env. Here is the update by hand for such an installation, from version 0.1.0 to a newer X.Y.Z (without the leading v). For an installation by hand, run the same commands in the directory of its docker-compose.yml. All commands for day-to-day operation are in the command reference.

1. Read the release notes of every version between yours and X.Y.Z on GitHub, especially Breaking changes and Update effort. If they list manual steps, follow them instead of the short path below.

2. Go to the stack and back up the database (written by root, readable only by root):

   cd /opt/restow
   sudo sh -c 'umask 077; docker compose exec -T postgres pg_dump -U restow -Fc restow > restow-$(date +%F).dump'

3. Optional: check both new images before you start them. The identity is the release workflow at exactly the tag of the new version, as in Verifying a release. For the full build:

   NEW=X.Y.Z
   for image in restow restow-web; do
     cosign verify "ghcr.io/restow-backup/$image:$NEW" \
       --certificate-identity "https://github.com/restow-backup/restow/.github/workflows/release.yml@refs/tags/v$NEW" \
       --certificate-oidc-issuer https://token.actions.githubusercontent.com
   done

   For the Community build, use restow-community restow-web-community in the loop.

4. Name the new version in .env. Keep a copy of the current file first (cp -p keeps its mode 0600), then change the two image lines, either with this one-liner, which touches only RESTOW_IMAGE and RESTOW_WEB_IMAGE (of either build) and leaves RESTOW_UPDATER_IMAGE alone:

   NEW=X.Y.Z
   sudo cp -p .env .env.before-update
   sudo sed -i -E 's#^(RESTOW_(WEB_)?IMAGE=ghcr\.io/restow-backup/restow(-web)?(-community)?):.*$#\1:'"$NEW"'#' .env
   sudo grep -E '^RESTOW_(WEB_)?IMAGE=' .env

   or by hand in an editor (for example sudo nano .env), so that the two lines read (full build):

   RESTOW_IMAGE=ghcr.io/restow-backup/restow:X.Y.Z
   RESTOW_WEB_IMAGE=ghcr.io/restow-backup/restow-web:X.Y.Z

5. Pull and start, then watch the migrations and the start (Ctrl+C ends the log, not the stack):

   sudo docker compose pull
   sudo docker compose up -d
   sudo docker compose logs -f api

   The log shows restow: applying database migrations, then restow: starting role 'api'. Check the result as in After the update.

Going back. If the new version does not come up and the release has no database migrations (its notes say so), put the old image lines back and start again:

cd /opt/restow
sudo cp -p .env.before-update .env
sudo docker compose up -d

If migrations ran, the old version must not be started on the migrated database: restore the dump from step 2 first, as described in Rolling back by hand.

Installation → Updates

The tab is visible to every provider administrator. It shows the running version, the release channel, the newest version with its tag and release date, a link to the release notes (and the notes themselves, rendered safely and collapsed), the status of the last check, and whether an update is available. From 0.3.6 an available update stands next to the running version, in the same size and highlighted, with the label available; before, it showed only in the status line. Since 0.4.2 the sidebar says it too (The sidebar notice). When a service needs a restart, the tab starts with a Restart required card.

Who may change it. Changing any setting, Check now, announcing, cancelling and dismissing an update need the Owner role of the provider team. The administrator created by the setup wizard is an owner. Everyone else sees the status only. See Team and roles. Changing the update source or the access token, and announcing an update, also need a recent sign-in (see Confirming it is you).

Turning the check on

The check is off by default. When an owner turns it on (Check for updates once a day), Restow reads the release list of the update source once a day and whenever you press Check now. After a failed check it tries again after an hour at the earliest.

What is read. Only the release list of the source. No request carries any data about your installation, and the result is cached. As with any web request, the source sees the address your server connects from. If a check fails, the tab says why and keeps showing the last good result:

Reason	Meaning
Rate limited	The source refused more requests for now. The tab shows when it lifts.
Token rejected	The access token is wrong, expired or revoked.
Not found	The repository does not exist, or it is private and no token is stored.
Access forbidden	The token may not read the releases of that repository.
Source error, no answer	The source answered with a server error, timed out or could not be reached.
Not a release list	The address answered, but not with releases.
No release published	The source has no release for this channel yet.
Redirect refused	The source redirected to another host. Restow does not follow it, enter the final address of the repository instead.

Source, channel and token

• Source. The default is the public releases of https://github.com/restow-backup/restow. You can point it at your own repository: a GitHub repository, or a Forgejo or Gitea repository (their releases API is compatible), for example https://git.example.com/acme/restow. Only https addresses are accepted, without credentials in the URL, because an access token travels with the request. The check connects to public addresses only: every resolved address is checked when the connection is made, and an address in a loopback, private or link-local network (for example a Forgejo on your LAN, or a host name that resolves there) is refused, unless the operator names its host in RESTOW_UPDATER_SOURCE_HOSTS (see Two modes). Saving such a source without that entry is refused with urn:restow:problem:invalid-update-source. The answer is read as a stream and given up past 8 MiB.
• Channel. Stable offers final releases only. Beta also offers pre-releases such as 0.3.0-rc.1. Versions are compared as Semantic Versions, so a pre-release precedes its release. A release such as 0.1.0 is an ordinary release that the stable channel offers as well.
• Access token. For a private repository, store a read-only access token. It is kept encrypted in the database, sent only as an Authorization header to the source’s own host, never shown again (the tab only says whether one is stored, and lets you replace or remove it), and never written to a log or the audit log. If you point the source at another host, the stored token is removed instead of following you there. A private repository is built from source (see below).

RESTOW_UPDATE_CHECK_URL and its precedence

RESTOW_UPDATE_CHECK_URL is an environment override of the tab’s settings. From strongest to weakest:

1. RESTOW_UPDATE_CHECK_URL, when set to an https address. The check is on, the source is that address, and the tab shows it read-only. No stored token is sent to it. Being the operator’s own setting, it may point at an internal address. Accepted: a GitHub https://api.github.com/repos/<owner>/<repo>/releases (or .../releases/latest), a Forgejo or Gitea .../api/v1/repos/<owner>/<repo>/releases, or any other https address that returns the same JSON. The channel can still be changed in the tab.
2. The settings of the tab (switch, source, channel, token).
3. The defaults: off, the public project releases, stable.

Confirming it is you

Changing the update source, storing, replacing or removing the access token, and announcing an update (also with a lead time of zero) need, besides the owner role, a sign-in from the last 10 minutes: with a passkey, with the password together with the authenticator code, or with an OIDC sign-in. An older session gets a Confirm it is you dialog. With a passkey you confirm in place and the change is made right away; otherwise you sign in again and repeat the change in the tab (entries you had not saved are not kept). Switching the check on or off, the channel, Check now, cancelling and dismissing need no fresh sign-in. The API answers such a request from an older session with 403 urn:restow:problem:recent-sign-in-required.

The “Update available” alert

Once per new version, Restow raises an Update available alert: an entry in the notification bell of the provider administrators, and the alert rules under Daily → Alerts that list the event Update available (e-mail, webhook) in every tenant. Only provider administrators can create a rule for this event, a tenant’s own administrators are not told about your updates. The same version never raises it twice. With a Business or Service Provider key that restricts updates, the alert names the newest release the key covers (Releases your key does not cover). See Alerts and reports.

Releases your key does not cover (since 0.4.5)

This applies to the full images with a Business or Service Provider license key. Without a key, with a beta license and in the Community build every release is offered and installs as before. A key that has an update period (see License and editions) covers the releases made up to the end of its last day; the release date is the one the images carry, which the release notes publish in a released: line.

• The update check marks a release the key does not cover. The Updates tab says so calmly: the version lies outside the update period, the installed version keeps working as before with all modules, and the newest version within the period, if there is one, installs as usual. It also says that you can renew the update period.
• The updater installs only covered releases. Announcing or planning another one is refused with 409 urn:restow:problem:update-not-covered.
• The bell and the Update available alert name only the newest covered release. A release outside the update period shows on the Updates tab only.
• A release started by hand is checked before it touches the database. docker compose pull and docker compose up -d bypass the update check, so the api checks the installed key before any migration. If the key does not cover the version, the api does not start and writes nothing to the database. Its log line names the version, the end of the update period and the last covered release date, and nothing needs to be restored to go back:
  • go back to the version you ran before (put the previous tags back into RESTOW_IMAGE and RESTOW_WEB_IMAGE, then docker compose up -d), or to any version the key covers, and install a renewed key there; or
  • start the new version as Community with RESTOW_RUN_AS_COMMUNITY=1 in .env: the Business and Service Provider modules stay off, no data is changed or deleted, and the license page says why. Installing a key that covers the version turns the modules on again at once.
• The opt-in updater sees such an api not start and puts the previous version back by itself (When something goes wrong); since no migration ran, no dump is needed.

The check never stops an installation without a key, the public demo, a development build, or any release on a beta license (every key issued so far). Releases from before 0.4.5 publish no released: line; for them the check uses their publication date.

Restart required (from 0.3.6)

Now and then a change takes effect only after one service has been restarted: something an update did in the background has finished, or a setting is read only at start. Restow then records which service needs a restart and why, and shows it in two places:

• A notice on every page, for provider administrators only: “Restart required: worker.”, with the link Show how to this tab. Nobody else sees it.

• A card at the top of Installation → Updates, Restart required, that lists each service (API, worker or scheduler) with its reason and the command to run on the server, for example:

  cd /opt/restow && docker compose restart worker

  When the opt-in updater runs, the command names the installation’s own directory. Without it, the card assumes /opt/restow and says to use the directory of your installation instead: the one with docker-compose.yml and .env.

Restart now. With the updater running, the owner of the provider team also gets a Restart now button for each service. After a confirmation, the updater runs docker compose restart <service> for exactly that service, and only for api, worker or scheduler: never the database, the edge, the updater or the mounter. The button is not available while an update is announced or running, or while another restart runs; the card then says so. It is not available either when no updater runs or when the updater is older than 0.3.6, and the card shows only the command. A restart briefly interrupts the service; running backup jobs save a checkpoint and continue afterwards, since 0.4.0 without using up an attempt (the worker’s 60 s stop grace period applies). The audit log records who asked (update.restart.requested).

Nothing restarts without a click. The notice stays until the service has started again, by the button, by the command or by the next update, and then disappears by itself.

Known issue in 0.3.6: nothing records a restart request yet. The notice and the button are in place for updates that need them, so for now there is nothing for the notice to show.

In the integration API

GET /api/v1/status carries a version object, so an RMM can flag installations that have fallen behind: running, commit (the source commit of the running build), latest, updateAvailable (null when not known), releaseUrl, updateCheck (disabled, pending, ok or failed), checkedAt, channel, latestTag, publishedAt, checkError (the reason code of a failed check), maintenance (an update that is announced or running: phase, targetVersion, startsAt) and, since 0.4.5 (contract 1.6.0), releasedAt (when the running build was released, the commit date of its release tag; null for a build without one). Fields are only ever added, never renamed or removed. See Integration API.

The opt-in updater

The Updates tab shows Install update only when the updater service is running. Without it, the tab shows the manual steps of this page instead.

The updater mounts the Docker socket: that is root on the host

Access to the Docker socket is root-equivalent access to the machine. Whoever can make the updater do something can start any container with any mount, read any file and change anything on the host. That is why the updater is a Compose profile that does nothing until you start it, and why it is built as small as it can be:

• It is a separate process and container (ROLE=updater, the application image, but its own pinned copy: RESTOW_UPDATER_IMAGE, see below). It has no database access and no application secret: it does not get .env as an environment, so no database password and not RESTOW_MASTER_KEY. It does read and rewrite the .env file in the mounted project directory, because it has to change the two image lines. It takes the database user and name from it for the backup, and registers every credential-like value in it so that none can reach a log.
• It listens only on the internal Docker network. No port is published.
• The api authenticates to it with a secret the updater generates on its first start into a Docker volume (restow-updater-shared, mounted read-only into the api). The secret is never in the repository, .env or any log.
• The read-only status the public maintenance page shows (phase, step, progress, a failure code) is the only thing the public edge forwards to it. It names no version and nothing about the installation; signed-in users see the versions in the web interface.
• Its own image never follows RESTOW_IMAGE, the line every update rewrites. It runs RESTOW_UPDATER_IMAGE, pinned by digest, and from 0.2.1 moves to a new image only after an update in image mode succeeded whose images passed the signature check: then it pins that release’s verified application image by digest and recreates itself (The updater updates itself). It never runs an image it built from source. It refuses to update while RESTOW_UPDATER_IMAGE is empty and the Compose file would take the updater’s image from RESTOW_IMAGE (the blocker “updater image not pinned”).
• In image mode it installs only release images signed by the project’s release workflow (see What is verified).
• Source mode, which builds an image from a repository and runs that repository’s Dockerfile on your host’s Docker, is off unless the operator names that repository in RESTOW_UPDATER_SOURCE_HOSTS on the host. Nobody can switch it on from the web interface.
• Changing the update source or token and announcing an update need a sign-in from the last 10 minutes, on top of the owner role (Confirming it is you).

If you would rather not have this on your host, do not enable it. Updating by hand is fully supported and takes a few commands.

Enabling it

From 0.2.1, run this once on the server, in the directory that holds docker-compose.yml and .env (/opt/restow for an installation made with the install script):

docker compose --profile updater up -d

Nothing has to be set in .env for it, and choosing an update source under Installation → Updates does not start it: only this command does.

• There is no separate updater image. The updater is the application image (the one in RESTOW_IMAGE) started with ROLE=updater. RESTOW_UPDATER_IMAGE and RESTOW_PROJECT_DIR are optional. While RESTOW_UPDATER_IMAGE is empty, the release Compose file starts the updater from RESTOW_IMAGE, and on that first start the updater writes RESTOW_UPDATER_IMAGE into .env itself, pinned by digest to the image it runs (ghcr.io/restow-backup/restow:0.4.7@sha256:...). That only makes sure the next rewrite of RESTOW_IMAGE does not reach the updater. If it cannot pin (a locally built image has no registry digest, .env is not writable), the tab shows the blocker “updater image not pinned”.
• The project directory is mounted at /project; set RESTOW_PROJECT_DIR only when the Compose file needs another path.
• Updating from 0.2.0 with the updater on: replace /opt/restow/docker-compose.yml with the one of the 0.2.1 release assets, install 0.2.1 from Installation → Updates, then delete the RESTOW_UPDATER_IMAGE line from .env (or set it to the 0.2.1 application image) and run docker compose --profile updater up -d updater once. From then on the updater follows signed updates itself.

To check the image’s signature before you give it the Docker socket (the install script already did this for the images it installed):

cosign verify ghcr.io/restow-backup/restow:0.4.7 \
  --certificate-identity https://github.com/restow-backup/restow/.github/workflows/release.yml@refs/tags/v0.4.7 \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Then open Installation → Updates. The install card says whether the updater is ready or what blocks it: Docker not reachable, the Docker command line image not pulled yet, Compose file not found, the project directory not matching, .env not writable, not enough free space, or the updater’s image not pinned. The first start pulls the Docker command line image, which needs access to Docker Hub; until then the card says it is preparing.

To remove it again: docker compose --profile updater rm -sf updater. The updater’s volume (restow-updater, holding the database dumps) stays until you remove it with docker volume rm.

The updater updates itself

From 0.2.1, after an update in image mode succeeded, and only when the release’s images passed the signature check of What is verified in that run, the updater moves itself to the same release:

1. It writes the release’s application image by the verified digest (<repository>:<version>@sha256:<digest>) into RESTOW_UPDATER_IMAGE: a digest, never a tag, so what runs next is exactly the content whose signature was checked.
2. A short-lived helper container (the pinned Docker command line image) runs docker compose --profile updater up -d --no-deps updater in the project directory, because a process cannot recreate its own container from inside.
3. The new updater reports in with the new version, and the tab shows it.

It never moves itself after an update built from source, when the signature check is off (RESTOW_UPDATER_VERIFY_SIGNATURES=false) or when you switch it off with RESTOW_UPDATER_SELF_UPDATE=false in .env (then recreate the updater). The tab then shows what to do by hand: set RESTOW_UPDATER_IMAGE to the image in RESTOW_IMAGE and run docker compose --profile updater up -d updater. If the self-update fails, the application update stays successful and the tab shows the command that finishes it.

The trust model. Up to 0.2.0 a person decided when the container that holds the Docker socket changed. From 0.2.1 the release workflow’s keyless signature decides: the updater moves only to an image that Sigstore’s transparency log records as signed by release.yml of exactly the tag being installed, and only by the digest that signature covers. If that is not acceptable for you, set RESTOW_UPDATER_SELF_UPDATE=false and move the updater by hand.

Switching a Community installation to the full build

From 0.2.1 a Community installation has the section Installation → Edition. It names the build, lists what Business and Service Provider add and switches to the full build of the version it runs now:

• With the updater: Switch to the full build is announced like an update and needs the owner role and a recent sign-in. The updater verifies both full images against the release signature of that tag, dumps the database, writes RESTOW_IMAGE=ghcr.io/restow-backup/restow:<version> and RESTOW_WEB_IMAGE=ghcr.io/restow-backup/restow-web:<version> into .env, recreates the services and rolls back to the Community images if the full build does not start. Both builds share one set of migrations, so nothing is migrated. Afterwards the updater moves itself to the full image too.
• Without the updater, the section shows the two .env lines to copy and docker compose pull && docker compose up -d.

There is no switch back to Community. A license key can be entered in the same section before the switch; the Community build keeps it sealed and unread, and the full build verifies and installs it on its first start. A key that does not verify is dropped and the rejection is audited.

Two modes

Restow picks the mode from the update source and shows it in the tab:

• Image (the default source, the project’s public releases). Verifies and pulls ghcr.io/restow-backup/restow:<version> for the application and ghcr.io/restow-backup/restow-web:<version> for the web edge (restow-community and restow-web-community on a Community installation; see What is verified), writes the two image references into .env (RESTOW_IMAGE, RESTOW_WEB_IMAGE) and recreates the services. If a release publishes no web image digest, the web edge stays as it is and the run says so.

• Source (any other repository, for example your own Forgejo). Downloads the tagged archive of the repository, with the stored token in an Authorization header (never in a URL, a command line or a log), builds the application and web images locally with docker build (the Dockerfile targets runtime and web, or runtime-community and web-community on a Community installation), then recreates the services the same way. This takes longer than a download.

  What is built there runs as the application, with the database and the master key. Source mode is therefore off unless the operator names the repository in .env and recreates the updater:

  # One repository (recommended), several separated by commas, or a whole host
  RESTOW_UPDATER_SOURCE_HOSTS=git.example.com/acme/restow

  docker compose --profile updater up -d updater

  An entry is host/owner/repository (only that repository) or host (every repository on that host). GitHub repositories are named github.com/owner/repo; a bare github.com would allow every repository on GitHub, so do not use it. Until the repository is listed, the tab shows the exact .env line and offers no install button, and the updater refuses the request as well: installing from a source that is not on the list is answered with 409 urn:restow:problem:update-source-not-allowed. The api reads the same variable to let the update check reach a listed host on a private network.

What is verified

In image mode, before anything is stopped:

1. The release must publish the digest of the application image in its notes (restow: sha256:..., or restow-community: sha256:... for the Community build). A release without one is refused: the tab does not offer it (409 urn:restow:problem:update-not-verifiable), and the updater ends such a run as unchanged. Update to it by hand after checking the release. The web edge changes only when the release also publishes the web image’s digest.
2. Each image is checked in the registry, by that digest, with cosign (keyless): it must carry a signature whose certificate was issued by https://token.actions.githubusercontent.com for exactly https://github.com/restow-backup/restow/.github/workflows/release.yml@refs/tags/v<version>, the release workflow run of the version being installed, and the signature must be in Sigstore’s transparency log. A digest someone typed into release notes, an image built anywhere else and the image of another release all fail this (fetch.signature_invalid, unchanged, nothing pulled).
3. Only then is the tag pulled, and the local image must carry the verified digest (fetch.digest_mismatch otherwise).

Each run shows under Release signature whether the signature was verified. cosign v3.1.3 runs in a short-lived container of an image pinned by digest (RESTOW_UPDATER_COSIGN_IMAGE), without capabilities, read-only, through the same Docker daemon, on the default Docker bridge network. During an update it needs outbound HTTPS to the registry (ghcr.io) and to Sigstore’s trust root (tuf-repo-cdn.sigstore.dev). Without that access the check fails and nothing is installed; updating by hand still works.

If you mirror the images (RESTOW_UPDATER_IMAGE_REPOSITORY, RESTOW_UPDATER_WEB_IMAGE_REPOSITORY), copy the signatures with them (cosign copy); otherwise every update fails with fetch.signature_invalid. A test installation or an unsigned mirror can switch the check off with RESTOW_UPDATER_VERIFY_SIGNATURES=false in a Compose override of the updater service (it is deliberately not in the Compose file); the digest is still required and checked, and every run records that the signature was not checked.

In source mode nothing is signed: the archive is fetched over https from a repository on the operator’s allowlist, with the stored token if there is one, and built as it is. The updater checks the host and repository against the allowlist, the size of the archive and that it holds a Dockerfile; it does not check who wrote the code, a signature of the tag, or what the Dockerfile does. Trust in that repository is the whole protection, which is why only the operator can allow it.

What an update does

The owner picks the version and a lead time (immediately, 1, 5, 15, 30 minutes or 1 hour) and confirms. Since 0.4.2 the install dialog asks When: From now (with that lead time, as before) or At a date and time (At a chosen time). From then on every signed-in person, not only administrators, sees a banner with a live countdown, and a notice when it is announced. The owner can cancel until the update starts. At the start, everyone sees a full-screen notice with the steps and the progress. The steps name the version they work on (“Downloading the image of version X.Y.Z”); before 0.4.2 some of them showed a literal {image} or {file} instead.

1. Prepare. Checks that Docker answers, the Compose file is there, .env is writable, there is room for the update (Disk space), the database is not being moved (Database), the target is newer than the running version, that the Compose file takes the image of api, worker, scheduler and web edge from RESTOW_IMAGE and RESTOW_WEB_IMAGE (so a mixed-version installation cannot result), and that it does not take the updater’s own image from them.
2. Verify and download, or build. Image mode verifies the signatures first (What is verified). Nothing is stopped yet: a failure here changes nothing.
3. Database backup. pg_dump -Fc of the database into the updater’s volume, checked for readability. The last three are kept, plus the one a run that needs attention depends on.
4. Stop worker and scheduler. Running jobs get up to 60 seconds to save a checkpoint and stop. They continue after the restart from their last checkpoint, and the stop does not use up one of their attempts. Machine and Proxmox VE backups that run on the agents are not stopped (see Runs in progress during an update).
5. Start the new version. The api is recreated with the new image and applies its database migrations as it starts.
6. Health check. Waits until the api reports ready and the new version (it gives up at once when the api container keeps restarting), then starts worker and scheduler, then recreates the web edge last.
7. Done.

While the api restarts, the edge answers browsers with a static maintenance page (in the browser’s language, light or dark; it shows the progress without naming versions) and answers API calls with a 503. The web interface treats that as “the server is restarting” and reloads by itself once the new version answers.

State lives in the updater’s volume, so it survives the api restarting. The result of every run also appears in the notification bell.

Runs in progress during an update

Since 0.4.0 the install dialog counts what runs and waits right now, in every tenant, and asks what should happen to it. It names the runs by group: Microsoft 365 and mail (backups, restores, imports, exports, restore checks), Servers and clients, Proxmox VE and File shares. With runs in progress it asks:

• Stop them cleanly (the default). Runs of Microsoft 365 and mail save a checkpoint and stop within a minute; they go back to the queue and continue after the update without using up an attempt. Runs on the agents (servers, clients, Proxmox VE) and of file shares are not stopped by the update, so it waits for them, 30 minutes at the latest. Since 0.4.2 an administrator can stop them one by one in the web app first (Proxmox VE, servers and clients, file shares).
• Wait until they finish. The update is announced, and its countdown starts, once nothing runs any more, after two hours at the latest.
• Do not wait. The update is announced right away, as before 0.4.0. What still runs when it starts is stopped, saves a checkpoint where it can and continues afterwards.

The confirm button stays disabled until the runs were read (or reading them failed), so a quick click cannot announce the update before you saw them.

In every case no new run starts from the confirmation until the update is over: the scheduler plans nothing (schedules that fall due meanwhile run on its first pass afterwards, nothing is skipped), the worker takes no new job from its queues, file share runs and Proxmox VE backup tasks wait, and the agents of servers and clients start no backup (below). This state is kept in the database, so it holds while the services restart. Everyone who is signed in sees a banner while the update waits; once it is announced, the countdown banner takes over. The pause lifts by itself when the update has finished or was cancelled, and in any case a few hours later (the longest wait, the lead time and two hours for the update), so an installation is never left paused. The audit log records update.jobs_paused and update.jobs_resumed.

While the update waits, the Updates tab shows what still runs and until when it waits at the latest. Start now announces the update at once, Cancel drops it, and new runs start again. Should the updater refuse the update when the wait is over (it became unavailable or blocked meanwhile), new runs start again and the administrators get a notice in the bell.

What happens to a run that the update stops:

• Microsoft 365, IMAP and mail. It saves a checkpoint, goes back to the queue and continues from there after the update. A stop by an update or a restart never uses up one of its attempts (also without a checkpoint), for up to 21 days after the run first started. Only when a run has no attempt left does it end as failed, and its explanation then says that its checkpoint was discarded and the next run starts over. Restore points of earlier runs are not affected.
• A worker that was killed instead of stopped (the stop timeout ran out, the host went down) cannot hand its jobs back. The next worker finds them two minutes after its start: a job that is still marked as running but started before every live worker goes back to the queue with its attempt, and its run shows as interrupted. Before 0.4.0 such a job waited for its expiration, 23 hours for a backup.
• Proxmox VE. The node helper keeps sending the blocks, the commit and the end of a running backup for five minutes while the server is unreachable (restow-pve 0.4.0; older helpers about 30 seconds), which covers an update with its database migrations. A backup run whose end never reached the server is closed as soon as the node’s heartbeat no longer lists it (restow-pve 0.4.0), otherwise after 48 hours.
• Servers and clients. A request of the agent that is in flight while the api restarts fails. The run counts as interrupted (not as a failed backup) only when the server is still unreachable at its end; otherwise it ends as failed and the next scheduled run backs up again. Wait or stop cleanly to avoid this.

The agents of servers and clients wait for the pause (agent 0.4.0, see the update pause). The answers to an agent’s configuration request and heartbeat say whether new runs are paused (startsPaused), and the agent asks again every minute while they are. While paused, or while the server does not answer (the api restarting), a backup that falls due, a retry and a catch-up wait and start once the pause is over and the server answers again. Such a wait is neither a missed backup (the catch-up rules of an agent policy do not apply to it) nor a failed one. A run that is already running continues and is counted, and waited for, by the dialog as before. Back up now from the web interface stays a waiting task on the server and is handed to the agent with its first heartbeat after the pause, whatever the agent’s version; restow-agent backup-now on the machine says that the instance is being updated and waits (Ctrl-C stops waiting, nothing was started). An agent older than 0.4.0 does not know the pause: a backup its own schedule starts during the pause runs, as before, and may fail if the api restarts under it.

An update by hand (docker compose pull && docker compose up -d) does none of the waiting: the worker gets its 60 seconds, and everything above about stopped runs applies.

Nodes and agents follow the update

Since 0.4.2 the Proxmox VE nodes (restow-pve) and the agents of servers and clients follow the server. Once the updated server runs, the pause of new runs does not lift at once: it moves to a second phase, and every node and agent behind the new release is told with its next heartbeat to update now. They install the signed release only while idle, and during the pause they are.

• When the pause lifts. As soon as every reachable node and agent reports the new version, and after 15 minutes in any case. Reachable means: called within the last seven minutes. An agent whose tenant (or the machine itself) paused agent updates is not waited for.

• What everyone sees. The banner says “Proxmox VE nodes and agents update now; new runs start once they are done, at the latest on …” instead of the countdown.

• The Updates tab shows the progress under Nodes and agents: how many are updated, updating, not reachable and with updates paused, and lists the ones still to come with their tenant, version and last contact (up to 200 by name; the counts cover all of them).

• The ones that were not reached show the command for an update by hand, as root on the machine:

  /opt/restow-pve/bin/restow-pve update                        # Proxmox VE node
  sudo restow-agent update                                     # agent 0.4.2 or newer (Linux, macOS)
  sudo systemctl restart restow-agent                          # Linux agent before 0.4.2: it checks a minute after it starts
  sudo launchctl kickstart -k system/com.restowbackup.agent    # macOS agent before 0.4.2

  restow-agent update installs the offered release at once, through the same signed download as the automatic self-update, restarts the service and says what it did. While a backup runs on the machine it refuses (“A backup is running; the update waits”) and the agent updates itself once it is idle. The tab names the command that fits each agent’s version. See Commands.

  restow-pve update installs the offered release on a node. Since 0.4.3 it restarts the Proxmox VE daemons and the restow-pve service itself and ends with “Updated to X.Y.Z.”; a helper before 0.4.3 installs the release and tells you to restart the services (systemctl restart restow-pve pvedaemon pvestatd pveproxy pvescheduler), which a node on restow-pve 0.4.1 needs after its update by hand.

• Update to X.Y.Z on a node and on a machine. Whenever a newer release is offered, the page of a node (Proxmox VE) and the page of a machine have a button Update to X.Y.Z, the offered version, also outside a server update. The node or agent installs the release with its next contact as soon as no backup or restore runs there, then restarts. Both are audited (pve.node.update_requested, endpoint.update_requested). Agent updates that the tenant or the machine paused stay paused, and the page says so.

An update by hand (docker compose pull && docker compose up -d) starts no fleet phase on its own: the nodes and agents update themselves within hours, as before, or at once with the commands above, or with Update nodes and agents now.

Update nodes and agents now

Since 0.4.3. When the server runs a newer release than its Proxmox VE nodes and agents, but no update led into the step in which they follow it (the server was updated by hand, or the answer to an announced update was lost), the nodes and agents that are behind show as Update available on the card Nodes and agents of the Updates tab. The same card has the button Update nodes and agents now.

• Who. An administrator with every tenant. The dialog asks first: “New runs pause for at most 15 minutes, as after an update. Reachable Proxmox VE nodes and agents that run an older release install the new one as soon as they are idle; the pause ends once all of them are done. Unreachable ones and agents with paused updates are not changed.”
• What happens. The same as after an update: new runs pause for at most 15 minutes, every reachable node and agent that is behind installs the release while idle, and the pause ends once they are done or the 15 minutes have passed. The progress shows on the card as above.
• When it is offered. Only while a reachable node or agent is behind and no update waits for runs, is announced or runs (no pause of new runs holds). The dialog adds the hint “No update led into this step (for example, the server was updated by hand).” If nothing reachable is behind any more when you confirm, the page says so (“No reachable node or agent runs an older release. Update unreachable ones by hand once they are online again.”), and nothing is paused.
• Audit. update.fleet_started, with manual: true.
• Nodes on restow-pve 0.4.1 do not act on it. See Upgrading to 0.4.3.

The route behind the button is listed under Routes of the web interface.

At a chosen time

Since 0.4.2 the install dialog asks When: From now (with a countdown, as before) or At a date and time. For the second you pick a date, a time in the installation’s time zone (the dialog names it, see The installation’s time zone) and the countdown before it.

• Restow keeps the plan and confirms it the countdown before the chosen time exactly as if you had confirmed it then: the same checks, the chosen handling of runs in progress, the same countdown for everyone. Nothing pauses before. Everyone who is signed in sees a notice from a day before (“… will be updated to version X.Y.Z on …”).
• Limits. Up to 30 days ahead, and at least a minute ahead of its own countdown. A time the clock skips (the hour that daylight saving time leaves out) is refused. Planning needs the same as announcing an update: the Owner role and a recent sign-in.
• Cancel planned update on the Updates tab drops the plan; you can plan or announce it again at any time.
• If the server was down at the planned time and comes back more than an hour later, the plan is dropped instead of started, and the audit log says so.
• A time that is not valid is refused with “Choose another time” (422, update-invalid-time): it is already past (or too close to count down to it), more than 30 days ahead, or skipped by a clock change.
• A plan keeps the zone it was made in. Changing the installation’s time zone afterwards does not move a planned update.

The sidebar notice

When an update is available, the version at the bottom of the sidebar turns into a highlighted, gently pulsing Update to X.Y.Z available that leads to Installation → Updates. Only people who may install updates see it. With “reduce motion” switched on in the operating system there is no animation.

The installation’s time zone

Planned updates are read in the installation’s time zone, the Updates tab shows their times in it, and a new tenant starts with it (each tenant keeps a zone of its own for its schedules, changed in its customer data). It is chosen in the setup wizard, with the time zone of your browser preselected, and changed by an owner under Installation → Server & operation → Server, in the card Time zone, which has a searchable list of zones. Only IANA names are accepted, such as Europe/Berlin, no fixed offsets.

The first of these that is set applies:

1. the zone chosen in the web interface,
2. RESTOW_TIME_ZONE in the server environment,
3. the process zone TZ,
4. Europe/Berlin.

An installation that was set up before 0.4.2 has no zone of its own, so the environment applies as before until an owner saves one; Use the server’s takes a choice back. The card says where the zone in force comes from (“Chosen here”, “From RESTOW_TIME_ZONE in the server environment”, “From the process time zone (TZ) of the server” or “The default”). A value in the environment that is not an IANA zone is skipped, so a typo never stops anything. Changing it is audited (settings.time_zone.changed). See Environment variables.

Disk space

An update writes a full database dump into the updater’s volume, pulls two new images and lets the new api run its migrations. On a nearly full disk the dump alone could fill it and stop PostgreSQL in the middle of the update. Since 0.4.0 the updater therefore adds up what the update needs before it starts and refuses it when that is not free (blocker disk_space, with the numbers), instead of requiring a fixed 1 GB:

Part	Estimate
Dump	the newest dump plus a quarter; before the first one, the size of the tables without their indexes (an upper bound)
Images	the size of the images that run now (2 GiB when it cannot be read)
Migrations	a tenth of the database, at least 1 GiB
Margin	RESTOW_UPDATER_MIN_FREE_MB (default 1024 MB)

Dump and images must fit where Docker keeps its data (the updater’s volume and the image store live there by default). The migrations’ room is counted there too, unless the database lives on another disk, where it must be free instead. Installation → Database shows the estimate, the disks and how to move the database to another disk.

The kept dumps are listed in the Updates tab under Database backups, with their size. The provider owner can download or delete one there (a recent sign-in, written to the audit log). A dump holds the whole database, the encrypted secrets included: keep a downloaded one as carefully as the master key. See Database.

When something goes wrong

The updater never guesses. It ends a failed run in exactly one of three states:

• Unchanged. It failed before anything was stopped (Docker unreachable, the release published no digest, the signature did not verify, the image could not be pulled or did not match its digest, the build failed, the backup failed). The old version never stopped.
• Rolled back. The new version did not come up and the database migrations had not run (the updater stops the new api first, then compares the number of applied migrations with the one before the update). The previous images are running again, .env is restored byte for byte, and the tab says which step failed and why. This is also what happens when a full-build release that the license key does not cover refuses to start, since 0.4.5 (Releases your key does not cover).
• Needs attention. The new api failed after its migrations ran. Migrations cannot be undone, so the updater does not start the old version on top of a migrated database. It stops api, worker and scheduler, keeps the database dump and tells you what to do. The web edge keeps showing the maintenance page.

“Checking whether the update was accepted…” (since 0.4.3). Before it accepts an update, the updater checks Docker, the Compose configuration, the disk space and the running version, which takes 5 to 12 seconds on real installations. The api waits up to a minute for the answer (30 seconds for cancelling, dismissing, restarts and deleting a dump). When the answer is still lost, it asks the updater for its state again: if the update shows there, it counts as announced (the pause of new runs, the audit entry update.scheduled, the step for nodes and agents) and the tab shows the countdown. If the updater does not answer at all, the pause of new runs stays until it answers again and ends on its own in any case, and the tab reads again and shows what the updater reports. Before 0.4.3 the api waited 5 seconds and the tab could say “The updater is not running” although the updater accepted the update a moment later; the update then ran without the pause and without the step in which nodes and agents follow the server, so they stayed on the old release. For those, use Update nodes and agents now. The tab could also say “The updater is not running” for a few seconds every now and then; the updater now answers at once with its last check and checks again in the background (an update still checks anew before it starts).

If the updater itself is restarted or killed in the middle of a run, the run is recorded as interrupted (unchanged when it had not stopped anything yet, otherwise needs attention) and nothing is started on its own.

For needs attention, restore the dump taken before the update. The tab shows the exact file name and the previous image references:

# copy the dump out of the updater's volume
docker compose --profile updater cp updater:/state/dumps/<file> ./<file>
# stop the application and restore
docker compose stop api worker scheduler
docker compose exec -T postgres pg_restore -U restow -d restow --clean --if-exists < <file>
# put the previous images back into .env (RESTOW_IMAGE, RESTOW_WEB_IMAGE), then
docker compose up -d

Backups made after the update are in the chunk store but not in the restored database, so run a backup after the rollback. Then report the failure with the log tail the tab shows. Dismiss clears a finished run from the page. Use it only after you have restored the installation, because the recovery commands disappear with it. The run stays in the audit log.

Audit events

Every step that matters is written to the audit log: update.check, update.settings.updated, update.scheduled, update.cancelled, update.started, update.succeeded, update.failed, update.acknowledged (dismissing a finished run), from 0.3.6 update.restart.requested (Restart now) and, from 0.4.0, update.jobs_paused and update.jobs_resumed (the pause of new runs), update.dump.downloaded and update.dump.deleted (the kept database backups) and the database move entries database.move.requested, database.move.started, database.move.succeeded, database.move.failed, database.old_copy.removal_requested and database.old_copy.removed (Database). From 0.4.2 there are update.planned (an update planned for a date and time; dropping it writes update.cancelled with planned: true), update.plan_failed (a plan that could not start), update.fleet_started (nodes and agents asked to update; update.jobs_resumed then names fleet_done or fleet_deadline as its reason), pve.node.update_requested and endpoint.update_requested (Update to X.Y.Z on a node or a machine) and settings.time_zone.changed (the installation’s time zone). The updater has no database access of its own, so it keeps a journal and the api writes it to the audit log afterwards, in order and once, even when the api was replaced in between. See Audit log.

Settings of the updater

The Compose service sets what it needs. These variables are read by the updater (ROLE=updater); the api reads RESTOW_UPDATER_SOURCE_HOSTS as well:

Variable	Default	Meaning
RESTOW_UPDATER_PROJECT_DIR	required, set from RESTOW_PROJECT_DIR	Absolute host path of the Compose project, mounted at the same path.
RESTOW_UPDATER_IMAGE_REPOSITORY	the release images of the updater’s own build (ghcr.io/restow-backup/restow or ghcr.io/restow-backup/restow-community)	Application image repository for image mode; set it only for a mirror, which needs the signatures too. Passed from .env.
RESTOW_UPDATER_WEB_IMAGE_REPOSITORY	ghcr.io/restow-backup/restow-web or ghcr.io/restow-backup/restow-web-community	Web image repository for image mode. Passed from .env.
RESTOW_UPDATER_SOURCE_HOSTS	empty (source mode off)	Repositories (host/owner/repo) or hosts source mode may build from, comma-separated. Passed from .env.
RESTOW_UPDATER_HEALTH_TIMEOUT_SECONDS	600	How long to wait for the new api (migrations can take a while).
RESTOW_UPDATER_MIN_FREE_MB	1024	Margin on top of what an update needs (Disk space).
RESTOW_UPDATER_CLI_IMAGE	docker:27-cli, pinned by digest	Image used to run docker and docker compose (see below). Must carry a digest (name:tag@sha256:...).
RESTOW_UPDATER_COSIGN_IMAGE	ghcr.io/sigstore/cosign/cosign:v3.1.3, pinned by digest	The cosign image that verifies the signatures. Must carry a digest.
RESTOW_UPDATER_VERIFY_SIGNATURES	true	false skips the signature check (test installations, unsigned mirrors); digests stay required. Set it in a Compose override only.

The Compose file itself reads RESTOW_UPDATER_IMAGE from .env for the updater service’s image (required in the release Compose file).

The published image contains no Docker command line. The updater therefore runs each docker and docker compose command in a short-lived helper container of RESTOW_UPDATER_CLI_IMAGE (by default from Docker Hub, pulled in the background when the updater starts; the updater cannot install anything before that and says so), through the mounted socket, without network access and with the project directory and the updater’s state volume mounted. If a docker binary is present in the updater’s own image, it is used directly instead. The helper containers carry no registry credentials: the images of the project’s public releases need none, and a private registry is not supported in this mode. The api reaches the updater at RESTOW_UPDATER_URL (default http://updater:8090). Nothing answers there unless the updater profile runs.

Rolling back by hand

• No migrations in the release: put the previous RESTOW_IMAGE and RESTOW_WEB_IMAGE back into .env and run docker compose up -d again. The updater does this by itself when a new version does not start.

• With migrations: migrations are not reversible. Stop the stack, restore the database dump taken before the update, then start the previous version:

  docker compose stop api worker scheduler
  docker compose exec -T postgres pg_restore -U restow -d restow --clean --if-exists < restow-YYYY-MM-DD.dump
  # put the previous RESTOW_IMAGE and RESTOW_WEB_IMAGE back into .env, then
  docker compose up -d

  Backups taken after the update are in the chunk store but not in the restored database, so run a backup after the rollback.

  After a database move (Installation → Database, since 0.4.0), the compose file you roll back with must still read RESTOW_POSTGRES_DATA from .env; the file of 0.3.6 does not, and PostgreSQL then refuses to start on the old copy. install.sh refuses such a file with exit code 8 (Database).

Versions and channels

Restow uses Semantic Versioning. Before 1.0.0, a minor version (0.2.0) adds features and may carry migrations. A patch version (0.2.1) fixes problems and carries migrations only when a fix needs one. Tags are vMAJOR.MINOR.PATCH, and pre-releases are marked -rc.N. The stable channel of the update check offers releases only, the beta channel also offers the pre-releases. The Docker tags of a release are a separate matter, see Verifying a release.