Troubleshooting

Start with the explanation in the web interface

Every failed job, item, sync and verification in Restow says what happened, why, and what to do about it, with a link to the right settings page where there is one. Under Technical details it lists what a support case needs: the HTTP status, the Microsoft Graph error code and the request IDs, with secrets removed. Look there first. Failure explanations lists the causes and what to do about each one. For servers and clients backed up with the agent, see Endpoint backup troubleshooting.

The links in these explanations lead to this page by default. RESTOW_DOCS_TROUBLESHOOTING_URL points them at your own runbook instead.

Installation

docker compose up -d fails or a container keeps restarting. Check docker compose ps and docker compose logs <service>. The most common cause is a missing required value in .env. See Get started → Fill in the required values; Compose refuses to start postgres at all if POSTGRES_PASSWORD is unset, and caddy if RESTOW_APP_DOMAIN is unset.
Compose stops with set RESTOW_IMAGE in .env or set RESTOW_WEB_IMAGE in .env. The release stack needs both image lines of one build in .env, for example ghcr.io/restow-backup/restow:0.4.7 and ghcr.io/restow-backup/restow-web:0.4.7 (or restow-community and restow-web-community). See Get started → Fill in the required values.
The api does not start and says RESTOW_SETUP_TOKEN must have at least 16 characters. A setup token you set yourself needs at least 16 characters, spaces and dashes not counted. Generate one with openssl rand -hex 16, or remove the line to let the api print a token to its log. See Setup wizard → Setup token.
docker compose --profile updater up -d does not start the updater. The release Compose file takes the updater’s image from RESTOW_UPDATER_IMAGE and, while that is empty, from RESTOW_IMAGE, so the command fails only when RESTOW_IMAGE is empty as well. Fill in RESTOW_IMAGE (and, if you set RESTOW_UPDATER_IMAGE yourself, use an image of the same build). See Release stack reference → updater and Updates → Enabling it.
Caddy can’t obtain a certificate. RESTOW_APP_DOMAIN must resolve (A/AAAA) to the host, and ports 80/443 must be reachable from the internet for the Let’s Encrypt challenge. This only applies in public mode.
api, worker or scheduler refuses to start with a Row Level Security error. The application role (DATABASE_URL) must not be a superuser and must not be able to bypass RLS; this check runs at startup on purpose. Re-check the three database roles in Get started.
install.sh stops with exit code 8 and says the database of this installation is in a directory. The database was moved to another disk (Installation → Database), and the docker-compose.yml that the installer would put in place or start does not read RESTOW_POSTGRES_DATA from .env; started that way, the stack would not find its database. Nothing was changed. Use the docker-compose.yml of release 0.4.0 or later. See Database.
The updater refuses an update with the blocker disk_space. Since 0.4.0 it adds up the database dump, the images and the migrations and refuses an update that does not fit. Free space or move the database to another disk. See Updates.
A notice says the disk is low. The disk under the database or under the local default repository has less than a tenth free, or less than 5 GiB. See Database.
After a server update, new runs stay paused for a while. Since 0.4.2 the pause does not lift when the new server starts: Proxmox VE nodes and agents install the release first, until all that report in are done and for 15 minutes at the longest. Installation → Updates shows the progress and the command for the ones that were not reached. See Nodes and agents follow the update.
After an update by hand, Proxmox VE nodes and agents stay on the old release. An update by hand starts no step in which they follow the server. On Installation → Updates they show as Update available, and Update nodes and agents now (since 0.4.3, for an administrator with every tenant) starts that step: new runs pause for at most 15 minutes while reachable nodes and agents install the release. A node on restow-pve 0.4.1 ignores it and needs one update by hand. See Upgrading to 0.4.3.
The Updates tab says “Checking whether the update was accepted…”, or said “The updater is not running” for a moment. The updater checks Docker, the Compose configuration and the disk space before it accepts an update, which can take up to about 12 seconds. Since 0.4.3 the api waits up to a minute and then asks the updater for its state, so the false “not running” is gone; the tab shows the countdown as soon as the updater answers. Before 0.4.3 such an update could run without the pause of new runs and without the step for nodes and agents: use Update nodes and agents now. See When something goes wrong.
A planned update is at another time than you expected, or did not start. Planned updates are read in the installation’s time zone, which the dialog names: check it under Installation → Server & operation → Server (the order is the zone chosen there, RESTOW_TIME_ZONE, TZ, then Europe/Berlin). A plan is dropped, with the audit entry update.plan_failed, when the server was down at the planned time and came back more than an hour later. See At a chosen time.
File share runs wait, or the mounter reports runner_network_missing. The mounter is off, or the docker-compose.yml is older than 0.4.0 and lacks the internal network runners. Run the installer again and start the mounter. See Upgrading to 0.4.0 and File shares: troubleshooting.

Backups

A job is stuck or very slow. Check the job’s progress for a throttling notice: Microsoft Graph throttling is shown, not hidden, and the first backup of a large tenant can legitimately take days. From 0.3.5 a Microsoft Graph request that gets no answer within 5 minutes is retried instead of hanging until the job’s time limit. If there is no throttling notice and no progress, check /healthz and /readyz, and confirm the worker and scheduler containers are running.
A long first backup was interrupted. From 0.3.5 a backup that is interrupted by its time limit (23 hours) or by a worker restart, after it saved a checkpoint, continues without using up one of its retries, for up to 21 days after it first started. The run view shows such an interruption as information, not as an error. For the worker to save the checkpoint when it stops, the worker service needs stop_grace_period: 60s (Updates).
A mailbox, OneDrive or site says “Not backed up: account disabled”, “Not backed up: no licence” or “Not backed up: locked” (since 0.4.7). Microsoft reports that there is nothing to back up: the account is disabled, it has no Exchange Online license, or the OneDrive or site is locked (HTTP 423, which Microsoft does to the OneDrive of an unlicensed account). It is no failure and raises no failure alert; the existing backups are kept. Enable the account, assign a license or lift the lock, and backups resume after the next directory sync. Until an administrator answers Yes, intended, the object reads amber and readiness stays yellow. See Objects that are no longer backed up.
A backup says “Held back by the service, retried” (since 0.4.7). Microsoft Graph or the storage asked Restow to slow down or was unavailable for longer than the run’s retries. It is amber, not a failure, and the next scheduled run tries again; Back up now tries sooner. If the object stays held back, Backup overdue speaks up. See Backups held back by a service.
A permission you granted to the app is still refused. Since 0.4.7 Restow fetches a new token after a 401 or 403 and the permission works within about a minute, without a new client secret. Before 0.4.7 a token kept the old permissions for up to about an hour. The exception is notification mail through a Microsoft 365 app registration: a Mail.Send permission granted later takes effect once the cached token expires. See Adding a permission later.
All backups of all tenants waited behind one backup (before 0.4.7). A queued backup of an object whose backup was still running (the next scheduled run, Back up now) used to hold up the backup queue of every tenant until the running one ended. Since 0.4.7 it waits for its object and every other backup passes.
The bell says the credential of the Microsoft 365 app, or of a source’s own app, expires or has expired (since 0.4.7). Create a new client secret (or certificate) in Entra, enter it with its new expiry date, and only then delete the old one. Without a date there is no reminder. See Expiry of the secret or certificate.
A backup shows as failed. Restow marks a backup as failed if its scheduled restore verification did not succeed, even if the backup run itself reported success. Treat that as the accurate status, not a false alarm. See Backups and schedules.
A file share backup ends with warnings or fails. The run names the cause (share.*) and what to do; the common ones are in File shares: troubleshooting and limits, the full list in Failure explanations.
Directory sync warns that SharePoint sites are neither found nor backed up (sites_unavailable). The backup’s app registration lacks Sites.Read.All or Sites.ReadWrite.All, or the customer’s administrator has not consented again. See SharePoint Online.
A restore point is marked Suspicious and retention is paused. Ransomware detection found signs in what the backup saw. Until an administrator marks the finding as a false alarm or releases the hold, no restore point of that object is deleted. See Ransomware detection.
Saving an S3 repository fails with endpoint_insecure. Since 0.4.0 a new or changed S3 endpoint must use https:// (except to localhost, 127.0.0.1 and [::1]). Use an HTTPS endpoint, or set S3_ALLOW_INSECURE=true for a test setup. See Immutable backups.
Space does not come back after a retention run on an immutable repository. Objects that are still locked cannot be deleted; the restore point stays visible and space returns once the lock has run out, up to the period plus 7 days later. See Immutable backups.
The storage check lists packs as temporarily unreadable, and objects say Attention. Since 0.4.2 a pack that a repository could not serve at the moment (a download that broke off, a reset connection, a timeout, S3 asking to slow down, or denied access) is not called damaged. Check the availability and the access rights of the storage; the next check reads those packs first, and Re-check these packs under Recovery readiness → Storage integrity reads only them again. They become Not restorable only when a pack is proven damaged or stayed unreadable in 3 consecutive checks over at least 24 hours. See Damaged or temporarily unreadable and the causes verify.storage_unreadable and verify.storage_unreadable_persistent in Failure explanations.
“Keep existing backups where they are” is greyed out, or refused with storage-switch-blocked-by-repositories. Containers or file shares of the tenant have restic repositories on the current primary, which only a move takes along; so does promoting a copy. Choose Move existing backups. See Replacing the primary repository.
A backup is shown as cancelled. Someone stopped it with Stop (Proxmox VE guests, servers and clients, file shares). It is not a failure and raises no alert. See Stop a backup and pause a queue.
A Proxmox VE guest says “Could not check: repository locked”. Another run held the lock of the container’s repository for two hours, so the restore check could not read the restore point. It is amber and not a failure: nothing is known about the restore point yet. Restow retries after ten minutes, removes stale locks itself and tries again after a day; Check now reads it sooner. Before 0.4.3 the same cause showed as a red “Restore check failed” with restic’s exit code 11, which stays red until the next check. See Restore checks that could not run and the Proxmox VE troubleshooting.

Restores

A restore did not go where you expected. By design, restore is built not to overwrite an existing original; if something with the same name already existed in the target location, Restow will have written the restored item in alongside it instead (a new folder for mail, a renamed file). Check the restore’s log entry, or its job detail page, for exactly which mode was used and where the item went. See Backup and restore.
You cannot find a restore in the audit log. Every restore is recorded in every edition, but the audit log viewer is a Business and Service Provider feature. If you have the viewer and cannot find one, check that you are looking in the correct tenant (Service Provider edition) and time range.

Archive and journaling

The Archive page says “Receiver not running”, “No reports yet” or “No report for more than 24 hours”. The reason and the checks are on Exchange journaling: the TLS certificate the receiver needs, DNS, TCP port 25 from the internet, the Microsoft 365 connector and the journal rule, and the mailbox for undeliverable journal reports.
Graph capture checks the mailboxes every 15 minutes instead of being notified (since 0.4.4). The Archive page names the reason. graph.notification_url_unavailable: the public URL is not https:// on port 443, or it names localhost, a private address or a .local, .internal or .lan host. graph.subscription_failed: Microsoft refused the subscription, usually because it cannot reach /api/v1/hooks/graph/notifications from the internet. See Network and ports.
A mailbox shows Error under Capture from the mailboxes, or the alert Archive capture failing for over an hour came. Find the mailbox in the table Per mailbox and check what you would check for a failed backup of it: the consent, the password, whether the server is reachable. When it works again, the status returns by itself; Catch up reads the mailbox again from the start. See Archiving.
IMAP archiving makes a provider refuse connections. Many providers allow only a few connections per account, and IDLE keeps one open. Set the source’s Archiving to Check at intervals only (Connecting sources).
No forwarded copies arrive. Check the status of the account under Forwarding to the archive, the rule at the provider with the address from the Archive page, and the receiver (port 25, certificate, JOURNAL_HOSTNAME). See Forwarding copies from the mail provider.
The receiver does not start after you set JOURNAL_SMTP_PORT. It starts only with a certificate and key of your own (JOURNAL_TLS_CERT_PATH, JOURNAL_TLS_KEY_PATH); the api log names the file that is missing, unusable or expired. See Exchange journaling → TLS certificate.

The setup wizard says “This is not the setup token of this server”. The token is new at every start of the api until the setup is complete; you probably copied an older one. Show the one printed last with docker compose logs api | grep 'SETUP TOKEN' and enter it (case and dashes do not matter). If RESTOW_SETUP_TOKEN is set in .env, enter that value instead; nothing is printed then. The API answers a wrong token with 403 urn:restow:problem:setup-token-invalid. See Setup wizard → Setup token.
The wizard says the setup token has changed. The api restarted while you were in the wizard. Enter the token it printed last; your entries are kept.
An owner lost their passkey, authenticator app or password. Recover the access on the server: docker compose exec api restow admin recover --email <owner>. It sets a new password, removes the owner’s authenticator app and passkeys, ends their sessions and writes an audit entry; it works for owners only. Other administrators are reset by an owner in the web interface. See Setup wizard → Recovering administrator access.
Passkeys are not offered yet. Passkeys only appear once your domain is verifiably and cleanly connected: valid HTTPS certificate, matching origin. Until then, sign-in uses the emergency password path with mandatory TOTP. Re-run the domain check after fixing DNS or certificates (see Setup wizard → Passkey readiness).
A Microsoft 365 admin-consent grant looks incomplete. Restow reports exactly which permissions were and were not granted; re-run the consent flow as an administrator with sufficient privileges in the tenant (see First steps).
People can’t sign in with their Microsoft account. Sign-in with Microsoft (Entra ID) is planned and not available in 0.4.7 (it did not come with 0.4.0): the sign-in code exists, but nothing links a Restow account to a Microsoft identity yet, and the login page does not offer it. Sign in with a passkey, or with the emergency password and an authenticator app.

Integrations and licensing

A webhook shows as “Failing”. Open its delivery log for the exact HTTP status or connection error from each attempt. A target in a loopback or private network is refused unless the operator explicitly allowed it. See Integration API → Webhooks.
A license key is rejected. Check that the key was issued for this installation’s own installation ID (shown under Installation → License, next to Install a key). A key issued for a different installation, or one that fails its Ed25519 signature check, is refused rather than partially applied. See License and editions.
A menu entry is greyed out with a lock, or there is nowhere to enter a license key. In the full images, Business and Service Provider entries stay locked until a key is installed; the lock leads to Installation → About, the section that Installation → License opens. The Community images have no license settings and no Business or Service Provider entries at all: switch to the full images to use them. See License and editions.

Error answers of the web interface and the API

The web interface turns these answers into a message with what to do; the API returns them as problem details (type is urn:restow:problem:<name>).

Problem type	What you see, and why	What to do
`recent-sign-in-required` (403)	“Confirm it is you”. Changing the update source or token, announcing an update, setting or changing an endpoint hook, and showing an endpoint’s repository password need a sign-in from the last 10 minutes.	Confirm with your passkey, or sign in again and repeat the change. See Updates → Confirming it is you.
`update-source-not-allowed` (409)	Installing an update built from a repository that `RESTOW_UPDATER_SOURCE_HOSTS` does not name. Source mode is off by default.	Add the repository to `RESTOW_UPDATER_SOURCE_HOSTS` in `.env` and recreate the updater, use the default source, or update by hand. See Updates → Two modes.
`update-not-verifiable` (409)	The release publishes no application image digest, so the updater cannot verify what it would install.	Check the release with cosign and update by hand. See Verifying a release.
`invalid-update-source` (422)	The update source you saved is not a usable `https` repository address, or it points to a private address or local-only host name that `RESTOW_UPDATER_SOURCE_HOSTS` does not name.	Enter the public address of the repository, or name an internal host in `RESTOW_UPDATER_SOURCE_HOSTS`. See Updates → Source, channel and token.
`pve-guest-excluded` (409)	Back up now or putting a Proxmox VE guest into a job, for a guest that an administrator excluded from backup (since 0.4.7).	Include again on the guest’s page or in its row of the inventory, then back it up or put it into a job. Taking it out of its job stays possible. See Excluding a guest from backup.
`endpoint-quota-exceeded` (403, to the agent)	A machine’s upload would go beyond its storage budget or the tenant’s. The backup stops; restores keep working.	Raise the machine’s Storage budget or `RESTOW_ENDPOINT_QUOTA_GIB` / `RESTOW_ENDPOINT_TENANT_QUOTA_GIB`, or back up less. See Endpoint backup troubleshooting.
`endpoint-hooks-not-allowed` (409)	A hook was set for a machine whose hook policy is off (or that reports none). Only root on the machine can allow hooks.	Run `sudo restow-agent hooks scripts` (or `any`) on the machine, or remove the hook. See Endpoint backup troubleshooting.
`endpoint-repository-locked` (503)	“The repository is busy with a backup or maintenance”: the server is working on that machine’s repository (for example retention or a check) while you browse or download.	Try again in a minute. See Endpoint backup troubleshooting.
`preview-busy` (503)	All parser processes for the mail preview are busy and the queue is full.	Try again. If it happens often, raise `PREVIEW_PARSE_WORKERS` (memory permitting). See Environment variables.
`preview-unreadable` (422)	An attachment of a message that cannot be read within the parser’s limits. The message itself is shown with its header data.	Download the whole message as EML, or restore it.
`payload-too-large` (413)	The request body is larger than the route allows (1 MiB for most requests, 16 MiB for restore and export selections, import requests and lists; the limit is in `limitBytes`).	Send smaller requests, for example split a large selection.
`feature-unavailable` (403)	Community images: a function that only Business or Service Provider offers (a second tenant, provider API keys, the views across all tenants, time-triggered reports).	Use the full images with a license key. See License and editions.
`edition-required` (403)	Full images: a Business or Service Provider function without a matching license key.	Install a key under Installation → License.
`export-quota-exceeded` (422)	The tenant’s unexpired exports already use `EXPORT_MAX_TENANT_BYTES` (default 50 GiB).	Wait for older exports to expire (`EXPORT_TTL_HOURS`), export less, or raise the limit. See Mail file import and export.
`import-staging-full` (422)	The tenant’s uploads waiting for import would go beyond `IMPORT_MAX_STAGING_BYTES` (default 100 GiB).	Wait for an import to finish, remove an upload, use the server import folder (it needs no staging), or raise the limit. See Mail file import and export.
`setup-token-invalid` (403)	A wrong or outdated setup token.	Copy the token printed last from the `api` log. See Setup wizard.
`cross-site-request` (403), `unsupported-media-type` (415)	A setup, set-password or agent enrollment request came from another site or without a JSON body.	Send JSON from the Restow address itself.

Where to look next

The audit log, for who did what and when.
Your notification mail transport’s test-send result (SMTP or Microsoft Graph sendMail), configured during setup. A failing transport means operational alerts are not reaching you even if everything else is healthy.
The status and roadmap page, if you are unsure whether something is meant to exist yet at all.