Backup jobs

A backup job is the definition of what is backed up, when, into which repository and for how long, for many mailboxes, OneDrives, SharePoint sites, IMAP accounts, machines, file shares or Proxmox VE guests at once. Jobs came with 0.2.0 and replace the backup and restore-check schedules per object of 0.1.0. Since 0.2.1 a machine is backed up only while it is a member of a job.

Mail jobs and machine jobs

	Mail job	Machine job
Where	Mail & SaaS → Jobs	Servers & clients → Jobs
Members	Mailboxes, OneDrives, SharePoint sites (since 0.4.0), Microsoft 365 groups (since 0.4.5, opt-in) and IMAP accounts	Servers and clients with the agent
Schedule	A backup schedule and an optional restore-check schedule	Every N hours, daily at a time, or when the machine connects
What	The objects you pick, or “every object that is in no other job”	Folders, exclusions, a bandwidth limit with time windows, optional hooks, an agent policy
Retention	A retention policy, or the tenant default	How many daily, weekly and monthly restore points to keep

An object or a machine belongs to at most one job. A mail job with the scope “every object that is in no other job” also catches objects added later; a tenant has at most one such job. The first active source of a new tenant creates one mail job over all objects (backup every 8 hours, restore check every Sunday at 03:00). Since 0.4.0 that covers every SharePoint site of the tenant too, unless the sites mode of the protection rules is set to the selected sites only. Microsoft 365 groups are the other way round: since 0.4.5 such a job covers every group that is protected, and no group is protected until the protection rules switch Protect all Microsoft 365 groups on or you include single groups (Microsoft 365 groups).

Per-member overrides. One mailbox can run on its own schedule; one machine can have other folders, exclusions, bandwidth, hooks or retention. An override replaces the job’s value for that member only.

The first question of a mail job (since 0.4.2)

The editor of a mail job starts with What should this job do?

Choice	What it means
Back up and archive	The default for a new job, and the recommended one. Restore folders, calendar, contacts and whole mailboxes, and keep every mail tamper-evident as well.
Back up	Restore folders, calendar, contacts and whole mailboxes as they were at a restore point.
Archive	Keep every mail tamper-evident, captured before users can change or delete it, for GoBD, retention periods and legal hold (see Backup vs. archive).

Archive and Back up and archive switch the archive on for the job’s mailboxes. They replace the switch Archive this job’s mailboxes that came with 0.3.1. Archive alone does not back up on a schedule: the job then has no schedule, mail is captured by the archive as it arrives, and calendar, contacts and folders are backed up only when someone starts the job (a run started by hand still backs up). Picking Archive turns the schedule off, and picking another purpose turns it on again. A machine job cannot archive: the API refuses it (archive_not_supported), because the archive is mail.

What the editor adds under the question:

With the archive, it shows How mail reaches the archive, the capture ways per kind of mailbox (below). Exchange journaling, if you choose it, is set up once per tenant, a journal address and a journal rule in Microsoft 365, and the editor links to Set up journaling (Exchange journaling). The choice says which mailboxes are expected in the archive; Exchange journals by its own rule, and Restow archives every journal report it receives.
Object Lock is not required. If the tenant’s storage has no Object Lock, the editor shows a warning that the archive is then immutable at application level only (whoever can reach the storage or the server can change or delete data); if the archive is in the installation’s default repository, it says that the repository page shows whether it has Object Lock. Neither blocks saving. See Archive and journaling.
With the archive next to backups (Back up and archive) and a schedule that repeats more often than every 8 hours, the editor suggests backing up less often: “Backups can run less often”. The archive already captures every mail as it arrives, so backups only keep what the archive does not hold (folders, calendar, contacts and the read status), and every 8 hours is usually enough and puts less load on the mail server. It is only a suggestion: the schedule stays as you set it until you choose Back up every 8 hours. It appears for a schedule that repeats at a fixed interval, not for a daily time.
With backup alone, it says what a backup cannot do: it sees the mailbox as it is when it runs, so mail that arrives and is deleted between two runs (the editor names the gap, for example “within the 8 hours between two backups”) is in no backup, and backups are not tamper-evident. Archiving captures every mail as it arrives (journaling and forwarding at transport; Graph and IMAP archiving near-time, which is not complete).

How mail reaches the archive (since 0.4.4)

With the archive on, the editor recognizes the kind of each mailbox in the job and offers only the capture ways that fit. A job with mailboxes of different kinds shows the ways for each kind.

Kind of mailbox	Ways	Defaults of a new job
Microsoft 365 mailboxes (`m365`)	Microsoft Graph (near-time), Exchange journaling (at transport)	Graph on, journaling off
IMAP accounts (`imap`)	IMAP archiving (near-time), Forwarding to the archive address (at transport)	IMAP archiving on, forwarding off
Google Workspace (IMAP) (`google`)	IMAP archiving (near-time), Routing to the archive address (at transport)	IMAP archiving on, routing off

The kind is derived, not stored. An IMAP account is a Google Workspace account when its source is an OAuth2 source of Google or its host is imap.gmail.com or imap.googlemail.com. An IMAP account on a Microsoft host (outlook.office365.com and others, kind imap_microsoft) gets the IMAP ways and a hint to connect the tenant as a Microsoft 365 source instead, because Graph and Exchange journaling capture its mailboxes more completely. OneDrives, SharePoint sites and imported mailboxes have nothing to capture.
Each way is labelled At transport or Near-time and says what it does; a way that the edition does not include is locked and names the edition (“Part of the Business edition”). Exchange journaling and forwarding also need the journal receiver, which the editor links to (Set up journaling, Set up forwarding).
At least one way per kind of mailbox in the job. Without one the editor says Choose at least one way for these mailboxes, and the API refuses the change (below).
Defaults. A new job, and a job that switches archiving on, starts with the defaults above, each with a one-time takeover of the mail already in the mailbox, marked as captured afterwards. See Archiving.
A job from before 0.4.4 keeps what it did. Its Microsoft 365 mailboxes have Exchange journaling only, and for its IMAP and Google Workspace accounts the editor says at the card that no way is chosen yet: they are not archived until you choose one. Nothing starts by itself. Name and schedule can be saved as before. See Upgrading to 0.4.4.
The same mail by several ways is archived once. Choosing both ways for a kind is fine: Restow recognizes the mail by Message-ID and content and keeps one item with the stronger way (at transport beats from the mailbox). See Archiving.
Switching off asks first. Switching archiving off, or a capture way, shows Stop archiving the mail of this job? or Switch these capture ways off?: new mail of these mailboxes is no longer archived from then on, and the archive kept so far stays unchanged until the end of its retention.
Pausing a mail job stops only its scheduled backups. Archiving goes on while the job is paused or switched off, and the question asked when you pause says so.

In the API a mail job carries archiveCapture on create, change and read, per kind of mailbox:

{
  "archive": true,
  "archiveCapture": {
    "m365": { "graph": true, "journal": false },
    "imap": { "sync": true, "forward": false },
    "google": { "sync": true, "forward": false }
  }
}

A kind left out takes its default. The members and candidates of a job carry captureKind (m365, google, imap or imap_microsoft), and GET /api/v1/backup-jobs/defaults?kind=mail names the kinds of the tenant’s active mail objects as mailSourceKinds, which is what the editor uses to decide which ways to offer. A job that archives without any way for a kind of mailbox it covers is refused with 422 archive_capture_no_way (problem type urn:restow:problem:invalid-backup-job; field is archiveCapture, issues[0].path names the kind, for example ["archiveCapture", "imap"], and kinds lists the kinds without a way); the defaults never cause it. A job from before 0.4.4 may carry pending: true on imap and google (no way chosen yet): the API accepts such a job until a way is chosen, keeps the marker only where the job already had it, never sets it, and drops it when a way is chosen. A machine, file share or copy job refuses archive and archiveCapture. The edition decides which ways work (Business and Service Provider; the Community build keeps the capture service idle), and the known issues of the ways are listed on Archiving.

See Archiving and Archive and journaling.

Since 0.4.0 there are more kinds of jobs. Like a mailbox or a machine, a file share belongs to at most one backup job.

Kind	Where	What it does
File shares	Servers & clients → File shares, buttons Jobs (also from the dialog Add file share)	Backs up SMB and NFS file shares from the server on a schedule (at most hourly), with the folders to back up, the file types to skip and a retention (daily, weekly, monthly). A file share in no job is not backed up. See Add and back up.
Copy of a file share	Servers & clients → File shares, button Copy jobs	On a schedule, writes the newest restore point of share A that passed its restore check into a folder of share B. It is not a backup: the target keeps no versions, and the job is labelled so. See Copy jobs.
Proxmox VE (a preview)	Servers & clients → Jobs, next to the machine jobs	Backs up Proxmox VE guests, see below.

Proxmox VE jobs in Servers & clients → Jobs. Once Proxmox VE is connected, its jobs are listed beside the machine jobs, and the list gets a Type column (Machines or Proxmox VE). A Proxmox VE row shows the guests of the job, the schedule, the last and the next run, “2 of 5 guests done” while a round runs, and a backup that someone started by hand as a row “Manual backup · web-01” until it ends. New job is a menu with one entry per type: Servers and clients (Restow agent) and Proxmox VE (VMs & containers); the Proxmox VE form lists the VMs and containers to put into the job. See Backup jobs and protection.

The job pages and the editor

The job list shows scope, schedule, repository, last and next run, status and the result of the last restore check. A job shows green only for a passed restore check, never for a backup that merely completed. A job opens in tabs: Overview, Scope, Settings and Runs.

Detail panel and layout (since 0.4.5). A click on a row, its name or its state opens a detail panel, like the one of VMs & containers; ?job=<id> in the address opens it, so a link to a job’s panel works. It lists every state, the full error text of the newest failed run and the warnings of the newest run with warnings, each in a monospace block you can copy, what needs attention, the restore checks, the key facts, the newest runs and the actions. The list keeps its layout: fixed column widths, one row height, one compact State badge per row (the most urgent of what applies, green only for a passed restore check), the progress of a running backup beside it, one-line cells without tooltips and action buttons of one size. Columns that do not fit a screen give way from the right; the panel has everything.

The editor is a side sheet that works with the keyboard. For machine jobs it shows the folders as a tree read from the latest backup and offers exclusions as chips: videos, disk images, temporary files, installers, music, trash and caches, your own patterns, and “skip files larger than X GB”.

Bandwidth with time windows. A machine job keeps its upload limit in kbit/s as the default and can add up to 24 windows with another limit for certain days and hours, for example Monday to Friday 08:00 to 18:00 at 1000 kbit/s (0 means unlimited). A window may cross midnight; windows must not overlap. The times are read in the time zone of the job’s schedule, and the limit that applies when a run starts is used for the whole run. Since 0.4.0, a limit left empty means that the limit of the agent policy applies (unlimited if it sets none), and Unlimited uploads without a limit even when a policy sets one; only a limit that the installation locked still applies.

The agent policy in the job editor

The editor of a machine job has a section Agent policy (since 0.4.0): when the machines of the job may start backups (Backup window), whether missed backups are caught up (Catch up missed backups), Only on AC power, and the number of Automatic retries. Each setting shows the value that applies and where it comes from (installation, tenant or job); what the job leaves unset comes from the tenant and the installation, and a setting that the installation locked cannot be set here. The upload limit is the job’s own Bandwidth field above. See Agent policies.

Create a recommended job

Next to every notice that something is in no job (the jobs pages, the inventory, the sources, the directory, VMs & containers) there is Create recommended job, and the readiness card of the dashboard has Protect everything without a job (since 0.4.0). Both first show a preview of what the click creates or adds to, then set up one job per kind: a mail job every 8 hours, machine jobs per operating system and profile, and one Proxmox VE job, daily at 22:00 in snapshot mode. A second click adds to the same job instead of making another one. File share and copy jobs are not part of it. It is for administrators, and it is recorded in the audit log. Afterwards you change the jobs like any other.

With the mail archive (since 0.4.7). While the installation offers capture from the mailbox (archive.mailboxCapture, Business and Service Provider) and the tenant has mailboxes or accounts that an archive can capture, the recommendation also switches the archive on:

A new mail job over all objects backs up and archives. The preview says “Also archives every mail of all objects of this job, new ones included”.
A mail job over all objects that exists and does not archive yet is switched to archiving (the preview says “Switch archiving on in …”). Its schedules stay, and the change is audited as backup_job.updated with changes.archive, like an edit in the job editor.
The click takes the default capture ways, which need nothing set up in the tenant: Microsoft Graph for Microsoft 365 mailboxes and IMAP sync for the other accounts. Exchange journaling and forwarding stay a choice in the editor (How mail reaches the archive).
What it leaves as it is. It never changes a mail job over selected objects; the preview names them (“Archiving stays as it is in …”). A mail job over all objects whose capture ways were chosen by hand (Microsoft Graph or IMAP sync switched off, for example journaling only) is left as it is, and the preview says so (“Please switch it on in the job”). A new job over all objects is made only when it covers something that is in no job yet.
What the dialog says. It says when the job over all objects archives already (“Nothing to do: … archives already”), the confirmation names the jobs in which archiving was switched on apart from the jobs that were set up, and the preview of a paused job names no schedule.
The list of mail jobs shows Switch archiving on while the installation offers the capture and the recommendation would switch an archive on in the mail job over all objects. It does not for tenants with only OneDrives, SharePoint sites or groups, nor for a job whose capture ways were chosen by hand.
Without the capture (the Community build, or a tenant with nothing to capture) the recommended job is made as before.

For Proxmox VE, the recommended job leaves out guests that were excluded from backup on purpose (Excluding a guest from backup). Existing jobs change only when an administrator applies the recommendation, never by an update.

Start now and Stop (since 0.4.5)

Run now is called Start now since 0.4.5. Both job lists, Mail & SaaS → Jobs and Servers & clients → Jobs, have the button on every row and in the row’s … menu: one button of a fixed size that says Start now, Stop while the job’s backups run or wait, and Stopping… once asked.

Start now works per job or for selected members. A mail job starts at once. A machine starts a requested backup at its next check-in, about every 5 minutes; until then the job and its members show Queued with the expected check-in time. A Proxmox VE job queues a backup of every guest it covers, as its schedule would, without moving the schedule; guests that already have a backup waiting, are on a node without restow-pve or are the unconfirmed Restow server VM are left out and counted in the message.

Stop asks first and says what it means for the kind of job:

Mail job: every backup of the job that waits is withdrawn. Running backups stop at their next safe point and keep what they stored so far. The next scheduled run starts as usual.
Machine job: requested backups that have not started are withdrawn. Machines that back up right now stop restic cleanly with their next contact, within about a minute. Nothing of a stopped run is saved, and the runs end as cancelled, not as failed.
File share job (and copy job): runs that wait are withdrawn, and running ones are stopped by their runner. Nothing of a stopped run is saved, and it ends as cancelled, not as failed.
Proxmox VE job: backups of the job’s guests that wait are withdrawn, and running ones are stopped on their node. Nothing of a stopped backup is kept, and it ends as cancelled, not as failed. The next scheduled round runs as usual.

Both are for tenant administrators and for the provider roles that may operate; they are closed in the public demo. They are recorded in the audit log: backup_job.run_cancelled on the job (and per machine endpoint.backup.cancelled), and for Proxmox VE jobs pve.job.run_requested and pve.job.cancelled. The routes are listed under API reference. Stopping one guest, one machine or one file share is described under Proxmox VE, servers and clients and file shares.

Machines without a job (from 0.2.1)

VMs and containers of Proxmox VE (a preview) have backup jobs of their own, on the VMs & containers page and, since 0.4.0, also in Servers & clients → Jobs (see above): see Backup jobs and protection.

Servers & clients → Inventory shows a Backup job column with the job as a link, or the warning badge Without backup. A filter “Without backup / In a job” and a banner count the machines without a job. Job administrators get two actions on each row and on the machine page:

Create a job opens the job editor with the machine preselected.
Add to job… puts the machine into an existing job; if it is in another job, Restow asks before it moves it.

Back up now is disabled for a machine in no job, and the enroll dialog says that a new machine is backed up only in a job. The 0.2.1 agent waits as well: under the schedule none it starts no scheduled backup, ignores a backup request, and restow-agent status says it is waiting for a backup job.

Agents older than 0.2.1 do not know the schedule none. Until they update themselves (every 6 hours, or when the machine is next online), such a machine reports one failed no_paths run per slot of its old schedule, with a backup.failed alert each time; nothing is read or written. If you paused agent updates (Tenant settings → Agents), resume them or put the machine into a job.

Running backups and the live run view

Since 0.4.0, the Jobs entries and VMs & containers in the sidebar show a spinner with the number of running backups and the waiting ones beside it. The tooltip lists them, and a click opens the run.

A run opens in a live drawer. Its charts move with time between two measurements, the numbers count up, the time left counts down, and a line says when the run last reported, or that Microsoft Graph throttles it. While the drawer is open, the title of the browser tab shows the percent done. The live connection stays open while the tab is in the background (it closes after ten minutes, or when the browser freezes the tab), so a page no longer reloads when you come back. For a running Proxmox VE backup, a progress bar shows for example “34% · 3.2 GB of 9.4 GB · 85 MB/s”; it needs restow-pve 0.4.0 or newer on the node. The progress of a first backup in Microsoft 365 is described in First steps.

All tenants in Jobs

Under All tenants, the Jobs pages of Mail & SaaS and of Servers & clients list every tenant with its jobs per kind (mail, machines and Proxmox VE), what runs and waits, how the newest backups ended, and the next run. You can show All tenants, With running jobs, With failures or Without any job, and sort by name, by Needs attention first or by next run. A row opens the tenant’s jobs, read-only: switch to the tenant to change a job.

Several machines at once

Right-click a row, press the context menu key or Shift+F10, or use the ⋯ button: the machine, job and protected-object tables open their row actions there. Selectable tables have checkboxes and a selection bar; New job from selection puts several machines or mailboxes into one new job.

Assigning a machine to a person (from 0.2.1)

A machine can be assigned to a person of the tenant’s protection directory (a user of the directory, not a login account). The machine table has an Assigned to column, sortable and filterable by person and by “nobody”, and Assign to… searches the directory. The assignment is audited as endpoint.assigned and cleared when the person leaves the directory.

API

Mail, machine, file share and copy jobs: /api/v1/backup-jobs (kind mail, endpoint, share or copy). Proxmox VE jobs are not among them. Since 0.4.4 a mail job has archiveCapture, its members and candidates have captureKind, and the defaults name mailSourceKinds (How mail reaches the archive); a missing way for a kind answers 422 archive_capture_no_way.
POST /api/v1/schedules with kind backup or verify answers 422 kind_replaced_by_jobs. Maintenance schedules (retention run, storage check, directory sync, archive sync) are unchanged.
PATCH /api/v1/endpoints/:id answers 409 endpoint-config-managed-by-job when it would change folders, exclusions, hooks, bandwidth or the schedule of a machine in a job.
From 0.2.1, POST /api/v1/endpoints/:id/tasks with backup_now, and a schedule other than none, answer 409 urn:restow:problem:endpoint-no-job for a machine in no job. assignedUserId assigns a person.

Every change to a job is audited (backup_job.created, .updated, .deleted, .scope.changed, .run_requested, and since 0.4.5 .run_cancelled). See also Backups and schedules for runs, History and repositories.