Mail file import and export

Some mail has to be kept although the mailbox it came from no longer exists: a former employee, a closed domain, an old mail server. Import takes such mail from files into Restow, where it can be browsed, previewed and restored like any backup. Export does the opposite: it writes mail that Restow holds back out as files you can hand to someone else or open in another program.

What can be imported

Source	What is read
EML	Single files, folders of files, or a ZIP of them.
MSG (Outlook)	Single files, folders, or a ZIP. Mail messages only: contacts, appointments and tasks stored in MSG form are reported as “not a mail message” and are not imported.
MBOX	Thunderbird (including `.sbd` folders), Apple Mail (`X.mbox/mbox`), Dovecot and Postfix exports, Google Takeout. Both CRLF and LF line endings work, and mboxrd quoting is undone.
ZIP	The folder structure inside the ZIP becomes the folder structure of the mailbox. It may contain EML, MSG and MBOX files.
MailStore	An export as an EML or MSG folder tree (the folder, or a ZIP of it).

Restow decides what a file is by its content (the first 64 KiB), never by its extension. Only mail is imported.

MailStore’s own internal archive format cannot be read. In MailStore, use the export to the file system and choose EML or MSG, then import that folder or a ZIP of it.

What is not supported

PST and OST files. They are recognised by their content and refused with a clear message, never parsed. PST and OST import is planned for a later release. Until then, export the mailbox from Outlook as MSG or EML files (drag the messages into a folder) or convert it to MBOX with a converter, and import that.
PST export and MSG export. Both are planned. There is no MSG writer with a clear, portable license yet, so the export dialog does not offer MSG, and PST is shown as planned. An EML export holds the same messages in their original form: Outlook opens EML files by drag and drop, and MBOX can be imported with common tools.
Calendar and contacts. Only mail is imported and exported.
Other containers. gz, 7z, rar, tar and Apple .emlx files are recognised and reported as not supported, not opened. Leftovers of operating systems (.DS_Store, Thumbs.db, desktop.ini, __MACOSX, Thunderbird .msf) are listed as “not a mail”, never as errors.

What an import produces

An import creates or extends an imported mailbox. It lives under one source per tenant named Imported mail files, which has no server and no credentials: nothing is ever backed up from it, so there is no connection to set up or test. It does not appear in backup schedules, and it never shows up as unprotected or overdue.

Same format as an IMAP backup. Each message is stored as its own EML in the encrypted chunk store, with the folder structure kept. Because of that, the restore explorer, preview, print view, download and IMAP restore work for imported mail without special cases, and the mailbox carries an “Imported” badge there.
Byte for byte, except MSG. EML and MBOX content is stored exactly as it is in the file. An MSG file contains no RFC 5322 message, so Restow reconstructs an EML from its stored properties: headers (when Outlook stored them), text, HTML, attachments and embedded messages. The report counts these messages as “rebuilt from MSG”.
Every import adds a snapshot. A new snapshot holds everything from the previous one plus the new messages. Nothing is ever removed, so you can build one legacy mailbox from several files, one after the other.

Folders

A loose a.eml or a.msg lands in the folder Imported.
Directories become folders (dir/sub/a.eml goes to dir/sub). Empty directories stay as empty folders.
An MBOX file x.mbox, or one called Inbox without an extension (recognised by content), becomes the folder x or Inbox. Thunderbird’s Inbox.sbd/ directories become subfolders of Inbox. An Apple Mail X.mbox/mbox becomes X.
In a ZIP, the path structure decides the folders. Entries without a directory go into a folder named after the ZIP file.

Getting the files in

Open Sources → Add source → Import mail files. There are two ways in.

Upload in the browser

Chunked and resumable. The file is sent in pieces (8 MiB by default), three at a time, each with its own SHA-256 checksum. If the network drops or the tab is closed, only the pieces in flight are lost. Choose the same file again (recognised by name and size) and only the missing pieces are sent.
Sealed from the first byte. Every piece is encrypted with the tenant key (AES-256-GCM) the moment it arrives and stored as its own object in the tenant’s primary storage target. The worker later reads the file from these pieces. No plaintext copy of the file exists on any disk, and a whole file is never held in memory, at most one piece or one single message.
Limits. IMPORT_MAX_FILE_BYTES (10 GiB per file by default), at most 20 open uploads per tenant, and an unfinished or unused upload expires after 48 hours (IMPORT_UPLOAD_TTL_HOURS). Cancelling an upload deletes its pieces at once. Folders cannot be uploaded: put a folder into a ZIP first.
Clean-up. The pieces are removed right after the import, on cancel and on expiry. The retention run clears any leftovers.

Server-side import folder

For very large archives that you do not want to send through the browser, put the files on the Restow server.

RESTOW_IMPORT_DIR is a directory on the host. The release stack mounts it read-only at /var/lib/restow/import in both the api and the worker container.
Each tenant has its own subfolder, <RESTOW_IMPORT_DIR>/<tenant slug>/, even an installation with a single tenant. The slug is in the tenant settings and the import page shows the full path. You create the subfolder yourself, and the directory must be readable for the user the containers run as.
Put files or whole folder trees there (for example a MailStore export) and select them in the interface.
Restow never writes to this folder and never empties it. Paths are checked against escapes (.., symbolic links pointing outside, special files), and both the API and the worker check that a path lies below the tenant’s own subfolder, so one tenant cannot see or import what another tenant put there.

The import job

Choose the source, the files and the target (a new imported mailbox with a name, or an existing one), then Start import. The import runs in the background, so you can leave the page and follow it under Imports.

Progress. Bytes read of the total (with percent and time left), counters for messages, duplicates, skipped and failed items, and the current phase.
Checkpoints. After every finished file and every 500 messages (or 256 MiB), the job saves a checkpoint. A restarted worker continues from there and never reads a message twice. If the list of files changes, the snapshot starts over.
Duplicates. A message with the same Message-ID and the same SHA-256 in the same folder is a duplicate: it is not stored again, but counted and listed with where it was found. The same message in a different folder is kept, because the folder is information, and shares its stored chunks through deduplication. Earlier imports into the same mailbox count too, so importing the same file twice gives “nothing new” the second time and no snapshot.
Failures are listed. Unreadable or damaged items (a broken MSG or EML, a truncated ZIP, a message over the size limit, an encrypted or nested ZIP) are listed with the reason and each explained in your language, see Failure explanations. Harmless things (files that are not mail, empty files, duplicates) are listed as skipped with their location. Nothing drops out silently.
Report. Messages, folders, attachments, duplicates, skipped, failed and bytes (stored and read). Per file: the format, the SHA-256 of the source file, its counts and its result (imported, partly imported, failed, not imported). Below that the list of items, failures first, at most 1,000 with the rest counted, and fixed notes such as “calendar and contacts are not imported” and “rebuilt from MSG”. The report can be downloaded as JSON.
Nothing readable, nothing stored. If a run finds not a single readable message, no empty snapshot is created. The job fails without retrying, and the report with every reason stays available.

Starting an import and uploading files needs at least the Administrator role, cancelling a running import needs Technician. See Team and roles.

Also archive (optional)

If you tick also ingest into the archive, every newly imported message becomes an archive object as well: the original bytes (the same chunks as in the snapshot), a place in the hash chain, the retention rules of the archive, and full-text search over subject, text and addresses. See Archiving.

Retention starts at the import date, not at the date of the mail. An old mail imported today is unalterable from today and counted from today. The sent date of the mail is kept and used in search and display.
Immutability applies from the moment of ingest, the same as for mail captured by the Graph sync.
Ingest is idempotent: what is already archived (same hash for the same mailbox) is not created twice, and a repeated job continues.

Restoring from an imported mailbox

An imported mailbox appears in the restore explorer like any other: folder tree, search, preview, print view and download. You can restore it into an existing IMAP account (appended with flags and date, with a Message-ID check) or into a Microsoft 365 mailbox, where the folder structure goes under a new restore folder. Existing items are never replaced. There is no “restore to original”, because an imported mailbox has no original.

Export

Start an export from the restore explorer (Export …) or from the archive, for a whole mailbox, a folder, a selection, or every result of a search filter. The sources are:

a backup (a Microsoft 365 mailbox or an IMAP account),
an imported mailbox,
the archive, as a selection or a search filter.

Format	What you get
EML in a ZIP	One `.eml` per message with the folder structure (empty folders stay), plus `MANIFEST.csv` (entry, SHA-256, size, Message-ID, date, sender, recipients, subject, status) and `SHA256SUMS`. The bytes are the stored originals.
MBOX (mboxrd)	A single folder becomes one `.mbox` file. Several folders become a ZIP with one `.mbox` per folder, plus the manifest and checksums. A message without a final line break gets one.

SHA256SUMS is in the format sha256sum reads. Unpack the ZIP and check every message and the manifest:

sha256sum -c SHA256SUMS

Only mail is exported. Calendar items and contacts are counted and named in the report, not exported. Messages that Microsoft Graph delivered only in parts have no original file and are listed as not exportable.

How it runs. A worker job builds the file and writes it encrypted, in the same sealed pieces as an upload, into the tenant’s storage target. Memory holds at most one message and one piece. While streaming, the SHA-256 of every message is checked against the manifest, so a damaged chunk stops the export instead of delivering a silently wrong file. The finished file can be downloaded for 24 hours (EXPORT_TTL_HOURS), then the retention run deletes it. The SHA-256 of the finished file is shown in the interface and recorded in the audit entry.

Audit. Requesting an export and every download are written to the audit log as export.requested and export.downloaded, and cancelling as export.cancelled. For someone else’s data, the entry carries your reason and who it was done on behalf of. See Audit log. Requesting and downloading an export needs at least the Technician role.

Security limits

Detection by content. A file is never trusted by its name.
ZIP limits. At most 2 million entries, 256 GiB unpacked and a compression ratio of 1,000. Encrypted and nested archives are reported and not opened. Entries are never written to a disk, and names are cleaned (.., absolute paths, backslashes).
One message at a time. A single message (an EML, an MSG, one message of an MBOX) is read into memory, limited by IMPORT_MAX_MESSAGE_BYTES (256 MiB). Larger ones are reported.
Everything is audited: import.upload.created, import.upload.completed, import.upload.cancelled, import.requested, import.cancelled, and the three export events above.
Storage need. Staging takes the size of the files until the import ends, then the messages are stored deduplicated in the chunk store. Exports take space until they expire.

Known limits

An imported mailbox cannot be deleted in 0.1.0, and neither can the Imported mail files source while it holds snapshots, archive objects or legal holds. Older snapshots are cleared by the backup retention, the newest one stays.
MSG. A body that exists only as RTF (or RTF-wrapped HTML) is not decoded, and Restow then takes the text body. Attachments that only point to a file are dropped. An embedded item that is not mail stays an .msg attachment. Headers come unchanged from the transport headers stored in the MSG, when there are any, so a subject changed later in Outlook is not reflected there. The MSG reader runs under a time limit and reports a hit as unreadable.
MBOX. The mboxrd variant is undone (a leading > before From lines). For mboxo files that already contained >From in the original, the two cannot be told apart. \Deleted from X-Status is not carried over: the message is imported.
UTF-16 EML files are not recognised as EML.
Metadata (subject, sender, full text) is read with a mail parser, which takes correspondingly longer for very large messages. The import stores the bytes unchanged regardless.
Changing the primary storage target does not copy staging or export files, because they are temporary. An unfinished upload has to be repeated afterwards.

Configuration

Set these in .env (see Environment variables). An empty value uses the default.

Variable	Meaning	Default
`RESTOW_IMPORT_DIR`	Host directory mounted read-only at `/var/lib/restow/import` in `api` and `worker`. Each tenant uses the subfolder `<directory>/<tenant slug>/`, which you create.	`./import` next to `docker-compose.yml`
`IMPORT_DIR`	Path inside the containers. Change it only together with the mount.	`/var/lib/restow/import`
`IMPORT_MAX_FILE_BYTES`	Largest file the upload accepts, in bytes.	10 GiB
`IMPORT_UPLOAD_TTL_HOURS`	How long an unfinished or unused upload is kept.	48
`IMPORT_SEGMENT_BYTES`	Size of one upload piece, in bytes (allowed 64 KiB to 32 MiB).	8 MiB
`IMPORT_MAX_MESSAGE_BYTES`	Largest single message that is read into memory.	256 MiB
`EXPORT_TTL_HOURS`	Hours a finished export can be downloaded before its file is deleted.	24

Planned

PST and OST import, PST export, MSG export (once a writer with a clear license exists), calendar and contacts from MSG and PST, reading MailStore’s own archive format (if it is openly documented), and password-protected ZIP files. See the roadmap.