Ransomware detection
When ransomware encrypts a file server, the next backup sees thousands of files change at once, files renamed to .locked, ransom notes in every folder and file contents that suddenly look like random bytes. Since 0.4.0 Restow judges every new restore point from these traces, marks the suspicious ones, alerts once per attack, points every restore view to the newest restore point without findings and pauses retention of the object until an administrator resolves the finding, so that clean old restore points are not pruned away while encrypted ones pile up.
What it judges, and when
Section titled “What it judges, and when”- Every new restore point, of machines (servers and clients), file shares, Proxmox VE containers and VMs, mailboxes, OneDrives, SharePoint sites, Microsoft 365 groups and IMAP accounts.
- Every two minutes, a worker judges every finished backup of the last 72 hours that has no judgement yet, oldest first. Runs older than 72 hours are not judged later, so an outage or an update does not flood you; the judgement logs such restore points as unjudged.
- From data the backup already has: the statistics of the run, the list of what changed against the previous restore point, file names and extensions, and a few small samples (the first and last 64 KiB of up to 8 new or changed files). It does not read the backed-up data a second time, never blocks a backup and does not scan for malware.
- Each object is compared with itself. The baseline is the object’s last 14 restore points that count as clean (judged without findings, called a false alarm, or made before the detection existed). A suspicious one that nobody called a false alarm stays out, so an attack that spans several backups does not teach the baseline that encryption is normal. An object backed up before the update is judged from the first pass on, because the baseline is read from the run history. A new backup chain begins a new baseline (since 0.4.6): when a container’s repository was missing or was reset (New backup chain and Reset repository), its first backup in the empty repository adds every file as new, which looks like an attack to the numbers of the old chain. Before 0.4.6 that first backup was judged against the old chain and flagged as suspicious; now the baseline begins with the newest chain, and that backup is called First backup, not Incremental.
The signals
Section titled “The signals”A restore point is suspicious when at least one signal fires. Every finding carries the numbers that made it (the measured value, the threshold it crossed, the object’s usual value), so you can see how far off it is. The defaults below are chosen so that ordinary days of ordinary objects stay quiet: a file server where a quarter of the files change every night has a high usual amount of change, and only a jump far beyond it alerts.
| Signal | Fires when (defaults) |
|---|---|
| Amount of change | New and changed files (or items), or bytes added, lie more than 8 median absolute deviations above the object’s median, are at least 5 times the median, and reach 500 files or 1 GiB |
| Deletions | At least 1,000 files deleted and at least 25 % of the files the previous restore point had |
| Renamed extensions | At least 100 files renamed with an appended or replaced extension (a.docx to a.docx.locked), at least 2 % of the files |
| Known names | At least 20 new or changed files carry a known ransomware extension, or at least 2 new files are named like a ransom note |
| Encrypted-looking files (entropy) | Of the sampled new or changed files that are not of a compressed format, at least 3 and at least half look encrypted (7.9 bits per byte or more) |
| Compression and deduplication | The run added at least 256 MiB of new data, and the new data barely compresses (stored share 0.95 or more) or barely deduplicates (new share 0.8 or more), clearly above the object’s usual value |
Some details:
- Amount of change says Still learning until the object has at least 5 clean restore points. The bound is the larger of the median plus 8 spreads, 5 times the median and the floor. The median and the spread are robust: one large monthly job in the history does not move them the way a mean would.
- Known names use versioned lists, matched on the server, so a list update needs no agent update. The extension list holds about 100 extensions of known families and leaves out extensions that ordinary software uses (such as
.enc). A ransom note is a fixed name or a text or web page whose name pairs an action word (decrypt,restore) with a subject word (readme,files). Notes are dropped into every folder, so the default of two copies keeps your ownhow_to_restore.txtfrom alerting. A name missing from the lists never makes anything clean. - Entropy: encrypted data looks like random bytes. The agent or runner measures the first and the last 64 KiB of a handful of new or changed files and nothing else of them. Compressed or encrypted formats (ZIP and Office files, JPEG, PNG, video, PDF, disk images, KeePass) are never judged, because their entropy says nothing.
- Compression and deduplication: encrypted data neither compresses nor deduplicates. Both ratios of the new data are judged against the object’s own history, so an object whose new data is always incompressible (media files, a VM with BitLocker or LUKS inside) never alerts on this signal. Its amount of change is still judged. Restow’s own chunk store does not compress, so for mailboxes, OneDrives, SharePoint sites, IMAP accounts and VM disks the chunk writer estimates how well the new data would compress by deflating the first 16 KiB of every new chunk.
Which objects have which signals
Section titled “Which objects have which signals”| Object | Amount of change | Deletions, renames, names, entropy | Compression and deduplication |
|---|---|---|---|
| Machine (server, client) | Yes | Needs the agent 0.4.0 or newer | Needs the agent 0.4.0 or newer for compression |
| File share | Yes | Needs the 0.4.0 runner (comes with the server) | Yes |
| Proxmox VE container | Bytes added; files from nodes with restow-pve 0.4.0 | Not available | Needs restow-pve 0.4.0 or newer |
| Proxmox VE VM | Bytes uploaded (changed blocks) | Not available | Yes, judged on the server |
| OneDrive | Items and bytes | Yes | Yes |
| SharePoint site | Items and bytes | Not available | Yes |
| Mailbox, IMAP account | Items and bytes | Not available | Yes |
| Microsoft 365 group (since 0.4.5) | Items and bytes | Not available | Yes |
A signal without data says Not available, never “clean”. That is the case for an older agent that sends no activity report, for a first backup (there is nothing to compare with), for a previous restore point that retention removed and for kinds of objects that have no such data. The reasons dialog of a restore point lists every signal with its state: finding, no finding, still learning, not available or switched off.
Where you see it
Section titled “Where you see it”- Suspicious badge (red) on the restore point and on the object in every restore view: the machine and file share pages, the guest page of a VM or container, the file restore, the explorer of mailboxes, OneDrives and SharePoint sites. A restore point without findings gets no badge, because no finding is no proof.
- The newest restore point without findings. When the newest restore point is suspicious, the restore views say since when restore points show signs of ransomware and offer Use the last clean restore point. A restore point made before the detection existed counts as without findings.
- What a click shows. The badge opens the reasons: each finding with its numbers, the state of every signal and what the detection cannot see.
Marking a false alarm
Section titled “Marking a false alarm”A planned migration of the project folders looks like an attack to the numbers. Mark as false alarm, with an optional note, tells Restow that the restore point is fine: it counts as clean, its numbers join the baseline (so the same amount of change no longer alerts) and the episode ends. The next finding is announced again. Undo the false alarm takes it back.
Provider technicians and above and tenant administrators can mark and undo it. Both are audited (suspicion.false_alarm, suspicion.false_alarm_revoked).
A restore point flagged before the update to 0.4.6 stays flagged. If the flag came from the first backup of a container’s new backup chain, mark it as a false alarm; while the retention hold is on, retention of the container waits until you do.
The retention hold
Section titled “The retention hold”The real damage is not the encrypted restore point. It is retention removing the clean old ones while encrypted ones accumulate: after a week of daily encrypted backups and seven daily restore points kept, nothing clean is left. So while an object has a suspicious restore point nobody resolved, retention, restic forget and prune delete no restore point of that object, and with them nothing they reference. The object shows Retention paused and a banner “Retention paused: suspicious restore point since …”.
- What goes on. Backups of the object continue, so its storage grows, and storage quotas and their warnings still apply: a backup that a used-up quota refuses fails as usual. Retention of other objects goes on; the shared chunk store only collects chunks that nothing references, so a held restore point keeps every chunk it needs.
- When it holds. A restore point holds while it is judged suspicious, is not called a false alarm and its hold was not released. While the detection is on, a backup of the object from the last six hours that has not been judged yet holds as well: a backup the pass cannot judge stops retention for six hours at most, not for days.
- When it ends. Only an administrator ends a hold, never a later clean restore point: ransomware that was cleaned up still leaves its encrypted restore points, and an attacker who stops encrypting for a day must not restart pruning. Two ways:
- Mark as false alarm: every finding that holds it counts as clean.
- Release hold: the findings stay marked, retention resumes with its next run and deletes what its rules no longer keep, suspicious restore points included. Release the hold only once the system is clean. This is for administrators (provider administrators from the administrator role up, and tenant administrators) with a sign-in from the last 10 minutes, and it is audited (
suspicion.hold_released, with every restore point it released and the note).
- It never goes on silently. A hold still open after 14 days is announced again with a warning, and every 14 days after that, even when the detection is switched off.
Releasing a hold also ends the episode, so the next finding is announced and holds again. If the repository is immutable, the lock also keeps the history from being deleted by someone else; the hold only stops Restow’s own retention.
Notifications
Section titled “Notifications”| Event | Severity | When |
|---|---|---|
Suspicious backup (signs of ransomware) (backup.suspicious) |
Error | Once per attack episode, for the first suspicious restore point after a clean one. Says that retention is paused. The mail lists every finding with its numbers |
Retention still paused after a suspicious backup (backup.suspicious_hold) |
Warning | When a hold is still open after 14 days, and every 14 days after. Says since when and how many restore points hold it |
An episode ends with a restore point without findings, a false alarm or the release of its hold. The next suspicious restore points of the same attack are marked but not announced again. Notification rules additionally throttle per object.
Both events reach the bell, the notification rules (mail) and the webhooks like every other event, as set in the notification matrix under Integrations → Notifications. See Notifications and reports.
Thresholds and switches
Section titled “Thresholds and switches”Installation → Ransomware detection holds the thresholds of every signal, each signal’s switch, the switch of the retention hold and its reminder days. The provider owner changes them, every provider administrator reads them. Every change is audited (settings.ransomware.updated). Back to the defaults resets them. A value outside its range is refused.
| Setting (as the page names it) | Default | Range |
|---|---|---|
| Check restore points for signs of ransomware | on | (off: nothing is judged) |
| Earlier clean restore points in the baseline | 14 | 3 to 60 |
| Restore points needed before the amount of change is judged | 5 | 2 to 30 |
| Amount of change: Deviations above the median | 8 | 2 to 50 |
| Multiple of the median | 5 | 1.5 to 100 |
| Minimum of new or changed files | 500 | 10 to 10,000,000 |
| Minimum added | 1 GiB | 1 MiB to 10 TiB |
| Mass deletions: Minimum of deleted files | 1,000 | 10 to 10,000,000 |
| Minimum share of the files before | 25 % | 1 to 100 % |
| Renamed extensions: Minimum of renamed files | 100 | 5 to 10,000,000 |
| Minimum share of the files | 2 % | 0.1 to 100 % |
| Known names: Files with a ransomware extension | 20 | 1 to 1,000,000 |
| Copies of a ransom note | 2 | 1 to 1,000 |
| Encrypted-looking files: Minimum of encrypted-looking samples | 3 | 1 to 16 |
| Entropy from (bits per byte) | 7.9 | 7 to 8 |
| Minimum share of the samples | 50 % | 10 to 100 % |
| Compression and deduplication: Minimum of new data | 256 MiB | 1 MiB to 10 TiB |
| Deviations above the median | 6 | 2 to 50 |
| Stored share of the new data from | 95 % | 50 to 100 % |
| Above the usual stored share by at least | 15 percentage points | 1 to 90 |
| New share of the data read from | 80 % | 10 to 100 % |
| Above the usual new share by at least | 30 percentage points | 1 to 90 |
| Retention hold: Pause retention of objects with a suspicious restore point | on | (off: nothing holds retention) |
| Announce an unresolved hold again after (days) | 14 | 1 to 365 |
Every signal can also be switched off on its own (the Check switch of its group).
After the update to 0.4.0
Section titled “After the update to 0.4.0”Ransomware detection and its retention hold are both on, and the detection judges the backups of the last 72 hours. A suspicious restore point therefore pauses retention of its object until an administrator resolves it. Thresholds and both switches are under Installation → Ransomware detection, and every provider administrator can read them. An object with fewer than 5 clean restore points says Still learning for the amount of change.
What it cannot detect
Section titled “What it cannot detect”- Slow or partial encryption. Ransomware that encrypts a few files per day, or only parts of large files, stays within the baseline. Intermittent encryption that leaves the first and last 64 KiB of a file intact is invisible to the entropy samples.
- Encryption without a trace in names (same extension, no note): only the amount of change, the entropy and the compression and deduplication of the new data can see it.
- Small or mixed runs. Below 256 MiB of new data the ratios are not judged, and encrypted data mixed into a large amount of ordinary new data moves them only partly.
- Data the backup does not see: excluded folders, files above a size limit, data changed after the backup, VMs (only the amount of changed blocks), mailboxes (only the amount of new items).
- An attacker who knows the detection can stay below the thresholds, encrypt into compressed formats (never judged by the entropy samples) or encrypt the backup history first. Immutable storage protects the history. The retention hold only stops Restow’s own retention from deleting it, and an administrator account that the attacker holds can release it (it needs a recent sign-in and is audited).
- Malware that does not encrypt (data theft, backdoors) leaves no trace in these signals.
- The first backup and objects with a short history have nothing to compare with.
Known limits in 0.4.7
Section titled “Known limits in 0.4.7”- The signals and defaults are built from the published behaviour of ransomware families and were tested with simulated attacks, not with live ransomware. Treat the defaults as a starting point and adjust them to your objects.
- The judgement reads new restore points batch after batch for up to a minute per pass. On very large installations a restore point that leaves its 72-hour window before a pass reaches it stays unjudged and is logged as such.
- The known issues of 0.4.0 apply as a whole, see the roadmap.
Where to read more
Section titled “Where to read more”- Immutable backups: keep the history from being deleted.
- Notifications and reports: who gets the two events.
- Backups and schedules: retention, the daily run that the hold pauses.
- How Restow checks that a backup can be restored: the restore check, which is a different thing from this detection.