Skip to main content

Restic — Emergency Interim Backup: Deployment & Restore (T3 Only)

T3 / CODE COMMANDERS ONLY — DO NOT ATTEMPT BELOW T3

This procedure deploys a non-standard backup tool into a production client environment, handles client-encryption keys that have no recovery path, and touches domain controllers and PHI-bearing volumes. A mistake here produces a backup that appears to work and cannot be restored from.

If you are T1 or T2 and a client has no working backup: escalate to T3 immediately. Do not attempt this page.

Audience: T3 / Code Commanders
Use when: NinjaOne Backup has stopped capturing both the local and cloud legs on a host, the root cause is not resolvable within the current maintenance window, and the host is left with no backup coverage.
Origin: Halo 1170038 — Lockhart clone-mode deactivation left a domain controller with 47 days of no backup and no onsite BDR.

1. Purpose & Scope

Restic is a break-glass bridge, not a backup platform. It exists to stop the bleeding on a host that currently has nothing, while the underlying NinjaOne fault is worked with the vendor. DTC's standard remains NinjaOne Hybrid Cloud Backup per page 1421.

Deploy restic only when all of the following are true:

  • NinjaOne Backup is producing no usable local and no usable cloud copy on the host
  • No Veeam BDR or other secondary backup covers the host
  • The fault cannot be resolved today — vendor case open, or remediation gated on a maintenance window
  • The host holds data whose loss would be materially damaging to the client

Do not deploy restic when:

  • NinjaOne is failing only the cloud leg and the local NAS copy is current — that is a degraded state, not an outage
  • A Veeam BDR at the site is protecting the host
  • The NinjaOne fault has a known same-day fix (see the NinjaOne error pages in this book)
  • You have not yet opened a NinjaOne support case

Every restic deployment carries an obligation to pressure the vendor. A NinjaOne case must be open before deployment and must be actively pursued afterwards. Restic is not a resting place. Review the deployment against its exit criteria at least weekly (Section 12).

2. What This Is Not — Read Before Deploying

These limitations are not caveats. They change what recovery is possible, and the client's risk position must be understood in these terms.

Limitation

Consequence

File-level, not image-level

No bare-metal recovery. Total loss of the host requires a full OS rebuild, role reinstall, and LOB application reinstall before any data can be restored. Recovery time is measured in days, not hours.

Not a supported AD restore path

VSS produces consistent NTDS files, but Microsoft's supported domain controller recovery is System State or BMR restored into DSRM. On a multi-DC domain, restoring AD files over a live DC risks USN rollback and silent replication corruption.

Unmonitored

No RMM visibility, no alerting, no integration with backup health checks. Silent failure surfaces nowhere. Verification is manual and is your responsibility.

Encryption key has no recovery

Losing the repository password renders every snapshot permanently unrecoverable. There is no reset, no escrow, and no vendor recovery.

Unverified until restore-tested

A completed backup is not confirmed recovery capability. Section 9 is mandatory, not optional.

3. Architecture

Two repositories, same encryption password, one client:

[ <SERVER01> ]
      |
      | restic backup --use-fs-snapshot  (VSS, nightly)
      v
[ \\<NAS>\backups\restic ]           <-- primary, fast restore, on-site
      |
      | restic copy  (deduplicated block transfer, nightly)
      v
[ s3:s3.<REGION>.backblazeb2.com/<BUCKET> ]   <-- offsite, survives site loss

Why copy rather than back up twice: the NAS repository is already deduplicated and compressed. restic copy transfers only blocks the destination lacks, avoiding a second VSS snapshot and a second full read of the source volumes. On a production server that difference is hours of avoided I/O.

Why both: the emergency this page addresses is NinjaOne failing to capture local and cloud. A NAS-only interim repeats that single point of failure — a site-level event would take the interim backup with it.

Backblaze B2 via the S3-compatible API, not the native B2 backend. Restic's own documentation recommends the S3 path because of error-handling deficiencies in the B2 library restic uses. Repository strings are therefore s3:, never b2:.

4. Prerequisites

  • Elevated PowerShell on the target host (VSS snapshot creation fails without it)
  • 64-bit restic binary on a 64-bit OS — restic checks that its architecture matches the OS before using VSS
  • Site NAS reachable on TCP 445 with writable share
  • NAS service account credentials from IT Glue under the site's NAS asset
  • Backblaze B2 console access
  • Free space on the NAS destination — check before committing (Section 5.3)
  • Open NinjaOne support case

NinjaOne will not give you the NAS password. The credentialId in a Lockhart policy is an internal reference, not a retrievable secret. Do not waste time in the NinjaOne console — go to IT Glue.

5. Deployment

5.1 — Install restic

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$dir = "C:\Program Files\restic"
New-Item -ItemType Directory -Path $dir -Force | Out-Null

$rel   = Invoke-RestMethod "https://api.github.com/repos/restic/restic/releases/latest"
$asset = $rel.assets | Where-Object { $_.name -like "*windows_amd64.zip" } | Select-Object -First 1
"Release: $($rel.tag_name)   Asset: $($asset.name)"

Invoke-WebRequest -Uri $asset.browser_download_url -OutFile "$env:TEMP\restic.zip"
Expand-Archive "$env:TEMP\restic.zip" -DestinationPath "$env:TEMP\resticx" -Force
Get-ChildItem "$env:TEMP\resticx" -Filter *.exe | Move-Item -Destination "$dir\restic.exe" -Force
Remove-Item "$env:TEMP\restic.zip","$env:TEMP\resticx" -Recurse -Force

& "$dir\restic.exe" version

Confirm the output reads windows/amd64. Record the version — it goes in the tracking sheet and the ticket.

5.2 — Generate and protect the repository password

$cfg = "C:\ProgramData\restic"
New-Item -ItemType Directory -Path $cfg -Force | Out-Null
New-Item -ItemType Directory -Path "$cfg\logs" -Force | Out-Null

$bytes = New-Object byte[] 32
[Security.Cryptography.RandomNumberGenerator]::Create().GetBytes($bytes)
$pw = [Convert]::ToBase64String($bytes)

[IO.File]::WriteAllText("$cfg\repo.pw", $pw, (New-Object Text.UTF8Encoding($false)))

icacls "$cfg\repo.pw" /inheritance:r /grant:r "SYSTEM:(F)" "BUILTIN\Administrators:(F)" | Out-Null
icacls "$cfg\repo.pw"

Write-Output ""
Write-Output "REPO PASSWORD - store in IT Glue now:"
Write-Output $pw

STOP. Put this password in IT Glue before running anything else. Once data is written, losing it means losing every snapshot in both repositories. There is no recovery path. Store under the client's asset, labelled with the hostname.

The same password is used for both repositories — one secret per client, one thing to get right during an incident. The trade-off is accepted: anyone who can read the ACL-protected file on the host can also read the B2 key stored alongside it.

5.3 — Initialise the NAS repository

$env:RESTIC_REPOSITORY    = "\\<NAS>\backups\restic"
$env:RESTIC_PASSWORD_FILE = "C:\ProgramData\restic\repo.pw"

Test-Path "\\<NAS>\backups"
Get-PSDrive C,D | Select-Object Name,
    @{N='UsedGB';E={[math]::Round($_.Used/1GB,1)}},
    @{N='FreeGB';E={[math]::Round($_.Free/1GB,1)}}

$fso = New-Object -ComObject Scripting.FileSystemObject
$d = $fso.GetDrive("\\<NAS>\backups")
[PSCustomObject]@{
    FreeGB  = [math]::Round($d.FreeSpace/1GB,1)
    TotalGB = [math]::Round($d.TotalSize/1GB,1)
}

& "C:\Program Files\restic\restic.exe" init

Space gate: restic deduplicates and compresses, so consumption lands well below raw source size — but do not proceed on a destination with less free space than roughly 60% of source used capacity without a deliberate decision. Record the pre-deployment free space; it goes in the tracking sheet.

Record the repository ID from the init output.

Note: Get-PSDrive against a UNC root does not populate Used/Free. The FileSystemObject COM method above is required for network destinations.

5.4 — Exclude file

$cfg = "C:\ProgramData\restic"

@'
# --- Volatile system files (cannot be restored, change constantly) ---
C:\pagefile.sys
C:\hiberfil.sys
C:\swapfile.sys
D:\pagefile.sys
D:\hiberfil.sys
D:\swapfile.sys

# --- Windows internals ---
C:\System Volume Information
D:\System Volume Information
C:\$RECYCLE.BIN
D:\$RECYCLE.BIN
C:\Recovery
C:\$WinREAgent
C:\Windows\CSC
C:\Windows\Prefetch

# --- Temp and caches ---
C:\Windows\Temp
C:\Windows\SoftwareDistribution\Download
C:\Users\*\AppData\Local\Temp
C:\Users\*\AppData\Local\CrashDumps
C:\Users\*\AppData\Local\Microsoft\Windows\INetCache
C:\Users\*\AppData\Local\Microsoft\Windows\WebCache

# --- Crash dumps ---
C:\Windows\MEMORY.DMP
C:\Windows\Minidump

# --- Restic's own password file (lives in IT Glue) ---
C:\ProgramData\restic\repo.pw
'@ | Set-Content "$cfg\excludes.txt" -Encoding UTF8

Get-Content "$cfg\excludes.txt" | Measure-Object -Line

Deliberately NOT excluded, and do not add them:

  • C:\Windows\NTDS and C:\Windows\SYSVOL — the AD database and policy store. Excluding these makes the backup worthless on a domain controller.
  • C:\Windows\WinSxS — bulky, but restic deduplicates it and removing it breaks any OS repair path.
  • Program Files and Program Files (x86) — LOB applications frequently store working data and configuration inside their install directories.

Adjust volume letters to the host. Enumerate fixed disks first — the drive admin shares (C$, D$, …) from Get-SmbShare are a reliable cross-check that no root volume has been missed.

5.5 — First full backup

$env:RESTIC_REPOSITORY    = "\\<NAS>\backups\restic"
$env:RESTIC_PASSWORD_FILE = "C:\ProgramData\restic\repo.pw"

& "C:\Program Files\restic\restic.exe" backup C:\ D:\ `
    --use-fs-snapshot `
    --exclude-file "C:\ProgramData\restic\excludes.txt" `
    --tag interim-<TICKET>

Flag / behaviour

Why

--use-fs-snapshot

Creates a VSS snapshot per volume and reads from it, allowing capture of files exclusively locked by another process — databases, NTDS, LOB data stores. Without this the backup is worthless.

--verbose

 omitted

Deliberate. Verbose streams every filename processed, which on a clinical file server means PHI-bearing paths in console output and logs. Omit it.

VSS timeout

Default is 120 seconds. If snapshot creation fails on a busy server, add 

-o vss.timeout=300s

. Do not add it pre-emptively.

Session persistence

Restic runs in the foreground and dies with its parent shell. If the RDP session drops, the run dies. It is resumable — rerun the identical command and it continues from existing packs.

Mid-run state

Restic writes the snapshot object last. During a run, 

restic snapshots

 legitimately returns zero. An empty repository mid-run is 

not

 evidence of failure.

Expect hours on a first full over SMB. Record the summary output — files processed, data added, duration, snapshot ID.

5.6 — Backblaze B2 bucket and key

Performed in the Backblaze B2 web console, not S3 Browser — application keys cannot be created in S3 Browser, and a key is required before any S3 client can connect.

Bucket — B2 console → Buckets → Create a Bucket:

  • Private. Never place a restic repository in a public bucket; repository structure, snapshot counts, sizes and timing would be world-readable even though contents are encrypted.
  • One bucket per client. A key scoped to a single bucket cannot reach another client's data — this isolation is the point.
  • Object Lock disabled unless there is a specific requirement
  • SSE-B2 default encryption is optional — restic already encrypts client-side, so this is a second layer, not the primary one
  • B2 bucket names are globally unique across all of Backblaze. A bare client name will collide; prefix it.
  • Record the Endpoint shown on the bucket page (e.g. s3.us-west-002.backblazeb2.com). The digits vary by account. A wrong endpoint is the most common cause of "bucket not found".

Application key — B2 console → Application Keys → Add a New Application Key:

Field

Value

Name of Key

RESTIC-<CLIENT>

 — letters, numbers and hyphens only

Allow access to Bucket(s)

The client's bucket. 

Not "All".

Type of Access

Read and Write.

 Restic reads the repository index before every write — Read Only cannot back up.

Allow List All Bucket Names

Ticked.

 Required for the S3 List Buckets operation, which restic's S3 backend performs.

File name prefix

Blank — dedicated bucket, no prefix needed

Duration (seconds)

Blank.

 A duration sets the key to expire, and when it does the backup stops silently. Never set one.

The application key is displayed exactly once. Copy it to IT Glue before leaving the page. The keyID is retrievable later; the secret is not. If lost, delete the key and create a replacement.

The Master Application Key does not work with the S3-compatible API. It must be a purpose-created key.

5.7 — Initialise the B2 repository and seed it

$R = "C:\Program Files\restic\restic.exe"
$env:RESTIC_PASSWORD_FILE = "C:\ProgramData\restic\repo.pw"

# Confirm the NAS repo holds a completed snapshot before proceeding
$env:RESTIC_REPOSITORY = "\\<NAS>\backups\restic"
& $R snapshots
& $R stats --mode raw-data

# Initialise B2, matching chunker parameters to the NAS repository
$env:AWS_ACCESS_KEY_ID     = "<keyID>"
$env:AWS_SECRET_ACCESS_KEY = "<applicationKey>"
$env:RESTIC_REPOSITORY     = "s3:s3.<REGION>.backblazeb2.com/<BUCKET>"

& $R init --from-repo "\\<NAS>\backups\restic" --copy-chunker-params

# Seed the offsite copy
& $R copy --from-repo "\\<NAS>\backups\restic"

--copy-chunker-params is mandatory. Without it the two repositories chunk data differently and every subsequent restic copy re-uploads the entire dataset from scratch instead of transferring deduplicated blocks.

restic stats --mode raw-data against the NAS repository gives the actual upload volume for B2. Check it against the site's upstream bandwidth before committing to the seed — this is the decision point on whether the site can carry an offsite copy at all.

Repository URLs are path-style only (endpoint/bucket). Virtual-host style (bucket.endpoint) is not supported. If init fails with bucket-not-found, try the bucket name in lowercase — the B2 console displays mixed case but the S3 API path may not accept it.

6. Automation

6.1 — Environment file

$cfg = "C:\ProgramData\restic"

@'
$env:RESTIC_PASSWORD_FILE  = "C:\ProgramData\restic\repo.pw"
$env:RESTIC_REPO_NAS       = "\\<NAS>\backups\restic"
$env:RESTIC_REPO_B2        = "s3:s3.<REGION>.backblazeb2.com/<BUCKET>"
$env:AWS_ACCESS_KEY_ID     = "<keyID>"
$env:AWS_SECRET_ACCESS_KEY = "<applicationKey>"
$env:RESTIC_BIN            = "C:\Program Files\restic\restic.exe"
$env:RESTIC_EXCLUDES       = "C:\ProgramData\restic\excludes.txt"
$env:RESTIC_TAG            = "interim-<TICKET>"
'@ | Set-Content "$cfg\restic-env.ps1" -Encoding UTF8

icacls "$cfg\restic-env.ps1" /inheritance:r /grant:r "SYSTEM:(F)" "BUILTIN\Administrators:(F)" "<DOMAIN>\<NAS_SVC_ACCOUNT>:(R)" | Out-Null
icacls "$cfg\restic-env.ps1"

This file holds the B2 credentials in plaintext and must carry the same ACL discipline as the password file. The NAS service account needs read access so the scheduled task can source it.

6.2 — Nightly backup script

@'
$ErrorActionPreference = "Stop"
$log = "C:\ProgramData\restic\logs\nightly_{0}.log" -f (Get-Date -Format "yyyyMMdd_HHmmss")
Start-Transcript -Path $log -Force | Out-Null

# Rotate: keep 30 most recent
Get-ChildItem "C:\ProgramData\restic\logs" -Filter "nightly_*.log" |
    Sort-Object LastWriteTime -Descending | Select-Object -Skip 30 | Remove-Item -Force -EA SilentlyContinue

$rc = 0
try {
    . "C:\ProgramData\restic\restic-env.ps1"

    $env:RESTIC_REPOSITORY = $env:RESTIC_REPO_NAS
    & $env:RESTIC_BIN backup C:\ D:\ --use-fs-snapshot --exclude-file $env:RESTIC_EXCLUDES --tag $env:RESTIC_TAG
    $backupExit = $LASTEXITCODE
    Write-Output "BACKUP EXIT: $backupExit"
    if ($backupExit -ne 0) { $rc = 2 }

    $env:RESTIC_REPOSITORY = $env:RESTIC_REPO_B2
    & $env:RESTIC_BIN copy --from-repo $env:RESTIC_REPO_NAS
    $copyExit = $LASTEXITCODE
    Write-Output "COPY EXIT: $copyExit"
    if ($copyExit -ne 0) { $rc = 2 }
}
catch {
    Write-Output "FATAL: $($_.Exception.Message)"
    $rc = 1
}
finally {
    Stop-Transcript | Out-Null
}
exit $rc
'@ | Set-Content "C:\ProgramData\restic\backup-nightly.ps1" -Encoding UTF8

Restic exit codes that matter: 0 success, 1 command failed, 3 backup completed but some source files could not be read. Exit 3 is not success — it means data is missing from the snapshot and must be investigated. Run restic --help on the deployed version for the full list.

6.3 — Weekly maintenance script

@'
$ErrorActionPreference = "Stop"
$log = "C:\ProgramData\restic\logs\maint_{0}.log" -f (Get-Date -Format "yyyyMMdd_HHmmss")
Start-Transcript -Path $log -Force | Out-Null

Get-ChildItem "C:\ProgramData\restic\logs" -Filter "maint_*.log" |
    Sort-Object LastWriteTime -Descending | Select-Object -Skip 12 | Remove-Item -Force -EA SilentlyContinue

$rc = 0
try {
    . "C:\ProgramData\restic\restic-env.ps1"

    foreach ($repo in @($env:RESTIC_REPO_NAS, $env:RESTIC_REPO_B2)) {
        Write-Output "=== $repo ==="
        $env:RESTIC_REPOSITORY = $repo

        & $env:RESTIC_BIN forget --keep-daily 14 --keep-within 1m --prune
        if ($LASTEXITCODE -ne 0) { $rc = 2 }

        & $env:RESTIC_BIN check
        if ($LASTEXITCODE -ne 0) { $rc = 2 }

        & $env:RESTIC_BIN snapshots
    }
}
catch {
    Write-Output "FATAL: $($_.Exception.Message)"
    $rc = 1
}
finally {
    Stop-Transcript | Out-Null
}
exit $rc
'@ | Set-Content "C:\ProgramData\restic\maintain.ps1" -Encoding UTF8

Retention is --keep-daily 14 --keep-within 1m, matching the DTC NinjaOne standard on page 1421 (14 daily revisions, one month total). The interim backup deliberately mirrors the standard so the client's recovery window does not change.

Prune runs weekly, not nightly. Prune over SMB against a large repository is slow and I/O-heavy; bolting it onto the nightly job risks the backup window.

6.4 — Register the scheduled tasks

Tasks run as the site NAS service account from IT Glue — the same identity NinjaOne uses for the local backup destination. This avoids minting a new credential and the account already has proven write access to the share.

# Interactive prompt avoids the password appearing in command history.
# (Read-Host / Get-Credential is prohibited in RMM scripts — this is a manual T3 procedure, not RMM.)
$cred = Get-Credential -Message "Site NAS service account (from IT Glue)"

$common = @{
    User        = $cred.UserName
    Password    = $cred.GetNetworkCredential().Password
    RunLevel    = "Highest"
}

Register-ScheduledTask -TaskName "Restic - Nightly Backup" @common `
    -Action  (New-ScheduledTaskAction -Execute "powershell.exe" `
              -Argument "-NoProfile -ExecutionPolicy Bypass -File C:\ProgramData\restic\backup-nightly.ps1") `
    -Trigger (New-ScheduledTaskTrigger -Daily -At 11pm) `
    -Description "Interim restic backup - NAS then B2 copy. See KB: Restic Emergency Interim Backup. Ticket <TICKET>."

Register-ScheduledTask -TaskName "Restic - Weekly Maintenance" @common `
    -Action  (New-ScheduledTaskAction -Execute "powershell.exe" `
              -Argument "-NoProfile -ExecutionPolicy Bypass -File C:\ProgramData\restic\maintain.ps1") `
    -Trigger (New-ScheduledTaskTrigger -Weekly -DaysOfWeek Sunday -At 3am) `
    -Description "Interim restic retention and integrity check. See KB: Restic Emergency Interim Backup. Ticket <TICKET>."

Get-ScheduledTask -TaskName "Restic*" | Select-Object TaskName, State

The account requires the Log on as a batch job right. If task registration succeeds but the task fails immediately with a logon error, that right is missing — grant it via secpol.msc → Local Policies → User Rights Assignment, or the equivalent GPO.

Schedule the backup at the same hour the NinjaOne plan used (DTC standard is 23:00 local) so the client's backup window does not shift.

7. Verification

Run the morning after the first scheduled execution. Do not report the deployment complete until this passes.

. "C:\ProgramData\restic\restic-env.ps1"

foreach ($repo in @($env:RESTIC_REPO_NAS, $env:RESTIC_REPO_B2)) {
    Write-Output "=== $repo ==="
    $env:RESTIC_REPOSITORY = $repo
    & $env:RESTIC_BIN snapshots --compact
    & $env:RESTIC_BIN stats --mode raw-data
}

Get-ScheduledTask -TaskName "Restic*" | Get-ScheduledTaskInfo |
    Select-Object TaskName, LastRunTime, LastTaskResult, NextRunTime | Format-Table -AutoSize

Get-ChildItem "C:\ProgramData\restic\logs" | Sort-Object LastWriteTime -Descending | Select-Object -First 3

Both repositories must show a snapshot from the scheduled run. LastTaskResult 0 on both tasks. A green task result with no new snapshot means the script exited before restic ran — read the transcript.

8. Restore

Three paths. Option A is the guaranteed one and is what you should reach for under pressure.

8.1 — Option A: restic restore (primary)

. "C:\ProgramData\restic\restic-env.ps1"
$env:RESTIC_REPOSITORY = $env:RESTIC_REPO_NAS    # or $env:RESTIC_REPO_B2 if the site is gone

# 1 - find the snapshot
& $env:RESTIC_BIN snapshots --compact

# 2 - locate the file if the path is unknown
& $env:RESTIC_BIN find "PatientLedger*"

# 3a - restore specific files to a staging folder (SAFE DEFAULT)
& $env:RESTIC_BIN restore <snapshotID> `
    --target "D:\restore-staging" `
    --include "/D/Storage/SomeFolder"

# 3b - restore an entire snapshot to a staging folder
& $env:RESTIC_BIN restore latest --target "D:\restore-staging"

Always restore to a staging folder, never over live data. Restore to D:\restore-staging, verify the content, then move it into place deliberately. Restoring directly over a production path with the application running is how a recoverable incident becomes an unrecoverable one.

Paths inside a snapshot use forward slashes with the drive letter as the first element — C:\Windows appears as /C/Windows. Use restic ls to confirm the exact form before building an --include.

8.2 — Option B: restic mount (browsing, requires WinFsp)

UNVERIFIED ON WINDOWS — verify before relying on this in a live recovery. Mount support on Windows is provided through WinFsp. DTC has not confirmed it functional on the currently deployed restic version, and Backblaze's own integration guide notes that some restic features are unavailable on Windows. Test it during the deployment, not during the disaster. If it does not work, use Option A or C.

# Install WinFsp first - https://winfsp.dev  (requires a reboot on some builds)

. "C:\ProgramData\restic\restic-env.ps1"
$env:RESTIC_REPOSITORY = $env:RESTIC_REPO_NAS

& $env:RESTIC_BIN mount X:

# Browse X:\snapshots\ in Explorer. Ctrl+C in the console to unmount.

The mount is read-only. Useful for locating a file when the client cannot describe where it lived, and for comparing revisions before choosing one.

8.3 — Option C: ls and dump (no mount required)

# List a snapshot's contents
& $env:RESTIC_BIN ls latest /D/Storage

# Stream a single file straight out of the repository
& $env:RESTIC_BIN dump latest "/D/Storage/Reports/file.pdf" > "D:\restore-staging\file.pdf"

# Dump an entire folder as a zip
& $env:RESTIC_BIN dump --archive zip latest "/D/Storage/Reports" > "D:\restore-staging\reports.zip"

This is the fallback when mount is unavailable and a single file is needed quickly. It requires no additional software.

9. Restore Test — Mandatory

A backup that has not been restored from is not a backup. Perform this within 24 hours of deployment and record the result on the ticket.

  1. Pick a file from the client's LOB data — real data, not a text file you created
  2. Restore it to D:\restore-staging using Option A
  3. Open it in its native application and confirm it is intact and current
  4. Repeat the restore from the B2 repository, not just the NAS — an untested offsite copy is not an offsite copy
  5. Test Option B (mount) once, to establish whether it works on this host
  6. Delete the staging folder
  7. Record on the ticket: file restored, source repository, snapshot ID, verification method, outcome

10. Disaster Recovery — Host Is Gone

Restic holds files, not an image. Recovery of a destroyed host is a rebuild, and the sequence matters.

  1. Build the replacement OS to the current DTC standard and patch it
  2. Restore roles before data. Domain controller: promote a new DC into the existing domain and let AD replicate — do not restore NTDS files from restic onto a live domain. If this was the only DC, escalate before proceeding; restoring a lone DC from file-level backup is a Microsoft-unsupported path and needs a deliberate decision.
  3. Reinstall LOB applications to the same versions and paths
  4. Restore data from restic to a staging folder, then move into place with services stopped
  5. Restore from B2 if the site NAS was lost with the host — the repository string is the only thing that changes
  6. Verify with the client before returning the system to service

Recovering from a different machine requires only the restic binary, the repository password from IT Glue, and the B2 key. Nothing is tied to the original host.

11. Rollback

If deployment goes wrong or must be reversed before completion:

Unregister-ScheduledTask -TaskName "Restic - Nightly Backup"     -Confirm:$false -EA SilentlyContinue
Unregister-ScheduledTask -TaskName "Restic - Weekly Maintenance" -Confirm:$false -EA SilentlyContinue

# Leaves the repositories intact - data is not destroyed by removing automation.
Get-ScheduledTask -TaskName "Restic*" -EA SilentlyContinue

Removing the tasks stops all activity without touching either repository. Nothing in this procedure modifies the host's existing NinjaOne configuration, so rollback cannot affect NinjaOne remediation.

12. Exit Criteria & Decommission

Review weekly. Restic comes out as soon as NinjaOne Hybrid Cloud Backup is confirmed working — not when it is believed working.

Decommission only when all hold:

  • NinjaOne has produced at least three consecutive successful backups with both local and cloud legs completing
  • A restore has been tested from the NinjaOne backup, per the integrity verification procedure in this book
  • The NinjaOne vendor case is closed or the root cause is confirmed resolved
# 1 - stop automation
Unregister-ScheduledTask -TaskName "Restic - Nightly Backup"     -Confirm:$false
Unregister-ScheduledTask -TaskName "Restic - Weekly Maintenance" -Confirm:$false

# 2 - RETAIN both repositories for 30 days past NinjaOne's first verified restore.
#     Do not delete on the same day you re-enable NinjaOne.

# 3 - after the retention window:
#     - delete the NAS repository folder
#     - delete the B2 bucket
#     - delete the B2 application key
#     - remove C:\Program Files\restic and C:\ProgramData\restic
#     - archive the repo password and B2 key in IT Glue as retired (do not delete
#       the IT Glue entry until both repositories are gone)

# 4 - update the tracking sheet and close the ticket

13. Tracking

Every restic deployment is recorded in the Restic Interim Deployment Tracker. An untracked deployment is one nobody will remember to remove.

Record: client, hostname, deployment date, ticket, NinjaOne fault and vendor case, NAS repository path, B2 bucket and endpoint, restic version, repository ID, retention, restore-test date and result, weekly review date, decommission date.

  • 1421 — NinjaOne Image Backup Plan Configuration Standard (the standard this bridges to)
  • 2966 — NinjaOne Backup: Architecture Deep Dive — Lockhart, Cloud Storage & Hybrid Model
  • 2967 — NinjaOne Backup: Support Escalation — When to Call & What to Bring
  • 2968 — NinjaOne Backup: Log File Locations & How to Read Them
  • 2969 — NinjaOne Backup: Error Code Master Reference
  • 3008 — NinjaOne Backup: Backup Integrity — Manual Verification & Spot-Check Procedure
  • 3715 — NinjaOne Backup: Backup Won't Start / Stuck on "Backup Started"
  • 3818 — NinjaOne Backup: SMB Credentials Rejected (System Error 86) — LmCompatibilityLevel / NTLMv2
  • 1034 — Disaster Recovery Runbook
  • 4116 — Open Backup Failure Tracker

Change Record

Date

Author

Change

2026-08-31

Z. Boogher

Page created. Derived from the interim deployment performed under Halo 1170038 following Lockhart clone-mode deactivation on a domain controller with no onsite BDR.

Back to top