Skip to main content

BDR Offline & Connectivity

Audience: T1 / T2 / T3
Version: 1.0
Last Updated: April 2026

Overview

This guide addresses scenarios where the BDR appliance itself is unreachable — cannot ping, RDP, or access the Veeam console.

Common symptoms:

  • Cannot ping BDR IP address
  • RDP connection refused or times out
  • NinjaRMM shows BDR as offline
  • Veeam console on BDR is inaccessible
  • All backup jobs fail or don't start

⚠️ CRITICAL: If the BDR is offline, all backups are failing. This is a severity 1 issue requiring immediate response.

Tier Definitions

Tier Scope Expected Resolution
T1 Basic connectivity, power/network checks 10-15 minutes
T2 Remote diagnostics, service recovery 30-60 minutes
T3 Physical intervention, hardware replacement 1+ hours, may require onsite

T1 — Basic Connectivity & Power Checks

1. Verify Network Connectivity

From your workstation or another server on the same network:

Test-Connection -ComputerName [BDR_IP] -Count 4

If ping fails:

  • BDR is powered off, network-disconnected, or IP changed
  • Proceed to physical/remote checks

If ping succeeds:

  • BDR is online but services may be down
  • Proceed to T2

2. Check Physical Status (if onsite or remote console available)

If you have physical access or iDRAC/IPMI/iLO:

  1. Check power status (front panel LEDs, remote console)
  2. Check network cable connected and link lights active
  3. Check console/monitor for errors or login prompt

If BDR is powered off:

  • Power on and wait for boot (5-10 minutes for full Veeam services)
  • Monitor for POST errors or boot failures

If BDR won't power on:

  • Check power cable, UPS status
  • Escalate to T3 for hardware failure

3. Check for IP Conflict or DHCP Lease Expiration

If BDR was on DHCP (not recommended, but happens):

From a workstation on the same network:

# Check ARP table for BDR MAC address
arp -a | Select-String [BDR_PARTIAL_IP]

If MAC is present but IP is different: BDR may have gotten a new DHCP lease.

Action:

  • Update DNS/bookmarks with new IP
  • Long-term fix: Configure static IP per DTC standard

4. Attempt RDP Connection

Even if ping fails, try RDP:

mstsc /v:[BDR_IP]

If RDP prompt appears: Network is working, Windows is running, but something else is wrong (services, firewall, etc.) → T2

If RDP times out or refused: Windows may not be fully booted or network is down → Continue T1 checks

5. Remote Power Control via IPMI (BDR Unreachable)

When to use this: The BDR is showing offline in NinjaOne and you cannot reach it any other way.

Background — what you're actually doing here

When a BDR won't respond, your first instinct might be to call the client and ask them to physically reboot it. Before you do that, try this.

Most BDR appliances have a hidden management connection built into the hardware. It has its own login page and stays powered on even when the BDR appears totally dead in NinjaOne.

Think of it like this: the BDR is a car that won't start. IPMI is a hidden panel under the hood that lets you crank the engine remotely, even if the dashboard is completely dark.

The steps below walk you through finding that hidden connection and using it to reboot the BDR without anyone touching it physically.

Step 1 — Remote into a machine at the client site

Use NinjaOne to remote into any online machine at the client's site — the server is preferred, but any workstation will work. You need to be on their network to find the BDR's hidden management connection.

Step 2 — Find the BDR's IP in NinjaOne

Before scanning, look up the offline BDR device in NinjaOne and note its private IP address. You'll use this to find it in the scan results.

Step 3 — Download and run Advanced IP Scanner

On the remote machine, open a browser and Google "Advanced IP Scanner." Click the first result and download it. You don't need to install it — just open it and run it.

Step 4 — Scan the network

When Advanced IP Scanner opens, you'll see an IP range already filled in (something like 192.168.0.1-254 or 192.168.1.1-254). Leave it as-is and click Scan. Wait for it to finish.

Step 5 — Find the IPMI entry

This part looks different depending on the BDR hardware. Look for one of the following:

  • An entry that expands to show "HTTP, Megarac SP" underneath it. It will not have a device name — just an IP address. This is a separate IP from the BDR's main IP.
  • The BDR's own IP (from Step 2) that expands to show an HTTP entry underneath it. This means the IPMI is on the same IP as the BDR, not a separate one.

Either way, the IPMI entry is what you're after. If neither appears, IPMI may not be available on this BDR — escalate to T2/T3.

Step 6 — Open the login page

Right-click the IPMI entry and select Tools > HTTP. A browser window will open on the remote machine showing a login page.

Step 7 — Log in

Open 1Password and search for the client name plus the word "IPMI" (example: "Georgetown IPMI" or "BDR IPMI"). Use those credentials to log in.

These are not the same as the Windows login. IPMI has its own separate username and password.

Step 8 — Reboot the BDR

Once logged in, find the Power Control section. What you see depends on the hardware:

  • Megarac interface (ASRock/Equus hardware): Navigate to Power Control and click the reboot option.
  • Supermicro interface: Power Control is at the bottom of the main System page you land on after login. Click Reset.

In both cases, confirm when prompted.

You may also see a Remote Console Preview — a live screenshot of the BDR's screen. This is useful to check if Windows is at a login screen, crashed, or mid-boot.

Step 9 — Wait and watch

Go back to NinjaOne and watch the BDR device. It will take 5 to 10 minutes to fully boot back up. Do not try anything else until it shows back online.

T2 — Remote Diagnostics & Service Recovery

1. Check NinjaRMM Status & Run Diagnostics

If NinjaRMM agent is installed on the BDR:

  1. Log into NinjaRMM dashboard
  2. Search for the BDR device
  3. Check last seen time
  4. If recently online: Run remote command to check services

NinjaRMM remote PowerShell:

Get-Service Veeam* | Select Name, Status, StartType
Get-EventLog -LogName System -Newest 50 | Where-Object {$_.EntryType -eq 'Error'} | Select TimeGenerated, Source, Message

2. Attempt WinRM / Remote PowerShell

From a Windows server on the same network (if WinRM is enabled on BDR):

Enter-PSSession -ComputerName [BDR_IP] -Credential (Get-Credential)

# Once connected:
Get-Service Veeam* | Select Name, Status
Get-EventLog -LogName System -Newest 20

If WinRM fails: Services may be stopped or Windows is unresponsive → T3

3. Check for Windows Update or Reboot Pending

If you get connected via RDP or WinRM:

# Check for pending reboot
Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired" -ErrorAction SilentlyContinue

# Check last boot time
(Get-CimInstance Win32_OperatingSystem).LastBootUpTime

# Check if Windows Update is running
Get-Process wuauclt, TrustedInstaller -ErrorAction SilentlyContinue

If pending reboot or update in progress:

  • Schedule maintenance window with client
  • Reboot BDR
  • Wait 10-15 minutes for full boot and Veeam services

4. Restart Veeam Services

If BDR is online but Veeam console won't open:

# Restart all Veeam services
Get-Service Veeam* | Restart-Service -Force

# Verify status
Get-Service Veeam* | Select Name, Status

Expected services:

  • VeeamBackupSvc - Veeam Backup Service
  • VeeamBrokerSvc - Veeam Broker Service
  • VeeamCatalogSvc - Veeam Catalog Service
  • VeeamCloudSvc - Veeam Cloud Connect Service (if enabled)
  • VeeamDeploymentService - Veeam Deployment Service
  • VeeamMountSvc - Veeam Mount Service
  • VeeamNFSSvc - Veeam NFS Service
  • VeeamRESTSvc - Veeam REST API Service
  • VeeamTransportSvc - Veeam Transport Service

All should be Running. If any fail to start, check Event Viewer → Application logs for Veeam errors.

5. Check Disk Space on BDR

Critical: If BDR C: drive is full, Windows and Veeam may become unresponsive.

Get-WmiObject Win32_LogicalDisk | Select DeviceID, @{N='FreeGB';E={[math]::Round($_.FreeSpace/1GB,2)}}, @{N='TotalGB';E={[math]::Round($_.Size/1GB,2)}}

If C: drive < 1 GB free:

  • Clear temp files: Remove-Item C:\Windows\Temp\* -Recurse -Force
  • Check for massive log files: Get-ChildItem C:\ProgramData\Veeam\Backup\*.log | Sort Length -Descending | Select -First 10
  • See BDR Storage Alerts & Capacity Issues for full cleanup procedures

T3 — Physical Intervention & Hardware Failure

1. Schedule Onsite Visit

If BDR is unresponsive and remote diagnostics fail:

  1. Notify client: BDR is offline, backups are not running
  2. Schedule emergency onsite (same-day if possible)
  3. Prepare:
    • Backup of Veeam configuration (if accessible remotely before failure)
    • Spare hardware (if BDR is old/failing)

2. Onsite Diagnostics

At client site:

  1. Check power:

    • UPS status (is BDR plugged into UPS? Is UPS working?)
    • Power cable connected
    • Front panel power button responsive

  2. Check network:

    • Cable connected to correct switch port
    • Link lights active on NIC and switch
    • Try different network cable/port

  3. Access local console:

    • Connect monitor and keyboard
    • Power on (if off) and observe POST
    • Look for BIOS errors, RAID warnings, or disk failures

  4. Boot into Safe Mode (if Windows won't start normally):

    • Reboot and press F8
    • Select "Safe Mode with Networking"
    • Once in, check Event Viewer for crash logs

3. Common Hardware Failures

Symptom Likely Cause Fix
No power, no POST PSU failure or motherboard Replace PSU, escalate if motherboard
POST succeeds, no OS boot Disk failure or OS corruption Boot from recovery media, check disk health
Blue screen on boot Driver or hardware incompatibility Boot Safe Mode, check Event Viewer
Network unreachable NIC failure or cable/switch issue Test cable, try different NIC/port
Intermittent crashes/reboots Memory or PSU failure Run memtest, check PSU voltages

4. Restore from Bare Metal Backup (if BDR is dead)

If BDR hardware has failed and cannot be repaired:

DTC Standard: BDRs should have local or offsite backup of the Veeam configuration.

Recovery steps:

  1. Deploy replacement BDR hardware (or repurpose server)
  2. Install Windows Server
  3. Install Veeam Backup & Replication
  4. Restore Veeam configuration from backup:
    • Veeam console: Main Menu → Configuration Backup → Restore
    • Point to backup location (USB, network share, or cloud)
  5. Re-establish repository paths and verify backup chains

⚠️ If no configuration backup exists: Manual rebuild required:

  • Recreate all backup jobs from scratch
  • Existing backup files on repository may be orphaned and require manual import

5. Temporary Workaround — Direct-Attached Repository

If BDR is dead but backup repository (external disk/NAS) is intact:

Short-term solution:

  1. Attach repository to another Windows server
  2. Install Veeam on that server
  3. Import existing backup chains
  4. Run jobs from temporary server until BDR is replaced

Note: This is not a long-term solution but keeps backups running during hardware procurement.

Prevention & Monitoring

1. Enable UPS Monitoring

Ensure BDR is on UPS with monitoring:

  • Configure UPS software to shut down BDR gracefully on low battery
  • NinjaRMM can monitor UPS status if agent is installed

2. Configure Email Alerts for BDR Downtime

In Veeam console:

  1. Main Menu → Notifications
  2. Enable Email notifications
  3. Add recipients
  4. Set alerts for:
    • Backup job failures
    • Repository warnings
    • Service failures

In NinjaRMM:

  • Set up device offline alert for BDR (5-minute threshold)

3. Regular Configuration Backups

DTC Standard: Veeam configuration should be backed up weekly.

In Veeam console:

  1. Main Menu → Configuration Backup
  2. Enable scheduled backups
  3. Save to:
    • Network share (not on BDR itself)
    • USB drive (rotated weekly)
    • Cloud storage (Backblaze B2, OneDrive, etc.)

4. Document BDR Build & Recovery Procedures

In HALO or BookStack:

  • BDR hardware specs (model, CPU, RAM, storage config)
  • Windows Server version and license
  • Veeam version and license
  • Network configuration (static IP, gateway, DNS)
  • Repository paths and retention settings

Use for: Quick rebuild if BDR fails catastrophically.

Quick Reference — BDR Offline Scenarios

Symptom Likely Cause Immediate Action
Cannot ping, cannot RDP Powered off, network cable unplugged, or IP changed Check physical connections, verify power
Ping works, RDP fails Windows Firewall blocking or RDP service stopped Check services, reboot if needed
RDP works, Veeam console won't open Veeam services crashed or stopped Restart Veeam services via PowerShell
Intermittent offline/online Network flapping, bad cable, or switch issue Replace cable, check switch logs
Offline after Windows Update Update failed or reboot pending Onsite console access, boot Safe Mode
Blue screen/crash on boot Hardware failure (disk, RAM, PSU) Onsite diagnostics, hardware replacement

Escalation & Client Communication

If BDR is offline and cannot be recovered remotely:

  1. Notify client immediately:

    • BDR is offline
    • All backups are currently not running
    • Estimated time to resolution

  2. Document in HALO:

    • Symptoms observed
    • Troubleshooting steps performed
    • Client notification timestamp
    • Onsite visit scheduled (if applicable)

  3. Escalate to team lead if:

    • Onsite visit required but no tech available
    • Hardware replacement needed
    • Client needs same-day resolution

Document History:

Version Date Author Changes
1.0 April 2026 DTC Initial creation - BDR offline troubleshooting
1.1 May 2026 Zachary Ireland Added T1 Step 5: Remote Power Control via IPMI — covers Megarac and Supermicro interfaces, Advanced IP Scanner process, and 1Password credential lookup
Back to top