Limehawk

Implementation Summary

Deployed automated monthly backup solution for Proxmox VE host configurations using bash script and cron.monthly. Backs up critical system directories to centralized SMB share with 6-month retention policy. Ensures rapid disaster recovery and cluster rebuild capability without relying on VM-level backups for host configuration.

> The Challenge

Proxmox VE provides excellent built-in backup solutions for virtual machines and containers, but host-level configuration is often overlooked. If a Proxmox host fails catastrophically, you need:

> Critical Host Configuration

•Network configuration (bridges, bonds, VLANs)
•Custom scripts and automation in /root and /opt
•Cluster configuration and certificates
•Storage mount points and credentials
•System-level customizations in /etc

Without host-level backups, rebuilding a failed Proxmox node means manually reconfiguring everything from memory or outdated documentation. This solution automates configuration backups to centralized storage for quick recovery.

> Solution Architecture

A bash script runs monthly via cron.monthly, creating timestamped tar archives of critical directories and storing them on a network SMB share. Automatic cleanup keeps only the latest 6 backups per directory to prevent unbounded storage growth.

> Key Features

→Monthly automated execution via cron
→Centralized storage on SMB share (accessible by multiple hosts)
→Timestamped archives for point-in-time recovery
→Automatic retention management (keeps latest 6, deletes older)
→Detailed logging to /var/log for audit trail
→Pre-flight check ensures SMB mount availability

> Directories Included in Backup

/etc

System configuration files including network settings, package configurations, systemd units, and Proxmox-specific configs.

Critical for: network bridges, storage mounts, service configurations

/root

Root user's home directory containing custom scripts, SSH keys, bash history, and administrative tools.

Critical for: automation scripts, credentials, custom tooling

/opt

Custom installed software and scripts not managed by package manager. Often contains third-party monitoring agents, custom applications.

Critical for: non-packaged software, monitoring agents, custom apps

/var/lib/pve-cluster

Proxmox cluster database containing VM/CT configurations, resource pools, user permissions, datacenter settings, and cluster membership info.

Critical for: cluster config, VM definitions, user permissions

> Backup Script

The script is placed in /etc/cron.monthly/ and runs automatically on the first of each month.

proxmox_config_to_nas(47 lines)

> Bash

#!/bin/bash

# Define backup location (Proxmox SMB mount point)
SMB_FOLDER="/mnt/pve/proxmox_hosts"
LOG_FILE="/var/log/proxmox_config_backup.log"
TIMESTAMP=$(date +'%Y-%m-%d_%I-%M-%S%p')
HOSTNAME=$(hostname)

# Directories to back up (Absolute Paths)
DIRS_TO_BACKUP=(
  "/etc"
  "/root"
  "/opt"
  "/var/lib/pve-cluster"
)

Script Behavior:

→Pre-flight check: Verifies SMB mount is available (exits if not)
→Creates timestamped tar.gz archives per directory
→--ignore-failed-read: Continues if files are locked/unreadable
→Logs all actions to /var/log/proxmox_config_backup.log
→Automatic cleanup: keeps 6 most recent backups per directory, deletes rest

> Backup File Naming

Files are named with hostname, directory, and timestamp for easy identification and sorting.

> Example filenames

proxmox01_etc_2025-01-28_11-06-13PM.tgz
proxmox01_root_2025-01-28_11-06-13PM.tgz
proxmox01_opt_2025-01-28_11-06-13PM.tgz
proxmox01_pve-cluster_2025-01-28_11-06-13PM.tgz

Format: {hostname}_{directory}_{timestamp}.tgz

> Implementation Steps

Step 1: Mount SMB Share

Add SMB mount to /etc/fstab for persistent mounting across reboots.

> Add to /etc/fstab

//fileserver.internal.example.com/backups/proxmox_hosts /mnt/pve/proxmox_hosts cifs username=backupuser,password=YOUR_PASSWORD,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0

> Mount manually

mount -a

> Verify mount

ls /mnt/pve/proxmox_hosts

Security Note: Consider using a credentials file instead of plaintext password in fstab. Create /root/.smbcredentials with permissions 600 containing username/password, then reference it with credentials=/root/.smbcredentials.

Step 2: Install Backup Script

> Create script file

nano /etc/cron.monthly/proxmox_config_to_nas

Paste the script above, save (CTRL+X, Y, ENTER)

> Make executable

chmod +x /etc/cron.monthly/proxmox_config_to_nas

Step 3: Test Backup Manually

> Run manually

/etc/cron.monthly/proxmox_config_to_nas

> Verify backups created

ls -lt /mnt/pve/proxmox_hosts/

> Check logs

cat /var/log/proxmox_config_backup.log

Step 4: Ensure Monthly Execution

Proxmox should automatically run scripts in /etc/cron.monthly/. If it doesn't, add to crontab manually.

> Edit crontab (if needed)

nano /etc/crontab

> Add this line (runs 2 AM on 1st of month)

0 2 1 * * root /etc/cron.monthly/proxmox_config_to_nas >/dev/null 2>&1

> Restart cron

systemctl restart cron

> Deploying to Multiple Hosts

The script uses $(hostname) to automatically namespace backups per host. Deploy the exact same script to multiple Proxmox servers without modification.

Multi-Host Benefits:

→Single script works across entire cluster
→Backups automatically organized by hostname
→Centralized backup location for all hosts
→Each host manages its own retention independently

> Outcome

Fully automated host configuration backup solution running across multiple Proxmox nodes. Monthly backups provide 6 months of retention with automatic cleanup. Centralized SMB storage enables rapid disaster recovery without manual configuration reconstruction.

Implementation Benefits:

→Zero ongoing maintenance required
→Rapid host rebuild capability (restore /etc, /root, cluster config)
→Works across entire Proxmox cluster without per-host customization
→Centralized backup location simplifies monitoring and verification
→Automatic retention prevents storage exhaustion

Key Takeaways:

→VM backups are useless if you can't rebuild the host they run on
→Network configuration and cluster membership data are critical for rapid recovery
→Simple bash + cron often beats complex backup solutions for host-level configs
→Timestamped archives provide audit trail and rollback capability

> Tools & Technologies

Proxmox VE

Open-source virtualization platform based on Debian. Enterprise-grade hypervisor with clustering support.

CIFS/SMB

Network file sharing protocol. Enables centralized backup storage accessible by multiple hosts.

cron

Time-based job scheduler in Unix-like systems. Enables automated recurring tasks.

tar

Archive utility for creating compressed file collections. Preserves permissions and metadata.

Automated Proxmox Host Configuration Backups to SMB Share