rewrite readme.md file

This commit is contained in:
Dejan R. 2025-11-27 16:33:06 +00:00
parent 73e445b7e6
commit d72ee8bf0f

View file

@ -1,252 +1,507 @@
# NetBird Backup & Restore Scripts
# Netbird Docker Compose - Backup & Restore Guide
Comprehensive backup and restore solution for NetBird Docker deployments with Zitadel integration.
## Overview
These scripts provide automated backup and restore functionality for your NetBird installation, including:
- Docker volumes (database, management data, certificates)
- Configuration files (docker-compose, environment variables)
- Traefik reverse proxy configuration
- Zitadel authentication data
## Prerequisites
- Docker and Docker Compose installed
- NetBird deployed via Docker Compose
- Sufficient disk space for backups
- Root or sudo access for Docker operations
## Files
### 1. `backup-netbird.sh`
Creates timestamped backups of your complete NetBird installation.
### 2. `restore-netbird.sh`
Restores NetBird from a backup archive.
## Installation
1. Place both scripts in your NetBird installation directory:
```bash
/home/Dejan/Docker/Netbird-compose/
```
2. Make scripts executable:
```bash
chmod +x backup-netbird.sh restore-netbird.sh
```
3. Ensure the backup directory exists:
```bash
mkdir -p /home/Dejan/Docker/Netbird-compose/backup
```
## Usage
### Creating a Backup
Run the backup script from your NetBird directory:
```bash
cd /home/Dejan/Docker/Netbird-compose
./backup-netbird.sh
```
**What happens during backup:**
1. Creates backup directory if it doesn't exist
2. Stops the NetBird Docker stack
3. Exports all Docker volumes to compressed archives
4. Archives configuration files
5. Restarts the NetBird stack
6. Creates a timestamped backup file: `netbird_backup_YYYY-MM-DD_HH-MM-SS.tar.gz`
**Backup includes:**
- Docker volumes: `netbird_zdb_data`, `netbird_management`, `netbird_zitadel_certs`
- Configuration: `docker-compose.yml`, `*.env` files
- NetBird config: `management.json`, `turnserver.conf`
- Authentication: `machinekey` directory
- Reverse proxy: `traefik-stack` directory
- The backup script itself
### Restoring from Backup
#### Option 1: Restore latest backup (automatic)
```bash
cd /home/Dejan/Docker/Netbird-compose
./restore-netbird.sh
```
#### Option 2: Restore specific backup
```bash
./restore-netbird.sh netbird_backup_2024-11-24_14-30-00.tar.gz
```
Or with full path:
```bash
./restore-netbird.sh /home/Dejan/Docker/Netbird-compose/backup/netbird_backup_2024-11-24_14-30-00.tar.gz
```
**What happens during restore:**
1. Validates backup file exists
2. Extracts backup to temporary directory
3. Stops current NetBird stack
4. Restores all Docker volumes
5. Restores configuration files
6. Cleans up temporary files
7. Starts NetBird stack with restored data
## Backup Schedule
### Manual Backups
Run before major changes:
- System updates
- Configuration modifications
- Docker Compose upgrades
### Automated Backups (Recommended)
Add to crontab for automatic daily backups:
```bash
crontab -e
```
Add this line for daily backup at 2 AM:
```cron
0 2 * * * /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
```
For weekly backups (Sunday at 3 AM):
```cron
0 3 * * 0 /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
```
## Backup Management
### View Available Backups
```bash
ls -lh /home/Dejan/Docker/Netbird-compose/backup/
```
### Delete Old Backups
Keep only last 7 backups:
```bash
cd /home/Dejan/Docker/Netbird-compose/backup
ls -1t netbird_backup_*.tar.gz | tail -n +8 | xargs rm -f
```
### Backup Retention Script
Create `cleanup-old-backups.sh`:
```bash
#!/bin/bash
BACKUP_DIR="/home/Dejan/Docker/Netbird-compose/backup"
KEEP_LAST=7
cd "$BACKUP_DIR"
ls -1t netbird_backup_*.tar.gz | tail -n +$((KEEP_LAST + 1)) | xargs -r rm -f
echo "Cleaned up old backups, kept last $KEEP_LAST"
```
## Troubleshooting
### Backup Script Fails
- **Insufficient disk space**: Check available space with `df -h`
- **Docker not running**: Start Docker service: `sudo systemctl start docker`
- **Permission denied**: Run with sudo: `sudo ./backup-netbird.sh`
### Restore Script Fails
- **Backup file not found**: Verify path and filename
- **Volume restore fails**: Check Docker is running and volumes are accessible
- **Stack won't start**: Check logs: `docker compose logs`
### Common Issues
**Stack doesn't start after restore:**
```bash
# Check container logs
docker compose logs -f
# Verify volumes exist
docker volume ls | grep netbird
# Try manual restart
docker compose down
docker compose up -d
```
**Configuration mismatch:**
```bash
# Verify restored files
ls -la /home/Dejan/Docker/Netbird-compose/
# Check environment variables
cat .env
```
## Important Notes
- **Downtime**: Both backup and restore scripts stop the NetBird stack temporarily
- **Backup size**: Varies based on database size and user count (typically 100MB-1GB)
- **Storage**: Ensure adequate backup storage space
- **Testing**: Test restore process in a development environment first
- **Security**: Backup files contain sensitive data - secure appropriately
## Backup Security
Protect your backups:
1. **Restrict permissions:**
```bash
chmod 700 /home/Dejan/Docker/Netbird-compose/backup
chmod 600 /home/Dejan/Docker/Netbird-compose/backup/*.tar.gz
```
2. **Encrypt backups (optional):**
```bash
gpg --symmetric --cipher-algo AES256 netbird_backup_*.tar.gz
```
3. **Off-site backup:**
```bash
# Example: rsync to remote server
rsync -avz backup/ user@backup-server:/backups/netbird/
```
## Migration Guide
To migrate NetBird to a new server:
1. **On old server:** Create backup
```bash
./backup-netbird.sh
```
2. **Transfer backup file:**
```bash
scp backup/netbird_backup_*.tar.gz user@new-server:/tmp/
```
3. **On new server:** Install Docker, Docker Compose, and NetBird structure
4. **Restore backup:**
```bash
./restore-netbird.sh /tmp/netbird_backup_*.tar.gz
```
5. **Update DNS/IP** if necessary
## Support
For issues or questions:
- Check NetBird documentation: https://docs.netbird.io
- Review Docker Compose logs: `docker compose logs`
- Verify volume integrity: `docker volume inspect <volume_name>`
## Version History
- **v1.0** - Initial backup and restore scripts with full volume and config support
Complete documentation for backing up and restoring your Netbird Docker Compose installation.
---
**Created for NetBird deployment at:** `/home/Dejan/Docker/Netbird-compose/`
**Last updated:** November 2024
## 📋 Table of Contents
- [Overview](#overview)
- [Prerequisites](#prerequisites)
- [Backup Script](#backup-script)
- [Restore Script](#restore-script)
- [Automated Backups](#automated-backups)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
---
## 🎯 Overview
This package includes two powerful scripts to manage your Netbird installation:
- **`backup-netbird.sh`** - Creates timestamped backups of your entire Netbird setup
- **`restore-netbird.sh`** - Restores your Netbird installation from any backup
Both scripts handle Docker services automatically and include comprehensive error handling.
---
## ⚙️ Prerequisites
- Root or sudo access
- Docker and Docker Compose installed
- Netbird installation at `/home/Dejan/Docker/Netbird-compose`
- Sufficient disk space for backups
---
## 💾 Backup Script
### What Gets Backed Up
The backup script saves:
- All configuration files (`.env`, `dashboard.env`, `zitadel.env`, etc.)
- Docker Compose configuration (`docker-compose.yml`)
- SSL certificates and keys (`acme.json`, `machinekey/`)
- Management configuration (`management.json`)
- Traefik configuration files
- All custom scripts
- Zitadel logs
**Excluded from backups:**
- The `backup/` directory itself
- `.git/` directory (if present)
- Old `.tar.gz` files
### How to Use
#### Basic Usage
```bash
# Navigate to your Netbird directory
cd /home/Dejan/Docker/Netbird-compose
# Make the script executable (first time only)
chmod +x backup-netbird.sh
# Run the backup
./backup-netbird.sh
```
#### What Happens During Backup
1. **Validation** - Checks for root access and directory existence
2. **Service Stop** - Gracefully stops all Docker Compose services
3. **Archive Creation** - Creates a compressed tar.gz archive with timestamp
4. **Service Restart** - Brings all services back online
5. **Verification** - Shows backup size and location
### Backup Output
Backups are saved in the `backup/` directory with timestamps:
```
backup/
├── netbird-backup-20241127_143022.tar.gz
├── netbird-backup-20241127_180530.tar.gz
└── netbird-backup-20241128_020000.tar.gz
```
**Filename format:** `netbird-backup-YYYYMMDD_HHMMSS.tar.gz`
### Backup Script Features
- ✅ Automatic service management
- ✅ Timestamped backups
- ✅ Colored console output
- ✅ Size reporting
- ✅ Error handling at each step
- ✅ Automatic backup directory creation
- ✅ List of recent backups after completion
### Optional: Automatic Cleanup
To keep only the last 5 backups, uncomment these lines in the script:
```bash
KEEP_BACKUPS=5
print_msg "Cleaning old backups, keeping last $KEEP_BACKUPS..."
cd "$BACKUP_DIR"
ls -t netbird-backup-*.tar.gz | tail -n +$((KEEP_BACKUPS + 1)) | xargs -r rm
```
---
## 🔄 Restore Script
### How to Use
#### Basic Usage
```bash
# Make the script executable (first time only)
chmod +x restore-netbird.sh
# Restore from a specific backup
./restore-netbird.sh netbird-backup-20241127_143022.tar.gz
```
#### Alternative Ways to Specify Backup
```bash
# Using just the filename (searches in backup/ directory)
./restore-netbird.sh netbird-backup-20241127_143022.tar.gz
# Using relative path
./restore-netbird.sh backup/netbird-backup-20241127_143022.tar.gz
# Using absolute path
./restore-netbird.sh /home/Dejan/Docker/Netbird-compose/backup/netbird-backup-20241127_143022.tar.gz
```
#### List Available Backups
```bash
# Show usage and list available backups
./restore-netbird.sh
```
### What Happens During Restore
1. **Validation** - Verifies backup file exists and is valid
2. **Preview** - Shows backup contents (first 20 files)
3. **Safety Backup** - Creates a pre-restore backup (recommended)
4. **Confirmation** - Asks for your approval before proceeding
5. **Service Stop** - Stops all running Docker services
6. **File Removal** - Removes current files (except backup/ directory)
7. **Extraction** - Extracts all files from backup archive
8. **Permissions** - Sets correct file permissions
9. **Verification** - Checks that critical files were restored
10. **Service Start** - Brings all services back online
11. **Status Report** - Shows running services status
### Interactive Prompts
The restore script will ask you two questions:
```
Do you want to create a safety backup before restoring? (recommended) [Y/n]
```
**Recommended:** Press Enter or type `Y` to create a safety backup
```
Are you sure you want to restore from this backup? [y/N]
```
**Type `y`** to proceed with the restore
### Restore Script Features
- ✅ Multiple backup file path formats supported
- ✅ Automatic backup file search
- ✅ Pre-restore safety backup
- ✅ Backup content preview
- ✅ Interactive confirmation prompts
- ✅ Automatic permission fixing
- ✅ Critical file verification
- ✅ Service status display
- ✅ Comprehensive error handling
### Safety Backups
When you create a safety backup, it's saved as:
```
backup/pre-restore-backup-YYYYMMDD_HHMMSS.tar.gz
```
This allows you to roll back if something goes wrong during restore.
---
## ⏰ Automated Backups
### Using Cron
Set up automatic backups using crontab:
```bash
# Edit root's crontab
sudo crontab -e
# Add one of these lines:
# Daily backup at 2:00 AM
0 2 * * * /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
# Every 6 hours
0 */6 * * * /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
# Weekly on Sunday at 3:00 AM
0 3 * * 0 /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
```
### Cron Schedule Examples
| Schedule | Cron Expression | Description |
|----------|----------------|-------------|
| Every day at 2 AM | `0 2 * * *` | Daily backup during low traffic |
| Every 12 hours | `0 */12 * * *` | Twice daily backups |
| Every 6 hours | `0 */6 * * *` | Four times daily |
| Weekly (Sunday 3 AM) | `0 3 * * 0` | Weekly backup |
| Every hour | `0 * * * *` | Hourly backup (generates many files) |
### Monitoring Automated Backups
```bash
# View backup log
tail -f /var/log/netbird-backup.log
# List recent backups
ls -lht /home/Dejan/Docker/Netbird-compose/backup/ | head -10
# Check cron is running
sudo systemctl status cron
# View crontab entries
sudo crontab -l
```
---
## 🎯 Best Practices
### Backup Strategy
1. **Regular Backups**
- Set up daily automated backups via cron
- Create manual backups before major changes
2. **Retention Policy**
- Keep at least 7 daily backups
- Keep at least 4 weekly backups
- Store critical backups off-site
3. **Before Updates**
- Always backup before updating Docker images
- Always backup before modifying configuration
- Always backup before system maintenance
4. **Test Restores**
- Periodically test restore process
- Verify backup integrity
- Document restore procedures
### Storage Management
```bash
# Check backup directory size
du -sh /home/Dejan/Docker/Netbird-compose/backup/
# Find backups older than 30 days
find /home/Dejan/Docker/Netbird-compose/backup/ -name "*.tar.gz" -mtime +30
# Remove backups older than 30 days (use with caution!)
find /home/Dejan/Docker/Netbird-compose/backup/ -name "*.tar.gz" -mtime +30 -delete
```
### Security
- Keep backups in a secure location
- Consider encrypting sensitive backups
- Restrict backup file permissions: `chmod 600 *.tar.gz`
- Store off-site copies for disaster recovery
---
## 🔧 Troubleshooting
### Common Issues
#### "Failed to stop services"
**Problem:** Docker services won't stop
**Solution:**
```bash
# Force stop all containers
docker-compose down --remove-orphans
# Or force remove
docker-compose rm -f -s -v
```
#### "Backup file not found"
**Problem:** Restore script can't find the backup
**Solution:**
```bash
# Check backup directory
ls -la /home/Dejan/Docker/Netbird-compose/backup/
# Use absolute path
./restore-netbird.sh /full/path/to/backup.tar.gz
```
#### "Permission denied"
**Problem:** Script can't be executed
**Solution:**
```bash
# Make scripts executable
chmod +x backup-netbird.sh restore-netbird.sh
# Run with sudo
sudo ./backup-netbird.sh
```
#### "No space left on device"
**Problem:** Insufficient disk space for backup
**Solution:**
```bash
# Check disk space
df -h
# Clean old backups
rm /home/Dejan/Docker/Netbird-compose/backup/netbird-backup-OLD*.tar.gz
# Clean Docker system
docker system prune -a
```
### Verify Backup Integrity
```bash
# Test if backup archive is valid
tar -tzf backup/netbird-backup-20241127_143022.tar.gz > /dev/null
# List backup contents
tar -tzf backup/netbird-backup-20241127_143022.tar.gz | less
# Extract specific file from backup (without full restore)
tar -xzf backup/netbird-backup-20241127_143022.tar.gz docker-compose.yml
```
### Manual Restore Steps
If the restore script fails, you can restore manually:
```bash
# 1. Stop services
cd /home/Dejan/Docker/Netbird-compose
docker-compose down
# 2. Backup current state (optional)
tar -czf /tmp/emergency-backup.tar.gz .
# 3. Extract backup
tar -xzf backup/netbird-backup-YYYYMMDD_HHMMSS.tar.gz
# 4. Fix permissions
chmod +x *.sh
chmod 600 acme.json
chmod 777 machinekey/
# 5. Start services
docker-compose up -d
```
---
## 📊 Script Workflow Diagrams
### Backup Process
```
Start
Check root access
Create backup directory (if needed)
Stop Docker Compose services
Create tar.gz archive with timestamp
Restart Docker Compose services
Display backup info and size
List recent backups
End
```
### Restore Process
```
Start
Check root access
Validate backup file exists
Show backup contents preview
Prompt: Create safety backup? (Y/n)
Create pre-restore backup (if yes)
Prompt: Confirm restore? (y/N)
Stop Docker Compose services
Remove current files
Extract backup archive
Set file permissions
Verify critical files
Start Docker Compose services
Display service status
End
```
---
## 📝 File Permissions Reference
After restore, these permissions are automatically set:
| File/Directory | Permission | Purpose |
|----------------|-----------|---------|
| `*.sh` | `755` (rwxr-xr-x) | Executable scripts |
| `machinekey/` | `777` (rwxrwxrwx) | Machine key storage |
| `acme.json` | `600` (rw-------) | SSL certificates |
| Other files | `644` (rw-r--r--) | Configuration files |
---
## 🆘 Support
### Getting Help
1. Check the troubleshooting section above
2. Verify Docker and Docker Compose are running
3. Check system logs: `journalctl -xe`
4. Check Docker logs: `docker-compose logs`
### Useful Commands
```bash
# Check script status
./backup-netbird.sh
./restore-netbird.sh
# View Docker services
docker-compose ps
# View Docker logs
docker-compose logs -f
# Check disk space
df -h
# List all backups with sizes
ls -lh backup/
```
---
## 📄 License
These scripts are provided as-is for use with Netbird Docker Compose installations.
---
**Last Updated:** November 2024
**Version:** 1.0
**Compatible with:** Netbird Docker Compose installations