rewrite readme.md file

scripts/readme.md

@@ -1,252 +1,507 @@
-# NetBird Backup & Restore Scripts
+# Netbird Docker Compose - Backup & Restore Guide
 
-Comprehensive backup and restore solution for NetBird Docker deployments with Zitadel integration.
+Complete documentation for backing up and restoring your Netbird Docker Compose installation.
-
-## Overview
-
-These scripts provide automated backup and restore functionality for your NetBird installation, including:
-- Docker volumes (database, management data, certificates)
-- Configuration files (docker-compose, environment variables)
-- Traefik reverse proxy configuration
-- Zitadel authentication data
-
-## Prerequisites
-
-- Docker and Docker Compose installed
-- NetBird deployed via Docker Compose
-- Sufficient disk space for backups
-- Root or sudo access for Docker operations
-
-## Files
-
-### 1. `backup-netbird.sh`
-Creates timestamped backups of your complete NetBird installation.
-
-### 2. `restore-netbird.sh`
-Restores NetBird from a backup archive.
-
-## Installation
-
-1. Place both scripts in your NetBird installation directory:
-   ```bash
-   /home/Dejan/Docker/Netbird-compose/
-   ```
-
-2. Make scripts executable:
-   ```bash
-   chmod +x backup-netbird.sh restore-netbird.sh
-   ```
-
-3. Ensure the backup directory exists:
-   ```bash
-   mkdir -p /home/Dejan/Docker/Netbird-compose/backup
-   ```
-
-## Usage
-
-### Creating a Backup
-
-Run the backup script from your NetBird directory:
-
-```bash
-cd /home/Dejan/Docker/Netbird-compose
-./backup-netbird.sh
-```
-
-**What happens during backup:**
-1. Creates backup directory if it doesn't exist
-2. Stops the NetBird Docker stack
-3. Exports all Docker volumes to compressed archives
-4. Archives configuration files
-5. Restarts the NetBird stack
-6. Creates a timestamped backup file: `netbird_backup_YYYY-MM-DD_HH-MM-SS.tar.gz`
-
-**Backup includes:**
-- Docker volumes: `netbird_zdb_data`, `netbird_management`, `netbird_zitadel_certs`
-- Configuration: `docker-compose.yml`, `*.env` files
-- NetBird config: `management.json`, `turnserver.conf`
-- Authentication: `machinekey` directory
-- Reverse proxy: `traefik-stack` directory
-- The backup script itself
-
-### Restoring from Backup
-
-#### Option 1: Restore latest backup (automatic)
-```bash
-cd /home/Dejan/Docker/Netbird-compose
-./restore-netbird.sh
-```
-
-#### Option 2: Restore specific backup
-```bash
-./restore-netbird.sh netbird_backup_2024-11-24_14-30-00.tar.gz
-```
-
-Or with full path:
-```bash
-./restore-netbird.sh /home/Dejan/Docker/Netbird-compose/backup/netbird_backup_2024-11-24_14-30-00.tar.gz
-```
-
-**What happens during restore:**
-1. Validates backup file exists
-2. Extracts backup to temporary directory
-3. Stops current NetBird stack
-4. Restores all Docker volumes
-5. Restores configuration files
-6. Cleans up temporary files
-7. Starts NetBird stack with restored data
-
-## Backup Schedule
-
-### Manual Backups
-Run before major changes:
-- System updates
-- Configuration modifications
-- Docker Compose upgrades
-
-### Automated Backups (Recommended)
-
-Add to crontab for automatic daily backups:
-
-```bash
-crontab -e
-```
-
-Add this line for daily backup at 2 AM:
-```cron
-0 2 * * * /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
-```
-
-For weekly backups (Sunday at 3 AM):
-```cron
-0 3 * * 0 /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
-```
-
-## Backup Management
-
-### View Available Backups
-```bash
-ls -lh /home/Dejan/Docker/Netbird-compose/backup/
-```
-
-### Delete Old Backups
-Keep only last 7 backups:
-```bash
-cd /home/Dejan/Docker/Netbird-compose/backup
-ls -1t netbird_backup_*.tar.gz | tail -n +8 | xargs rm -f
-```
-
-### Backup Retention Script
-Create `cleanup-old-backups.sh`:
-```bash
-#!/bin/bash
-BACKUP_DIR="/home/Dejan/Docker/Netbird-compose/backup"
-KEEP_LAST=7
-cd "$BACKUP_DIR"
-ls -1t netbird_backup_*.tar.gz | tail -n +$((KEEP_LAST + 1)) | xargs -r rm -f
-echo "Cleaned up old backups, kept last $KEEP_LAST"
-```
-
-## Troubleshooting
-
-### Backup Script Fails
-- **Insufficient disk space**: Check available space with `df -h`
-- **Docker not running**: Start Docker service: `sudo systemctl start docker`
-- **Permission denied**: Run with sudo: `sudo ./backup-netbird.sh`
-
-### Restore Script Fails
-- **Backup file not found**: Verify path and filename
-- **Volume restore fails**: Check Docker is running and volumes are accessible
-- **Stack won't start**: Check logs: `docker compose logs`
-
-### Common Issues
-
-**Stack doesn't start after restore:**
-```bash
-# Check container logs
-docker compose logs -f
-
-# Verify volumes exist
-docker volume ls | grep netbird
-
-# Try manual restart
-docker compose down
-docker compose up -d
-```
-
-**Configuration mismatch:**
-```bash
-# Verify restored files
-ls -la /home/Dejan/Docker/Netbird-compose/
-
-# Check environment variables
-cat .env
-```
-
-## Important Notes
-
-- **Downtime**: Both backup and restore scripts stop the NetBird stack temporarily
-- **Backup size**: Varies based on database size and user count (typically 100MB-1GB)
-- **Storage**: Ensure adequate backup storage space
-- **Testing**: Test restore process in a development environment first
-- **Security**: Backup files contain sensitive data - secure appropriately
-
-## Backup Security
-
-Protect your backups:
-
-1. **Restrict permissions:**
-   ```bash
-   chmod 700 /home/Dejan/Docker/Netbird-compose/backup
-   chmod 600 /home/Dejan/Docker/Netbird-compose/backup/*.tar.gz
-   ```
-
-2. **Encrypt backups (optional):**
-   ```bash
-   gpg --symmetric --cipher-algo AES256 netbird_backup_*.tar.gz
-   ```
-
-3. **Off-site backup:**
-   ```bash
-   # Example: rsync to remote server
-   rsync -avz backup/ user@backup-server:/backups/netbird/
-   ```
-
-## Migration Guide
-
-To migrate NetBird to a new server:
-
-1. **On old server:** Create backup
-   ```bash
-   ./backup-netbird.sh
-   ```
-
-2. **Transfer backup file:**
-   ```bash
-   scp backup/netbird_backup_*.tar.gz user@new-server:/tmp/
-   ```
-
-3. **On new server:** Install Docker, Docker Compose, and NetBird structure
-
-4. **Restore backup:**
-   ```bash
-   ./restore-netbird.sh /tmp/netbird_backup_*.tar.gz
-   ```
-
-5. **Update DNS/IP** if necessary
-
-## Support
-
-For issues or questions:
-- Check NetBird documentation: https://docs.netbird.io
-- Review Docker Compose logs: `docker compose logs`
-- Verify volume integrity: `docker volume inspect <volume_name>`
-
-## Version History
-
-- **v1.0** - Initial backup and restore scripts with full volume and config support
 
 ---
 
-**Created for NetBird deployment at:** `/home/Dejan/Docker/Netbird-compose/`  
+## 📋 Table of Contents
-**Last updated:** November 2024
+
+- [Overview](#overview)
+- [Prerequisites](#prerequisites)
+- [Backup Script](#backup-script)
+- [Restore Script](#restore-script)
+- [Automated Backups](#automated-backups)
+- [Best Practices](#best-practices)
+- [Troubleshooting](#troubleshooting)
+
+---
+
+## 🎯 Overview
+
+This package includes two powerful scripts to manage your Netbird installation:
+
+- **`backup-netbird.sh`** - Creates timestamped backups of your entire Netbird setup
+- **`restore-netbird.sh`** - Restores your Netbird installation from any backup
+
+Both scripts handle Docker services automatically and include comprehensive error handling.
+
+---
+
+## ⚙️ Prerequisites
+
+- Root or sudo access
+- Docker and Docker Compose installed
+- Netbird installation at `/home/Dejan/Docker/Netbird-compose`
+- Sufficient disk space for backups
+
+---
+
+## 💾 Backup Script
+
+### What Gets Backed Up
+
+The backup script saves:
+- All configuration files (`.env`, `dashboard.env`, `zitadel.env`, etc.)
+- Docker Compose configuration (`docker-compose.yml`)
+- SSL certificates and keys (`acme.json`, `machinekey/`)
+- Management configuration (`management.json`)
+- Traefik configuration files
+- All custom scripts
+- Zitadel logs
+
+**Excluded from backups:**
+- The `backup/` directory itself
+- `.git/` directory (if present)
+- Old `.tar.gz` files
+
+### How to Use
+
+#### Basic Usage
+
+```bash
+# Navigate to your Netbird directory
+cd /home/Dejan/Docker/Netbird-compose
+
+# Make the script executable (first time only)
+chmod +x backup-netbird.sh
+
+# Run the backup
+./backup-netbird.sh
+```
+
+#### What Happens During Backup
+
+1. **Validation** - Checks for root access and directory existence
+2. **Service Stop** - Gracefully stops all Docker Compose services
+3. **Archive Creation** - Creates a compressed tar.gz archive with timestamp
+4. **Service Restart** - Brings all services back online
+5. **Verification** - Shows backup size and location
+
+### Backup Output
+
+Backups are saved in the `backup/` directory with timestamps:
+
+```
+backup/
+├── netbird-backup-20241127_143022.tar.gz
+├── netbird-backup-20241127_180530.tar.gz
+└── netbird-backup-20241128_020000.tar.gz
+```
+
+**Filename format:** `netbird-backup-YYYYMMDD_HHMMSS.tar.gz`
+
+### Backup Script Features
+
+- ✅ Automatic service management
+- ✅ Timestamped backups
+- ✅ Colored console output
+- ✅ Size reporting
+- ✅ Error handling at each step
+- ✅ Automatic backup directory creation
+- ✅ List of recent backups after completion
+
+### Optional: Automatic Cleanup
+
+To keep only the last 5 backups, uncomment these lines in the script:
+
+```bash
+KEEP_BACKUPS=5
+print_msg "Cleaning old backups, keeping last $KEEP_BACKUPS..."
+cd "$BACKUP_DIR"
+ls -t netbird-backup-*.tar.gz | tail -n +$((KEEP_BACKUPS + 1)) | xargs -r rm
+```
+
+---
+
+## 🔄 Restore Script
+
+### How to Use
+
+#### Basic Usage
+
+```bash
+# Make the script executable (first time only)
+chmod +x restore-netbird.sh
+
+# Restore from a specific backup
+./restore-netbird.sh netbird-backup-20241127_143022.tar.gz
+```
+
+#### Alternative Ways to Specify Backup
+
+```bash
+# Using just the filename (searches in backup/ directory)
+./restore-netbird.sh netbird-backup-20241127_143022.tar.gz
+
+# Using relative path
+./restore-netbird.sh backup/netbird-backup-20241127_143022.tar.gz
+
+# Using absolute path
+./restore-netbird.sh /home/Dejan/Docker/Netbird-compose/backup/netbird-backup-20241127_143022.tar.gz
+```
+
+#### List Available Backups
+
+```bash
+# Show usage and list available backups
+./restore-netbird.sh
+```
+
+### What Happens During Restore
+
+1. **Validation** - Verifies backup file exists and is valid
+2. **Preview** - Shows backup contents (first 20 files)
+3. **Safety Backup** - Creates a pre-restore backup (recommended)
+4. **Confirmation** - Asks for your approval before proceeding
+5. **Service Stop** - Stops all running Docker services
+6. **File Removal** - Removes current files (except backup/ directory)
+7. **Extraction** - Extracts all files from backup archive
+8. **Permissions** - Sets correct file permissions
+9. **Verification** - Checks that critical files were restored
+10. **Service Start** - Brings all services back online
+11. **Status Report** - Shows running services status
+
+### Interactive Prompts
+
+The restore script will ask you two questions:
+
+```
+Do you want to create a safety backup before restoring? (recommended) [Y/n]
+```
+**Recommended:** Press Enter or type `Y` to create a safety backup
+
+```
+Are you sure you want to restore from this backup? [y/N]
+```
+**Type `y`** to proceed with the restore
+
+### Restore Script Features
+
+- ✅ Multiple backup file path formats supported
+- ✅ Automatic backup file search
+- ✅ Pre-restore safety backup
+- ✅ Backup content preview
+- ✅ Interactive confirmation prompts
+- ✅ Automatic permission fixing
+- ✅ Critical file verification
+- ✅ Service status display
+- ✅ Comprehensive error handling
+
+### Safety Backups
+
+When you create a safety backup, it's saved as:
+
+```
+backup/pre-restore-backup-YYYYMMDD_HHMMSS.tar.gz
+```
+
+This allows you to roll back if something goes wrong during restore.
+
+---
+
+## ⏰ Automated Backups
+
+### Using Cron
+
+Set up automatic backups using crontab:
+
+```bash
+# Edit root's crontab
+sudo crontab -e
+
+# Add one of these lines:
+
+# Daily backup at 2:00 AM
+0 2 * * * /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
+
+# Every 6 hours
+0 */6 * * * /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
+
+# Weekly on Sunday at 3:00 AM
+0 3 * * 0 /home/Dejan/Docker/Netbird-compose/backup-netbird.sh >> /var/log/netbird-backup.log 2>&1
+```
+
+### Cron Schedule Examples
+
+| Schedule | Cron Expression | Description |
+|----------|----------------|-------------|
+| Every day at 2 AM | `0 2 * * *` | Daily backup during low traffic |
+| Every 12 hours | `0 */12 * * *` | Twice daily backups |
+| Every 6 hours | `0 */6 * * *` | Four times daily |
+| Weekly (Sunday 3 AM) | `0 3 * * 0` | Weekly backup |
+| Every hour | `0 * * * *` | Hourly backup (generates many files) |
+
+### Monitoring Automated Backups
+
+```bash
+# View backup log
+tail -f /var/log/netbird-backup.log
+
+# List recent backups
+ls -lht /home/Dejan/Docker/Netbird-compose/backup/ | head -10
+
+# Check cron is running
+sudo systemctl status cron
+
+# View crontab entries
+sudo crontab -l
+```
+
+---
+
+## 🎯 Best Practices
+
+### Backup Strategy
+
+1. **Regular Backups**
+   - Set up daily automated backups via cron
+   - Create manual backups before major changes
+
+2. **Retention Policy**
+   - Keep at least 7 daily backups
+   - Keep at least 4 weekly backups
+   - Store critical backups off-site
+
+3. **Before Updates**
+   - Always backup before updating Docker images
+   - Always backup before modifying configuration
+   - Always backup before system maintenance
+
+4. **Test Restores**
+   - Periodically test restore process
+   - Verify backup integrity
+   - Document restore procedures
+
+### Storage Management
+
+```bash
+# Check backup directory size
+du -sh /home/Dejan/Docker/Netbird-compose/backup/
+
+# Find backups older than 30 days
+find /home/Dejan/Docker/Netbird-compose/backup/ -name "*.tar.gz" -mtime +30
+
+# Remove backups older than 30 days (use with caution!)
+find /home/Dejan/Docker/Netbird-compose/backup/ -name "*.tar.gz" -mtime +30 -delete
+```
+
+### Security
+
+- Keep backups in a secure location
+- Consider encrypting sensitive backups
+- Restrict backup file permissions: `chmod 600 *.tar.gz`
+- Store off-site copies for disaster recovery
+
+---
+
+## 🔧 Troubleshooting
+
+### Common Issues
+
+#### "Failed to stop services"
+
+**Problem:** Docker services won't stop
+
+**Solution:**
+```bash
+# Force stop all containers
+docker-compose down --remove-orphans
+
+# Or force remove
+docker-compose rm -f -s -v
+```
+
+#### "Backup file not found"
+
+**Problem:** Restore script can't find the backup
+
+**Solution:**
+```bash
+# Check backup directory
+ls -la /home/Dejan/Docker/Netbird-compose/backup/
+
+# Use absolute path
+./restore-netbird.sh /full/path/to/backup.tar.gz
+```
+
+#### "Permission denied"
+
+**Problem:** Script can't be executed
+
+**Solution:**
+```bash
+# Make scripts executable
+chmod +x backup-netbird.sh restore-netbird.sh
+
+# Run with sudo
+sudo ./backup-netbird.sh
+```
+
+#### "No space left on device"
+
+**Problem:** Insufficient disk space for backup
+
+**Solution:**
+```bash
+# Check disk space
+df -h
+
+# Clean old backups
+rm /home/Dejan/Docker/Netbird-compose/backup/netbird-backup-OLD*.tar.gz
+
+# Clean Docker system
+docker system prune -a
+```
+
+### Verify Backup Integrity
+
+```bash
+# Test if backup archive is valid
+tar -tzf backup/netbird-backup-20241127_143022.tar.gz > /dev/null
+
+# List backup contents
+tar -tzf backup/netbird-backup-20241127_143022.tar.gz | less
+
+# Extract specific file from backup (without full restore)
+tar -xzf backup/netbird-backup-20241127_143022.tar.gz docker-compose.yml
+```
+
+### Manual Restore Steps
+
+If the restore script fails, you can restore manually:
+
+```bash
+# 1. Stop services
+cd /home/Dejan/Docker/Netbird-compose
+docker-compose down
+
+# 2. Backup current state (optional)
+tar -czf /tmp/emergency-backup.tar.gz .
+
+# 3. Extract backup
+tar -xzf backup/netbird-backup-YYYYMMDD_HHMMSS.tar.gz
+
+# 4. Fix permissions
+chmod +x *.sh
+chmod 600 acme.json
+chmod 777 machinekey/
+
+# 5. Start services
+docker-compose up -d
+```
+
+---
+
+## 📊 Script Workflow Diagrams
+
+### Backup Process
+
+```
+Start
+  ↓
+Check root access
+  ↓
+Create backup directory (if needed)
+  ↓
+Stop Docker Compose services
+  ↓
+Create tar.gz archive with timestamp
+  ↓
+Restart Docker Compose services
+  ↓
+Display backup info and size
+  ↓
+List recent backups
+  ↓
+End
+```
+
+### Restore Process
+
+```
+Start
+  ↓
+Check root access
+  ↓
+Validate backup file exists
+  ↓
+Show backup contents preview
+  ↓
+Prompt: Create safety backup? (Y/n)
+  ↓
+Create pre-restore backup (if yes)
+  ↓
+Prompt: Confirm restore? (y/N)
+  ↓
+Stop Docker Compose services
+  ↓
+Remove current files
+  ↓
+Extract backup archive
+  ↓
+Set file permissions
+  ↓
+Verify critical files
+  ↓
+Start Docker Compose services
+  ↓
+Display service status
+  ↓
+End
+```
+
+---
+
+## 📝 File Permissions Reference
+
+After restore, these permissions are automatically set:
+
+| File/Directory | Permission | Purpose |
+|----------------|-----------|---------|
+| `*.sh` | `755` (rwxr-xr-x) | Executable scripts |
+| `machinekey/` | `777` (rwxrwxrwx) | Machine key storage |
+| `acme.json` | `600` (rw-------) | SSL certificates |
+| Other files | `644` (rw-r--r--) | Configuration files |
+
+---
+
+## 🆘 Support
+
+### Getting Help
+
+1. Check the troubleshooting section above
+2. Verify Docker and Docker Compose are running
+3. Check system logs: `journalctl -xe`
+4. Check Docker logs: `docker-compose logs`
+
+### Useful Commands
+
+```bash
+# Check script status
+./backup-netbird.sh
+./restore-netbird.sh
+
+# View Docker services
+docker-compose ps
+
+# View Docker logs
+docker-compose logs -f
+
+# Check disk space
+df -h
+
+# List all backups with sizes
+ls -lh backup/
+```
+
+---
+
+## 📄 License
+
+These scripts are provided as-is for use with Netbird Docker Compose installations.
+
+---
+
+**Last Updated:** November 2024  
+**Version:** 1.0  
+**Compatible with:** Netbird Docker Compose installations