Fixing Incomplete Rsync Backups — From Bash Pitfalls to a Reliable Python Script

While automating daily backups with rsync, I noticed that only the first directory was copied — the rest of the folders remained empty. At first glance, the options looked correct, yet the problem persisted across several servers. The root cause turned out to be a combination of Bash behavior and environment differences that made the process unreliable.

The Symptom

The Bash script executed three rsync commands to copy /home, /var/www, and /etc. Only the first one succeeded. No error message appeared, but the script stopped silently. Here is a simplified extract of the failing part:

run_set() {
  local dest="$1"
  mkdir -p "$dest/home_admin" "$dest/var_www" "$dest/etc"

  rsync "${OPTS[@]}" "$SRC_HOME" "$dest/home_admin/"
  rsync "${OPTS[@]}" "$SRC_WWW" "$dest/var_www/"
  rsync "${OPTS[@]}" "$SRC_ETC" "$dest/etc/"
}

Each rsync returned exit code 23 or 24 (“partial transfer” or “vanished files”) — normal for active systems — but with set -e enabled, Bash aborted the script after the first call.

Root Cause

set -e behavior: Bash exits on any non-zero exit code.
rsync exit codes 23/24: harmless warnings (changed or disappeared files).
Python 2.7 environment: the system still defaulted to Python 2.7, which caused compatibility issues when running modern scripts.

The combination of those factors caused the backup script to fail silently after the first transfer. The fix required two steps:

Wrap rsync calls in a “safe” handler that allows exit codes 0, 23, 24.
Replace the fragile Bash logic with a clean Python script compatible with Python 2.7 (or Python 3 on newer systems).

The Solution — Python 2.7 Compatible Rsync Backup Script

Below is the working version that resolved the issue. It creates weekday-based backups (Monday → Sunday) and a monthly snapshot on the first day of each month. It works both locally (CIFS mount) and remotely (SSH, e.g., Hetzner Storage Box).

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Python 2.7 compatible rsync backup
import os, sys, subprocess
from datetime import datetime

SRC_SETS = [
    ("/home/admin/", "home_admin"),
    ("/var/www/", "var_www"),
    ("/etc/", "etc"),
]

LOCAL_TARGET_BASE = "/mnt/backupbox/vindazo_nl_web_backups"
REMOTE_HOST = ""  # or "u500444@u500444.your-storagebox.de"
REMOTE_PORT = 23
REMOTE_BASE = "/backup/vindazo_nl_web_backups"

RSYNC_OPTS = [
    "-aH", "--delete", "--partial", "--stats",
    "--no-perms", "--numeric-ids", "--inplace",
    "--no-whole-file", "--copy-links", "--copy-dirlinks"
]

def rsync_safe(src, dest, extra):
    cmd = ["rsync"] + RSYNC_OPTS + extra + [src, dest]
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    out, _ = p.communicate()
    sys.stdout.write(out or "")
    if p.returncode not in (0, 23, 24):
        raise RuntimeError("rsync failed (%d)" % p.returncode)

# simplified main
weekday = datetime.now().strftime("%A")
today = datetime.now().strftime("%Y-%m-%d")
dest = "%s/weekday/%s" % (LOCAL_TARGET_BASE, weekday)
for src, name in SRC_SETS:
    target = "%s/%s/" % (dest, name)
    if not os.path.isdir(target): os.makedirs(target)
    rsync_safe(src, target, [])
print("Backup completed on %s" % today)

Advantages of the Python Approach

No unexpected set -e aborts; all rsync calls run independently.
Clearer error handling and logging via Python exceptions.
Compatible with both local CIFS mounts and SSH (Hetzner StorageBox).
Easier extension (monthly rotation, exclude lists, e-mail alerts).

Conclusion

This change eliminated all incomplete transfers and made daily backups predictable again. In production environments, it is worth replacing shell-based logic with a structured Python script that explicitly handles rsync return codes and environment differences.

Author: Sergej Dergatsjev — Online Solutions Group BV

Search This Blog