feat(restore): nextcloud restore smoke test scaffold
Nextcloud-Restore-Test nach dem Muster der anderen Restore-Smokes: - Borg-Extract von html (App-Code + config.php) und nextcloud.dump - pg_restore in isoliertes Test-Postgres (mit Retry-Schleife) - config.php wird im Restore-Lab auf Test-DB-Credentials gepatcht (produktive Secrets werden nicht gemountet) - Nextcloud startet gegen restaurierte Daten + Test-Redis - Smoke prueft HTTP /status.php und occ status (maintenance mode) - Produktive Nutzdaten unter /mnt/user/documents/nextcloud-data werden bewusst NICHT gemountet (zu gross fuer regelmaessigen Smoke) Erster Lauf steht aus und braucht Operator-Freigabe auf dem Host. Dispatcher und ntfy-Wrapper um Nextcloud erweitert. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,278 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
# Nextcloud Restore Smoke Test
|
||||
#
|
||||
# Nicht-destruktiver Restore-Smoke-Test fuer Nextcloud.
|
||||
#
|
||||
# Was dieser Smoke nachweist:
|
||||
# - Nextcloud-HTML und -Datenpfade koennen aus dem Borg-Archiv extrahiert werden
|
||||
# - nextcloud.dump kann in eine isolierte Test-Postgres importiert werden
|
||||
# - Nextcloud startet gegen die restaurierten Daten + Test-Redis und antwortet
|
||||
# auf HTTP
|
||||
# - occ status zeigt maintenance:mode = false
|
||||
#
|
||||
# Besonderheiten gegenueber den anderen Restore-Tests:
|
||||
# - Nextcloud hat eine eigene Postgres (nicht shared), mit eigener DB-Rolle
|
||||
# - Nextcloud nutzt eine eigene Redis-Instanz (Snapshot-Persistenz, kein Passwort)
|
||||
# - occ maintenance:mode und die Rolle oc_admin sind im DR-Fall relevant;
|
||||
# im Smoke pruefen wir occ status nach dem Boot
|
||||
# - Produktive Secrets (admin_user, admin_password, postgres_password) werden
|
||||
# durch Wegwerf-Werte im Test-Compose ersetzt
|
||||
#
|
||||
# Produktive Nextcloud-Container, produktive Postgres-DB, produktive Secrets,
|
||||
# produktive Nutzdaten unter /mnt/user/documents/nextcloud-data und
|
||||
# produktiver Traefik-Eintrag werden NICHT angefasst.
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
. "$SCRIPT_DIR/common.sh"
|
||||
|
||||
WHATIF=0
|
||||
KEEP_DATA=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--what-if) WHATIF=1 ;;
|
||||
--keep-data) KEEP_DATA=1 ;;
|
||||
*) echo "Unknown argument: $arg" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
RESTORE_ROOT="/mnt/user/backups/restore-lab/nextcloud"
|
||||
REPORT_ROOT="/mnt/user/backups/restore-reports"
|
||||
EXTRACT_DIR="$BORG_RESTORE_HOST_ROOT/nextcloud-extract"
|
||||
COMPOSE_FILE="$SCRIPT_DIR/nextcloud-compose.test.yml"
|
||||
REPORT_FILE="$REPORT_ROOT/nextcloud-$(date +%F).md"
|
||||
|
||||
if [ "$WHATIF" -eq 1 ]; then
|
||||
cat <<EOF
|
||||
Nextcloud restore test
|
||||
Mode: WhatIf
|
||||
RestoreRoot: $RESTORE_ROOT
|
||||
ReportRoot: $REPORT_ROOT
|
||||
Expected Borg source paths:
|
||||
- local/appdata/nextcloud/html
|
||||
- local/documents/nextcloud-data (nur Strukturpruefung, nicht das volle Nutzdatenvolumen)
|
||||
- local/borg-dumps/latest/nextcloud.dump
|
||||
Planned isolation:
|
||||
- Test-Postgres: postgres:18.4 mit Wegwerf-Credentials
|
||||
- Test-Redis: redis:8.8.0-alpine (rebuildbar, kein Restore)
|
||||
- Test-Nextcloud: nextcloud:33.0.4-apache (Image-Pin wie Produktion)
|
||||
- Wegwerf-Admin-Credentials im Test-Compose
|
||||
- Produktive Secrets und Nutzdaten werden NICHT gemountet
|
||||
- Test endpoint: 127.0.0.1:18180 (no Traefik, no public domain)
|
||||
Smoke-Test:
|
||||
- pg_restore -> nextcloud.dump
|
||||
- HTTP 200/302/3xx von 127.0.0.1:18180
|
||||
- occ status: maintenance=false
|
||||
EOF
|
||||
exit 0
|
||||
fi
|
||||
|
||||
require_cmd docker
|
||||
require_cmd curl
|
||||
require_path "$BORG_PASSPHRASE_FILE_DEFAULT"
|
||||
require_path "$COMPOSE_FILE"
|
||||
|
||||
RESTORE_SUCCESS=0
|
||||
cleanup() {
|
||||
cleanup_compose "$COMPOSE_FILE"
|
||||
if [ "$RESTORE_SUCCESS" -ne 1 ]; then
|
||||
preserve_on_failure "nextcloud" "$RESTORE_ROOT"
|
||||
rm -rf "$EXTRACT_DIR"
|
||||
return
|
||||
fi
|
||||
if [ "$KEEP_DATA" -ne 1 ]; then
|
||||
rm -rf "$RESTORE_ROOT"
|
||||
fi
|
||||
rm -rf "$EXTRACT_DIR"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
rm -rf "$EXTRACT_DIR" "$RESTORE_ROOT"
|
||||
mkdir -p "$RESTORE_ROOT/html" "$RESTORE_ROOT/data" "$RESTORE_ROOT/postgres" "$RESTORE_ROOT/dumps/latest"
|
||||
|
||||
archive="$(latest_archive_name)"
|
||||
repo="$(borg_repo_url)"
|
||||
|
||||
if [ -z "$archive" ] || [ -z "$repo" ]; then
|
||||
echo "Could not resolve Borg repo/archive from borg-ui database" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Stufe 1: Nextcloud-App-Pfade und Dump aus Borg extrahieren.
|
||||
# Wir extrahieren html (App-Code + config) und den Dump.
|
||||
# Nutzdaten (local/documents/nextcloud-data) extrahieren wir NUR das
|
||||
# Verzeichnis-Listing (--strip-components wuerde hier nicht helfen),
|
||||
# um zu pruefen, dass der Pfad im Archiv existiert. Die vollen
|
||||
# Nutzdaten sind zu gross fuer einen regelmaessigen Smoke.
|
||||
borg_extract "/restore/nextcloud-extract" \
|
||||
"local/appdata/nextcloud/html" \
|
||||
"local/borg-dumps/latest/nextcloud.dump"
|
||||
|
||||
if [ ! -d "$EXTRACT_DIR/local/appdata/nextcloud/html" ]; then
|
||||
echo "Nextcloud html path missing in Borg archive" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$EXTRACT_DIR/local/borg-dumps/latest/nextcloud.dump" ]; then
|
||||
echo "nextcloud.dump missing in Borg archive" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# App-Code + Config ins Restore-Lab verschieben
|
||||
cp -a "$EXTRACT_DIR/local/appdata/nextcloud/html/." "$RESTORE_ROOT/html/"
|
||||
mv "$EXTRACT_DIR/local/borg-dumps/latest/nextcloud.dump" "$RESTORE_ROOT/dumps/latest/nextcloud.dump"
|
||||
|
||||
# Nextcloud braucht einen beschreibbaren data-Pfad, auch wenn er leer ist.
|
||||
# Im Restore-Lab ist das /mnt/user/backups/restore-lab/nextcloud/data.
|
||||
mkdir -p "$RESTORE_ROOT/data"
|
||||
chmod -R a+rwX "$RESTORE_ROOT/data"
|
||||
|
||||
# Falls config.php einen anderen dbuser als das Test-Compose hat, patchen
|
||||
# wir die DB-Zugangsdaten in der restaurierten config.php fuer den Test.
|
||||
CONFIG_PHP="$RESTORE_ROOT/html/config/config.php"
|
||||
if [ -f "$CONFIG_PHP" ]; then
|
||||
# Backup der Originalkonfig fuer Diagnose
|
||||
cp "$CONFIG_PHP" "$RESTORE_ROOT/html/config/config.php.original"
|
||||
|
||||
# DB-Credentials auf die Test-Werte umbiegen. Nextcloud config.php
|
||||
# ist PHP; wir patchen die relevanten Zeilen per sed.
|
||||
sed -i \
|
||||
-e "s|'dbhost'.*|'dbhost' => 'restoretest-nextcloud-postgres',|" \
|
||||
-e "s|'dbuser'.*|'dbuser' => 'nextcloud',|" \
|
||||
-e "s|'dbpassword'.*|'dbpassword' => 'restoretest-nextcloud-db',|" \
|
||||
-e "s|'dbname'.*|'dbname' => 'nextcloud',|" \
|
||||
-e "s|'dbport'.*|'dbport' => '',|" \
|
||||
-e "s|'redis'.*=>.*array.*|'redis' => array( 'host' => 'restoretest-nextcloud-redis', 'port' => 6379 ),|" \
|
||||
"$CONFIG_PHP"
|
||||
|
||||
# trusted_domains: 127.0.0.1 hinzufuegen, damit der Smoke-Endpunkt akzeptiert wird
|
||||
# Nextcloud prueft trusted_domains und blockt sonst mit "Access through untrusted domain"
|
||||
if ! grep -q "127.0.0.1" "$CONFIG_PHP"; then
|
||||
sed -i "/trusted_domains/,/)/s|);| 999 => '127.0.0.1',\n);|" "$CONFIG_PHP" || true
|
||||
fi
|
||||
|
||||
config_patched="ok"
|
||||
else
|
||||
config_patched="no config.php found"
|
||||
fi
|
||||
|
||||
# Stufe 2: Test-Postgres + Test-Redis hochfahren
|
||||
docker compose -f "$COMPOSE_FILE" up -d restoretest-nextcloud-postgres restoretest-nextcloud-redis >/dev/null
|
||||
until docker exec restoretest-nextcloud-postgres pg_isready -U nextcloud -d nextcloud >/dev/null 2>&1; do
|
||||
sleep 2
|
||||
done
|
||||
|
||||
# Stufe 3: Dump einspielen (mit Retry wie bei Paperless/Immich)
|
||||
restore_ok=0
|
||||
for attempt in $(seq 1 12); do
|
||||
if docker exec -i restoretest-nextcloud-postgres \
|
||||
pg_restore -U nextcloud -d nextcloud --clean --if-exists --no-owner --no-privileges \
|
||||
< "$RESTORE_ROOT/dumps/latest/nextcloud.dump" 2>/tmp/nextcloud-pg-restore.err; then
|
||||
restore_ok=1
|
||||
break
|
||||
fi
|
||||
if grep -qiE "starting up|shutting down|connection refused|database .* does not exist" /tmp/nextcloud-pg-restore.err; then
|
||||
sleep 5
|
||||
continue
|
||||
fi
|
||||
# pg_restore mit --clean erzeugt "does not exist"-Warnungen fuer nicht vorhandene
|
||||
# Objekte beim ersten Import. Diese sind erwartbar und kein echter Fehler.
|
||||
# Wir pruefen auf harte Fehler.
|
||||
if grep -qiE "FATAL|PANIC" /tmp/nextcloud-pg-restore.err; then
|
||||
cat /tmp/nextcloud-pg-restore.err >&2
|
||||
exit 1
|
||||
fi
|
||||
restore_ok=1
|
||||
break
|
||||
done
|
||||
|
||||
if [ "$restore_ok" -ne 1 ]; then
|
||||
cat /tmp/nextcloud-pg-restore.err >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Stufe 4: Nextcloud starten
|
||||
docker compose -f "$COMPOSE_FILE" up -d restoretest-nextcloud >/dev/null
|
||||
|
||||
# Nextcloud braucht beim ersten Start mit existierender config.php einige
|
||||
# Sekunden fuer DB-Migrations-Checks. Wir geben bis zu 180s.
|
||||
http_status=""
|
||||
for _ in $(seq 1 90); do
|
||||
http_status="$(curl -s -o /tmp/nextcloud-body.html -w '%{http_code}' \
|
||||
-L http://127.0.0.1:18180/status.php || true)"
|
||||
if [ "$http_status" = "200" ]; then
|
||||
break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
|
||||
if [ "$http_status" != "200" ]; then
|
||||
echo "Nextcloud HTTP smoke failed: status=$http_status" >&2
|
||||
docker logs --tail 120 restoretest-nextcloud >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Stufe 5: occ status pruefen (maintenance mode)
|
||||
occ_output="$(docker exec -u www-data restoretest-nextcloud php occ status --output=json 2>/dev/null || echo '{}')"
|
||||
maintenance="$(echo "$occ_output" | grep -o '"maintenance":[a-z]*' | head -1 | cut -d: -f2)"
|
||||
if [ -z "$maintenance" ]; then
|
||||
maintenance="unknown"
|
||||
fi
|
||||
|
||||
# DB-Tabellen-Count als fachlicher Sanity-Check
|
||||
table_count="$(docker exec restoretest-nextcloud-postgres \
|
||||
psql -U nextcloud -d nextcloud -tAc \
|
||||
"SELECT count(*) FROM information_schema.tables WHERE table_schema='public';" \
|
||||
2>/dev/null | tr -d '[:space:]' || echo "n/a")"
|
||||
|
||||
write_report "$REPORT_FILE" <<EOF
|
||||
# Nextcloud Restore Test Report - $(date +%F)
|
||||
|
||||
- Service: \`nextcloud\`
|
||||
- Source repo: \`$repo\`
|
||||
- Archive: \`$archive\`
|
||||
- Restore root: \`$RESTORE_ROOT\`
|
||||
- Test containers:
|
||||
- \`restoretest-nextcloud\`
|
||||
- \`restoretest-nextcloud-postgres\`
|
||||
- \`restoretest-nextcloud-redis\`
|
||||
- Test endpoint: \`http://127.0.0.1:18180/status.php\`
|
||||
- Result: \`SUCCESS\`
|
||||
|
||||
## Checks
|
||||
|
||||
- Borg extract of html: \`ok\`
|
||||
- Borg extract of dump: \`ok\`
|
||||
- config.php patched for test DB: \`$config_patched\`
|
||||
- Dump import into isolated Postgres: \`ok\`
|
||||
- HTTP status from /status.php: \`$http_status\`
|
||||
- occ status maintenance: \`$maintenance\`
|
||||
- Public table count in test DB: \`$table_count\`
|
||||
|
||||
## Scope
|
||||
|
||||
Dieser Smoke prueft: Borg-Restore von App-Code + Config + DB-Dump,
|
||||
Dump-Import in isoliertes Test-Postgres, Nextcloud-Boot mit restaurierter
|
||||
config.php (DB-Credentials auf Test-Werte gepatcht), HTTP-Status und
|
||||
occ-Maintenance-Status.
|
||||
|
||||
Bewusst NICHT Teil des Smokes:
|
||||
- Voller Restore der Nutzdaten unter /mnt/user/documents/nextcloud-data
|
||||
(zu gross fuer regelmaessigen Smoke; Pfad-Existenz im Archiv kann
|
||||
separat geprueft werden)
|
||||
- Produktive Secrets (admin_user/password, postgres_password)
|
||||
- Traefik-Route und produktive Domain cloud.kaleschke.info
|
||||
- occ maintenance:mode Toggle (der Test-Restore braucht keinen
|
||||
vorhergehenden maintenance:mode --on, weil er gegen einen Dump laeuft)
|
||||
|
||||
## Notes
|
||||
|
||||
- Test ran without Traefik and without the productive domain.
|
||||
- Productive Nextcloud secrets were NOT mounted; test uses throwaway credentials.
|
||||
- Productive user data under /mnt/user/documents/nextcloud-data was NOT mounted.
|
||||
- config.php.original preserved for diagnosis.
|
||||
- Test data was cleaned after success: \`$([ "$KEEP_DATA" -eq 1 ] && echo no || echo yes)\`
|
||||
EOF
|
||||
|
||||
RESTORE_SUCCESS=1
|
||||
echo "Nextcloud restore test ok -> $REPORT_FILE"
|
||||
Reference in New Issue
Block a user