gitea-saas-core/OPS.md

gitea-saas-core — operations (issue #40)

Containerised deployment of the billing service. The app is the assembled Flask app from issue #38 (wsgi.py -> gitea_saas.app.create_app), served by gunicorn inside a slim Python container. This document covers day-to-day operation: start/stop, backups, key rotation, and what to check when it is down.

Ledger reality check (read first)

• The ledger is a single SQLite file at /data/saas.db inside the saas_data named volume. The volume is the durability story: losing the volume loses the ledger. Nothing else holds the data.
• The store is single-writer: the Dockerfile deliberately runs exactly one gunicorn worker, because multiple writer processes on one SQLite file corrupt or lock. Never raise --workers (or run a second saas container against the same volume).
• SQLite WAL is on and writes are serialised by the app's single lock; threads are safe, workers are not.
• An untested restore is worse than no backup: run the restore procedure below against a scratch/compose copy before you ever need it, and after any change to it.

Quick start

git clone <repo> && cd <repo>
export SAAS_API_KEY='<a real secret>'     # absent key -> app boots FAIL-CLOSED (503 on everything but /health)
docker compose up -d --build
curl -fsS http://127.0.0.1:8080/health    # -> {"ok":true}

For convenience the key can live in a .env file next to docker-compose.yml (SAAS_API_KEY=<secret>) and compose will pick it up automatically. .env is local configuration — do not commit it.

Environment variables

Variable	Default	Meaning
SAAS_API_KEY	(empty -> 503s)	X-API-Key accepted by every authed route. Required before traffic.
SAAS_DB_PATH	/data/saas.db	SQLite ledger path inside the container (volume).
SAAS_PORT	8080	Port gunicorn binds inside the container.

Day to day

docker compose ps                 # state + health of the service
docker compose logs --tail=200    # recent logs (stdout/stderr)
docker compose up -d              # (re)create/start, e.g. after a deploy
docker compose restart            # restart the process
docker compose down               # stop; volume is KEPT
docker compose down -v            # !!! DESTROYS THE LEDGER — never, without a verified backup

Health endpoint is public (no API key): /health and /healthz return {"ok":true}. Authed routes need X-API-Key: <key>; with no key configured they return 503 SERVICE_UNAVAILABLE (fail-closed by design).

Backup

Online, consistent (uses SQLite's backup API, safe while the service is running — do not cp the live file):

docker compose exec saas python -c "import sqlite3; s=sqlite3.connect('/data/saas.db'); d=sqlite3.connect('/data/saas-backup.db'); s.backup(d); d.close(); s.close(); print('backup OK')"
docker compose cp saas:/data/saas-backup.db ./saas-backup-$(date +%F).db

Check the archive landed and is readable before moving on:

ls -l saas-backup-*.db
docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone()); print(c.execute('select count(*) from clients').fetchone()[0], 'clients')"

Keep the copy off-machine (the compose down /data copy dies with the host). A crontab line like docker compose ... exec saas python -c "..." && docker compose cp ... off-site is enough.

Restore (tested end-to-end once — see the issue PR for the recorded run)

Run from the directory that contains the backup file. gitea-saas is the compose project name (pinned in docker-compose.yml), so the volume is gitea-saas_saas_data.

The container runs as the non-root app user, so the directory holding the backup file must be traversable by it (e.g. mode 0755, file 0644); a backup stashed in a private home directory will fail with "Permission denied".

docker compose down                     # no writer while we swap files
docker run --rm -v gitea-saas_saas_data:/data -v "$(pwd)":/restore \
  gitea-saas-core:local sh -c "rm -f /data/saas.db-wal /data/saas.db-shm && cp /restore/saas-backup.db /data/saas.db"
docker compose up -d

Why the rm of -wal/-shm: the running app leaves WAL sidecar files; if they survived when we replace saas.db, SQLite would replay stale frames onto the restored file. Removing them first is required, not cosmetic.

Verify the restored ledger:

curl -fsS http://127.0.0.1:8080/health
docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone()); print('clients:', c.execute('select coalesce(group_concat(name, \", \"), \"(none)\") from clients').fetchone()[0]); print('ledger rows:', c.execute('select count(*) from ledger').fetchone()[0])"

Rotate SAAS_API_KEY

docker compose exec saas env            # sanity: what the container sees
# edit .env (or re-export) with the new key, then:
docker compose up -d                    # recreates the container with the new env

Verify the old key now 401s and the new one works:

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/api/clients -H 'X-API-Key: old-key'
curl -s http://127.0.0.1:8080/api/clients -H 'X-API-Key: new-key'

When it is down — checklist

1. docker compose ps — is the container running? restart: unless-stopped brings it back after most exits; a container stuck in Restarting loops is crashing at boot.
2. docker compose logs --tail=200 saas — traceback? DB path error? PermissionError on /data? If the volume mount is missing the app still starts but the ledger is ephemeral — check ls inside.
3. curl -fsS http://127.0.0.1:8080/health — health is public and keyless; a time-out here while the container is up means the app is not listening (bind/port issue): docker compose port saas 8080, ss -tlnp | grep 8080.
4. Authed calls 401? Wrong/rotated key. Authed calls 503? SAAS_API_KEY is unset/empty in the container env — fail-closed on purpose.
5. Ledger questions: docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone())" — should print ('ok',). Check the volume's disk space: df -h /var/lib/docker/volumes/gitea-saas_saas_data/_data.
6. Still broken and the ledger looks intact? Restore from the last verified backup (above) after confirming the backup file itself passes integrity_check.

Scaling (and why not to do it casually)

This deployment is intentionally a single SQLite writer. To grow you must move the ledger off SQLite onto a network-capable database (Postgres etc.) first — raising --workers against SQLite will corrupt or lock the ledger. Until then: one worker, threads for concurrency, backups before traffic.