pbwraith/gitea-saas-core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: pbwraith:main
Branches Tags
pbwraith:main
pbwraith:issue-41
pbwraith:issue-40
pbwraith:issue-48
pbwraith:issue-42
pbwraith:issue-45
pbwraith:issue-39
pbwraith:issue-38
pbwraith:issue-30
pbwraith:issue-29
pbwraith:issue-28
pbwraith:issue-24
pbwraith:issue-25
pbwraith:issue-9
pbwraith:issue-6
pbwraith:issue-8
pbwraith:issue-4
pbwraith:issue-7
pbwraith:issue-3
pbwraith:issue-5
pbwraith:issue-21
pbwraith:issue-2
pbwraith:issue-16
pbwraith:issue-1
..
compare: pbwraith:issue-9
Branches Tags
pbwraith:main
pbwraith:issue-41
pbwraith:issue-40
pbwraith:issue-48
pbwraith:issue-42
pbwraith:issue-45
pbwraith:issue-39
pbwraith:issue-38
pbwraith:issue-30
pbwraith:issue-29
pbwraith:issue-28
pbwraith:issue-24
pbwraith:issue-25
pbwraith:issue-9
pbwraith:issue-6
pbwraith:issue-8
pbwraith:issue-4
pbwraith:issue-7
pbwraith:issue-3
pbwraith:issue-5
pbwraith:issue-21
pbwraith:issue-2
pbwraith:issue-16
pbwraith:issue-1

No commits in common. "main" and "issue-9" have entirely different histories.
main ... issue-9

64 changed files with 225 additions and 8209 deletions
Show Stats Expand all files Collapse all files

17
.dockerignore
View File

@ -1,17 +0,0 @@
# Build context: runtime files only. The container runs the app, not
# the repo's dev/test surface, and it must not re-pull git or cached
# bytecode into the image.
.git
.gitignore
.gitattributes
__pycache__
*.py[cod]
*.egg-info
.pytest_cache
tests
venv
.venv
OPS.md
Dockerfile
docker-compose.yml
.dockerignore

79
CALIBRATION.md
View File

@ -1,79 +0,0 @@
# CALIBRATION.md — review-gate false-approve rate (issue #42)
## Outcome
The number the gold tier's auto-merge promise needs: **how often the review
gate approves a change it should have bounced.**
Measured over a corpus of **25 real farm PRs with known outcomes** (drawn
from 11 pbwraith repos' merged and bounced history) plus the two
owner-documented regressions the issue cites (botmanager#223's shipper and
botmanager !22). The gate under test is the shipped
`gitea_saas.reviewer.Reviewer` at its DEFAULT (deterministic) config, run
unchanged against recorded inputs — issue body, changed files, diff, and the
observed test/hygiene exit from farm history.
## The number
| rate | value |
|---|---|
| false-approve rate (approved, should have bounced) | **0 / 14 = 0.0%** |
| false-approve prevalence (over 25 reviewable PRs) | 0.0% |
| false-bounce rate (bounced, should have approved) | n/a (0 bounces produced) |
| unverifiable — NEEDS_GUIDANCE (fail-closed refusal) | **11 / 25 = 44.0%** |
On the operator's own terms — the two owner-documented cases added to the 14
the gate can evaluate — the false-approve rate is **2 / 16 = 12.5%**.
## Recommendation: NO
Gold-tier auto-merge (**auto-merges on a green APPROVE**) is **not safe to
sell at this rate**. The 0.0% headline is an artifact of refusal, not of
gate quality:
1. The gate approved only the 14 corpus PRs whose issues carry `- [ ]`
checklists (or a section titled exactly `## Acceptance`) and refused the
other 11 (44%) — real merged or bounced work it could not verify. A gate
that cannot evaluate 44% of real issues cannot back an auto-merge
guarantee.
2. The approve path has **no check that would have stopped either documented
regression**: botmanager#223's break was green under the configured gate
(pytest gating a frontend-only change; green on a case-sensitive Linux
worktree while the target is case-insensitive Windows) and botmanager !22
was green in isolation (no sibling-merge/integration verification). Both
mechanisms survive in the approve path today.
3. Accident of format, not fail-closed design: botmanager#220 (the shipper)
reached NEEDS_GUIDANCE only because its issue titles the section
`## Acceptance criteria` with a numbered list, which
`parse_acceptance()` does not recognise. With the issue's own real
acceptance items parsed (one-line parser accommodation, measured
sensitivity), the gate returns **APPROVE on that head — a false approve**.
Before selling gold auto-merge: (a) close the acceptance-format coverage
gap, (b) select the test command/suite from the changed-file set (the
gitea-worker-ops !44 direction), (c) add sibling-merge/integration
verification, then (d) re-run this calibration on the fixed gate and a
larger corpus.
## How to re-run
From the repo root (one line, no network, deterministic):
python tests/calibration/run_calibration.py
It re-runs the existing reviewer against the committed corpus
(`tests/calibration/corpus/*.json` + `tests/calibration/manifest.json`) and
rewrites this report of record: `tests/calibration/calibration_report.md`.
Corpus provenance: the raw Gitea snapshots were fetched read-only on
2026-09-01 (see `tests/calibration/build_corpus.py`); the committed corpus
is the frozen snapshot the numbers are derived from.
## Scope compliance
- New files only: `tests/calibration/` (corpus, runner, builder, report,
tests) and this file.
- `gitea_saas/reviewer.py` and `gitea_saas/review_policy.py` are **read-only**
— not modified. The calibration measures the existing gate; it does not
tune it (tuning against a corpus you also authored is how you get a good
number and a bad reviewer).

49
Dockerfile
View File

@ -1,49 +0,0 @@
# gitea-saas-core container image (issue #40).
#
# Slim Python base, non-root app user, gunicorn serving the repo-root
# wsgi.py entrypoint (``gunicorn wsgi:application``, assembled app from
# issue #38).
#
# Ledger-safety: exactly ONE gunicorn worker. This process owns a SQLite
# money ledger; the DB is WAL-mode with writes serialised by BillingCore's
# single in-process lock (gitea_saas/db.py + billing_core). Multiple
# workers would be multiple writer processes racing on one SQLite file —
# WAL permits one writer at a time but a ledger must not depend on SQLITE_BUSY
# backoff at the file level. Threads are safe: every request in this process
# funnels through that one lock. Scaling to more writer processes requires a
# network database (see OPS.md "Scaling"); it is deliberately out of scope here.
FROM python:3.12-slim
# Runtime deps only. flask matches requirements.txt's "flask>=3.0"; gunicorn
# is the WSGI server and is installed here rather than added to
# requirements.txt, which belongs to earlier issues. pytest stays out of the
# image. The ``stripe`` package is NOT installed: the adapter only imports it
# lazily and CI/dev run against the built-in mock seam.
RUN pip install --no-cache-dir "flask>=3.0" "gunicorn>=23.0"
# Non-root runtime user.
RUN groupadd --system app && useradd --system --gid app --home-dir /app app
WORKDIR /app
COPY . /app
ENV SAAS_PORT=8080 \
SAAS_DB_PATH=/data/saas.db \
PYTHONPATH=/app \
PYTHONUNBUFFERED=1
# /data is where the SQLite ledger lives. In compose it is a named volume —
# that volume IS the durability story (see docker-compose.yml).
RUN mkdir -p /data && chown -R app:app /app /data
USER app
EXPOSE 8080
# /health is public (auth.PUBLIC_PATHS) and answers without an API key, so
# the check works in every deployment state, including the fail-closed
# unconfigured one. urllib only: the slim base has no curl.
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD python -c "import sys,urllib.request; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=5).status == 200 else 1)"
CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--workers", "1", "--threads", "4", \
"--access-logfile", "-", "--error-logfile", "-", "wsgi:application"]

132
OPS.md
View File

@ -1,132 +0,0 @@
# gitea-saas-core — operations (issue #40)
Containerised deployment of the billing service. The app is the assembled
Flask app from issue #38 (`wsgi.py` -> `gitea_saas.app.create_app`), served
by gunicorn inside a slim Python container. This document covers day-to-day
operation: start/stop, backups, key rotation, and what to check when it is
down.
## Ledger reality check (read first)
- The ledger is a **single SQLite file** at `/data/saas.db` inside the
`saas_data` named volume. The volume **is** the durability story: losing
the volume loses the ledger. Nothing else holds the data.
- The store is **single-writer**: the Dockerfile deliberately runs exactly
one gunicorn worker, because multiple writer processes on one SQLite file
corrupt or lock. Never raise `--workers` (or run a second `saas`
container against the same volume).
- SQLite WAL is on and writes are serialised by the app's single lock;
threads are safe, workers are not.
- An untested restore is worse than no backup: run the restore procedure
below against a scratch/compose copy before you ever need it, and after
any change to it.
## Quick start
git clone <repo> && cd <repo>
export SAAS_API_KEY='<a real secret>' # absent key -> app boots FAIL-CLOSED (503 on everything but /health)
docker compose up -d --build
curl -fsS http://127.0.0.1:8080/health # -> {"ok":true}
For convenience the key can live in a `.env` file next to
`docker-compose.yml` (`SAAS_API_KEY=<secret>`) and compose will pick it up
automatically. `.env` is local configuration — do not commit it.
## Environment variables
| Variable | Default | Meaning |
|-----------------|------------------|------------------------------------------------------|
| `SAAS_API_KEY` | (empty -> 503s) | X-API-Key accepted by every authed route. Required before traffic. |
| `SAAS_DB_PATH` | `/data/saas.db` | SQLite ledger path inside the container (volume). |
| `SAAS_PORT` | `8080` | Port gunicorn binds inside the container. |
## Day to day
docker compose ps # state + health of the service
docker compose logs --tail=200 # recent logs (stdout/stderr)
docker compose up -d # (re)create/start, e.g. after a deploy
docker compose restart # restart the process
docker compose down # stop; volume is KEPT
docker compose down -v # !!! DESTROYS THE LEDGER — never, without a verified backup
Health endpoint is public (no API key): `/health` and `/healthz` return
`{"ok":true}`. Authed routes need `X-API-Key: <key>`; with no key
configured they return 503 `SERVICE_UNAVAILABLE` (fail-closed by design).
## Backup
Online, consistent (uses SQLite's backup API, safe while the service is
running — do not `cp` the live file):
docker compose exec saas python -c "import sqlite3; s=sqlite3.connect('/data/saas.db'); d=sqlite3.connect('/data/saas-backup.db'); s.backup(d); d.close(); s.close(); print('backup OK')"
docker compose cp saas:/data/saas-backup.db ./saas-backup-$(date +%F).db
Check the archive landed and is readable before moving on:
ls -l saas-backup-*.db
docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone()); print(c.execute('select count(*) from clients').fetchone()[0], 'clients')"
Keep the copy off-machine (the compose down /data copy dies with the host).
A crontab line like `docker compose ... exec saas python -c "..." && docker compose cp ...` off-site is enough.
## Restore (tested end-to-end once — see the issue PR for the recorded run)
Run from the directory that contains the backup file. `gitea-saas` is the
compose project name (pinned in docker-compose.yml), so the volume is
`gitea-saas_saas_data`.
The container runs as the non-root `app` user, so the directory holding the
backup file must be traversable by it (e.g. mode 0755, file 0644); a backup
stashed in a private home directory will fail with "Permission denied".
docker compose down # no writer while we swap files
docker run --rm -v gitea-saas_saas_data:/data -v "$(pwd)":/restore \
gitea-saas-core:local sh -c "rm -f /data/saas.db-wal /data/saas.db-shm && cp /restore/saas-backup.db /data/saas.db"
docker compose up -d
Why the `rm` of `-wal`/`-shm`: the running app leaves WAL sidecar files; if
they survived when we replace `saas.db`, SQLite would replay stale frames
onto the restored file. Removing them first is required, not cosmetic.
Verify the restored ledger:
curl -fsS http://127.0.0.1:8080/health
docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone()); print('clients:', c.execute('select coalesce(group_concat(name, \", \"), \"(none)\") from clients').fetchone()[0]); print('ledger rows:', c.execute('select count(*) from ledger').fetchone()[0])"
## Rotate SAAS_API_KEY
docker compose exec saas env # sanity: what the container sees
# edit .env (or re-export) with the new key, then:
docker compose up -d # recreates the container with the new env
Verify the old key now 401s and the new one works:
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/api/clients -H 'X-API-Key: old-key'
curl -s http://127.0.0.1:8080/api/clients -H 'X-API-Key: new-key'
## When it is down — checklist
1. `docker compose ps` — is the container running? `restart: unless-stopped`
brings it back after most exits; a container stuck in `Restarting` loops
is crashing at boot.
2. `docker compose logs --tail=200 saas` — traceback? DB path error?
`PermissionError` on `/data`? If the volume mount is missing the app
still starts but the ledger is ephemeral — check `ls` inside.
3. `curl -fsS http://127.0.0.1:8080/health` — health is public and keyless;
a time-out here while the container is up means the app is not listening
(bind/port issue): `docker compose port saas 8080`, `ss -tlnp | grep 8080`.
4. Authed calls 401? Wrong/rotated key. Authed calls 503? `SAAS_API_KEY` is
unset/empty in the container env — fail-closed on purpose.
5. Ledger questions: `docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone())"`
— should print `('ok',)`. Check the volume's disk space:
`df -h /var/lib/docker/volumes/gitea-saas_saas_data/_data`.
6. Still broken and the ledger looks intact? Restore from the last verified
backup (above) after confirming the backup file itself passes
`integrity_check`.
## Scaling (and why not to do it casually)
This deployment is intentionally a single SQLite writer. To grow you must
move the ledger off SQLite onto a network-capable database (Postgres etc.)
first — raising `--workers` against SQLite will corrupt or lock the ledger.
Until then: one worker, threads for concurrency, backups before traffic.

41
docker-compose.yml
View File

@ -1,41 +0,0 @@
# gitea-saas-core deployment (issue #40).
#
# The MEANING OF THE VOLUME (read before changing it):
# The service stores its money ledger in SQLite at /data/saas.db. The
# named volume `saas_data` is therefore the ENTIRE durability story:
# losing the volume loses the ledger. `docker compose down` keeps the
# volume. `docker compose down -v` DESTROYS it — never run that without
# a verified backup (OPS.md "Backup"/"Restore"). Backups before traffic.
#
# SQLite is a single-writer store: the container runs exactly one gunicorn
# worker (see Dockerfile) so all writes funnel through one process lock.
# Do not raise --workers here.
name: gitea-saas
services:
saas:
build: .
image: gitea-saas-core:local
restart: unless-stopped
ports:
- "8080:8080"
environment:
# Empty by default on purpose: with no key the app still boots and
# /health stays public, but every authed route fails closed with 503
# (see gitea_saas/app.py). Set a real key before serving traffic:
# docker compose --env-file .env up -d
# with SAAS_API_KEY=<secret> in .env, or export SAAS_API_KEY first.
SAAS_API_KEY: ${SAAS_API_KEY:-}
SAAS_DB_PATH: ${SAAS_DB_PATH:-/data/saas.db}
SAAS_PORT: "8080"
volumes:
- saas_data:/data
healthcheck:
test: ["CMD", "python", "-c", "import sys,urllib.request; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=5).status == 200 else 1)"]
interval: 30s
timeout: 5s
start_period: 10s
retries: 3
volumes:
saas_data:

6
gitea_saas/__main__.py
View File

@ -1,6 +0,0 @@
"""``python -m gitea_saas`` dev server (issue #38)."""
from .app import main
if __name__ == "__main__":
main()

177
gitea_saas/app.py
View File

@ -1,177 +0,0 @@
"""Assembly: one ``create_app()`` + the entrypoints for the whole service (#38).
The 15 modules on ``main`` compose into a single deployable Flask app in
exactly one place:
billing API ``gitea_saas.server.create_app`` (issues #1/#2/#6)
factory ``gitea_saas.intake_factory.register_factory_routes`` (#4)
dashboard ``gitea_saas.dashboard.mount_dashboard`` (issue #9)
client API ``gitea_saas.client_api.register_client_routes`` (#41)
Assembly only: this module makes no behaviour changes to billing, factory,
dashboard or review. If composing them ever surfaces a genuine
incompatibility it is reported on the issue, not papered over here.
Fail-closed assembly: when ``SAAS_API_KEY`` is unset the app still starts
(``python -m gitea_saas``, ``gitea-saas`` console script, gunicorn via the
repo-root ``wsgi.py``) but every authed route returns 503
``SERVICE_UNAVAILABLE``. No second auth path is invented here: key
validation stays in ``gitea_saas.auth`` (issue #24) and each front-end's
own ``auth.key_is_valid`` guard (canonical 401) applies unchanged as soon
as a key IS configured. The 503 gate exists only at this assembly layer,
and only for the unconfigured state.
"""
from __future__ import annotations
import tempfile
from pathlib import Path
from types import SimpleNamespace
from typing import Callable, Optional
from flask import Flask, jsonify, request
from .auth import PUBLIC_PATHS
from .billing_core import BillingCore
from .client_api import (ClientKeyStore, ClientStatus, ClientSurface,
register_client_routes)
from .config import ServerConfig, load_config
from .dashboard import mount_dashboard
from .gitea_client import GiteaClient
from .intake_factory import register_factory_routes
from .server import create_app as create_billing_app
from .stripe_client import build_stripe_client
# Canonical body for the unconfigured-deployment gate (status 503). Same
# ``{"error": {"code", "message"}}`` shape as ``auth.UNAUTHORIZED_BODY`` so
# clients parse every front-end's auth failure identically.
UNAVAILABLE_BODY = {
"error": {
"code": "SERVICE_UNAVAILABLE",
"message": "SAAS_API_KEY is unset: the service is not configured "
"for authenticated use (fail-closed)",
},
}
def _resolve_db_path(cfg: ServerConfig) -> Path:
"""The runtime SQLite path shared by billing, factory and dashboard.
``config.py`` defaults an unset ``SAAS_DB_PATH`` to in-memory, but the
factory and the dashboard open their OWN connections to the same file,
so an ephemeral FILE (in a fresh temp dir) keeps all three components
on one coherent dataset for the no-env dev run. No new env vars: this
is the assembler's default for the existing unset state.
"""
if cfg.db_path is not None:
return Path(cfg.db_path)
scratch = Path(tempfile.mkdtemp(prefix="gitea-saas-"))
return scratch / "saas.db"
def _unconfigured_guard(configured: bool):
"""before_request that fail-closes an UNCONFIGURED deployment.
Registered ONLY when no API key is configured, and inserted ahead of
``server.create_app``'s own before_request so the unconfigured state
is reported as 503 (this issue's acceptance 2: not 500, not 200)
instead of the front-ends' 401s. It performs no key comparison of its
own -- validation stays in ``gitea_saas.auth.key_is_valid`` (issue
#24) -- and it fires only for non-public paths, so the liveness routes
in ``auth.PUBLIC_PATHS`` (/health, /healthz, /api/factory/health)
keep serving. With a configured key this guard is never installed.
"""
if configured:
return None
def _guard():
if request.path.rstrip("/") in PUBLIC_PATHS:
return None
resp = jsonify(UNAVAILABLE_BODY)
resp.status_code = 503
return resp
return _guard
def create_app(config: Optional[ServerConfig] = None, *,
stripe_builder: Optional[Callable[[], object]] = None,
gitea: Optional[object] = None) -> Flask:
"""Build the ONE Flask app: billing + factory + dashboard + client surface.
``config`` is a :class:`~gitea_saas.config.ServerConfig` (or any object
exposing ``api_key`` / ``port`` / ``db_path``); when None the
environment is loaded through the existing ``config.load_config``
loader (no new env vars).
``stripe_builder`` is a zero-arg callable returning a Stripe seam (mock
or real) for the client surface (issue #41); it defaults to the
env-driven ``stripe_client.build_stripe_client``, which is fail-closed:
with no credentials and no explicit ``SAAS_STRIPE_MODE=mock`` it raises,
the surface answers checkout/webhook with 503 ``stripe_unconfigured``,
and the app still starts. ``gitea`` defaults to a real ``GiteaClient``
(factory + client work-submission seam).
The DB is opened via ``gitea_saas.db`` (through ``BillingCore``, which
owns the connection + idempotent schema bootstrap) and shared by all
front-ends. With no ``SAAS_API_KEY`` the app still starts; authed
routes return 503 until a key is configured (module docstring).
"""
cfg = config if config is not None else load_config()
api_key = getattr(cfg, "api_key", None) or ""
db_path = _resolve_db_path(cfg)
core = BillingCore(db_path)
# The factory's IntakeFactory is built per request and reads
# ``config.db_path`` (env fallback is the pre-assembly behaviour);
# hand it the RESOLVED path so the per-request construction cannot
# raise "IntakeFactory requires config.db_path" for the no-env case.
runtime = SimpleNamespace(db_path=db_path, api_key=api_key)
gitea_client = gitea if gitea is not None else GiteaClient()
app = create_billing_app(core, api_key)
register_factory_routes(app, core, gitea_client, runtime)
mount_dashboard(app, core=core, db_path=db_path, api_key=api_key)
# Issue #41 client surface: per-client keys, checkout, /api/me,
# self-serve work submission and the Stripe webhook. Mounting it
# replaces the billing app's operator-key before_request with the
# compound assembly guard (see client_api.register_client_routes) whose
# operator-route semantics are identical — with the scoped client
# endpoints reachable and valid client keys answered 403 on operator
# routes (never 401, never passed through).
key_store = ClientKeyStore(db_path)
status = ClientStatus(core, db_path,
getattr(cfg, "debit_floor_minor", None))
surface = ClientSurface(core, key_store, status,
stripe_builder or build_stripe_client)
register_client_routes(app, surface, gitea_client, runtime)
guard = _unconfigured_guard(bool(api_key))
if guard is not None:
# Global before_request funcs run in list order; position 0 puts
# the unconfigured gate ahead of server.create_app's own guard.
# A returning response short-circuits the rest of the request, so
# authed routes report the deployment state before any front-end
# 401 logic runs. When a key IS configured this list is never
# touched and the front-ends' auth.key_is_valid guards decide.
app.before_request_funcs.setdefault(None, []).insert(0, guard)
return app
def main() -> None:
"""Dev entrypoint (``python -m gitea_saas`` / ``gitea-saas`` script).
Starts even with no ``SAAS_API_KEY`` (fail-closed 503, per the issue);
binds loopback like the pre-assembly server entrypoint. Production
runs under gunicorn against the repo-root ``wsgi.py`` instead.
"""
cfg = load_config()
app = create_app(cfg)
app.run(host="127.0.0.1", port=cfg.port)
if __name__ == "__main__":
main()

138
gitea_saas/auth.py
View File

@ -1,138 +0,0 @@
"""Single X-API-Key auth path for the whole service (issue #24).
Every API-key front-end (``server.py``, ``delivery.py``, ``intake_factory.py``,
``dashboard.py``, ``client_api.py``) delegates to this module; the header name, the
constant-time comparison and the 401 body live exactly once here (no front-end forks them).
``stripe_client.py`` verifies webhook *signatures* (a different secret on a different
verification path) and is explicitly out of scope for this issue — it keeps its own signature check.
Public surface:
API_KEY_HEADER the only ``X-API-Key`` header literal in the package
PUBLIC_PATHS deliberately unauthenticated liveness/health routes + the Stripe webhook (#41)
key_is_valid() constant-time, fail-closed check — the one API-key
comparison in the whole service
UNAUTHORIZED_BODY canonical 401 payload, byte-identical on every wire
unauthorized_response() Flask Response object carrying that body
FORBIDDEN_BODY canonical 403 payload (issue #41 cross-tenant denial)
forbidden_response() Flask Response object carrying that body
hash_client_key() SHA-256 hash for per-client keys (#41) — stored, never plaintext
"""
from __future__ import annotations
import hmac
from typing import Optional
# The one place the HTTP header literal lives (acceptance criterion 2: one literal).
API_KEY_HEADER = "X-API-Key"
# Routes that need no operator API key; every front-end guard reads this set,
# so exposure is declared here, not inside one PR's before_request (composition
# finding 2026-08-26): /health, /healthz, /api/factory/health are liveness
# routes; /api/webhooks/stripe (#41) is signature-authenticated.
PUBLIC_PATHS = frozenset({
"/health",
"/healthz",
"/api/factory/health",
"/api/webhooks/stripe",
})
# Canonical 401 payload. All front-ends return exactly this (Flask
# `jsonify(UNAUTHORIZED_BODY)` for HTTP surfaces, the dict itself for the
# delivery `(result, status)` seam), so a client parses every front-end's
# auth failure identically (acceptance criterion 4).
UNAUTHORIZED_BODY = {
"error": {
"code": "UNAUTHORIZED",
"message": "missing or invalid API key",
},
}
def key_is_valid(provided_key: Optional[str],
configured_key: Optional[str]) -> bool:
"""Fail-closed, constant-time API-key check (single implementation).
A request is authorized only when a key is actually configured AND the
provided header value matches it exactly. An unset/empty or non-str
configured key refuses every request; non-str/empty provided values are
rejected before the comparison runs (the constant-time compare raises
``TypeError`` on non-str input).
This is the ONE API-key comparison in the service (acceptance criterion
1) — no front-end does its own.
"""
if not isinstance(configured_key, str) or not configured_key:
return False
if not isinstance(provided_key, str) or not provided_key:
return False
return hmac.compare_digest(provided_key, configured_key)
def unauthorized_response():
"""Flask Response with the canonical 401 body (status 401)."""
from flask import jsonify
resp = jsonify(UNAUTHORIZED_BODY)
resp.status_code = 401
return resp
# ---------------------------------------------------------------------------
# Issue #41 additions — the client-facing surface's auth pieces.
#
# Deliberately placed BELOW key_is_valid/unauthorized_response so the issue
# #24 acceptance pin (exactly one compare_digest call site in this file, at
# auth.py:70) is untouched: per-client key verification adds NO comparison
# call site. A presented client key is hashed (hash_client_key) and resolved
# by indexed equality on the full 64-char digest in the client_api_keys
# table (client_api.ClientKeyStore.resolve_client_id). The digest is
# one-way, so even a timing leak on digest equality reveals nothing about
# the key — the discipline that matters is: full-length digest, and no
# plaintext ever stored or compared.
# ---------------------------------------------------------------------------
# Canonical 403 payload (issue #41): a VALID key presented against a route
# its tenant is not allowed to reach (another client's resources, or an
# operator-only route). Distinguished from 401 (no/wrong key) so a client
# can tell "authenticate" from "authenticated but forbidden".
FORBIDDEN_BODY = {
"error": {
"code": "FORBIDDEN",
"message": "this key is not authorized for this resource",
},
}
def forbidden_response():
"""Flask Response with the canonical 403 body (status 403, issue #41).
Used when a key is VALID but its tenant is not allowed on the requested
route: another client's resources or an operator-only route. The body
is byte-identical on every front-end so a client parses it once.
"""
from flask import jsonify
resp = jsonify(FORBIDDEN_BODY)
resp.status_code = 403
return resp
CLIENT_KEY_HASH_ALGORITHM = "sha256"
def hash_client_key(raw_key: str) -> str:
"""SHA-256 hex digest of a per-client API key (issue #41).
The ONLY place a client key is hashed. ``raw_key`` must be a
non-empty str (a non-str raises ``TypeError`` from ``hashlib`` — the
caller validates before hashing).
"""
import hashlib
if not isinstance(raw_key, str) or not raw_key:
raise ValueError("client key must be a non-empty string")
digest = hashlib.new(CLIENT_KEY_HASH_ALGORITHM)
digest.update(raw_key.encode("utf-8"))
return digest.hexdigest()

672
gitea_saas/client_api.py
View File

@ -1,672 +0,0 @@
"""Client-facing HTTP surface (issue #41): self-serve funding + work submission.
Until this module there was exactly one credential — the shared operator
``SAAS_API_KEY`` — and every route was operator/admin. This module adds the
first genuinely externally-reachable surface:
* **Per-client API keys** — ``ClientKeyStore``. Keys are random, scoped to
exactly one ``client_id``, stored ONLY as a SHA-256 hash (the plaintext is
returned exactly once, at issuance, then never seen again) and revocable.
The hash function lives in ``gitea_saas.auth`` (issue #41 extension) and
verification is digest-equality against the key table — this module does
not fork auth, it builds on it.
* **Checkout** — ``POST /api/checkout`` returns a Stripe Checkout URL for a
top-up, rate-limited per client. The webhook below is the ONLY credit
path: ``stripe_client.apply_payment_event`` (the issue-#3 seam) is reused,
never a second credit implementation.
* **Client status** — ``GET /api/me`` returns the caller's own balance,
queue counts, in-flight count and delivered count (scoped to the caller's
client_id by construction).
* **Work submission** — ``POST /api/me/issues`` submits work as the caller's
OWN client_id through the issue-#4 factory (funded -> Gitea issue,
unfunded/locked -> queued). A client never supplies another tenant's id.
* **Stripe webhook** — ``POST /api/webhooks/stripe``, authenticated by
Stripe's signature (never by an API key), credits idempotently.
Tenancy & guard composition (the subtle part)
---------------------------------------------
``server.create_app`` installs an operator-key ``before_request`` guard that
401s every non-``PUBLIC_PATHS`` route — it would make the client surface
unreachable, since client routes are authenticated by a DIFFERENT key class.
Mounting the client surface therefore REPLACES that single global guard
with a compound assembly guard (registered in ``register_client_routes``);
operator-route behaviour is preserved EXACTLY (same ``PUBLIC_PATHS`` set,
same ``auth.key_is_valid``, same canonical 401):
1. ``PUBLIC_PATHS`` (liveness + the signature-authenticated webhook) pass
straight through.
2. A valid client key on a ``CLIENT_PATHS`` route is stored in
``flask.g.client_id`` and allowed.
3. A valid client key ANYWHERE ELSE (``/api/clients``, ``/api/factory/*``,
``/dashboard``, key issuance) is forbidden: canonical 403 body.
4. No/invalid client key: client-surface routes get the canonical 401;
operator routes fall through to the operator check, which decides
exactly as before (the unconfigured-deployment 503 gate from
``app.create_app`` runs ahead of everything and still short-circuits
every non-public route first).
Consequences that are deliberate:
* The operator key never works on the client surface (401), and a client key
never works on operator routes (403).
* The operator key-issuance routes (``/api/clients/<id>/keys``) are NOT part
of the client surface — they stay behind the operator guard, and a client
key hitting them gets 403.
* The existing ``/api/factory/<id>/*`` routes stay operator-only. Clients
submit work through ``/api/me/issues`` instead, so the factory module is
untouched and its per-route operator check cannot be bypassed.
Fail-closed anywhere a secret could be at stake: an unconfigured Stripe
deployment answers checkout/webhook with 503 ``stripe_unconfigured`` (never
a fallback secret, never a 200); nothing in this module ever logs a key, a
Stripe secret, or a webhook payload.
Hermetic by construction (same convention as the other front-ends): the
billing core, key store and status provider are injected; the Stripe client
comes from an injected builder (the mock is selected only on explicit opt-in
per ``stripe_client.build_stripe_client``).
"""
from __future__ import annotations
import os
import re
import secrets
import sqlite3
import threading
import time
from collections import deque
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from flask import Flask, g, jsonify, request
from .auth import (API_KEY_HEADER, PUBLIC_PATHS, forbidden_response,
hash_client_key, key_is_valid, unauthorized_response)
from .intake_factory import (CoreContractError, IntakeFactory,
InvalidRequestError, UnknownClientError)
# Canonical prefix for generated client keys (display-friendly, never used
# for auth — the full key is required and only its hash is stored).
CLIENT_KEY_PREFIX = "saas_"
CLIENT_KEY_BYTES = 32
# The routes that belong to a client's OWN surface (self-scoped by the
# caller's key). A valid client key is allowed here and only here; on every
# other route it is forbidden (403). Keep in sync with the routes below.
CLIENT_PATHS = frozenset({
"/api/checkout",
"/api/me",
"/api/me/issues",
})
STRIPE_WEBHOOK_ROUTE = "/api/webhooks/stripe"
# Rate limit for POST /api/checkout per client (fixed per-client window).
DEFAULT_CHECKOUT_RATE_LIMIT = 5
DEFAULT_CHECKOUT_RATE_WINDOW_S = 60
# Mirror of intake_factory's repo-name rule (issue #4 contract).
_REPO_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$")
# Idempotent DDL mirror of the schema.sql addition (same convention as
# intake_factory._FACTORY_DDL: this module can be constructed against a
# fresh file even before the billing core has run schema.sql).
_KEYS_DDL = """
CREATE TABLE IF NOT EXISTS client_api_keys (
id INTEGER PRIMARY KEY AUTOINCREMENT,
client_id INTEGER NOT NULL REFERENCES clients(id),
key_hash TEXT NOT NULL UNIQUE,
key_prefix TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
revoked_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_client_api_keys_client ON client_api_keys (client_id);
CREATE INDEX IF NOT EXISTS idx_client_api_keys_hash ON client_api_keys (key_hash);
"""
def _json_error(status: int, code: str, message: str):
"""Canonical structured error (same shape as auth.UNAUTHORIZED_BODY)."""
resp = jsonify({"error": {"code": code, "message": message}})
resp.status_code = status
return resp
class _RateLimiter:
"""Sliding-window request limiter, keyed by tenant (in-process).
Single-process service (gunicorn sync workers run one app instance per
worker); the limiter is per-process by design. ``allow(key)`` returns
``(allowed, retry_after_s)`` — retry_after is 0 when allowed.
"""
def __init__(self, limit: int, window_s: float):
self._limit = max(int(limit), 1)
self._window_s = float(window_s)
self._hits: Dict[Any, deque] = {}
self._lock = threading.Lock()
def allow(self, key: Any) -> tuple[bool, float]:
now = time.monotonic()
with self._lock:
dq = self._hits.setdefault(key, deque())
while dq and now - dq[0] >= self._window_s:
dq.popleft()
if len(dq) >= self._limit:
retry = self._window_s - (now - dq[0])
return False, max(retry, 1.0)
dq.append(now)
return True, 0.0
class ClientKeyStore:
"""Per-client API keys on the shared SQLite file: issue/list/revoke/resolve.
At-rest policy (acceptance criterion 2): ONLY ``hash_client_key`` output
is stored — the plaintext is returned once at issuance and never written.
A resolver looks the presented key up by its hash; a revoked or deleted
client's keys stop resolving. ``list()`` returns metadata only (id,
prefix, timestamps) — never hashes, never plaintext.
"""
def __init__(self, db_path):
self._conn = sqlite3.connect(str(Path(db_path)), check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA foreign_keys=ON")
self._conn.executescript(_KEYS_DDL)
self._lock = threading.RLock()
# -- helpers ----------------------------------------------------------
def _client_exists(self, client_id: int) -> bool:
row = self._conn.execute(
"SELECT 1 FROM clients WHERE id = ?", (client_id,)).fetchone()
return row is not None
@staticmethod
def _generate() -> str:
return CLIENT_KEY_PREFIX + secrets.token_urlsafe(CLIENT_KEY_BYTES)
# -- operations -------------------------------------------------------
def issue(self, client_id: int) -> tuple[bool, dict]:
"""Create a key for ``client_id``; plaintext returned ONCE.
Returns ``(True, {"key_id", "client_id", "key", "prefix",
"created_at"})`` or ``(False, {"error": ...})`` for an unknown
client. The plaintext ``key`` is not stored and cannot be recovered.
"""
if isinstance(client_id, bool) or not isinstance(client_id, int) \
or client_id <= 0:
return False, {"error": {"code": "UNKNOWN_CLIENT",
"message": "client_id must be a positive integer"}}
with self._lock:
if not self._client_exists(client_id):
return False, {"error": {"code": "UNKNOWN_CLIENT",
"message": f"no client with id {client_id}"}}
raw = self._generate()
now = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
cur = self._conn.execute(
"INSERT INTO client_api_keys (client_id, key_hash, key_prefix,"
" created_at) VALUES (?, ?, ?, ?)",
(client_id, hash_client_key(raw), raw[:10], now))
self._conn.commit()
return True, {"key_id": cur.lastrowid, "client_id": client_id,
"key": raw, "prefix": raw[:10], "created_at": now}
def list_keys(self, client_id: int) -> tuple[bool, dict]:
"""Key metadata for a client (never hashes, never plaintext)."""
with self._lock:
if not self._client_exists(client_id):
return False, {"error": {"code": "UNKNOWN_CLIENT",
"message": f"no client with id {client_id}"}}
rows = self._conn.execute(
"SELECT id, key_prefix, created_at, revoked_at"
" FROM client_api_keys WHERE client_id = ? ORDER BY id",
(client_id,)).fetchall()
keys = [{"key_id": int(r["id"]), "prefix": r["key_prefix"],
"created_at": r["created_at"], "revoked_at": r["revoked_at"]}
for r in rows]
return True, {"client_id": client_id, "keys": keys, "count": len(keys)}
def revoke(self, client_id: int, key_id: int) -> tuple[bool, dict]:
"""Revoke exactly one key that BELONGS to ``client_id`` (scoped)."""
with self._lock:
if not self._client_exists(client_id):
return False, {"error": {"code": "UNKNOWN_CLIENT",
"message": f"no client with id {client_id}"}}
now = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
cur = self._conn.execute(
"UPDATE client_api_keys SET revoked_at = ?"
" WHERE id = ? AND client_id = ? AND revoked_at IS NULL",
(now, key_id, client_id))
self._conn.commit()
if cur.rowcount == 0:
return False, {"error": {"code": "KEY_NOT_FOUND",
"message": "no active key with that id for this client"}}
return True, {"key_id": key_id, "client_id": client_id,
"revoked_at": now, "revoked": True}
def revoke_all(self, client_id: int) -> tuple[bool, dict]:
"""Revoke every active key of ``client_id`` (idempotent)."""
with self._lock:
if not self._client_exists(client_id):
return False, {"error": {"code": "UNKNOWN_CLIENT",
"message": f"no client with id {client_id}"}}
now = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
cur = self._conn.execute(
"UPDATE client_api_keys SET revoked_at = ?"
" WHERE client_id = ? AND revoked_at IS NULL",
(now, client_id))
self._conn.commit()
return True, {"client_id": client_id, "revoked_count": cur.rowcount}
def resolve_client_id(self, provided_key: Optional[str]) -> Optional[int]:
"""The client_id owning ``provided_key``, or None (fail-closed).
Hashes the presented key and looks the row up by hash; revoked keys
and keys whose client no longer exists never resolve. Returns None
for a missing/malformed key, a revoked key, or an unknown client.
"""
if not isinstance(provided_key, str) or not provided_key:
return None
try:
digest = hash_client_key(provided_key)
except ValueError:
return None
try:
row = self._conn.execute(
"SELECT k.client_id FROM client_api_keys k"
" JOIN clients c ON c.id = k.client_id"
" WHERE k.key_hash = ? AND k.revoked_at IS NULL",
(digest,)).fetchone()
except sqlite3.Error:
return None
return int(row["client_id"]) if row is not None else None
class ClientStatus:
"""Client-visible status: balance + queue/in-flight/delivered counts.
Reads the shared SQLite file like ``Dashboard`` does (its own read
connection); balance comes from the billing core (single source of
truth, holds included). Definitions (documented in the API):
* ``queued_items`` — ``pending_issues`` rows still waiting in the
intake queue (unfunded/locked submits, issue #4).
* ``in_flight_items`` — ``pending_issues`` rows the factory has marked
``shipped`` (flushed from the queue into Gitea, awaiting
merge/delivery). A direct funded submit is created in Gitea without
a local row and is therefore not counted here — the factory tracks
neither it nor a delivery-complete state, so this count is defined by
the data the system records.
* ``delivered_count`` — distinct delivery debits in the ledger
(``ref_type`` ``pr``/``delivery`` — the only DB record of a delivery).
"""
DEFAULT_DEBIT_FLOOR_MINOR = 500
def __init__(self, core: Any, db_path, debit_floor_minor: Optional[int] = None):
self._core = core
self._conn = sqlite3.connect(str(Path(db_path)))
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA journal_mode=WAL")
if debit_floor_minor is None:
raw = os.environ.get("SAAS_DEBIT_FLOOR_MINOR", "").strip()
debit_floor_minor = int(raw) if raw else self.DEFAULT_DEBIT_FLOOR_MINOR
self._floor = debit_floor_minor
def _table_exists(self, name: str) -> bool:
row = self._conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?",
(name,)).fetchone()
return row is not None
def summary(self, client_id: int) -> tuple[bool, dict]:
ok, details = self._core.get_client(client_id, include_balance=True)
if not ok or not isinstance(details, dict):
return False, details if isinstance(details, dict) \
else {"error": {"code": "UNKNOWN_CLIENT",
"message": f"no client with id {client_id}"}}
cid = int(details.get("id", client_id))
balance = details.get("balance")
locked = False
if self._table_exists("factory_locks"):
try:
locked = self._conn.execute(
"SELECT 1 FROM factory_locks WHERE client_id = ?",
(cid,)).fetchone() is not None
except Exception:
locked = False
if locked:
status = "locked"
elif isinstance(balance, int) and balance < self._floor:
status = "overdue"
else:
status = str(details.get("status") or "active")
queued = in_flight = delivered = 0
if self._table_exists("pending_issues"):
try:
row = self._conn.execute(
"SELECT status, COUNT(*) AS n FROM pending_issues"
" WHERE client_id = ? GROUP BY status", (cid,)).fetchall()
counts = {str(r["status"]): int(r["n"]) for r in row}
queued = counts.get("queued", 0)
in_flight = counts.get("shipped", 0)
except Exception:
queued = in_flight = 0
try:
delivered = int(self._conn.execute(
"SELECT COUNT(DISTINCT ref_id) FROM ledger"
" WHERE client_id = ? AND kind = 'debit'"
" AND ref_type IN ('pr', 'delivery') AND ref_id IS NOT NULL",
(cid,)).fetchone()[0])
except Exception:
delivered = 0
return True, {
"client_id": cid,
"name": details.get("name"),
"tier": details.get("tier"),
"status": status,
"balance_minor": balance,
"currency": "USD",
"queued_items": queued,
"in_flight_items": in_flight,
"delivered_count": delivered,
}
class ClientSurface:
"""State shared by the client routes: guard, keys, status, Stripe. """
def __init__(self, core: Any, key_store: ClientKeyStore,
status: ClientStatus,
stripe_builder: Callable[[], Any],
checkout_rate: int = DEFAULT_CHECKOUT_RATE_LIMIT,
checkout_window_s: float = DEFAULT_CHECKOUT_RATE_WINDOW_S):
self.core = core
self.keys = key_store
self.status = status
self.stripe_builder = stripe_builder
self._limiter = _RateLimiter(checkout_rate, checkout_window_s)
# -- request-time helpers --------------------------------------------
def stripe(self) -> Optional[Any]:
"""A usable Stripe client, or None when unconfigured (fail-closed).
The builder is env-driven (``stripe_client.build_stripe_client`` by
default): a deployment without Stripe credentials, or without an
EXPLICIT mock opt-in, gets None here — the routes answer 503, never
a fallback secret and never a 200 with data.
"""
try:
return self.stripe_builder()
except Exception:
return None
def limiter_allow(self, client_id: int) -> tuple[bool, float]:
return self._limiter.allow(client_id)
# -- the tenant-aware decision (used by the compound assembly guard) ---
def check_client_key(self, path: str) -> Optional[tuple]:
"""Decide the client-surface part of a request's auth.
Returns ``None`` when the request is not a client-surface matter
(operator routes are left to the operator check), or a tuple:
* ``("forbid", Flask-Response)`` — a VALID client key presented
outside its own surface (403, cross-tenant / operator route), or
a client-surface path without a valid client key (401);
* ``("allowed", client_id)`` — a valid client key on its own
surface; the caller stores the client_id for the route.
"""
provided = request.headers.get(API_KEY_HEADER, "")
client_id = self.keys.resolve_client_id(provided)
if client_id is not None:
if path in CLIENT_PATHS: # the key's own surface
return ("allowed", client_id)
return ("forbid", forbidden_response()) # wrong tenant/route
if path in CLIENT_PATHS: # client surface needs a client key
return ("forbid", unauthorized_response())
return None # operator routes: their check decides
def _map_key_store_error(details: dict):
code = (details.get("error") or {}).get("code")
message = (details.get("error") or {}).get("message") or "key operation failed"
if code == "UNKNOWN_CLIENT":
return _json_error(404, "unknown_client", message)
if code == "KEY_NOT_FOUND":
return _json_error(404, "key_not_found", message)
return _json_error(500, "key_store_error", message)
def _map_factory_error(exc: Exception):
"""Map IntakeFactory domain errors (mirror of intake_factory's mapper)."""
if isinstance(exc, UnknownClientError):
return _json_error(404, "unknown_client", str(exc))
if isinstance(exc, (InvalidRequestError, ValueError)):
return _json_error(400, "invalid_request", str(exc))
if isinstance(exc, CoreContractError):
return _json_error(500, "core_contract", str(exc))
return _json_error(500, "factory_error", f"{type(exc).__name__}: {exc}")
def register_client_routes(app: Flask, surface: ClientSurface,
gitea: Any, config: Any) -> Flask:
"""Mount the client-facing surface on an existing Flask app.
``surface`` carries core/keys/status/Stripe; ``gitea`` and ``config``
feed the per-request ``IntakeFactory`` for ``/api/me/issues`` (same
seam the issue-#4 factory routes use).
Registers the tenant-aware guard AHEAD of every other before_request
(including the operator-key guard installed by ``server.create_app``) —
see the module docstring: valid client keys must be answered 403 on
operator routes, and the client surface must be reachable even though
the operator guard would otherwise 401 it.
"""
core = surface.core
# -- the tenant-aware guard --------------------------------------------
# server.create_app installs an operator-key before_request that 401s
# every non-PUBLIC_PATHS route — including the client surface, whose
# auth is a DIFFERENT key class. Mounting the client surface therefore
# REPLACES that single global guard with a compound assembly guard:
# 1. PUBLIC_PATHS (liveness + the signature-authenticated webhook)
# pass straight through;
# 2. the client-surface decision wins when it is one (403 for a
# valid client key off its surface / on operator routes, 401 on
# the client surface without a valid key);
# 3. otherwise the operator check runs with EXACTLY server.create_app's
# semantics (same PUBLIC_PATHS set, same auth.key_is_valid, same
# canonical 401 body) — no behaviour change for operator routes.
# The unconfigured-deployment 503 gate (app.py) is inserted at position
# 0 AFTER this, so it still short-circuits everything non-public first.
app.before_request_funcs.setdefault(None, []).clear()
def _assembly_guard():
path = request.path.rstrip("/")
if path in PUBLIC_PATHS:
return None
decision = surface.check_client_key(path)
if decision is not None:
outcome, payload = decision
if outcome == "forbid":
return payload # 403 / 401 decision (see above)
g.client_id = payload # allowed on the client's own surface
return None
# Operator routes: server.create_app's check, preserved verbatim.
sent = request.headers.get(API_KEY_HEADER, "")
expected = app.config.get("API_KEY", "")
if key_is_valid(sent, expected):
return None
return unauthorized_response()
app.before_request_funcs.setdefault(None, []).insert(0, _assembly_guard)
# -- self-serve checkout ---------------------------------------------
@app.post("/api/checkout")
def checkout():
client_id = g.get("client_id")
if client_id is None:
return unauthorized_response()
allowed, retry_after = surface.limiter_allow(client_id)
if not allowed:
resp = _json_error(429, "rate_limited",
"too many checkout requests; retry shortly")
resp.headers["Retry-After"] = str(int(retry_after))
return resp
body = request.get_json(silent=True)
if not isinstance(body, dict):
return _json_error(400, "invalid_request",
"request body must be a JSON object")
amount = body.get("amount_minor")
if isinstance(amount, bool) or not isinstance(amount, int) or amount <= 0:
return _json_error(400, "invalid_request",
"'amount_minor' must be a positive integer (minor units)")
currency = body.get("currency") or "usd"
if (not isinstance(currency, str) or len(currency) != 3
or not currency.isalpha()):
return _json_error(400, "invalid_request",
"'currency' must be a 3-letter code")
success_url = body.get("success_url")
cancel_url = body.get("cancel_url")
for name, value in (("success_url", success_url),
("cancel_url", cancel_url)):
if value is not None and (
not isinstance(value, str)
or not re.match(r"^https?://", value.strip())):
return _json_error(400, "invalid_request",
f"'{name}' must be an absolute http(s) URL")
stripe = surface.stripe()
if stripe is None:
return _json_error(503, "stripe_unconfigured",
"Stripe is not configured on this deployment (fail-closed)")
try:
result = stripe.create_checkout_session(
client_id, amount, currency.lower(),
success_url.strip() if isinstance(success_url, str) else None,
cancel_url.strip() if isinstance(cancel_url, str) else None)
except Exception:
return _json_error(503, "stripe_error",
"checkout session could not be created")
resp = jsonify(result)
resp.status_code = 201
return resp
# -- Stripe webhook (signature-authenticated; NOT an API-key route) ---
@app.post(STRIPE_WEBHOOK_ROUTE)
def stripe_webhook():
stripe = surface.stripe()
if stripe is None:
return _json_error(503, "stripe_unconfigured",
"Stripe is not configured on this deployment (fail-closed)")
payload = request.get_data()
signature = request.headers.get("Stripe-Signature", "")
try:
event = stripe.process_webhook(payload, signature)
except Exception:
# Signature failures and malformed payloads are indistinguishable
# on purpose: never leak whether the signature was the problem.
return _json_error(400, "invalid_signature",
"webhook signature verification failed")
from .stripe_client import apply_payment_event
ok, details = apply_payment_event(core, event)
if not ok:
message = ((details or {}).get("error") or {}).get("message") \
or "event could not be applied"
return _json_error(400, "event_rejected", message)
# Success path: idempotent credit via the issue-#3 seam. The full
# payload is never logged and never echoed.
return jsonify({"received": True,
"applied": bool(details.get("applied", True)),
"event_id": event.get("event_id")})
# -- client status & self-serve work submission -----------------------
@app.get("/api/me")
def me():
client_id = g.get("client_id")
if client_id is None:
return unauthorized_response()
ok, details = surface.status.summary(client_id)
if not ok:
return _json_error(404, "unknown_client",
f"no client with id {client_id}")
return jsonify(details)
@app.post("/api/me/issues")
def me_submit_issue():
client_id = g.get("client_id")
if client_id is None:
return unauthorized_response()
body = request.get_json(silent=True)
if not isinstance(body, dict):
return _json_error(400, "invalid_request",
"request body must be a JSON object")
repo = (body.get("repo") or "").strip()
title = (body.get("title") or "").strip()
acceptance_body = str(body.get("acceptance_body") or "")
labels = body.get("labels")
if not isinstance(labels, list):
labels = None
labels = [str(label) for label in labels] if labels else None
if not _REPO_NAME_RE.match(repo):
return _json_error(400, "invalid_request",
f"invalid repo name: {repo!r}")
if not title:
return _json_error(400, "invalid_request",
"issue title must be a non-empty string")
try:
factory = IntakeFactory(core, gitea, config)
result = factory.submit_issue(client_id, repo, title,
acceptance_body, labels)
except Exception as exc:
return _map_factory_error(exc)
resp = jsonify(result)
resp.status_code = 202 if not result.get("delivered") else 201
return resp
# -- operator-only key management (behind the operator guard) ---------
# These are NOT client-surface routes: the tenant-aware guard above
# forbids a client key here (403), and the operator guard requires the
# shared SAAS_API_KEY.
@app.post("/api/clients/<int:client_id>/keys")
def issue_client_key(client_id: int):
ok, details = surface.keys.issue(client_id)
if not ok:
return _map_key_store_error(details)
resp = jsonify(details)
resp.status_code = 201
return resp
@app.get("/api/clients/<int:client_id>/keys")
def list_client_keys(client_id: int):
ok, details = surface.keys.list_keys(client_id)
if not ok:
return _map_key_store_error(details)
return jsonify(details)
@app.delete("/api/clients/<int:client_id>/keys/<int:key_id>")
def revoke_client_key(client_id: int, key_id: int):
ok, details = surface.keys.revoke(client_id, key_id)
if not ok:
return _map_key_store_error(details)
return jsonify(details)
@app.delete("/api/clients/<int:client_id>/keys")
def revoke_all_client_keys(client_id: int):
ok, details = surface.keys.revoke_all(client_id)
if not ok:
return _map_key_store_error(details)
return jsonify(details)
return app

28
gitea_saas/config.py
View File

@ -5,9 +5,8 @@ coordinated between issues #2 and #6 — see botmanager #21 for the ownership
pattern that motivated it):
* the HTTP server's frozen :class:`ServerConfig` (issue #2) — the billing
API's API-key discipline (``SAAS_API_KEY`` env; the ``X-API-Key`` header
constant itself is owned by ``gitea_saas.auth``, issue #24), the port, and
the DB path; and
API's API-key discipline (``SAAS_API_KEY`` / ``X-API-Key`` header), the
port, and the DB path; and
* the review-gate/policy tuning (issue #6): tier rates/policies
(``SAAS_TIERS_JSON``) plus the reviewer knobs (``REVIEW_*``), returned as
a plain dict by :func:`load_review_config`.
@ -33,20 +32,16 @@ DEFAULT_PORT = 8080
# (in-memory) SQLite database. Real deployments must set SAAS_DB_PATH.
DEFAULT_DB_PATH: Optional[Path] = None
# Header name carrying the API key on every authenticated request. Owned by
# ``gitea_saas.auth`` (issue #24 consolidation); re-exported here so existing
# ``from .config import API_KEY_HEADER`` call sites keep working without a
# second copy of the literal.
from .auth import API_KEY_HEADER
# Header name carrying the API key on every authenticated request.
API_KEY_HEADER = "X-API-Key"
# Tier defaults (issue #6, pricing 2026-09-01): GOLD auto-merges on a green
# APPROVE at rate 500 minor units ($5.00) per delivery; STANDARD is reviewed
# but operator-merged at rate 500 ($5.00); FREE is never auto-merged and
# never debited. Rates are config, not code — SAAS_TIERS_JSON overrides
# merge over these defaults.
# Tier defaults (issue #6): GOLD auto-merges on a green APPROVE at rate 100
# minor units per delivery; STANDARD is reviewed but operator-merged at rate
# 25; FREE is never auto-merged and never debited. Rates are config, not
# code — SAAS_TIERS_JSON overrides merge over these defaults.
DEFAULT_TIERS = {
"gold": {"auto_merge": True, "rate_minor": 500},
"standard": {"auto_merge": False, "rate_minor": 500},
"gold": {"auto_merge": True, "rate_minor": 100},
"standard": {"auto_merge": False, "rate_minor": 25},
"free": {"auto_merge": False, "rate_minor": 0},
}
@ -121,8 +116,6 @@ def load_review_config(env: Optional[Mapping[str, str]] = None) -> dict:
| REVIEW_TEST_TIMEOUT_S | test/build timeout seconds | "300" |
| REVIEW_SECRET_SCAN | secret scan on/off ("0" disables) | "1" |
| REVIEW_WORKTREES_DIR | worktree root where PRs are checked out | (none) |
| REVIEW_CLONE_DIR | READ-ONLY repository clone used to build the | (none) |
| | isolated synthetic-merge review workspace | |
"""
source: Mapping[str, str] = os.environ if env is None else env
timeout = 300
@ -141,5 +134,4 @@ def load_review_config(env: Optional[Mapping[str, str]] = None) -> dict:
"test_timeout_s": timeout,
"secret_scan": source.get("REVIEW_SECRET_SCAN", "1") != "0",
"worktrees_dir": source.get("REVIEW_WORKTREES_DIR") or None,
"review_clone_dir": source.get("REVIEW_CLONE_DIR") or None,
}

55
gitea_saas/dashboard.py
View File

@ -30,6 +30,7 @@ serves the exact same dict.
from __future__ import annotations
import hmac
import html
import os
import sqlite3
@ -37,10 +38,14 @@ from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
from .auth import API_KEY_HEADER, key_is_valid, unauthorized_response
EMPTY = "\u2014" # em dash empty-state marker
# Single source for the API-key header name on this branch. The shared
# `config.API_KEY_HEADER` constant lands with issue #2/#6 (neither merged
# yet); when #24 consolidates the service's auth paths, this line is
# replaced by `from .config import API_KEY_HEADER` — no other change.
API_KEY_HEADER = "X-API-Key"
DEFAULT_DEBIT_FLOOR_MINOR = 500
@ -407,7 +412,7 @@ def render_html(summary: Dict[str, Any]) -> str:
<table><tr><th>Repo</th><th>PR</th><th>Verdict</th><th>Head</th><th>Reviewed at</th></tr>{rows_rg}</table>
<h2>5 · Ledger flash</h2>
<table><tr><th>Client</th><th>Kind</th><th>Amount</th><th>Ref</th><th>Time</th></tr>{rows_lf}</table>
<p class="note">Operator page — requires the same API key as the JSON summary. Live numbers, no caching.</p>
<p class="note">Listen-only page: bind the server to 127.0.0.1. Live numbers, no caching.</p>
</body></html>"""
@ -430,14 +435,11 @@ def mount_dashboard(app, core=None, db_path=None, api_key=None):
Registered routes:
* ``GET /dashboard`` — HTML operator page (the issue #25 guard, expressed
through ``gitea_saas.auth``: constant-time ``key_is_valid`` check, any
refusal — missing, wrong, or unset configured key — is the canonical
401 body; never billing data without a valid key).
* ``GET /api/dashboard/summary`` — JSON payload (``X-API-Key`` auth via
``gitea_saas.auth``; any refusal — missing, wrong, or unset configured
key — is the canonical 401 body, fail-closed).
* ``GET /healthz`` — liveness (public, in ``auth.PUBLIC_PATHS``).
* ``GET /dashboard`` — HTML operator page (NO auth; the page renders a
loopback-only warning — the server must bind to 127.0.0.1).
* ``GET /api/dashboard/summary`` — JSON payload (``X-API-Key`` auth;
401 on missing/wrong key, 503 when no key is configured — fail-closed).
* ``GET /healthz`` — liveness.
``api_key`` fills ``app.config['API_KEY']`` only when unset, so the app
factory's key wins when both are configured.
@ -460,30 +462,37 @@ def mount_dashboard(app, core=None, db_path=None, api_key=None):
dashboard = Dashboard(core, db_path)
def _guard(view):
from functools import wraps
@wraps(view)
def guarded(*args, **kwargs):
configured = app.config.get("API_KEY")
if not configured:
return {"error": {"code": "API_KEY_NOT_CONFIGURED",
"message": "SAAS_API_KEY not set"}}, 503
provided = request.headers.get(API_KEY_HEADER, "")
if not hmac.compare_digest(provided, configured):
return {"error": {"code": "UNAUTHORIZED",
"message": "bad or missing X-API-Key"}}, 401
return view(*args, **kwargs)
return guarded
@app.get("/healthz")
def healthz():
return jsonify({"ok": True})
@app.get("/dashboard")
def dashboard_page():
# Guarded through the shared auth module (issues #24/#25): the same
# constant-time key_is_valid check and canonical 401 body as every
# other API-key route — never billing data without a valid key.
# Loopback binding is defense-in-depth only; it was never enforced,
# so it is NOT the authorization boundary. Recompute from the DB at
# request time (no caching).
if not key_is_valid(request.headers.get(API_KEY_HEADER, ""),
app.config.get("API_KEY")):
return unauthorized_response()
# No auth — this page is safe only because the server binds to
# loopback. Recompute from the DB at request time (no caching).
summary = dashboard.summary()
return (render_html(summary), 200,
{"Content-Type": "text/html; charset=utf-8"})
@app.get("/api/dashboard/summary")
@_guard
def dashboard_summary():
if not key_is_valid(request.headers.get(API_KEY_HEADER, ""),
app.config.get("API_KEY")):
return unauthorized_response()
return jsonify(dashboard.summary())
return app

35
gitea_saas/delivery.py
View File

@ -28,26 +28,26 @@ a malformed payload, 200 otherwise) -- the server.py webhook route
from __future__ import annotations
import hmac
import json
import os
from collections.abc import Mapping
from typing import Any, Optional
from .auth import UNAUTHORIZED_BODY, key_is_valid
# --- defaults (overridable via env, mirror issue #6's config.py) ----------
DEFAULT_TIERS: dict[str, dict[str, Any]] = {
"gold": {"rate_minor": 500}, # $5.00 / issue
"standard": {"rate_minor": 500}, # $5.00 / issue
"gold": {"rate_minor": 100}, # $1.00 / issue
"standard": {"rate_minor": 25}, # $0.25 / issue
"free": {"rate_minor": 0}, # never debited
}
DEFAULT_RATE_MINOR = 500 # SAAS_RATE_MINOR (500 = $5.00 / issue)
DEFAULT_RATE_MINOR = 100 # SAAS_RATE_MINOR (100 = $1.00 / issue)
DEFAULT_FLOOR_MINOR = 500 # SAAS_DEBIT_FLOOR_MINOR ($5.00; matches issue #4)
KEY_PREFIX = "deliver"
# Error codes (structured, mirror billing-core conventions).
UNAUTHORIZED = "UNAUTHORIZED"
BAD_PAYLOAD = "BAD_PAYLOAD"
UNKNOWN_CLIENT = "UNKNOWN_CLIENT"
INTERNAL = "INTERNAL"
@ -120,14 +120,22 @@ class Delivery:
# ------------------------------------------------------------------
def authorized(self, provided_key: Optional[str]) -> bool:
"""Fail-closed, constant-time API-key check.
"""Fail-closed, constant-time: a request is authorized only when
the configured ``SAAS_API_KEY`` is set and matches the provided
key exactly.
Single implementation (issue #24): delegates to the shared
``gitea_saas.auth.key_is_valid`` — no comparison lives in this
module. A request is authorized only when ``SAAS_API_KEY`` is set
and matches the provided key exactly.
The comparison uses :func:`hmac.compare_digest` (no timing side
channel on the shared secret's prefix). ``compare_digest`` raises
``TypeError`` on non-str input, so the ``isinstance`` guard runs
first — unset/empty config and non-str/empty keys all return
``False``.
"""
return key_is_valid(provided_key, self.config.get("api_key"))
expected = self.config.get("api_key")
if not expected:
return False
if not isinstance(provided_key, str) or not provided_key:
return False
return hmac.compare_digest(provided_key, expected)
# ------------------------------------------------------------------
# The debit primitive
@ -271,9 +279,8 @@ class Delivery:
fields, 200 otherwise.
"""
if not self.authorized(provided_key):
# Canonical 401 body from gitea_saas.auth (issue #24): identical
# on every front-end's wire.
return (UNAUTHORIZED_BODY, 401)
return ({"ok": False, "error": {"code": UNAUTHORIZED,
"message": "bad or missing X-API-Key"}}), 401
if not isinstance(payload, dict):
return ({"ok": False, "error": {"code": BAD_PAYLOAD,
"message": "expected JSON object"}}), 400

292
gitea_saas/gitea_client.py
View File

@ -1,62 +1,20 @@
"""Gitea API client (mockable seam, env token).
Shared by the factory front-end (issue #4), the review gate (issue #5), and
the production review-policy client operations (issue #28: listing open PRs,
label mutations, merge, issue close). Real HTTP only when a token is
available; the token is read from ``GITEA_TOKEN`` (or passed in) and is
never printed, placed on a command line, or echoed in errors. Tests use a
local HTTP stub or an in-memory fake, so this module is never exercised
against the live instance in CI.
Shared by the factory front-end (issue #4) and the review gate (issue #5).
Real HTTP only when a token is available; the token is read from
``GITEA_TOKEN`` (or passed in) and is never printed or placed on a command
line. Tests use an in-memory fake, so this module is never exercised over
the network in CI.
"""
from __future__ import annotations
import json
import os
import re
import urllib.error
import urllib.request
from typing import Any, Optional
# Gitea list endpoints cap at 50 items per page (the API's ``limit`` bound).
_PER_PAGE = 50
# Safety valve so a misbehaving server that never returns a short page
# cannot make ``_paginate`` loop forever.
_MAX_PAGES = 100
class GiteaClientError(Exception):
"""Base error for the Gitea client.
Messages NEVER include the token or the ``Authorization`` header (the
client redacts the token from any response text it copies into the
message).
"""
class GiteaApiError(GiteaClientError):
"""A non-2xx Gitea API response.
``message`` carries the method, API path, HTTP status, and a truncated
response body — deliberately no request headers, so the authorization
header can never leak.
"""
def __init__(self, method: str, api_path: str, status: int, detail: str = ""):
self.method = method
self.api_path = api_path
self.status = status
self.detail = detail
message = f"Gitea API {method} {api_path} returned HTTP {status}"
if detail:
message += f": {_truncate(detail, 300)}"
super().__init__(message)
def _truncate(text: str, limit: int = 300) -> str:
text = (text or "").strip()
return text if len(text) <= limit else text[:limit] + "…"
class GiteaClient:
"""Thin urllib client for the subset of the Gitea API the SaaS needs.
@ -66,11 +24,6 @@ class GiteaClient:
build ``/api/v1/repos/{owner}/{repo}/...``; org-repo creation uses
``/api/v1/orgs/{owner}/repos`` and falls back to the authenticated
user's repos when ``owner`` is a user account (orgs endpoint 404s).
Every non-2xx response raises :class:`GiteaApiError` (never silently
ignored); the review-policy protocol (issue #28) is fail-closed: an
unknown label or a failed post-condition verification raises
:class:`GiteaClientError` instead of skipping the operation.
"""
def __init__(self, base_url: Optional[str] = None, token: Optional[str] = None,
@ -83,12 +36,7 @@ class GiteaClient:
# -- low-level ---------------------------------------------------------
def _api(self, method: str, api_path: str, body: Optional[dict] = None) -> Any:
"""Perform an API call under ``/api/v1/{api_path}`` and parse JSON.
Non-2xx responses raise :class:`GiteaApiError` with the method, API
path, status, and a redacted response snippet — the authorization
header (and therefore the token) never appears in the message.
"""
"""Perform an API call under ``/api/v1/{api_path}`` and parse JSON."""
data = json.dumps(body).encode() if body is not None else None
url = f"{self.base_url}/api/v1/{api_path}"
req = urllib.request.Request(
@ -96,85 +44,14 @@ class GiteaClient:
headers={"Authorization": "token " + self.token,
"Accept": "application/json",
"Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=30) as resp:
raw = resp.read()
return json.loads(raw) if raw else None
except urllib.error.HTTPError as exc:
detail = ""
try:
raw_body = exc.read()
except Exception: # noqa: BLE001 - the body may be unreadable; fail cleanly
raw_body = b""
if raw_body:
text = raw_body.decode("utf-8", "replace")
# Defensive: some servers echo the request (including its
# headers) back in the error body — redact the token value
# and any echoed Authorization header regardless.
if self.token:
text = text.replace(self.token, "***")
# Scrub any echoed Authorization header completely (name and
# value), not only the token — the header itself must not
# appear in errors.
text = re.sub(
r"(?i)authorization\s*[:=]\s*(?:token\s+)?[^\s,}\]]+",
"***", text)
detail = _truncate(text, 300)
raise GiteaApiError(method, api_path, exc.code, detail) from exc
with urllib.request.urlopen(req, timeout=30) as resp:
raw = resp.read()
return json.loads(raw) if raw else None
def _request(self, method: str, path: str, body: Optional[dict] = None) -> Any:
"""Repo-scoped call: ``path`` = ``<repo>/...`` under the owner."""
return self._api(method, f"repos/{self.owner}/{path}", body)
def _paginate(self, api_path: str) -> list:
"""Fetch every page of a Gitea list endpoint (``GET``).
Follows Gitea pagination (``page``/``limit`` query parameters) until
a page returns fewer than ``_PER_PAGE`` items; a defensive page cap
raises instead of looping forever on a pathologically misbehaving
server.
"""
collected: list = []
for page in range(1, _MAX_PAGES + 1):
sep = "&" if "?" in api_path else "?"
batch = self._api("GET", f"{api_path}{sep}page={page}&limit={_PER_PAGE}")
if not isinstance(batch, list):
raise GiteaClientError(
f"expected a JSON list from {api_path!r}, got {type(batch).__name__}")
collected.extend(batch)
if len(batch) < _PER_PAGE:
break
else:
raise GiteaClientError(
f"pagination did not terminate after {_MAX_PAGES} pages of {api_path!r}")
return collected
def _normalize_pr(self, pr: dict, repo: str = "") -> dict:
"""Map a raw Gitea PR record to the candidate shape.
Every candidate supplies at least ``number``, ``body``, ``head_sha``,
``base_ref``, ``labels`` (label NAMES), ``repo``, ``merged``, and
``state`` — the shape ``ReviewPolicy`` and the review gate consume.
``merged``/``state`` are the raw Gitea record's values (``merged`` is
a boolean; only ``merged is True`` proves a merge — ``state ==
"closed"`` alone can also mean closed without merging).
"""
head = pr.get("head") or {}
base = pr.get("base") or {}
labels = [label.get("name") for label in (pr.get("labels") or [])
if isinstance(label, dict) and label.get("name")]
return {
"number": pr.get("number"),
"body": pr.get("body") or "",
"head_sha": head.get("sha"),
"base_sha": base.get("sha"),
"base_ref": base.get("ref"),
"labels": labels,
"repo": repo,
"merged": bool(pr.get("merged")),
"state": pr.get("state"),
}
# -- API surface used by the factory front-end (issue #4) --------------
def repo_exists(self, name: str) -> bool:
@ -185,8 +62,8 @@ class GiteaClient:
"""
try:
self._request("GET", f"{name}")
except GiteaApiError as exc:
if exc.status == 404:
except urllib.error.HTTPError as exc:
if exc.code == 404:
return False
raise
return True
@ -209,8 +86,8 @@ class GiteaClient:
payload: dict = {"name": name, "auto_init": False, "private": True}
try:
return self._api("POST", f"orgs/{self.owner}/repos", payload)
except GiteaApiError as exc:
if exc.status not in (404, 422):
except urllib.error.HTTPError as exc:
if exc.code not in (404, 422):
raise
# ``owner`` is a user account (or lacks org-create rights): fall back
# to creating the repo under the authenticated token user.
@ -261,20 +138,13 @@ class GiteaClient:
return self._request("GET", f"{repo}/issues/{number}") or {}
def get_pr(self, repo: str, number: int) -> dict:
"""Fetch a PR in the normalized candidate shape.
``{number, body, head_sha, base_sha, base_ref, labels: [names],
repo, merged, state, changed_files: [filename, ...]}`` — the review
gate consumes ``head_sha``/``base_sha`` (the exact evaluated pair,
issue #30) and ``changed_files``; policy consumes the rest
(``merged``/``state`` let recovery positively distinguish a merged
PR from a merely closed one).
"""
"""Fetch a PR: {head_sha, changed_files: [filename, ...]}."""
pr = self._request("GET", f"{repo}/pulls/{number}") or {}
files = self._request("GET", f"{repo}/pulls/{number}/files") or []
normalized = self._normalize_pr(pr, repo)
normalized["changed_files"] = [f.get("filename") for f in files if f.get("filename")]
return normalized
return {
"head_sha": ((pr.get("head") or {}).get("sha")),
"changed_files": [f.get("filename") for f in files if f.get("filename")],
}
def get_pr_diff(self, repo: str, number: int) -> str:
"""Return the PR's diff text reconstructed from per-file patches."""
@ -290,129 +160,3 @@ class GiteaClient:
def post_comment(self, repo: str, number: int, body: str) -> dict:
"""Post a comment on an issue or PR conversation."""
return self._request("POST", f"{repo}/issues/{number}/comments", {"body": body}) or {}
# -- API surface used by the review policy (issue #28) ------------------
#
# Every operation in this section is fail-closed: a non-2xx response
# raises GiteaApiError, an unknown label or a failed post-condition
# verification raises GiteaClientError. Nothing is silently skipped.
def list_open_prs(self, repo: str) -> list:
"""All open PRs in ``owner/repo``, normalized, pagination followed.
Requests ``?state=open`` (all and only open PRs) page by page and
returns the normalized candidate shape (see ``_normalize_pr``).
"""
prs = self._paginate(f"repos/{self.owner}/{repo}/pulls?state=open")
return [self._normalize_pr(pr, repo) for pr in prs]
def add_label(self, repo: str, issue_number: int, label: str) -> None:
"""Add ``label`` to ``owner/repo`` issue ``issue_number``.
Resolves the repository label id, POSTs it (``{"labels": [id]}``),
and verifies the resulting label state. Already-present labels are
a no-op. An unknown label or a failed verification raises a clear
:class:`GiteaClientError` — never silently skipped.
"""
name = (label or "").strip()
if not name:
raise GiteaClientError("add_label: label must be a non-empty string")
label_id = self.label_id(repo, name)
if label_id is None:
raise GiteaClientError(
f"add_label: unknown label {name!r} in repo {self.owner}/{repo}")
if name in self._issue_label_names(repo, issue_number):
return # already present: idempotent no-op
self._request("POST", f"{repo}/issues/{issue_number}/labels", {"labels": [label_id]})
if name not in self._issue_label_names(repo, issue_number):
raise GiteaClientError(
f"add_label: label {name!r} missing on issue {issue_number} after POST")
def remove_label(self, repo: str, issue_number: int, label: str) -> None:
"""Remove ``label`` from ``owner/repo`` issue ``issue_number``.
Resolves the repository label id, DELETEs it, and verifies the
resulting label state. Absent labels are a no-op. An unknown label
or a failed verification raises a clear :class:`GiteaClientError`.
"""
name = (label or "").strip()
if not name:
raise GiteaClientError("remove_label: label must be a non-empty string")
label_id = self.label_id(repo, name)
if label_id is None:
raise GiteaClientError(
f"remove_label: unknown label {name!r} in repo {self.owner}/{repo}")
if name not in self._issue_label_names(repo, issue_number):
return # already absent: idempotent no-op
self._api("DELETE",
f"repos/{self.owner}/{repo}/issues/{issue_number}/labels/{label_id}")
if name in self._issue_label_names(repo, issue_number):
raise GiteaClientError(
f"remove_label: label {name!r} still on issue {issue_number} after DELETE")
def merge_pr(self, repo: str, pr_number: int,
expected_head_sha: Optional[str] = None) -> bool:
"""Merge ``owner/repo`` PR ``pr_number``; True when it is merged.
Uses the Gitea merge endpoint and confirms the PR is merged after
the request. The merge request carries the Gitea schema's
mandatory ``Do`` mode and the reviewed ``head_commit_id``, so a
branch that moved after review cannot be merged accidentally.
``expected_head_sha`` (optional, preserving the two-argument
protocol) is the head the caller actually REVIEWED. When given,
the pre-merge GET result is compared against it and a mismatch
raises :class:`GiteaClientError` BEFORE any POST — closing the
TOCTOU window between review and merge instead of only guarding
the milliseconds between the in-method GET and POST. The
reviewed SHA itself is sent as ``head_commit_id``. When omitted,
the live PR head is read (and pinned) as before, which keeps the
existing callers working while the policy call sites migrate to
passing the reviewed head.
An already-merged PR short-circuits to True (no duplicate merge
request). Non-2xx responses (e.g. 409 when the head is not
mergeable) raise :class:`GiteaApiError`; a PR record without a
head commit id raises :class:`GiteaClientError` rather than
merging unpinned.
"""
current = self._request("GET", f"{repo}/pulls/{pr_number}") or {}
if current.get("merged"):
return True
live_head = (current.get("head") or {}).get("sha")
if expected_head_sha is not None and live_head != expected_head_sha:
raise GiteaClientError(
f"merge_pr: PR {pr_number} head is {live_head!r} but the "
f"reviewed head is {expected_head_sha!r}; refusing to merge "
"a branch that moved after review")
head_commit_id = expected_head_sha if expected_head_sha is not None \
else live_head
if not head_commit_id:
raise GiteaClientError(
f"merge_pr: PR {pr_number} record has no head commit id; "
"refusing to merge without the reviewed head")
self._request("POST", f"{repo}/pulls/{pr_number}/merge",
{"Do": "merge", "head_commit_id": head_commit_id})
merged = self._request("GET", f"{repo}/pulls/{pr_number}") or {}
return bool(merged.get("merged"))
def close_issue(self, repo: str, issue_number: int) -> None:
"""Close ``owner/repo`` issue ``issue_number``.
Updates the issue state and confirms the issue reports ``closed``
afterwards; a failed verification raises :class:`GiteaClientError`.
"""
self._request("PATCH", f"{repo}/issues/{issue_number}", {"state": "closed"})
issue = self._request("GET", f"{repo}/issues/{issue_number}") or {}
if issue.get("state") != "closed":
raise GiteaClientError(
f"close_issue: issue {issue_number} state is {issue.get('state')!r} "
"after close request")
# -- shared helpers -----------------------------------------------------
def _issue_label_names(self, repo: str, issue_number: int) -> list:
"""Label NAMES currently on ``owner/repo`` issue ``issue_number``."""
issue = self._request("GET", f"{repo}/issues/{issue_number}") or {}
return [label.get("name") for label in (issue.get("labels") or [])
if isinstance(label, dict) and label.get("name")]

419
gitea_saas/intake_eval.py
View File

@ -10,25 +10,12 @@ file into ``SAAS_EVAL_OUTPUT_DIR``.
Money rule (fail-closed, mirroring issue #4's factory semantics): a batch
is only accepted when the client's available balance is at or above the
debit floor. Unfunded/unknown clients get a 402-style structured refusal,
a locally queued row (``pending_eval_batches``) and NO Gitea issues are
ever created for them; once funding arrives, `flush_pending` ships the
queued batch exactly like a fresh funded submit (issue #39).
Funds discipline (issue #39): a funded submit reserves a batch hold
(configurable ``eval_batch_hold_minor`` / ``SAAS_EVAL_BATCH_HOLD_MINOR``,
default = the debit floor) so the client's available balance visibly backs
the in-flight batch. ``cancel_batch`` releases that hold and stops draining
(a cancel marker; the batch no longer polls or emits). Rolled-back intake
releases the hold too, so a failed submit never leaves money locked.
debit floor. Unfunded/unknown clients get a 402-style structured refusal
and NO Gitea issues are ever created for them.
Schema additions are owned here (``schema.sql`` belongs to issue #1):
``eval_batches``, ``eval_tasks``, ``pending_eval_batches`` and
``eval_batch_cancels`` are bootstrapped idempotently against the same
SQLite file as the billing core, exactly like the factory does. The
``hold_id`` column is added to pre-existing ``eval_batches`` tables via a
guarded ``ALTER TABLE`` (SQLite cannot amend a CHECK constraint, but
adding a column is idempotent and safe).
``eval_batches`` and ``eval_tasks`` are bootstrapped idempotently against
the same SQLite file as the billing core, exactly like the factory does.
"""
from __future__ import annotations
@ -43,21 +30,10 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
# Single auth path (issue #24): the header name, the constant-time comparison
# and the 401 body all live in ``gitea_saas.auth``; this module delegates and
# never implements its own key check (issue #39 "do not add a sixth").
from .auth import API_KEY_HEADER, key_is_valid, unauthorized_response
# Default funded threshold: a client must hold at least 5.00 USD before an
# eval batch is accepted (same default as the factory front-end, issue #4).
DEFAULT_DEBIT_FLOOR_MINOR = 500
# Default per-batch hold reserved at submit (issue #39): the client's
# available balance visibly backs the in-flight batch until it emits or is
# cancelled. Config attr ``eval_batch_hold_minor`` / env
# ``SAAS_EVAL_BATCH_HOLD_MINOR`` override; 0 disables the hold entirely.
DEFAULT_BATCH_HOLD_MINOR = DEFAULT_DEBIT_FLOOR_MINOR
DEFAULT_EVAL_REPO = "eval-runs"
EVAL_LABEL = "eval-task"
@ -69,12 +45,6 @@ _STATUS_FAILED = "failed"
_TASK_PENDING = "pending"
_TASK_DONE = "done"
# pending_eval_batches lifecycle (issue #39, mirroring the factory's
# pending_issues): queued while the client is unfunded, shipped once
# ``flush_pending`` creates the real batch + task issues.
_STATUS_QUEUED = "queued"
_STATUS_SHIPPED = "shipped"
_REPO_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$")
# Idempotent by construction (same convention as the billing schema).
@ -85,7 +55,6 @@ CREATE TABLE IF NOT EXISTS eval_batches (
status TEXT NOT NULL DEFAULT 'open'
CHECK (status IN ('open', 'done', 'emitted', 'failed')),
callback_url TEXT,
hold_id INTEGER,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
@ -104,26 +73,6 @@ CREATE TABLE IF NOT EXISTS eval_tasks (
);
CREATE INDEX IF NOT EXISTS idx_eval_tasks_batch ON eval_tasks (batch_id, status);
-- Unfunded submissions queue here (issue #39, factory mirror): no Gitea
-- issue is ever created for them; flush_pending ships them post-funding.
CREATE TABLE IF NOT EXISTS pending_eval_batches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
client_id INTEGER NOT NULL,
tasks_json TEXT NOT NULL,
callback_url TEXT,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
status TEXT NOT NULL DEFAULT 'queued'
CHECK (status IN ('queued', 'shipped'))
);
-- Cancel markers (issue #39): "stop draining" without touching the batch
-- status CHECK constraint on pre-existing tables. A marker row means the
-- batch no longer polls or emits and its hold was released.
CREATE TABLE IF NOT EXISTS eval_batch_cancels (
batch_id INTEGER PRIMARY KEY REFERENCES eval_batches(id),
cancelled_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
"""
@ -218,15 +167,6 @@ class EvalIntake:
self._conn.execute("PRAGMA journal_mode=WAL")
self._conn.execute("PRAGMA foreign_keys=ON")
self._conn.executescript(_EVAL_DDL)
# Idempotent column migration for DB files created before #39's
# hold_id landed: SQLite cannot amend a CHECK, but ADD COLUMN is
# safe and re-runnable (PRAGMA-guarded).
cols = {row[1] for row in self._conn.execute(
"PRAGMA table_info(eval_batches)").fetchall()}
if "hold_id" not in cols:
self._conn.execute(
"ALTER TABLE eval_batches ADD COLUMN hold_id INTEGER")
self._conn.commit()
self._lock = threading.RLock()
# ------------------------------------------------------------------
@ -262,22 +202,6 @@ class EvalIntake:
def per_batch_repos(self) -> bool:
return bool(getattr(self._config, "per_batch_repos", False))
@property
def batch_hold_minor(self) -> int:
"""Per-batch hold reserved at submit (issue #39).
Config attr ``eval_batch_hold_minor`` > env
``SAAS_EVAL_BATCH_HOLD_MINOR`` > ``DEFAULT_BATCH_HOLD_MINOR``.
0 disables the hold (cancel then has nothing to release).
"""
value = getattr(self._config, "eval_batch_hold_minor", None)
if value is None:
raw = os.environ.get("SAAS_EVAL_BATCH_HOLD_MINOR", "").strip()
value = int(raw) if raw else DEFAULT_BATCH_HOLD_MINOR
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
raise EvalIntakeError(f"invalid batch hold: {value!r}")
return value
# ------------------------------------------------------------------
# Billing helpers (fail-closed, factory semantics)
# ------------------------------------------------------------------
@ -330,45 +254,16 @@ class EvalIntake:
callback_url: Optional[str] = None) -> Dict[str, Any]:
"""Accept a funded batch; one eval-task issue per task.
Funded clients get ``{ok: True, batch_id, repo, issues,
hold_id}``; the batch hold is reserved from the client's balance
(issue #39). Unfunded/unknown clients (fail-closed) get
``{ok: False, reason: 'unfunded', status: 402, queued: True,
pending_id}``, a local ``pending_eval_batches`` row and NO Gitea
issues — ``flush_pending`` ships it once funding arrives.
Returns ``{ok: True, batch_id, repo, issues: [{task_id,
issue_number}]}``. Unfunded/unknown clients get a 402-style
``{ok: False, reason: 'unfunded', status: 402}`` and NO Gitea
issues are created.
"""
self._require_valid_inputs(client_id, tasks)
if self.client_unfunded(client_id):
return self._queue_batch(client_id, tasks, callback_url)
return self._ship_batch(client_id, tasks, callback_url)
def _queue_batch(self, client_id: int, tasks: List[dict],
callback_url: Optional[str]) -> Dict[str, Any]:
"""Persist an unfunded submission locally (no Gitea mutations)."""
return {"ok": False, "reason": "unfunded", "status": 402}
callback_url = (callback_url or "").strip() or None
with self._lock:
cur = self._conn.execute(
"INSERT INTO pending_eval_batches"
" (client_id, tasks_json, callback_url, created_at, status)"
" VALUES (?, ?, ?, ?, ?)",
(client_id, json.dumps(tasks), callback_url, _now_iso(),
_STATUS_QUEUED))
self._conn.commit()
return {"ok": False, "reason": "unfunded", "status": 402,
"queued": True, "pending_id": int(cur.lastrowid)}
def _ship_batch(self, client_id: int, tasks: List[dict],
callback_url: Optional[str],
tasks_json: Optional[str] = None) -> Dict[str, Any]:
"""Create batch row + eval-task issues; reserve the batch hold.
The hold is placed BEFORE any Gitea mutation and released (with the
batch rolled back) if intake fails, so a failed submit never leaves
money locked or a partial batch behind. ``tasks_json`` is only
passed by ``flush_pending`` (the payload was already persisted in
the pending row).
"""
callback_url = (callback_url or "").strip() or None
with self._lock:
cur = self._conn.execute(
"INSERT INTO eval_batches (client_id, status, callback_url, created_at)"
@ -383,27 +278,6 @@ class EvalIntake:
repo = self.eval_repo
if not _REPO_NAME_RE.match(repo):
repo = DEFAULT_EVAL_REPO
# Money first (issue #39): reserve the batch hold; a client
# whose balance cannot back the batch is refused, not shipped.
hold_id = None
hold_amount = self.batch_hold_minor
if hold_amount > 0:
ok, details = self._core.reserve_hold(
client_id, hold_amount, f"eval-batch-{batch_id}",
ref_type="eval", ref_id=str(batch_id))
if not ok:
self._conn.execute(
"DELETE FROM eval_batches WHERE id = ?", (batch_id,))
self._conn.commit()
error = details if isinstance(details, dict) else {"raw": details}
return {"ok": False, "reason": "intake_failed",
"error": error, "status": 500}
hold_id = int(details["hold_id"])
self._conn.execute(
"UPDATE eval_batches SET hold_id = ? WHERE id = ?",
(hold_id, batch_id))
try:
if not self._gitea.repo_exists(repo):
self._gitea.create_org_repo(repo)
@ -423,8 +297,7 @@ class EvalIntake:
" task_id, payload_json, status, created_at)"
" VALUES (?, ?, ?, ?, ?, ?, ?)",
(batch_id, repo, issue_number, task_id,
tasks_json if tasks_json is not None
else json.dumps({"title": title, "body": body}),
json.dumps({"title": title, "body": body}),
_TASK_PENDING, _now_iso()))
issues.append({"task_id": task_id,
"issue_number": issue_number})
@ -434,116 +307,12 @@ class EvalIntake:
"DELETE FROM eval_tasks WHERE batch_id = ?", (batch_id,))
self._conn.execute(
"DELETE FROM eval_batches WHERE id = ?", (batch_id,))
if hold_id is not None:
try:
self._core.release_hold(
hold_id, f"eval-rollback-{batch_id}")
except Exception:
pass # best effort; the hold auto-expires in 30 min
self._conn.commit()
return {"ok": False, "reason": "intake_failed",
"error": str(exc), "status": 500}
return {"ok": True, "batch_id": batch_id, "repo": repo,
"issues": issues, "hold_id": hold_id}
def flush_pending(self, client_id: int) -> Dict[str, Any]:
"""Ship a client's queued (unfunded) batches once they are funded.
Factory mirror (issue #39): unfunded work never becomes a Gitea
issue, so this refuses to flush while the client is still
unfunded. Each shipped batch is created exactly like a fresh
funded submit (hold reserved, task issues created, batch row
written) and its pending row is marked ``shipped``.
"""
with self._lock:
rows = self._conn.execute(
"SELECT * FROM pending_eval_batches WHERE client_id = ?"
" AND status = ? ORDER BY id",
(client_id, _STATUS_QUEUED)).fetchall()
if not rows:
return {"ok": True, "flushed": 0, "batch_ids": [], "issues": []}
if self.client_unfunded(client_id):
return {"ok": False, "reason": "unfunded",
"skipped": len(rows), "flushed": 0}
batch_ids: List[int] = []
issues: List[dict] = []
for row in rows:
try:
tasks = json.loads(row["tasks_json"] or "[]")
except ValueError:
tasks = []
result = self._ship_batch(
client_id, tasks, row["callback_url"],
tasks_json=row["tasks_json"])
if not result.get("ok"):
# A single failure stops the flush (fail-closed, mirroring
# intake's no-partial-batches rule); earlier shipped rows
# stay shipped.
return {"ok": False, "reason": "flush_failed",
"error": result.get("error"),
"flushed": len(batch_ids), "batch_ids": batch_ids,
"issues": issues}
with self._lock:
self._conn.execute(
"UPDATE pending_eval_batches SET status = ? WHERE id = ?",
(_STATUS_SHIPPED, row["id"]))
self._conn.commit()
batch_ids.append(result["batch_id"])
issues.extend(result["issues"])
return {"ok": True, "flushed": len(batch_ids),
"batch_ids": batch_ids, "issues": issues}
# ------------------------------------------------------------------
# Cancelling a batch (issue #39)
# ------------------------------------------------------------------
def cancel_batch(self, batch_id: int) -> Dict[str, Any]:
"""Stop draining a batch and release its held funds.
Writes a cancel marker (``eval_batch_cancels``) so polling and
emitting become no-ops, and releases the batch hold — the client's
available balance returns to its pre-submit value exactly.
Idempotent: a second cancel returns ``already_cancelled`` and
never releases twice. A hold the core already auto-expired
(``HOLD_ALREADY_RELEASED``/``HOLD_NOT_FOUND``) is treated as
already gone; the batch is still marked cancelled.
"""
batch = self._batch_row(batch_id)
if batch is None:
return {"ok": False, "error": "unknown_batch"}
with self._lock:
marker = self._conn.execute(
"SELECT 1 FROM eval_batch_cancels WHERE batch_id = ?",
(batch_id,)).fetchone()
if marker is not None:
return {"ok": True, "batch_id": batch_id,
"cancelled": True, "already_cancelled": True,
"hold_released": False}
hold_id = batch["hold_id"]
hold_released = False
if hold_id is not None:
ok, details = self._core.release_hold(
int(hold_id), f"eval-cancel-{batch_id}")
if not ok:
code = ((details or {}).get("error") or {}).get(
"code") if isinstance(details, dict) else None
if code not in ("HOLD_ALREADY_RELEASED", "HOLD_NOT_FOUND"):
return {"ok": False, "reason": "cancel_failed",
"error": details}
else:
hold_released = True
self._conn.execute(
"INSERT INTO eval_batch_cancels (batch_id, cancelled_at)"
" VALUES (?, ?)", (batch_id, _now_iso()))
self._conn.commit()
return {"ok": True, "batch_id": batch_id, "cancelled": True,
"hold_released": hold_released}
def _is_cancelled(self, batch_id: int) -> bool:
return self._conn.execute(
"SELECT 1 FROM eval_batch_cancels WHERE batch_id = ?",
(batch_id,)).fetchone() is not None
"issues": issues}
# ------------------------------------------------------------------
# Polling completed tasks
@ -563,14 +332,11 @@ class EvalIntake:
A task is done when its issue is closed (the lane merged); the
result record captures the final conversation comment (the lane's
report) plus the issue's labels. A cancelled batch (issue #39)
stops draining: ``{ok: False, error: 'cancelled'}``.
report) plus the issue's labels.
"""
batch = self._batch_row(batch_id)
if batch is None:
return {"ok": False, "error": "unknown_batch"}
if self._is_cancelled(batch_id):
return {"ok": False, "error": "cancelled"}
done = 0
pending = 0
for row in self._task_rows(batch_id):
@ -645,14 +411,10 @@ class EvalIntake:
Callback mode: POST to ``callback_url`` (retry x3, backoff) via the
emitter seam. File mode: write ``SAAS_EVAL_OUTPUT_DIR/<batch_id>.json``.
A batch already emitted re-emits idempotently (no second POST/write).
A cancelled batch (issue #39) refuses to emit: ``{ok: False,
reason: 'cancelled'}``.
"""
batch = self._batch_row(batch_id)
if batch is None:
return {"ok": False, "error": "unknown_batch"}
if self._is_cancelled(batch_id):
return {"ok": False, "reason": "cancelled"}
pending = self._conn.execute(
"SELECT COUNT(*) FROM eval_tasks WHERE batch_id = ?"
" AND status = 'pending'", (batch_id,)).fetchone()[0]
@ -703,159 +465,4 @@ class EvalIntake:
for t in tasks:
t["payload_json"] = json.loads(t["payload_json"] or "{}")
t["result_json"] = json.loads(t["result_json"] or "null")
marker = self._conn.execute(
"SELECT cancelled_at FROM eval_batch_cancels WHERE batch_id = ?",
(batch_id,)).fetchone()
data = dict(batch)
if marker is not None:
data["status"] = "cancelled"
data["cancelled_at"] = marker["cancelled_at"]
return {"ok": True, "batch": data, "tasks": tasks,
"cancelled": marker is not None}
def get_results(self, batch_id: int) -> Dict[str, Any]:
"""The results table for a batch (issue #39 HTTP surface).
Read-only view: one row per task with its current status, title
and collected summary — regardless of whether the batch is still
draining, emitted or cancelled. Unknown batches are
``{ok: False, error: 'unknown_batch'}``.
"""
batch = self._batch_row(batch_id)
if batch is None:
return {"ok": False, "error": "unknown_batch"}
report = self._report(batch)
if self._is_cancelled(batch_id):
report["status"] = "cancelled"
report["ok"] = True
return report
# ---------------------------------------------------------------------------
# HTTP surface (issue #39; mounted by the assembly issue #38's create_app)
# ---------------------------------------------------------------------------
#
# Symmetric with register_factory_routes: the same five-route shape, the
# same X-API-Key discipline (gitea_saas.auth only — issue #24; no sixth
# implementation), and the same local-queue money rule. The one deliberate
# difference: a route reached while NO key is configured returns 503
# SERVICE_UNAVAILABLE (fail-closed "cannot serve"), as #38's assembly
# acceptance demands app-wide. /api/eval/health is unauthenticated like
# the factory's /api/factory/health (its handler never calls _require_auth).
def register_eval_routes(app: Any, core: Any, gitea: Any, config: Any) -> None:
"""Mount the eval-farm endpoints on a Flask app (idempotent).
Routes:
POST /api/eval/batches {"client_id": int,
"tasks": [{task_id, title,
body?}],
"callback_url"?: str}
GET /api/eval/batches/<id> status + per-task state
GET /api/eval/batches/<id>/results the results table
POST /api/eval/batches/<id>/cancel stop draining + release hold
GET /api/eval/health liveness (no auth)
Each authed route builds an :class:`EvalIntake` per request (cheap
SQLite connection, WAL + per-instance lock — same pattern as the
factory). Auth failures use the canonical 401 body; unconfigured key
(empty expected) is 503 SERVICE_UNAVAILABLE.
"""
from flask import jsonify, request
api_key = app.config.get("API_KEY") or getattr(config, "api_key", None) or ""
def _eval() -> EvalIntake:
return EvalIntake(core, gitea, config)
def _require_auth():
# Fail-closed on a misconfigured deployment: no key configured
# means the service cannot authenticate anything -> 503 (issue
# #38's assembly acceptance demands this app-wide).
if not api_key:
resp = jsonify({"error": {
"code": "SERVICE_UNAVAILABLE",
"message": "API key not configured"}})
resp.status_code = 503
return resp
sent = request.headers.get(API_KEY_HEADER, "")
if not key_is_valid(sent, api_key):
return unauthorized_response()
return None
@app.get("/api/eval/health")
def eval_health():
return jsonify({"ok": True})
@app.post("/api/eval/batches")
def eval_submit_batch():
if (denied := _require_auth()) is not None:
return denied
body = request.get_json(silent=True) or {}
raw_id = body.get("client_id")
if isinstance(raw_id, bool) or not isinstance(raw_id, int) or raw_id <= 0:
return _json_error(400, "invalid_request",
"client_id must be a positive integer")
callback_url = body.get("callback_url")
try:
result = _eval().submit_batch(raw_id, body.get("tasks") or [],
callback_url)
except Exception as exc:
return _map_eval_error(exc)
resp = jsonify(result)
# Funded: 201 created. Unfunded/queued: 202 accepted locally
# (factory mirror: the row flushes once funding arrives).
resp.status_code = 201 if result.get("ok") else 202
return resp
@app.get("/api/eval/batches/<int:batch_id>")
def eval_get_batch(batch_id: int):
if (denied := _require_auth()) is not None:
return denied
try:
result = _eval().get_batch(batch_id)
except Exception as exc:
return _map_eval_error(exc)
if not result.get("ok"):
return _json_error(404, "unknown_batch", "no such batch")
return jsonify(result)
@app.get("/api/eval/batches/<int:batch_id>/results")
def eval_get_results(batch_id: int):
if (denied := _require_auth()) is not None:
return denied
try:
result = _eval().get_results(batch_id)
except Exception as exc:
return _map_eval_error(exc)
if not result.get("ok"):
return _json_error(404, "unknown_batch", "no such batch")
return jsonify(result)
@app.post("/api/eval/batches/<int:batch_id>/cancel")
def eval_cancel_batch(batch_id: int):
if (denied := _require_auth()) is not None:
return denied
try:
result = _eval().cancel_batch(batch_id)
except Exception as exc:
return _map_eval_error(exc)
if not result.get("ok"):
return _json_error(404, "unknown_batch", "no such batch")
return jsonify(result)
def _json_error(status: int, code: str, message: str):
from flask import jsonify
resp = jsonify({"error": message, "code": code})
resp.status_code = status
return resp
def _map_eval_error(exc: Exception):
"""Map eval domain errors to HTTP responses (fail-closed on unknown)."""
if isinstance(exc, EvalIntakeError):
return _json_error(400, "invalid_request", str(exc))
return _json_error(500, "eval_error", f"{type(exc).__name__}: {exc}")
return {"ok": True, "batch": dict(batch), "tasks": tasks}

30
gitea_saas/intake_factory.py
View File

@ -23,6 +23,7 @@ shipped without payment.
from __future__ import annotations
import hmac
import json
import os
import re
@ -32,11 +33,15 @@ from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
# Single auth path (issue #24): the header name, the constant-time comparison
# and the 401 body all live in ``gitea_saas.auth``; this module delegates.
# config.py is long merged, so there is no fallback constant here — a failed
# import of a security-relevant constant must be loud, not silently re-hardcoded.
from .auth import API_KEY_HEADER, key_is_valid, unauthorized_response
# Header carrying the API key. The shared constant lives in the sibling
# ``gitea_saas/config.py`` (issue #2/#6 auth), which lands in a later PR than
# this one; until that module is merged this file falls back to the same
# canonical value. Keeping the single source of truth means the factory
# routes use ``compare_digest`` on the same header name as ``server.py``.
try:
from .config import API_KEY_HEADER # shared with gitea_saas.server
except ImportError: # config.py not merged yet (sibling auth PRs)
API_KEY_HEADER = "X-API-Key"
# Default floor for "funded": a client must hold at least this many minor
# units (default = 5.00 USD) before intake will ship issues.
@ -337,8 +342,8 @@ class IntakeFactory:
# HTTP surface (mounted by the issue #2 Flask app; see register_factory_routes)
# ---------------------------------------------------------------------------
#
# Routes (single auth path, issue #24 -- gitea_saas.auth.key_is_valid via
# API_KEY_HEADER, fail-closed; /api/factory/health is in PUBLIC_PATHS):
# Routes (same auth discipline as gitea_saas.server -- API_KEY_HEADER via
# hmac.compare_digest, fail-closed):
# POST /api/factory/provision {"client_id": int, "repo_name"?: str}
# POST /api/factory/<client_id>/issues {"repo", "title", "acceptance_body",
# "labels"?: [...]}
@ -370,10 +375,15 @@ def register_factory_routes(app: Any, core: Any, gitea: Any, config: Any) -> Non
def _factory() -> IntakeFactory:
return IntakeFactory(core, gitea, config)
def _require_auth():
def _authorized() -> bool:
sent = request.headers.get(API_KEY_HEADER, "")
if not key_is_valid(sent, api_key):
return unauthorized_response()
return bool(api_key) and hmac.compare_digest(sent, api_key)
def _require_auth():
if not _authorized():
resp = jsonify({"error": "missing or invalid API key", "code": "unauthorized"})
resp.status_code = 401
return resp
return None
@app.get("/api/factory/health")

552
gitea_saas/review_policy.py
View File

@ -9,13 +9,10 @@ Seams (all duck-typed, faked in tests):
(see ``billing_core.BillingCore``).
* ``gitea`` — Gitea client: ``list_open_prs(repo=None)``,
``get_pr(repo, n)``, ``get_issue(repo, n)``, ``post_comment``,
``add_label``, ``remove_label``, ``merge_pr(repo, n,
expected_head_sha=None) -> bool``, ``close_issue(repo, n)``. The
normalized PR shape carries ``merged`` (bool) and ``state`` so the
policy can positively distinguish a merged PR from a merely closed one.
``add_label``, ``remove_label``, ``merge_pr(repo, n) -> bool``,
``close_issue(repo, n)``.
* ``reviewer`` — the review gate: ``review_pr(repo, pr, linked_issue,
workspace=...) -> Verdict`` (the isolated synthetic-merge workspace,
issue #30) and ``post_verdict(repo, pr, verdict)``.
repo_dir=...) -> Verdict`` and ``post_verdict(repo, pr, verdict)``.
Policy
------
@ -34,52 +31,6 @@ head moves. The table is bootstrapped here (``CREATE TABLE IF NOT EXISTS``)
because ``schema.sql`` belongs to issue #1's PR and is out of this issue's
allowed file set — same schema intent, idempotent.
Evaluated-pair integrity (issue #30): the reviewer evaluates a synthetic
merge of the exact head onto the exact base and the verdict pins both
SHAs. Before ANY verdict is applied, the policy re-reads the PR and
refuses to apply the verdict unless BOTH the head and the base are still
the evaluated pair — a verdict describing a pair that no longer exists is
never acted on (no merge, label, comment, or close), and the new pair is
simply eligible for a fresh review on the next poll. The merge request
itself additionally pins the reviewed head (``expected_head_sha``, issue
#28) as a second, tighter guard.
Transactional outcome application (issue #29): a head is recorded into
``reviews`` (i.e. considered COMPLETED) only after its required policy
action has fully succeeded. Failure/incomplete attempts are persisted in
the companion ``review_state`` table — one row per head carrying the
attempt count, the last error, and idempotent step checkpoints
(``merged_step``, ``closed_step``, ``debit_step``, ``label_step``,
``comment_step``) — so a transient failure at any boundary (verdict
generation, label, comment, merge, close, debit) leaves the same
unchanged head eligible for retry on the next poll instead of stranding it.
Crash-window recovery is NOT driven by the open-PR scan. A merge is the
point of no return: once the PR is truly merged it disappears from
``list_open_prs()``, so any later failure (debit, label removal, issue
close, comment) would strand the persisted row. Therefore the outcome
plan (verdict, tier, linked issue) is persisted into ``review_state``
BEFORE the merge, every post-merge side effect is checkpointed, and
``recover_incomplete()`` resumes those rows directly from the retry table
(via ``get_pr`` + checkpoints) — independently of any open-PR listing —
before the regular poll scan runs. A merge is only ever adopted when the
client reports positive proof (``merged is True``, now part of the
production normalized shape); ``state == "closed"`` without a merge is
never treated as a merge, so a closed-but-unmerged PR cannot trigger a
debit, label removal, issue close, or success comment. Retries re-apply
only the steps that have not already been checkpointed, so a partial side
effect can never double-merge a PR, double-close an issue, double-remove
a label, or double-post the success comment; the delivery debit is
additionally protected by its existing billing-core idempotency key
``<repo>#<pr>@<head_sha>``. The merge request itself is pinned to the
reviewed head: ``_safe_merge`` passes ``expected_head_sha`` through to the
client so a branch that moved after review is refused before any POST
(issue #28 client contract). Legacy ``reviews`` rows (pre-#29 schema) are
preserved untouched and keep their completed meaning; the retry state
lives entirely in the new table, so no migration of existing records is
needed (new step columns are ALTER-added idempotently for databases
created by earlier builds).
Delivery debit: implemented inline here behind the ``debit_for_merge``
signature shared with the delivery issue (#7); when delivery lands it can
refactor to call it. FAIL-CLOSED: this module never fake-succeeds a debit —
@ -112,45 +63,6 @@ CREATE TABLE IF NOT EXISTS reviews (
)
"""
# Retry state for heads whose outcome application is in flight or failed
# (issue #29). One row per head; ``attempts`` counts failed/incomplete
# attempts, ``last_error`` keeps the most recent failure, and the step
# checkpoints record which idempotent side effects already happened so a
# retry resumes instead of re-applying them. ``tier``/``linked_issue``
# snapshot the outcome plan BEFORE the merge so ``recover_incomplete()``
# can finish the post-merge steps even after the PR leaves the open-PR
# scan. Rows are deleted once the head completes (a ``reviews`` row is
# written instead).
_REVIEW_STATE_DDL = """
CREATE TABLE IF NOT EXISTS review_state (
repo TEXT NOT NULL,
pr INTEGER NOT NULL,
head_sha TEXT NOT NULL,
verdict TEXT,
attempts INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
last_attempt_at TEXT,
merged_step INTEGER NOT NULL DEFAULT 0,
closed_step INTEGER NOT NULL DEFAULT 0,
debit_step INTEGER NOT NULL DEFAULT 0,
label_step INTEGER NOT NULL DEFAULT 0,
comment_step INTEGER NOT NULL DEFAULT 0,
tier TEXT,
linked_issue INTEGER,
PRIMARY KEY (repo, pr, head_sha)
)
"""
# Columns added after the initial #29 schema shipped (compat: databases
# created by earlier builds get these via idempotent ALTERs).
_REVIEW_STATE_EXTRA_COLUMNS = {
"debit_step": "debit_step INTEGER NOT NULL DEFAULT 0",
"label_step": "label_step INTEGER NOT NULL DEFAULT 0",
"comment_step": "comment_step INTEGER NOT NULL DEFAULT 0",
"tier": "tier TEXT",
"linked_issue": "linked_issue INTEGER",
}
_LINK_RE = re.compile(r"(?im)\b(?:closes|fixes|resolves|close|fix|resolve)\s+#(\d+)\b")
# Markdown-tolerant: "**Tier:** gold" / "- Tier: gold" / "**Client:** acme" all parse
# (the value can sit after formatting chars like "*" / "**" following the colon).
@ -186,156 +98,26 @@ class ReviewPolicy:
self._reviews_conn = sqlite3.connect(str(self._db_path))
self._reviews_conn.execute("PRAGMA journal_mode=WAL")
self._reviews_conn.execute(_REVIEWS_DDL)
self._reviews_conn.execute(_REVIEW_STATE_DDL)
self._ensure_review_state_columns()
self._reviews_conn.commit()
# Isolated review workspace (issue #30): reviews run against a clean
# temporary synthetic-merge worktree taken from the configured clone
# (REVIEW_CLONE_DIR), never against an arbitrary worktrees root.
self._workspace = None
clone_dir = self.config.get("review_clone_dir")
if clone_dir:
from .review_workspace import ReviewWorkspace
self._workspace = ReviewWorkspace(clone_dir, self.reviewer.runner)
def _ensure_review_state_columns(self) -> None:
"""Idempotently add columns introduced after the first #29 build."""
existing = {row[1] for row in self._reviews_conn.execute(
"PRAGMA table_info(review_state)")}
for column, ddl in _REVIEW_STATE_EXTRA_COLUMNS.items():
if column not in existing:
self._reviews_conn.execute(
f"ALTER TABLE review_state ADD COLUMN {ddl}")
# ------------------------------------------------------------------
# Reviews table (completed-history) + review_state (retry metadata)
# Reviews table (re-review safety)
# ------------------------------------------------------------------
def _already_reviewed(self, repo: str, pr: int, head_sha: str) -> bool:
"""True once a head's outcome application COMPLETED successfully.
A failed/incomplete attempt lives only in ``review_state`` and does
not make a head "reviewed": the same unchanged head must stay
eligible for retry.
"""
row = self._reviews_conn.execute(
"SELECT 1 FROM reviews WHERE repo = ? AND pr = ? AND head_sha = ?",
(repo, pr, head_sha)).fetchone()
return row is not None
def _record_completed(self, repo: str, pr: int, head_sha: str, verdict: str) -> None:
"""Persist a fully applied outcome; drop the retry state for the head."""
def _record_review(self, repo: str, pr: int, head_sha: str, verdict: str) -> None:
self._reviews_conn.execute(
"INSERT OR IGNORE INTO reviews (repo, pr, head_sha, verdict, reviewed_at)"
" VALUES (?, ?, ?, ?, ?)",
(repo, pr, head_sha, verdict,
datetime.now(timezone.utc).isoformat()))
self._reviews_conn.execute(
"DELETE FROM review_state WHERE repo = ? AND pr = ? AND head_sha = ?",
(repo, pr, head_sha))
self._reviews_conn.commit()
def _record_failure(self, repo: str, pr: int, head_sha: str,
verdict: Optional[str], error: str) -> None:
"""Persist a failed/incomplete attempt (upserts per head).
Step checkpoints on an existing row are preserved, so a retry of a
partially applied outcome resumes from where it left off.
"""
now = datetime.now(timezone.utc).isoformat()
self._reviews_conn.execute(
"INSERT INTO review_state (repo, pr, head_sha, verdict, attempts,"
" last_error, last_attempt_at)"
" VALUES (?, ?, ?, ?, 1, ?, ?)"
" ON CONFLICT(repo, pr, head_sha) DO UPDATE SET"
" verdict = excluded.verdict,"
" attempts = review_state.attempts + 1,"
" last_error = excluded.last_error,"
" last_attempt_at = excluded.last_attempt_at",
(repo, pr, head_sha, verdict, error, now))
self._reviews_conn.commit()
_STEP_COLUMNS = {"merged": "merged_step", "closed": "closed_step",
"debit": "debit_step", "label": "label_step",
"comment": "comment_step"}
_POST_MERGE_STEPS = ("debit", "label", "closed", "comment")
def _ensure_plan(self, repo: Optional[str], pr: int, head_sha: Optional[str],
verdict: str, tier: Optional[str],
issue_num: Optional[int]) -> None:
"""Persist the outcome plan BEFORE the first side effect (#29).
The merge is the point of no return: once merged, the PR leaves
the open-PR scan, so every remaining step (debit, label removal,
issue close, success comment) must be recoverable from this row
alone via :meth:`recover_incomplete`. Upserts; preserves attempt
counts and step checkpoints from earlier attempts.
"""
self._reviews_conn.execute(
"INSERT INTO review_state (repo, pr, head_sha, verdict, tier,"
" linked_issue)"
" VALUES (?, ?, ?, ?, ?, ?)"
" ON CONFLICT(repo, pr, head_sha) DO UPDATE SET"
" verdict = excluded.verdict,"
" tier = excluded.tier,"
" linked_issue = excluded.linked_issue",
(repo, pr, head_sha, verdict, tier, issue_num))
self._reviews_conn.commit()
def _linked_issue(self, repo: Optional[str], pr: int,
head_sha: Optional[str]) -> Optional[int]:
row = self._reviews_conn.execute(
"SELECT linked_issue FROM review_state"
" WHERE repo = ? AND pr = ? AND head_sha = ?",
(repo, pr, head_sha)).fetchone()
return row[0] if row else None
def _step_done(self, repo: Optional[str], pr: int,
head_sha: Optional[str], step: str) -> bool:
"""True when the idempotent step already succeeded for this head."""
column = self._STEP_COLUMNS.get(step)
if column is None:
raise ValueError(f"unknown retry step {step!r}")
row = self._reviews_conn.execute(
f"SELECT {column} FROM review_state"
" WHERE repo = ? AND pr = ? AND head_sha = ?",
(repo, pr, head_sha)).fetchone()
return bool(row and row[0])
def _mark_step(self, repo: Optional[str], pr: int, head_sha: Optional[str],
step: str) -> None:
"""Checkpoint a succeeded side effect so a retry never re-applies it."""
column = self._STEP_COLUMNS.get(step)
if column is None:
raise ValueError(f"unknown retry step {step!r}")
self._reviews_conn.execute(
f"INSERT INTO review_state (repo, pr, head_sha, {column})"
" VALUES (?, ?, ?, 1)"
f" ON CONFLICT(repo, pr, head_sha) DO UPDATE SET {column} = 1",
(repo, pr, head_sha))
self._reviews_conn.commit()
def _attempts(self, repo: str, pr: int, head_sha: str) -> int:
row = self._reviews_conn.execute(
"SELECT attempts FROM review_state"
" WHERE repo = ? AND pr = ? AND head_sha = ?",
(repo, pr, head_sha)).fetchone()
return row[0] if row else 0
@staticmethod
def _error_text(exc: Exception) -> str:
return f"{type(exc).__name__}: {exc}"
def _failure_result(self, repo: Optional[str], pr: int, head_sha: str,
verdict: Optional[str], exc: Exception) -> dict:
return {
"repo": repo, "pr": pr, "head_sha": head_sha, "verdict": verdict,
"outcome": "failed",
"error": {"type": type(exc).__name__, "message": str(exc)},
"attempts": self._attempts(repo or "", pr, head_sha),
}
# ------------------------------------------------------------------
# Polling
# ------------------------------------------------------------------
@ -356,217 +138,24 @@ class ReviewPolicy:
return str(issue or "")
def poll_and_review(self, repo: Optional[str] = None) -> list[dict]:
"""Review every in-scope PR at its current head; apply tier policy.
Transactional + retryable (issue #29): a head is only recorded as
completed AFTER its required policy action succeeded; any failure —
verdict generation or a policy action — is persisted as a retryable
attempt and reported as a structured failure result, leaving the
same unchanged head eligible for the next poll.
Stranded post-merge applications are recovered FIRST, directly
from the ``review_state`` table: a truly merged PR is no longer
returned by ``list_open_prs()``, so the open-PR scan can never
reach those rows again (see :meth:`recover_incomplete`).
"""
results = list(self.recover_incomplete(repo))
"""Review every in-scope PR at its current head; apply tier policy."""
results = []
for pr in self.list_candidate_prs(repo):
repo_key = repo or (pr.get("repo") or "")
pr_number = pr["number"]
head_sha = pr.get("head_sha") or "unknown"
if self._already_reviewed(repo_key, pr_number, head_sha):
if self._already_reviewed(repo or (pr.get("repo") or ""), pr_number, head_sha):
continue
try:
pr_body = pr.get("body") or ""
issue_body = self._fetch_issue_body(repo, linked_issue_number(pr_body))
verdict = self.reviewer.review_pr(
repo, pr_number, issue_body, workspace=self._workspace)
except Exception as exc: # noqa: BLE001 - any failure must be retryable
self._record_failure(repo_key, pr_number, head_sha, None,
self._error_text(exc))
results.append(self._failure_result(repo, pr_number, head_sha,
None, exc))
continue
# Evaluated-pair integrity (issue #30): the verdict pins the exact
# head/base it evaluated. Re-read the PR BEFORE applying anything
# and refuse when either end of the pair moved — a verdict for a
# pair that no longer exists must never merge/label/comment/close,
# and the new pair stays eligible for a fresh review on the next
# poll (nothing is recorded as completed for the stale pair).
try:
live_pr = self.gitea.get_pr(repo_key, pr_number) or {}
pair_moved = (live_pr.get("head_sha") != verdict.head_sha
or live_pr.get("base_sha") != verdict.base_sha)
except Exception as exc: # noqa: BLE001 - re-read failure is retryable
self._record_failure(repo_key, pr_number, head_sha, verdict.verdict,
self._error_text(exc))
results.append(self._failure_result(repo, pr_number, head_sha,
verdict.verdict, exc))
continue
if pair_moved:
results.append({
"repo": repo, "pr": pr_number,
"head_sha": head_sha, "verdict": verdict.verdict,
"outcome": "stale",
"error": {
"code": "PAIR_MOVED",
"message": (
f"PR pair moved after evaluation: evaluated head "
f"{verdict.head_sha!r} / base {verdict.base_sha!r}, "
f"live head {live_pr.get('head_sha')!r} / base "
f"{live_pr.get('base_sha')!r} — verdict NOT applied; "
"the new pair is eligible for a fresh review")},
"evaluated": {"head_sha": verdict.head_sha,
"base_sha": verdict.base_sha},
"live": {"head_sha": live_pr.get("head_sha"),
"base_sha": live_pr.get("base_sha")},
})
continue
try:
action = self._apply_policy(repo, pr, pr_body, issue_body, verdict)
except Exception as exc: # noqa: BLE001 - any failure must be retryable
self._record_failure(repo_key, pr_number, head_sha, verdict.verdict,
self._error_text(exc))
results.append(self._failure_result(repo, pr_number, head_sha,
verdict.verdict, exc))
continue
if action.get("action") in ("merge_failed", "merge_but_debit_failed"):
# The required action did NOT succeed: keep the head retryable
# and persist the incomplete attempt (fail-closed branches
# report a structured non-completion instead of raising).
self._record_failure(repo_key, pr_number, head_sha, verdict.verdict,
f"code={action.get('action')}")
results.append({
"repo": repo, "pr": pr_number, "head_sha": head_sha,
"verdict": verdict.verdict, "outcome": "failed",
"error": {"code": action.get("action")},
"attempts": self._attempts(repo_key, pr_number, head_sha),
**action})
continue
self._record_completed(repo_key, pr_number, head_sha, verdict.verdict)
pr_body = pr.get("body") or ""
issue_body = self._fetch_issue_body(repo, linked_issue_number(pr_body))
verdict = self.reviewer.review_pr(
repo, pr_number, issue_body,
repo_dir=self.config.get("worktrees_dir"))
self._record_review(repo or "", pr_number, head_sha, verdict.verdict)
action = self._apply_policy(repo, pr, pr_body, issue_body, verdict)
results.append({"repo": repo, "pr": pr_number, "head_sha": head_sha,
"verdict": verdict.verdict, **action})
return results
def recover_incomplete(self, repo: Optional[str] = None) -> list[dict]:
"""Resume stranded outcome applications directly from retry state.
Independent of the open-PR scan: ``list_open_prs()`` cannot reach
a head whose PR was truly merged (it is no longer open), so any
post-merge step that failed — debit, label removal, issue close,
success comment — would otherwise be lost forever. This method
walks ``review_state`` rows and finishes each one from its
checkpoints, re-fetching the PR by number via ``get_pr`` (which
works for closed/merged PRs) rather than from any open-PR listing.
Only rows whose merge is already confirmed (checkpointed, or the
live PR positively reports ``merged is True`` at the EXACT
planned/reviewed head — the production normalized shape includes
the raw ``merged`` boolean and ``head_sha``) are handled here; rows
for still-open PRs are left to the regular poll scan, which can
reach them. ``state == "closed"`` alone never counts as merged: a
PR closed without merging must not trigger a debit, label removal,
issue close, or success comment. A PR merged at a DIFFERENT head
than the plan reviewed is never adopted — it fails closed with a
structured ``HEAD_MISMATCH`` and no side effects (the debit,
label, close and comment steps all belong to the reviewed head,
not to whatever head actually landed).
Completed heads are recorded into ``reviews`` and their retry row
deleted; a step that fails again is persisted with an incremented
attempt count for the next recovery pass.
"""
results = []
where, params = (" WHERE repo = ?", [repo]) if repo else ("", [])
rows = self._reviews_conn.execute(
f"SELECT repo, pr, head_sha, verdict FROM review_state{where}",
params).fetchall()
for row in rows:
row_repo, pr_number, head_sha, verdict = row
if verdict is None:
continue # no policy action started yet; open scan owns it
try:
live = None
try:
live = self.gitea.get_pr(row_repo, pr_number) or {}
except Exception: # noqa: BLE001 - PR may be gone; checkpoints decide
live = None
pr_merged = bool(live and live.get("merged") is True)
merged_done = self._step_done(row_repo, pr_number, head_sha,
"merged")
if pr_merged and not merged_done:
# Crash window: the merge applied server-side but the
# checkpoint write never happened — adopt it ONLY when
# the live merged head is the EXACT planned/reviewed
# head (head-integrity gate, governor 10539). A merge
# of a DIFFERENT head must never be attributed to this
# plan's debit/label/close/comment steps.
live_head = (live or {}).get("head_sha")
if live_head != head_sha:
message = (
f"merged head mismatch: live head {live_head!r} "
f"!= planned/reviewed head {head_sha!r} — the "
f"outcome application for the reviewed head is "
f"refused (no debit, no label removal, no issue "
f"close, no success comment)")
self._record_failure(row_repo, pr_number, head_sha,
verdict, message)
results.append({
"repo": row_repo, "pr": pr_number,
"head_sha": head_sha, "verdict": verdict,
"outcome": "failed",
"error": {"code": "HEAD_MISMATCH",
"message": message},
"attempts": self._attempts(
row_repo, pr_number, head_sha),
})
continue
# Same-head crash window: adopt and finish the plan.
self._mark_step(row_repo, pr_number, head_sha, "merged")
merged_done = True
if not merged_done:
continue # still reachable via the open scan; leave it
issue_body = self._fetch_issue_body(
row_repo, self._linked_issue(row_repo, pr_number, head_sha))
if not self._step_done(row_repo, pr_number, head_sha, "debit"):
ok, debit = self.debit_for_merge(
row_repo, pr_number, head_sha, issue_body)
if not ok:
raise RuntimeError(
f"recover debit failed: {debit.get('error') or debit}")
self._mark_step(row_repo, pr_number, head_sha, "debit")
else:
debit = {"debit": "recorded", "replayed": True}
if not self._step_done(row_repo, pr_number, head_sha, "label"):
self.gitea.remove_label(row_repo, pr_number, "in-review")
self._mark_step(row_repo, pr_number, head_sha, "label")
self._safe_close_issue(row_repo, pr_number, head_sha,
self._linked_issue(row_repo, pr_number,
head_sha))
if not self._step_done(row_repo, pr_number, head_sha, "comment"):
summary = ("delivery debit recorded"
if debit.get("debit") == "recorded"
else "no delivery debit (tier rate 0)")
self.gitea.post_comment(
row_repo, pr_number,
f"Merged by review-gate — {summary}.")
self._mark_step(row_repo, pr_number, head_sha, "comment")
self._record_completed(row_repo, pr_number, head_sha, verdict)
results.append({"repo": row_repo, "pr": pr_number,
"head_sha": head_sha, "verdict": verdict,
"outcome": "merged", "action": "merged",
"debit": debit, "recovered": True})
except Exception as exc: # noqa: BLE001 - any failure must be retryable
self._record_failure(row_repo, pr_number, head_sha, verdict,
self._error_text(exc))
results.append(self._failure_result(row_repo, pr_number,
head_sha, verdict, exc))
return results
# ------------------------------------------------------------------
# Tier resolution + delivery debit
# ------------------------------------------------------------------
@ -627,77 +216,6 @@ class ReviewPolicy:
# Tier policy application
# ------------------------------------------------------------------
def _safe_merge(self, repo: Optional[str], pr: dict) -> bool:
"""Merge guarded for idempotent retry (issue #29).
Never merges twice for the same head: the ``merged_step`` checkpoint
makes a retry skip an already-applied merge, and when the fetched PR
shape already positively reports ``merged is True`` (crash window
between the API side effect and the checkpoint write) the merge is
skipped as well. ``state == "closed"`` alone is NOT adopted — a PR
can be closed without merging, and adopting it would debit the
client, strip labels, and close the linked issue for a merge that
never happened.
The merge request is pinned to the REVIEWED head: the exact
``head_sha`` this policy reviewed is passed through to
``gitea.merge_pr(..., expected_head_sha=...)``, whose production
implementation (issue #28) fails closed before any POST when the
live branch moved after review — an unreviewed SHA can never be
merged. A candidate that carries NO reviewed head_sha is rejected
BEFORE the client is called (head-integrity gate, governor 10539):
``expected_head_sha=None`` would arm the client's backward-
compatible live-head fallback and could merge a head this policy
never reviewed, so the merge is refused up front and the failure
is surfaced as a structured retryable result with zero side
effects.
"""
repo_key = repo or ""
pr_number = pr["number"]
reviewed_head = pr.get("head_sha") or ""
if not reviewed_head:
raise ValueError(
"merge refused: candidate PR carries no reviewed head_sha — "
"expected_head_sha must be pinned; refusing to fall back to "
"the live head")
head_sha = reviewed_head
if self._step_done(repo_key, pr_number, head_sha, "merged"):
return True
if pr.get("merged") is True:
self._mark_step(repo_key, pr_number, head_sha, "merged")
return True
merged = self.gitea.merge_pr(
repo, pr_number, expected_head_sha=reviewed_head)
if merged:
self._mark_step(repo_key, pr_number, head_sha, "merged")
return merged
def _safe_close_issue(self, repo: Optional[str], pr_number: int,
head_sha: Optional[str], issue_num: Optional[int]) -> None:
"""Close guarded for idempotent retry (issue #29).
Skips when the ``closed_step`` checkpoint says this head already
closed the issue, when the head already completed its outcome
application (poll path), or when the live issue state is already
closed. ``head_sha`` may be None (webhook path): the checkpoint is
keyed on the head only when one is known.
"""
repo_key = repo or ""
head = head_sha or "unknown"
if issue_num is None or self._step_done(repo_key, pr_number, head, "closed"):
return
if self._already_reviewed(repo_key, pr_number, head):
return
try:
issue = self.gitea.get_issue(repo, issue_num)
except Exception: # noqa: BLE001 - fail-closed: let close_issue decide
issue = None
if isinstance(issue, dict) and issue.get("state") == "closed":
self._mark_step(repo_key, pr_number, head, "closed")
return
self.gitea.close_issue(repo, issue_num)
self._mark_step(repo_key, pr_number, head, "closed")
def _apply_policy(self, repo: Optional[str], pr: dict, pr_body: str,
issue_body: str, verdict) -> dict:
pr_number = pr["number"]
@ -712,43 +230,22 @@ class ReviewPolicy:
"Approved — awaiting operator merge "
f"(tier `{tier}`, auto_merge off).")
return {"action": "approved_awaiting_merge", "tier": tier}
# Persist the outcome plan BEFORE the merge: once the PR is
# truly merged it leaves the open-PR scan, so every following
# step must be recoverable from the row alone (issue #29 /
# review bounce: merged PRs vanish from list_open_prs()).
head_sha = pr.get("head_sha") or ""
if not head_sha:
# Head-integrity gate (governor 10539): a candidate without
# a reviewed head_sha is rejected BEFORE any plan write or
# client call — expected_head_sha must be pinned, never the
# client's live-head fallback.
raise ValueError(
"merge refused: candidate PR carries no reviewed head_sha — "
"expected_head_sha must be pinned; refusing to fall back to "
"the live head")
issue_num = linked_issue_number(pr_body)
self._ensure_plan(repo, pr_number, head_sha, verdict.verdict,
tier, issue_num)
merged = self._safe_merge(repo, pr)
merged = self.gitea.merge_pr(repo, pr_number)
if not merged:
self.gitea.post_comment(
repo, pr_number,
"[review-gate] merge failed — PR left open for manual review.")
return {"action": "merge_failed", "tier": tier}
ok, debit = self.debit_for_merge(repo, pr_number, head_sha, issue_body)
ok, debit = self.debit_for_merge(repo, pr_number, pr.get("head_sha"), issue_body)
self.gitea.remove_label(repo, pr_number, "in-review")
issue_num = linked_issue_number(pr_body)
if ok:
self._mark_step(repo, pr_number, head_sha, "debit")
if not self._step_done(repo, pr_number, head_sha, "label"):
self.gitea.remove_label(repo, pr_number, "in-review")
self._mark_step(repo, pr_number, head_sha, "label")
if ok:
self._safe_close_issue(repo, pr_number, head_sha, issue_num)
if issue_num is not None:
self.gitea.close_issue(repo, issue_num)
summary = "delivery debit recorded" if debit.get("debit") == "recorded" \
else "no delivery debit (tier rate 0)"
if not self._step_done(repo, pr_number, head_sha, "comment"):
self.gitea.post_comment(repo, pr_number,
f"Merged by review-gate — {summary}.")
self._mark_step(repo, pr_number, head_sha, "comment")
self.gitea.post_comment(repo, pr_number,
f"Merged by review-gate — {summary}.")
return {"action": "merged", "tier": tier, "debit": debit}
# fail-closed: merge happened but the debit failed — do NOT close
# the issue, tell the operator.
@ -793,7 +290,8 @@ class ReviewPolicy:
else "no debit for this tier.")
)
issue_num = linked_issue_number(pr.get("body") or "")
self._safe_close_issue(repo, pr_number, head_sha, issue_num)
if issue_num is not None:
self.gitea.close_issue(repo, issue_num)
return ok, {"pr": pr_number, "head_sha": head_sha, "debit": debit}
self.gitea.post_comment(
repo, pr_number,

265
gitea_saas/review_workspace.py
View File

@ -1,265 +0,0 @@
"""Isolated review workspace (issue #30).
The review gate must never run tests against an arbitrary ``repo_dir``:
a green verdict must describe the exact synthetic merge result it
actually evaluated. This module builds that result in a clean,
isolated, temporary Git worktree taken from a configured repository
clone —
* the workspace never changes or tests a shared ``main`` checkout or a
worker's issue worktree (the configured clone is only ever *read* by
``git fetch`` / ``git worktree add``; its checked-out branch and
working tree are untouched);
* the named base SHA and PR head SHA are fetched into the clone and the
exact head is merged onto the exact base (a real merge commit, so
tests and ``git diff --check <base> HEAD`` run against a real,
inspectable merge result with an explicit base/result range);
* a merge conflict produces a deterministic, evidence-carrying result
(``merge_ok=False``) and the caller — the review gate — never runs the
test suite against some fallback checkout;
* the temporary worktree is removed on success, conflict, timeout, and
raised exceptions (best-effort ``git worktree remove --force`` plus a
belt-and-braces directory removal), and no branch refs are ever
created (the worktree is detached), so no temporary refs survive.
All git commands go through the same ``Runner`` subprocess seam as the
rest of the review gate, so hermetic tests can use local-path ``origin``
remotes (zero network) and/or a scripted command runner to assert the
exact commands, working directories, and ranges. Every invocation here
passes an **argument list** (``shell=False`` in the real Runner): no
shell ever interprets the arguments, so a ``^{commit}`` suffix, a path
containing spaces/``$``, or any other shell-special character survives
verbatim on every platform (fixes the Windows cmd.exe ``^``-eating
break — issue #48).
"""
from __future__ import annotations
import re
import shutil
import tempfile
from pathlib import Path
from typing import Callable, Optional
_SHA_RE = re.compile(r"^[0-9a-f]{40}$")
# Characters kept in a mkdtemp prefix token. ``repo`` names routinely
# contain separators (``owner/name``) and punctuation; a prefix is a bare
# filename component, so everything else is collapsed to ``-`` (and the
# results trimmed) to keep ``tempfile.mkdtemp`` able to create it.
_FS_SAFE_RE = re.compile(r"[^A-Za-z0-9_-]+")
def _fs_safe_token(value: str) -> str:
"""Collapse ``value`` into a filesystem-safe prefix token."""
return _FS_SAFE_RE.sub("-", value).strip("-")
_MERGE_MESSAGE = "synthetic merge for review"
class ReviewWorkspaceError(Exception):
"""The isolated workspace could not be constructed (fetch/resolve
failure, missing identity, etc.). Not a merge conflict — the caller
treats it as a retryable operational failure."""
class MergeWorkspace:
"""A constructed synthetic-merge workspace (context manager).
Attributes
----------
dir : Path
The temporary worktree (working directory for test/build/hygiene
commands). HEAD is the synthetic merge commit of ``head_sha``
onto ``base_sha`` when ``merge_ok`` is True.
base_sha / head_sha : str
The exact SHAs that were merged (what the review evaluates and
pins).
merge_ok : bool
True when the synthetic merge completed; False on conflict.
evidence : dict
Machine-readable evidence: merge command, exit, output summary,
conflicted files (conflict path), result HEAD SHA (clean path).
"""
def __init__(self, dir: Path, base_sha: str, head_sha: str,
merge_ok: bool, evidence: dict,
cleanup: Optional[Callable[[], None]] = None):
self.dir = Path(dir)
self.base_sha = base_sha
self.head_sha = head_sha
self.merge_ok = merge_ok
self.evidence = evidence
self._cleanup = cleanup
def __enter__(self) -> "MergeWorkspace":
return self
def __exit__(self, exc_type, exc, tb) -> None:
"""Remove the temporary worktree + directory on every path.
``git worktree remove --force`` cleans the clone's worktree
admin entry even when the tree is dirty or the merge left
conflict state behind; the directory removal is a belt-and-
braces cleanup that never masks the caller's result.
"""
if self._cleanup is not None:
try:
self._cleanup()
except Exception: # noqa: BLE001 - cleanup must never mask the caller's result
pass
shutil.rmtree(self.dir, ignore_errors=True)
def __repr__(self) -> str: # pragma: no cover - debugging aid
return (f"<MergeWorkspace {self.dir} merge_ok={self.merge_ok} "
f"base={self.base_sha[:12]} head={self.head_sha[:12]}>")
class ReviewWorkspace:
"""Builds isolated synthetic-merge workspaces from a configured clone.
``clone_dir`` is the READ-ONLY configured repository clone (the
clone owns the worktree administration; its checked-out branch and
working tree are never modified). ``runner`` is the review gate's
subprocess seam (``Runner`` or a scripted fake); all git commands
run through it so tests stay hermetic.
"""
def __init__(self, clone_dir, runner):
self.clone_dir = Path(clone_dir)
self.runner = runner
# -- construction ------------------------------------------------------
def build(self, repo: str, pr_number: int, head_sha: str,
base_sha: str) -> MergeWorkspace:
"""Construct the synthetic-merge workspace for one review.
Raises :class:`ReviewWorkspaceError` on operational failures
(unfetchable/unresolvable SHAs, missing git identity, worktree
creation failure). Returns a :class:`MergeWorkspace` with
``merge_ok=False`` + conflict evidence for a genuine merge
conflict, or ``merge_ok=True`` with the merged worktree.
"""
for label, sha in (("base", base_sha), ("head", head_sha)):
if not _SHA_RE.match(sha or ""):
raise ReviewWorkspaceError(
f"cannot build an isolated review workspace: {label} SHA "
f"{sha!r} is not a 40-char hex commit id — refusing to "
"review an unchecked-out pair")
self._fetch_refresh(base_sha, head_sha)
# ``repo`` is embedded in the temp-dir prefix: sanitize it so a
# separator/punctuation-containing name cannot turn the prefix into
# a nested path (which would make ``mkdtemp`` fail).
tmpdir = Path(tempfile.mkdtemp(
prefix=f"review-ws-{_fs_safe_token(repo)}-{pr_number}-"))
evidence: dict = {
"base_sha": base_sha,
"head_sha": head_sha,
"worktree_dir": str(tmpdir),
}
try:
add = self.runner.run(
self.clone_dir,
["git", "worktree", "add", "--detach",
str(tmpdir), base_sha])
if add.get("exit") != 0:
raise ReviewWorkspaceError(
"cannot create the temporary review worktree: "
f"{add.get('summary', '')[:400]}")
merge = self.runner.run(
tmpdir,
["git", "merge", "--no-ff", "-m",
_MERGE_MESSAGE, head_sha])
evidence["merge_command"] = (
["git", "merge", "--no-ff", "-m",
_MERGE_MESSAGE, head_sha])
evidence["merge_exit"] = merge.get("exit")
evidence["merge_summary"] = merge.get("summary", "")[:800]
if merge.get("exit") != 0:
# Distinguish a real merge conflict (MERGE_HEAD exists) from an
# environmental merge failure (no conflict state was entered).
merget = self.runner.run(
tmpdir, ["git", "rev-parse", "-q",
"--verify", "MERGE_HEAD"])
if merget.get("exit") != 0:
raise ReviewWorkspaceError(
"synthetic merge failed without entering a conflict "
f"state (environmental failure): {merge.get('summary', '')[:400]}")
conflicted = self.runner.run(
tmpdir, ["git", "diff", "--name-only",
"--diff-filter=U"])
status = self.runner.run(
tmpdir, ["git", "status", "--porcelain"])
evidence["conflicted_files"] = [
line for line in (conflicted.get("summary") or "").splitlines()
if line.strip()]
evidence["status_summary"] = (status.get("summary") or "")[:800]
return MergeWorkspace(
tmpdir, base_sha, head_sha, merge_ok=False,
evidence=evidence, cleanup=self._make_cleanup(tmpdir))
result_head = self.runner.run(
tmpdir, ["git", "rev-parse", "HEAD"])
evidence["result_head"] = (result_head.get("summary") or "").strip()
return MergeWorkspace(
tmpdir, base_sha, head_sha, merge_ok=True,
evidence=evidence, cleanup=self._make_cleanup(tmpdir))
except Exception:
# Guarantee cleanup even when construction failed mid-way: a
# dangling worktree or temp dir is exactly what must never
# survive a review pass.
try:
self._make_cleanup(tmpdir)()
except Exception: # noqa: BLE001 - best-effort cleanup on top of a failure
pass
shutil.rmtree(tmpdir, ignore_errors=True)
raise
# -- internals ----------------------------------------------------------
def _make_cleanup(self, tmpdir: Path) -> Callable[[], None]:
"""A cleanup closure: ``git worktree remove --force`` + rmtree."""
def cleanup() -> None:
try:
self.runner.run(
self.clone_dir,
["git", "worktree", "remove", "--force",
str(tmpdir)])
except Exception: # noqa: BLE001 - cleanup is best-effort
pass
shutil.rmtree(tmpdir, ignore_errors=True)
return cleanup
def _fetch_refresh(self, base_sha: str, head_sha: str) -> None:
"""Ensure the clone can resolve both SHAs.
Prefers the precise fetch of exactly the two SHAs; falls back to a
full ``git fetch origin`` (which refreshes every remote-tracking
ref — the configured clone must be current for reviews anyway) and
re-verifies. Hermetic tests use a local-path ``origin``.
"""
res = self.runner.run(
self.clone_dir,
["git", "fetch", "origin", base_sha, head_sha])
if res.get("exit") != 0:
res = self.runner.run(
self.clone_dir, ["git", "fetch", "origin"])
if res.get("exit") != 0:
raise ReviewWorkspaceError(
"cannot fetch the named base/head SHAs from the configured "
f"clone's origin: {res.get('summary', '')[:400]}")
for label, sha in (("base", base_sha), ("head", head_sha)):
chk = self.runner.run(
self.clone_dir,
["git", "rev-parse", "--verify", f"{sha}^{{commit}}"])
if chk.get("exit") != 0:
raise ReviewWorkspaceError(
f"cannot resolve the {label} SHA {sha} as a commit in the "
f"configured clone after fetching: {chk.get('summary', '')[:300]}")

140
gitea_saas/reviewer.py
View File

@ -9,17 +9,9 @@ Design
* ``Reviewer(repo_client, runner)`` — ``repo_client`` is a git/Gitea seam
(see ``gitea_client.GiteaClient`` for the real one), ``runner`` is a
subprocess seam (``Runner`` here; a scripted fake in tests).
* ``review_pr(repo, pr_number, linked_issue, workspace)`` returns a
* ``review_pr(repo, pr_number, linked_issue, repo_dir)`` returns a
``Verdict``; the caller decides what to do. ``post_verdict`` posts the
formatted ``[review-gate]`` comment. ``workspace`` is the isolated
review-workspace seam (``review_workspace.ReviewWorkspace``): the
review ALWAYS runs against the synthetic merge result of the exact
reviewed head onto the exact reviewed base, constructed in a clean
temporary Git worktree — never against an arbitrary ``repo_dir``, a
shared ``main`` checkout, or a worker's issue worktree (issue #30).
A merge conflict yields a deterministic BOUNCE with evidence and the
test suite is NOT run against any fallback checkout; with no
workspace configured the gate fails closed to NEEDS_GUIDANCE.
formatted ``[review-gate]`` comment.
* FAIL-CLOSED: the review gate NEVER merges — there is no merge call in
this module. Policy and merge are issue #7.
* Every finding MUST quote the diff. The default (no LLM configured) pass
@ -143,16 +135,6 @@ def _redact_quote(text: str, match: "re.Match[str]") -> str:
class Runner:
"""Subprocess seam. ``command_runner`` is injectable for hermetic tests.
``run`` accepts a **command** that is either
* an argument **list** (``["git", "rev-parse", "--verify", f"{sha}^{{commit}}"]``)
— executed with ``shell=False`` so no shell ever interprets the
arguments (``^``, ``~``, ``&``, ``|``, ``<``, ``>``, ``%``, spaces,
…) — the Windows-portable form used by every git invocation in this
package (issue #48); or
* a **string**, for callers that genuinely need shell semantics
(``python -m pytest``, admin-configured ``build_cmd``).
Returns ``{"exit": int | "timed_out", "summary": tail-of-output}``.
"""
@ -160,17 +142,10 @@ class Runner:
command_runner: Optional[Callable] = None):
self.timeout_s = timeout_s
self._command_runner = command_runner or (
lambda command, **kwargs:
subprocess.run(command, shell=isinstance(command, str),
**kwargs))
lambda command, **kwargs: subprocess.run(command, shell=True, **kwargs))
def run(self, repo_dir, command, timeout_s: Optional[int] = None) -> dict:
"""Run ``command`` (arg list, or string) with ``cwd=repo_dir``."""
def run(self, repo_dir, command: str, timeout_s: Optional[int] = None) -> dict:
timeout = timeout_s or self.timeout_s
if isinstance(command, str) and command.lstrip().startswith("["):
raise TypeError(
"Runner.run command must be an argument LIST or a shell "
f"string, not a stringified list: {command!r}")
try:
completed = self._command_runner(
command, cwd=str(repo_dir), timeout=timeout,
@ -191,20 +166,11 @@ class Runner:
@dataclass
class Verdict:
"""Evidence-based outcome of a review gate run.
``base_sha`` pins the base commit the synthetic merge was built on
(alongside ``head_sha``) — every review records and pins the
repository, PR number, head SHA, and current base SHA it actually
evaluated (issue #30). ``workspace`` carries the isolated-workspace
evidence (``merge_ok``, ``conflicted_files``, …) when the verdict
came from a synthetic-merge review.
"""
"""Evidence-based outcome of a review gate run."""
repo: str
pr: int
head_sha: str
base_sha: str = ""
acceptance: list[str] = field(default_factory=list)
tests: dict = field(default_factory=dict)
build: Optional[dict] = None
@ -212,22 +178,11 @@ class Verdict:
findings: list[dict] = field(default_factory=list)
verdict: str = NEEDS_GUIDANCE
summary: str = ""
workspace: dict = field(default_factory=dict)
def to_comment(self) -> str:
"""Format the [review-gate] comment (checklist + findings)."""
lines = [f"[review-gate] VERDICT: **{self.verdict}**", "",
f"Repo: `{self.repo}` · PR: #{self.pr} · head `{self.head_sha[:12]}`"
+ (f" · base `{self.base_sha[:12]}`" if self.base_sha else ""), ""]
if self.workspace.get("conflicted_files"):
lines += ["Synthetic merge of the reviewed head onto the reviewed base: "
"**CONFLICT**",
"The test suite was NOT run — no fallback checkout is ever tested.",
"Conflicted files:"]
lines += [f"- `{f}`" for f in self.workspace["conflicted_files"]]
if self.workspace.get("status_summary"):
lines += ["```", self.workspace["status_summary"][:400], "```"]
lines += [""]
f"Repo: `{self.repo}` · PR: #{self.pr} · head `{self.head_sha[:12]}`", ""]
lines.append(f"Acceptance parsed ({len(self.acceptance)}):")
if self.acceptance:
lines += [f"- [ ] {a}" for a in self.acceptance]
@ -275,24 +230,11 @@ class Reviewer:
# -- main entry points -------------------------------------------------
def review_pr(self, repo: str, pr_number: int, linked_issue,
workspace=None) -> Verdict:
"""Review one PR against the isolated synthetic-merge workspace.
``workspace`` is the issue-#30 seam (``ReviewWorkspace``): its
``build(repo, pr_number, head_sha, base_sha)`` context manager
yields the merged temporary worktree. A green verdict ALWAYS
describes that exact merge result — never an arbitrary
``repo_dir``. With no workspace (or no resolvable base/head
SHAs) the gate fails closed to NEEDS_GUIDANCE; on a merge
conflict it returns a deterministic BOUNCE and the test suite is
not run.
"""
def review_pr(self, repo: str, pr_number: int, linked_issue, repo_dir) -> Verdict:
issue_body = linked_issue.get("body") if isinstance(linked_issue, dict) else str(linked_issue or "")
pr = self.repo_client.get_pr(repo, pr_number)
head_sha = pr.get("head_sha") or "unknown"
base_sha = pr.get("base_sha") or "unknown"
changed_files = pr.get("changed_files") or []
diff_text = self.repo_client.get_pr_diff(repo, pr_number) or ""
@ -301,62 +243,28 @@ class Reviewer:
scope_violations = [f for f in changed_files
if allowed and not _matches_allowed(f, allowed)]
if workspace is None:
return Verdict(
repo=repo, pr=pr_number, head_sha=head_sha, base_sha=base_sha,
acceptance=acceptance, tests=dict(exit="not_run"),
scope_violations=scope_violations, verdict=NEEDS_GUIDANCE,
summary=("no isolated review workspace configured — cannot "
"construct and test the synthetic merge; refusing to "
"review an arbitrary checkout (never silent-approve)"))
# The whole review happens inside the workspace: the working tree it
# yields IS the synthetic merge of the exact head onto the exact base.
with workspace.build(repo, pr_number, head_sha, base_sha) as ws:
if not ws.merge_ok:
return Verdict(
repo=repo, pr=pr_number, head_sha=head_sha, base_sha=base_sha,
acceptance=acceptance, tests=dict(
exit="not_run",
summary="test suite not run: the synthetic merge of the "
"reviewed head onto the reviewed base conflicts"),
scope_violations=scope_violations, verdict=BOUNCE,
summary=("synthetic merge conflict between the reviewed head "
"and the current base — see the CONFLICT evidence; "
"the test suite was not run against any fallback "
"checkout"),
workspace={**ws.evidence, "merge_ok": False})
tests = self.runner.run(ws.dir, "python -m pytest -q",
tests = self.runner.run(repo_dir, "python -m pytest -q",
timeout_s=self.config["test_timeout_s"])
build = None
if self.config.get("build_cmd"):
build = self.runner.run(repo_dir, self.config["build_cmd"],
timeout_s=self.config["test_timeout_s"])
build = None
if self.config.get("build_cmd"):
build = self.runner.run(ws.dir, self.config["build_cmd"],
timeout_s=self.config["test_timeout_s"])
build["command"] = self.config["build_cmd"]
build["command"] = self.config["build_cmd"]
# Explicit base/result range: the whitespace gate checks only the
# synthetic merge result vs the exact base it was built on.
# Argument list (shell=False), same Windows-safety rationale as
# review_workspace (issue #48): the SHA and "HEAD" are untrusted
# data and must reach git verbatim.
hygiene = self.runner.run(
ws.dir,
["git", "diff", "--check", ws.base_sha, "HEAD"],
timeout_s=self.config["test_timeout_s"])
hygiene = self.runner.run(repo_dir, "git diff --check",
timeout_s=self.config["test_timeout_s"])
findings = self._collect_findings(diff_text, tests, hygiene)
findings = self._collect_findings(diff_text, tests, hygiene)
verdict, summary = self._decide(
issue_body, acceptance, tests, build, scope_violations,
findings, hygiene)
verdict, summary = self._decide(
issue_body, acceptance, tests, build, scope_violations, findings, hygiene)
return Verdict(
repo=repo, pr=pr_number, head_sha=ws.head_sha, base_sha=ws.base_sha,
acceptance=acceptance, tests=tests, build=build,
scope_violations=scope_violations, findings=findings,
verdict=verdict, summary=summary,
workspace={**ws.evidence, "merge_ok": True})
return Verdict(
repo=repo, pr=pr_number, head_sha=head_sha,
acceptance=acceptance, tests=tests, build=build,
scope_violations=scope_violations, findings=findings,
verdict=verdict, summary=summary,
)
def post_verdict(self, repo: str, pr_number: int, verdict: Verdict) -> str:
"""Post the formatted [review-gate] comment. Returns the comment."""

18
gitea_saas/schema.sql
View File

@ -42,24 +42,6 @@ CREATE TABLE IF NOT EXISTS holds (
released_at TEXT
);
-- Per-client API keys (issue #41): the client-facing surface's credentials.
-- Only the SHA-256 hash of a key is stored -- never the plaintext. A row is
-- scoped to exactly one client, so a key can only ever resolve to its owner's
-- client_id (cross-tenant denial is enforced by the client surface using this
-- scope). revoked_at NULL = active; a revoked key fails resolution and is
-- never returned by any listing (prefix + created_at only, for display).
CREATE TABLE IF NOT EXISTS client_api_keys (
id INTEGER PRIMARY KEY AUTOINCREMENT,
client_id INTEGER NOT NULL REFERENCES clients(id),
key_hash TEXT NOT NULL UNIQUE,
key_prefix TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
revoked_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_client_api_keys_client ON client_api_keys (client_id);
CREATE INDEX IF NOT EXISTS idx_client_api_keys_hash ON client_api_keys (key_hash);
-- Balance / listing lookups.
CREATE INDEX IF NOT EXISTS idx_ledger_client_created ON ledger (client_id, created_at, id);
CREATE INDEX IF NOT EXISTS idx_ledger_client_kind ON ledger (client_id, kind);

17
gitea_saas/server.py
View File

@ -44,15 +44,14 @@ tests hermetic (Flask test client + temp DB) and free of any network/Stripe.
from __future__ import annotations
import hmac
import sys
from pathlib import Path
from typing import Any, Dict, Optional, Tuple
from flask import Flask, jsonify, request
from .auth import (API_KEY_HEADER, PUBLIC_PATHS, key_is_valid,
unauthorized_response)
from .config import load_config
from .config import API_KEY_HEADER, load_config
class ApiError(Exception):
@ -220,10 +219,7 @@ def create_app(core: Any, api_key: str) -> Flask:
"""Build the Flask app around a billing ``core`` instance.
``api_key`` is matched against the ``X-API-Key`` header on every route
except the liveness routes in ``auth.PUBLIC_PATHS`` (``/health``,
``/healthz``, ``/api/factory/health``). Missing or wrong keys get the
canonical 401 body from ``gitea_saas.auth`` (issue #24 consolidation —
this app delegates, it does not implement its own check).
except ``/health``. Missing or wrong keys get ``401``.
"""
app = Flask(__name__)
app.config["CORE"] = core
@ -231,12 +227,13 @@ def create_app(core: Any, api_key: str) -> Flask:
@app.before_request
def _require_api_key():
if request.path.rstrip("/") in PUBLIC_PATHS:
if request.path.rstrip("/") == "/health":
return None
sent = request.headers.get(API_KEY_HEADER, "")
expected = app.config["API_KEY"]
if not key_is_valid(sent, expected):
return unauthorized_response()
if not expected or not isinstance(sent, str) or not hmac.compare_digest(
sent, expected):
return _error(401, "unauthorized", "missing or invalid API key")
return None
@app.errorhandler(ApiError)

6
pyproject.toml
View File

@ -3,12 +3,6 @@ name = "gitea-saas-core"
version = "0.1.0"
description = "Shared billing core (clients/ledger/holds, idempotent, fail-closed) for gitea-saas"
requires-python = ">=3.10"
dependencies = [
"flask>=3.0",
]
[project.scripts]
gitea-saas = "gitea_saas.app:main"
[tool.pytest.ini_options]
testpaths = ["tests"]

269
tests/calibration/build_corpus.py
View File

@ -1,269 +0,0 @@
#!/usr/bin/env python3
"""Build the calibration corpus for gitea-saas-core#42.
Turns the raw Gitea snapshots in /tmp/corpus_raw (fetched 2026-09-01 by
corpus_fetch.py, read-only API calls) into the committed corpus under
tests/calibration/corpus/ plus tests/calibration/manifest.json.
The manifest is the curated measurement metadata: for every real PR the
KNOWN correct outcome (should_approve / should_bounce), the recorded gate
inputs (test exit etc. as observed in farm history), and the category.
The corpus JSON files carry the immutable recorded inputs the reviewer is
re-run against (issue body, changed files, diff).
Nothing in here touches gitea_saas/reviewer.py or review_policy.py.
"""
from __future__ import annotations
import json
import re
import sys
from pathlib import Path
RAW_DIR = Path("/tmp/corpus_raw")
OUT_DIR = Path(__file__).resolve().parent
# ---------------------------------------------------------------------------
# Curated manifest: id, repo, pr, known outcome, category, recorded gate inputs
# ---------------------------------------------------------------------------
# categories:
# merged-clean - merged into main, no bounce record
# merged-after-bounce - merged; had a bounce cycle earlier in its history
# bounced-open - open PR still carrying `bounced` (never merged)
# wrong-check-target - owner-documented false-approve (frontend change gated
# by a backend-only test run)
# sibling-merge-regression - owner-documented false-approve (test green in
# isolation, red in the tree after sibling merge)
COMMON_MERGED = {
"reviewable": True,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": ("merged into origin/main (gate green at merge); "
"repo main suite: 235 passed on 2026-09-01"),
}
MANIFEST = [
# -- merged, clean -------------------------------------------------------
dict(id="gitea-saas-core-10", repo="gitea-saas-core", pr=10,
known_outcome="should_approve", category="merged-clean",
note="Billing ledger core (#1); merged; allowed-file set respected.",
**COMMON_MERGED),
dict(id="gitea-saas-core-19", repo="gitea-saas-core", pr=19,
known_outcome="should_approve", category="merged-clean",
note="Billing core client API consistency (#16); merged.",
**COMMON_MERGED),
dict(id="gitea-saas-core-27", repo="gitea-saas-core", pr=27,
known_outcome="should_approve", category="merged-clean",
note="X-API-Key auth consolidation (#24); merged.",
**COMMON_MERGED),
dict(id="botmanager-191", repo="botmanager", pr=191,
known_outcome="should_approve", category="merged-clean",
note="fix(#188) no fabricated RSNs; merged.",
**COMMON_MERGED),
dict(id="botmanager-200", repo="botmanager", pr=200,
known_outcome="should_approve", category="merged-clean",
note="Closes #198 display_name backfill; merged.",
**COMMON_MERGED),
dict(id="botmanager-209", repo="botmanager", pr=209,
known_outcome="should_approve", category="merged-after-bounce",
note="Closes #208 deploy_live --check; two bounce-fix attempts "
"(review comments record pytest 1184 passed) before merge.",
**COMMON_MERGED),
dict(id="botmanager-231", repo="botmanager", pr=231,
known_outcome="should_approve", category="merged-clean",
note="feat(#226) responsive fleet row; merged.",
**COMMON_MERGED),
dict(id="speakeasy-41", repo="speakeasy", pr=41,
known_outcome="should_approve", category="merged-clean",
note="fix(#38) route through Windows curl.exe from WSL; merged.",
**COMMON_MERGED),
dict(id="speakeasy-43", repo="speakeasy", pr=43,
known_outcome="should_approve", category="merged-clean",
note="fix(#42) memberExpireDays into profile; merged.",
**COMMON_MERGED),
dict(id="nexus-aio-61", repo="nexus-aio", pr=61,
known_outcome="should_approve", category="merged-clean",
note="refactor(#60) activity registry; merged.",
**COMMON_MERGED),
dict(id="nexus-aio-98", repo="nexus-aio", pr=98,
known_outcome="should_approve", category="merged-clean",
note="fix(#94) normalise game text before comparison; merged.",
**COMMON_MERGED),
dict(id="rs-pilot-42", repo="rs-pilot", pr=42,
known_outcome="should_approve", category="merged-clean",
note="feat(#41) offline dry-run mode; merged.",
**COMMON_MERGED),
dict(id="rs-pilot-49", repo="rs-pilot", pr=49,
known_outcome="should_approve", category="merged-clean",
note="fix(#47) rank prompt candidates by track record; merged.",
**COMMON_MERGED),
dict(id="after-hours-vn-19", repo="after-hours-vn", pr=19,
known_outcome="should_approve", category="merged-clean",
note="fix(#17) SDK-only import guard; merged.",
**COMMON_MERGED),
dict(id="gitea-worker-ops-43", repo="gitea-worker-ops", pr=43,
known_outcome="should_approve", category="merged-clean",
note="fix(#42) dispatcher capacity double-subtraction; merged.",
**COMMON_MERGED),
dict(id="dice-and-deeds-27", repo="dice-and-deeds", pr=27,
known_outcome="should_approve", category="merged-clean",
note="test(#17) one-command fresh-checkout suite; merged.",
**COMMON_MERGED),
dict(id="microkit-10", repo="microkit", pr=10,
known_outcome="should_approve", category="merged-clean",
note="feat(#9) F2P alcher mode; merged.",
**COMMON_MERGED),
dict(id="gopher64-netplay-21", repo="gopher64-netplay", pr=21,
known_outcome="should_approve", category="merged-clean",
note="Add .gitattributes LF fix (#19); merged.",
**COMMON_MERGED),
# -- merged after a bounce cycle ----------------------------------------
dict(id="gitea-saas-core-22", repo="gitea-saas-core", pr=22,
known_outcome="should_approve", category="merged-after-bounce",
note="SaaS dashboard (#9); merged AFTER a bounce cycle (bounced "
"label still attached). Final landed state is the fixed head.",
**COMMON_MERGED),
dict(id="bot-contracts-8", repo="bot-contracts", pr=8,
known_outcome="should_approve", category="merged-after-bounce",
note="fix(#7) repo-relative evidence paths; merged after a bounce "
"cycle (bounced label still attached).",
**COMMON_MERGED),
# -- open, bounced (never merged) ---------------------------------------
dict(id="nexus-aio-40", repo="nexus-aio", pr=40,
known_outcome="should_bounce", category="bounced-open",
reviewable=True,
test_exit=None, hygiene_exit=0,
test_exit_source="not recorded in farm history (bounce decided by "
"human/LLM review findings, not a test failure)",
note="Closes #10; open PR with bounced+needs-guidance labels."),
dict(id="nexus-aio-44", repo="nexus-aio", pr=44,
known_outcome="should_bounce", category="bounced-open",
reviewable=True,
test_exit=None, hygiene_exit=0,
test_exit_source="not recorded in farm history (bounce decided by "
"human/LLM review findings, not a test failure)",
note="[skill] Attack (combat shape) Closes #22; bounced+needs-guidance."),
dict(id="gopher64-netplay-9", repo="gopher64-netplay", pr=9,
known_outcome="should_bounce", category="bounced-open",
reviewable=True,
test_exit=None, hygiene_exit=0,
test_exit_source="not recorded in farm history (bounce decided by "
"human/LLM review findings, not a test failure)",
note="Fix GGRS frame counter reset (#5); bounced+manual-only. Diff "
"392KB truncated to 150KB in corpus."),
dict(id="gopher64-netplay-18", repo="gopher64-netplay", pr=18,
known_outcome="should_bounce", category="bounced-open",
reviewable=True,
test_exit=None, hygiene_exit=0,
test_exit_source="not recorded in farm history (bounce decided by "
"human/LLM review findings, not a test failure)",
note="netplay session resume (#3); bounced+manual-only. Diff 454KB "
"truncated to 150KB in corpus."),
# -- owner-documented false-approve case 1: wrong check target ----------
dict(id="botmanager-220", repo="botmanager", pr=220,
reviewable=True,
known_outcome="should_bounce", category="wrong-check-target",
test_exit=0, hygiene_exit=0,
test_exit_source=("owner-documented: the configured gate reported "
"GREEN on this frontend-only change (gate ran pytest "
"against a change pytest cannot test). PR review "
"comments additionally record npm ci + npm test "
"100/100 + npm run build green on a LINUX review "
"worktree. The crash (FleetSummary components "
"undefined) manifests on case-insensitive Windows "
"filesystems only: botmanager#223."),
note=("Merged via 'feat(#215): action-first Operator Fleet workspace "
"redesign'. It shipped the undefined-component break that "
"botmanager#223 measured on origin/master AND the review-gate "
"fix landed in botmanager-224/gitea-worker-ops !44. The correct "
"outcome was BOUNCE. Recorded inputs are the actual reviewed "
"head (7f350a1)."),
acceptance_override_farm=(
"Acceptance bullets parsed from the issue's '## Acceptance "
"criteria' section, which the shipped parse_acceptance() does not "
"recognise (it matches only '## Acceptance' or '- [ ]' "
"checklists). Farm-configured-gate sensitivity only; not the "
"default measurement.")),
# -- owner-documented false-approve case 2: sibling-merge regression -----
dict(id="botmanager-bang22", repo="botmanager", pr=None,
reviewable=False,
known_outcome="should_bounce", category="sibling-merge-regression",
original_ref="botmanager !22 (original tracker numbering, pre-Gitea "
"renumbering; first-era Gitea PR #22 is not retrievable "
"from the current repo)",
test_exit_source=("owner-documented: the PR's own scope-guard test "
"passed IN ISOLATION; it could never pass once a "
"sibling PR merged - green in isolation, red in the "
"tree."),
note=("No diff/issue snapshot survives on this Gitea instance (PRs "
"are numbered from 119; first-era PRs 1-118 existed - see "
"pr_bm_*.json artifacts - but are not retrievable). Recorded "
"here as the owner documented it (gitea-saas-core#42 body). "
"Closest surviving Gitea record of the same class: botmanager "
"#201, bounced at head 4b4aebc because its branch was cut "
"before sibling #200 merged and the rebase dropped #200's code "
"- its own 22 new tests passed in isolation.")),
]
def main() -> None:
entries = []
for m in MANIFEST:
entry = dict(**m)
if m.get("reviewable", False):
raw = json.loads((RAW_DIR / f"{m['repo']}-{m['pr']}.json").read_text())
corpus = {
"id": m["id"],
"repo": m["repo"],
"pr": m["pr"],
"title": raw["title"],
"head_sha": raw["head_sha"],
"base_sha": raw["base_sha"],
"labels": raw["labels"],
"merged": raw["merged"],
"issue_number": raw["issue"],
"issue_body": raw["issue_body"],
"changed_files": raw["changed_files"],
"diff": raw["diff"],
"diff_truncated": raw["diff_truncated"],
"diff_bytes": raw["diff_bytes"],
}
if m["id"] == "botmanager-220" and "acceptance_override_farm" in m:
corpus["acceptance_override_farm"] = _farm_acceptance(
raw["issue_body"] or "")
(OUT_DIR / "corpus" / f"{m['id']}.json").write_text(
json.dumps(corpus, indent=1))
entries.append(entry)
(OUT_DIR / "manifest.json").write_text(json.dumps(
{"generated_utc": "2026-09-01T11:20:00Z",
"source": ("pbwraith Gitea API snapshots fetched 2026-09-01 "
"(read-only); see build_corpus.py"),
"entries": entries}, indent=1))
print(f"wrote {len(entries)} manifest entries, "
f"{sum(1 for e in entries if e.get('reviewable'))} reviewable")
def _farm_acceptance(issue_body: str) -> list[str]:
"""Extract acceptance bullets the farm's own review used.
Matches '## Acceptance criteria' / '## Acceptance' sections with '- '
bullets (the farm's format), verbatim from the issue body. Used ONLY for
the clearly-labelled farm-configured sensitivity pass.
"""
m = re.search(r"(?im)^#{1,3}\s*Acceptance[^\n]*$([\s\S]*?)(?=^#{1,3}\s|\Z)",
issue_body)
if not m:
return []
items = []
for line in m.group(1).splitlines():
s = line.strip()
if s.startswith(("-", "*")) and s not in ("-", "*"):
items.append(s.lstrip("-*").strip())
elif re.match(r"^\d+[.)]\s+", s):
items.append(re.sub(r"^\d+[.)]\s+", "", s))
return items
if __name__ == "__main__":
sys.exit(main())

99
tests/calibration/calibration_report.md
View File

@ -1,99 +0,0 @@
# Review-gate calibration report (gitea-saas-core#42)
- Generated: 2026-09-01T11:20:00Z (re-derive with `python tests/calibration/run_calibration.py`)
- Corpus: 25 reviewable real farm PRs + 1 owner-documented case(s), drawn from 11 pbwraith repos
- Gate under test: `gitea_saas.reviewer.Reviewer`, DEFAULT (deterministic) config - unchanged code, run against recorded inputs (issue body, changed files, diff, observed test/hygiene exit)
## Headline rates
| rate | value | denominator |
|---|---|---|
| false-approve rate | **0.0%** | 14 gate approvals |
| false-approve prevalence | 0.0% | 25 reviewable PRs |
| false-bounce rate | n/a | 0 gate bounces |
| unverifiable (NEEDS_GUIDANCE) | 44.0% | 25 reviewable PRs |
## Verdict matrix
- correct approves: 14
- **false approves (approved, should have bounced): 0**
- correct bounces: 0
- false bounces (bounced, should have approved): 0
- NEEDS_GUIDANCE (gate refused to verify): 11
## Per-entry results
| id | repo#pr | known | verdict | bucket | category |
|---|---|---|---|---|---|
| after-hours-vn-19 | after-hours-vn#19 | approve | APPROVE | correct-approve | merged-clean |
| botmanager-191 | botmanager#191 | approve | APPROVE | correct-approve | merged-clean |
| botmanager-200 | botmanager#200 | approve | APPROVE | correct-approve | merged-clean |
| dice-and-deeds-27 | dice-and-deeds#27 | approve | APPROVE | correct-approve | merged-clean |
| gitea-saas-core-10 | gitea-saas-core#10 | approve | APPROVE | correct-approve | merged-clean |
| gitea-saas-core-19 | gitea-saas-core#19 | approve | APPROVE | correct-approve | merged-clean |
| gitea-saas-core-22 | gitea-saas-core#22 | approve | APPROVE | correct-approve | merged-after-bounce |
| microkit-10 | microkit#10 | approve | APPROVE | correct-approve | merged-clean |
| nexus-aio-61 | nexus-aio#61 | approve | APPROVE | correct-approve | merged-clean |
| nexus-aio-98 | nexus-aio#98 | approve | APPROVE | correct-approve | merged-clean |
| rs-pilot-42 | rs-pilot#42 | approve | APPROVE | correct-approve | merged-clean |
| rs-pilot-49 | rs-pilot#49 | approve | APPROVE | correct-approve | merged-clean |
| speakeasy-41 | speakeasy#41 | approve | APPROVE | correct-approve | merged-clean |
| speakeasy-43 | speakeasy#43 | approve | APPROVE | correct-approve | merged-clean |
| bot-contracts-8 | bot-contracts#8 | approve | NEEDS_GUIDANCE | guidance | merged-after-bounce |
| botmanager-209 | botmanager#209 | approve | NEEDS_GUIDANCE | guidance | merged-after-bounce |
| botmanager-220 | botmanager#220 | bounce | NEEDS_GUIDANCE | guidance | wrong-check-target |
| botmanager-231 | botmanager#231 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
| gitea-saas-core-27 | gitea-saas-core#27 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
| gitea-worker-ops-43 | gitea-worker-ops#43 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
| gopher64-netplay-18 | gopher64-netplay#18 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
| gopher64-netplay-21 | gopher64-netplay#21 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
| gopher64-netplay-9 | gopher64-netplay#9 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
| nexus-aio-40 | nexus-aio#40 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
| nexus-aio-44 | nexus-aio#44 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
## Per-category breakdown
| category | total | false-approve | false-bounce | guidance |
|---|---|---|---|---|
| bounced-open | 4 | 0 | 0 | 4 |
| merged-after-bounce | 3 | 0 | 0 | 2 |
| merged-clean | 17 | 0 | 0 | 4 |
| wrong-check-target | 1 | 0 | 0 | 1 |
## Owner-documented cases (gitea-saas-core#42)
The two farm regressions the issue cites are in the corpus with their recorded outcomes. Both had a GREEN gate in farm history; both shipped or would have shipped a broken tree:
### 1. botmanager !220 (`botmanager#220`) - wrong check target
- Known outcome: **should_bounce** (it shipped the undefined-component break measured in botmanager#223; the review-gate fix landed in botmanager#224 / gitea-worker-ops !44).
- Recorded gate inputs: test exit 0 (pytest on a frontend-only change; review comments record npm ci / npm test 100/100 / build green on a LINUX worktree - the break manifests on case-insensitive Windows).
- Product gate verdict on the recorded inputs: NEEDS_GUIDANCE (accepted by accident: the issue's acceptance section is titled '## Acceptance criteria' with a numbered list, which the shipped parse_acceptance() does not recognise - '## Acceptance' / '- [ ]' only).
- Farm-configured-parser sensitivity (the issue's own real acceptance items, 8 parsed): verdict **APPROVE** -> **FALSE-APPROVE**. I.e. with a one-line parser accommodation the gate auto-merges a change that broke master.
### 2. botmanager !22 (original tracker ref) - sibling-merge regression
- Known outcome: **should_bounce** (owner-documented: the PR's own scope-guard test passed in isolation and could never pass once a sibling PR merged - green in isolation, red in the tree).
- Not re-runnable: recorded as owner-documented (original head data not retrievable) (first-era PR #22 predates the current repo's renumbering; closest surviving record of the same class is botmanager#201, bounced at head 4b4aebc after a rebase silently dropped sibling #200's code while its own 22 new tests passed in isolation).
## Coverage: why 11 of 25 reviewable PRs get NEEDS_GUIDANCE
The shipped `parse_acceptance()` reads `- [ ]` checklists anywhere or bullets under a section titled exactly `## Acceptance`. Farm issues mix formats: `## Acceptance criteria` (numbered), `## Scope` bullets, or no acceptance section at all. 11 corpus issues use one of those formats, so the gate refuses to verify them (fail-closed, never silent-approve). That is SAFE but means the gate can evaluate only ~56% of this farm's real issues.
## Limitations
- Recorded test exits are reconstructed from farm history (merged implies gate-green at merge; bounced entries' exits were not recorded - their bounce was decided by human/LLM review findings the deterministic gate's inputs cannot represent).
- Diffs are current-head snapshots (two >150KB gopher64 diffs truncated); merged PRs' diffs approximate the reviewed content.
- The deterministic gate is measured; the farm's LLM-augmented review is not re-run (nondeterministic, and the issue's instruction is: measure the existing reviewer).
## Recommendation
**NO - gold-tier auto-merge is not safe to sell at this rate (0.0% false-approve rate (0/14)).**
Reasons, each backed by the corpus:
1. **Zero is an artifact of refusal, not of quality.** The gate approved only the 14 checklist-formatted issues and refused the other 11 (44%) real farm PRs it could not verify. A gate that cannot evaluate 44% of real work cannot back an auto-merge guarantee.
2. **The approve path has no check that would have stopped either documented regression.** botmanager#223's break was green under the configured gate (wrong suite: pytest vs frontend change; green on case-sensitive Linux vs case-insensitive Windows target) and botmanager !22 was green in isolation (no sibling-merge/integration verification). Both mechanisms survive in the gate's approve path today.
3. **On the operator's own terms the rate is 2/16 = 12.5%.** Add the two owner-documented cases to the 14 the gate can evaluate: both had green gate inputs and both should have bounced (sensitivity pass on botmanager#220 -> FALSE-APPROVE; botmanager !22 documented).
4. **Accidental safety is not a product.** botmanager#220 reached NEEDS_GUIDANCE only because its issue says 'Acceptance criteria' instead of 'Acceptance' - bad luck, not fail-closed design.
Before selling gold auto-merge, close, in order: (a) the acceptance format coverage gap, (b) test-command/target-suite selection per changed-file set (the gitea-worker-ops !44 direction), (c) sibling-merge/integration verification, and (d) re-run this calibration on the fixed gate and on a larger corpus.

20
tests/calibration/corpus/after-hours-vn-19.json
View File

File diff suppressed because one or more lines are too long

23
tests/calibration/corpus/bot-contracts-8.json
View File

File diff suppressed because one or more lines are too long

20
tests/calibration/corpus/botmanager-191.json
View File

File diff suppressed because one or more lines are too long

21
tests/calibration/corpus/botmanager-200.json
View File

File diff suppressed because one or more lines are too long

19
tests/calibration/corpus/botmanager-209.json
View File

File diff suppressed because one or more lines are too long

51
tests/calibration/corpus/botmanager-220.json
View File

File diff suppressed because one or more lines are too long

27
tests/calibration/corpus/botmanager-231.json
View File

File diff suppressed because one or more lines are too long

25
tests/calibration/corpus/dice-and-deeds-27.json
View File

File diff suppressed because one or more lines are too long

23
tests/calibration/corpus/gitea-saas-core-10.json
View File

File diff suppressed because one or more lines are too long

19
tests/calibration/corpus/gitea-saas-core-19.json
View File

File diff suppressed because one or more lines are too long

21
tests/calibration/corpus/gitea-saas-core-22.json
View File

File diff suppressed because one or more lines are too long

28
tests/calibration/corpus/gitea-saas-core-27.json
View File

File diff suppressed because one or more lines are too long

19
tests/calibration/corpus/gitea-worker-ops-43.json
View File

File diff suppressed because one or more lines are too long

50
tests/calibration/corpus/gopher64-netplay-18.json
View File

File diff suppressed because one or more lines are too long

18
tests/calibration/corpus/gopher64-netplay-21.json
View File

@ -1,18 +0,0 @@
{
"id": "gopher64-netplay-21",
"repo": "gopher64-netplay",
"pr": 21,
"title": "Add .gitattributes: *.sh text eol=lf for Windows clones (#19)",
"head_sha": "cfa469428cd691f8e9814956b340989ccb4743ef",
"base_sha": "c857feada3a9ee838aed425ac5602a94cc61523a",
"labels": [],
"merged": true,
"issue_number": 19,
"issue_body": "## `scripts/gate.sh` breaks in any Windows clone, because this repo has no `.gitattributes`\n\nSmall, cheap, and worth fixing now that `scripts/gate.sh` is the repo's keystone - eight open PRs are gated by it, so anything that makes it fail confusingly costs more here than it would elsewhere.\n\n## The measurement\n\n```\n$ git ls-tree origin/main --name-only | grep gitattributes\n(nothing)\n```\n\nNo `.gitattributes` on `main`, and none on `issue-16` either. The repo ships three shell scripts:\n\n```\ndata/icon/generate.sh\ndata/shader/output.sh\nscripts/gate.sh\n```\n\nThe committed blobs are clean - `git cat-file -p origin/issue-16:scripts/gate.sh | tr -cd '\\r' | wc -c` is **0**. The problem is not what is stored, it is what gets written on checkout.\n\n## Why it bites\n\nGit for Windows defaults to `core.autocrlf=true`. With no `.gitattributes` to override it, cloning this repo on Windows writes CRLF into every `.sh` working copy, and the script then dies at exec:\n\n```\n$ ./scripts/gate.sh\n/usr/bin/env: 'bash\\r': No such file or directory\nEXIT=127\n```\n\nThis already happened during review of `!17`: the first gate run failed exactly this way, and it took a round of investigation to establish it was a clone artifact rather than a defect in the PR. `env: 'bash\\r'` does not name line endings, and the natural reading is \"the script is broken\" - which is the wrong conclusion about a script whose blob is byte-clean.\n\nThe workaround is `git config core.autocrlf false` plus a fresh checkout, but that is per-clone tribal knowledge. The repo should assert its own requirement.\n\n## What to do\n\nAdd `.gitattributes` at the repo root:\n\n```\n*.sh text eol=lf\n```\n\nConsider `* text=auto` alongside it if you want the general rule, but the `.sh` line is the one that fixes the observed failure.\n\n## The part that is easy to get wrong\n\n**`.gitattributes` alone does not fix an already-checked-out working copy.** Git does not rewrite a file on branch switch when its blob is unchanged, so a clone that already has CRLF in `scripts/gate.sh` will still have CRLF after merging this change - on a branch whose blobs are perfectly clean. Existing clones need a forced re-checkout:\n\n```\nrm -f .git/index && git reset --hard HEAD\n```\n\nIf any committed blob *did* contain CR (none do here, so this should be a no-op), fixing it needs `git add --renormalize .` as a separate commit - the attributes file does not retroactively clean stored objects.\n\n## Acceptance criteria\n\n1. `.gitattributes` exists on `main` and contains `*.sh text eol=lf`.\n2. On a **fresh Windows clone made with `core.autocrlf=true`** (the default - set it explicitly for the test rather than relying on the box's config), `tr -cd '\\r' < scripts/gate.sh | wc -c` returns **0**. State the command and the number.\n3. `head -c 24 scripts/gate.sh | od -c` shows a clean `#!/usr/bin/env bash` shebang with no `\\r`.\n4. All three `.sh` files are covered, not just `scripts/gate.sh`.\n\n## Measurement warning\n\nDo **not** verify this with `grep -c $'\\r'`. It is broken in this environment's Git Bash and returns the *line count*, not the CR count - it has produced false findings on three separate occasions here, and the tell is that the number always equals `wc -l`. Use `tr -cd '\\r' < file | wc -c`, and treat actually running `./scripts/gate.sh --help` (or any invocation that reaches the interpreter) as the deciding test.\n\n## Out of scope\n\n- Any change to `scripts/gate.sh`'s contents or to the build configuration. This is line endings only.\n- The disk-space blocker currently preventing the gate from completing (gitea-worker-ops#24). This issue is verifiable by inspection and a fresh clone; it does not need a working build.\n",
"changed_files": [
".gitattributes"
],
"diff": "diff --git a/.gitattributes b/.gitattributes\nnew file mode 100644\nindex 0000000..e4ba11e\n--- /dev/null\n+++ b/.gitattributes\n@@ -0,0 +1,7 @@\n+# Force LF line endings for shell scripts.\n+# Without this, Windows clones (git's core.autocrlf=true default) check out\n+# CRLF and the shebang dies with: /usr/bin/env: 'bash\\r': No such file or directory (exit 127)\n+# Keep every *.sh blob byte-clean; existing clones need a forced re-checkout\n+# (rm -f .git/index && git reset --hard HEAD) because git does not rewrite a\n+# file whose blob is unchanged.\n+*.sh text eol=lf\n",
"diff_truncated": false,
"diff_bytes": 566
}

50
tests/calibration/corpus/gopher64-netplay-9.json
View File

File diff suppressed because one or more lines are too long

22
tests/calibration/corpus/microkit-10.json
View File

File diff suppressed because one or more lines are too long

26
tests/calibration/corpus/nexus-aio-40.json
View File

File diff suppressed because one or more lines are too long

26
tests/calibration/corpus/nexus-aio-44.json
View File

File diff suppressed because one or more lines are too long

39
tests/calibration/corpus/nexus-aio-61.json
View File

File diff suppressed because one or more lines are too long

22
tests/calibration/corpus/nexus-aio-98.json
View File

File diff suppressed because one or more lines are too long

22
tests/calibration/corpus/rs-pilot-42.json
View File

File diff suppressed because one or more lines are too long

22
tests/calibration/corpus/rs-pilot-49.json
View File

File diff suppressed because one or more lines are too long

19
tests/calibration/corpus/speakeasy-41.json
View File

File diff suppressed because one or more lines are too long

21
tests/calibration/corpus/speakeasy-43.json
View File

File diff suppressed because one or more lines are too long

318
tests/calibration/manifest.json
View File

@ -1,318 +0,0 @@
{
"generated_utc": "2026-09-01T11:20:00Z",
"source": "pbwraith Gitea API snapshots fetched 2026-09-01 (read-only); see build_corpus.py",
"entries": [
{
"id": "gitea-saas-core-10",
"repo": "gitea-saas-core",
"pr": 10,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "Billing ledger core (#1); merged; allowed-file set respected.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "gitea-saas-core-19",
"repo": "gitea-saas-core",
"pr": 19,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "Billing core client API consistency (#16); merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "gitea-saas-core-27",
"repo": "gitea-saas-core",
"pr": 27,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "X-API-Key auth consolidation (#24); merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "botmanager-191",
"repo": "botmanager",
"pr": 191,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#188) no fabricated RSNs; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "botmanager-200",
"repo": "botmanager",
"pr": 200,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "Closes #198 display_name backfill; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "botmanager-209",
"repo": "botmanager",
"pr": 209,
"known_outcome": "should_approve",
"category": "merged-after-bounce",
"note": "Closes #208 deploy_live --check; two bounce-fix attempts (review comments record pytest 1184 passed) before merge.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "botmanager-231",
"repo": "botmanager",
"pr": 231,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "feat(#226) responsive fleet row; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "speakeasy-41",
"repo": "speakeasy",
"pr": 41,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#38) route through Windows curl.exe from WSL; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "speakeasy-43",
"repo": "speakeasy",
"pr": 43,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#42) memberExpireDays into profile; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "nexus-aio-61",
"repo": "nexus-aio",
"pr": 61,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "refactor(#60) activity registry; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "nexus-aio-98",
"repo": "nexus-aio",
"pr": 98,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#94) normalise game text before comparison; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "rs-pilot-42",
"repo": "rs-pilot",
"pr": 42,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "feat(#41) offline dry-run mode; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "rs-pilot-49",
"repo": "rs-pilot",
"pr": 49,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#47) rank prompt candidates by track record; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "after-hours-vn-19",
"repo": "after-hours-vn",
"pr": 19,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#17) SDK-only import guard; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "gitea-worker-ops-43",
"repo": "gitea-worker-ops",
"pr": 43,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "fix(#42) dispatcher capacity double-subtraction; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "dice-and-deeds-27",
"repo": "dice-and-deeds",
"pr": 27,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "test(#17) one-command fresh-checkout suite; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "microkit-10",
"repo": "microkit",
"pr": 10,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "feat(#9) F2P alcher mode; merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "gopher64-netplay-21",
"repo": "gopher64-netplay",
"pr": 21,
"known_outcome": "should_approve",
"category": "merged-clean",
"note": "Add .gitattributes LF fix (#19); merged.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "gitea-saas-core-22",
"repo": "gitea-saas-core",
"pr": 22,
"known_outcome": "should_approve",
"category": "merged-after-bounce",
"note": "SaaS dashboard (#9); merged AFTER a bounce cycle (bounced label still attached). Final landed state is the fixed head.",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "bot-contracts-8",
"repo": "bot-contracts",
"pr": 8,
"known_outcome": "should_approve",
"category": "merged-after-bounce",
"note": "fix(#7) repo-relative evidence paths; merged after a bounce cycle (bounced label still attached).",
"reviewable": true,
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
},
{
"id": "nexus-aio-40",
"repo": "nexus-aio",
"pr": 40,
"known_outcome": "should_bounce",
"category": "bounced-open",
"reviewable": true,
"test_exit": null,
"hygiene_exit": 0,
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
"note": "Closes #10; open PR with bounced+needs-guidance labels."
},
{
"id": "nexus-aio-44",
"repo": "nexus-aio",
"pr": 44,
"known_outcome": "should_bounce",
"category": "bounced-open",
"reviewable": true,
"test_exit": null,
"hygiene_exit": 0,
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
"note": "[skill] Attack (combat shape) Closes #22; bounced+needs-guidance."
},
{
"id": "gopher64-netplay-9",
"repo": "gopher64-netplay",
"pr": 9,
"known_outcome": "should_bounce",
"category": "bounced-open",
"reviewable": true,
"test_exit": null,
"hygiene_exit": 0,
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
"note": "Fix GGRS frame counter reset (#5); bounced+manual-only. Diff 392KB truncated to 150KB in corpus."
},
{
"id": "gopher64-netplay-18",
"repo": "gopher64-netplay",
"pr": 18,
"known_outcome": "should_bounce",
"category": "bounced-open",
"reviewable": true,
"test_exit": null,
"hygiene_exit": 0,
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
"note": "netplay session resume (#3); bounced+manual-only. Diff 454KB truncated to 150KB in corpus."
},
{
"id": "botmanager-220",
"repo": "botmanager",
"pr": 220,
"reviewable": true,
"known_outcome": "should_bounce",
"category": "wrong-check-target",
"test_exit": 0,
"hygiene_exit": 0,
"test_exit_source": "owner-documented: the configured gate reported GREEN on this frontend-only change (gate ran pytest against a change pytest cannot test). PR review comments additionally record npm ci + npm test 100/100 + npm run build green on a LINUX review worktree. The crash (FleetSummary components undefined) manifests on case-insensitive Windows filesystems only: botmanager#223.",
"note": "Merged via 'feat(#215): action-first Operator Fleet workspace redesign'. It shipped the undefined-component break that botmanager#223 measured on origin/master AND the review-gate fix landed in botmanager-224/gitea-worker-ops !44. The correct outcome was BOUNCE. Recorded inputs are the actual reviewed head (7f350a1).",
"acceptance_override_farm": "Acceptance bullets parsed from the issue's '## Acceptance criteria' section, which the shipped parse_acceptance() does not recognise (it matches only '## Acceptance' or '- [ ]' checklists). Farm-configured-gate sensitivity only; not the default measurement."
},
{
"id": "botmanager-bang22",
"repo": "botmanager",
"pr": null,
"reviewable": false,
"known_outcome": "should_bounce",
"category": "sibling-merge-regression",
"original_ref": "botmanager !22 (original tracker numbering, pre-Gitea renumbering; first-era Gitea PR #22 is not retrievable from the current repo)",
"test_exit_source": "owner-documented: the PR's own scope-guard test passed IN ISOLATION; it could never pass once a sibling PR merged - green in isolation, red in the tree.",
"note": "No diff/issue snapshot survives on this Gitea instance (PRs are numbered from 119; first-era PRs 1-118 existed - see pr_bm_*.json artifacts - but are not retrievable). Recorded here as the owner documented it (gitea-saas-core#42 body). Closest surviving Gitea record of the same class: botmanager #201, bounced at head 4b4aebc because its branch was cut before sibling #200 merged and the rebase dropped #200's code - its own 22 new tests passed in isolation."
}
]
}

387
tests/calibration/run_calibration.py
View File

@ -1,387 +0,0 @@
#!/usr/bin/env python3
"""Run the review-gate calibration for gitea-saas-core#42.
One-line command (from the repo root):
python tests/calibration/run_calibration.py
Re-runs the EXISTING review gate (gitea_saas.reviewer.Reviewer, read-only —
no changes to reviewer.py / review_policy.py) against the committed corpus
(tests/calibration/corpus/*.json, manifest.json), compares each verdict to
the KNOWN correct outcome, and writes tests/calibration/calibration_report.md
with the false-approve / false-bounce / guidance rates.
Recorded inputs per corpus entry are what the gate actually saw in farm
history: issue body, changed files, diff, and the observed test/hygiene exit.
The deterministic gate is reproduced exactly (default config: secret scan on,
no build command, no LLM pass).
The two owner-documented false-approve cases (gitea-saas-core#42 body:
botmanager#223 crash shipper = PR !220, and botmanager !22) are reported in a
separate, clearly-labelled section: botmanager#220 is additionally evaluated
under a "farm-configured parser" sensitivity pass (its issue's real
'## Acceptance criteria' numbered list, which the shipped parse_acceptance()
does not recognise).
"""
from __future__ import annotations
import json
import sys
from pathlib import Path
from types import SimpleNamespace
HERE = Path(__file__).resolve().parent
ROOT = HERE.parent.parent
sys.path.insert(0, str(ROOT))
from gitea_saas.reviewer import ( # noqa: E402
APPROVE,
BOUNCE,
NEEDS_GUIDANCE,
Reviewer,
Verdict,
)
REVIEWABLE_EXCLUDE_NOTE = "recorded as owner-documented (original head data not retrievable)"
# ---------------------------------------------------------------------------
# Seams (the same duck-typed pattern the repo's own hermetic tests use:
# tests/test_reviewer.py)
# ---------------------------------------------------------------------------
class FakeRepoClient:
"""Serves one corpus entry's recorded PR data."""
def __init__(self, entry: dict):
self.entry = entry
def get_pr(self, repo, number):
return {
"head_sha": self.entry["head_sha"],
"base_sha": self.entry.get("base_sha") or ("b" * 40),
"changed_files": list(self.entry["changed_files"]),
}
def get_pr_diff(self, repo, number):
return self.entry["diff"]
class FakeWorkspace:
"""Merge-OK workspace seam: the synthetic merge is assumed to apply."""
def build(self, repo, pr_number, head_sha, base_sha):
return _FakeMergeContext()
class _FakeMergeContext:
def __enter__(self):
return SimpleNamespace(dir=Path("/fake/ws"), base_sha="b" * 40,
head_sha="a" * 40, merge_ok=True,
evidence={"merge_ok": True})
def __exit__(self, *exc):
return False
class FakeRunner:
"""Subprocess seam: returns the corpus-recorded exits per command."""
def __init__(self, entry: dict):
self.entry = entry
def run(self, repo_dir, command, timeout_s=None):
if "pytest" in command:
exit_code = self.entry["test_exit"]
return {"exit": exit_code,
"summary": self.entry.get("test_summary", "") or
("" if exit_code is None
else "[recorded farm-history test exit]")}
if "git diff --check" in command:
return {"exit": self.entry.get("hygiene_exit", 0),
"summary": ""}
return {"exit": 0, "summary": ""}
# ---------------------------------------------------------------------------
# Classification
# ---------------------------------------------------------------------------
def classify(verdict: str, known: str) -> str:
if known == "should_approve":
if verdict == APPROVE:
return "correct-approve"
if verdict == BOUNCE:
return "FALSE-BOUNCE"
return "guidance"
# known == should_bounce
if verdict == BOUNCE:
return "correct-bounce"
if verdict == APPROVE:
return "FALSE-APPROVE"
return "guidance"
def _fmt_rates(stats: dict) -> dict:
approve_total = stats["correct-approve"] + stats["FALSE-APPROVE"]
bounce_total = stats["correct-bounce"] + stats["FALSE-BOUNCE"]
reviewable = stats["reviewable"]
fa = stats["FALSE-APPROVE"]
return {
"false_approve_rate": (fa / approve_total if approve_total else None),
"false_approve_prevalence": (fa / reviewable if reviewable else None),
"false_bounce_rate": (stats["FALSE-BOUNCE"] / bounce_total
if bounce_total else None),
"guidance_share": (stats["guidance"] / reviewable if reviewable else None),
"approve_total": approve_total,
"bounce_total": bounce_total,
}
def _pct(x) -> str:
return f"{x * 100:.1f}%" if x is not None else "n/a"
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def load_all() -> tuple[list[dict], dict]:
manifest = json.loads((HERE / "manifest.json").read_text())
entries = []
for m in manifest["entries"]:
row = dict(m)
if m.get("reviewable"):
row["corpus"] = json.loads(
(HERE / "corpus" / f"{m['id']}.json").read_text())
entries.append(row)
return entries, manifest
def evaluate(entry: dict, farm_parser: bool = False) -> Verdict:
"""Run the real Reviewer (deterministic default config) on the entry."""
corpus = entry["corpus"]
issue_body = corpus["issue_body"] or ""
if farm_parser and corpus.get("acceptance_override_farm"):
# Sensitivity only: emulate a parser that recognises the farm's
# '## Acceptance criteria' numbered list by converting the issue's
# own real acceptance items into the checklist form the shipped
# parser reads. The items themselves are quoted verbatim from the
# issue body by build_corpus.py.
items = corpus["acceptance_override_farm"]
issue_body = (issue_body
+ "\n\n## Acceptance\n"
+ "".join(f"- [ ] {item}\n" for item in items))
reviewer = Reviewer(FakeRepoClient(corpus), FakeRunner(entry), config={})
return reviewer.review_pr(corpus["repo"], corpus["pr"],
{"body": issue_body}, workspace=FakeWorkspace())
def main(argv: list[str] | None = None) -> int:
entries, manifest = load_all()
reviewable = [e for e in entries if e.get("reviewable")]
documented = [e for e in entries if not e.get("reviewable")]
stats = {"reviewable": len(reviewable), "guidance": 0,
"correct-approve": 0, "FALSE-APPROVE": 0,
"correct-bounce": 0, "FALSE-BOUNCE": 0}
rows = []
for e in reviewable:
v = evaluate(e)
bucket = classify(v.verdict, e["known_outcome"])
stats[bucket] += 1
rows.append({
"id": e["id"], "repo": e["repo"],
"pr": e.get("pr") or "-",
"known": "approve" if e["known_outcome"] == "should_approve"
else "bounce",
"verdict": v.verdict, "bucket": bucket,
"category": e["category"],
"note": (e.get("note") or "")[:110],
})
# farm-configured sensitivity pass on the one reviewable owner case
sens = None
for e in reviewable:
if e["id"] == "botmanager-220" and e["corpus"].get("acceptance_override_farm"):
v = evaluate(e, farm_parser=True)
sens = {"id": e["id"], "verdict": v.verdict,
"bucket": classify(v.verdict, e["known_outcome"]),
"acceptance_parsed": len(v.acceptance)}
rates = _fmt_rates(stats)
report = render_report(manifest, rows, stats, rates, documented, sens)
(HERE / "calibration_report.md").write_text(report)
print(report)
return 0
def render_report(manifest, rows, stats, rates, documented, sens) -> str:
L = []
add = L.append
add("# Review-gate calibration report (gitea-saas-core#42)")
add("")
add(f"- Generated: {manifest.get('generated_utc', '2026-09-01')} "
f"(re-derive with `python tests/calibration/run_calibration.py`)")
add(f"- Corpus: {len(rows)} reviewable real farm PRs + "
f"{len(documented)} owner-documented case(s), drawn from "
f"{len({r['repo'] for r in rows})} pbwraith repos")
add(f"- Gate under test: `gitea_saas.reviewer.Reviewer`, DEFAULT "
f"(deterministic) config - unchanged code, run against recorded "
f"inputs (issue body, changed files, diff, observed test/hygiene exit)")
add("")
add("## Headline rates")
add("")
add(f"| rate | value | denominator |")
add(f"|---|---|---|")
add(f"| false-approve rate | **{_pct(rates['false_approve_rate'])}** "
f"| {rates['approve_total']} gate approvals |")
add(f"| false-approve prevalence | {_pct(rates['false_approve_prevalence'])} "
f"| {stats['reviewable']} reviewable PRs |")
add(f"| false-bounce rate | {_pct(rates['false_bounce_rate'])} "
f"| {rates['bounce_total']} gate bounces |")
add(f"| unverifiable (NEEDS_GUIDANCE) | {_pct(rates['guidance_share'])} "
f"| {stats['reviewable']} reviewable PRs |")
add("")
add("## Verdict matrix")
add("")
add(f"- correct approves: {stats['correct-approve']}")
add(f"- **false approves (approved, should have bounced): "
f"{stats['FALSE-APPROVE']}**")
add(f"- correct bounces: {stats['correct-bounce']}")
add(f"- false bounces (bounced, should have approved): "
f"{stats['FALSE-BOUNCE']}")
add(f"- NEEDS_GUIDANCE (gate refused to verify): {stats['guidance']}")
add("")
add("## Per-entry results")
add("")
add("| id | repo#pr | known | verdict | bucket | category |")
add("|---|---|---|---|---|---|")
for r in sorted(rows, key=lambda r: (r["bucket"] != "correct-approve",
r["repo"], str(r["pr"]))):
add(f"| {r['id']} | {r['repo']}#{r['pr']} | {r['known']} | "
f"{r['verdict']} | {r['bucket']} | {r['category']} |")
add("")
add("## Per-category breakdown")
add("")
cats = {}
for r in rows:
cats.setdefault(r["category"], {"total": 0, "fa": 0, "fb": 0,
"guidance": 0, "ok": 0})
cats[r["category"]]["total"] += 1
cats[r["category"]][
"fa" if r["bucket"] == "FALSE-APPROVE"
else "fb" if r["bucket"] == "FALSE-BOUNCE"
else "guidance" if r["bucket"] == "guidance"
else "ok"] += 1
add("| category | total | false-approve | false-bounce | guidance |")
add("|---|---|---|---|---|")
for cat, c in sorted(cats.items()):
add(f"| {cat} | {c['total']} | {c.get('fa', 0)} | {c.get('fb', 0)} "
f"| {c['guidance']} |")
add("")
add("## Owner-documented cases (gitea-saas-core#42)")
add("")
add("The two farm regressions the issue cites are in the corpus with their "
"recorded outcomes. Both had a GREEN gate in farm history; both shipped "
"or would have shipped a broken tree:")
add("")
add("### 1. botmanager !220 (`botmanager#220`) - wrong check target")
add("")
add("- Known outcome: **should_bounce** (it shipped the undefined-component "
"break measured in botmanager#223; the review-gate fix landed in "
"botmanager#224 / gitea-worker-ops !44).")
add("- Recorded gate inputs: test exit 0 (pytest on a frontend-only "
"change; review comments record npm ci / npm test 100/100 / build "
"green on a LINUX worktree - the break manifests on case-insensitive "
"Windows).")
add(f"- Product gate verdict on the recorded inputs: "
f"{next(r['verdict'] for r in rows if r['id'] == 'botmanager-220')} "
f"(accepted by accident: the issue's acceptance section is titled "
f"'## Acceptance criteria' with a numbered list, which the shipped "
f"parse_acceptance() does not recognise - '## Acceptance' / '- [ ]' "
f"only).")
if sens is not None:
add(f"- Farm-configured-parser sensitivity (the issue's own real "
f"acceptance items, {sens['acceptance_parsed']} parsed): verdict "
f"**{sens['verdict']}** -> **{sens['bucket']}**. I.e. with a "
f"one-line parser accommodation the gate auto-merges a change that "
f"broke master.")
add("")
add("### 2. botmanager !22 (original tracker ref) - sibling-merge regression")
add("")
add("- Known outcome: **should_bounce** (owner-documented: the PR's own "
"scope-guard test passed in isolation and could never pass once a "
"sibling PR merged - green in isolation, red in the tree).")
add(f"- Not re-runnable: {REVIEWABLE_EXCLUDE_NOTE} (first-era PR #22 "
f"predates the current repo's renumbering; closest surviving record "
f"of the same class is botmanager#201, bounced at head 4b4aebc after "
f"a rebase silently dropped sibling #200's code while its own 22 new "
f"tests passed in isolation).")
add("")
add("## Coverage: why 11 of 25 reviewable PRs get NEEDS_GUIDANCE")
add("")
add("The shipped `parse_acceptance()` reads `- [ ]` checklists anywhere or "
"bullets under a section titled exactly `## Acceptance`. Farm issues "
"mix formats: `## Acceptance criteria` (numbered), `## Scope` bullets, "
"or no acceptance section at all. 11 corpus issues use one of those "
"formats, so the gate refuses to verify them (fail-closed, "
"never silent-approve). That is SAFE but means the gate can evaluate "
"only ~56% of this farm's real issues.")
add("")
add("## Limitations")
add("")
add("- Recorded test exits are reconstructed from farm history (merged "
"implies gate-green at merge; bounced entries' exits were not recorded "
"- their bounce was decided by human/LLM review findings the "
"deterministic gate's inputs cannot represent).")
add("- Diffs are current-head snapshots (two >150KB gopher64 diffs "
"truncated); merged PRs' diffs approximate the reviewed content.")
add("- The deterministic gate is measured; the farm's LLM-augmented "
"review is not re-run (nondeterministic, and the issue's instruction "
"is: measure the existing reviewer).")
add("")
add("## Recommendation")
add("")
fa_denom = rates["approve_total"]
fa = stats["FALSE-APPROVE"]
if rates["false_approve_rate"] is None:
headline = "0 approvals produced - nothing to sell"
else:
headline = (f"{_pct(rates['false_approve_rate'])} false-approve rate "
f"({fa}/{fa_denom})")
add(f"**NO - gold-tier auto-merge is not safe to sell at this rate "
f"({headline}).**")
add("")
add("Reasons, each backed by the corpus:")
add("1. **Zero is an artifact of refusal, not of quality.** The gate "
"approved only the 14 checklist-formatted issues and refused the other "
"11 (44%) real farm PRs it could not verify. A gate that cannot "
"evaluate 44% of real work cannot back an auto-merge guarantee.")
add("2. **The approve path has no check that would have stopped either "
"documented regression.** botmanager#223's break was green under the "
"configured gate (wrong suite: pytest vs frontend change; green on "
"case-sensitive Linux vs case-insensitive Windows target) and "
"botmanager !22 was green in isolation (no sibling-merge/integration "
"verification). Both mechanisms survive in the gate's approve path "
"today.")
add("3. **On the operator's own terms the rate is 2/16 = 12.5%.** Add the "
"two owner-documented cases to the 14 the gate can evaluate: both had "
"green gate inputs and both should have bounced (sensitivity pass on "
f"botmanager#220 -> {sens['bucket'] if sens else 'FALSE-APPROVE'}; "
"botmanager !22 documented).")
add("4. **Accidental safety is not a product.** botmanager#220 reached "
"NEEDS_GUIDANCE only because its issue says 'Acceptance criteria' "
"instead of 'Acceptance' - bad luck, not fail-closed design.")
add("")
add("Before selling gold auto-merge, close, in order: (a) the acceptance "
"format coverage gap, (b) test-command/target-suite selection per "
"changed-file set (the gitea-worker-ops !44 direction), (c) "
"sibling-merge/integration verification, and (d) re-run this "
"calibration on the fixed gate and on a larger corpus.")
add("")
return "\n".join(L)
if __name__ == "__main__":
sys.exit(main())

80
tests/calibration/test_calibration.py
View File

@ -1,80 +0,0 @@
"""Guard tests for the review-gate calibration harness (issue #42).
The calibration's deliverable is a report, not a pass/fail — but the repo
gate must prove the harness holds together: corpus size, both
owner-documented cases present, rates computable, and the report
regenerating deterministically.
Run with: python -m pytest tests/calibration/test_calibration.py -q
"""
import json
import sys
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE.parent.parent))
import run_calibration # noqa: E402
def _load():
manifest = json.loads((HERE / "manifest.json").read_text())
return manifest["entries"]
def test_corpus_size_meets_acceptance():
entries = _load()
reviewable = [e for e in entries if e.get("reviewable")]
# Acceptance 1: false-approve rate over at least 15 real PRs.
assert len(reviewable) >= 15
for e in reviewable:
assert (HERE / "corpus" / f"{e['id']}.json").exists()
def test_both_botmanager_cases_recorded_with_outcomes():
entries = _load()
by_id = {e["id"]: e for e in entries}
# Acceptance 2: the two botmanager cases are in the corpus with outcomes.
assert by_id["botmanager-220"]["known_outcome"] == "should_bounce"
assert by_id["botmanager-bang22"]["known_outcome"] == "should_bounce"
assert by_id["botmanager-220"]["reviewable"]
assert not by_id["botmanager-bang22"]["reviewable"]
corpus220 = json.loads(
(HERE / "corpus" / "botmanager-220.json").read_text())
assert "acceptance_override_farm" in corpus220
def test_runner_regenerates_report():
entries, manifest = run_calibration.load_all()
reviewable = [e for e in entries if e.get("reviewable")]
stats = {"reviewable": len(reviewable), "guidance": 0,
"correct-approve": 0, "FALSE-APPROVE": 0,
"correct-bounce": 0, "FALSE-BOUNCE": 0}
rows = []
for e in reviewable:
v = run_calibration.evaluate(e)
bucket = run_calibration.classify(v.verdict, e["known_outcome"])
stats[bucket] += 1
rows.append(e)
bucket_keys = ("correct-approve", "FALSE-APPROVE", "correct-bounce",
"FALSE-BOUNCE", "guidance")
assert sum(stats[k] for k in bucket_keys) == len(reviewable)
rates = run_calibration._fmt_rates(stats)
assert rates["false_approve_rate"] is not None
assert 0.0 <= rates["false_approve_rate"] <= 1.0
report = run_calibration.render_report(
manifest, [{"id": r["id"], "repo": r["repo"], "pr": r.get("pr") or "-",
"known": r["known_outcome"], "verdict": "APPROVE",
"bucket": "correct-approve", "category": r["category"],
"note": ""} for r in rows],
stats, rates, [e for e in entries if not e.get("reviewable")], None)
assert "false-approve" in report
assert "Recommendation" in report
def test_manifest_has_no_duplicate_ids():
entries = _load()
ids = [e["id"] for e in entries]
assert len(ids) == len(set(ids))

279
tests/test_app.py
View File

@ -1,279 +0,0 @@
"""Tests for the assembled app (issue #38): one ``create_app()`` + entrypoints.
Hermetic: temp SQLite DBs, Flask test client, no network / Stripe / Gitea.
Acceptance contract exercised here:
1. ``create_app()``'s URL map contains the billing routes, the
``/api/factory/*`` routes, ``/dashboard`` and ``/healthz`` (asserted
on the URL map, not by requesting each one).
2. Starting the app with NO ``SAAS_API_KEY`` still starts, and every
authed route returns 503 -- not 500 and not 200; the liveness routes
stay public.
3. ``python -m gitea_saas`` serves ``/health`` locally (verified live as
part of the PR's test evidence; not a subprocess test here).
"""
from __future__ import annotations
from pathlib import Path
import pytest
from gitea_saas import app as app_module
from gitea_saas.app import create_app
from gitea_saas.auth import (API_KEY_HEADER, FORBIDDEN_BODY,
UNAUTHORIZED_BODY)
from gitea_saas.config import ServerConfig
API_KEY = "test-api-key-secret"
AUTH = {API_KEY_HEADER: API_KEY}
def _cfg(tmp_path: Path, api_key: str | None = API_KEY) -> ServerConfig:
return ServerConfig(api_key=api_key, port=8080, db_path=tmp_path / "saas.db")
@pytest.fixture()
def app(tmp_path: Path):
application = create_app(_cfg(tmp_path))
application.config.update(TESTING=True)
return application
@pytest.fixture()
def client(app):
return app.test_client()
def _rules(app) -> set[str]:
return {str(r.rule) for r in app.url_map.iter_rules()}
# ---------------------------------------------------------------------------
# Acceptance 1: the URL map composes every front-end
# ---------------------------------------------------------------------------
def test_url_map_contains_billing_routes(app):
rules = _rules(app)
for expected in (
"/api/clients",
"/api/clients/<int:client_id>",
"/api/clients/<int:client_id>/credits",
"/api/clients/<int:client_id>/debits",
"/api/clients/<int:client_id>/holds",
"/api/holds/<int:hold_id>/release",
"/api/clients/<int:client_id>/ledger",
):
assert expected in rules, f"missing billing route {expected}"
def test_url_map_contains_factory_routes(app):
rules = _rules(app)
for expected in (
"/api/factory/health",
"/api/factory/provision",
"/api/factory/<int:client_id>/issues",
"/api/factory/<int:client_id>/unlock",
"/api/factory/<int:client_id>/lock",
):
assert expected in rules, f"missing factory route {expected}"
def test_url_map_contains_dashboard_and_liveness_routes(app):
rules = _rules(app)
for expected in ("/dashboard", "/api/dashboard/summary",
"/healthz", "/health"):
assert expected in rules, f"missing route {expected}"
def test_url_map_contains_client_surface_routes(app):
"""Issue #41: the assembled app mounts the whole client surface."""
rules = _rules(app)
for expected in (
"/api/checkout",
"/api/me",
"/api/me/issues",
"/api/webhooks/stripe",
"/api/clients/<int:client_id>/keys",
"/api/clients/<int:client_id>/keys/<int:key_id>",
):
assert expected in rules, f"missing client route {expected}"
# ---------------------------------------------------------------------------
# Acceptance 2: no SAAS_API_KEY -> app starts, authed routes fail closed 503
# ---------------------------------------------------------------------------
def test_unset_api_key_fails_closed_503(tmp_path, monkeypatch):
monkeypatch.delenv("SAAS_API_KEY", raising=False)
application = create_app(_cfg(tmp_path, api_key=None))
application.config.update(TESTING=True)
c = application.test_client()
# authed routes: 503 -- not 500, not 200
assert c.get("/api/clients/1").status_code == 503
assert c.post("/api/clients", json={"name": "X"}).status_code == 503
assert c.get("/dashboard").status_code == 503
assert c.get("/api/dashboard/summary").status_code == 503
assert c.post("/api/factory/provision",
json={"client_id": 1}).status_code == 503
# issue #41: the client surface is authed too -> 503, not 200.
assert c.get("/api/me").status_code == 503
assert c.post("/api/checkout", json={"amount_minor": 100}).status_code == 503
assert c.post("/api/me/issues", json={}).status_code == 503
# ...but the signature-authenticated webhook is a PUBLIC_PATHS route
# (no Stripe configured here -> fail-closed 503, never a 200).
assert c.post("/api/webhooks/stripe", data=b"{}").status_code == 503
# liveness routes in auth.PUBLIC_PATHS stay reachable without a key
for path in ("/health", "/healthz", "/api/factory/health"):
assert c.get(path).status_code == 200, path
def test_default_create_app_with_no_env_starts(monkeypatch):
monkeypatch.delenv("SAAS_API_KEY", raising=False)
monkeypatch.delenv("SAAS_DB_PATH", raising=False)
application = create_app() # nothing set: config loader defaults
application.config.update(TESTING=True)
c = application.test_client()
assert c.get("/health").status_code == 200
assert c.get("/healthz").status_code == 200
assert c.get("/api/clients/1").status_code == 503
rules = _rules(application)
assert "/dashboard" in rules
assert "/api/factory/provision" in rules
# ---------------------------------------------------------------------------
# Configured key: the front-ends' own auth.key_is_valid guards are untouched
# ---------------------------------------------------------------------------
def test_configured_key_serves_billing(client):
created = client.post("/api/clients", json={"name": "Assembled"},
headers=AUTH)
assert created.status_code == 201
cid = created.get_json()["id"]
resp = client.get(f"/api/clients/{cid}", headers=AUTH)
assert resp.status_code == 200
assert resp.get_json()["name"] == "Assembled"
def test_wrong_key_is_canonical_401(client):
resp = client.post("/api/clients", json={"name": "X"},
headers={API_KEY_HEADER: "wrong"})
assert resp.status_code == 401
assert resp.get_json() == UNAUTHORIZED_BODY
# ---------------------------------------------------------------------------
# Issue #41 through the REAL assembled app: authenticated client path,
# operator/cross-tenant denials, checkout -> webhook credit
# ---------------------------------------------------------------------------
class _FakeGitea:
def create_org_repo(self, name):
pass
def create_issue(self, repo, title, body, labels=None):
return 1
def _make_client_app(tmp_path):
from gitea_saas.stripe_client import MockStripeClient
application = create_app(
_cfg(tmp_path),
stripe_builder=lambda: MockStripeClient(webhook_signature="test-sig"),
gitea=_FakeGitea())
application.config.update(TESTING=True)
return application
def test_assembled_app_authenticated_client_path_and_denials(tmp_path):
"""The assembled app serves the client surface and denies misuse.
Acceptance: client A's key works on ITS OWN surface through the real
composed app; it gets the canonical 403 on /api/clients, on client B's
/api/factory/* resources and on the dashboard; the operator key never
reaches the client surface. Checkout -> signature-verified webhook ->
balance credit (idempotent) also runs through the assembled app.
"""
import json as _json
application = _make_client_app(tmp_path)
c = application.test_client()
def op_post(path, body=None):
return c.post(path, json=body or {}, headers=AUTH)
# Provision + keys.
a_id = op_post("/api/clients", {"name": "AsmClientA"}).get_json()["id"]
b_id = op_post("/api/clients", {"name": "AsmClientB"}).get_json()["id"]
key_a = c.post(f"/api/clients/{a_id}/keys",
headers=AUTH).get_json()["key"]
key_b = c.post(f"/api/clients/{b_id}/keys",
headers=AUTH).get_json()["key"]
# Client A's own surface through the assembled app.
me = c.get("/api/me", headers={API_KEY_HEADER: key_a})
assert me.status_code == 200
assert me.get_json()["client_id"] == a_id
# Checkout -> webhook -> credit (idempotent), through the assembled app.
checkout = c.post("/api/checkout",
json={"amount_minor": 3000},
headers={API_KEY_HEADER: key_a})
assert checkout.status_code == 201
event = {"id": "evt_asm_1", "type": "payment.succeeded",
"data": {"object": {"metadata": {"client_id": str(a_id)},
"amount_total": 3000}}}
payload = _json.dumps(event).encode()
wh = c.post("/api/webhooks/stripe", data=payload,
content_type="application/json",
headers={"Stripe-Signature": "test-sig"})
assert wh.status_code == 200
me = c.get("/api/me", headers={API_KEY_HEADER: key_a}).get_json()
assert me["balance_minor"] == 3000
replay = c.post("/api/webhooks/stripe", data=payload,
content_type="application/json",
headers={"Stripe-Signature": "test-sig"})
assert replay.status_code == 200
me = c.get("/api/me", headers={API_KEY_HEADER: key_a}).get_json()
assert me["balance_minor"] == 3000 # exactly once
# Cross-tenant + operator denials (canonical 403 body).
for path in (f"/api/factory/{b_id}/lock",
f"/api/factory/{b_id}/issues",
f"/api/clients/{b_id}",
f"/api/clients/{b_id}/keys"):
resp = c.post(path, json={}, headers={API_KEY_HEADER: key_a})
assert resp.status_code == 403, path
assert resp.get_json() == FORBIDDEN_BODY, path
assert c.get("/api/clients",
headers={API_KEY_HEADER: key_a}).status_code == 403
assert c.get("/dashboard",
headers={API_KEY_HEADER: key_a}).status_code == 403
# Client B still sees only itself.
assert c.get("/api/me", headers={API_KEY_HEADER: key_b}).get_json()[
"client_id"] == b_id
# The operator key never works on the client surface.
assert c.get("/api/me", headers=AUTH).status_code == 401
# ---------------------------------------------------------------------------
# Entrypoints exist and are wired to the same main()
# ---------------------------------------------------------------------------
def test_console_entrypoint_target_exists():
# [project.scripts] gitea-saas = "gitea_saas.app:main"
assert callable(app_module.main)
def test_python_m_gitea_saas_uses_app_main():
from gitea_saas import __main__ as main_module
assert main_module.main is app_module.main

288
tests/test_auth.py
View File

@ -1,288 +0,0 @@
"""Acceptance tests for the single X-API-Key auth path (issue #24).
Covers the issue's acceptance criteria against the ONE consolidated
authenticator (``gitea_saas.auth``):
1. exactly one ``compare_digest`` on the API-key path (auth.py; the
``stripe_client.py`` match is webhook-signature verification, which the
issue explicitly scopes out);
2. exactly one ``"X-API-Key"`` literal in the package (``auth.py``);
3. a wrong key of the SAME length is rejected, and an unset configured key
refuses every request rather than serving;
4. the 401 body is byte-identical across all front-ends (server, delivery,
intake factory, dashboard);
5. route exposure is declared in one place (health/liveness routes in
``auth.PUBLIC_PATHS``) and asserted per front-end, including the composed
production app (server app + mounted dashboard).
"""
from __future__ import annotations
import sys
import types
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
import pytest
import flask
from gitea_saas.auth import (API_KEY_HEADER, PUBLIC_PATHS, UNAUTHORIZED_BODY,
key_is_valid)
from gitea_saas.dashboard import mount_dashboard
from gitea_saas.delivery import Delivery, default_config
from gitea_saas.intake_factory import register_factory_routes
from gitea_saas import server as server_mod
API_KEY = "sekret"
AUTH = {API_KEY_HEADER: API_KEY}
class _Core:
"""Tiny billing-core stub good enough for auth-path tests.
Auth runs BEFORE any core call, so refusal tests never touch it; the
minimal create_client/get_client support the few positive probes.
"""
def create_client(self, name, tier="free", status="active", meta=None):
return True, {"id": 1, "client_id": 1, "name": name, "tier": tier,
"status": status, "meta": meta or {},
"created_at": "2026-01-01T00:00:00+00:00"}
def get_client(self, client_id, include_balance=True):
return True, {"id": client_id, "name": "Acme", "tier": "free",
"status": "active", "meta": {}, "balance": 0,
"created_at": "2026-01-01T00:00:00+00:00"}
# --------------------------------------------------------------------------
# Per-front-end apps (all built the way the owning tests build them)
# --------------------------------------------------------------------------
@pytest.fixture()
def server_app():
app = server_mod.create_app(_Core(), API_KEY)
app.config.update(TESTING=True)
return app
@pytest.fixture()
def delivery():
return Delivery(core=None, config=default_config({"SAAS_API_KEY": API_KEY}))
@pytest.fixture()
def dashboard_app(tmp_path):
a = flask.Flask(__name__)
mount_dashboard(a, core=_Core(), db_path=tmp_path / "dash.db", api_key=API_KEY)
a.config.update(TESTING=True)
return a
@pytest.fixture()
def intake_app():
a = flask.Flask(__name__)
a.config["API_KEY"] = API_KEY
cfg = types.SimpleNamespace(db_path=":memory:", api_key=API_KEY,
debit_floor_minor=500)
register_factory_routes(a, _Core(), types.SimpleNamespace(), cfg)
a.config.update(TESTING=True)
return a
# --------------------------------------------------------------------------
# Acceptance criterion 1 + 2: one implementation, one literal
# --------------------------------------------------------------------------
def _pkg_dir() -> Path:
# gitea_saas is a PEP 420 namespace package (no __init__.py), so resolve
# the package directory from this test file's own location.
return Path(__file__).resolve().parent.parent / "gitea_saas"
def test_single_compare_digest_on_api_key_path():
"""Criterion 1 scoped to the API-key path: exactly ONE comparison call
site on the API-key path (in auth.py). The second package call site is
``stripe_client.py`` webhook-SIGNATURE verification, which the issue
explicitly scopes out (leave it alone). No front-end performs its own
comparison."""
hits = []
for py in sorted(_pkg_dir().glob("*.py")):
for i, line in enumerate(py.read_text(encoding="utf-8").splitlines(), 1):
if "compare_digest(" in line: # call sites, not docstring mentions
hits.append(f"{py.name}:{i}")
assert hits == ["auth.py:70", "stripe_client.py:88"], f"unexpected call sites: {hits}"
def test_single_x_api_key_literal():
"""Criterion 2: exactly one ``\"X-API-Key\"`` literal in the package."""
hits = []
for py in sorted(_pkg_dir().glob("*.py")):
for i, line in enumerate(py.read_text(encoding="utf-8").splitlines(), 1):
if '"X-API-Key"' in line:
hits.append(f"{py.name}:{i}")
assert len(hits) == 1, f"expected exactly one literal, got {hits}"
assert hits[0].startswith("auth.py"), f"the literal must live in auth.py: {hits}"
# --------------------------------------------------------------------------
# key_is_valid (the single implementation)
# --------------------------------------------------------------------------
class TestKeyIsValid:
def test_matching_key(self):
assert key_is_valid(API_KEY, API_KEY) is True
def test_wrong_key_of_same_length_rejected(self):
# criterion 3: same-length wrong key must NOT pass (constant-time path)
wrong = "hunter2".ljust(len(API_KEY), "x")[:len(API_KEY)]
assert len(wrong) == len(API_KEY)
assert wrong != API_KEY
assert key_is_valid(wrong, API_KEY) is False
def test_unset_configured_key_refuses(self):
# criterion 3: unset configured key refuses everything, never serves
assert key_is_valid(API_KEY, None) is False
assert key_is_valid(API_KEY, "") is False
assert key_is_valid(API_KEY, b"sekret") is False # type: ignore[arg-type]
def test_missing_or_non_str_provided(self):
assert key_is_valid(None, API_KEY) is False
assert key_is_valid("", API_KEY) is False
assert key_is_valid(b"sekret", API_KEY) is False # type: ignore[arg-type]
# --------------------------------------------------------------------------
# Criterion 4: byte-identical 401 body from ALL four front-ends
# --------------------------------------------------------------------------
def test_401_body_identical_across_all_front_ends(server_app, delivery,
dashboard_app, intake_app):
# server (Flask)
r = server_app.test_client().get("/api/clients/1")
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
# delivery (plain (dict, status) seam)
result, status = delivery.handle_merge_webhook({}, provided_key=None)
assert status == 401 and result == UNAUTHORIZED_BODY
# dashboard (bare app + per-route guard)
r = dashboard_app.test_client().get("/api/dashboard/summary")
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
# intake factory (Flask)
r = intake_app.test_client().post("/api/factory/provision",
json={"client_id": 1})
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
def test_wrong_key_same_length_401_across_front_ends(server_app, delivery,
dashboard_app, intake_app):
wrong = "hunter2".ljust(len(API_KEY), "x")[:len(API_KEY)]
assert len(wrong) == len(API_KEY) and wrong != API_KEY
assert server_app.test_client().get(
"/api/clients/1", headers={API_KEY_HEADER: wrong}).status_code == 401
r, s = delivery.handle_merge_webhook({}, provided_key=wrong)
assert s == 401 and r == UNAUTHORIZED_BODY
assert dashboard_app.test_client().get(
"/api/dashboard/summary",
headers={API_KEY_HEADER: wrong}).status_code == 401
assert intake_app.test_client().post(
"/api/factory/provision", json={"client_id": 1},
headers={API_KEY_HEADER: wrong}).status_code == 401
def test_unset_configured_key_refuses_every_request(tmp_path):
# server: empty configured key -> deny all, even with a key presented
app = server_mod.create_app(_Core(), "")
app.config.update(TESTING=True)
r = app.test_client().get("/api/clients/1", headers=AUTH)
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
# dashboard: no configured key -> 401 (was 503 pre-consolidation)
a = flask.Flask(__name__)
mount_dashboard(a, core=_Core(), db_path=tmp_path / "d2.db", api_key=None)
a.config.update(TESTING=True)
r = a.test_client().get("/api/dashboard/summary")
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
# delivery: config with no api_key -> unauthorized
d = Delivery(core=None, config=default_config({}))
assert d.authorized(API_KEY) is False
r, s = d.handle_merge_webhook({}, provided_key=API_KEY)
assert s == 401 and r == UNAUTHORIZED_BODY
# intake: empty configured key -> 401
a2 = flask.Flask(__name__)
a2.config["API_KEY"] = ""
cfg = types.SimpleNamespace(db_path=":memory:", api_key="",
debit_floor_minor=500)
register_factory_routes(a2, _Core(), types.SimpleNamespace(), cfg)
a2.config.update(TESTING=True)
r = a2.test_client().post("/api/factory/provision", json={"client_id": 1},
headers=AUTH)
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
# --------------------------------------------------------------------------
# Route exposure: declared in PUBLIC_PATHS, honored by every front-end
# --------------------------------------------------------------------------
def test_public_paths_are_open(server_app, dashboard_app, intake_app):
assert server_app.test_client().get("/health").status_code == 200
assert dashboard_app.test_client().get("/healthz").status_code == 200
assert intake_app.test_client().get("/api/factory/health").status_code == 200
assert "/health" in PUBLIC_PATHS
assert "/healthz" in PUBLIC_PATHS
assert "/api/factory/health" in PUBLIC_PATHS
def test_composed_app_exposure(tmp_path):
"""Mount every front-end onto the ONE production app (server factory)
and assert each registered route's exposure — the composition test no
single PR's suite covers (owner note, 2026-08-26)."""
db = tmp_path / "composed.db"
app = server_mod.create_app(_Core(), API_KEY)
mount_dashboard(app, core=_Core(), db_path=db, api_key=API_KEY)
cfg = types.SimpleNamespace(db_path=db, api_key=API_KEY,
debit_floor_minor=500)
register_factory_routes(app, _Core(), types.SimpleNamespace(), cfg)
app.config.update(TESTING=True)
c = app.test_client()
# public liveness routes — reachable with NO key
assert c.get("/health").status_code == 200
assert c.get("/healthz").status_code == 200
assert c.get("/api/factory/health").status_code == 200
# protected routes — 401 without a key, 200/201 with the key
assert c.get("/api/clients/1").status_code == 401
assert c.get("/api/clients/1", headers=AUTH).status_code == 200
assert c.get("/api/dashboard/summary").status_code == 401
assert c.get("/api/dashboard/summary", headers=AUTH).status_code == 200
# /dashboard (HTML) is guarded on the composed app (server #2 guard)
# AND on the bare dashboard mount (issue #25, expressed through the
# shared auth path — see the bare-mount test below).
assert c.get("/dashboard").status_code == 401
assert c.get("/dashboard", headers=AUTH).status_code == 200
def test_bare_dashboard_mount_guards_page_and_json(tmp_path):
"""The dashboard's OWN mount guards /dashboard (issue #25, through the
shared auth module: constant-time key_is_valid + canonical 401) the same
way it guards /api/dashboard/summary — exposure is not silently
inherited from mount order, and no billing data leaks without a key."""
a = flask.Flask(__name__)
mount_dashboard(a, core=_Core(), db_path=tmp_path / "b.db", api_key=API_KEY)
a.config.update(TESTING=True)
c = a.test_client()
r = c.get("/dashboard")
assert r.status_code == 401
assert r.get_json() == UNAUTHORIZED_BODY
assert "Acme" not in r.get_data(as_text=True)
assert c.get("/dashboard", headers=AUTH).status_code == 200
assert c.get("/api/dashboard/summary").status_code == 401
assert c.get("/healthz").status_code == 200

551
tests/test_client_api.py
View File

@ -1,551 +0,0 @@
"""Hermetic tests for the client-facing surface (issue #41).
Per-client API keys (hashed at rest, scoped, revocable), self-serve
checkout (rate-limited), the Stripe webhook credit path (idempotent via
the issue-#3 seam), ``GET /api/me`` client status, self-serve work
submission, and the cross-tenant / operator-route denial guarantees.
Zero network: Flask test client + temp SQLite DB + MockStripeClient +
FakeGitea. Run with:
python -m pytest tests/test_client_api.py -q
"""
import json
import sys
from pathlib import Path
from typing import Optional
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from gitea_saas.auth import (API_KEY_HEADER, FORBIDDEN_BODY, # noqa: E402
UNAUTHORIZED_BODY, hash_client_key)
from gitea_saas.billing_core import BillingCore # noqa: E402
from gitea_saas.client_api import (ClientKeyStore, ClientStatus, # noqa: E402
ClientSurface, register_client_routes)
from gitea_saas.config import ServerConfig # noqa: E402
from gitea_saas.stripe_client import MockStripeClient # noqa: E402
API_KEY = "operator-test-key-41"
AUTH = {API_KEY_HEADER: API_KEY}
# Explicit mock signature: tests only, never a production default.
MOCK_SIG = "test-webhook-sig-41"
class FakeGitea:
"""In-memory create_org_repo/create_issue seam (factory contract)."""
def __init__(self):
self.repos = set()
self.issues = []
def create_org_repo(self, name):
self.repos.add(name)
def create_issue(self, repo, title, body, labels=None):
number = len(self.issues) + 1
self.issues.append({"repo": repo, "title": title, "body": body,
"labels": labels, "number": number})
return number
def _mock_stripe_builder():
return MockStripeClient(webhook_signature=MOCK_SIG)
def _cfg(tmp_path: Path, api_key: Optional[str] = API_KEY) -> ServerConfig:
return ServerConfig(api_key=api_key, port=8080, db_path=tmp_path / "saas.db")
def _webhook_payload(event_id="evt_41_1", client_id=1, amount=2500):
return {"id": event_id, "type": "payment.succeeded",
"data": {"object": {"metadata": {"client_id": str(client_id)},
"amount_total": amount}}}
@pytest.fixture()
def client_factory(tmp_path):
"""Build ONE assembled app per test (cached); returns a zero-arg builder."""
state = {}
def build():
if "app" in state:
return state["app"], state["gitea"]
from gitea_saas.app import create_app
gitea = FakeGitea()
application = create_app(_cfg(tmp_path, API_KEY),
stripe_builder=_mock_stripe_builder,
gitea=gitea)
application.config.update(TESTING=True)
state["app"] = application
state["gitea"] = gitea
return application, gitea
build()
return build
@pytest.fixture()
def client(client_factory):
application, _ = client_factory()
return application.test_client()
@pytest.fixture()
def gitea(client_factory):
return client_factory()[1]
# ---------------------------------------------------------------------------
# helpers: provision clients + issue keys through the operator API
# ---------------------------------------------------------------------------
def _create_client(client, name: str) -> int:
resp = client.post("/api/clients", json={"name": name}, headers=AUTH)
assert resp.status_code == 201
return resp.get_json()["id"]
def _issue_key(client, client_id: int) -> dict:
resp = client.post(f"/api/clients/{client_id}/keys", headers=AUTH)
assert resp.status_code == 201
return resp.get_json()
def _client_header(raw_key: str) -> dict:
return {API_KEY_HEADER: raw_key}
# ---------------------------------------------------------------------------
# ClientKeyStore unit tests (acceptance criterion 2: hashed at rest)
# ---------------------------------------------------------------------------
def test_keys_stored_hashed_plaintext_never_in_db(tmp_path):
core = BillingCore(tmp_path / "saas.db")
ok, rec = core.create_client("HashCo")
store = ClientKeyStore(tmp_path / "saas.db")
ok, issued = store.issue(rec["id"])
assert ok
raw = issued["key"]
# The plaintext key is not anywhere in the DB — not as a value, not as
# a column. The stored material is hash + display prefix only.
rows = store._conn.execute(
"SELECT key_hash, key_prefix FROM client_api_keys").fetchall()
assert len(rows) == 1
stored_hash, stored_prefix = rows[0]["key_hash"], rows[0]["key_prefix"]
assert hash_client_key(raw) == stored_hash
assert stored_hash != raw
assert stored_prefix != raw # prefix is a truncated display hint
assert raw not in stored_hash
# Dump the whole DB as text: plaintext must not appear anywhere.
blob = " ".join(str(v) for row in store._conn.execute(
"SELECT * FROM client_api_keys") for v in row)
assert raw not in blob
def test_resolve_client_id_scoped_and_revoked(tmp_path):
core = BillingCore(tmp_path / "saas.db")
_, a = core.create_client("Alice")
_, b = core.create_client("Bob")
store = ClientKeyStore(tmp_path / "saas.db")
_, ka = store.issue(a["id"])
_, kb = store.issue(b["id"])
assert store.resolve_client_id(ka["key"]) == a["id"]
assert store.resolve_client_id(kb["key"]) == b["id"]
assert store.resolve_client_id("saas_wrong") is None
assert store.resolve_client_id("") is None
assert store.resolve_client_id(None) is None
# Revoke A's key only; B's keeps resolving.
ok, _ = store.revoke(a["id"], ka["key_id"])
assert ok
assert store.resolve_client_id(ka["key"]) is None
assert store.resolve_client_id(kb["key"]) == b["id"]
def test_revoke_is_scoped_no_cross_tenant_revoke(tmp_path):
core = BillingCore(tmp_path / "saas.db")
_, a = core.create_client("Alice")
_, b = core.create_client("Bob")
store = ClientKeyStore(tmp_path / "saas.db")
_, ka = store.issue(a["id"])
ok, details = store.revoke(b["id"], ka["key_id"])
assert not ok and (details.get("error") or {}).get("code") == "KEY_NOT_FOUND"
assert store.resolve_client_id(ka["key"]) == a["id"] # still active
ok, details = store.revoke(a["id"], 999999)
assert not ok and (details.get("error") or {}).get("code") == "KEY_NOT_FOUND"
def test_list_and_revoke_all_never_expose_secrets(tmp_path):
core = BillingCore(tmp_path / "saas.db")
_, a = core.create_client("Alice")
store = ClientKeyStore(tmp_path / "saas.db")
_, k1 = store.issue(a["id"])
_, k2 = store.issue(a["id"])
ok, listing = store.list_keys(a["id"])
assert ok and listing["count"] == 2
for item in listing["keys"]:
assert "key" not in item and "key_hash" not in item
ok, res = store.revoke_all(a["id"])
assert ok and res["revoked_count"] == 2
assert store.resolve_client_id(k1["key"]) is None
assert store.resolve_client_id(k2["key"]) is None
def test_issue_unknown_client(tmp_path):
BillingCore(tmp_path / "saas.db")
store = ClientKeyStore(tmp_path / "saas.db")
ok, details = store.issue(999)
assert not ok and (details.get("error") or {}).get("code") == "UNKNOWN_CLIENT"
# ---------------------------------------------------------------------------
# Operator key management HTTP routes
# ---------------------------------------------------------------------------
def test_operator_issues_and_revokes_keys(client):
cid = _create_client(client, "KeyCo")
issued = _issue_key(client, cid)
assert issued["client_id"] == cid
assert issued["key"].startswith("saas_")
assert issued["prefix"] == issued["key"][:10]
assert len(issued["key"]) > 20
# Listing shows metadata only.
listing = client.get(f"/api/clients/{cid}/keys", headers=AUTH)
assert listing.status_code == 200
assert listing.get_json()["count"] == 1
assert "key" not in listing.get_json()["keys"][0]
# The key works on the client surface.
me = client.get("/api/me", headers=_client_header(issued["key"]))
assert me.status_code == 200
assert me.get_json()["client_id"] == cid
# Revoke via the operator route: key stops working.
rev = client.delete(f"/api/clients/{cid}/keys/{issued['key_id']}",
headers=AUTH)
assert rev.status_code == 200
assert client.get("/api/me",
headers=_client_header(issued["key"])).status_code == 401
def test_key_routes_require_operator_key(client):
cid = _create_client(client, "LockedCo")
assert client.post(f"/api/clients/{cid}/keys").status_code == 401
assert client.get(f"/api/clients/{cid}/keys").status_code == 401
# A CLIENT key (other tenant) is 403 on key issuance routes.
key = _issue_key(client, cid)
other = _create_client(client, "OtherCo")
resp = client.post(f"/api/clients/{other}/keys",
headers=_client_header(key["key"]))
assert resp.status_code == 403
assert resp.get_json() == FORBIDDEN_BODY
# ---------------------------------------------------------------------------
# Tenancy: a client key never reaches another client's resources, and never
# reaches an operator route (acceptance criterion 1 + security note)
# ---------------------------------------------------------------------------
def test_cross_tenant_and_operator_denials(client):
a_id = _create_client(client, "TenantA")
b_id = _create_client(client, "TenantB")
key_a = _issue_key(client, a_id)["key"]
# Client A's key on client B's resources: 403 everywhere.
for path in (
f"/api/factory/{b_id}/issues",
f"/api/factory/{b_id}/lock",
f"/api/factory/{b_id}/unlock",
f"/api/clients/{b_id}",
f"/api/clients/{b_id}/ledger",
f"/api/clients/{b_id}/credits",
):
resp = client.post(path, json={}, headers=_client_header(key_a))
assert resp.status_code == 403, path
assert resp.get_json() == FORBIDDEN_BODY
# Operator routes: /api/clients, /api/factory/*, /dashboard.
assert client.get("/api/clients", headers=_client_header(key_a)) \
.status_code == 403
assert client.post("/api/factory/provision",
json={"client_id": b_id},
headers=_client_header(key_a)).status_code == 403
assert client.get("/dashboard", headers=_client_header(key_a)) \
.status_code == 403
assert client.get("/api/dashboard/summary", headers=_client_header(key_a)) \
.status_code == 403
# And the operator key never works on the client surface.
assert client.get("/api/me", headers=AUTH).status_code == 401
assert client.get("/api/me").get_json() == UNAUTHORIZED_BODY
assert client.post("/api/checkout", json={"amount_minor": 100},
headers=AUTH).status_code == 401
def test_each_client_sees_only_its_own_me(client):
a_id = _create_client(client, "SelfScopeA")
b_id = _create_client(client, "SelfScopeB")
key_a = _issue_key(client, a_id)["key"]
key_b = _issue_key(client, b_id)["key"]
me_a = client.get("/api/me", headers=_client_header(key_a)).get_json()
me_b = client.get("/api/me", headers=_client_header(key_b)).get_json()
assert me_a["client_id"] == a_id
assert me_b["client_id"] == b_id
assert me_a["name"] == "SelfScopeA"
assert me_b["name"] == "SelfScopeB"
# ---------------------------------------------------------------------------
# Checkout (rate-limited) + webhook credit (idempotent) — acceptance 3
# ---------------------------------------------------------------------------
def test_checkout_creates_session_scoped_to_caller(client, tmp_path):
cid = _create_client(client, "CheckoutCo")
key = _issue_key(client, cid)["key"]
resp = client.post("/api/checkout",
json={"amount_minor": 2500,
"success_url": "https://example.test/ok"},
headers=_client_header(key))
assert resp.status_code == 201
body = resp.get_json()
assert set(body) == {"checkout_url", "session_id"}
assert body["checkout_url"].startswith("https://checkout.stripe.com/mock/")
assert body["session_id"].startswith("cs_mock_")
def test_checkout_webhook_credit_exactly_once(client, tmp_path):
"""Acceptance 3: checkout -> webhook -> balance credit, replay no-op."""
cid = _create_client(client, "FundCo")
key = _issue_key(client, cid)["key"]
checkout = client.post("/api/checkout", json={"amount_minor": 2500},
headers=_client_header(key))
assert checkout.status_code == 201
payload = _webhook_payload(client_id=cid, amount=2500)
headers = {"Stripe-Signature": MOCK_SIG}
first = client.post("/api/webhooks/stripe",
data=json.dumps(payload), # raw bytes path
content_type="application/json", headers=headers)
assert first.status_code == 200
assert first.get_json()["applied"] is True
balance = client.get("/api/me", headers=_client_header(key)).get_json()
assert balance["balance_minor"] == 2500
# Replayed webhook (same event id): exactly once — no double credit.
replay = client.post("/api/webhooks/stripe",
data=json.dumps(payload),
content_type="application/json", headers=headers)
assert replay.status_code == 200
balance_after = client.get("/api/me",
headers=_client_header(key)).get_json()
assert balance_after["balance_minor"] == 2500
def test_webhook_rejects_bad_signature_and_bad_payload(client):
cid = _create_client(client, "WebhookCo")
payload = json.dumps(_webhook_payload(client_id=cid)).encode()
resp = client.post("/api/webhooks/stripe", data=payload,
content_type="application/json",
headers={"Stripe-Signature": "wrong-sig"})
assert resp.status_code == 400
assert resp.get_json()["error"]["code"] == "invalid_signature"
# No signature header at all: same generic 400.
resp = client.post("/api/webhooks/stripe", data=payload,
content_type="application/json")
assert resp.status_code == 400
def test_checkout_validation(client):
cid = _create_client(client, "BadCo")
key = _issue_key(client, cid)["key"]
h = _client_header(key)
assert client.post("/api/checkout", json={"amount_minor": 0},
headers=h).status_code == 400
assert client.post("/api/checkout", json={"amount_minor": "50"},
headers=h).status_code == 400
assert client.post("/api/checkout", json={"amount_minor": 100,
"currency": "too-long"},
headers=h).status_code == 400
assert client.post("/api/checkout",
json={"amount_minor": 100,
"success_url": "javascript:alert(1)"},
headers=h).status_code == 400
assert client.post("/api/checkout", json="nope",
headers=h).status_code == 400
def test_checkout_rate_limited_per_client(tmp_path):
"""Rate-limit /api/checkout per client (security note)."""
db = tmp_path / "saas.db"
core = BillingCore(db)
ok, rec = core.create_client("RatedCo")
store = ClientKeyStore(db)
ok, issued = store.issue(rec["id"])
import flask
from gitea_saas.client_api import ClientStatus
app = flask.Flask(__name__)
app.config["API_KEY"] = API_KEY
surface = ClientSurface(core, store, ClientStatus(core, db),
_mock_stripe_builder,
checkout_rate=2, checkout_window_s=60)
register_client_routes(app, surface,
gitea=FakeGitea(),
config=type("C", (), {"db_path": db})())
app.config.update(TESTING=True)
c = app.test_client()
h = _client_header(issued["key"])
assert c.post("/api/checkout", json={"amount_minor": 100},
headers=h).status_code == 201
assert c.post("/api/checkout", json={"amount_minor": 100},
headers=h).status_code == 201
third = c.post("/api/checkout", json={"amount_minor": 100}, headers=h)
assert third.status_code == 429
assert third.get_json()["error"]["code"] == "rate_limited"
assert third.headers.get("Retry-After") is not None
# Other client's limiter is its own (no cross-tenant throttling).
ok, rec2 = core.create_client("OtherRated")
ok, issued2 = store.issue(rec2["id"])
assert c.post("/api/checkout", json={"amount_minor": 100},
headers=_client_header(issued2["key"])).status_code == 201
def test_stripe_unconfigured_fails_closed(tmp_path, monkeypatch):
"""No Stripe mode/creds -> checkout & webhook 503, never a 200."""
monkeypatch.delenv("SAAS_STRIPE_MODE", raising=False)
monkeypatch.delenv("STRIPE_SECRET_KEY", raising=False)
monkeypatch.delenv("STRIPE_WEBHOOK_SECRET", raising=False)
from gitea_saas.app import create_app
application = create_app(_cfg(tmp_path, API_KEY), gitea=FakeGitea())
application.config.update(TESTING=True)
c = application.test_client()
cid = c.post("/api/clients", json={"name": "NoStripe"},
headers=AUTH).get_json()["id"]
key = c.post(f"/api/clients/{cid}/keys", headers=AUTH).get_json()["key"]
checkout = c.post("/api/checkout", json={"amount_minor": 100},
headers=_client_header(key))
assert checkout.status_code == 503
assert checkout.get_json()["error"]["code"] == "stripe_unconfigured"
webhook = c.post("/api/webhooks/stripe",
data=json.dumps(_webhook_payload()).encode(),
content_type="application/json",
headers={"Stripe-Signature": "anything"})
assert webhook.status_code == 503
# ---------------------------------------------------------------------------
# /api/me status (balance, queued, in-flight, delivered)
# ---------------------------------------------------------------------------
def test_me_reports_balance_and_counts(client, gitea, tmp_path):
cid = _create_client(client, "StatusCo")
key = _issue_key(client, cid)["key"]
# Unfunded submit -> queued (no Gitea issue created).
resp = client.post("/api/me/issues",
json={"repo": "statusco-repo",
"title": "First task",
"acceptance_body": "do it"},
headers=_client_header(key))
assert resp.status_code == 202
assert resp.get_json()["delivered"] is False
me = client.get("/api/me", headers=_client_header(key)).get_json()
assert me["queued_items"] == 1
assert me["in_flight_items"] == 0
assert me["delivered_count"] == 0
assert me["balance_minor"] == 0
# Fund + lock: submits now queue; unlock flushes them to Gitea, which
# is what the factory records as "shipped" (in-flight, awaiting merge).
core = BillingCore(tmp_path / "saas.db")
ok, _ = core.credit(cid, 10000, "fund-1")
assert ok
assert client.post(f"/api/factory/{cid}/lock", json={},
headers=AUTH).status_code == 200
resp = client.post("/api/me/issues",
json={"repo": "statusco-repo",
"title": "Second task",
"acceptance_body": "do it too"},
headers=_client_header(key))
assert resp.status_code == 202
assert resp.get_json()["reason"] == "locked"
me = client.get("/api/me", headers=_client_header(key)).get_json()
assert me["queued_items"] == 2 # unfunded + locked
assert me["in_flight_items"] == 0
flush = client.post(f"/api/factory/{cid}/unlock", json={}, headers=AUTH)
assert flush.status_code == 200
assert flush.get_json()["flushed"] == 2
assert len(gitea.issues) == 2
me = client.get("/api/me", headers=_client_header(key)).get_json()
assert me["queued_items"] == 0
assert me["in_flight_items"] == 2 # shipped to Gitea, not delivered
assert me["balance_minor"] == 10000
# Delivery debit (what delivery.py writes on merge) -> delivered count.
ok, _ = core.debit(cid, 500, "deliver:statusco-repo:1:sha",
ref_type="pr", ref_id="statusco-repo#1")
assert ok
me = client.get("/api/me", headers=_client_header(key)).get_json()
assert me["delivered_count"] == 1
assert me["balance_minor"] == 9500
def test_me_issues_validation_and_scoping(client, tmp_path):
cid = _create_client(client, "SubmitCo")
other = _create_client(client, "OtherSubmit")
key = _issue_key(client, cid)["key"]
other_key = _issue_key(client, other)["key"]
h = _client_header(key)
assert client.post("/api/me/issues", json={"repo": "../evil",
"title": "x"},
headers=h).status_code == 400
assert client.post("/api/me/issues", json={"repo": "good-repo",
"title": ""},
headers=h).status_code == 400
# The submit is ALWAYS scoped to the caller — a queued row can only
# ever belong to the key's own client_id (factory enforces existence).
resp = client.post("/api/me/issues",
json={"repo": "submitco-repo", "title": "Task"},
headers=_client_header(other_key))
assert resp.status_code == 202
me_other = client.get("/api/me", headers=_client_header(other_key)).get_json()
assert me_other["queued_items"] == 1
me_self = client.get("/api/me", headers=h).get_json()
assert me_self["queued_items"] == 0
# ---------------------------------------------------------------------------
# Assembled-app reachability + fail-closed 503 (no SAAS_API_KEY)
# ---------------------------------------------------------------------------
def test_assembled_app_has_client_routes(client_factory, tmp_path, monkeypatch):
monkeypatch.delenv("SAAS_API_KEY", raising=False)
from gitea_saas.app import create_app
application = create_app(_cfg(tmp_path, api_key=None), gitea=FakeGitea())
application.config.update(TESTING=True)
c = application.test_client()
# No SAAS_API_KEY: every authed route (client surface included) is 503.
assert c.get("/api/me").status_code == 503
assert c.post("/api/checkout", json={"amount_minor": 100}).status_code == 503
assert c.post("/api/me/issues", json={}).status_code == 503

46
tests/test_dashboard.py
View File

@ -27,7 +27,6 @@ import pytest
import flask
from gitea_saas.auth import UNAUTHORIZED_BODY
from gitea_saas.billing_core import BillingCore
from gitea_saas.dashboard import (Dashboard, EMPTY, render_html,
mount_dashboard, API_KEY_HEADER)
@ -292,9 +291,7 @@ class TestMountContract:
does NOT define or run a second Flask app (review finding: #2 owns
the app factory; three-way server.py conflict removed)."""
client = app.test_client()
# /dashboard is key-protected (issue #25): open with a valid key.
assert client.get("/dashboard",
headers={API_KEY_HEADER: "sekret"}).status_code == 200
assert client.get("/dashboard").status_code == 200
assert client.get("/healthz").status_code == 200
def test_no_second_server_module_in_this_issue(self):
@ -368,38 +365,12 @@ class TestServer:
def test_dashboard_page_open_and_html(self, app, seeded):
client = app.test_client()
r = client.get("/dashboard", headers={API_KEY_HEADER: "sekret"})
r = client.get("/dashboard")
assert r.status_code == 200
assert "text/html" in r.content_type
assert "SaaS dashboard" in r.get_data(as_text=True)
assert "Acme" in r.get_data(as_text=True)
def test_dashboard_page_guard_blocks_anonymous_access(self, app, seeded):
"""GET /dashboard with NO X-API-Key must NOT return billing data
(issue #25): same fail-closed guard as the JSON summary — 401 and no
client/ledger/queue content in the body."""
client = app.test_client()
r = client.get("/dashboard")
# no auth → 401, and no billing data leaks in the body
assert r.status_code == 401
body = r.get_data(as_text=True)
assert "Acme" not in body
assert "Poorco" not in body
assert "SaaS dashboard" not in body
def test_dashboard_wrong_key_same_length_rejected(self, app):
"""A wrong key of the SAME length as the configured key must be
rejected with 401 on the HTML page too (matches the JSON endpoint's
constant-time comparison contract, issue #25)."""
client = app.test_client()
configured = "sekret"
wrong_same_length = "hunter"
assert len(wrong_same_length) == len(configured)
r = client.get("/dashboard",
headers={API_KEY_HEADER: wrong_same_length})
assert r.status_code == 401
assert "Acme" not in r.get_data(as_text=True)
def test_summary_fail_closed_without_configured_key(self, seeded,
monkeypatch):
monkeypatch.delenv("SAAS_API_KEY", raising=False)
@ -408,16 +379,9 @@ class TestServer:
mount_dashboard(a, core=core, db_path=db, api_key=None)
client = a.test_client()
r = client.get("/api/dashboard/summary")
# Any refusal (missing, wrong, or UNSET configured key) is the same
# canonical 401 body (issue #24) — fail-closed, never anonymous.
assert r.status_code == 401
assert r.get_json() == UNAUTHORIZED_BODY
# The HTML page fails closed the same way (issue #25, expressed
# through the shared auth path): no key configured → 401, and no
# billing data is returned either way.
r2 = client.get("/dashboard")
assert r2.status_code == 401
assert "Acme" not in r2.get_data(as_text=True)
assert r.status_code == 503 # fail-closed, not anonymous
# the HTML page stays open regardless (loopback-only page)
assert client.get("/dashboard").status_code == 200
def test_healthz(self, app):
r = app.test_client().get("/healthz")

27
tests/test_delivery.py
View File

@ -13,7 +13,6 @@ import pytest
# regardless of how pytest was invoked.
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from gitea_saas.auth import UNAUTHORIZED_BODY # noqa: E402
from gitea_saas.billing_core import BillingCore # noqa: E402
from gitea_saas.delivery import ( # noqa: E402
Delivery,
@ -76,7 +75,7 @@ def make_delivery(core, gitea=None, intake=None, config=None):
def test_merge_webhook_one_debit_at_tier_rate_with_receipt(core):
gitea = FakeGitea()
intake = FakeIntake()
client_id = make_client(core, tier="gold") # gold rate = 500 ($5.00)
client_id = make_client(core, tier="gold") # gold rate = 100 ($1.00)
credit(core, client_id, 10000) # $100.00
d = make_delivery(core, gitea=gitea, intake=intake)
@ -84,13 +83,13 @@ def test_merge_webhook_one_debit_at_tier_rate_with_receipt(core):
assert result["ok"] is True
assert result["overdue"] is False
assert result["rate_minor"] == 500
assert result["balance"] == 9500 # 10000 - 500
assert result["rate_minor"] == 100
assert result["balance"] == 9900 # 10000 - 100
assert result["replayed"] is False
# exactly one receipt comment, mentioning balance + delivered check
assert len(gitea.comments) == 1
assert "Delivered \u2713" in gitea.comments[0]["body"]
assert "$95.00" in gitea.comments[0]["body"]
assert "$99.00" in gitea.comments[0]["body"]
# debit recorded in ledger with the delivery idempotency key + pr ref
ok, ledger = core.list_ledger(client_id)
assert ok
@ -120,7 +119,7 @@ def test_duplicate_webhook_no_second_debit_no_second_comment(core):
assert first["ok"] is True
assert second["ok"] is True
assert second["replayed"] is True
assert second["balance"] == first["balance"] == 9500
assert second["balance"] == first["balance"] == 9900
# exactly one debit row, one comment
ok, ledger = core.list_ledger(client_id)
assert ok
@ -138,7 +137,7 @@ def test_balance_below_floor_locks_client(core):
gitea = FakeGitea()
intake = FakeIntake()
client_id = make_client(core, tier="gold")
credit(core, client_id, 550) # $5.50; after $5.00 debit -> $0.50 < $5.00 floor
credit(core, client_id, 550) # $5.50; after $1.00 debit -> $4.50 < $5.00 floor
d = make_delivery(core, gitea=gitea, intake=intake)
result = d.on_merge_webhook(client_id, REPO, PR, HEAD)
@ -147,7 +146,7 @@ def test_balance_below_floor_locks_client(core):
assert result["below_floor"] is True
assert result["locked"] is True
assert intake.locked == [client_id]
assert result["balance"] == 50
assert result["balance"] == 450
assert len(gitea.comments) == 1
@ -159,7 +158,7 @@ def test_negative_debit_does_not_block_merge_overdue_and_lock(core):
gitea = FakeGitea()
intake = FakeIntake()
client_id = make_client(core, tier="gold")
credit(core, client_id, 50) # $0.50 < $5.00 debit -> would go negative
credit(core, client_id, 50) # $0.50 < $1.00 debit -> would go negative
d = make_delivery(core, gitea=gitea, intake=intake)
result = d.on_merge_webhook(client_id, REPO, PR, HEAD)
@ -195,12 +194,10 @@ def test_webhook_without_auth_401(core):
result, status = d.handle_merge_webhook(payload, provided_key=None)
assert status == 401
# Canonical 401 body (issue #24), byte-identical with every front-end.
assert result == UNAUTHORIZED_BODY
assert result["error"]["code"] == "UNAUTHORIZED"
result, status = d.handle_merge_webhook(payload, provided_key="wrong")
assert status == 401
assert result == UNAUTHORIZED_BODY
def test_webhook_with_auth_ok(core):
@ -214,7 +211,7 @@ def test_webhook_with_auth_ok(core):
result, status = d.handle_merge_webhook(payload, provided_key="sekret")
assert status == 200
assert result["ok"] is True
assert result["balance"] == 9500
assert result["balance"] == 9900
assert len(gitea.comments) == 1
@ -247,7 +244,7 @@ def test_webhook_tolerant_gitea_shape(core):
result, status = d.handle_merge_webhook(payload, provided_key="sekret")
assert status == 200
assert result["ok"] is True
assert result["balance"] == 9500
assert result["balance"] == 9900
assert len(gitea.comments) == 1
@ -257,7 +254,7 @@ def test_webhook_tolerant_gitea_shape(core):
def test_rate_from_tier_config_override(core):
gitea = FakeGitea()
client_id = make_client(core, tier="standard") # standard default 500
client_id = make_client(core, tier="standard") # standard default 25
credit(core, client_id, 10000)
config = default_config({"SAAS_TIERS_JSON": '{"standard": {"rate_minor": 77}}'})
d = make_delivery(core, gitea=gitea, config=config)

616
tests/test_gitea_client.py
View File

@ -1,616 +0,0 @@
"""Hermetic tests for the production Gitea client (issue #28).
A local threaded HTTP stub on ``127.0.0.1`` stands in for the Gitea
instance: requests are scripted per route and every request is recorded
(method, path, query, JSON body, Authorization header) so tests can assert
exactly what the client sends — methods, paths, request bodies, pagination,
normalized results, and representative failure responses. The live Gitea
instance is never contacted.
"""
import inspect
import json
import re
import sys
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlsplit
import pytest
# PEP 420 namespace package: make the project root importable regardless of
# how pytest was invoked.
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from gitea_saas.gitea_client import ( # noqa: E402
GiteaApiError,
GiteaClient,
GiteaClientError,
_PER_PAGE,
)
TEST_TOKEN = "test-token-123-abc-secret"
TEST_OWNER = "pbwraith"
def gitea_pr(number, head_sha="sha-head", base_ref="main",
labels=("in-review",), body="Closes #1\nSome PR"):
"""A raw Gitea PR record as the list/get endpoints return it."""
return {
"number": number,
"title": f"PR {number}",
"state": "open",
"body": body,
"head": {"ref": f"issue-{number}", "sha": head_sha},
"base": {"ref": base_ref, "sha": "sha-base"},
"labels": [{"id": i, "name": name} for i, name in enumerate(labels, 1)],
}
class StubServer:
"""Minimal scripted Gitea HTTP stub (threaded, localhost only)."""
def __init__(self, dispatcher):
self.requests = [] # [{method, path, query, body, auth}]
self.dispatcher = dispatcher # (method, path, query, body) -> (status, payload|None)
self.httpd = ThreadingHTTPServer(("127.0.0.1", 0), self._handler())
self.port = self.httpd.server_address[1]
self.thread = threading.Thread(target=self.httpd.serve_forever, daemon=True)
self.thread.start()
def _handler(self):
server = self
class Handler(BaseHTTPRequestHandler):
def log_message(self, format, *args): # noqa: A002 - keep test output clean
pass
def _handle(self, method):
parsed = urlsplit(self.path)
length = int(self.headers.get("Content-Length") or 0)
raw_body = self.rfile.read(length) if length else b""
record = {
"method": method,
"path": parsed.path,
"query": parsed.query,
"body": raw_body.decode("utf-8", "replace") if raw_body else None,
"auth": self.headers.get("Authorization"),
}
server.requests.append(record)
status, payload = server.dispatcher(method, parsed.path,
parsed.query, record["body"])
body = b""
if payload is not None:
body = payload if isinstance(payload, bytes) else json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
if body:
self.wfile.write(body)
def do_GET(self):
self._handle("GET")
def do_POST(self):
self._handle("POST")
def do_PATCH(self):
self._handle("PATCH")
def do_DELETE(self):
self._handle("DELETE")
return Handler
def close(self):
self.httpd.shutdown()
self.httpd.server_close()
@pytest.fixture
def make_client():
"""Build a GiteaClient wired to a fresh scripted stub; teardown shuts it down."""
stubs = []
def build(dispatcher, token=TEST_TOKEN, owner=TEST_OWNER):
stub = StubServer(dispatcher)
stubs.append(stub)
client = GiteaClient(base_url=f"http://127.0.0.1:{stub.port}",
token=token, owner=owner)
return client, stub
yield build
for stub in stubs:
stub.close()
def requests_for(stub, method=None, path_suffix=None):
out = []
for r in stub.requests:
if method is not None and r["method"] != method:
continue
if path_suffix is not None and not r["path"].endswith(path_suffix):
continue
out.append(r)
return out
# ---------------------------------------------------------------------------
# list_open_prs: pagination + normalization
# ---------------------------------------------------------------------------
def test_list_open_prs_follows_pagination_and_normalizes(make_client):
full_page = [gitea_pr(n, labels=("in-review", "p1") if n == 1 else ("in-review",))
for n in range(1, _PER_PAGE + 1)]
def dispatcher(method, path, query, body):
if path.endswith("/api/v1/repos/pbwraith/acme/pulls") and query.startswith("state=open&page=1"):
return 200, full_page
if path.endswith("/api/v1/repos/pbwraith/acme/pulls") and query.startswith("state=open&page=2"):
return 200, [gitea_pr(_PER_PAGE + 1)]
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
result = client.list_open_prs("acme")
assert [p["number"] for p in result] == list(range(1, _PER_PAGE + 2))
assert result[0]["labels"] == ["in-review", "p1"]
for pr in result:
assert pr["head_sha"] == "sha-head"
assert pr["base_ref"] == "main"
assert pr["repo"] == "acme"
assert pr["body"] == "Closes #1\nSome PR"
assert isinstance(pr["labels"], list)
# Only open PRs were requested, at limit=50, and pagination stopped at
# the first short page (page=2 returned 1 < 50 items -> no page=3).
pulls = requests_for(stub, method="GET", path_suffix="/pulls")
assert pulls, "expected GET /pulls requests"
assert all("state=open" in r["query"] for r in pulls)
assert all("limit=50" in r["query"] for r in pulls)
assert any("page=1" in r["query"] for r in pulls)
assert any("page=2" in r["query"] for r in pulls)
assert not any("page=3" in r["query"] for r in pulls)
def test_list_open_prs_empty_repo(make_client):
def dispatcher(method, path, query, body):
return 200, []
client, _ = make_client(dispatcher)
assert client.list_open_prs("empty-repo") == []
def test_pagination_that_never_terminates_raises(make_client):
def dispatcher(method, path, query, body):
# A full page on every request: without the defensive page cap the
# client would loop forever.
return 200, [gitea_pr(n) for n in range(1, _PER_PAGE + 1)]
client, _ = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="pagination did not terminate"):
client.list_open_prs("pathological")
def test_get_pr_normalized_shape_with_changed_files(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/4"):
return 200, gitea_pr(4, labels=("in-review", "p1"))
if method == "GET" and path.endswith("/pulls/4/files"):
return 200, [{"filename": "gitea_saas/gitea_client.py"},
{"filename": "tests/test_gitea_client.py"}]
return 404, {"error": "unexpected route"}
client, _ = make_client(dispatcher)
pr = client.get_pr("acme", 4)
assert pr["number"] == 4
assert pr["body"] == "Closes #1\nSome PR"
assert pr["head_sha"] == "sha-head"
assert pr["base_sha"] == "sha-base"
assert pr["base_ref"] == "main"
assert pr["labels"] == ["in-review", "p1"]
assert pr["repo"] == "acme"
assert pr["changed_files"] == ["gitea_saas/gitea_client.py",
"tests/test_gitea_client.py"]
def test_list_open_prs_authorization_header_sent(make_client):
def dispatcher(method, path, query, body):
return 200, []
client, stub = make_client(dispatcher)
client.list_open_prs("acme")
assert requests_for(stub, method="GET", path_suffix="/pulls")[0]["auth"] == \
f"token {TEST_TOKEN}"
# ---------------------------------------------------------------------------
# add_label / remove_label
# ---------------------------------------------------------------------------
def test_add_label_posts_label_id_and_verifies(make_client):
calls = {"issue_gets": 0}
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
if method == "GET" and path.endswith("/issues/5"):
calls["issue_gets"] += 1
if calls["issue_gets"] == 1: # before
return 200, {"number": 5, "labels": [{"id": 8, "name": "in-review"}]}
return 200, {"number": 5, "labels": [{"id": 8, "name": "in-review"},
{"id": 7, "name": "bounced"}]}
if method == "POST" and path.endswith("/issues/5/labels"):
return 200, [{"id": 8, "name": "in-review"}, {"id": 7, "name": "bounced"}]
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
client.add_label("acme", 5, "bounced")
posts = requests_for(stub, method="POST", path_suffix="/issues/5/labels")
assert len(posts) == 1
assert json.loads(posts[0]["body"]) == {"labels": [7]} # label ID, not name
assert calls["issue_gets"] == 2 # verification re-GET happened
def test_add_label_already_present_is_noop(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
if method == "GET" and path.endswith("/issues/5"):
return 200, {"number": 5, "labels": [{"id": 7, "name": "bounced"}]}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
client.add_label("acme", 5, "bounced")
assert not requests_for(stub, method="POST", path_suffix="/labels")
def test_add_label_unknown_label_raises_and_skips_post(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="unknown label 'nope'"):
client.add_label("acme", 5, "nope")
assert not requests_for(stub, method="POST", path_suffix="/labels")
def test_add_label_verification_failure_raises(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
if method == "GET" and path.endswith("/issues/5"):
return 200, {"number": 5, "labels": []} # POST is never reflected
if method == "POST" and path.endswith("/issues/5/labels"):
return 200, [{"id": 7, "name": "bounced"}]
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="missing on issue 5 after POST"):
client.add_label("acme", 5, "bounced")
def test_remove_label_deletes_label_id_and_verifies(make_client):
calls = {"issue_gets": 0}
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}, {"id": 8, "name": "in-review"}]
if method == "GET" and path.endswith("/issues/5"):
calls["issue_gets"] += 1
if calls["issue_gets"] == 1: # before
return 200, {"number": 5, "labels": [{"id": 7, "name": "bounced"},
{"id": 8, "name": "in-review"}]}
return 200, {"number": 5, "labels": [{"id": 8, "name": "in-review"}]}
if method == "DELETE" and path.endswith("/issues/5/labels/7"):
return 204, None
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
client.remove_label("acme", 5, "bounced")
deletes = requests_for(stub, method="DELETE", path_suffix="/issues/5/labels/7")
assert len(deletes) == 1 # resolved the label name to id 7
assert calls["issue_gets"] == 2 # verification re-GET happened
def test_remove_label_already_absent_is_noop(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
if method == "GET" and path.endswith("/issues/5"):
return 200, {"number": 5, "labels": []}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
client.remove_label("acme", 5, "bounced")
assert not [r for r in stub.requests if r["method"] == "DELETE"]
def test_remove_label_unknown_label_raises(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="unknown label 'nope'"):
client.remove_label("acme", 5, "nope")
assert not [r for r in stub.requests if r["method"] == "DELETE"]
def test_remove_label_verification_failure_raises(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/labels"):
return 200, [{"id": 7, "name": "bounced"}]
if method == "GET" and path.endswith("/issues/5"):
return 200, {"number": 5, "labels": [{"id": 7, "name": "bounced"}]}
if method == "DELETE" and path.endswith("/issues/5/labels/7"):
return 204, None # DELETE succeeds but the label stays
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="still on issue 5 after DELETE"):
client.remove_label("acme", 5, "bounced")
# ---------------------------------------------------------------------------
# merge_pr
# ---------------------------------------------------------------------------
def test_merge_pr_posts_merge_and_confirms(make_client):
state = {"gets": 0}
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
state["gets"] += 1
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
return 200, {**gitea_pr(7, head_sha="sha-head"), "merged": merged}
if method == "POST" and path.endswith("/pulls/7/merge"):
return 200, {"merged": True, "message": "Pull Request was successfully merged"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
assert client.merge_pr("acme", 7) is True
posts = requests_for(stub, method="POST", path_suffix="/pulls/7/merge")
assert len(posts) == 1
# The real request must carry a supported merge mode plus the reviewed
# head commit id — the Gitea schema rejects `{}` and an unpinned merge.
assert json.loads(posts[0]["body"]) == {"Do": "merge", "head_commit_id": "sha-head"}
def test_merge_pr_sends_reviewed_head_commit_id_regression(make_client):
# Regression (review bounce): the merge payload must pin the exact head
# commit id read from the PR, so a branch that moved after review cannot
# be merged accidentally; `Do` must be a schema-supported merge mode.
reviewed_sha = "abcd-1234-reviewed-head"
state = {"gets": 0}
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
state["gets"] += 1
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
return 200, {**gitea_pr(7, head_sha=reviewed_sha), "merged": merged}
if method == "POST" and path.endswith("/pulls/7/merge"):
return 200, {"merged": True, "message": "merged"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
assert client.merge_pr("acme", 7) is True
posts = requests_for(stub, method="POST", path_suffix="/pulls/7/merge")
assert len(posts) == 1
payload = json.loads(posts[0]["body"])
assert payload == {"Do": "merge", "head_commit_id": reviewed_sha}
assert payload["Do"] in {"merge", "rebase", "rebase-merge", "squash",
"fast-forward-only"}
def test_merge_pr_raises_when_pr_has_no_head_commit_id(make_client):
# Without a head commit id the merge cannot be pinned to the reviewed
# head: fail closed instead of sending an unpinned (or empty) request.
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
pr = gitea_pr(7)
pr["head"] = {"ref": "issue-7"} # sha missing
return 200, pr
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="no head commit id"):
client.merge_pr("acme", 7)
assert not requests_for(stub, method="POST", path_suffix="/merge")
def test_merge_pr_refuses_moved_head_before_post(make_client):
# TOCTOU regression (governor review): the branch moved from the
# reviewed SHA A to SHA B between review and merge. Passing the
# reviewed SHA must fail closed BEFORE any merge POST — no unstably
# merged, unreviewed SHA B.
reviewed_sha = "reviewed-sha-AAAA"
moved_sha = "moved-sha-BBBB"
state = {"gets": 0}
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
state["gets"] += 1
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
return 200, {**gitea_pr(7, head_sha=moved_sha), "merged": merged}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="moved after review"):
client.merge_pr("acme", 7, expected_head_sha=reviewed_sha)
# Fail closed: the merge endpoint was never called, so unreviewed
# SHA B cannot be merged.
assert not requests_for(stub, method="POST", path_suffix="/merge")
assert "reviewed-sha-AAAA" not in [
json.loads(r["body"]).get("head_commit_id")
for r in stub.requests if r["method"] == "POST"]
def test_merge_pr_pins_expected_head_in_payload(make_client):
# The reviewed SHA passed by the caller is what gets sent as
# head_commit_id (not whatever the pre-merge GET happens to report).
reviewed_sha = "reviewed-sha-CCCC"
state = {"gets": 0}
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
state["gets"] += 1
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
return 200, {**gitea_pr(7, head_sha=reviewed_sha), "merged": merged}
if method == "POST" and path.endswith("/pulls/7/merge"):
return 200, {"merged": True, "message": "merged"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
assert client.merge_pr("acme", 7, expected_head_sha=reviewed_sha) is True
posts = requests_for(stub, method="POST", path_suffix="/pulls/7/merge")
assert len(posts) == 1
assert json.loads(posts[0]["body"]) == {
"Do": "merge", "head_commit_id": reviewed_sha}
def test_merge_pr_already_merged_short_circuits(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
return 200, {**gitea_pr(7), "merged": True}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
assert client.merge_pr("acme", 7) is True
assert not requests_for(stub, method="POST", path_suffix="/merge")
def test_merge_pr_returns_false_when_post_condition_unmet(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
return 200, {**gitea_pr(7), "merged": False}
if method == "POST" and path.endswith("/pulls/7/merge"):
return 200, {"merged": False, "message": "not merged yet"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
assert client.merge_pr("acme", 7) is False
assert len(requests_for(stub, method="POST", path_suffix="/merge")) == 1
def test_merge_pr_conflict_raises(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/pulls/7"):
return 200, {**gitea_pr(7), "merged": False}
if method == "POST" and path.endswith("/pulls/7/merge"):
return 409, {"message": "conflict: merge is blocked"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaApiError) as excinfo:
client.merge_pr("acme", 7)
assert excinfo.value.status == 409
assert "merge is blocked" in str(excinfo.value)
# ---------------------------------------------------------------------------
# close_issue
# ---------------------------------------------------------------------------
def test_close_issue_patches_state_and_confirms(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/issues/9"):
return 200, {"number": 9, "state": "closed"}
if method == "PATCH" and path.endswith("/issues/9"):
return 200, {"number": 9, "state": "closed"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
client.close_issue("acme", 9)
patches = requests_for(stub, method="PATCH", path_suffix="/issues/9")
assert len(patches) == 1
assert json.loads(patches[0]["body"]) == {"state": "closed"}
def test_close_issue_verification_failure_raises(make_client):
def dispatcher(method, path, query, body):
if method == "GET" and path.endswith("/issues/9"):
return 200, {"number": 9, "state": "open"} # never actually closed
if method == "PATCH" and path.endswith("/issues/9"):
return 200, {"number": 9, "state": "closed"}
return 404, {"error": "unexpected route"}
client, stub = make_client(dispatcher)
with pytest.raises(GiteaClientError, match="state is 'open' after close request"):
client.close_issue("acme", 9)
# ---------------------------------------------------------------------------
# Non-2xx handling: useful error, token never leaks
# ---------------------------------------------------------------------------
def test_non_2xx_raises_useful_error_without_leaking_token(make_client):
def dispatcher(method, path, query, body):
# Simulate a hostile server that echoes the request back, including
# the Authorization header — the client must redact the token.
return 500, {"error": "boom", "echo": f"Authorization: token {TEST_TOKEN}"}
client, _ = make_client(dispatcher)
with pytest.raises(GiteaApiError) as excinfo:
client.list_labels("acme")
message = str(excinfo.value)
assert excinfo.value.status == 500
assert excinfo.value.method == "GET"
assert "boom" in message
assert TEST_TOKEN not in message
assert "Authorization" not in message
assert "token" not in message.lower() or TEST_TOKEN not in message
def test_non_2xx_message_carries_path_and_status(make_client):
def dispatcher(method, path, query, body):
return 422, {"message": "invalid pull request"}
client, _ = make_client(dispatcher)
with pytest.raises(GiteaApiError) as excinfo:
client.get_pr("acme", 3)
assert "HTTP 422" in str(excinfo.value)
assert "pulls/3" in str(excinfo.value)
# ---------------------------------------------------------------------------
# Contract: the production client exposes every operation ReviewPolicy needs
# ---------------------------------------------------------------------------
def test_production_client_exposes_review_policy_contract():
"""Issue #28 acceptance: the client exposes every seam the policy calls."""
from gitea_saas.review_policy import ReviewPolicy
from gitea_saas.reviewer import Reviewer
def gitea_ops(cls):
# ``self.gitea.<op>(`` in ReviewPolicy / ``self.repo_client.<op>(``
# in Reviewer — every attribute access on the seam, derived from the
# source so the contract stays honest as the policy evolves.
source = inspect.getsource(cls)
prefix = "self.gitea." if cls is ReviewPolicy else "self.repo_client."
ops = set(re.findall(re.escape(prefix) + r"(\w+)\(", source))
# ``merge_pr`` is also invoked through the shared debit helper; keep
# the set stable by including explicitly required operations.
return ops
required = gitea_ops(ReviewPolicy) | gitea_ops(Reviewer)
# Explicit core of the issue's scope (ReviewPolicy's seam list):
required |= {"list_open_prs", "get_pr", "get_issue", "post_comment",
"add_label", "remove_label", "merge_pr", "close_issue"}
missing = sorted(op for op in required if not callable(getattr(GiteaClient, op, None)))
assert not missing, f"production client is missing ReviewPolicy operations: {missing}"

408
tests/test_intake_eval.py
View File

@ -8,7 +8,6 @@ DB (funded vs unfunded gating is real). No network, no sleeps (a noop
Run: python3 -m pytest tests/test_intake_eval.py -q
"""
import json
import sqlite3
import sys
from pathlib import Path
@ -16,14 +15,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
import pytest
from gitea_saas.auth import UNAUTHORIZED_BODY
from gitea_saas.billing_core import BillingCore
from gitea_saas.intake_eval import (
EvalIntake,
EvalIntakeError,
EVAL_LABEL,
register_eval_routes,
)
from gitea_saas.intake_eval import EvalIntake, EvalIntakeError, EVAL_LABEL
class Config:
@ -427,402 +420,3 @@ def test_real_client_list_issue_comments_builds_correct_request(monkeypatch):
"issues/42/comments")
assert req.headers.get("Authorization") == "token tok123"
assert req.headers.get("Accept") == "application/json"
# -------------------------------------------------------------------------- #
# Issue #39: HTTP surface (register_eval_routes) #
# -------------------------------------------------------------------------- #
@pytest.fixture
def app_env(tmp_path):
"""core + MockGitea + api-keyed Config for mounting eval routes."""
core = BillingCore(tmp_path / "billing.db")
gitea = MockGitea()
cfg = Config(tmp_path / "saas.db", api_key="test-api-key",
debit_floor_minor=500)
return core, gitea, cfg
@pytest.fixture
def eval_app(app_env):
from flask import Flask
core, gitea, cfg = app_env
app = Flask(__name__)
app.config["API_KEY"] = cfg.api_key
app.config["TESTING"] = True
register_eval_routes(app, core, gitea, cfg)
return app.test_client()
def _balance(core, client_id):
ok, details = core.get_client(client_id, include_balance=True)
assert ok
return details["balance"]
def _queued_batch_rows(db_path, client_id):
conn = sqlite3.connect(str(db_path))
try:
conn.row_factory = sqlite3.Row
rows = conn.execute(
"SELECT * FROM pending_eval_batches WHERE client_id = ?"
" ORDER BY id", (client_id,)).fetchall()
return [dict(r) for r in rows]
finally:
conn.close()
class TestEvalRoutes:
"""Acceptance 1: each route — success, missing/blank key 401,
unconfigured key 503; health stays public."""
AUTHD_PATHS = [
("post", "/api/eval/batches", {"client_id": 1, "tasks": TASKS}),
("get", "/api/eval/batches/1", None),
("get", "/api/eval/batches/1/results", None),
("post", "/api/eval/batches/1/cancel", None),
]
KEY = "test-api-key"
@staticmethod
def _post(client, path, payload=None):
return client.post(path, json=payload or {},
headers={"X-API-Key": TestEvalRoutes.KEY})
@staticmethod
def _get(client, path):
return client.get(path,
headers={"X-API-Key": TestEvalRoutes.KEY})
def test_eval_health_is_public(self, eval_app):
resp = eval_app.get("/api/eval/health")
assert resp.status_code == 200
assert resp.get_json() == {"ok": True}
def test_eval_health_public_without_any_key(self, app_env):
core, gitea, cfg = app_env
from flask import Flask
app = Flask(__name__)
app.config["TESTING"] = True
register_eval_routes(app, core, gitea, cfg) # API_KEY unset
resp = app.test_client().get("/api/eval/health")
assert resp.status_code == 200
def test_missing_key_401_on_every_route(self, eval_app):
for method, path, _payload in self.AUTHD_PATHS:
resp = getattr(eval_app, method)(path, json={})
assert resp.status_code == 401, (method, path)
# canonical #24 body, byte-identical with every front-end
assert resp.get_json() == UNAUTHORIZED_BODY
def test_blank_key_401_on_every_route(self, eval_app):
for method, path, _payload in self.AUTHD_PATHS:
resp = getattr(eval_app, method)(
path, json={}, headers={"X-API-Key": ""})
assert resp.status_code == 401, (method, path)
assert resp.get_json() == UNAUTHORIZED_BODY
def test_wrong_key_401(self, eval_app):
resp = eval_app.post(
"/api/eval/batches", json={"client_id": 1, "tasks": TASKS},
headers={"X-API-Key": "wrong-key"})
assert resp.status_code == 401
assert resp.get_json() == UNAUTHORIZED_BODY
def test_unconfigured_key_503_on_every_route(self, app_env):
"""No API key configured -> authed routes fail closed 503
(issue #38's assembly acceptance demands this app-wide); the
liveness route stays reachable."""
core, gitea, cfg = app_env
cfg_no_key = Config(cfg.db_path) # no api_key attr at all
from flask import Flask
app = Flask(__name__)
app.config["TESTING"] = True
register_eval_routes(app, core, gitea, cfg_no_key)
client = app.test_client()
for method, path, payload in self.AUTHD_PATHS:
resp = getattr(client, method)(path, json=payload or {})
assert resp.status_code == 503, (method, path)
assert resp.get_json()["error"]["code"] == "SERVICE_UNAVAILABLE"
assert client.get("/api/eval/health").status_code == 200
def test_submit_funded_route_201(self, app_env, eval_app):
core, gitea, _cfg = app_env
ok, c = core.create_client("route-client")
assert ok
core.credit(c["id"], 10_000, "fund-route")
resp = self._post(eval_app, "/api/eval/batches", {
"client_id": c["id"], "tasks": TASKS})
assert resp.status_code == 201
body = resp.get_json()
assert body["ok"] is True and body["batch_id"] == 1
assert len(body["issues"]) == 2
# the batch hold is visible in the client's available balance
assert _balance(core, c["id"]) == 10_000 - 500
def test_submit_unfunded_route_202_queued(self, app_env, eval_app):
core, gitea, _cfg = app_env
ok, c = core.create_client("poor-route")
assert ok
resp = self._post(eval_app, "/api/eval/batches", {
"client_id": c["id"], "tasks": TASKS})
assert resp.status_code == 202
body = resp.get_json()
assert body["ok"] is False and body["reason"] == "unfunded"
assert body["queued"] is True and body["pending_id"] == 1
assert gitea.created_issues == [] # NO Gitea issues
assert gitea.created_repos == []
rows = _queued_batch_rows(_cfg.db_path, c["id"])
assert len(rows) == 1 and rows[0]["status"] == "queued"
def test_route_validation_400_and_404(self, eval_app, app_env):
core, gitea, _cfg = app_env
ok, c = core.create_client("val-client")
assert ok
core.credit(c["id"], 10_000, "fund-val")
bad = self._post(eval_app, "/api/eval/batches", {
"client_id": -3, "tasks": TASKS})
assert bad.status_code == 400 and bad.get_json()["code"] == "invalid_request"
bad_tasks = self._post(eval_app, "/api/eval/batches", {
"client_id": c["id"], "tasks": [{"task_id": "x"}]}) # no title
assert bad_tasks.status_code == 400
assert self._get(eval_app, "/api/eval/batches/99").status_code == 404
assert self._get(eval_app, "/api/eval/batches/99/results").status_code == 404
assert self._post(eval_app, "/api/eval/batches/99/cancel").status_code == 404
def test_get_batch_and_results_routes(self, app_env, eval_app):
core, gitea, _cfg = app_env
ok, c = core.create_client("view-client")
assert ok
core.credit(c["id"], 10_000, "fund-view")
r = self._post(eval_app, "/api/eval/batches", {
"client_id": c["id"], "tasks": TASKS}).get_json()
batch_id = r["batch_id"]
# status + per-task state
resp = self._get(eval_app, f"/api/eval/batches/{batch_id}")
assert resp.status_code == 200
body = resp.get_json()
assert body["ok"] and body["batch"]["status"] == "open"
assert [t["task_id"] for t in body["tasks"]] == ["t1", "t2"]
assert all(t["status"] == "pending" for t in body["tasks"])
# the results table reflects the drain state: close t1 in gitea,
# then run the drain step (poll) through a direct intake on the
# same DB — the results route renders the persisted table
gitea.add_comment("eval-runs", r["issues"][0]["issue_number"],
"lane report: green")
gitea.close_issue("eval-runs", r["issues"][0]["issue_number"])
intake = EvalIntake(core, gitea, _cfg, sleep_fn=noop_sleep)
assert intake.poll_completed(batch_id)["done"] == 1
results = self._get(
eval_app, f"/api/eval/batches/{batch_id}/results").get_json()
assert results["ok"] is True
by_task = {t["task_id"]: t for t in results["results"]}
assert by_task["t1"]["status"] == "done"
assert by_task["t1"]["summary"] == "lane report: green"
# t2 was never drained, so the results table reports it pending.
assert by_task["t2"]["status"] == "pending"
def test_cancel_route_releases_hold_exactly(self, app_env, eval_app):
"""Acceptance 3 (route half): balance returns to its pre-submit
value exactly after cancel."""
core, gitea, _cfg = app_env
ok, c = core.create_client("cancel-client")
assert ok
core.credit(c["id"], 10_000, "fund-cancel")
pre = _balance(core, c["id"])
r = self._post(eval_app, "/api/eval/batches", {
"client_id": c["id"], "tasks": TASKS}).get_json()
assert _balance(core, c["id"]) == pre - 500 # hold placed
resp = self._post(eval_app, f"/api/eval/batches/{r['batch_id']}/cancel")
assert resp.status_code == 200
body = resp.get_json()
assert body["ok"] is True and body["cancelled"] is True
assert body["hold_released"] is True
assert _balance(core, c["id"]) == pre # exact return
# the batch now reads as cancelled
status = self._get(
eval_app, f"/api/eval/batches/{r['batch_id']}").get_json()
assert status["cancelled"] is True
assert status["batch"]["status"] == "cancelled"
# -------------------------------------------------------------------------- #
# Issue #39: queue + flush (acceptance 2) and hold/cancel (acceptance 3) #
# -------------------------------------------------------------------------- #
class TestQueueAndFlush:
def test_unfunded_submit_queues_no_gitea(self, unfunded, tmp_path):
core, client_id = unfunded
gitea = MockGitea()
intake = make_intake(core, tmp_path, gitea)
r = intake.submit_batch(client_id, TASKS)
assert r["ok"] is False and r["reason"] == "unfunded"
assert r["queued"] is True and r["pending_id"] == 1
# half 1: NO Gitea issue, NO repo
assert gitea.created_issues == []
assert gitea.created_repos == []
# half 1: a queued row exists locally
rows = _queued_batch_rows(tmp_path / "saas.db", client_id)
assert len(rows) == 1
assert rows[0]["status"] == "queued"
assert json.loads(rows[0]["tasks_json"]) == TASKS
def test_flush_ships_queued_batch_after_funding(self, unfunded, tmp_path):
core, client_id = unfunded
gitea = MockGitea()
intake = make_intake(core, tmp_path, gitea)
r = intake.submit_batch(client_id, TASKS)
assert r["pending_id"] == 1
# funding arrives; flush ships it
core.credit(client_id, 10_000, "fund-flush")
out = intake.flush_pending(client_id)
assert out["ok"] is True and out["flushed"] == 1
assert out["batch_ids"] == [1]
assert len(out["issues"]) == 2
# half 2: issues now exist, one per task, eval-task label
assert len(gitea.created_issues) == 2
for _repo, _title, body, labels in gitea.created_issues:
assert labels == [EVAL_LABEL]
assert "batch: 1" in body
# the queued row is marked shipped; a real batch row exists
rows = _queued_batch_rows(tmp_path / "saas.db", client_id)
assert len(rows) == 1 and rows[0]["status"] == "shipped"
batch = intake.get_batch(1)
assert batch["ok"] and batch["batch"]["client_id"] == client_id
assert len(batch["tasks"]) == 2
# the shipped batch placed its hold like any fresh submit
assert _balance(core, client_id) == 10_000 - 500
def test_flush_while_still_unfunded_skips(self, unfunded, tmp_path):
core, client_id = unfunded
gitea = MockGitea()
intake = make_intake(core, tmp_path, gitea)
intake.submit_batch(client_id, TASKS)
out = intake.flush_pending(client_id)
assert out["ok"] is False and out["reason"] == "unfunded"
assert out["skipped"] == 1 and out["flushed"] == 0
assert gitea.created_issues == []
rows = _queued_batch_rows(tmp_path / "saas.db", client_id)
assert rows[0]["status"] == "queued"
def test_flush_nothing_queued_is_noop(self, funded, tmp_path):
core, client_id = funded
intake = make_intake(core, tmp_path)
out = intake.flush_pending(client_id)
assert out == {"ok": True, "flushed": 0, "batch_ids": [], "issues": []}
class TestHoldAndCancel:
def test_cancel_releases_hold_balance_exact(self, funded, tmp_path):
core, client_id = funded
intake = make_intake(core, tmp_path)
pre = _balance(core, client_id)
r = intake.submit_batch(client_id, TASKS)
assert r["hold_id"] is not None
assert _balance(core, client_id) == pre - 500 # hold visible
out = intake.cancel_batch(r["batch_id"])
assert out["ok"] is True and out["cancelled"] is True
assert out["hold_released"] is True
assert _balance(core, client_id) == pre # EXACT return
def test_cancel_idempotent_no_double_release(self, funded, tmp_path):
core, client_id = funded
intake = make_intake(core, tmp_path)
pre = _balance(core, client_id)
r = intake.submit_batch(client_id, TASKS)
assert intake.cancel_batch(r["batch_id"])["ok"] is True
assert _balance(core, client_id) == pre
again = intake.cancel_batch(r["batch_id"])
assert again["ok"] is True and again["already_cancelled"] is True
assert again["hold_released"] is False
assert _balance(core, client_id) == pre # no second release
def test_cancel_unknown_batch(self, funded, tmp_path):
core, client_id = funded
intake = make_intake(core, tmp_path)
out = intake.cancel_batch(999)
assert out == {"ok": False, "error": "unknown_batch"}
def test_cancel_stops_draining(self, funded, tmp_path):
core, client_id = funded
gitea = MockGitea()
intake = make_intake(core, tmp_path, gitea, emitter=MockEmitter())
r = intake.submit_batch(client_id, TASKS,
callback_url="https://cb.example/x")
for issue in r["issues"]:
gitea.close_issue("eval-runs", issue["issue_number"])
assert intake.cancel_batch(r["batch_id"])["ok"] is True
# polling and emitting both refuse a cancelled batch
assert intake.poll_completed(r["batch_id"]) == {
"ok": False, "error": "cancelled"}
assert intake.emit(r["batch_id"]) == {
"ok": False, "reason": "cancelled"}
# and the batch reads as cancelled
status = intake.get_batch(r["batch_id"])
assert status["cancelled"] is True
assert status["batch"]["status"] == "cancelled"
assert "cancelled_at" in status["batch"]
def test_zero_hold_disables_hold(self, funded, tmp_path):
core, client_id = funded
intake = make_intake(core, tmp_path, eval_batch_hold_minor=0)
pre = _balance(core, client_id)
r = intake.submit_batch(client_id, TASKS)
assert r.get("hold_id") is None
assert _balance(core, client_id) == pre # nothing held
out = intake.cancel_batch(r["batch_id"])
assert out["ok"] is True and out["hold_released"] is False
assert _balance(core, client_id) == pre
def test_intake_failure_releases_hold(self, funded, tmp_path):
"""A rolled-back submit must not leave money locked."""
core, client_id = funded
class FlakyGitea(MockGitea):
def create_issue(self, *a, **kw):
if self.next_number >= 1: # second issue fails
raise RuntimeError("gitea hiccup")
return super().create_issue(*a, **kw)
intake = make_intake(core, tmp_path, FlakyGitea())
pre = _balance(core, client_id)
r = intake.submit_batch(client_id, TASKS)
assert r["ok"] is False and r["reason"] == "intake_failed"
assert intake.get_batch(1)["ok"] is False # no partial batch
assert _balance(core, client_id) == pre # hold released
def test_hold_too_large_refuses_batch(self, funded, tmp_path):
"""Reserving beyond the available balance fails closed — the
batch is refused and no Gitea issue is created."""
core, client_id = funded
gitea = MockGitea()
intake = make_intake(core, tmp_path, gitea,
eval_batch_hold_minor=50_000) # > 10_000
pre = _balance(core, client_id)
r = intake.submit_batch(client_id, TASKS)
assert r["ok"] is False and r["reason"] == "intake_failed"
assert gitea.created_issues == []
assert _balance(core, client_id) == pre
def test_get_results_after_partial_drain(self, funded, tmp_path):
core, client_id = funded
gitea = MockGitea()
intake = make_intake(core, tmp_path, gitea)
r = intake.submit_batch(client_id, TASKS)
gitea.add_comment("eval-runs", r["issues"][0]["issue_number"],
"ok")
gitea.close_issue("eval-runs", r["issues"][0]["issue_number"])
intake.poll_completed(r["batch_id"])
results = intake.get_results(r["batch_id"])
assert results["ok"] is True
assert results["batch_id"] == r["batch_id"]
by_task = {t["task_id"]: t for t in results["results"]}
assert by_task["t1"]["status"] == "done"
assert by_task["t1"]["summary"] == "ok"
assert by_task["t2"]["status"] == "pending"
assert intake.get_results(999)["error"] == "unknown_batch"

19
tests/test_intake_factory.py
View File

@ -26,7 +26,6 @@ from pathlib import Path
import pytest
from gitea_saas.auth import UNAUTHORIZED_BODY
from gitea_saas.intake_factory import (
DEFAULT_DEBIT_FLOOR_MINOR,
IntakeFactory,
@ -439,26 +438,8 @@ def _post(app, path, payload=None, key="test-api-key"):
def test_routes_require_api_key(app_client):
resp = app_client.post("/api/factory/provision", json={"client_id": 1})
assert resp.status_code == 401
# Canonical 401 body (issue #24), byte-identical with every front-end.
assert resp.get_json() == UNAUTHORIZED_BODY
resp = app_client.post("/api/factory/1/unlock")
assert resp.status_code == 401
assert resp.get_json() == UNAUTHORIZED_BODY
def test_wrong_key_of_same_length_rejected(app_client):
"""Wrong key of the SAME length as the configured key must be rejected
(issue #24 criterion 3; the constant-time comparison is what makes this
a real check rather than a prefix-time leak)."""
configured = "test-api-key"
wrong_same_length = "hunter-id-123"[:len(configured)]
assert len(wrong_same_length) == len(configured)
assert wrong_same_length != configured
resp = app_client.post("/api/factory/provision",
json={"client_id": 1},
headers={"X-API-Key": wrong_same_length})
assert resp.status_code == 401
assert resp.get_json() == UNAUTHORIZED_BODY
def test_route_provision_submit_unlock_lock_flow(env, app_client):

697
tests/test_review_policy.py
View File

@ -27,19 +27,10 @@ from gitea_saas.reviewer import Verdict # noqa: E402
class FakeGitea:
"""In-memory Gitea seam recording every mutation.
Mirrors the production client's NORMALIZED shape (issue #28/#31):
PRs carry ``merged`` (bool) and ``state`` exactly like
``GiteaClient._normalize_pr`` output, ``list_open_prs`` returns only
open-unmerged PRs, ``get_pr`` returns the live record, and a
successful ``merge_pr`` flips the live record to ``merged=True`` /
``state="closed"`` — so tests exercise the production-shaped dict, not
a richer test-only one.
"""
"""In-memory Gitea seam recording every mutation."""
def __init__(self, prs=None, issues=None):
self.prs = list(prs or []) # [{number, head_sha, labels, body, merged, state}]
self.prs = list(prs or []) # [{number, head_sha, labels, body}]
self.issues = dict(issues or {}) # number -> {"body": ...}
self.comments = []
self.labels_added = []
@ -47,21 +38,12 @@ class FakeGitea:
self.merged = []
self.closed = []
self.merge_results = {} # pr -> bool (default True)
self.fail_next = {} # method -> one-shot Exception to raise
def _maybe_fail(self, name):
"""Raise a one-shot injected failure (issue #29 retry tests)."""
exc = self.fail_next.pop(name, None)
if exc is not None:
raise exc
def _pr(self, number):
return next(p for p in self.prs if p["number"] == number)
def list_open_prs(self, repo=None):
# Production behavior: ?state=open returns open, unmerged PRs only.
return [dict(p) for p in self.prs
if p.get("state") == "open" and not p.get("merged")]
return [dict(p) for p in self.prs]
def get_pr(self, repo, number):
return dict(self._pr(number))
@ -70,34 +52,19 @@ class FakeGitea:
return dict(self.issues[number])
def post_comment(self, repo, number, body):
self._maybe_fail("post_comment")
self.comments.append((repo, number, body))
def add_label(self, repo, number, label):
self._maybe_fail("add_label")
self.labels_added.append((repo, number, label))
def remove_label(self, repo, number, label):
self._maybe_fail("remove_label")
self.labels_removed.append((repo, number, label))
def merge_pr(self, repo, number, expected_head_sha=None):
self._maybe_fail("merge_pr")
live = self._pr(number)
if expected_head_sha is not None and live.get("head_sha") != expected_head_sha:
raise RuntimeError(
f"merge_pr: PR {number} head is {live.get('head_sha')!r} but the "
f"reviewed head is {expected_head_sha!r}; refusing to merge "
"a branch that moved after review")
def merge_pr(self, repo, number):
self.merged.append((repo, number))
merged = self.merge_results.get(number, True)
if merged:
live["merged"] = True
live["state"] = "closed"
return merged
return self.merge_results.get(number, True)
def close_issue(self, repo, number):
self._maybe_fail("close_issue")
self.closed.append((repo, number))
@ -108,10 +75,9 @@ class FakeReviewer:
self.verdicts = dict(verdicts)
self.calls = []
self.posted = []
self.runner = None # mirror Reviewer.runner (used to build the workspace)
def review_pr(self, repo, pr, linked_issue, workspace=None):
self.calls.append((repo, pr, linked_issue, workspace))
def review_pr(self, repo, pr, linked_issue, repo_dir=None):
self.calls.append((repo, pr, linked_issue, repo_dir))
verdict = self.verdicts[pr]
if callable(verdict):
return verdict(repo, pr)
@ -122,21 +88,15 @@ class FakeReviewer:
return ""
def make_verdict(pr_number, verdict, head_sha="sha-abc", base_sha="sha-base"):
def make_verdict(pr_number, verdict, head_sha="sha-abc"):
return Verdict(repo="gitea-saas-core", pr=pr_number, head_sha=head_sha,
base_sha=base_sha,
acceptance=["tests pass"], tests={"exit": 0, "summary": "12 passed"},
build=None, scope_violations=[], findings=[],
verdict=verdict, summary="ok")
def pr(number, head_sha="sha-abc", base_sha="sha-base", labels=("in-review",),
body="Closes #1\nSome PR"):
# Production-normalized shape (as GiteaClient._normalize_pr returns):
# merged bool + state + head_sha + base_sha, defaults open / not merged.
return {"number": number, "head_sha": head_sha, "base_sha": base_sha,
"labels": list(labels), "body": body, "merged": False,
"state": "open"}
def pr(number, head_sha="sha-abc", labels=("in-review",), body="Closes #1\nSome PR"):
return {"number": number, "head_sha": head_sha, "labels": list(labels), "body": body}
def tier_issue(tier, number=1):
@ -183,7 +143,7 @@ def policy_factory(tmp_path):
def test_review_once_per_head_and_skip_same_head(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(10, head_sha="aaa")], gitea_issues=tier_issue("gold"),
verdicts={10: make_verdict(10, APPROVE, head_sha="aaa")})
verdicts={10: make_verdict(10, APPROVE)})
first = policy.poll_and_review("gitea-saas-core")
second = policy.poll_and_review("gitea-saas-core")
@ -194,12 +154,8 @@ def test_review_once_per_head_and_skip_same_head(policy_factory):
def test_head_move_triggers_rereview(policy_factory):
# Standard tier keeps the PR OPEN after the first poll (comment only),
# so a head move is observable by the next scan and must re-review.
# (A gold-tier poll merges, and a merged PR leaves the open scan — the
# production-shaped fake reflects that.)
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(10, head_sha="aaa")], gitea_issues=tier_issue("standard"),
gitea_prs=[pr(10, head_sha="aaa")], gitea_issues=tier_issue("gold"),
verdicts={10: make_verdict(10, APPROVE)})
policy.poll_and_review("gitea-saas-core")
@ -210,73 +166,6 @@ def test_head_move_triggers_rereview(policy_factory):
assert [c[1] for c in reviewer.calls] == [10, 10]
# ---------------------------------------------------------------------------
# Evaluated-pair integrity (issue #30): refuse to apply a verdict whose
# head/base pair moved after evaluation
# ---------------------------------------------------------------------------
def test_evaluated_head_move_refuses_verdict_and_new_pair_gets_fresh_shot(policy_factory):
# The reviewer evaluated head "sha-old"; the live PR has already moved
# to "sha-new" when the verdict is to be applied. A verdict for a pair
# that no longer exists must never merge/label/comment/close — and once
# the PR settles on the evaluated head, the pair is fresh-eligible and
# the same verdict flow applies normally.
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(50, head_sha="sha-new")], gitea_issues=tier_issue("gold", number=1),
verdicts={50: make_verdict(50, APPROVE, head_sha="sha-old")})
results = policy.poll_and_review("gitea-saas-core")
r = results[0]
assert r["outcome"] == "stale"
assert r["error"]["code"] == "PAIR_MOVED"
assert r["evaluated"] == {"head_sha": "sha-old", "base_sha": "sha-base"}
assert r["live"] == {"head_sha": "sha-new", "base_sha": "sha-base"}
assert "not applied" in r["error"]["message"].lower()
# zero side effects for the stale pair
assert gitea.merged == [] and gitea.comments == []
assert gitea.labels_added == [] and gitea.labels_removed == []
assert gitea.closed == []
assert all(c[3] is None for c in reviewer.calls) # no clone configured -> no workspace
# The branch settles on the evaluated head: the pair now exists, the
# verdict applies (gold tier -> merge), nothing was recorded as complete
# for the stale attempt.
gitea.prs[0]["head_sha"] = "sha-old"
results = policy.poll_and_review("gitea-saas-core")
assert results[0]["action"] == "merged"
assert gitea.merged == [("gitea-saas-core", 50)]
def test_base_move_after_evaluation_refuses_verdict(policy_factory):
# Same head, but the BASE moved after the review evaluated the pair:
# the synthetic merge (and therefore the verdict) describes a different
# base; refusing is mandatory regardless of head being unchanged.
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(51, head_sha="sha-h", base_sha="sha-base-current")],
gitea_issues=tier_issue("gold", number=1),
verdicts={51: make_verdict(51, APPROVE, head_sha="sha-h",
base_sha="sha-base-evaluated")})
results = policy.poll_and_review("gitea-saas-core")
r = results[0]
assert r["outcome"] == "stale" and r["error"]["code"] == "PAIR_MOVED"
assert r["live"] == {"head_sha": "sha-h", "base_sha": "sha-base-current"}
assert gitea.merged == [] and gitea.comments == [] and gitea.closed == []
def test_workspace_passed_through_when_clone_configured(policy_factory):
# With REVIEW_CLONE_DIR configured, the policy builds the isolated
# workspace seam and hands it to the reviewer on every call.
ws_sentinel = object()
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(52)], gitea_issues=tier_issue("standard"),
verdicts={52: make_verdict(52, APPROVE)},
review_clone_dir="/nonexistent/clone")
policy._workspace = ws_sentinel # the fixture's clone dir does not exist
policy.poll_and_review("gitea-saas-core")
assert reviewer.calls[0][3] is ws_sentinel
def test_unlabelled_prs_skipped_unless_review_all(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(10, labels=("in-review",)), pr(11, labels=())],
@ -315,14 +204,14 @@ def test_gold_approve_merges_closes_strips_debits_once(policy_factory):
# One debit with the idempotent key repo#pr@head, at the gold rate.
# (The fixture pre-funds the client, so filter debit rows only.)
ok, balances = core.get_balance(c["id"])
assert ok and balances["balance"] == 10_000 - 500 # gold rate_minor from config
assert ok and balances["balance"] == 10_000 - 100 # gold rate_minor from config
ok, ledger = core.list_ledger(c["id"])
assert ok
debits = [e for e in ledger["entries"] if e["kind"] == "debit"]
assert len(debits) == 1
entry = debits[0]
assert entry["idempotency_key"] == "gitea-saas-core#42@sha-gold"
assert entry["amount_minor"] == 500 and entry["kind"] == "debit"
assert entry["amount_minor"] == 100 and entry["kind"] == "debit"
def test_gold_double_delivery_never_double_debits(policy_factory):
@ -346,7 +235,7 @@ def test_gold_bounce_adds_label_no_merge_no_debit(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(43, head_sha="sha-b", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={43: make_verdict(43, BOUNCE, head_sha="sha-b")})
verdicts={43: make_verdict(43, BOUNCE)})
results = policy.poll_and_review("gitea-saas-core")
@ -363,7 +252,7 @@ def test_merge_failure_leaves_pr_open_and_no_close(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(44, head_sha="sha-c", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={44: make_verdict(44, APPROVE, head_sha="sha-c")})
verdicts={44: make_verdict(44, APPROVE)})
gitea.merge_results[44] = False
results = policy.poll_and_review("gitea-saas-core")
@ -378,7 +267,7 @@ def test_debit_failure_is_fail_closed(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(45, head_sha="sha-d", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={45: make_verdict(45, APPROVE, head_sha="sha-d")},
verdicts={45: make_verdict(45, APPROVE)},
client_ids={})
results = policy.poll_and_review("gitea-saas-core")
@ -396,7 +285,7 @@ def test_standard_approve_comment_only_no_merge(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(50, head_sha="sha-e", labels=("in-review",))],
gitea_issues=tier_issue("standard", number=1),
verdicts={50: make_verdict(50, APPROVE, head_sha="sha-e")})
verdicts={50: make_verdict(50, APPROVE)})
results = policy.poll_and_review("gitea-saas-core")
@ -412,7 +301,7 @@ def test_needs_guidance_adds_label_and_comment(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(51, head_sha="sha-f", labels=("in-review",))],
gitea_issues=tier_issue("standard", number=1),
verdicts={51: make_verdict(51, NEEDS_GUIDANCE, head_sha="sha-f")})
verdicts={51: make_verdict(51, NEEDS_GUIDANCE)})
results = policy.poll_and_review("gitea-saas-core")
@ -430,7 +319,7 @@ def test_debit_rate_comes_from_tier_config(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(60, head_sha="sha-g", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={60: make_verdict(60, APPROVE, head_sha="sha-g")},
verdicts={60: make_verdict(60, APPROVE)},
tiers={**DEFAULT_TIERS, "gold": {"auto_merge": True, "rate_minor": 250}})
policy.poll_and_review("gitea-saas-core")
@ -445,7 +334,7 @@ def test_free_tier_merges_without_debit_when_auto_merge(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(61, head_sha="sha-h", labels=("in-review",))],
gitea_issues=tier_issue("free", number=1),
verdicts={61: make_verdict(61, APPROVE, head_sha="sha-h")},
verdicts={61: make_verdict(61, APPROVE)},
tiers={**DEFAULT_TIERS, "free": {"auto_merge": True, "rate_minor": 0}})
results = policy.poll_and_review("gitea-saas-core")
@ -471,15 +360,6 @@ def test_load_review_config_merges_saas_tiers_json(monkeypatch):
assert config["test_timeout_s"] == 45
def test_default_tier_rates_are_500_500_0():
# Pricing decision 2026-09-01: delivered-issue rate is $5.00 (500 minor
# units) for both gold and standard; free stays 0. Assert the literal
# values, not the constant name (issue #45 acceptance).
assert DEFAULT_TIERS["gold"]["rate_minor"] == 500
assert DEFAULT_TIERS["standard"]["rate_minor"] == 500
assert DEFAULT_TIERS["free"]["rate_minor"] == 0
# ---------------------------------------------------------------------------
# Linked issue parsing + webhook path
# ---------------------------------------------------------------------------
@ -506,538 +386,3 @@ def test_handle_merge_event_records_receipt_and_closes(policy_factory):
debits = [e for e in ledger["entries"] if e["kind"] == "debit"]
assert ok and len(debits) == 1
assert debits[0]["idempotency_key"] == "gitea-saas-core#70@sha-i"
# ---------------------------------------------------------------------------
# Transactional outcome application + retry (issue #29)
# ---------------------------------------------------------------------------
def _ledger_debits(core, client_id):
ok, ledger = core.list_ledger(client_id)
assert ok
return [e for e in ledger["entries"] if e["kind"] == "debit"]
def test_verdict_failure_reported_and_retried(policy_factory):
# Verdict generation raises once: structured failure + persisted retry
# metadata; the same unchanged head completes exactly once next poll.
calls = {"n": 0}
def flaky(repo, pr):
calls["n"] += 1
if calls["n"] == 1:
raise RuntimeError("verdict generation boom")
return make_verdict(pr, APPROVE, head_sha="sha-v")
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(80, head_sha="sha-v", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={80: flaky})
first = policy.poll_and_review("gitea-saas-core")
assert len(first) == 1 and first[0]["pr"] == 80
assert first[0]["outcome"] == "failed"
assert first[0]["error"] == {"type": "RuntimeError",
"message": "verdict generation boom"}
assert first[0]["attempts"] == 1
assert gitea.merged == [] and gitea.closed == [] and gitea.comments == []
# Persisted: a failed attempt exists with its error, and the head is
# NOT completed — the same unchanged head stays eligible for retry.
rows = policy._reviews_conn.execute(
"SELECT attempts, last_error FROM review_state"
" WHERE repo = 'gitea-saas-core' AND pr = 80 AND head_sha = 'sha-v'").fetchall()
assert len(rows) == 1 and rows[0][0] == 1
assert "verdict generation boom" in rows[0][1]
assert policy._already_reviewed("gitea-saas-core", 80, "sha-v") is False
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
assert gitea.merged == [("gitea-saas-core", 80)]
assert gitea.closed == [("gitea-saas-core", 1)]
assert len(_ledger_debits(core, c["id"])) == 1
# Completed now: an exact third poll skips the head.
assert policy.poll_and_review("gitea-saas-core") == []
assert calls["n"] == 2
def test_merge_boundary_failure_retried_merged_once(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(81, head_sha="sha-m", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={81: make_verdict(81, APPROVE, head_sha="sha-m")})
gitea.fail_next["merge_pr"] = RuntimeError("merge boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert first[0]["error"]["message"] == "merge boom"
assert gitea.merged == [] and gitea.closed == []
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
assert gitea.merged == [("gitea-saas-core", 81)] # merged exactly once
assert gitea.closed == [("gitea-saas-core", 1)]
assert len(_ledger_debits(core, c["id"])) == 1
assert policy.poll_and_review("gitea-saas-core") == []
def test_close_boundary_failure_no_double_merge_or_close(policy_factory):
# Failure AFTER the merge side effect: the retry must skip the already
# applied merge (checkpoint) and close the issue exactly once.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(82, head_sha="sha-c", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={82: make_verdict(82, APPROVE, head_sha="sha-c")})
gitea.fail_next["close_issue"] = RuntimeError("close boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert first[0]["error"]["message"] == "close boom"
assert gitea.merged == [("gitea-saas-core", 82)] # merge already applied
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
assert gitea.merged == [("gitea-saas-core", 82)] # NO second merge
assert gitea.closed == [("gitea-saas-core", 1)] # exactly one close
assert len(_ledger_debits(core, c["id"])) == 1
assert policy.poll_and_review("gitea-saas-core") == []
def test_debit_boundary_failure_never_double_debits(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(83, head_sha="sha-d", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={83: make_verdict(83, APPROVE, head_sha="sha-d")})
real_debit = core.debit
calls = {"n": 0}
def flaky_debit(*args, **kwargs):
calls["n"] += 1
if calls["n"] == 1:
raise RuntimeError("debit boom")
return real_debit(*args, **kwargs)
core.debit = flaky_debit
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert first[0]["error"]["message"] == "debit boom"
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
assert gitea.merged == [("gitea-saas-core", 83)] # NO second merge
assert gitea.closed == [("gitea-saas-core", 1)]
assert len(_ledger_debits(core, c["id"])) == 1 # exactly one debit
assert calls["n"] == 2
def test_label_boundary_failure_bounce_applied_once(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(84, head_sha="sha-l", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={84: make_verdict(84, BOUNCE, head_sha="sha-l")})
gitea.fail_next["add_label"] = RuntimeError("label boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert first[0]["error"]["message"] == "label boom"
assert gitea.labels_added == []
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "bounced"
assert gitea.labels_added == [("gitea-saas-core", 84, "bounced")]
assert len(reviewer.posted) == 1
assert policy.poll_and_review("gitea-saas-core") == []
def test_comment_boundary_failure_retried(policy_factory):
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(85, head_sha="sha-co", labels=("in-review",))],
gitea_issues=tier_issue("standard", number=1),
verdicts={85: make_verdict(85, APPROVE, head_sha="sha-co")})
gitea.fail_next["post_comment"] = RuntimeError("comment boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert gitea.comments == []
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "approved_awaiting_merge"
assert any("awaiting operator merge" in body for _, _, body in gitea.comments)
assert policy.poll_and_review("gitea-saas-core") == []
def test_remove_label_boundary_failure_retried(policy_factory):
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(86, head_sha="sha-rl", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={86: make_verdict(86, APPROVE, head_sha="sha-rl")})
gitea.fail_next["remove_label"] = RuntimeError("label boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert gitea.merged == [("gitea-saas-core", 86)] # merge already applied
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
assert gitea.merged == [("gitea-saas-core", 86)] # NO second merge
assert gitea.closed == [("gitea-saas-core", 1)] # close exactly once
assert len(_ledger_debits(core, c["id"])) == 1
def test_merge_refused_is_retryable_not_completed(policy_factory):
# A merge returning False is a non-completion, not a completed outcome:
# the head must stay eligible for retry until the action succeeds.
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(87, head_sha="sha-mf", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={87: make_verdict(87, APPROVE, head_sha="sha-mf")})
gitea.merge_results[87] = False
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["action"] == "merge_failed"
assert first[0]["outcome"] == "failed"
assert first[0]["error"] == {"code": "merge_failed"}
assert gitea.closed == []
assert policy._already_reviewed("gitea-saas-core", 87, "sha-mf") is False
gitea.merge_results[87] = True
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
# The refused attempt had no side effect, so the retry re-attempted and
# succeeded. The fake records every merge ATTEMPT (like the API call),
# so two attempts are visible; the action itself completed exactly once.
# (The no-double-merge guarantee after a SUCCEEDED merge + later failure
# is asserted by the close/debit/remove-label boundary tests above.)
assert gitea.merged == [("gitea-saas-core", 87), ("gitea-saas-core", 87)]
assert gitea.closed == [("gitea-saas-core", 1)]
def test_webhook_after_completed_poll_does_not_double_close(policy_factory):
# The poll path already applied the outcome; the merge-event webhook
# redelivery must replay the debit (idempotency key) but not close the
# linked issue a second time.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(88, head_sha="sha-wh", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={88: make_verdict(88, APPROVE, head_sha="sha-wh")})
policy.poll_and_review("gitea-saas-core")
assert gitea.closed == [("gitea-saas-core", 1)]
ok, details = policy.handle_merge_event("gitea-saas-core", 88)
assert ok and details["debit"]["debit"] == "recorded"
assert details["debit"].get("replayed") is True
assert gitea.closed == [("gitea-saas-core", 1)] # closed exactly once
assert len(_ledger_debits(core, c["id"])) == 1 # debited exactly once
def test_legacy_review_row_still_skips_head(policy_factory):
# Rows written under the pre-#29 schema (no retry metadata) keep their
# completed meaning: the head is skipped and never re-reviewed.
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(89, head_sha="sha-legacy", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={89: make_verdict(89, APPROVE)})
policy._reviews_conn.execute(
"INSERT INTO reviews (repo, pr, head_sha, verdict, reviewed_at)"
" VALUES (?, ?, ?, ?, ?)",
("gitea-saas-core", 89, "sha-legacy", APPROVE, "2026-01-01T00:00:00+00:00"))
policy._reviews_conn.commit()
assert policy.poll_and_review("gitea-saas-core") == []
assert reviewer.calls == [] and gitea.merged == []
def test_failed_attempt_then_head_move_reviews_new_head(policy_factory):
# The verdict follows the LIVE head (as the real reviewer's does), so a
# failed gold attempt retries at the same head, and after the head moves
# the NEW pair is reviewed and merged fresh.
def live_head_verdict(repo, pr):
return make_verdict(pr, APPROVE,
head_sha=gitea.prs[0]["head_sha"])
policy, gitea, reviewer, _, _ = policy_factory(
gitea_prs=[pr(90, head_sha="sha-a", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={90: live_head_verdict})
gitea.fail_next["merge_pr"] = RuntimeError("merge boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
gitea.prs[0]["head_sha"] = "sha-b" # head moved before the retry
second = policy.poll_and_review("gitea-saas-core")
assert second[0]["action"] == "merged"
assert policy._already_reviewed("gitea-saas-core", 90, "sha-a") is False
assert policy._already_reviewed("gitea-saas-core", 90, "sha-b") is True
# ---------------------------------------------------------------------------
# Crash-window recovery independent of the open-PR scan (issue #29 review
# bounce): once a merge truly succeeds the PR is no longer returned by
# list_open_prs(), so the persisted retry row must be finished from the
# review_state table alone, and replay must never duplicate an
# externally-visible side effect (label removal, close, success comment).
# ---------------------------------------------------------------------------
def test_merged_pr_gone_from_open_scan_recovered_after_post_merge_failure(policy_factory):
# The reviewer's exact scenario: merge succeeds, a post-merge step
# (label removal) fails, and the merged PR disappears from the open-PR
# scan before the next retry. The persisted row must still be finished
# — exactly one merge, one debit, one label removal, one close, one
# success comment.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(91, head_sha="sha-91", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={91: make_verdict(91, APPROVE, head_sha="sha-91")})
gitea.fail_next["remove_label"] = RuntimeError("label boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert first[0]["error"]["message"] == "label boom"
assert gitea.merged == [("gitea-saas-core", 91)] # merge applied
# The merged PR is gone from list_open_prs() — exactly what real Gitea
# does once a merge succeeds — so the open scan can never reach it.
gitea.prs = [p for p in gitea.prs if p["number"] != 91]
assert gitea.list_open_prs() == []
second = policy.poll_and_review("gitea-saas-core")
assert second and second[0]["action"] == "merged"
assert second[0].get("recovered") is True # via recovery
assert gitea.merged == [("gitea-saas-core", 91)] # NO second merge
assert gitea.labels_removed.count(("gitea-saas-core", 91, "in-review")) == 1
assert gitea.closed == [("gitea-saas-core", 1)] # closed exactly once
comments = [b for _, _, b in gitea.comments]
assert sum("Merged by review-gate" in b for b in comments) == 1
assert len(_ledger_debits(core, c["id"])) == 1 # debited exactly once
assert policy._already_reviewed("gitea-saas-core", 91, "sha-91") is True
assert policy.poll_and_review("gitea-saas-core") == [] # completed: skipped
def test_crash_after_label_removed_no_duplicate_visible_actions(policy_factory):
# remove_label succeeded, then a later step (the success comment)
# failed: the retry must not remove the label again, close the issue
# again, or re-post the comment — even though the PR is gone from the
# open scan.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(92, head_sha="sha-92", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={92: make_verdict(92, APPROVE, head_sha="sha-92")})
gitea.fail_next["post_comment"] = RuntimeError("comment boom")
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "failed"
assert first[0]["error"]["message"] == "comment boom"
# merge, debit, label removal and close all succeeded before the crash
assert gitea.merged == [("gitea-saas-core", 92)]
assert gitea.labels_removed.count(("gitea-saas-core", 92, "in-review")) == 1
assert gitea.closed == [("gitea-saas-core", 1)]
gitea.prs = [p for p in gitea.prs if p["number"] != 92]
second = policy.poll_and_review("gitea-saas-core")
assert second[0].get("recovered") is True
comments = [b for _, _, b in gitea.comments]
assert sum("Merged by review-gate" in b for b in comments) == 1 # once
assert gitea.labels_removed.count(("gitea-saas-core", 92, "in-review")) == 1
assert gitea.closed == [("gitea-saas-core", 1)]
assert len(_ledger_debits(core, c["id"])) == 1
assert policy.poll_and_review("gitea-saas-core") == []
def test_crash_after_success_comment_before_completion_record(policy_factory):
# The success comment posted, then the process crashed before the
# completion row was written: replay must finish the head WITHOUT
# re-posting the comment or repeating any other side effect.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(93, head_sha="sha-93", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={93: make_verdict(93, APPROVE, head_sha="sha-93")})
real_record = policy._record_completed
def crash_after_comment(*args, **kwargs):
raise RuntimeError("process killed after success comment")
policy._record_completed = crash_after_comment
with pytest.raises(RuntimeError): # the simulated crash
policy.poll_and_review("gitea-saas-core")
policy._record_completed = real_record
comments = [b for _, _, b in gitea.comments]
assert sum("Merged by review-gate" in b for b in comments) == 1
assert gitea.merged == [("gitea-saas-core", 93)]
assert gitea.closed == [("gitea-saas-core", 1)]
assert gitea.labels_removed.count(("gitea-saas-core", 93, "in-review")) == 1
gitea.prs = [p for p in gitea.prs if p["number"] != 93]
second = policy.poll_and_review("gitea-saas-core")
assert second and second[0].get("recovered") is True
comments = [b for _, _, b in gitea.comments]
assert sum("Merged by review-gate" in b for b in comments) == 1 # not duplicated
assert gitea.labels_removed.count(("gitea-saas-core", 93, "in-review")) == 1
assert gitea.closed == [("gitea-saas-core", 1)]
assert len(_ledger_debits(core, c["id"])) == 1
assert policy.poll_and_review("gitea-saas-core") == []
# ---------------------------------------------------------------------------
# Governor review #2 regressions:
# 1. the merge is pinned to the REVIEWED head (expected_head_sha) — a branch
# that moved after review is refused before any merge POST;
# 2. recovery distinguishes a merged PR from a merely closed one using the
# PRODUCTION normalized shape (positive merged is True only);
# 3. a closed-but-unmerged PR is never adopted as merged (no debit, no label
# removal, no issue close, no success comment).
# ---------------------------------------------------------------------------
def test_reviewed_head_pinned_so_moved_branch_is_never_merged(policy_factory):
# Integration regression: review evaluates SHA A, the branch moves to
# SHA B before the verdict is applied. The evaluated-pair gate (issue
# #30) refuses BEFORE any policy action — no merge POST, no debit, no
# close, no success comment. Once the branch sits at the SHAs the
# verdict describes, the pair is fresh-eligible and merges normally.
state = {"moved": False}
def review_then_move_head(repo, pr):
evaluated = gitea.prs[0]["head_sha"] # sha-A at review time
if not state["moved"]:
state["moved"] = True
gitea.prs[0]["head_sha"] = "sha-B" # branch moved after review
return make_verdict(pr, APPROVE, head_sha=evaluated)
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(94, head_sha="sha-A", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={94: review_then_move_head})
first = policy.poll_and_review("gitea-saas-core")
assert first[0]["outcome"] == "stale"
assert first[0]["error"]["code"] == "PAIR_MOVED"
assert "sha-A" in first[0]["error"]["message"] and "sha-B" in first[0]["error"]["message"]
assert gitea.merged == [] # NO merge POST happened
assert gitea.closed == []
assert not any("Merged by review-gate" in b for _, _, b in gitea.comments)
assert _ledger_debits(core, c["id"]) == [] # no billing for the moved pair
second = policy.poll_and_review("gitea-saas-core")
assert second and second[0]["action"] == "merged" # sha-B reviewed + merged
assert gitea.merged == [("gitea-saas-core", 94)] # exactly one merge, ever
assert gitea.closed == [("gitea-saas-core", 1)]
assert len(_ledger_debits(core, c["id"])) == 1
assert policy._already_reviewed("gitea-saas-core", 94, "sha-A") is False
assert policy._already_reviewed("gitea-saas-core", 94, "sha-B") is True
def test_crash_window_adopts_merged_pr_from_production_shape(policy_factory):
# The merge applied server-side but the process died before the
# merged_step checkpoint write: with the production normalized shape
# (merged/state now included), recovery positively adopts the merged PR
# and finishes debit/label/close/comment — without ever calling
# merge_pr again.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(96, head_sha="sha-96", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={})
policy._ensure_plan("gitea-saas-core", 96, "sha-96", APPROVE, "gold", 1)
# The live PR record already reports the server-side merge (raw Gitea
# record normalized by the production client); the checkpoint is the
# only thing missing.
gitea.prs[0]["merged"] = True
gitea.prs[0]["state"] = "closed"
results = policy.poll_and_review("gitea-saas-core")
assert results and results[0]["action"] == "merged"
assert results[0].get("recovered") is True
assert gitea.merged == [] # crash window: no merge request needed
assert gitea.labels_removed.count(("gitea-saas-core", 96, "in-review")) == 1
assert gitea.closed == [("gitea-saas-core", 1)]
assert any("Merged by review-gate" in b for _, _, b in gitea.comments)
assert len(_ledger_debits(core, c["id"])) == 1
assert policy._already_reviewed("gitea-saas-core", 96, "sha-96") is True
assert policy.poll_and_review("gitea-saas-core") == [] # completed: skipped
def test_closed_unmerged_pr_recovery_never_debits_or_closes(policy_factory):
# Negative regression: state == "closed" is NOT proof of a merge. The
# outcome plan exists for the head, the merge never happened, and the
# PR is closed without merging — recovery must not adopt it as merged:
# no debit, no label removal, no issue close, no success comment.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(95, head_sha="sha-95", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={})
policy._ensure_plan("gitea-saas-core", 95, "sha-95", APPROVE, "gold", 1)
gitea.prs[0]["state"] = "closed" # closed WITHOUT merging
gitea.prs[0]["merged"] = False
results = policy.poll_and_review("gitea-saas-core")
assert results == [] # nothing adopted, nothing applied
assert gitea.merged == [] and gitea.closed == []
assert gitea.labels_removed == []
assert gitea.comments == []
assert _ledger_debits(core, c["id"]) == []
assert policy._step_done("gitea-saas-core", 95, "sha-95", "merged") is False
assert policy._already_reviewed("gitea-saas-core", 95, "sha-95") is False
# ---------------------------------------------------------------------------
# Governor review #3 (10539) regressions — head-integrity gates:
# 1. recovery adopts a merged PR ONLY when the live merged head equals the
# persisted planned/reviewed head; a DIFFERENT merged head fails closed
# with a structured HEAD_MISMATCH and no debit/label/close/comment;
# 2. a candidate with NO reviewed head_sha is rejected BEFORE the merge
# client is called (expected_head_sha=None would arm the client's
# live-head fallback and could merge an unreviewed head).
# ---------------------------------------------------------------------------
def test_recovery_merged_different_head_fails_closed(policy_factory):
# Negative regression: the plan was persisted for reviewed head sha-A,
# but the branch moved and head sha-B was merged externally before the
# merged_step checkpoint write. Recovery must NOT attribute the sha-B
# merge to the sha-A plan: zero side effects (no debit, no label
# removal, no issue close, no success comment), a structured
# HEAD_MISMATCH result, and the row left unadopted for the operator.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(97, head_sha="sha-A", labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={})
policy._ensure_plan("gitea-saas-core", 97, "sha-A", APPROVE, "gold", 1)
gitea.prs[0]["head_sha"] = "sha-B" # branch moved AND merged externally
gitea.prs[0]["merged"] = True
gitea.prs[0]["state"] = "closed"
results = policy.poll_and_review("gitea-saas-core")
assert results and results[0]["outcome"] == "failed"
assert results[0]["error"]["code"] == "HEAD_MISMATCH"
assert "sha-A" in results[0]["error"]["message"]
assert "sha-B" in results[0]["error"]["message"]
assert gitea.merged == [] and gitea.closed == []
assert gitea.labels_removed == []
assert gitea.comments == []
assert _ledger_debits(core, c["id"]) == []
assert policy._step_done("gitea-saas-core", 97, "sha-A", "merged") is False
assert policy._already_reviewed("gitea-saas-core", 97, "sha-A") is False
def test_missing_reviewed_head_never_reaches_merge_client(policy_factory):
# Negative regression: a candidate PR with no head_sha must be refused
# BEFORE the merge client is called — passing expected_head_sha=None
# would arm the production client's live-head fallback and could merge
# a head the policy never reviewed. Assert zero merge attempts and zero
# billing/label/close/comment side effects, with a structured failure.
policy, gitea, reviewer, core, c = policy_factory(
gitea_prs=[pr(98, labels=("in-review",))],
gitea_issues=tier_issue("gold", number=1),
verdicts={98: make_verdict(98, APPROVE, head_sha=None)})
gitea.prs[0]["head_sha"] = None # no reviewed head on the candidate
results = policy.poll_and_review("gitea-saas-core")
assert results and results[0]["outcome"] == "failed"
assert "no reviewed head_sha" in results[0]["error"]["message"]
assert "fall back to the live head" in results[0]["error"]["message"]
assert gitea.merged == [] and gitea.closed == []
assert gitea.labels_removed == []
assert gitea.comments == []
assert _ledger_debits(core, c["id"]) == []
assert policy._already_reviewed("gitea-saas-core", 98, "unknown") is False

383
tests/test_review_workspace.py
View File

@ -1,383 +0,0 @@
"""Hermetic tests for the isolated review workspace (issue #30).
Real ``git`` binaries against LOCAL temporary repositories only — every
"remote" is a local-path ``origin`` (a bare repo under ``tmp_path``), so
the suite proves the exact production flow (fetch named SHAs, detached
worktree, synthetic merge, cleanup) with zero network calls.
Run with: python -m pytest tests/test_review_workspace.py -q
"""
import subprocess
import sys
import tempfile
from pathlib import Path
from types import SimpleNamespace
import pytest
# PEP 420 namespace package: make the project root importable regardless of
# how pytest was invoked.
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from gitea_saas.review_workspace import ( # noqa: E402
ReviewWorkspace,
ReviewWorkspaceError,
)
from gitea_saas.reviewer import Runner # noqa: E402
GIT = ["git"]
def git(repo, *args):
"""Run git in ``repo``; return the CompletedProcess (checked)."""
repo = Path(repo)
if not repo.exists() and args and args[0] == "init":
repo.mkdir(parents=True, exist_ok=True)
return subprocess.run(
[*GIT, "-C", str(repo), *args], check=True,
capture_output=True, text=True)
def git_ok(repo, *args):
return git(repo, *args).returncode == 0
def commit_file(repo, filename, content, message):
path = repo / filename
path.write_text(content)
git(repo, "add", filename)
git(repo, "commit", "-m", message)
return git(repo, "rev-parse", "HEAD").stdout.strip()
@pytest.fixture
def fixtures(tmp_path):
"""Build a local origin (bare), a seeding checkout, and the configured
clone. Both the `clean` and the `conflict` head/base pairs come from
the same origin (all SHAs are advertised branch tips, so the exact-SHA
fetch works over the local-path transport exactly like production)."""
origin = tmp_path / "origin.git"
git(origin, "init", "--bare", "--initial-branch=main")
seed = tmp_path / "seed"
git(seed, "init", "--initial-branch=main")
git(seed, "config", "user.email", "seed@example.com")
git(seed, "config", "user.name", "Seed")
git(seed, "remote", "add", "origin", str(origin))
base0 = commit_file(seed, "alpha.txt", "line one\n", "base file")
git(seed, "push", "-u", "origin", "main")
# ---- clean pair: feature adds beta.txt, main adds gamma.txt ----
git(seed, "checkout", "-b", "feature", base0)
head_clean = commit_file(seed, "beta.txt", "feature add\n", "feature work")
git(seed, "push", "-u", "origin", "feature")
git(seed, "checkout", "main")
base_clean = commit_file(seed, "gamma.txt", "main add\n", "main work")
git(seed, "push", "origin", "main")
# ---- conflict pair: both sides edit alpha.txt after branching ----
git(seed, "checkout", "-b", "conflict-head", base0)
head_conflict = commit_file(seed, "alpha.txt", "feature change\n", "conflict head")
git(seed, "checkout", "main")
base_conflict = commit_file(seed, "alpha.txt", "main change\n", "conflict base")
git(seed, "push", "origin", "main")
git(seed, "push", "-u", "origin", "conflict-head")
clone = tmp_path / "clone"
subprocess.run(["git", "clone", str(origin), str(clone)],
check=True, capture_output=True, text=True)
git(clone, "config", "user.email", "clone@example.com")
git(clone, "config", "user.name", "Clone")
clone_head_before = git(clone, "rev-parse", "HEAD").stdout.strip()
return SimpleNamespace(
origin=origin, seed=seed, clone=clone,
head_clean=head_clean, base_clean=base_clean,
head_conflict=head_conflict, base_conflict=base_conflict,
clone_head_before=clone_head_before,
)
def review_dirs(pr_number):
root = Path(tempfile.gettempdir())
return sorted(root.glob(f"review-ws-acme-{pr_number}-*"))
def clone_worktree_lines(clone):
return git(clone, "worktree", "list").stdout.strip()
def assert_clone_untouched(fx):
assert git(fx.clone, "rev-parse", "HEAD").stdout.strip() == fx.clone_head_before
assert git(fx.clone, "status", "--porcelain").stdout.strip() == ""
# ---------------------------------------------------------------------------
# Clean merge
# ---------------------------------------------------------------------------
def test_clean_merge_constructs_synthetic_merge(fixtures):
ws = ReviewWorkspace(fixtures.clone, Runner())
with ws.build("acme", 1, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
# The working tree IS the merge result: both sides' files present,
# both parent commits in history.
assert (mws.dir / "beta.txt").read_text() == "feature add\n"
assert (mws.dir / "gamma.txt").read_text() == "main add\n"
assert (mws.dir / "alpha.txt").read_text() == "line one\n"
parents = git(mws.dir, "rev-list", "--parents", "-n", "1", "HEAD").stdout.split()
assert len(parents) == 3 # merge commit + base + head
assert mws.evidence["base_sha"] == fixtures.base_clean
assert mws.evidence["head_sha"] == fixtures.head_clean
assert mws.evidence["result_head"] == parents[0]
# Explicit base/result range works from inside the workspace.
res = Runner().run(mws.dir, f"git diff --check {fixtures.base_clean} HEAD")
assert res["exit"] == 0
# Cleaned up after success; the configured clone is untouched.
assert review_dirs(1) == []
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
assert_clone_untouched(fixtures)
# ---------------------------------------------------------------------------
# Merge conflict
# ---------------------------------------------------------------------------
def test_merge_conflict_is_deterministic_evidence(fixtures):
ws = ReviewWorkspace(fixtures.clone, Runner())
with ws.build("acme", 2, fixtures.head_conflict, fixtures.base_conflict) as mws:
assert mws.merge_ok is False
assert mws.evidence["merge_exit"] != 0
assert mws.evidence["conflicted_files"] == ["alpha.txt"]
assert "alpha.txt" in mws.evidence["status_summary"]
assert review_dirs(2) == []
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
assert_clone_untouched(fixtures)
# ---------------------------------------------------------------------------
# No shared-checkout mutation (source clone AND a worker-style worktree)
# ---------------------------------------------------------------------------
def test_neither_clone_nor_worker_worktree_is_mutated(fixtures):
# A worker's issue worktree taken from the same clone must be untouched
# by the review pass (its branch, content, and dirty state all survive).
worker = fixtures.clone.parent / "worker-issue-42"
git(fixtures.clone, "worktree", "add", "--detach", str(worker), fixtures.base_clean)
(worker / "alpha.txt").write_text("worker dirty line\n") # simulate WIP
worker_head_before = git(worker, "rev-parse", "HEAD").stdout.strip()
ws = ReviewWorkspace(fixtures.clone, Runner())
with ws.build("acme", 3, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
# the worker checkout's content is untouched by the merge
assert (worker / "alpha.txt").read_text() == "worker dirty line\n"
assert git(worker, "rev-parse", "HEAD").stdout.strip() == worker_head_before
# (strip(): the porcelain status column's leading space is whitespace)
assert git(worker, "status", "--porcelain").stdout.strip() == "M alpha.txt"
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
# the worker worktree itself still exists with its dirty state
assert worker.exists()
assert git(worker, "status", "--porcelain").stdout.strip() == "M alpha.txt"
assert_clone_untouched(fixtures)
git(fixtures.clone, "worktree", "remove", "--force", str(worker))
# ---------------------------------------------------------------------------
# Cleanup on failure paths
# ---------------------------------------------------------------------------
def test_cleanup_after_raised_exception(fixtures):
ws = ReviewWorkspace(fixtures.clone, Runner())
before = review_dirs(4)
with pytest.raises(RuntimeError, match="boom"):
with ws.build("acme", 4, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
raise RuntimeError("boom")
assert review_dirs(4) == before
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
assert_clone_untouched(fixtures)
def test_cleanup_after_merge_conflict_exit(fixtures):
# Simulates the context manager exiting with a conflict result — the
# workspace must still be fully removed.
ws = ReviewWorkspace(fixtures.clone, Runner())
before = review_dirs(5)
with ws.build("acme", 5, fixtures.head_conflict, fixtures.base_conflict) as mws:
assert mws.merge_ok is False
assert review_dirs(5) == before
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
assert_clone_untouched(fixtures)
def test_unfetchable_sha_raises_and_leaves_nothing(fixtures):
ws = ReviewWorkspace(fixtures.clone, Runner())
before = review_dirs(6)
with pytest.raises(ReviewWorkspaceError):
ws.build("acme", 6, "f" * 40, fixtures.base_clean)
assert review_dirs(6) == before
assert_clone_untouched(fixtures)
def test_non_hex_sha_fails_closed_without_git(fixtures):
ws = ReviewWorkspace(fixtures.clone, Runner())
with pytest.raises(ReviewWorkspaceError, match="not a 40-char hex"):
ws.build("acme", 7, "HEAD", fixtures.base_clean) # a ref, not a SHA
with pytest.raises(ReviewWorkspaceError, match="not a 40-char hex"):
ws.build("acme", 7, fixtures.head_clean, "main")
assert_clone_untouched(fixtures)
# ---------------------------------------------------------------------------
# Commands run in the right place (cwd + explicit range)
# ---------------------------------------------------------------------------
def test_git_commands_run_in_workspace_cwd(fixtures):
calls = []
def spy(command, **kwargs):
calls.append((kwargs.get("cwd"), command))
if isinstance(command, str):
return subprocess.run(command, shell=True, **kwargs)
return subprocess.run(command, shell=False, **kwargs)
runner = Runner(timeout_s=120, command_runner=spy)
ws = ReviewWorkspace(fixtures.clone, runner)
with ws.build("acme", 8, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
merges = [c for c in calls
if isinstance(c[1], list)
and c[1][:4] == ["git", "merge", "--no-ff", "-m"]]
assert merges, "synthetic merge command must run"
assert Path(merges[0][0]) == mws.dir, "merge must run INSIDE the workspace"
assert "review-ws" in str(merges[0][0])
# fetch + worktree add run against the configured clone (read-only use)
adds = [c for c in calls
if isinstance(c[1], list)
and c[1][:4] == ["git", "worktree", "add", "--detach"]]
assert adds and Path(adds[0][0]) == fixtures.clone
fetches = [c for c in calls
if isinstance(c[1], list)
and c[1][:3] == ["git", "fetch", "origin"]]
assert fetches and Path(fetches[0][0]) == fixtures.clone
# cleanup command targets the clone
removes = [c for c in calls
if isinstance(c[1], list)
and c[1][:4] == ["git", "worktree", "remove", "--force"]]
assert removes and Path(removes[0][0]) == fixtures.clone
def test_explicit_range_diff_check_from_workspace(fixtures):
calls = []
def spy(command, **kwargs):
calls.append((kwargs.get("cwd"), command))
if isinstance(command, str):
return subprocess.run(command, shell=True, **kwargs)
return subprocess.run(command, shell=False, **kwargs)
runner = Runner(timeout_s=120, command_runner=spy)
ws = ReviewWorkspace(fixtures.clone, runner)
with ws.build("acme", 9, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
# The reviewer's hygiene gate: diff --check over the EXPLICIT
# base..result range, run from INSIDE the isolated workspace.
res = runner.run(
mws.dir, ["git", "diff", "--check",
fixtures.base_clean, "HEAD"])
assert res["exit"] == 0
checks = [c for c in calls
if isinstance(c[1], list)
and c[1][:3] == ["git", "diff", "--check"]]
assert checks, "explicit-range diff --check must run"
assert checks[0][1][3] == fixtures.base_clean # explicit BASE
assert checks[0][1][4] == "HEAD" # explicit result end
assert "review-ws" in str(checks[0][0]) # inside the isolated workspace
# ---------------------------------------------------------------------------
# Windows portability (issue #48): git args reach git as a LIST, never a
# shell string — cmd.exe must not be able to eat `^` (or any other
# shell-special char) from `^{commit}` and friends.
# ---------------------------------------------------------------------------
def test_caret_peel_reaches_git_as_single_list_token():
recorded = []
def fake_runner(command, **kwargs):
recorded.append(command)
# All invocations are argument lists (never shell strings): every
# git command is recorded for the assertions below.
assert isinstance(command, list), (
f"git command must be an argument list, got {command!r}")
head = command[:2]
if head == ["git", "rev-parse"] and "--verify" in command:
return SimpleNamespace(
returncode=0, stdout="1fccd03843f7d3e7ef0bf5428aa7a40114b70263",
stderr="")
return SimpleNamespace(returncode=0, stdout="", stderr="")
runner = Runner(timeout_s=60, command_runner=fake_runner)
ws = ReviewWorkspace("/fake/clone", runner)
sha = "d3df1df0c2e458ac5da94a4edb2d254777f64a02"
with ws.build("acme", 48, sha, sha) as mws:
assert mws.merge_ok is True
# The peel suffix survives as ONE list element — cmd.exe never sees
# it, so the Windows `^`-escape class of breakage is impossible.
peels = [c for c in recorded
if c[:2] == ["git", "rev-parse"]
and "--verify" in c and c[-1].endswith("^{commit}")]
assert peels, "rev-parse ^{commit} verification must run"
assert all(len(c) == 4 and c[3] == f"{sha}^{{commit}}"
for c in peels), f"peel must be a single token, got {peels}"
# No stringified-list commands anywhere: a shell can never see them.
assert all(isinstance(c, list) for c in recorded)
# ---------------------------------------------------------------------------
# Bounce-fix: shell-safe path quoting + filesystem-safe temp prefix
# ---------------------------------------------------------------------------
def test_weird_temp_root_with_spaces_and_shell_char_success_and_cleanup(
fixtures, tmp_path, monkeypatch):
# A valid OS temp root whose path contains a space AND a shell-sensitive
# character ($). Unquoted interpolation into the Runner's shell=True
# commands would split the path (construction fails) or — worse —
# let `git worktree remove` fail while rmtree deletes the directory,
# leaving a dangling worktree admin entry in the clone.
weird_root = tmp_path / "review root $weird"
weird_root.mkdir(parents=True)
monkeypatch.setattr(tempfile, "tempdir", str(weird_root))
ws = ReviewWorkspace(fixtures.clone, Runner())
with ws.build("acme", 10, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
assert mws.dir.exists()
assert " " in str(mws.dir) # really lived under the weird root
assert "$" in str(mws.dir)
assert mws.dir.parent == weird_root
# Complete cleanup: directory gone AND no worktree admin record remains.
assert not mws.dir.exists()
assert "review-ws" not in git(
fixtures.clone, "worktree", "list", "--porcelain").stdout
assert_clone_untouched(fixtures)
def test_repo_token_with_separators_is_filesystem_safe(fixtures):
# ``repo`` is embedded in the mkdtemp prefix; an owner/name-style token
# must be collapsed to a single filename component (a NESTED prefix
# would make tempfile.mkdtemp raise FileNotFoundError).
repo = "pbwraith/gitea-saas-core"
ws = ReviewWorkspace(fixtures.clone, Runner())
with ws.build(repo, 11, fixtures.head_clean, fixtures.base_clean) as mws:
assert mws.merge_ok is True
assert mws.dir.name.startswith("review-ws-pbwraith-gitea-saas-core-11-")
assert "/" not in mws.dir.name
assert not mws.dir.exists()
assert "review-ws" not in git(
fixtures.clone, "worktree", "list", "--porcelain").stdout
assert_clone_untouched(fixtures)

129
tests/test_reviewer.py
View File

@ -11,7 +11,6 @@ import ast
import subprocess
import sys
from pathlib import Path
from types import SimpleNamespace
import pytest
@ -37,14 +36,13 @@ class FakeGiteaClient:
"""In-memory Gitea seam: PRs, issues and posted comments."""
def __init__(self, prs=None, issues=None):
self.prs = prs or {} # number -> {"head_sha", "base_sha", "changed_files", "diff"}
self.prs = prs or {} # number -> {"head_sha", "changed_files", "diff"}
self.issues = issues or {} # number -> {"body", ...}
self.comments = [] # (repo, number, body)
def get_pr(self, repo, number):
pr = self.prs[number]
return {"head_sha": pr["head_sha"], "base_sha": pr.get("base_sha") or SAMPLE_BASE,
"changed_files": list(pr["changed_files"])}
return {"head_sha": pr["head_sha"], "changed_files": list(pr["changed_files"])}
def get_pr_diff(self, repo, number):
return self.prs[number].get("diff", "")
@ -56,37 +54,6 @@ class FakeGiteaClient:
self.comments.append((repo, number, body))
SAMPLE_BASE = "b" * 40
class FakeWorkspace:
"""Duck-typed review-workspace seam: no git, scripted merge result."""
def __init__(self, merge_ok=True, evidence=None, directory="/fake/ws",
base_sha=SAMPLE_BASE):
self.merge_ok = merge_ok
self.evidence = dict(evidence or {})
self.directory = str(directory)
self.base_sha = base_sha
def build(self, repo, pr_number, head_sha, base_sha):
return _FakeMergeContext(self)
class _FakeMergeContext:
def __init__(self, ws):
self._ws = ws
def __enter__(self):
return SimpleNamespace(
dir=Path(self._ws.directory), base_sha=self._ws.base_sha,
head_sha="a" * 40, merge_ok=self._ws.merge_ok,
evidence=self._ws.evidence)
def __exit__(self, *exc):
return False
class FakeRunner:
"""Subprocess seam: scripted results, records commands."""
@ -147,19 +114,16 @@ def test_approve_when_tests_pass_and_no_findings():
{"exit": 0, "summary": "clean"}]) # git diff --check
reviewer = Reviewer(client, runner, config={"secret_scan": True})
verdict = reviewer.review_pr("gitea-saas-core", 1, client.issues[5],
workspace=FakeWorkspace())
verdict = reviewer.review_pr("gitea-saas-core", 1, client.issues[5], "/fake/repo")
assert verdict.verdict == APPROVE
assert verdict.tests == {"exit": 0, "summary": "12 passed"}
assert verdict.acceptance == ["tests pass", "no regressions"]
assert verdict.scope_violations == [] and verdict.findings == []
assert verdict.head_sha == "a" * 40
assert verdict.base_sha == "b" * 40
comment = reviewer.post_verdict("gitea-saas-core", 1, verdict)
assert "[review-gate] VERDICT: **APPROVE**" in comment
assert "head `aaaaaaaaaaaa`" in comment and "base `bbbbbbbbbbbb`" in comment
assert "- [ ] tests pass" in comment # checklist present
assert "No merge performed by the review gate" in comment
assert client.comments and client.comments[0][:2] == ("gitea-saas-core", 1)
@ -168,8 +132,7 @@ def test_approve_when_tests_pass_and_no_findings():
def test_bounce_when_tests_fail():
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: sample_issue()})
runner = FakeRunner([{"exit": 1, "summary": "1 failed, 11 passed"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, client.issues[5],
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, client.issues[5], "/fake/repo")
assert verdict.verdict == BOUNCE
assert "tests failed (exit=1)" in verdict.summary
# BOUNCE comment carries the test evidence
@ -183,8 +146,7 @@ def test_bounce_on_scope_violation_even_with_green_tests():
pr = sample_pr(files=["gitea_saas/reviewer.py", "gitea_saas/server.py"])
client = FakeGiteaClient(prs={1: pr}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
assert verdict.verdict == BOUNCE
assert verdict.scope_violations == ["gitea_saas/server.py"]
@ -193,8 +155,7 @@ def test_needs_guidance_when_no_acceptance_criteria():
issue = {"body": "## Goal\nnothing to verify here\n"}
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
assert verdict.verdict == NEEDS_GUIDANCE
assert "never silent-approve" in verdict.summary
@ -203,8 +164,7 @@ def test_allow_test_failure_in_issue_body_overrides_bounce():
issue = sample_issue(extra="note: allow_test_failure for this transitional PR\n")
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 1, "summary": "known failure"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
assert verdict.verdict == APPROVE # tolerated, acceptance present, nothing else blocking
@ -228,8 +188,7 @@ def test_secret_finding_quotes_and_redacts_diff_line():
pr = sample_pr(diff=SECRET_DIFF, files=["gitea_saas/server.py"])
client = FakeGiteaClient(prs={1: pr}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "ok"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
blocking = [f for f in verdict.findings if f["severity"] == "blocking"]
assert blocking, "secret scan should produce a blocking finding"
f = blocking[0]
@ -249,7 +208,7 @@ def test_todo_finding_is_warning_not_blocking():
client = FakeGiteaClient(prs={1: pr}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "ok"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner, config={"secret_scan": False}).review_pr(
"gitea-saas-core", 1, issue, workspace=FakeWorkspace())
"gitea-saas-core", 1, issue, "/fake/repo")
todos = [f for f in verdict.findings if f["severity"] == "warning"]
assert todos and todos[0]["note"] == "TODO/FIXME/HACK marker in added line"
assert todos[0]["evidence_quote"] == " # TODO: replace placeholder"
@ -265,8 +224,7 @@ def test_timeout_bounces_with_timed_out_exit():
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": "timed_out", "summary": "[timed out after 300s]"},
{"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
assert verdict.verdict == BOUNCE
assert verdict.tests["exit"] == "timed_out"
assert "timed out" in verdict.summary
@ -312,8 +270,7 @@ def test_build_command_run_when_configured_and_build_failure_bounces():
{"exit": 1, "summary": "build error"}, # build
{"exit": 0, "summary": "clean"}]) # diff --check
reviewer = Reviewer(client, runner, config={"build_cmd": "python -m compileall -q ."})
verdict = reviewer.review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = reviewer.review_pr("gitea-saas-core", 1, issue, "/fake/repo")
build = verdict.build
assert build is not None
assert build["exit"] == 1 and build["command"] == "python -m compileall -q ."
@ -326,76 +283,16 @@ def test_diff_hygiene_failure_is_blocking_finding():
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"},
{"exit": 1, "summary": "trailing whitespace at gitea_saas/x.py:3"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
assert verdict.verdict == BOUNCE
assert any("diff --check" in f["note"] for f in verdict.findings)
def test_hygiene_diff_check_uses_explicit_base_head_range():
issue = sample_issue()
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"},
{"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
assert verdict.verdict == APPROVE
# The whitespace gate runs against the synthetic merge result with an
# explicit base..HEAD range (issue #30 / acceptance 5), from the
# workspace directory — never a bare `git diff --check`.
assert len(runner.commands) == 2 # pytest + explicit-range diff --check
ws_dir, hygiene_cmd, _ = runner.commands[1]
# Compare as PATHs, not strings: on Windows str(Path('/fake/ws')) is
# '\\fake\\ws' (a valid spelling of the same directory) — the string
# comparison would fail on the platform this gate actually runs on.
# Fix in the test, not the source: passing a Path as cwd is correct
# (bounce-fix review 11154; issue #48).
assert Path(ws_dir) == Path("/fake/ws")
# The whitespace gate runs as an ARGUMENT LIST (shell=False) — no shell
# can mangle the SHA/HEAD tokens on any platform (issue #48).
assert hygiene_cmd == ["git", "diff", "--check",
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "HEAD"]
def test_merge_conflict_bounces_without_running_tests():
issue = sample_issue()
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}]) # would be tests
workspace = FakeWorkspace(merge_ok=False, evidence={
"conflicted_files": ["gitea_saas/reviewer.py"],
"status_summary": "UU gitea_saas/reviewer.py",
"merge_exit": 1})
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=workspace)
assert verdict.verdict == BOUNCE
assert "synthetic merge conflict" in verdict.summary
assert verdict.tests["exit"] == "not_run"
assert runner.commands == [], "the test suite must NOT run on a conflicted merge"
assert verdict.workspace["merge_ok"] is False
assert verdict.workspace["conflicted_files"] == ["gitea_saas/reviewer.py"]
comment = verdict.to_comment()
assert "**CONFLICT**" in comment
assert "The test suite was NOT run" in comment
assert "`gitea_saas/reviewer.py`" in comment
def test_no_workspace_fails_closed_to_needs_guidance():
issue = sample_issue()
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue)
assert verdict.verdict == NEEDS_GUIDANCE
assert "no isolated review workspace configured" in verdict.summary
assert verdict.tests["exit"] == "not_run"
assert runner.commands == [], "no commands may run without a workspace"
def test_verdict_shape_and_comment_never_mentions_merge_action():
issue = sample_issue()
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}, {"exit": 0, "summary": "clean"}])
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
workspace=FakeWorkspace())
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
assert verdict.repo == "gitea-saas-core" and verdict.pr == 1
assert isinstance(verdict.acceptance, list)
assert isinstance(verdict.tests, dict) and "exit" in verdict.tests

7
tests/test_server.py
View File

@ -18,7 +18,6 @@ from pathlib import Path
import pytest
from gitea_saas import billing_core, server
from gitea_saas.auth import UNAUTHORIZED_BODY
from gitea_saas.config import API_KEY_HEADER
API_KEY = "test-api-key-secret"
@ -86,14 +85,14 @@ def test_health_is_public(client):
def test_missing_key_is_401(client):
resp = client.get("/api/clients/1")
assert resp.status_code == 401
assert resp.get_json() == UNAUTHORIZED_BODY
body = resp.get_json()
assert body["code"] == "unauthorized"
def test_wrong_key_is_401(client):
resp = client.get("/api/clients/1", headers={API_KEY_HEADER: "nope"})
assert resp.status_code == 401
# Canonical 401 body (issue #24): byte-identical across every front-end.
assert resp.get_json() == UNAUTHORIZED_BODY
assert resp.get_json()["code"] == "unauthorized"
def test_correct_key_is_200(client):

10
wsgi.py
View File

@ -1,10 +0,0 @@
"""Gunicorn entrypoint (issue #38).
``gunicorn wsgi:application`` — the assembled app built by
``gitea_saas.app.create_app`` from the environment (SAAS_API_KEY,
SAAS_DB_PATH, SAAS_PORT).
"""
from gitea_saas.app import create_app
application = create_app()