Compare commits
33 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d75c1cc235 | |||
| 67d03e2fed | |||
|
|
f39343ea81 | ||
|
|
5be23b9390 | ||
| 2ef1e668fa | |||
| 9da95125f6 | |||
| 5fda4c09b6 | |||
| a66f88ea10 | |||
| 729536d3eb | |||
|
|
6326199966 | ||
|
|
3d3e592832 | ||
|
|
cf798a809f | ||
|
|
e085457d0a | ||
|
|
8620d4d36e | ||
|
|
0f9d2a9b30 | ||
| 71cc2634ce | |||
|
|
069a74a95c | ||
| b4e577b31b | |||
|
|
d8faabf2e5 | ||
| 8588033d73 | |||
|
|
b46c356d64 | ||
|
|
30e71d44ba | ||
|
|
d522ea23e4 | ||
| fc55c41dd0 | |||
|
|
f6b40d0108 | ||
|
|
c3915d2533 | ||
|
|
1f2c85b4ff | ||
|
|
6cff1599d2 | ||
|
|
8b17668746 | ||
|
|
3819b49498 | ||
|
|
64ab8478dc | ||
|
|
08febd56ca | ||
| ccccfbf879 |
17
.dockerignore
Normal file
17
.dockerignore
Normal file
@ -0,0 +1,17 @@
|
||||
# Build context: runtime files only. The container runs the app, not
|
||||
# the repo's dev/test surface, and it must not re-pull git or cached
|
||||
# bytecode into the image.
|
||||
.git
|
||||
.gitignore
|
||||
.gitattributes
|
||||
__pycache__
|
||||
*.py[cod]
|
||||
*.egg-info
|
||||
.pytest_cache
|
||||
tests
|
||||
venv
|
||||
.venv
|
||||
OPS.md
|
||||
Dockerfile
|
||||
docker-compose.yml
|
||||
.dockerignore
|
||||
79
CALIBRATION.md
Normal file
79
CALIBRATION.md
Normal file
@ -0,0 +1,79 @@
|
||||
# CALIBRATION.md — review-gate false-approve rate (issue #42)
|
||||
|
||||
## Outcome
|
||||
|
||||
The number the gold tier's auto-merge promise needs: **how often the review
|
||||
gate approves a change it should have bounced.**
|
||||
|
||||
Measured over a corpus of **25 real farm PRs with known outcomes** (drawn
|
||||
from 11 pbwraith repos' merged and bounced history) plus the two
|
||||
owner-documented regressions the issue cites (botmanager#223's shipper and
|
||||
botmanager !22). The gate under test is the shipped
|
||||
`gitea_saas.reviewer.Reviewer` at its DEFAULT (deterministic) config, run
|
||||
unchanged against recorded inputs — issue body, changed files, diff, and the
|
||||
observed test/hygiene exit from farm history.
|
||||
|
||||
## The number
|
||||
|
||||
| rate | value |
|
||||
|---|---|
|
||||
| false-approve rate (approved, should have bounced) | **0 / 14 = 0.0%** |
|
||||
| false-approve prevalence (over 25 reviewable PRs) | 0.0% |
|
||||
| false-bounce rate (bounced, should have approved) | n/a (0 bounces produced) |
|
||||
| unverifiable — NEEDS_GUIDANCE (fail-closed refusal) | **11 / 25 = 44.0%** |
|
||||
|
||||
On the operator's own terms — the two owner-documented cases added to the 14
|
||||
the gate can evaluate — the false-approve rate is **2 / 16 = 12.5%**.
|
||||
|
||||
## Recommendation: NO
|
||||
|
||||
Gold-tier auto-merge (**auto-merges on a green APPROVE**) is **not safe to
|
||||
sell at this rate**. The 0.0% headline is an artifact of refusal, not of
|
||||
gate quality:
|
||||
|
||||
1. The gate approved only the 14 corpus PRs whose issues carry `- [ ]`
|
||||
checklists (or a section titled exactly `## Acceptance`) and refused the
|
||||
other 11 (44%) — real merged or bounced work it could not verify. A gate
|
||||
that cannot evaluate 44% of real issues cannot back an auto-merge
|
||||
guarantee.
|
||||
2. The approve path has **no check that would have stopped either documented
|
||||
regression**: botmanager#223's break was green under the configured gate
|
||||
(pytest gating a frontend-only change; green on a case-sensitive Linux
|
||||
worktree while the target is case-insensitive Windows) and botmanager !22
|
||||
was green in isolation (no sibling-merge/integration verification). Both
|
||||
mechanisms survive in the approve path today.
|
||||
3. Accident of format, not fail-closed design: botmanager#220 (the shipper)
|
||||
reached NEEDS_GUIDANCE only because its issue titles the section
|
||||
`## Acceptance criteria` with a numbered list, which
|
||||
`parse_acceptance()` does not recognise. With the issue's own real
|
||||
acceptance items parsed (one-line parser accommodation, measured
|
||||
sensitivity), the gate returns **APPROVE on that head — a false approve**.
|
||||
|
||||
Before selling gold auto-merge: (a) close the acceptance-format coverage
|
||||
gap, (b) select the test command/suite from the changed-file set (the
|
||||
gitea-worker-ops !44 direction), (c) add sibling-merge/integration
|
||||
verification, then (d) re-run this calibration on the fixed gate and a
|
||||
larger corpus.
|
||||
|
||||
## How to re-run
|
||||
|
||||
From the repo root (one line, no network, deterministic):
|
||||
|
||||
python tests/calibration/run_calibration.py
|
||||
|
||||
It re-runs the existing reviewer against the committed corpus
|
||||
(`tests/calibration/corpus/*.json` + `tests/calibration/manifest.json`) and
|
||||
rewrites this report of record: `tests/calibration/calibration_report.md`.
|
||||
|
||||
Corpus provenance: the raw Gitea snapshots were fetched read-only on
|
||||
2026-09-01 (see `tests/calibration/build_corpus.py`); the committed corpus
|
||||
is the frozen snapshot the numbers are derived from.
|
||||
|
||||
## Scope compliance
|
||||
|
||||
- New files only: `tests/calibration/` (corpus, runner, builder, report,
|
||||
tests) and this file.
|
||||
- `gitea_saas/reviewer.py` and `gitea_saas/review_policy.py` are **read-only**
|
||||
— not modified. The calibration measures the existing gate; it does not
|
||||
tune it (tuning against a corpus you also authored is how you get a good
|
||||
number and a bad reviewer).
|
||||
49
Dockerfile
Normal file
49
Dockerfile
Normal file
@ -0,0 +1,49 @@
|
||||
# gitea-saas-core container image (issue #40).
|
||||
#
|
||||
# Slim Python base, non-root app user, gunicorn serving the repo-root
|
||||
# wsgi.py entrypoint (``gunicorn wsgi:application``, assembled app from
|
||||
# issue #38).
|
||||
#
|
||||
# Ledger-safety: exactly ONE gunicorn worker. This process owns a SQLite
|
||||
# money ledger; the DB is WAL-mode with writes serialised by BillingCore's
|
||||
# single in-process lock (gitea_saas/db.py + billing_core). Multiple
|
||||
# workers would be multiple writer processes racing on one SQLite file —
|
||||
# WAL permits one writer at a time but a ledger must not depend on SQLITE_BUSY
|
||||
# backoff at the file level. Threads are safe: every request in this process
|
||||
# funnels through that one lock. Scaling to more writer processes requires a
|
||||
# network database (see OPS.md "Scaling"); it is deliberately out of scope here.
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Runtime deps only. flask matches requirements.txt's "flask>=3.0"; gunicorn
|
||||
# is the WSGI server and is installed here rather than added to
|
||||
# requirements.txt, which belongs to earlier issues. pytest stays out of the
|
||||
# image. The ``stripe`` package is NOT installed: the adapter only imports it
|
||||
# lazily and CI/dev run against the built-in mock seam.
|
||||
RUN pip install --no-cache-dir "flask>=3.0" "gunicorn>=23.0"
|
||||
|
||||
# Non-root runtime user.
|
||||
RUN groupadd --system app && useradd --system --gid app --home-dir /app app
|
||||
|
||||
WORKDIR /app
|
||||
COPY . /app
|
||||
|
||||
ENV SAAS_PORT=8080 \
|
||||
SAAS_DB_PATH=/data/saas.db \
|
||||
PYTHONPATH=/app \
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
# /data is where the SQLite ledger lives. In compose it is a named volume —
|
||||
# that volume IS the durability story (see docker-compose.yml).
|
||||
RUN mkdir -p /data && chown -R app:app /app /data
|
||||
|
||||
USER app
|
||||
EXPOSE 8080
|
||||
|
||||
# /health is public (auth.PUBLIC_PATHS) and answers without an API key, so
|
||||
# the check works in every deployment state, including the fail-closed
|
||||
# unconfigured one. urllib only: the slim base has no curl.
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||
CMD python -c "import sys,urllib.request; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=5).status == 200 else 1)"
|
||||
|
||||
CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--workers", "1", "--threads", "4", \
|
||||
"--access-logfile", "-", "--error-logfile", "-", "wsgi:application"]
|
||||
132
OPS.md
Normal file
132
OPS.md
Normal file
@ -0,0 +1,132 @@
|
||||
# gitea-saas-core — operations (issue #40)
|
||||
|
||||
Containerised deployment of the billing service. The app is the assembled
|
||||
Flask app from issue #38 (`wsgi.py` -> `gitea_saas.app.create_app`), served
|
||||
by gunicorn inside a slim Python container. This document covers day-to-day
|
||||
operation: start/stop, backups, key rotation, and what to check when it is
|
||||
down.
|
||||
|
||||
## Ledger reality check (read first)
|
||||
|
||||
- The ledger is a **single SQLite file** at `/data/saas.db` inside the
|
||||
`saas_data` named volume. The volume **is** the durability story: losing
|
||||
the volume loses the ledger. Nothing else holds the data.
|
||||
- The store is **single-writer**: the Dockerfile deliberately runs exactly
|
||||
one gunicorn worker, because multiple writer processes on one SQLite file
|
||||
corrupt or lock. Never raise `--workers` (or run a second `saas`
|
||||
container against the same volume).
|
||||
- SQLite WAL is on and writes are serialised by the app's single lock;
|
||||
threads are safe, workers are not.
|
||||
- An untested restore is worse than no backup: run the restore procedure
|
||||
below against a scratch/compose copy before you ever need it, and after
|
||||
any change to it.
|
||||
|
||||
## Quick start
|
||||
|
||||
git clone <repo> && cd <repo>
|
||||
export SAAS_API_KEY='<a real secret>' # absent key -> app boots FAIL-CLOSED (503 on everything but /health)
|
||||
docker compose up -d --build
|
||||
curl -fsS http://127.0.0.1:8080/health # -> {"ok":true}
|
||||
|
||||
For convenience the key can live in a `.env` file next to
|
||||
`docker-compose.yml` (`SAAS_API_KEY=<secret>`) and compose will pick it up
|
||||
automatically. `.env` is local configuration — do not commit it.
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|-----------------|------------------|------------------------------------------------------|
|
||||
| `SAAS_API_KEY` | (empty -> 503s) | X-API-Key accepted by every authed route. Required before traffic. |
|
||||
| `SAAS_DB_PATH` | `/data/saas.db` | SQLite ledger path inside the container (volume). |
|
||||
| `SAAS_PORT` | `8080` | Port gunicorn binds inside the container. |
|
||||
|
||||
## Day to day
|
||||
|
||||
docker compose ps # state + health of the service
|
||||
docker compose logs --tail=200 # recent logs (stdout/stderr)
|
||||
docker compose up -d # (re)create/start, e.g. after a deploy
|
||||
docker compose restart # restart the process
|
||||
docker compose down # stop; volume is KEPT
|
||||
docker compose down -v # !!! DESTROYS THE LEDGER — never, without a verified backup
|
||||
|
||||
Health endpoint is public (no API key): `/health` and `/healthz` return
|
||||
`{"ok":true}`. Authed routes need `X-API-Key: <key>`; with no key
|
||||
configured they return 503 `SERVICE_UNAVAILABLE` (fail-closed by design).
|
||||
|
||||
## Backup
|
||||
|
||||
Online, consistent (uses SQLite's backup API, safe while the service is
|
||||
running — do not `cp` the live file):
|
||||
|
||||
docker compose exec saas python -c "import sqlite3; s=sqlite3.connect('/data/saas.db'); d=sqlite3.connect('/data/saas-backup.db'); s.backup(d); d.close(); s.close(); print('backup OK')"
|
||||
docker compose cp saas:/data/saas-backup.db ./saas-backup-$(date +%F).db
|
||||
|
||||
Check the archive landed and is readable before moving on:
|
||||
|
||||
ls -l saas-backup-*.db
|
||||
docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone()); print(c.execute('select count(*) from clients').fetchone()[0], 'clients')"
|
||||
|
||||
Keep the copy off-machine (the compose down /data copy dies with the host).
|
||||
A crontab line like `docker compose ... exec saas python -c "..." && docker compose cp ...` off-site is enough.
|
||||
|
||||
## Restore (tested end-to-end once — see the issue PR for the recorded run)
|
||||
|
||||
Run from the directory that contains the backup file. `gitea-saas` is the
|
||||
compose project name (pinned in docker-compose.yml), so the volume is
|
||||
`gitea-saas_saas_data`.
|
||||
|
||||
The container runs as the non-root `app` user, so the directory holding the
|
||||
backup file must be traversable by it (e.g. mode 0755, file 0644); a backup
|
||||
stashed in a private home directory will fail with "Permission denied".
|
||||
|
||||
docker compose down # no writer while we swap files
|
||||
docker run --rm -v gitea-saas_saas_data:/data -v "$(pwd)":/restore \
|
||||
gitea-saas-core:local sh -c "rm -f /data/saas.db-wal /data/saas.db-shm && cp /restore/saas-backup.db /data/saas.db"
|
||||
docker compose up -d
|
||||
|
||||
Why the `rm` of `-wal`/`-shm`: the running app leaves WAL sidecar files; if
|
||||
they survived when we replace `saas.db`, SQLite would replay stale frames
|
||||
onto the restored file. Removing them first is required, not cosmetic.
|
||||
|
||||
Verify the restored ledger:
|
||||
|
||||
curl -fsS http://127.0.0.1:8080/health
|
||||
docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone()); print('clients:', c.execute('select coalesce(group_concat(name, \", \"), \"(none)\") from clients').fetchone()[0]); print('ledger rows:', c.execute('select count(*) from ledger').fetchone()[0])"
|
||||
|
||||
## Rotate SAAS_API_KEY
|
||||
|
||||
docker compose exec saas env # sanity: what the container sees
|
||||
# edit .env (or re-export) with the new key, then:
|
||||
docker compose up -d # recreates the container with the new env
|
||||
|
||||
Verify the old key now 401s and the new one works:
|
||||
|
||||
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/api/clients -H 'X-API-Key: old-key'
|
||||
curl -s http://127.0.0.1:8080/api/clients -H 'X-API-Key: new-key'
|
||||
|
||||
## When it is down — checklist
|
||||
|
||||
1. `docker compose ps` — is the container running? `restart: unless-stopped`
|
||||
brings it back after most exits; a container stuck in `Restarting` loops
|
||||
is crashing at boot.
|
||||
2. `docker compose logs --tail=200 saas` — traceback? DB path error?
|
||||
`PermissionError` on `/data`? If the volume mount is missing the app
|
||||
still starts but the ledger is ephemeral — check `ls` inside.
|
||||
3. `curl -fsS http://127.0.0.1:8080/health` — health is public and keyless;
|
||||
a time-out here while the container is up means the app is not listening
|
||||
(bind/port issue): `docker compose port saas 8080`, `ss -tlnp | grep 8080`.
|
||||
4. Authed calls 401? Wrong/rotated key. Authed calls 503? `SAAS_API_KEY` is
|
||||
unset/empty in the container env — fail-closed on purpose.
|
||||
5. Ledger questions: `docker compose exec saas python -c "import sqlite3; c=sqlite3.connect('/data/saas.db'); print(c.execute('PRAGMA integrity_check').fetchone())"`
|
||||
— should print `('ok',)`. Check the volume's disk space:
|
||||
`df -h /var/lib/docker/volumes/gitea-saas_saas_data/_data`.
|
||||
6. Still broken and the ledger looks intact? Restore from the last verified
|
||||
backup (above) after confirming the backup file itself passes
|
||||
`integrity_check`.
|
||||
|
||||
## Scaling (and why not to do it casually)
|
||||
|
||||
This deployment is intentionally a single SQLite writer. To grow you must
|
||||
move the ledger off SQLite onto a network-capable database (Postgres etc.)
|
||||
first — raising `--workers` against SQLite will corrupt or lock the ledger.
|
||||
Until then: one worker, threads for concurrency, backups before traffic.
|
||||
41
docker-compose.yml
Normal file
41
docker-compose.yml
Normal file
@ -0,0 +1,41 @@
|
||||
# gitea-saas-core deployment (issue #40).
|
||||
#
|
||||
# The MEANING OF THE VOLUME (read before changing it):
|
||||
# The service stores its money ledger in SQLite at /data/saas.db. The
|
||||
# named volume `saas_data` is therefore the ENTIRE durability story:
|
||||
# losing the volume loses the ledger. `docker compose down` keeps the
|
||||
# volume. `docker compose down -v` DESTROYS it — never run that without
|
||||
# a verified backup (OPS.md "Backup"/"Restore"). Backups before traffic.
|
||||
#
|
||||
# SQLite is a single-writer store: the container runs exactly one gunicorn
|
||||
# worker (see Dockerfile) so all writes funnel through one process lock.
|
||||
# Do not raise --workers here.
|
||||
name: gitea-saas
|
||||
|
||||
services:
|
||||
saas:
|
||||
build: .
|
||||
image: gitea-saas-core:local
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8080:8080"
|
||||
environment:
|
||||
# Empty by default on purpose: with no key the app still boots and
|
||||
# /health stays public, but every authed route fails closed with 503
|
||||
# (see gitea_saas/app.py). Set a real key before serving traffic:
|
||||
# docker compose --env-file .env up -d
|
||||
# with SAAS_API_KEY=<secret> in .env, or export SAAS_API_KEY first.
|
||||
SAAS_API_KEY: ${SAAS_API_KEY:-}
|
||||
SAAS_DB_PATH: ${SAAS_DB_PATH:-/data/saas.db}
|
||||
SAAS_PORT: "8080"
|
||||
volumes:
|
||||
- saas_data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import sys,urllib.request; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=5).status == 200 else 1)"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
start_period: 10s
|
||||
retries: 3
|
||||
|
||||
volumes:
|
||||
saas_data:
|
||||
6
gitea_saas/__main__.py
Normal file
6
gitea_saas/__main__.py
Normal file
@ -0,0 +1,6 @@
|
||||
"""``python -m gitea_saas`` dev server (issue #38)."""
|
||||
|
||||
from .app import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
177
gitea_saas/app.py
Normal file
177
gitea_saas/app.py
Normal file
@ -0,0 +1,177 @@
|
||||
"""Assembly: one ``create_app()`` + the entrypoints for the whole service (#38).
|
||||
|
||||
The 15 modules on ``main`` compose into a single deployable Flask app in
|
||||
exactly one place:
|
||||
|
||||
billing API ``gitea_saas.server.create_app`` (issues #1/#2/#6)
|
||||
factory ``gitea_saas.intake_factory.register_factory_routes`` (#4)
|
||||
dashboard ``gitea_saas.dashboard.mount_dashboard`` (issue #9)
|
||||
client API ``gitea_saas.client_api.register_client_routes`` (#41)
|
||||
|
||||
Assembly only: this module makes no behaviour changes to billing, factory,
|
||||
dashboard or review. If composing them ever surfaces a genuine
|
||||
incompatibility it is reported on the issue, not papered over here.
|
||||
|
||||
Fail-closed assembly: when ``SAAS_API_KEY`` is unset the app still starts
|
||||
(``python -m gitea_saas``, ``gitea-saas`` console script, gunicorn via the
|
||||
repo-root ``wsgi.py``) but every authed route returns 503
|
||||
``SERVICE_UNAVAILABLE``. No second auth path is invented here: key
|
||||
validation stays in ``gitea_saas.auth`` (issue #24) and each front-end's
|
||||
own ``auth.key_is_valid`` guard (canonical 401) applies unchanged as soon
|
||||
as a key IS configured. The 503 gate exists only at this assembly layer,
|
||||
and only for the unconfigured state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Callable, Optional
|
||||
|
||||
from flask import Flask, jsonify, request
|
||||
|
||||
from .auth import PUBLIC_PATHS
|
||||
from .billing_core import BillingCore
|
||||
from .client_api import (ClientKeyStore, ClientStatus, ClientSurface,
|
||||
register_client_routes)
|
||||
from .config import ServerConfig, load_config
|
||||
from .dashboard import mount_dashboard
|
||||
from .gitea_client import GiteaClient
|
||||
from .intake_factory import register_factory_routes
|
||||
from .server import create_app as create_billing_app
|
||||
from .stripe_client import build_stripe_client
|
||||
|
||||
# Canonical body for the unconfigured-deployment gate (status 503). Same
|
||||
# ``{"error": {"code", "message"}}`` shape as ``auth.UNAUTHORIZED_BODY`` so
|
||||
# clients parse every front-end's auth failure identically.
|
||||
UNAVAILABLE_BODY = {
|
||||
"error": {
|
||||
"code": "SERVICE_UNAVAILABLE",
|
||||
"message": "SAAS_API_KEY is unset: the service is not configured "
|
||||
"for authenticated use (fail-closed)",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _resolve_db_path(cfg: ServerConfig) -> Path:
|
||||
"""The runtime SQLite path shared by billing, factory and dashboard.
|
||||
|
||||
``config.py`` defaults an unset ``SAAS_DB_PATH`` to in-memory, but the
|
||||
factory and the dashboard open their OWN connections to the same file,
|
||||
so an ephemeral FILE (in a fresh temp dir) keeps all three components
|
||||
on one coherent dataset for the no-env dev run. No new env vars: this
|
||||
is the assembler's default for the existing unset state.
|
||||
"""
|
||||
if cfg.db_path is not None:
|
||||
return Path(cfg.db_path)
|
||||
scratch = Path(tempfile.mkdtemp(prefix="gitea-saas-"))
|
||||
return scratch / "saas.db"
|
||||
|
||||
|
||||
def _unconfigured_guard(configured: bool):
|
||||
"""before_request that fail-closes an UNCONFIGURED deployment.
|
||||
|
||||
Registered ONLY when no API key is configured, and inserted ahead of
|
||||
``server.create_app``'s own before_request so the unconfigured state
|
||||
is reported as 503 (this issue's acceptance 2: not 500, not 200)
|
||||
instead of the front-ends' 401s. It performs no key comparison of its
|
||||
own -- validation stays in ``gitea_saas.auth.key_is_valid`` (issue
|
||||
#24) -- and it fires only for non-public paths, so the liveness routes
|
||||
in ``auth.PUBLIC_PATHS`` (/health, /healthz, /api/factory/health)
|
||||
keep serving. With a configured key this guard is never installed.
|
||||
"""
|
||||
if configured:
|
||||
return None
|
||||
|
||||
def _guard():
|
||||
if request.path.rstrip("/") in PUBLIC_PATHS:
|
||||
return None
|
||||
resp = jsonify(UNAVAILABLE_BODY)
|
||||
resp.status_code = 503
|
||||
return resp
|
||||
|
||||
return _guard
|
||||
|
||||
|
||||
def create_app(config: Optional[ServerConfig] = None, *,
|
||||
stripe_builder: Optional[Callable[[], object]] = None,
|
||||
gitea: Optional[object] = None) -> Flask:
|
||||
"""Build the ONE Flask app: billing + factory + dashboard + client surface.
|
||||
|
||||
``config`` is a :class:`~gitea_saas.config.ServerConfig` (or any object
|
||||
exposing ``api_key`` / ``port`` / ``db_path``); when None the
|
||||
environment is loaded through the existing ``config.load_config``
|
||||
loader (no new env vars).
|
||||
|
||||
``stripe_builder`` is a zero-arg callable returning a Stripe seam (mock
|
||||
or real) for the client surface (issue #41); it defaults to the
|
||||
env-driven ``stripe_client.build_stripe_client``, which is fail-closed:
|
||||
with no credentials and no explicit ``SAAS_STRIPE_MODE=mock`` it raises,
|
||||
the surface answers checkout/webhook with 503 ``stripe_unconfigured``,
|
||||
and the app still starts. ``gitea`` defaults to a real ``GiteaClient``
|
||||
(factory + client work-submission seam).
|
||||
|
||||
The DB is opened via ``gitea_saas.db`` (through ``BillingCore``, which
|
||||
owns the connection + idempotent schema bootstrap) and shared by all
|
||||
front-ends. With no ``SAAS_API_KEY`` the app still starts; authed
|
||||
routes return 503 until a key is configured (module docstring).
|
||||
"""
|
||||
cfg = config if config is not None else load_config()
|
||||
api_key = getattr(cfg, "api_key", None) or ""
|
||||
db_path = _resolve_db_path(cfg)
|
||||
core = BillingCore(db_path)
|
||||
|
||||
# The factory's IntakeFactory is built per request and reads
|
||||
# ``config.db_path`` (env fallback is the pre-assembly behaviour);
|
||||
# hand it the RESOLVED path so the per-request construction cannot
|
||||
# raise "IntakeFactory requires config.db_path" for the no-env case.
|
||||
runtime = SimpleNamespace(db_path=db_path, api_key=api_key)
|
||||
|
||||
gitea_client = gitea if gitea is not None else GiteaClient()
|
||||
|
||||
app = create_billing_app(core, api_key)
|
||||
register_factory_routes(app, core, gitea_client, runtime)
|
||||
mount_dashboard(app, core=core, db_path=db_path, api_key=api_key)
|
||||
|
||||
# Issue #41 client surface: per-client keys, checkout, /api/me,
|
||||
# self-serve work submission and the Stripe webhook. Mounting it
|
||||
# replaces the billing app's operator-key before_request with the
|
||||
# compound assembly guard (see client_api.register_client_routes) whose
|
||||
# operator-route semantics are identical — with the scoped client
|
||||
# endpoints reachable and valid client keys answered 403 on operator
|
||||
# routes (never 401, never passed through).
|
||||
key_store = ClientKeyStore(db_path)
|
||||
status = ClientStatus(core, db_path,
|
||||
getattr(cfg, "debit_floor_minor", None))
|
||||
surface = ClientSurface(core, key_store, status,
|
||||
stripe_builder or build_stripe_client)
|
||||
register_client_routes(app, surface, gitea_client, runtime)
|
||||
|
||||
guard = _unconfigured_guard(bool(api_key))
|
||||
if guard is not None:
|
||||
# Global before_request funcs run in list order; position 0 puts
|
||||
# the unconfigured gate ahead of server.create_app's own guard.
|
||||
# A returning response short-circuits the rest of the request, so
|
||||
# authed routes report the deployment state before any front-end
|
||||
# 401 logic runs. When a key IS configured this list is never
|
||||
# touched and the front-ends' auth.key_is_valid guards decide.
|
||||
app.before_request_funcs.setdefault(None, []).insert(0, guard)
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Dev entrypoint (``python -m gitea_saas`` / ``gitea-saas`` script).
|
||||
|
||||
Starts even with no ``SAAS_API_KEY`` (fail-closed 503, per the issue);
|
||||
binds loopback like the pre-assembly server entrypoint. Production
|
||||
runs under gunicorn against the repo-root ``wsgi.py`` instead.
|
||||
"""
|
||||
cfg = load_config()
|
||||
app = create_app(cfg)
|
||||
app.run(host="127.0.0.1", port=cfg.port)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
138
gitea_saas/auth.py
Normal file
138
gitea_saas/auth.py
Normal file
@ -0,0 +1,138 @@
|
||||
"""Single X-API-Key auth path for the whole service (issue #24).
|
||||
|
||||
Every API-key front-end (``server.py``, ``delivery.py``, ``intake_factory.py``,
|
||||
``dashboard.py``, ``client_api.py``) delegates to this module; the header name, the
|
||||
constant-time comparison and the 401 body live exactly once here (no front-end forks them).
|
||||
|
||||
``stripe_client.py`` verifies webhook *signatures* (a different secret on a different
|
||||
verification path) and is explicitly out of scope for this issue — it keeps its own signature check.
|
||||
|
||||
Public surface:
|
||||
API_KEY_HEADER the only ``X-API-Key`` header literal in the package
|
||||
PUBLIC_PATHS deliberately unauthenticated liveness/health routes + the Stripe webhook (#41)
|
||||
key_is_valid() constant-time, fail-closed check — the one API-key
|
||||
comparison in the whole service
|
||||
UNAUTHORIZED_BODY canonical 401 payload, byte-identical on every wire
|
||||
unauthorized_response() Flask Response object carrying that body
|
||||
FORBIDDEN_BODY canonical 403 payload (issue #41 cross-tenant denial)
|
||||
forbidden_response() Flask Response object carrying that body
|
||||
hash_client_key() SHA-256 hash for per-client keys (#41) — stored, never plaintext
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
from typing import Optional
|
||||
|
||||
# The one place the HTTP header literal lives (acceptance criterion 2: one literal).
|
||||
API_KEY_HEADER = "X-API-Key"
|
||||
|
||||
# Routes that need no operator API key; every front-end guard reads this set,
|
||||
# so exposure is declared here, not inside one PR's before_request (composition
|
||||
# finding 2026-08-26): /health, /healthz, /api/factory/health are liveness
|
||||
# routes; /api/webhooks/stripe (#41) is signature-authenticated.
|
||||
PUBLIC_PATHS = frozenset({
|
||||
"/health",
|
||||
"/healthz",
|
||||
"/api/factory/health",
|
||||
"/api/webhooks/stripe",
|
||||
})
|
||||
|
||||
# Canonical 401 payload. All front-ends return exactly this (Flask
|
||||
# `jsonify(UNAUTHORIZED_BODY)` for HTTP surfaces, the dict itself for the
|
||||
# delivery `(result, status)` seam), so a client parses every front-end's
|
||||
# auth failure identically (acceptance criterion 4).
|
||||
UNAUTHORIZED_BODY = {
|
||||
"error": {
|
||||
"code": "UNAUTHORIZED",
|
||||
"message": "missing or invalid API key",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def key_is_valid(provided_key: Optional[str],
|
||||
configured_key: Optional[str]) -> bool:
|
||||
"""Fail-closed, constant-time API-key check (single implementation).
|
||||
|
||||
A request is authorized only when a key is actually configured AND the
|
||||
provided header value matches it exactly. An unset/empty or non-str
|
||||
configured key refuses every request; non-str/empty provided values are
|
||||
rejected before the comparison runs (the constant-time compare raises
|
||||
``TypeError`` on non-str input).
|
||||
|
||||
This is the ONE API-key comparison in the service (acceptance criterion
|
||||
1) — no front-end does its own.
|
||||
"""
|
||||
if not isinstance(configured_key, str) or not configured_key:
|
||||
return False
|
||||
if not isinstance(provided_key, str) or not provided_key:
|
||||
return False
|
||||
return hmac.compare_digest(provided_key, configured_key)
|
||||
|
||||
|
||||
def unauthorized_response():
|
||||
"""Flask Response with the canonical 401 body (status 401)."""
|
||||
from flask import jsonify
|
||||
|
||||
resp = jsonify(UNAUTHORIZED_BODY)
|
||||
resp.status_code = 401
|
||||
return resp
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Issue #41 additions — the client-facing surface's auth pieces.
|
||||
#
|
||||
# Deliberately placed BELOW key_is_valid/unauthorized_response so the issue
|
||||
# #24 acceptance pin (exactly one compare_digest call site in this file, at
|
||||
# auth.py:70) is untouched: per-client key verification adds NO comparison
|
||||
# call site. A presented client key is hashed (hash_client_key) and resolved
|
||||
# by indexed equality on the full 64-char digest in the client_api_keys
|
||||
# table (client_api.ClientKeyStore.resolve_client_id). The digest is
|
||||
# one-way, so even a timing leak on digest equality reveals nothing about
|
||||
# the key — the discipline that matters is: full-length digest, and no
|
||||
# plaintext ever stored or compared.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Canonical 403 payload (issue #41): a VALID key presented against a route
|
||||
# its tenant is not allowed to reach (another client's resources, or an
|
||||
# operator-only route). Distinguished from 401 (no/wrong key) so a client
|
||||
# can tell "authenticate" from "authenticated but forbidden".
|
||||
FORBIDDEN_BODY = {
|
||||
"error": {
|
||||
"code": "FORBIDDEN",
|
||||
"message": "this key is not authorized for this resource",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def forbidden_response():
|
||||
"""Flask Response with the canonical 403 body (status 403, issue #41).
|
||||
|
||||
Used when a key is VALID but its tenant is not allowed on the requested
|
||||
route: another client's resources or an operator-only route. The body
|
||||
is byte-identical on every front-end so a client parses it once.
|
||||
"""
|
||||
from flask import jsonify
|
||||
|
||||
resp = jsonify(FORBIDDEN_BODY)
|
||||
resp.status_code = 403
|
||||
return resp
|
||||
|
||||
|
||||
CLIENT_KEY_HASH_ALGORITHM = "sha256"
|
||||
|
||||
|
||||
def hash_client_key(raw_key: str) -> str:
|
||||
"""SHA-256 hex digest of a per-client API key (issue #41).
|
||||
|
||||
The ONLY place a client key is hashed. ``raw_key`` must be a
|
||||
non-empty str (a non-str raises ``TypeError`` from ``hashlib`` — the
|
||||
caller validates before hashing).
|
||||
"""
|
||||
import hashlib
|
||||
|
||||
if not isinstance(raw_key, str) or not raw_key:
|
||||
raise ValueError("client key must be a non-empty string")
|
||||
digest = hashlib.new(CLIENT_KEY_HASH_ALGORITHM)
|
||||
digest.update(raw_key.encode("utf-8"))
|
||||
return digest.hexdigest()
|
||||
672
gitea_saas/client_api.py
Normal file
672
gitea_saas/client_api.py
Normal file
@ -0,0 +1,672 @@
|
||||
"""Client-facing HTTP surface (issue #41): self-serve funding + work submission.
|
||||
|
||||
Until this module there was exactly one credential — the shared operator
|
||||
``SAAS_API_KEY`` — and every route was operator/admin. This module adds the
|
||||
first genuinely externally-reachable surface:
|
||||
|
||||
* **Per-client API keys** — ``ClientKeyStore``. Keys are random, scoped to
|
||||
exactly one ``client_id``, stored ONLY as a SHA-256 hash (the plaintext is
|
||||
returned exactly once, at issuance, then never seen again) and revocable.
|
||||
The hash function lives in ``gitea_saas.auth`` (issue #41 extension) and
|
||||
verification is digest-equality against the key table — this module does
|
||||
not fork auth, it builds on it.
|
||||
* **Checkout** — ``POST /api/checkout`` returns a Stripe Checkout URL for a
|
||||
top-up, rate-limited per client. The webhook below is the ONLY credit
|
||||
path: ``stripe_client.apply_payment_event`` (the issue-#3 seam) is reused,
|
||||
never a second credit implementation.
|
||||
* **Client status** — ``GET /api/me`` returns the caller's own balance,
|
||||
queue counts, in-flight count and delivered count (scoped to the caller's
|
||||
client_id by construction).
|
||||
* **Work submission** — ``POST /api/me/issues`` submits work as the caller's
|
||||
OWN client_id through the issue-#4 factory (funded -> Gitea issue,
|
||||
unfunded/locked -> queued). A client never supplies another tenant's id.
|
||||
* **Stripe webhook** — ``POST /api/webhooks/stripe``, authenticated by
|
||||
Stripe's signature (never by an API key), credits idempotently.
|
||||
|
||||
Tenancy & guard composition (the subtle part)
|
||||
---------------------------------------------
|
||||
``server.create_app`` installs an operator-key ``before_request`` guard that
|
||||
401s every non-``PUBLIC_PATHS`` route — it would make the client surface
|
||||
unreachable, since client routes are authenticated by a DIFFERENT key class.
|
||||
Mounting the client surface therefore REPLACES that single global guard
|
||||
with a compound assembly guard (registered in ``register_client_routes``);
|
||||
operator-route behaviour is preserved EXACTLY (same ``PUBLIC_PATHS`` set,
|
||||
same ``auth.key_is_valid``, same canonical 401):
|
||||
|
||||
1. ``PUBLIC_PATHS`` (liveness + the signature-authenticated webhook) pass
|
||||
straight through.
|
||||
2. A valid client key on a ``CLIENT_PATHS`` route is stored in
|
||||
``flask.g.client_id`` and allowed.
|
||||
3. A valid client key ANYWHERE ELSE (``/api/clients``, ``/api/factory/*``,
|
||||
``/dashboard``, key issuance) is forbidden: canonical 403 body.
|
||||
4. No/invalid client key: client-surface routes get the canonical 401;
|
||||
operator routes fall through to the operator check, which decides
|
||||
exactly as before (the unconfigured-deployment 503 gate from
|
||||
``app.create_app`` runs ahead of everything and still short-circuits
|
||||
every non-public route first).
|
||||
|
||||
Consequences that are deliberate:
|
||||
* The operator key never works on the client surface (401), and a client key
|
||||
never works on operator routes (403).
|
||||
* The operator key-issuance routes (``/api/clients/<id>/keys``) are NOT part
|
||||
of the client surface — they stay behind the operator guard, and a client
|
||||
key hitting them gets 403.
|
||||
* The existing ``/api/factory/<id>/*`` routes stay operator-only. Clients
|
||||
submit work through ``/api/me/issues`` instead, so the factory module is
|
||||
untouched and its per-route operator check cannot be bypassed.
|
||||
|
||||
Fail-closed anywhere a secret could be at stake: an unconfigured Stripe
|
||||
deployment answers checkout/webhook with 503 ``stripe_unconfigured`` (never
|
||||
a fallback secret, never a 200); nothing in this module ever logs a key, a
|
||||
Stripe secret, or a webhook payload.
|
||||
|
||||
Hermetic by construction (same convention as the other front-ends): the
|
||||
billing core, key store and status provider are injected; the Stripe client
|
||||
comes from an injected builder (the mock is selected only on explicit opt-in
|
||||
per ``stripe_client.build_stripe_client``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
from collections import deque
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
from flask import Flask, g, jsonify, request
|
||||
|
||||
from .auth import (API_KEY_HEADER, PUBLIC_PATHS, forbidden_response,
|
||||
hash_client_key, key_is_valid, unauthorized_response)
|
||||
from .intake_factory import (CoreContractError, IntakeFactory,
|
||||
InvalidRequestError, UnknownClientError)
|
||||
|
||||
# Canonical prefix for generated client keys (display-friendly, never used
|
||||
# for auth — the full key is required and only its hash is stored).
|
||||
CLIENT_KEY_PREFIX = "saas_"
|
||||
CLIENT_KEY_BYTES = 32
|
||||
|
||||
# The routes that belong to a client's OWN surface (self-scoped by the
|
||||
# caller's key). A valid client key is allowed here and only here; on every
|
||||
# other route it is forbidden (403). Keep in sync with the routes below.
|
||||
CLIENT_PATHS = frozenset({
|
||||
"/api/checkout",
|
||||
"/api/me",
|
||||
"/api/me/issues",
|
||||
})
|
||||
|
||||
STRIPE_WEBHOOK_ROUTE = "/api/webhooks/stripe"
|
||||
|
||||
# Rate limit for POST /api/checkout per client (fixed per-client window).
|
||||
DEFAULT_CHECKOUT_RATE_LIMIT = 5
|
||||
DEFAULT_CHECKOUT_RATE_WINDOW_S = 60
|
||||
|
||||
# Mirror of intake_factory's repo-name rule (issue #4 contract).
|
||||
_REPO_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$")
|
||||
|
||||
# Idempotent DDL mirror of the schema.sql addition (same convention as
|
||||
# intake_factory._FACTORY_DDL: this module can be constructed against a
|
||||
# fresh file even before the billing core has run schema.sql).
|
||||
_KEYS_DDL = """
|
||||
CREATE TABLE IF NOT EXISTS client_api_keys (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
client_id INTEGER NOT NULL REFERENCES clients(id),
|
||||
key_hash TEXT NOT NULL UNIQUE,
|
||||
key_prefix TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
revoked_at TEXT
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_client_api_keys_client ON client_api_keys (client_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_client_api_keys_hash ON client_api_keys (key_hash);
|
||||
"""
|
||||
|
||||
|
||||
def _json_error(status: int, code: str, message: str):
|
||||
"""Canonical structured error (same shape as auth.UNAUTHORIZED_BODY)."""
|
||||
resp = jsonify({"error": {"code": code, "message": message}})
|
||||
resp.status_code = status
|
||||
return resp
|
||||
|
||||
|
||||
class _RateLimiter:
|
||||
"""Sliding-window request limiter, keyed by tenant (in-process).
|
||||
|
||||
Single-process service (gunicorn sync workers run one app instance per
|
||||
worker); the limiter is per-process by design. ``allow(key)`` returns
|
||||
``(allowed, retry_after_s)`` — retry_after is 0 when allowed.
|
||||
"""
|
||||
|
||||
def __init__(self, limit: int, window_s: float):
|
||||
self._limit = max(int(limit), 1)
|
||||
self._window_s = float(window_s)
|
||||
self._hits: Dict[Any, deque] = {}
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def allow(self, key: Any) -> tuple[bool, float]:
|
||||
now = time.monotonic()
|
||||
with self._lock:
|
||||
dq = self._hits.setdefault(key, deque())
|
||||
while dq and now - dq[0] >= self._window_s:
|
||||
dq.popleft()
|
||||
if len(dq) >= self._limit:
|
||||
retry = self._window_s - (now - dq[0])
|
||||
return False, max(retry, 1.0)
|
||||
dq.append(now)
|
||||
return True, 0.0
|
||||
|
||||
|
||||
class ClientKeyStore:
|
||||
"""Per-client API keys on the shared SQLite file: issue/list/revoke/resolve.
|
||||
|
||||
At-rest policy (acceptance criterion 2): ONLY ``hash_client_key`` output
|
||||
is stored — the plaintext is returned once at issuance and never written.
|
||||
A resolver looks the presented key up by its hash; a revoked or deleted
|
||||
client's keys stop resolving. ``list()`` returns metadata only (id,
|
||||
prefix, timestamps) — never hashes, never plaintext.
|
||||
"""
|
||||
|
||||
def __init__(self, db_path):
|
||||
self._conn = sqlite3.connect(str(Path(db_path)), check_same_thread=False)
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA foreign_keys=ON")
|
||||
self._conn.executescript(_KEYS_DDL)
|
||||
self._lock = threading.RLock()
|
||||
|
||||
# -- helpers ----------------------------------------------------------
|
||||
|
||||
def _client_exists(self, client_id: int) -> bool:
|
||||
row = self._conn.execute(
|
||||
"SELECT 1 FROM clients WHERE id = ?", (client_id,)).fetchone()
|
||||
return row is not None
|
||||
|
||||
@staticmethod
|
||||
def _generate() -> str:
|
||||
return CLIENT_KEY_PREFIX + secrets.token_urlsafe(CLIENT_KEY_BYTES)
|
||||
|
||||
# -- operations -------------------------------------------------------
|
||||
|
||||
def issue(self, client_id: int) -> tuple[bool, dict]:
|
||||
"""Create a key for ``client_id``; plaintext returned ONCE.
|
||||
|
||||
Returns ``(True, {"key_id", "client_id", "key", "prefix",
|
||||
"created_at"})`` or ``(False, {"error": ...})`` for an unknown
|
||||
client. The plaintext ``key`` is not stored and cannot be recovered.
|
||||
"""
|
||||
if isinstance(client_id, bool) or not isinstance(client_id, int) \
|
||||
or client_id <= 0:
|
||||
return False, {"error": {"code": "UNKNOWN_CLIENT",
|
||||
"message": "client_id must be a positive integer"}}
|
||||
with self._lock:
|
||||
if not self._client_exists(client_id):
|
||||
return False, {"error": {"code": "UNKNOWN_CLIENT",
|
||||
"message": f"no client with id {client_id}"}}
|
||||
raw = self._generate()
|
||||
now = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
cur = self._conn.execute(
|
||||
"INSERT INTO client_api_keys (client_id, key_hash, key_prefix,"
|
||||
" created_at) VALUES (?, ?, ?, ?)",
|
||||
(client_id, hash_client_key(raw), raw[:10], now))
|
||||
self._conn.commit()
|
||||
return True, {"key_id": cur.lastrowid, "client_id": client_id,
|
||||
"key": raw, "prefix": raw[:10], "created_at": now}
|
||||
|
||||
def list_keys(self, client_id: int) -> tuple[bool, dict]:
|
||||
"""Key metadata for a client (never hashes, never plaintext)."""
|
||||
with self._lock:
|
||||
if not self._client_exists(client_id):
|
||||
return False, {"error": {"code": "UNKNOWN_CLIENT",
|
||||
"message": f"no client with id {client_id}"}}
|
||||
rows = self._conn.execute(
|
||||
"SELECT id, key_prefix, created_at, revoked_at"
|
||||
" FROM client_api_keys WHERE client_id = ? ORDER BY id",
|
||||
(client_id,)).fetchall()
|
||||
keys = [{"key_id": int(r["id"]), "prefix": r["key_prefix"],
|
||||
"created_at": r["created_at"], "revoked_at": r["revoked_at"]}
|
||||
for r in rows]
|
||||
return True, {"client_id": client_id, "keys": keys, "count": len(keys)}
|
||||
|
||||
def revoke(self, client_id: int, key_id: int) -> tuple[bool, dict]:
|
||||
"""Revoke exactly one key that BELONGS to ``client_id`` (scoped)."""
|
||||
with self._lock:
|
||||
if not self._client_exists(client_id):
|
||||
return False, {"error": {"code": "UNKNOWN_CLIENT",
|
||||
"message": f"no client with id {client_id}"}}
|
||||
now = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
cur = self._conn.execute(
|
||||
"UPDATE client_api_keys SET revoked_at = ?"
|
||||
" WHERE id = ? AND client_id = ? AND revoked_at IS NULL",
|
||||
(now, key_id, client_id))
|
||||
self._conn.commit()
|
||||
if cur.rowcount == 0:
|
||||
return False, {"error": {"code": "KEY_NOT_FOUND",
|
||||
"message": "no active key with that id for this client"}}
|
||||
return True, {"key_id": key_id, "client_id": client_id,
|
||||
"revoked_at": now, "revoked": True}
|
||||
|
||||
def revoke_all(self, client_id: int) -> tuple[bool, dict]:
|
||||
"""Revoke every active key of ``client_id`` (idempotent)."""
|
||||
with self._lock:
|
||||
if not self._client_exists(client_id):
|
||||
return False, {"error": {"code": "UNKNOWN_CLIENT",
|
||||
"message": f"no client with id {client_id}"}}
|
||||
now = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
cur = self._conn.execute(
|
||||
"UPDATE client_api_keys SET revoked_at = ?"
|
||||
" WHERE client_id = ? AND revoked_at IS NULL",
|
||||
(now, client_id))
|
||||
self._conn.commit()
|
||||
return True, {"client_id": client_id, "revoked_count": cur.rowcount}
|
||||
|
||||
def resolve_client_id(self, provided_key: Optional[str]) -> Optional[int]:
|
||||
"""The client_id owning ``provided_key``, or None (fail-closed).
|
||||
|
||||
Hashes the presented key and looks the row up by hash; revoked keys
|
||||
and keys whose client no longer exists never resolve. Returns None
|
||||
for a missing/malformed key, a revoked key, or an unknown client.
|
||||
"""
|
||||
if not isinstance(provided_key, str) or not provided_key:
|
||||
return None
|
||||
try:
|
||||
digest = hash_client_key(provided_key)
|
||||
except ValueError:
|
||||
return None
|
||||
try:
|
||||
row = self._conn.execute(
|
||||
"SELECT k.client_id FROM client_api_keys k"
|
||||
" JOIN clients c ON c.id = k.client_id"
|
||||
" WHERE k.key_hash = ? AND k.revoked_at IS NULL",
|
||||
(digest,)).fetchone()
|
||||
except sqlite3.Error:
|
||||
return None
|
||||
return int(row["client_id"]) if row is not None else None
|
||||
|
||||
|
||||
class ClientStatus:
|
||||
"""Client-visible status: balance + queue/in-flight/delivered counts.
|
||||
|
||||
Reads the shared SQLite file like ``Dashboard`` does (its own read
|
||||
connection); balance comes from the billing core (single source of
|
||||
truth, holds included). Definitions (documented in the API):
|
||||
|
||||
* ``queued_items`` — ``pending_issues`` rows still waiting in the
|
||||
intake queue (unfunded/locked submits, issue #4).
|
||||
* ``in_flight_items`` — ``pending_issues`` rows the factory has marked
|
||||
``shipped`` (flushed from the queue into Gitea, awaiting
|
||||
merge/delivery). A direct funded submit is created in Gitea without
|
||||
a local row and is therefore not counted here — the factory tracks
|
||||
neither it nor a delivery-complete state, so this count is defined by
|
||||
the data the system records.
|
||||
* ``delivered_count`` — distinct delivery debits in the ledger
|
||||
(``ref_type`` ``pr``/``delivery`` — the only DB record of a delivery).
|
||||
"""
|
||||
|
||||
DEFAULT_DEBIT_FLOOR_MINOR = 500
|
||||
|
||||
def __init__(self, core: Any, db_path, debit_floor_minor: Optional[int] = None):
|
||||
self._core = core
|
||||
self._conn = sqlite3.connect(str(Path(db_path)))
|
||||
self._conn.row_factory = sqlite3.Row
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
if debit_floor_minor is None:
|
||||
raw = os.environ.get("SAAS_DEBIT_FLOOR_MINOR", "").strip()
|
||||
debit_floor_minor = int(raw) if raw else self.DEFAULT_DEBIT_FLOOR_MINOR
|
||||
self._floor = debit_floor_minor
|
||||
|
||||
def _table_exists(self, name: str) -> bool:
|
||||
row = self._conn.execute(
|
||||
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?",
|
||||
(name,)).fetchone()
|
||||
return row is not None
|
||||
|
||||
def summary(self, client_id: int) -> tuple[bool, dict]:
|
||||
ok, details = self._core.get_client(client_id, include_balance=True)
|
||||
if not ok or not isinstance(details, dict):
|
||||
return False, details if isinstance(details, dict) \
|
||||
else {"error": {"code": "UNKNOWN_CLIENT",
|
||||
"message": f"no client with id {client_id}"}}
|
||||
cid = int(details.get("id", client_id))
|
||||
balance = details.get("balance")
|
||||
locked = False
|
||||
if self._table_exists("factory_locks"):
|
||||
try:
|
||||
locked = self._conn.execute(
|
||||
"SELECT 1 FROM factory_locks WHERE client_id = ?",
|
||||
(cid,)).fetchone() is not None
|
||||
except Exception:
|
||||
locked = False
|
||||
if locked:
|
||||
status = "locked"
|
||||
elif isinstance(balance, int) and balance < self._floor:
|
||||
status = "overdue"
|
||||
else:
|
||||
status = str(details.get("status") or "active")
|
||||
|
||||
queued = in_flight = delivered = 0
|
||||
if self._table_exists("pending_issues"):
|
||||
try:
|
||||
row = self._conn.execute(
|
||||
"SELECT status, COUNT(*) AS n FROM pending_issues"
|
||||
" WHERE client_id = ? GROUP BY status", (cid,)).fetchall()
|
||||
counts = {str(r["status"]): int(r["n"]) for r in row}
|
||||
queued = counts.get("queued", 0)
|
||||
in_flight = counts.get("shipped", 0)
|
||||
except Exception:
|
||||
queued = in_flight = 0
|
||||
try:
|
||||
delivered = int(self._conn.execute(
|
||||
"SELECT COUNT(DISTINCT ref_id) FROM ledger"
|
||||
" WHERE client_id = ? AND kind = 'debit'"
|
||||
" AND ref_type IN ('pr', 'delivery') AND ref_id IS NOT NULL",
|
||||
(cid,)).fetchone()[0])
|
||||
except Exception:
|
||||
delivered = 0
|
||||
return True, {
|
||||
"client_id": cid,
|
||||
"name": details.get("name"),
|
||||
"tier": details.get("tier"),
|
||||
"status": status,
|
||||
"balance_minor": balance,
|
||||
"currency": "USD",
|
||||
"queued_items": queued,
|
||||
"in_flight_items": in_flight,
|
||||
"delivered_count": delivered,
|
||||
}
|
||||
|
||||
|
||||
class ClientSurface:
|
||||
"""State shared by the client routes: guard, keys, status, Stripe. """
|
||||
|
||||
def __init__(self, core: Any, key_store: ClientKeyStore,
|
||||
status: ClientStatus,
|
||||
stripe_builder: Callable[[], Any],
|
||||
checkout_rate: int = DEFAULT_CHECKOUT_RATE_LIMIT,
|
||||
checkout_window_s: float = DEFAULT_CHECKOUT_RATE_WINDOW_S):
|
||||
self.core = core
|
||||
self.keys = key_store
|
||||
self.status = status
|
||||
self.stripe_builder = stripe_builder
|
||||
self._limiter = _RateLimiter(checkout_rate, checkout_window_s)
|
||||
|
||||
# -- request-time helpers --------------------------------------------
|
||||
|
||||
def stripe(self) -> Optional[Any]:
|
||||
"""A usable Stripe client, or None when unconfigured (fail-closed).
|
||||
|
||||
The builder is env-driven (``stripe_client.build_stripe_client`` by
|
||||
default): a deployment without Stripe credentials, or without an
|
||||
EXPLICIT mock opt-in, gets None here — the routes answer 503, never
|
||||
a fallback secret and never a 200 with data.
|
||||
"""
|
||||
try:
|
||||
return self.stripe_builder()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def limiter_allow(self, client_id: int) -> tuple[bool, float]:
|
||||
return self._limiter.allow(client_id)
|
||||
|
||||
# -- the tenant-aware decision (used by the compound assembly guard) ---
|
||||
|
||||
def check_client_key(self, path: str) -> Optional[tuple]:
|
||||
"""Decide the client-surface part of a request's auth.
|
||||
|
||||
Returns ``None`` when the request is not a client-surface matter
|
||||
(operator routes are left to the operator check), or a tuple:
|
||||
|
||||
* ``("forbid", Flask-Response)`` — a VALID client key presented
|
||||
outside its own surface (403, cross-tenant / operator route), or
|
||||
a client-surface path without a valid client key (401);
|
||||
* ``("allowed", client_id)`` — a valid client key on its own
|
||||
surface; the caller stores the client_id for the route.
|
||||
"""
|
||||
provided = request.headers.get(API_KEY_HEADER, "")
|
||||
client_id = self.keys.resolve_client_id(provided)
|
||||
if client_id is not None:
|
||||
if path in CLIENT_PATHS: # the key's own surface
|
||||
return ("allowed", client_id)
|
||||
return ("forbid", forbidden_response()) # wrong tenant/route
|
||||
if path in CLIENT_PATHS: # client surface needs a client key
|
||||
return ("forbid", unauthorized_response())
|
||||
return None # operator routes: their check decides
|
||||
|
||||
|
||||
def _map_key_store_error(details: dict):
|
||||
code = (details.get("error") or {}).get("code")
|
||||
message = (details.get("error") or {}).get("message") or "key operation failed"
|
||||
if code == "UNKNOWN_CLIENT":
|
||||
return _json_error(404, "unknown_client", message)
|
||||
if code == "KEY_NOT_FOUND":
|
||||
return _json_error(404, "key_not_found", message)
|
||||
return _json_error(500, "key_store_error", message)
|
||||
|
||||
|
||||
def _map_factory_error(exc: Exception):
|
||||
"""Map IntakeFactory domain errors (mirror of intake_factory's mapper)."""
|
||||
if isinstance(exc, UnknownClientError):
|
||||
return _json_error(404, "unknown_client", str(exc))
|
||||
if isinstance(exc, (InvalidRequestError, ValueError)):
|
||||
return _json_error(400, "invalid_request", str(exc))
|
||||
if isinstance(exc, CoreContractError):
|
||||
return _json_error(500, "core_contract", str(exc))
|
||||
return _json_error(500, "factory_error", f"{type(exc).__name__}: {exc}")
|
||||
|
||||
|
||||
def register_client_routes(app: Flask, surface: ClientSurface,
|
||||
gitea: Any, config: Any) -> Flask:
|
||||
"""Mount the client-facing surface on an existing Flask app.
|
||||
|
||||
``surface`` carries core/keys/status/Stripe; ``gitea`` and ``config``
|
||||
feed the per-request ``IntakeFactory`` for ``/api/me/issues`` (same
|
||||
seam the issue-#4 factory routes use).
|
||||
|
||||
Registers the tenant-aware guard AHEAD of every other before_request
|
||||
(including the operator-key guard installed by ``server.create_app``) —
|
||||
see the module docstring: valid client keys must be answered 403 on
|
||||
operator routes, and the client surface must be reachable even though
|
||||
the operator guard would otherwise 401 it.
|
||||
"""
|
||||
core = surface.core
|
||||
|
||||
# -- the tenant-aware guard --------------------------------------------
|
||||
# server.create_app installs an operator-key before_request that 401s
|
||||
# every non-PUBLIC_PATHS route — including the client surface, whose
|
||||
# auth is a DIFFERENT key class. Mounting the client surface therefore
|
||||
# REPLACES that single global guard with a compound assembly guard:
|
||||
# 1. PUBLIC_PATHS (liveness + the signature-authenticated webhook)
|
||||
# pass straight through;
|
||||
# 2. the client-surface decision wins when it is one (403 for a
|
||||
# valid client key off its surface / on operator routes, 401 on
|
||||
# the client surface without a valid key);
|
||||
# 3. otherwise the operator check runs with EXACTLY server.create_app's
|
||||
# semantics (same PUBLIC_PATHS set, same auth.key_is_valid, same
|
||||
# canonical 401 body) — no behaviour change for operator routes.
|
||||
# The unconfigured-deployment 503 gate (app.py) is inserted at position
|
||||
# 0 AFTER this, so it still short-circuits everything non-public first.
|
||||
|
||||
app.before_request_funcs.setdefault(None, []).clear()
|
||||
|
||||
def _assembly_guard():
|
||||
path = request.path.rstrip("/")
|
||||
if path in PUBLIC_PATHS:
|
||||
return None
|
||||
decision = surface.check_client_key(path)
|
||||
if decision is not None:
|
||||
outcome, payload = decision
|
||||
if outcome == "forbid":
|
||||
return payload # 403 / 401 decision (see above)
|
||||
g.client_id = payload # allowed on the client's own surface
|
||||
return None
|
||||
# Operator routes: server.create_app's check, preserved verbatim.
|
||||
sent = request.headers.get(API_KEY_HEADER, "")
|
||||
expected = app.config.get("API_KEY", "")
|
||||
if key_is_valid(sent, expected):
|
||||
return None
|
||||
return unauthorized_response()
|
||||
|
||||
app.before_request_funcs.setdefault(None, []).insert(0, _assembly_guard)
|
||||
|
||||
# -- self-serve checkout ---------------------------------------------
|
||||
|
||||
@app.post("/api/checkout")
|
||||
def checkout():
|
||||
client_id = g.get("client_id")
|
||||
if client_id is None:
|
||||
return unauthorized_response()
|
||||
allowed, retry_after = surface.limiter_allow(client_id)
|
||||
if not allowed:
|
||||
resp = _json_error(429, "rate_limited",
|
||||
"too many checkout requests; retry shortly")
|
||||
resp.headers["Retry-After"] = str(int(retry_after))
|
||||
return resp
|
||||
body = request.get_json(silent=True)
|
||||
if not isinstance(body, dict):
|
||||
return _json_error(400, "invalid_request",
|
||||
"request body must be a JSON object")
|
||||
amount = body.get("amount_minor")
|
||||
if isinstance(amount, bool) or not isinstance(amount, int) or amount <= 0:
|
||||
return _json_error(400, "invalid_request",
|
||||
"'amount_minor' must be a positive integer (minor units)")
|
||||
currency = body.get("currency") or "usd"
|
||||
if (not isinstance(currency, str) or len(currency) != 3
|
||||
or not currency.isalpha()):
|
||||
return _json_error(400, "invalid_request",
|
||||
"'currency' must be a 3-letter code")
|
||||
success_url = body.get("success_url")
|
||||
cancel_url = body.get("cancel_url")
|
||||
for name, value in (("success_url", success_url),
|
||||
("cancel_url", cancel_url)):
|
||||
if value is not None and (
|
||||
not isinstance(value, str)
|
||||
or not re.match(r"^https?://", value.strip())):
|
||||
return _json_error(400, "invalid_request",
|
||||
f"'{name}' must be an absolute http(s) URL")
|
||||
stripe = surface.stripe()
|
||||
if stripe is None:
|
||||
return _json_error(503, "stripe_unconfigured",
|
||||
"Stripe is not configured on this deployment (fail-closed)")
|
||||
try:
|
||||
result = stripe.create_checkout_session(
|
||||
client_id, amount, currency.lower(),
|
||||
success_url.strip() if isinstance(success_url, str) else None,
|
||||
cancel_url.strip() if isinstance(cancel_url, str) else None)
|
||||
except Exception:
|
||||
return _json_error(503, "stripe_error",
|
||||
"checkout session could not be created")
|
||||
resp = jsonify(result)
|
||||
resp.status_code = 201
|
||||
return resp
|
||||
|
||||
# -- Stripe webhook (signature-authenticated; NOT an API-key route) ---
|
||||
|
||||
@app.post(STRIPE_WEBHOOK_ROUTE)
|
||||
def stripe_webhook():
|
||||
stripe = surface.stripe()
|
||||
if stripe is None:
|
||||
return _json_error(503, "stripe_unconfigured",
|
||||
"Stripe is not configured on this deployment (fail-closed)")
|
||||
payload = request.get_data()
|
||||
signature = request.headers.get("Stripe-Signature", "")
|
||||
try:
|
||||
event = stripe.process_webhook(payload, signature)
|
||||
except Exception:
|
||||
# Signature failures and malformed payloads are indistinguishable
|
||||
# on purpose: never leak whether the signature was the problem.
|
||||
return _json_error(400, "invalid_signature",
|
||||
"webhook signature verification failed")
|
||||
from .stripe_client import apply_payment_event
|
||||
ok, details = apply_payment_event(core, event)
|
||||
if not ok:
|
||||
message = ((details or {}).get("error") or {}).get("message") \
|
||||
or "event could not be applied"
|
||||
return _json_error(400, "event_rejected", message)
|
||||
# Success path: idempotent credit via the issue-#3 seam. The full
|
||||
# payload is never logged and never echoed.
|
||||
return jsonify({"received": True,
|
||||
"applied": bool(details.get("applied", True)),
|
||||
"event_id": event.get("event_id")})
|
||||
|
||||
# -- client status & self-serve work submission -----------------------
|
||||
|
||||
@app.get("/api/me")
|
||||
def me():
|
||||
client_id = g.get("client_id")
|
||||
if client_id is None:
|
||||
return unauthorized_response()
|
||||
ok, details = surface.status.summary(client_id)
|
||||
if not ok:
|
||||
return _json_error(404, "unknown_client",
|
||||
f"no client with id {client_id}")
|
||||
return jsonify(details)
|
||||
|
||||
@app.post("/api/me/issues")
|
||||
def me_submit_issue():
|
||||
client_id = g.get("client_id")
|
||||
if client_id is None:
|
||||
return unauthorized_response()
|
||||
body = request.get_json(silent=True)
|
||||
if not isinstance(body, dict):
|
||||
return _json_error(400, "invalid_request",
|
||||
"request body must be a JSON object")
|
||||
repo = (body.get("repo") or "").strip()
|
||||
title = (body.get("title") or "").strip()
|
||||
acceptance_body = str(body.get("acceptance_body") or "")
|
||||
labels = body.get("labels")
|
||||
if not isinstance(labels, list):
|
||||
labels = None
|
||||
labels = [str(label) for label in labels] if labels else None
|
||||
if not _REPO_NAME_RE.match(repo):
|
||||
return _json_error(400, "invalid_request",
|
||||
f"invalid repo name: {repo!r}")
|
||||
if not title:
|
||||
return _json_error(400, "invalid_request",
|
||||
"issue title must be a non-empty string")
|
||||
try:
|
||||
factory = IntakeFactory(core, gitea, config)
|
||||
result = factory.submit_issue(client_id, repo, title,
|
||||
acceptance_body, labels)
|
||||
except Exception as exc:
|
||||
return _map_factory_error(exc)
|
||||
resp = jsonify(result)
|
||||
resp.status_code = 202 if not result.get("delivered") else 201
|
||||
return resp
|
||||
|
||||
# -- operator-only key management (behind the operator guard) ---------
|
||||
# These are NOT client-surface routes: the tenant-aware guard above
|
||||
# forbids a client key here (403), and the operator guard requires the
|
||||
# shared SAAS_API_KEY.
|
||||
|
||||
@app.post("/api/clients/<int:client_id>/keys")
|
||||
def issue_client_key(client_id: int):
|
||||
ok, details = surface.keys.issue(client_id)
|
||||
if not ok:
|
||||
return _map_key_store_error(details)
|
||||
resp = jsonify(details)
|
||||
resp.status_code = 201
|
||||
return resp
|
||||
|
||||
@app.get("/api/clients/<int:client_id>/keys")
|
||||
def list_client_keys(client_id: int):
|
||||
ok, details = surface.keys.list_keys(client_id)
|
||||
if not ok:
|
||||
return _map_key_store_error(details)
|
||||
return jsonify(details)
|
||||
|
||||
@app.delete("/api/clients/<int:client_id>/keys/<int:key_id>")
|
||||
def revoke_client_key(client_id: int, key_id: int):
|
||||
ok, details = surface.keys.revoke(client_id, key_id)
|
||||
if not ok:
|
||||
return _map_key_store_error(details)
|
||||
return jsonify(details)
|
||||
|
||||
@app.delete("/api/clients/<int:client_id>/keys")
|
||||
def revoke_all_client_keys(client_id: int):
|
||||
ok, details = surface.keys.revoke_all(client_id)
|
||||
if not ok:
|
||||
return _map_key_store_error(details)
|
||||
return jsonify(details)
|
||||
|
||||
return app
|
||||
@ -5,8 +5,9 @@ coordinated between issues #2 and #6 — see botmanager #21 for the ownership
|
||||
pattern that motivated it):
|
||||
|
||||
* the HTTP server's frozen :class:`ServerConfig` (issue #2) — the billing
|
||||
API's API-key discipline (``SAAS_API_KEY`` / ``X-API-Key`` header), the
|
||||
port, and the DB path; and
|
||||
API's API-key discipline (``SAAS_API_KEY`` env; the ``X-API-Key`` header
|
||||
constant itself is owned by ``gitea_saas.auth``, issue #24), the port, and
|
||||
the DB path; and
|
||||
* the review-gate/policy tuning (issue #6): tier rates/policies
|
||||
(``SAAS_TIERS_JSON``) plus the reviewer knobs (``REVIEW_*``), returned as
|
||||
a plain dict by :func:`load_review_config`.
|
||||
@ -32,16 +33,20 @@ DEFAULT_PORT = 8080
|
||||
# (in-memory) SQLite database. Real deployments must set SAAS_DB_PATH.
|
||||
DEFAULT_DB_PATH: Optional[Path] = None
|
||||
|
||||
# Header name carrying the API key on every authenticated request.
|
||||
API_KEY_HEADER = "X-API-Key"
|
||||
# Header name carrying the API key on every authenticated request. Owned by
|
||||
# ``gitea_saas.auth`` (issue #24 consolidation); re-exported here so existing
|
||||
# ``from .config import API_KEY_HEADER`` call sites keep working without a
|
||||
# second copy of the literal.
|
||||
from .auth import API_KEY_HEADER
|
||||
|
||||
# Tier defaults (issue #6): GOLD auto-merges on a green APPROVE at rate 100
|
||||
# minor units per delivery; STANDARD is reviewed but operator-merged at rate
|
||||
# 25; FREE is never auto-merged and never debited. Rates are config, not
|
||||
# code — SAAS_TIERS_JSON overrides merge over these defaults.
|
||||
# Tier defaults (issue #6, pricing 2026-09-01): GOLD auto-merges on a green
|
||||
# APPROVE at rate 500 minor units ($5.00) per delivery; STANDARD is reviewed
|
||||
# but operator-merged at rate 500 ($5.00); FREE is never auto-merged and
|
||||
# never debited. Rates are config, not code — SAAS_TIERS_JSON overrides
|
||||
# merge over these defaults.
|
||||
DEFAULT_TIERS = {
|
||||
"gold": {"auto_merge": True, "rate_minor": 100},
|
||||
"standard": {"auto_merge": False, "rate_minor": 25},
|
||||
"gold": {"auto_merge": True, "rate_minor": 500},
|
||||
"standard": {"auto_merge": False, "rate_minor": 500},
|
||||
"free": {"auto_merge": False, "rate_minor": 0},
|
||||
}
|
||||
|
||||
@ -116,6 +121,8 @@ def load_review_config(env: Optional[Mapping[str, str]] = None) -> dict:
|
||||
| REVIEW_TEST_TIMEOUT_S | test/build timeout seconds | "300" |
|
||||
| REVIEW_SECRET_SCAN | secret scan on/off ("0" disables) | "1" |
|
||||
| REVIEW_WORKTREES_DIR | worktree root where PRs are checked out | (none) |
|
||||
| REVIEW_CLONE_DIR | READ-ONLY repository clone used to build the | (none) |
|
||||
| | isolated synthetic-merge review workspace | |
|
||||
"""
|
||||
source: Mapping[str, str] = os.environ if env is None else env
|
||||
timeout = 300
|
||||
@ -134,4 +141,5 @@ def load_review_config(env: Optional[Mapping[str, str]] = None) -> dict:
|
||||
"test_timeout_s": timeout,
|
||||
"secret_scan": source.get("REVIEW_SECRET_SCAN", "1") != "0",
|
||||
"worktrees_dir": source.get("REVIEW_WORKTREES_DIR") or None,
|
||||
"review_clone_dir": source.get("REVIEW_CLONE_DIR") or None,
|
||||
}
|
||||
|
||||
@ -30,7 +30,6 @@ serves the exact same dict.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import html
|
||||
import os
|
||||
import sqlite3
|
||||
@ -38,13 +37,9 @@ from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
EMPTY = "\u2014" # em dash empty-state marker
|
||||
from .auth import API_KEY_HEADER, key_is_valid, unauthorized_response
|
||||
|
||||
# Single source for the API-key header name on this branch. The shared
|
||||
# `config.API_KEY_HEADER` constant lands with issue #2/#6 (neither merged
|
||||
# yet); when #24 consolidates the service's auth paths, this line is
|
||||
# replaced by `from .config import API_KEY_HEADER` — no other change.
|
||||
API_KEY_HEADER = "X-API-Key"
|
||||
EMPTY = "\u2014" # em dash empty-state marker
|
||||
|
||||
DEFAULT_DEBIT_FLOOR_MINOR = 500
|
||||
|
||||
@ -412,7 +407,7 @@ def render_html(summary: Dict[str, Any]) -> str:
|
||||
<table><tr><th>Repo</th><th>PR</th><th>Verdict</th><th>Head</th><th>Reviewed at</th></tr>{rows_rg}</table>
|
||||
<h2>5 · Ledger flash</h2>
|
||||
<table><tr><th>Client</th><th>Kind</th><th>Amount</th><th>Ref</th><th>Time</th></tr>{rows_lf}</table>
|
||||
<p class="note">Listen-only page: bind the server to 127.0.0.1. Live numbers, no caching.</p>
|
||||
<p class="note">Operator page — requires the same API key as the JSON summary. Live numbers, no caching.</p>
|
||||
</body></html>"""
|
||||
|
||||
|
||||
@ -435,11 +430,14 @@ def mount_dashboard(app, core=None, db_path=None, api_key=None):
|
||||
|
||||
Registered routes:
|
||||
|
||||
* ``GET /dashboard`` — HTML operator page (NO auth; the page renders a
|
||||
loopback-only warning — the server must bind to 127.0.0.1).
|
||||
* ``GET /api/dashboard/summary`` — JSON payload (``X-API-Key`` auth;
|
||||
401 on missing/wrong key, 503 when no key is configured — fail-closed).
|
||||
* ``GET /healthz`` — liveness.
|
||||
* ``GET /dashboard`` — HTML operator page (the issue #25 guard, expressed
|
||||
through ``gitea_saas.auth``: constant-time ``key_is_valid`` check, any
|
||||
refusal — missing, wrong, or unset configured key — is the canonical
|
||||
401 body; never billing data without a valid key).
|
||||
* ``GET /api/dashboard/summary`` — JSON payload (``X-API-Key`` auth via
|
||||
``gitea_saas.auth``; any refusal — missing, wrong, or unset configured
|
||||
key — is the canonical 401 body, fail-closed).
|
||||
* ``GET /healthz`` — liveness (public, in ``auth.PUBLIC_PATHS``).
|
||||
|
||||
``api_key`` fills ``app.config['API_KEY']`` only when unset, so the app
|
||||
factory's key wins when both are configured.
|
||||
@ -462,37 +460,30 @@ def mount_dashboard(app, core=None, db_path=None, api_key=None):
|
||||
|
||||
dashboard = Dashboard(core, db_path)
|
||||
|
||||
def _guard(view):
|
||||
from functools import wraps
|
||||
|
||||
@wraps(view)
|
||||
def guarded(*args, **kwargs):
|
||||
configured = app.config.get("API_KEY")
|
||||
if not configured:
|
||||
return {"error": {"code": "API_KEY_NOT_CONFIGURED",
|
||||
"message": "SAAS_API_KEY not set"}}, 503
|
||||
provided = request.headers.get(API_KEY_HEADER, "")
|
||||
if not hmac.compare_digest(provided, configured):
|
||||
return {"error": {"code": "UNAUTHORIZED",
|
||||
"message": "bad or missing X-API-Key"}}, 401
|
||||
return view(*args, **kwargs)
|
||||
return guarded
|
||||
|
||||
@app.get("/healthz")
|
||||
def healthz():
|
||||
return jsonify({"ok": True})
|
||||
|
||||
@app.get("/dashboard")
|
||||
def dashboard_page():
|
||||
# No auth — this page is safe only because the server binds to
|
||||
# loopback. Recompute from the DB at request time (no caching).
|
||||
# Guarded through the shared auth module (issues #24/#25): the same
|
||||
# constant-time key_is_valid check and canonical 401 body as every
|
||||
# other API-key route — never billing data without a valid key.
|
||||
# Loopback binding is defense-in-depth only; it was never enforced,
|
||||
# so it is NOT the authorization boundary. Recompute from the DB at
|
||||
# request time (no caching).
|
||||
if not key_is_valid(request.headers.get(API_KEY_HEADER, ""),
|
||||
app.config.get("API_KEY")):
|
||||
return unauthorized_response()
|
||||
summary = dashboard.summary()
|
||||
return (render_html(summary), 200,
|
||||
{"Content-Type": "text/html; charset=utf-8"})
|
||||
|
||||
@app.get("/api/dashboard/summary")
|
||||
@_guard
|
||||
def dashboard_summary():
|
||||
if not key_is_valid(request.headers.get(API_KEY_HEADER, ""),
|
||||
app.config.get("API_KEY")):
|
||||
return unauthorized_response()
|
||||
return jsonify(dashboard.summary())
|
||||
|
||||
return app
|
||||
|
||||
@ -28,26 +28,26 @@ a malformed payload, 200 otherwise) -- the server.py webhook route
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
from typing import Any, Optional
|
||||
|
||||
from .auth import UNAUTHORIZED_BODY, key_is_valid
|
||||
|
||||
# --- defaults (overridable via env, mirror issue #6's config.py) ----------
|
||||
|
||||
DEFAULT_TIERS: dict[str, dict[str, Any]] = {
|
||||
"gold": {"rate_minor": 100}, # $1.00 / issue
|
||||
"standard": {"rate_minor": 25}, # $0.25 / issue
|
||||
"gold": {"rate_minor": 500}, # $5.00 / issue
|
||||
"standard": {"rate_minor": 500}, # $5.00 / issue
|
||||
"free": {"rate_minor": 0}, # never debited
|
||||
}
|
||||
DEFAULT_RATE_MINOR = 100 # SAAS_RATE_MINOR (100 = $1.00 / issue)
|
||||
DEFAULT_RATE_MINOR = 500 # SAAS_RATE_MINOR (500 = $5.00 / issue)
|
||||
DEFAULT_FLOOR_MINOR = 500 # SAAS_DEBIT_FLOOR_MINOR ($5.00; matches issue #4)
|
||||
|
||||
KEY_PREFIX = "deliver"
|
||||
|
||||
# Error codes (structured, mirror billing-core conventions).
|
||||
UNAUTHORIZED = "UNAUTHORIZED"
|
||||
BAD_PAYLOAD = "BAD_PAYLOAD"
|
||||
UNKNOWN_CLIENT = "UNKNOWN_CLIENT"
|
||||
INTERNAL = "INTERNAL"
|
||||
@ -120,22 +120,14 @@ class Delivery:
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def authorized(self, provided_key: Optional[str]) -> bool:
|
||||
"""Fail-closed, constant-time: a request is authorized only when
|
||||
the configured ``SAAS_API_KEY`` is set and matches the provided
|
||||
key exactly.
|
||||
"""Fail-closed, constant-time API-key check.
|
||||
|
||||
The comparison uses :func:`hmac.compare_digest` (no timing side
|
||||
channel on the shared secret's prefix). ``compare_digest`` raises
|
||||
``TypeError`` on non-str input, so the ``isinstance`` guard runs
|
||||
first — unset/empty config and non-str/empty keys all return
|
||||
``False``.
|
||||
Single implementation (issue #24): delegates to the shared
|
||||
``gitea_saas.auth.key_is_valid`` — no comparison lives in this
|
||||
module. A request is authorized only when ``SAAS_API_KEY`` is set
|
||||
and matches the provided key exactly.
|
||||
"""
|
||||
expected = self.config.get("api_key")
|
||||
if not expected:
|
||||
return False
|
||||
if not isinstance(provided_key, str) or not provided_key:
|
||||
return False
|
||||
return hmac.compare_digest(provided_key, expected)
|
||||
return key_is_valid(provided_key, self.config.get("api_key"))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# The debit primitive
|
||||
@ -279,8 +271,9 @@ class Delivery:
|
||||
fields, 200 otherwise.
|
||||
"""
|
||||
if not self.authorized(provided_key):
|
||||
return ({"ok": False, "error": {"code": UNAUTHORIZED,
|
||||
"message": "bad or missing X-API-Key"}}), 401
|
||||
# Canonical 401 body from gitea_saas.auth (issue #24): identical
|
||||
# on every front-end's wire.
|
||||
return (UNAUTHORIZED_BODY, 401)
|
||||
if not isinstance(payload, dict):
|
||||
return ({"ok": False, "error": {"code": BAD_PAYLOAD,
|
||||
"message": "expected JSON object"}}), 400
|
||||
|
||||
@ -1,20 +1,62 @@
|
||||
"""Gitea API client (mockable seam, env token).
|
||||
|
||||
Shared by the factory front-end (issue #4) and the review gate (issue #5).
|
||||
Real HTTP only when a token is available; the token is read from
|
||||
``GITEA_TOKEN`` (or passed in) and is never printed or placed on a command
|
||||
line. Tests use an in-memory fake, so this module is never exercised over
|
||||
the network in CI.
|
||||
Shared by the factory front-end (issue #4), the review gate (issue #5), and
|
||||
the production review-policy client operations (issue #28: listing open PRs,
|
||||
label mutations, merge, issue close). Real HTTP only when a token is
|
||||
available; the token is read from ``GITEA_TOKEN`` (or passed in) and is
|
||||
never printed, placed on a command line, or echoed in errors. Tests use a
|
||||
local HTTP stub or an in-memory fake, so this module is never exercised
|
||||
against the live instance in CI.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from typing import Any, Optional
|
||||
|
||||
# Gitea list endpoints cap at 50 items per page (the API's ``limit`` bound).
|
||||
_PER_PAGE = 50
|
||||
# Safety valve so a misbehaving server that never returns a short page
|
||||
# cannot make ``_paginate`` loop forever.
|
||||
_MAX_PAGES = 100
|
||||
|
||||
|
||||
class GiteaClientError(Exception):
|
||||
"""Base error for the Gitea client.
|
||||
|
||||
Messages NEVER include the token or the ``Authorization`` header (the
|
||||
client redacts the token from any response text it copies into the
|
||||
message).
|
||||
"""
|
||||
|
||||
|
||||
class GiteaApiError(GiteaClientError):
|
||||
"""A non-2xx Gitea API response.
|
||||
|
||||
``message`` carries the method, API path, HTTP status, and a truncated
|
||||
response body — deliberately no request headers, so the authorization
|
||||
header can never leak.
|
||||
"""
|
||||
|
||||
def __init__(self, method: str, api_path: str, status: int, detail: str = ""):
|
||||
self.method = method
|
||||
self.api_path = api_path
|
||||
self.status = status
|
||||
self.detail = detail
|
||||
message = f"Gitea API {method} {api_path} returned HTTP {status}"
|
||||
if detail:
|
||||
message += f": {_truncate(detail, 300)}"
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
def _truncate(text: str, limit: int = 300) -> str:
|
||||
text = (text or "").strip()
|
||||
return text if len(text) <= limit else text[:limit] + "…"
|
||||
|
||||
|
||||
class GiteaClient:
|
||||
"""Thin urllib client for the subset of the Gitea API the SaaS needs.
|
||||
@ -24,6 +66,11 @@ class GiteaClient:
|
||||
build ``/api/v1/repos/{owner}/{repo}/...``; org-repo creation uses
|
||||
``/api/v1/orgs/{owner}/repos`` and falls back to the authenticated
|
||||
user's repos when ``owner`` is a user account (orgs endpoint 404s).
|
||||
|
||||
Every non-2xx response raises :class:`GiteaApiError` (never silently
|
||||
ignored); the review-policy protocol (issue #28) is fail-closed: an
|
||||
unknown label or a failed post-condition verification raises
|
||||
:class:`GiteaClientError` instead of skipping the operation.
|
||||
"""
|
||||
|
||||
def __init__(self, base_url: Optional[str] = None, token: Optional[str] = None,
|
||||
@ -36,7 +83,12 @@ class GiteaClient:
|
||||
# -- low-level ---------------------------------------------------------
|
||||
|
||||
def _api(self, method: str, api_path: str, body: Optional[dict] = None) -> Any:
|
||||
"""Perform an API call under ``/api/v1/{api_path}`` and parse JSON."""
|
||||
"""Perform an API call under ``/api/v1/{api_path}`` and parse JSON.
|
||||
|
||||
Non-2xx responses raise :class:`GiteaApiError` with the method, API
|
||||
path, status, and a redacted response snippet — the authorization
|
||||
header (and therefore the token) never appears in the message.
|
||||
"""
|
||||
data = json.dumps(body).encode() if body is not None else None
|
||||
url = f"{self.base_url}/api/v1/{api_path}"
|
||||
req = urllib.request.Request(
|
||||
@ -44,14 +96,85 @@ class GiteaClient:
|
||||
headers={"Authorization": "token " + self.token,
|
||||
"Accept": "application/json",
|
||||
"Content-Type": "application/json"})
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
raw = resp.read()
|
||||
return json.loads(raw) if raw else None
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
raw = resp.read()
|
||||
return json.loads(raw) if raw else None
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = ""
|
||||
try:
|
||||
raw_body = exc.read()
|
||||
except Exception: # noqa: BLE001 - the body may be unreadable; fail cleanly
|
||||
raw_body = b""
|
||||
if raw_body:
|
||||
text = raw_body.decode("utf-8", "replace")
|
||||
# Defensive: some servers echo the request (including its
|
||||
# headers) back in the error body — redact the token value
|
||||
# and any echoed Authorization header regardless.
|
||||
if self.token:
|
||||
text = text.replace(self.token, "***")
|
||||
# Scrub any echoed Authorization header completely (name and
|
||||
# value), not only the token — the header itself must not
|
||||
# appear in errors.
|
||||
text = re.sub(
|
||||
r"(?i)authorization\s*[:=]\s*(?:token\s+)?[^\s,}\]]+",
|
||||
"***", text)
|
||||
detail = _truncate(text, 300)
|
||||
raise GiteaApiError(method, api_path, exc.code, detail) from exc
|
||||
|
||||
def _request(self, method: str, path: str, body: Optional[dict] = None) -> Any:
|
||||
"""Repo-scoped call: ``path`` = ``<repo>/...`` under the owner."""
|
||||
return self._api(method, f"repos/{self.owner}/{path}", body)
|
||||
|
||||
def _paginate(self, api_path: str) -> list:
|
||||
"""Fetch every page of a Gitea list endpoint (``GET``).
|
||||
|
||||
Follows Gitea pagination (``page``/``limit`` query parameters) until
|
||||
a page returns fewer than ``_PER_PAGE`` items; a defensive page cap
|
||||
raises instead of looping forever on a pathologically misbehaving
|
||||
server.
|
||||
"""
|
||||
collected: list = []
|
||||
for page in range(1, _MAX_PAGES + 1):
|
||||
sep = "&" if "?" in api_path else "?"
|
||||
batch = self._api("GET", f"{api_path}{sep}page={page}&limit={_PER_PAGE}")
|
||||
if not isinstance(batch, list):
|
||||
raise GiteaClientError(
|
||||
f"expected a JSON list from {api_path!r}, got {type(batch).__name__}")
|
||||
collected.extend(batch)
|
||||
if len(batch) < _PER_PAGE:
|
||||
break
|
||||
else:
|
||||
raise GiteaClientError(
|
||||
f"pagination did not terminate after {_MAX_PAGES} pages of {api_path!r}")
|
||||
return collected
|
||||
|
||||
def _normalize_pr(self, pr: dict, repo: str = "") -> dict:
|
||||
"""Map a raw Gitea PR record to the candidate shape.
|
||||
|
||||
Every candidate supplies at least ``number``, ``body``, ``head_sha``,
|
||||
``base_ref``, ``labels`` (label NAMES), ``repo``, ``merged``, and
|
||||
``state`` — the shape ``ReviewPolicy`` and the review gate consume.
|
||||
``merged``/``state`` are the raw Gitea record's values (``merged`` is
|
||||
a boolean; only ``merged is True`` proves a merge — ``state ==
|
||||
"closed"`` alone can also mean closed without merging).
|
||||
"""
|
||||
head = pr.get("head") or {}
|
||||
base = pr.get("base") or {}
|
||||
labels = [label.get("name") for label in (pr.get("labels") or [])
|
||||
if isinstance(label, dict) and label.get("name")]
|
||||
return {
|
||||
"number": pr.get("number"),
|
||||
"body": pr.get("body") or "",
|
||||
"head_sha": head.get("sha"),
|
||||
"base_sha": base.get("sha"),
|
||||
"base_ref": base.get("ref"),
|
||||
"labels": labels,
|
||||
"repo": repo,
|
||||
"merged": bool(pr.get("merged")),
|
||||
"state": pr.get("state"),
|
||||
}
|
||||
|
||||
# -- API surface used by the factory front-end (issue #4) --------------
|
||||
|
||||
def repo_exists(self, name: str) -> bool:
|
||||
@ -62,8 +185,8 @@ class GiteaClient:
|
||||
"""
|
||||
try:
|
||||
self._request("GET", f"{name}")
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
except GiteaApiError as exc:
|
||||
if exc.status == 404:
|
||||
return False
|
||||
raise
|
||||
return True
|
||||
@ -86,8 +209,8 @@ class GiteaClient:
|
||||
payload: dict = {"name": name, "auto_init": False, "private": True}
|
||||
try:
|
||||
return self._api("POST", f"orgs/{self.owner}/repos", payload)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code not in (404, 422):
|
||||
except GiteaApiError as exc:
|
||||
if exc.status not in (404, 422):
|
||||
raise
|
||||
# ``owner`` is a user account (or lacks org-create rights): fall back
|
||||
# to creating the repo under the authenticated token user.
|
||||
@ -138,13 +261,20 @@ class GiteaClient:
|
||||
return self._request("GET", f"{repo}/issues/{number}") or {}
|
||||
|
||||
def get_pr(self, repo: str, number: int) -> dict:
|
||||
"""Fetch a PR: {head_sha, changed_files: [filename, ...]}."""
|
||||
"""Fetch a PR in the normalized candidate shape.
|
||||
|
||||
``{number, body, head_sha, base_sha, base_ref, labels: [names],
|
||||
repo, merged, state, changed_files: [filename, ...]}`` — the review
|
||||
gate consumes ``head_sha``/``base_sha`` (the exact evaluated pair,
|
||||
issue #30) and ``changed_files``; policy consumes the rest
|
||||
(``merged``/``state`` let recovery positively distinguish a merged
|
||||
PR from a merely closed one).
|
||||
"""
|
||||
pr = self._request("GET", f"{repo}/pulls/{number}") or {}
|
||||
files = self._request("GET", f"{repo}/pulls/{number}/files") or []
|
||||
return {
|
||||
"head_sha": ((pr.get("head") or {}).get("sha")),
|
||||
"changed_files": [f.get("filename") for f in files if f.get("filename")],
|
||||
}
|
||||
normalized = self._normalize_pr(pr, repo)
|
||||
normalized["changed_files"] = [f.get("filename") for f in files if f.get("filename")]
|
||||
return normalized
|
||||
|
||||
def get_pr_diff(self, repo: str, number: int) -> str:
|
||||
"""Return the PR's diff text reconstructed from per-file patches."""
|
||||
@ -160,3 +290,129 @@ class GiteaClient:
|
||||
def post_comment(self, repo: str, number: int, body: str) -> dict:
|
||||
"""Post a comment on an issue or PR conversation."""
|
||||
return self._request("POST", f"{repo}/issues/{number}/comments", {"body": body}) or {}
|
||||
|
||||
# -- API surface used by the review policy (issue #28) ------------------
|
||||
#
|
||||
# Every operation in this section is fail-closed: a non-2xx response
|
||||
# raises GiteaApiError, an unknown label or a failed post-condition
|
||||
# verification raises GiteaClientError. Nothing is silently skipped.
|
||||
|
||||
def list_open_prs(self, repo: str) -> list:
|
||||
"""All open PRs in ``owner/repo``, normalized, pagination followed.
|
||||
|
||||
Requests ``?state=open`` (all and only open PRs) page by page and
|
||||
returns the normalized candidate shape (see ``_normalize_pr``).
|
||||
"""
|
||||
prs = self._paginate(f"repos/{self.owner}/{repo}/pulls?state=open")
|
||||
return [self._normalize_pr(pr, repo) for pr in prs]
|
||||
|
||||
def add_label(self, repo: str, issue_number: int, label: str) -> None:
|
||||
"""Add ``label`` to ``owner/repo`` issue ``issue_number``.
|
||||
|
||||
Resolves the repository label id, POSTs it (``{"labels": [id]}``),
|
||||
and verifies the resulting label state. Already-present labels are
|
||||
a no-op. An unknown label or a failed verification raises a clear
|
||||
:class:`GiteaClientError` — never silently skipped.
|
||||
"""
|
||||
name = (label or "").strip()
|
||||
if not name:
|
||||
raise GiteaClientError("add_label: label must be a non-empty string")
|
||||
label_id = self.label_id(repo, name)
|
||||
if label_id is None:
|
||||
raise GiteaClientError(
|
||||
f"add_label: unknown label {name!r} in repo {self.owner}/{repo}")
|
||||
if name in self._issue_label_names(repo, issue_number):
|
||||
return # already present: idempotent no-op
|
||||
self._request("POST", f"{repo}/issues/{issue_number}/labels", {"labels": [label_id]})
|
||||
if name not in self._issue_label_names(repo, issue_number):
|
||||
raise GiteaClientError(
|
||||
f"add_label: label {name!r} missing on issue {issue_number} after POST")
|
||||
|
||||
def remove_label(self, repo: str, issue_number: int, label: str) -> None:
|
||||
"""Remove ``label`` from ``owner/repo`` issue ``issue_number``.
|
||||
|
||||
Resolves the repository label id, DELETEs it, and verifies the
|
||||
resulting label state. Absent labels are a no-op. An unknown label
|
||||
or a failed verification raises a clear :class:`GiteaClientError`.
|
||||
"""
|
||||
name = (label or "").strip()
|
||||
if not name:
|
||||
raise GiteaClientError("remove_label: label must be a non-empty string")
|
||||
label_id = self.label_id(repo, name)
|
||||
if label_id is None:
|
||||
raise GiteaClientError(
|
||||
f"remove_label: unknown label {name!r} in repo {self.owner}/{repo}")
|
||||
if name not in self._issue_label_names(repo, issue_number):
|
||||
return # already absent: idempotent no-op
|
||||
self._api("DELETE",
|
||||
f"repos/{self.owner}/{repo}/issues/{issue_number}/labels/{label_id}")
|
||||
if name in self._issue_label_names(repo, issue_number):
|
||||
raise GiteaClientError(
|
||||
f"remove_label: label {name!r} still on issue {issue_number} after DELETE")
|
||||
|
||||
def merge_pr(self, repo: str, pr_number: int,
|
||||
expected_head_sha: Optional[str] = None) -> bool:
|
||||
"""Merge ``owner/repo`` PR ``pr_number``; True when it is merged.
|
||||
|
||||
Uses the Gitea merge endpoint and confirms the PR is merged after
|
||||
the request. The merge request carries the Gitea schema's
|
||||
mandatory ``Do`` mode and the reviewed ``head_commit_id``, so a
|
||||
branch that moved after review cannot be merged accidentally.
|
||||
|
||||
``expected_head_sha`` (optional, preserving the two-argument
|
||||
protocol) is the head the caller actually REVIEWED. When given,
|
||||
the pre-merge GET result is compared against it and a mismatch
|
||||
raises :class:`GiteaClientError` BEFORE any POST — closing the
|
||||
TOCTOU window between review and merge instead of only guarding
|
||||
the milliseconds between the in-method GET and POST. The
|
||||
reviewed SHA itself is sent as ``head_commit_id``. When omitted,
|
||||
the live PR head is read (and pinned) as before, which keeps the
|
||||
existing callers working while the policy call sites migrate to
|
||||
passing the reviewed head.
|
||||
|
||||
An already-merged PR short-circuits to True (no duplicate merge
|
||||
request). Non-2xx responses (e.g. 409 when the head is not
|
||||
mergeable) raise :class:`GiteaApiError`; a PR record without a
|
||||
head commit id raises :class:`GiteaClientError` rather than
|
||||
merging unpinned.
|
||||
"""
|
||||
current = self._request("GET", f"{repo}/pulls/{pr_number}") or {}
|
||||
if current.get("merged"):
|
||||
return True
|
||||
live_head = (current.get("head") or {}).get("sha")
|
||||
if expected_head_sha is not None and live_head != expected_head_sha:
|
||||
raise GiteaClientError(
|
||||
f"merge_pr: PR {pr_number} head is {live_head!r} but the "
|
||||
f"reviewed head is {expected_head_sha!r}; refusing to merge "
|
||||
"a branch that moved after review")
|
||||
head_commit_id = expected_head_sha if expected_head_sha is not None \
|
||||
else live_head
|
||||
if not head_commit_id:
|
||||
raise GiteaClientError(
|
||||
f"merge_pr: PR {pr_number} record has no head commit id; "
|
||||
"refusing to merge without the reviewed head")
|
||||
self._request("POST", f"{repo}/pulls/{pr_number}/merge",
|
||||
{"Do": "merge", "head_commit_id": head_commit_id})
|
||||
merged = self._request("GET", f"{repo}/pulls/{pr_number}") or {}
|
||||
return bool(merged.get("merged"))
|
||||
|
||||
def close_issue(self, repo: str, issue_number: int) -> None:
|
||||
"""Close ``owner/repo`` issue ``issue_number``.
|
||||
|
||||
Updates the issue state and confirms the issue reports ``closed``
|
||||
afterwards; a failed verification raises :class:`GiteaClientError`.
|
||||
"""
|
||||
self._request("PATCH", f"{repo}/issues/{issue_number}", {"state": "closed"})
|
||||
issue = self._request("GET", f"{repo}/issues/{issue_number}") or {}
|
||||
if issue.get("state") != "closed":
|
||||
raise GiteaClientError(
|
||||
f"close_issue: issue {issue_number} state is {issue.get('state')!r} "
|
||||
"after close request")
|
||||
|
||||
# -- shared helpers -----------------------------------------------------
|
||||
|
||||
def _issue_label_names(self, repo: str, issue_number: int) -> list:
|
||||
"""Label NAMES currently on ``owner/repo`` issue ``issue_number``."""
|
||||
issue = self._request("GET", f"{repo}/issues/{issue_number}") or {}
|
||||
return [label.get("name") for label in (issue.get("labels") or [])
|
||||
if isinstance(label, dict) and label.get("name")]
|
||||
|
||||
@ -10,12 +10,25 @@ file into ``SAAS_EVAL_OUTPUT_DIR``.
|
||||
|
||||
Money rule (fail-closed, mirroring issue #4's factory semantics): a batch
|
||||
is only accepted when the client's available balance is at or above the
|
||||
debit floor. Unfunded/unknown clients get a 402-style structured refusal
|
||||
and NO Gitea issues are ever created for them.
|
||||
debit floor. Unfunded/unknown clients get a 402-style structured refusal,
|
||||
a locally queued row (``pending_eval_batches``) and NO Gitea issues are
|
||||
ever created for them; once funding arrives, `flush_pending` ships the
|
||||
queued batch exactly like a fresh funded submit (issue #39).
|
||||
|
||||
Funds discipline (issue #39): a funded submit reserves a batch hold
|
||||
(configurable ``eval_batch_hold_minor`` / ``SAAS_EVAL_BATCH_HOLD_MINOR``,
|
||||
default = the debit floor) so the client's available balance visibly backs
|
||||
the in-flight batch. ``cancel_batch`` releases that hold and stops draining
|
||||
(a cancel marker; the batch no longer polls or emits). Rolled-back intake
|
||||
releases the hold too, so a failed submit never leaves money locked.
|
||||
|
||||
Schema additions are owned here (``schema.sql`` belongs to issue #1):
|
||||
``eval_batches`` and ``eval_tasks`` are bootstrapped idempotently against
|
||||
the same SQLite file as the billing core, exactly like the factory does.
|
||||
``eval_batches``, ``eval_tasks``, ``pending_eval_batches`` and
|
||||
``eval_batch_cancels`` are bootstrapped idempotently against the same
|
||||
SQLite file as the billing core, exactly like the factory does. The
|
||||
``hold_id`` column is added to pre-existing ``eval_batches`` tables via a
|
||||
guarded ``ALTER TABLE`` (SQLite cannot amend a CHECK constraint, but
|
||||
adding a column is idempotent and safe).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@ -30,10 +43,21 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
# Single auth path (issue #24): the header name, the constant-time comparison
|
||||
# and the 401 body all live in ``gitea_saas.auth``; this module delegates and
|
||||
# never implements its own key check (issue #39 "do not add a sixth").
|
||||
from .auth import API_KEY_HEADER, key_is_valid, unauthorized_response
|
||||
|
||||
# Default funded threshold: a client must hold at least 5.00 USD before an
|
||||
# eval batch is accepted (same default as the factory front-end, issue #4).
|
||||
DEFAULT_DEBIT_FLOOR_MINOR = 500
|
||||
|
||||
# Default per-batch hold reserved at submit (issue #39): the client's
|
||||
# available balance visibly backs the in-flight batch until it emits or is
|
||||
# cancelled. Config attr ``eval_batch_hold_minor`` / env
|
||||
# ``SAAS_EVAL_BATCH_HOLD_MINOR`` override; 0 disables the hold entirely.
|
||||
DEFAULT_BATCH_HOLD_MINOR = DEFAULT_DEBIT_FLOOR_MINOR
|
||||
|
||||
DEFAULT_EVAL_REPO = "eval-runs"
|
||||
EVAL_LABEL = "eval-task"
|
||||
|
||||
@ -45,6 +69,12 @@ _STATUS_FAILED = "failed"
|
||||
_TASK_PENDING = "pending"
|
||||
_TASK_DONE = "done"
|
||||
|
||||
# pending_eval_batches lifecycle (issue #39, mirroring the factory's
|
||||
# pending_issues): queued while the client is unfunded, shipped once
|
||||
# ``flush_pending`` creates the real batch + task issues.
|
||||
_STATUS_QUEUED = "queued"
|
||||
_STATUS_SHIPPED = "shipped"
|
||||
|
||||
_REPO_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$")
|
||||
|
||||
# Idempotent by construction (same convention as the billing schema).
|
||||
@ -55,6 +85,7 @@ CREATE TABLE IF NOT EXISTS eval_batches (
|
||||
status TEXT NOT NULL DEFAULT 'open'
|
||||
CHECK (status IN ('open', 'done', 'emitted', 'failed')),
|
||||
callback_url TEXT,
|
||||
hold_id INTEGER,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
|
||||
@ -73,6 +104,26 @@ CREATE TABLE IF NOT EXISTS eval_tasks (
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_eval_tasks_batch ON eval_tasks (batch_id, status);
|
||||
|
||||
-- Unfunded submissions queue here (issue #39, factory mirror): no Gitea
|
||||
-- issue is ever created for them; flush_pending ships them post-funding.
|
||||
CREATE TABLE IF NOT EXISTS pending_eval_batches (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
client_id INTEGER NOT NULL,
|
||||
tasks_json TEXT NOT NULL,
|
||||
callback_url TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
status TEXT NOT NULL DEFAULT 'queued'
|
||||
CHECK (status IN ('queued', 'shipped'))
|
||||
);
|
||||
|
||||
-- Cancel markers (issue #39): "stop draining" without touching the batch
|
||||
-- status CHECK constraint on pre-existing tables. A marker row means the
|
||||
-- batch no longer polls or emits and its hold was released.
|
||||
CREATE TABLE IF NOT EXISTS eval_batch_cancels (
|
||||
batch_id INTEGER PRIMARY KEY REFERENCES eval_batches(id),
|
||||
cancelled_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
|
||||
);
|
||||
"""
|
||||
|
||||
|
||||
@ -167,6 +218,15 @@ class EvalIntake:
|
||||
self._conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._conn.execute("PRAGMA foreign_keys=ON")
|
||||
self._conn.executescript(_EVAL_DDL)
|
||||
# Idempotent column migration for DB files created before #39's
|
||||
# hold_id landed: SQLite cannot amend a CHECK, but ADD COLUMN is
|
||||
# safe and re-runnable (PRAGMA-guarded).
|
||||
cols = {row[1] for row in self._conn.execute(
|
||||
"PRAGMA table_info(eval_batches)").fetchall()}
|
||||
if "hold_id" not in cols:
|
||||
self._conn.execute(
|
||||
"ALTER TABLE eval_batches ADD COLUMN hold_id INTEGER")
|
||||
self._conn.commit()
|
||||
self._lock = threading.RLock()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@ -202,6 +262,22 @@ class EvalIntake:
|
||||
def per_batch_repos(self) -> bool:
|
||||
return bool(getattr(self._config, "per_batch_repos", False))
|
||||
|
||||
@property
|
||||
def batch_hold_minor(self) -> int:
|
||||
"""Per-batch hold reserved at submit (issue #39).
|
||||
|
||||
Config attr ``eval_batch_hold_minor`` > env
|
||||
``SAAS_EVAL_BATCH_HOLD_MINOR`` > ``DEFAULT_BATCH_HOLD_MINOR``.
|
||||
0 disables the hold (cancel then has nothing to release).
|
||||
"""
|
||||
value = getattr(self._config, "eval_batch_hold_minor", None)
|
||||
if value is None:
|
||||
raw = os.environ.get("SAAS_EVAL_BATCH_HOLD_MINOR", "").strip()
|
||||
value = int(raw) if raw else DEFAULT_BATCH_HOLD_MINOR
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
|
||||
raise EvalIntakeError(f"invalid batch hold: {value!r}")
|
||||
return value
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Billing helpers (fail-closed, factory semantics)
|
||||
# ------------------------------------------------------------------
|
||||
@ -254,16 +330,45 @@ class EvalIntake:
|
||||
callback_url: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Accept a funded batch; one eval-task issue per task.
|
||||
|
||||
Returns ``{ok: True, batch_id, repo, issues: [{task_id,
|
||||
issue_number}]}``. Unfunded/unknown clients get a 402-style
|
||||
``{ok: False, reason: 'unfunded', status: 402}`` and NO Gitea
|
||||
issues are created.
|
||||
Funded clients get ``{ok: True, batch_id, repo, issues,
|
||||
hold_id}``; the batch hold is reserved from the client's balance
|
||||
(issue #39). Unfunded/unknown clients (fail-closed) get
|
||||
``{ok: False, reason: 'unfunded', status: 402, queued: True,
|
||||
pending_id}``, a local ``pending_eval_batches`` row and NO Gitea
|
||||
issues — ``flush_pending`` ships it once funding arrives.
|
||||
"""
|
||||
self._require_valid_inputs(client_id, tasks)
|
||||
if self.client_unfunded(client_id):
|
||||
return {"ok": False, "reason": "unfunded", "status": 402}
|
||||
callback_url = (callback_url or "").strip() or None
|
||||
return self._queue_batch(client_id, tasks, callback_url)
|
||||
return self._ship_batch(client_id, tasks, callback_url)
|
||||
|
||||
def _queue_batch(self, client_id: int, tasks: List[dict],
|
||||
callback_url: Optional[str]) -> Dict[str, Any]:
|
||||
"""Persist an unfunded submission locally (no Gitea mutations)."""
|
||||
callback_url = (callback_url or "").strip() or None
|
||||
with self._lock:
|
||||
cur = self._conn.execute(
|
||||
"INSERT INTO pending_eval_batches"
|
||||
" (client_id, tasks_json, callback_url, created_at, status)"
|
||||
" VALUES (?, ?, ?, ?, ?)",
|
||||
(client_id, json.dumps(tasks), callback_url, _now_iso(),
|
||||
_STATUS_QUEUED))
|
||||
self._conn.commit()
|
||||
return {"ok": False, "reason": "unfunded", "status": 402,
|
||||
"queued": True, "pending_id": int(cur.lastrowid)}
|
||||
|
||||
def _ship_batch(self, client_id: int, tasks: List[dict],
|
||||
callback_url: Optional[str],
|
||||
tasks_json: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Create batch row + eval-task issues; reserve the batch hold.
|
||||
|
||||
The hold is placed BEFORE any Gitea mutation and released (with the
|
||||
batch rolled back) if intake fails, so a failed submit never leaves
|
||||
money locked or a partial batch behind. ``tasks_json`` is only
|
||||
passed by ``flush_pending`` (the payload was already persisted in
|
||||
the pending row).
|
||||
"""
|
||||
callback_url = (callback_url or "").strip() or None
|
||||
with self._lock:
|
||||
cur = self._conn.execute(
|
||||
"INSERT INTO eval_batches (client_id, status, callback_url, created_at)"
|
||||
@ -278,6 +383,27 @@ class EvalIntake:
|
||||
repo = self.eval_repo
|
||||
if not _REPO_NAME_RE.match(repo):
|
||||
repo = DEFAULT_EVAL_REPO
|
||||
|
||||
# Money first (issue #39): reserve the batch hold; a client
|
||||
# whose balance cannot back the batch is refused, not shipped.
|
||||
hold_id = None
|
||||
hold_amount = self.batch_hold_minor
|
||||
if hold_amount > 0:
|
||||
ok, details = self._core.reserve_hold(
|
||||
client_id, hold_amount, f"eval-batch-{batch_id}",
|
||||
ref_type="eval", ref_id=str(batch_id))
|
||||
if not ok:
|
||||
self._conn.execute(
|
||||
"DELETE FROM eval_batches WHERE id = ?", (batch_id,))
|
||||
self._conn.commit()
|
||||
error = details if isinstance(details, dict) else {"raw": details}
|
||||
return {"ok": False, "reason": "intake_failed",
|
||||
"error": error, "status": 500}
|
||||
hold_id = int(details["hold_id"])
|
||||
self._conn.execute(
|
||||
"UPDATE eval_batches SET hold_id = ? WHERE id = ?",
|
||||
(hold_id, batch_id))
|
||||
|
||||
try:
|
||||
if not self._gitea.repo_exists(repo):
|
||||
self._gitea.create_org_repo(repo)
|
||||
@ -297,7 +423,8 @@ class EvalIntake:
|
||||
" task_id, payload_json, status, created_at)"
|
||||
" VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||
(batch_id, repo, issue_number, task_id,
|
||||
json.dumps({"title": title, "body": body}),
|
||||
tasks_json if tasks_json is not None
|
||||
else json.dumps({"title": title, "body": body}),
|
||||
_TASK_PENDING, _now_iso()))
|
||||
issues.append({"task_id": task_id,
|
||||
"issue_number": issue_number})
|
||||
@ -307,12 +434,116 @@ class EvalIntake:
|
||||
"DELETE FROM eval_tasks WHERE batch_id = ?", (batch_id,))
|
||||
self._conn.execute(
|
||||
"DELETE FROM eval_batches WHERE id = ?", (batch_id,))
|
||||
if hold_id is not None:
|
||||
try:
|
||||
self._core.release_hold(
|
||||
hold_id, f"eval-rollback-{batch_id}")
|
||||
except Exception:
|
||||
pass # best effort; the hold auto-expires in 30 min
|
||||
self._conn.commit()
|
||||
return {"ok": False, "reason": "intake_failed",
|
||||
"error": str(exc), "status": 500}
|
||||
|
||||
return {"ok": True, "batch_id": batch_id, "repo": repo,
|
||||
"issues": issues}
|
||||
"issues": issues, "hold_id": hold_id}
|
||||
|
||||
def flush_pending(self, client_id: int) -> Dict[str, Any]:
|
||||
"""Ship a client's queued (unfunded) batches once they are funded.
|
||||
|
||||
Factory mirror (issue #39): unfunded work never becomes a Gitea
|
||||
issue, so this refuses to flush while the client is still
|
||||
unfunded. Each shipped batch is created exactly like a fresh
|
||||
funded submit (hold reserved, task issues created, batch row
|
||||
written) and its pending row is marked ``shipped``.
|
||||
"""
|
||||
with self._lock:
|
||||
rows = self._conn.execute(
|
||||
"SELECT * FROM pending_eval_batches WHERE client_id = ?"
|
||||
" AND status = ? ORDER BY id",
|
||||
(client_id, _STATUS_QUEUED)).fetchall()
|
||||
if not rows:
|
||||
return {"ok": True, "flushed": 0, "batch_ids": [], "issues": []}
|
||||
if self.client_unfunded(client_id):
|
||||
return {"ok": False, "reason": "unfunded",
|
||||
"skipped": len(rows), "flushed": 0}
|
||||
batch_ids: List[int] = []
|
||||
issues: List[dict] = []
|
||||
for row in rows:
|
||||
try:
|
||||
tasks = json.loads(row["tasks_json"] or "[]")
|
||||
except ValueError:
|
||||
tasks = []
|
||||
result = self._ship_batch(
|
||||
client_id, tasks, row["callback_url"],
|
||||
tasks_json=row["tasks_json"])
|
||||
if not result.get("ok"):
|
||||
# A single failure stops the flush (fail-closed, mirroring
|
||||
# intake's no-partial-batches rule); earlier shipped rows
|
||||
# stay shipped.
|
||||
return {"ok": False, "reason": "flush_failed",
|
||||
"error": result.get("error"),
|
||||
"flushed": len(batch_ids), "batch_ids": batch_ids,
|
||||
"issues": issues}
|
||||
with self._lock:
|
||||
self._conn.execute(
|
||||
"UPDATE pending_eval_batches SET status = ? WHERE id = ?",
|
||||
(_STATUS_SHIPPED, row["id"]))
|
||||
self._conn.commit()
|
||||
batch_ids.append(result["batch_id"])
|
||||
issues.extend(result["issues"])
|
||||
return {"ok": True, "flushed": len(batch_ids),
|
||||
"batch_ids": batch_ids, "issues": issues}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Cancelling a batch (issue #39)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def cancel_batch(self, batch_id: int) -> Dict[str, Any]:
|
||||
"""Stop draining a batch and release its held funds.
|
||||
|
||||
Writes a cancel marker (``eval_batch_cancels``) so polling and
|
||||
emitting become no-ops, and releases the batch hold — the client's
|
||||
available balance returns to its pre-submit value exactly.
|
||||
Idempotent: a second cancel returns ``already_cancelled`` and
|
||||
never releases twice. A hold the core already auto-expired
|
||||
(``HOLD_ALREADY_RELEASED``/``HOLD_NOT_FOUND``) is treated as
|
||||
already gone; the batch is still marked cancelled.
|
||||
"""
|
||||
batch = self._batch_row(batch_id)
|
||||
if batch is None:
|
||||
return {"ok": False, "error": "unknown_batch"}
|
||||
with self._lock:
|
||||
marker = self._conn.execute(
|
||||
"SELECT 1 FROM eval_batch_cancels WHERE batch_id = ?",
|
||||
(batch_id,)).fetchone()
|
||||
if marker is not None:
|
||||
return {"ok": True, "batch_id": batch_id,
|
||||
"cancelled": True, "already_cancelled": True,
|
||||
"hold_released": False}
|
||||
hold_id = batch["hold_id"]
|
||||
hold_released = False
|
||||
if hold_id is not None:
|
||||
ok, details = self._core.release_hold(
|
||||
int(hold_id), f"eval-cancel-{batch_id}")
|
||||
if not ok:
|
||||
code = ((details or {}).get("error") or {}).get(
|
||||
"code") if isinstance(details, dict) else None
|
||||
if code not in ("HOLD_ALREADY_RELEASED", "HOLD_NOT_FOUND"):
|
||||
return {"ok": False, "reason": "cancel_failed",
|
||||
"error": details}
|
||||
else:
|
||||
hold_released = True
|
||||
self._conn.execute(
|
||||
"INSERT INTO eval_batch_cancels (batch_id, cancelled_at)"
|
||||
" VALUES (?, ?)", (batch_id, _now_iso()))
|
||||
self._conn.commit()
|
||||
return {"ok": True, "batch_id": batch_id, "cancelled": True,
|
||||
"hold_released": hold_released}
|
||||
|
||||
def _is_cancelled(self, batch_id: int) -> bool:
|
||||
return self._conn.execute(
|
||||
"SELECT 1 FROM eval_batch_cancels WHERE batch_id = ?",
|
||||
(batch_id,)).fetchone() is not None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Polling completed tasks
|
||||
@ -332,11 +563,14 @@ class EvalIntake:
|
||||
|
||||
A task is done when its issue is closed (the lane merged); the
|
||||
result record captures the final conversation comment (the lane's
|
||||
report) plus the issue's labels.
|
||||
report) plus the issue's labels. A cancelled batch (issue #39)
|
||||
stops draining: ``{ok: False, error: 'cancelled'}``.
|
||||
"""
|
||||
batch = self._batch_row(batch_id)
|
||||
if batch is None:
|
||||
return {"ok": False, "error": "unknown_batch"}
|
||||
if self._is_cancelled(batch_id):
|
||||
return {"ok": False, "error": "cancelled"}
|
||||
done = 0
|
||||
pending = 0
|
||||
for row in self._task_rows(batch_id):
|
||||
@ -411,10 +645,14 @@ class EvalIntake:
|
||||
Callback mode: POST to ``callback_url`` (retry x3, backoff) via the
|
||||
emitter seam. File mode: write ``SAAS_EVAL_OUTPUT_DIR/<batch_id>.json``.
|
||||
A batch already emitted re-emits idempotently (no second POST/write).
|
||||
A cancelled batch (issue #39) refuses to emit: ``{ok: False,
|
||||
reason: 'cancelled'}``.
|
||||
"""
|
||||
batch = self._batch_row(batch_id)
|
||||
if batch is None:
|
||||
return {"ok": False, "error": "unknown_batch"}
|
||||
if self._is_cancelled(batch_id):
|
||||
return {"ok": False, "reason": "cancelled"}
|
||||
pending = self._conn.execute(
|
||||
"SELECT COUNT(*) FROM eval_tasks WHERE batch_id = ?"
|
||||
" AND status = 'pending'", (batch_id,)).fetchone()[0]
|
||||
@ -465,4 +703,159 @@ class EvalIntake:
|
||||
for t in tasks:
|
||||
t["payload_json"] = json.loads(t["payload_json"] or "{}")
|
||||
t["result_json"] = json.loads(t["result_json"] or "null")
|
||||
return {"ok": True, "batch": dict(batch), "tasks": tasks}
|
||||
marker = self._conn.execute(
|
||||
"SELECT cancelled_at FROM eval_batch_cancels WHERE batch_id = ?",
|
||||
(batch_id,)).fetchone()
|
||||
data = dict(batch)
|
||||
if marker is not None:
|
||||
data["status"] = "cancelled"
|
||||
data["cancelled_at"] = marker["cancelled_at"]
|
||||
return {"ok": True, "batch": data, "tasks": tasks,
|
||||
"cancelled": marker is not None}
|
||||
|
||||
def get_results(self, batch_id: int) -> Dict[str, Any]:
|
||||
"""The results table for a batch (issue #39 HTTP surface).
|
||||
|
||||
Read-only view: one row per task with its current status, title
|
||||
and collected summary — regardless of whether the batch is still
|
||||
draining, emitted or cancelled. Unknown batches are
|
||||
``{ok: False, error: 'unknown_batch'}``.
|
||||
"""
|
||||
batch = self._batch_row(batch_id)
|
||||
if batch is None:
|
||||
return {"ok": False, "error": "unknown_batch"}
|
||||
report = self._report(batch)
|
||||
if self._is_cancelled(batch_id):
|
||||
report["status"] = "cancelled"
|
||||
report["ok"] = True
|
||||
return report
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTTP surface (issue #39; mounted by the assembly issue #38's create_app)
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Symmetric with register_factory_routes: the same five-route shape, the
|
||||
# same X-API-Key discipline (gitea_saas.auth only — issue #24; no sixth
|
||||
# implementation), and the same local-queue money rule. The one deliberate
|
||||
# difference: a route reached while NO key is configured returns 503
|
||||
# SERVICE_UNAVAILABLE (fail-closed "cannot serve"), as #38's assembly
|
||||
# acceptance demands app-wide. /api/eval/health is unauthenticated like
|
||||
# the factory's /api/factory/health (its handler never calls _require_auth).
|
||||
|
||||
|
||||
def register_eval_routes(app: Any, core: Any, gitea: Any, config: Any) -> None:
|
||||
"""Mount the eval-farm endpoints on a Flask app (idempotent).
|
||||
|
||||
Routes:
|
||||
POST /api/eval/batches {"client_id": int,
|
||||
"tasks": [{task_id, title,
|
||||
body?}],
|
||||
"callback_url"?: str}
|
||||
GET /api/eval/batches/<id> status + per-task state
|
||||
GET /api/eval/batches/<id>/results the results table
|
||||
POST /api/eval/batches/<id>/cancel stop draining + release hold
|
||||
GET /api/eval/health liveness (no auth)
|
||||
|
||||
Each authed route builds an :class:`EvalIntake` per request (cheap
|
||||
SQLite connection, WAL + per-instance lock — same pattern as the
|
||||
factory). Auth failures use the canonical 401 body; unconfigured key
|
||||
(empty expected) is 503 SERVICE_UNAVAILABLE.
|
||||
"""
|
||||
from flask import jsonify, request
|
||||
|
||||
api_key = app.config.get("API_KEY") or getattr(config, "api_key", None) or ""
|
||||
|
||||
def _eval() -> EvalIntake:
|
||||
return EvalIntake(core, gitea, config)
|
||||
|
||||
def _require_auth():
|
||||
# Fail-closed on a misconfigured deployment: no key configured
|
||||
# means the service cannot authenticate anything -> 503 (issue
|
||||
# #38's assembly acceptance demands this app-wide).
|
||||
if not api_key:
|
||||
resp = jsonify({"error": {
|
||||
"code": "SERVICE_UNAVAILABLE",
|
||||
"message": "API key not configured"}})
|
||||
resp.status_code = 503
|
||||
return resp
|
||||
sent = request.headers.get(API_KEY_HEADER, "")
|
||||
if not key_is_valid(sent, api_key):
|
||||
return unauthorized_response()
|
||||
return None
|
||||
|
||||
@app.get("/api/eval/health")
|
||||
def eval_health():
|
||||
return jsonify({"ok": True})
|
||||
|
||||
@app.post("/api/eval/batches")
|
||||
def eval_submit_batch():
|
||||
if (denied := _require_auth()) is not None:
|
||||
return denied
|
||||
body = request.get_json(silent=True) or {}
|
||||
raw_id = body.get("client_id")
|
||||
if isinstance(raw_id, bool) or not isinstance(raw_id, int) or raw_id <= 0:
|
||||
return _json_error(400, "invalid_request",
|
||||
"client_id must be a positive integer")
|
||||
callback_url = body.get("callback_url")
|
||||
try:
|
||||
result = _eval().submit_batch(raw_id, body.get("tasks") or [],
|
||||
callback_url)
|
||||
except Exception as exc:
|
||||
return _map_eval_error(exc)
|
||||
resp = jsonify(result)
|
||||
# Funded: 201 created. Unfunded/queued: 202 accepted locally
|
||||
# (factory mirror: the row flushes once funding arrives).
|
||||
resp.status_code = 201 if result.get("ok") else 202
|
||||
return resp
|
||||
|
||||
@app.get("/api/eval/batches/<int:batch_id>")
|
||||
def eval_get_batch(batch_id: int):
|
||||
if (denied := _require_auth()) is not None:
|
||||
return denied
|
||||
try:
|
||||
result = _eval().get_batch(batch_id)
|
||||
except Exception as exc:
|
||||
return _map_eval_error(exc)
|
||||
if not result.get("ok"):
|
||||
return _json_error(404, "unknown_batch", "no such batch")
|
||||
return jsonify(result)
|
||||
|
||||
@app.get("/api/eval/batches/<int:batch_id>/results")
|
||||
def eval_get_results(batch_id: int):
|
||||
if (denied := _require_auth()) is not None:
|
||||
return denied
|
||||
try:
|
||||
result = _eval().get_results(batch_id)
|
||||
except Exception as exc:
|
||||
return _map_eval_error(exc)
|
||||
if not result.get("ok"):
|
||||
return _json_error(404, "unknown_batch", "no such batch")
|
||||
return jsonify(result)
|
||||
|
||||
@app.post("/api/eval/batches/<int:batch_id>/cancel")
|
||||
def eval_cancel_batch(batch_id: int):
|
||||
if (denied := _require_auth()) is not None:
|
||||
return denied
|
||||
try:
|
||||
result = _eval().cancel_batch(batch_id)
|
||||
except Exception as exc:
|
||||
return _map_eval_error(exc)
|
||||
if not result.get("ok"):
|
||||
return _json_error(404, "unknown_batch", "no such batch")
|
||||
return jsonify(result)
|
||||
|
||||
|
||||
def _json_error(status: int, code: str, message: str):
|
||||
from flask import jsonify
|
||||
|
||||
resp = jsonify({"error": message, "code": code})
|
||||
resp.status_code = status
|
||||
return resp
|
||||
|
||||
|
||||
def _map_eval_error(exc: Exception):
|
||||
"""Map eval domain errors to HTTP responses (fail-closed on unknown)."""
|
||||
if isinstance(exc, EvalIntakeError):
|
||||
return _json_error(400, "invalid_request", str(exc))
|
||||
return _json_error(500, "eval_error", f"{type(exc).__name__}: {exc}")
|
||||
|
||||
@ -23,7 +23,6 @@ shipped without payment.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
@ -33,15 +32,11 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
# Header carrying the API key. The shared constant lives in the sibling
|
||||
# ``gitea_saas/config.py`` (issue #2/#6 auth), which lands in a later PR than
|
||||
# this one; until that module is merged this file falls back to the same
|
||||
# canonical value. Keeping the single source of truth means the factory
|
||||
# routes use ``compare_digest`` on the same header name as ``server.py``.
|
||||
try:
|
||||
from .config import API_KEY_HEADER # shared with gitea_saas.server
|
||||
except ImportError: # config.py not merged yet (sibling auth PRs)
|
||||
API_KEY_HEADER = "X-API-Key"
|
||||
# Single auth path (issue #24): the header name, the constant-time comparison
|
||||
# and the 401 body all live in ``gitea_saas.auth``; this module delegates.
|
||||
# config.py is long merged, so there is no fallback constant here — a failed
|
||||
# import of a security-relevant constant must be loud, not silently re-hardcoded.
|
||||
from .auth import API_KEY_HEADER, key_is_valid, unauthorized_response
|
||||
|
||||
# Default floor for "funded": a client must hold at least this many minor
|
||||
# units (default = 5.00 USD) before intake will ship issues.
|
||||
@ -342,8 +337,8 @@ class IntakeFactory:
|
||||
# HTTP surface (mounted by the issue #2 Flask app; see register_factory_routes)
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Routes (same auth discipline as gitea_saas.server -- API_KEY_HEADER via
|
||||
# hmac.compare_digest, fail-closed):
|
||||
# Routes (single auth path, issue #24 -- gitea_saas.auth.key_is_valid via
|
||||
# API_KEY_HEADER, fail-closed; /api/factory/health is in PUBLIC_PATHS):
|
||||
# POST /api/factory/provision {"client_id": int, "repo_name"?: str}
|
||||
# POST /api/factory/<client_id>/issues {"repo", "title", "acceptance_body",
|
||||
# "labels"?: [...]}
|
||||
@ -375,15 +370,10 @@ def register_factory_routes(app: Any, core: Any, gitea: Any, config: Any) -> Non
|
||||
def _factory() -> IntakeFactory:
|
||||
return IntakeFactory(core, gitea, config)
|
||||
|
||||
def _authorized() -> bool:
|
||||
sent = request.headers.get(API_KEY_HEADER, "")
|
||||
return bool(api_key) and hmac.compare_digest(sent, api_key)
|
||||
|
||||
def _require_auth():
|
||||
if not _authorized():
|
||||
resp = jsonify({"error": "missing or invalid API key", "code": "unauthorized"})
|
||||
resp.status_code = 401
|
||||
return resp
|
||||
sent = request.headers.get(API_KEY_HEADER, "")
|
||||
if not key_is_valid(sent, api_key):
|
||||
return unauthorized_response()
|
||||
return None
|
||||
|
||||
@app.get("/api/factory/health")
|
||||
|
||||
@ -9,10 +9,13 @@ Seams (all duck-typed, faked in tests):
|
||||
(see ``billing_core.BillingCore``).
|
||||
* ``gitea`` — Gitea client: ``list_open_prs(repo=None)``,
|
||||
``get_pr(repo, n)``, ``get_issue(repo, n)``, ``post_comment``,
|
||||
``add_label``, ``remove_label``, ``merge_pr(repo, n) -> bool``,
|
||||
``close_issue(repo, n)``.
|
||||
``add_label``, ``remove_label``, ``merge_pr(repo, n,
|
||||
expected_head_sha=None) -> bool``, ``close_issue(repo, n)``. The
|
||||
normalized PR shape carries ``merged`` (bool) and ``state`` so the
|
||||
policy can positively distinguish a merged PR from a merely closed one.
|
||||
* ``reviewer`` — the review gate: ``review_pr(repo, pr, linked_issue,
|
||||
repo_dir=...) -> Verdict`` and ``post_verdict(repo, pr, verdict)``.
|
||||
workspace=...) -> Verdict`` (the isolated synthetic-merge workspace,
|
||||
issue #30) and ``post_verdict(repo, pr, verdict)``.
|
||||
|
||||
Policy
|
||||
------
|
||||
@ -31,6 +34,52 @@ head moves. The table is bootstrapped here (``CREATE TABLE IF NOT EXISTS``)
|
||||
because ``schema.sql`` belongs to issue #1's PR and is out of this issue's
|
||||
allowed file set — same schema intent, idempotent.
|
||||
|
||||
Evaluated-pair integrity (issue #30): the reviewer evaluates a synthetic
|
||||
merge of the exact head onto the exact base and the verdict pins both
|
||||
SHAs. Before ANY verdict is applied, the policy re-reads the PR and
|
||||
refuses to apply the verdict unless BOTH the head and the base are still
|
||||
the evaluated pair — a verdict describing a pair that no longer exists is
|
||||
never acted on (no merge, label, comment, or close), and the new pair is
|
||||
simply eligible for a fresh review on the next poll. The merge request
|
||||
itself additionally pins the reviewed head (``expected_head_sha``, issue
|
||||
#28) as a second, tighter guard.
|
||||
|
||||
Transactional outcome application (issue #29): a head is recorded into
|
||||
``reviews`` (i.e. considered COMPLETED) only after its required policy
|
||||
action has fully succeeded. Failure/incomplete attempts are persisted in
|
||||
the companion ``review_state`` table — one row per head carrying the
|
||||
attempt count, the last error, and idempotent step checkpoints
|
||||
(``merged_step``, ``closed_step``, ``debit_step``, ``label_step``,
|
||||
``comment_step``) — so a transient failure at any boundary (verdict
|
||||
generation, label, comment, merge, close, debit) leaves the same
|
||||
unchanged head eligible for retry on the next poll instead of stranding it.
|
||||
|
||||
Crash-window recovery is NOT driven by the open-PR scan. A merge is the
|
||||
point of no return: once the PR is truly merged it disappears from
|
||||
``list_open_prs()``, so any later failure (debit, label removal, issue
|
||||
close, comment) would strand the persisted row. Therefore the outcome
|
||||
plan (verdict, tier, linked issue) is persisted into ``review_state``
|
||||
BEFORE the merge, every post-merge side effect is checkpointed, and
|
||||
``recover_incomplete()`` resumes those rows directly from the retry table
|
||||
(via ``get_pr`` + checkpoints) — independently of any open-PR listing —
|
||||
before the regular poll scan runs. A merge is only ever adopted when the
|
||||
client reports positive proof (``merged is True``, now part of the
|
||||
production normalized shape); ``state == "closed"`` without a merge is
|
||||
never treated as a merge, so a closed-but-unmerged PR cannot trigger a
|
||||
debit, label removal, issue close, or success comment. Retries re-apply
|
||||
only the steps that have not already been checkpointed, so a partial side
|
||||
effect can never double-merge a PR, double-close an issue, double-remove
|
||||
a label, or double-post the success comment; the delivery debit is
|
||||
additionally protected by its existing billing-core idempotency key
|
||||
``<repo>#<pr>@<head_sha>``. The merge request itself is pinned to the
|
||||
reviewed head: ``_safe_merge`` passes ``expected_head_sha`` through to the
|
||||
client so a branch that moved after review is refused before any POST
|
||||
(issue #28 client contract). Legacy ``reviews`` rows (pre-#29 schema) are
|
||||
preserved untouched and keep their completed meaning; the retry state
|
||||
lives entirely in the new table, so no migration of existing records is
|
||||
needed (new step columns are ALTER-added idempotently for databases
|
||||
created by earlier builds).
|
||||
|
||||
Delivery debit: implemented inline here behind the ``debit_for_merge``
|
||||
signature shared with the delivery issue (#7); when delivery lands it can
|
||||
refactor to call it. FAIL-CLOSED: this module never fake-succeeds a debit —
|
||||
@ -63,6 +112,45 @@ CREATE TABLE IF NOT EXISTS reviews (
|
||||
)
|
||||
"""
|
||||
|
||||
# Retry state for heads whose outcome application is in flight or failed
|
||||
# (issue #29). One row per head; ``attempts`` counts failed/incomplete
|
||||
# attempts, ``last_error`` keeps the most recent failure, and the step
|
||||
# checkpoints record which idempotent side effects already happened so a
|
||||
# retry resumes instead of re-applying them. ``tier``/``linked_issue``
|
||||
# snapshot the outcome plan BEFORE the merge so ``recover_incomplete()``
|
||||
# can finish the post-merge steps even after the PR leaves the open-PR
|
||||
# scan. Rows are deleted once the head completes (a ``reviews`` row is
|
||||
# written instead).
|
||||
_REVIEW_STATE_DDL = """
|
||||
CREATE TABLE IF NOT EXISTS review_state (
|
||||
repo TEXT NOT NULL,
|
||||
pr INTEGER NOT NULL,
|
||||
head_sha TEXT NOT NULL,
|
||||
verdict TEXT,
|
||||
attempts INTEGER NOT NULL DEFAULT 0,
|
||||
last_error TEXT,
|
||||
last_attempt_at TEXT,
|
||||
merged_step INTEGER NOT NULL DEFAULT 0,
|
||||
closed_step INTEGER NOT NULL DEFAULT 0,
|
||||
debit_step INTEGER NOT NULL DEFAULT 0,
|
||||
label_step INTEGER NOT NULL DEFAULT 0,
|
||||
comment_step INTEGER NOT NULL DEFAULT 0,
|
||||
tier TEXT,
|
||||
linked_issue INTEGER,
|
||||
PRIMARY KEY (repo, pr, head_sha)
|
||||
)
|
||||
"""
|
||||
|
||||
# Columns added after the initial #29 schema shipped (compat: databases
|
||||
# created by earlier builds get these via idempotent ALTERs).
|
||||
_REVIEW_STATE_EXTRA_COLUMNS = {
|
||||
"debit_step": "debit_step INTEGER NOT NULL DEFAULT 0",
|
||||
"label_step": "label_step INTEGER NOT NULL DEFAULT 0",
|
||||
"comment_step": "comment_step INTEGER NOT NULL DEFAULT 0",
|
||||
"tier": "tier TEXT",
|
||||
"linked_issue": "linked_issue INTEGER",
|
||||
}
|
||||
|
||||
_LINK_RE = re.compile(r"(?im)\b(?:closes|fixes|resolves|close|fix|resolve)\s+#(\d+)\b")
|
||||
# Markdown-tolerant: "**Tier:** gold" / "- Tier: gold" / "**Client:** acme" all parse
|
||||
# (the value can sit after formatting chars like "*" / "**" following the colon).
|
||||
@ -98,26 +186,156 @@ class ReviewPolicy:
|
||||
self._reviews_conn = sqlite3.connect(str(self._db_path))
|
||||
self._reviews_conn.execute("PRAGMA journal_mode=WAL")
|
||||
self._reviews_conn.execute(_REVIEWS_DDL)
|
||||
self._reviews_conn.execute(_REVIEW_STATE_DDL)
|
||||
self._ensure_review_state_columns()
|
||||
self._reviews_conn.commit()
|
||||
# Isolated review workspace (issue #30): reviews run against a clean
|
||||
# temporary synthetic-merge worktree taken from the configured clone
|
||||
# (REVIEW_CLONE_DIR), never against an arbitrary worktrees root.
|
||||
self._workspace = None
|
||||
clone_dir = self.config.get("review_clone_dir")
|
||||
if clone_dir:
|
||||
from .review_workspace import ReviewWorkspace
|
||||
self._workspace = ReviewWorkspace(clone_dir, self.reviewer.runner)
|
||||
|
||||
def _ensure_review_state_columns(self) -> None:
|
||||
"""Idempotently add columns introduced after the first #29 build."""
|
||||
existing = {row[1] for row in self._reviews_conn.execute(
|
||||
"PRAGMA table_info(review_state)")}
|
||||
for column, ddl in _REVIEW_STATE_EXTRA_COLUMNS.items():
|
||||
if column not in existing:
|
||||
self._reviews_conn.execute(
|
||||
f"ALTER TABLE review_state ADD COLUMN {ddl}")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Reviews table (re-review safety)
|
||||
# Reviews table (completed-history) + review_state (retry metadata)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _already_reviewed(self, repo: str, pr: int, head_sha: str) -> bool:
|
||||
"""True once a head's outcome application COMPLETED successfully.
|
||||
|
||||
A failed/incomplete attempt lives only in ``review_state`` and does
|
||||
not make a head "reviewed": the same unchanged head must stay
|
||||
eligible for retry.
|
||||
"""
|
||||
row = self._reviews_conn.execute(
|
||||
"SELECT 1 FROM reviews WHERE repo = ? AND pr = ? AND head_sha = ?",
|
||||
(repo, pr, head_sha)).fetchone()
|
||||
return row is not None
|
||||
|
||||
def _record_review(self, repo: str, pr: int, head_sha: str, verdict: str) -> None:
|
||||
def _record_completed(self, repo: str, pr: int, head_sha: str, verdict: str) -> None:
|
||||
"""Persist a fully applied outcome; drop the retry state for the head."""
|
||||
self._reviews_conn.execute(
|
||||
"INSERT OR IGNORE INTO reviews (repo, pr, head_sha, verdict, reviewed_at)"
|
||||
" VALUES (?, ?, ?, ?, ?)",
|
||||
(repo, pr, head_sha, verdict,
|
||||
datetime.now(timezone.utc).isoformat()))
|
||||
self._reviews_conn.execute(
|
||||
"DELETE FROM review_state WHERE repo = ? AND pr = ? AND head_sha = ?",
|
||||
(repo, pr, head_sha))
|
||||
self._reviews_conn.commit()
|
||||
|
||||
def _record_failure(self, repo: str, pr: int, head_sha: str,
|
||||
verdict: Optional[str], error: str) -> None:
|
||||
"""Persist a failed/incomplete attempt (upserts per head).
|
||||
|
||||
Step checkpoints on an existing row are preserved, so a retry of a
|
||||
partially applied outcome resumes from where it left off.
|
||||
"""
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
self._reviews_conn.execute(
|
||||
"INSERT INTO review_state (repo, pr, head_sha, verdict, attempts,"
|
||||
" last_error, last_attempt_at)"
|
||||
" VALUES (?, ?, ?, ?, 1, ?, ?)"
|
||||
" ON CONFLICT(repo, pr, head_sha) DO UPDATE SET"
|
||||
" verdict = excluded.verdict,"
|
||||
" attempts = review_state.attempts + 1,"
|
||||
" last_error = excluded.last_error,"
|
||||
" last_attempt_at = excluded.last_attempt_at",
|
||||
(repo, pr, head_sha, verdict, error, now))
|
||||
self._reviews_conn.commit()
|
||||
|
||||
_STEP_COLUMNS = {"merged": "merged_step", "closed": "closed_step",
|
||||
"debit": "debit_step", "label": "label_step",
|
||||
"comment": "comment_step"}
|
||||
|
||||
_POST_MERGE_STEPS = ("debit", "label", "closed", "comment")
|
||||
|
||||
def _ensure_plan(self, repo: Optional[str], pr: int, head_sha: Optional[str],
|
||||
verdict: str, tier: Optional[str],
|
||||
issue_num: Optional[int]) -> None:
|
||||
"""Persist the outcome plan BEFORE the first side effect (#29).
|
||||
|
||||
The merge is the point of no return: once merged, the PR leaves
|
||||
the open-PR scan, so every remaining step (debit, label removal,
|
||||
issue close, success comment) must be recoverable from this row
|
||||
alone via :meth:`recover_incomplete`. Upserts; preserves attempt
|
||||
counts and step checkpoints from earlier attempts.
|
||||
"""
|
||||
self._reviews_conn.execute(
|
||||
"INSERT INTO review_state (repo, pr, head_sha, verdict, tier,"
|
||||
" linked_issue)"
|
||||
" VALUES (?, ?, ?, ?, ?, ?)"
|
||||
" ON CONFLICT(repo, pr, head_sha) DO UPDATE SET"
|
||||
" verdict = excluded.verdict,"
|
||||
" tier = excluded.tier,"
|
||||
" linked_issue = excluded.linked_issue",
|
||||
(repo, pr, head_sha, verdict, tier, issue_num))
|
||||
self._reviews_conn.commit()
|
||||
|
||||
def _linked_issue(self, repo: Optional[str], pr: int,
|
||||
head_sha: Optional[str]) -> Optional[int]:
|
||||
row = self._reviews_conn.execute(
|
||||
"SELECT linked_issue FROM review_state"
|
||||
" WHERE repo = ? AND pr = ? AND head_sha = ?",
|
||||
(repo, pr, head_sha)).fetchone()
|
||||
return row[0] if row else None
|
||||
|
||||
def _step_done(self, repo: Optional[str], pr: int,
|
||||
head_sha: Optional[str], step: str) -> bool:
|
||||
"""True when the idempotent step already succeeded for this head."""
|
||||
column = self._STEP_COLUMNS.get(step)
|
||||
if column is None:
|
||||
raise ValueError(f"unknown retry step {step!r}")
|
||||
row = self._reviews_conn.execute(
|
||||
f"SELECT {column} FROM review_state"
|
||||
" WHERE repo = ? AND pr = ? AND head_sha = ?",
|
||||
(repo, pr, head_sha)).fetchone()
|
||||
return bool(row and row[0])
|
||||
|
||||
def _mark_step(self, repo: Optional[str], pr: int, head_sha: Optional[str],
|
||||
step: str) -> None:
|
||||
"""Checkpoint a succeeded side effect so a retry never re-applies it."""
|
||||
column = self._STEP_COLUMNS.get(step)
|
||||
if column is None:
|
||||
raise ValueError(f"unknown retry step {step!r}")
|
||||
self._reviews_conn.execute(
|
||||
f"INSERT INTO review_state (repo, pr, head_sha, {column})"
|
||||
" VALUES (?, ?, ?, 1)"
|
||||
f" ON CONFLICT(repo, pr, head_sha) DO UPDATE SET {column} = 1",
|
||||
(repo, pr, head_sha))
|
||||
self._reviews_conn.commit()
|
||||
|
||||
def _attempts(self, repo: str, pr: int, head_sha: str) -> int:
|
||||
row = self._reviews_conn.execute(
|
||||
"SELECT attempts FROM review_state"
|
||||
" WHERE repo = ? AND pr = ? AND head_sha = ?",
|
||||
(repo, pr, head_sha)).fetchone()
|
||||
return row[0] if row else 0
|
||||
|
||||
@staticmethod
|
||||
def _error_text(exc: Exception) -> str:
|
||||
return f"{type(exc).__name__}: {exc}"
|
||||
|
||||
def _failure_result(self, repo: Optional[str], pr: int, head_sha: str,
|
||||
verdict: Optional[str], exc: Exception) -> dict:
|
||||
return {
|
||||
"repo": repo, "pr": pr, "head_sha": head_sha, "verdict": verdict,
|
||||
"outcome": "failed",
|
||||
"error": {"type": type(exc).__name__, "message": str(exc)},
|
||||
"attempts": self._attempts(repo or "", pr, head_sha),
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Polling
|
||||
# ------------------------------------------------------------------
|
||||
@ -138,24 +356,217 @@ class ReviewPolicy:
|
||||
return str(issue or "")
|
||||
|
||||
def poll_and_review(self, repo: Optional[str] = None) -> list[dict]:
|
||||
"""Review every in-scope PR at its current head; apply tier policy."""
|
||||
results = []
|
||||
"""Review every in-scope PR at its current head; apply tier policy.
|
||||
|
||||
Transactional + retryable (issue #29): a head is only recorded as
|
||||
completed AFTER its required policy action succeeded; any failure —
|
||||
verdict generation or a policy action — is persisted as a retryable
|
||||
attempt and reported as a structured failure result, leaving the
|
||||
same unchanged head eligible for the next poll.
|
||||
|
||||
Stranded post-merge applications are recovered FIRST, directly
|
||||
from the ``review_state`` table: a truly merged PR is no longer
|
||||
returned by ``list_open_prs()``, so the open-PR scan can never
|
||||
reach those rows again (see :meth:`recover_incomplete`).
|
||||
"""
|
||||
results = list(self.recover_incomplete(repo))
|
||||
for pr in self.list_candidate_prs(repo):
|
||||
repo_key = repo or (pr.get("repo") or "")
|
||||
pr_number = pr["number"]
|
||||
head_sha = pr.get("head_sha") or "unknown"
|
||||
if self._already_reviewed(repo or (pr.get("repo") or ""), pr_number, head_sha):
|
||||
if self._already_reviewed(repo_key, pr_number, head_sha):
|
||||
continue
|
||||
pr_body = pr.get("body") or ""
|
||||
issue_body = self._fetch_issue_body(repo, linked_issue_number(pr_body))
|
||||
verdict = self.reviewer.review_pr(
|
||||
repo, pr_number, issue_body,
|
||||
repo_dir=self.config.get("worktrees_dir"))
|
||||
self._record_review(repo or "", pr_number, head_sha, verdict.verdict)
|
||||
action = self._apply_policy(repo, pr, pr_body, issue_body, verdict)
|
||||
try:
|
||||
pr_body = pr.get("body") or ""
|
||||
issue_body = self._fetch_issue_body(repo, linked_issue_number(pr_body))
|
||||
verdict = self.reviewer.review_pr(
|
||||
repo, pr_number, issue_body, workspace=self._workspace)
|
||||
except Exception as exc: # noqa: BLE001 - any failure must be retryable
|
||||
self._record_failure(repo_key, pr_number, head_sha, None,
|
||||
self._error_text(exc))
|
||||
results.append(self._failure_result(repo, pr_number, head_sha,
|
||||
None, exc))
|
||||
continue
|
||||
# Evaluated-pair integrity (issue #30): the verdict pins the exact
|
||||
# head/base it evaluated. Re-read the PR BEFORE applying anything
|
||||
# and refuse when either end of the pair moved — a verdict for a
|
||||
# pair that no longer exists must never merge/label/comment/close,
|
||||
# and the new pair stays eligible for a fresh review on the next
|
||||
# poll (nothing is recorded as completed for the stale pair).
|
||||
try:
|
||||
live_pr = self.gitea.get_pr(repo_key, pr_number) or {}
|
||||
pair_moved = (live_pr.get("head_sha") != verdict.head_sha
|
||||
or live_pr.get("base_sha") != verdict.base_sha)
|
||||
except Exception as exc: # noqa: BLE001 - re-read failure is retryable
|
||||
self._record_failure(repo_key, pr_number, head_sha, verdict.verdict,
|
||||
self._error_text(exc))
|
||||
results.append(self._failure_result(repo, pr_number, head_sha,
|
||||
verdict.verdict, exc))
|
||||
continue
|
||||
if pair_moved:
|
||||
results.append({
|
||||
"repo": repo, "pr": pr_number,
|
||||
"head_sha": head_sha, "verdict": verdict.verdict,
|
||||
"outcome": "stale",
|
||||
"error": {
|
||||
"code": "PAIR_MOVED",
|
||||
"message": (
|
||||
f"PR pair moved after evaluation: evaluated head "
|
||||
f"{verdict.head_sha!r} / base {verdict.base_sha!r}, "
|
||||
f"live head {live_pr.get('head_sha')!r} / base "
|
||||
f"{live_pr.get('base_sha')!r} — verdict NOT applied; "
|
||||
"the new pair is eligible for a fresh review")},
|
||||
"evaluated": {"head_sha": verdict.head_sha,
|
||||
"base_sha": verdict.base_sha},
|
||||
"live": {"head_sha": live_pr.get("head_sha"),
|
||||
"base_sha": live_pr.get("base_sha")},
|
||||
})
|
||||
continue
|
||||
try:
|
||||
action = self._apply_policy(repo, pr, pr_body, issue_body, verdict)
|
||||
except Exception as exc: # noqa: BLE001 - any failure must be retryable
|
||||
self._record_failure(repo_key, pr_number, head_sha, verdict.verdict,
|
||||
self._error_text(exc))
|
||||
results.append(self._failure_result(repo, pr_number, head_sha,
|
||||
verdict.verdict, exc))
|
||||
continue
|
||||
if action.get("action") in ("merge_failed", "merge_but_debit_failed"):
|
||||
# The required action did NOT succeed: keep the head retryable
|
||||
# and persist the incomplete attempt (fail-closed branches
|
||||
# report a structured non-completion instead of raising).
|
||||
self._record_failure(repo_key, pr_number, head_sha, verdict.verdict,
|
||||
f"code={action.get('action')}")
|
||||
results.append({
|
||||
"repo": repo, "pr": pr_number, "head_sha": head_sha,
|
||||
"verdict": verdict.verdict, "outcome": "failed",
|
||||
"error": {"code": action.get("action")},
|
||||
"attempts": self._attempts(repo_key, pr_number, head_sha),
|
||||
**action})
|
||||
continue
|
||||
self._record_completed(repo_key, pr_number, head_sha, verdict.verdict)
|
||||
results.append({"repo": repo, "pr": pr_number, "head_sha": head_sha,
|
||||
"verdict": verdict.verdict, **action})
|
||||
return results
|
||||
|
||||
def recover_incomplete(self, repo: Optional[str] = None) -> list[dict]:
|
||||
"""Resume stranded outcome applications directly from retry state.
|
||||
|
||||
Independent of the open-PR scan: ``list_open_prs()`` cannot reach
|
||||
a head whose PR was truly merged (it is no longer open), so any
|
||||
post-merge step that failed — debit, label removal, issue close,
|
||||
success comment — would otherwise be lost forever. This method
|
||||
walks ``review_state`` rows and finishes each one from its
|
||||
checkpoints, re-fetching the PR by number via ``get_pr`` (which
|
||||
works for closed/merged PRs) rather than from any open-PR listing.
|
||||
|
||||
Only rows whose merge is already confirmed (checkpointed, or the
|
||||
live PR positively reports ``merged is True`` at the EXACT
|
||||
planned/reviewed head — the production normalized shape includes
|
||||
the raw ``merged`` boolean and ``head_sha``) are handled here; rows
|
||||
for still-open PRs are left to the regular poll scan, which can
|
||||
reach them. ``state == "closed"`` alone never counts as merged: a
|
||||
PR closed without merging must not trigger a debit, label removal,
|
||||
issue close, or success comment. A PR merged at a DIFFERENT head
|
||||
than the plan reviewed is never adopted — it fails closed with a
|
||||
structured ``HEAD_MISMATCH`` and no side effects (the debit,
|
||||
label, close and comment steps all belong to the reviewed head,
|
||||
not to whatever head actually landed).
|
||||
Completed heads are recorded into ``reviews`` and their retry row
|
||||
deleted; a step that fails again is persisted with an incremented
|
||||
attempt count for the next recovery pass.
|
||||
"""
|
||||
results = []
|
||||
where, params = (" WHERE repo = ?", [repo]) if repo else ("", [])
|
||||
rows = self._reviews_conn.execute(
|
||||
f"SELECT repo, pr, head_sha, verdict FROM review_state{where}",
|
||||
params).fetchall()
|
||||
for row in rows:
|
||||
row_repo, pr_number, head_sha, verdict = row
|
||||
if verdict is None:
|
||||
continue # no policy action started yet; open scan owns it
|
||||
try:
|
||||
live = None
|
||||
try:
|
||||
live = self.gitea.get_pr(row_repo, pr_number) or {}
|
||||
except Exception: # noqa: BLE001 - PR may be gone; checkpoints decide
|
||||
live = None
|
||||
pr_merged = bool(live and live.get("merged") is True)
|
||||
merged_done = self._step_done(row_repo, pr_number, head_sha,
|
||||
"merged")
|
||||
if pr_merged and not merged_done:
|
||||
# Crash window: the merge applied server-side but the
|
||||
# checkpoint write never happened — adopt it ONLY when
|
||||
# the live merged head is the EXACT planned/reviewed
|
||||
# head (head-integrity gate, governor 10539). A merge
|
||||
# of a DIFFERENT head must never be attributed to this
|
||||
# plan's debit/label/close/comment steps.
|
||||
live_head = (live or {}).get("head_sha")
|
||||
if live_head != head_sha:
|
||||
message = (
|
||||
f"merged head mismatch: live head {live_head!r} "
|
||||
f"!= planned/reviewed head {head_sha!r} — the "
|
||||
f"outcome application for the reviewed head is "
|
||||
f"refused (no debit, no label removal, no issue "
|
||||
f"close, no success comment)")
|
||||
self._record_failure(row_repo, pr_number, head_sha,
|
||||
verdict, message)
|
||||
results.append({
|
||||
"repo": row_repo, "pr": pr_number,
|
||||
"head_sha": head_sha, "verdict": verdict,
|
||||
"outcome": "failed",
|
||||
"error": {"code": "HEAD_MISMATCH",
|
||||
"message": message},
|
||||
"attempts": self._attempts(
|
||||
row_repo, pr_number, head_sha),
|
||||
})
|
||||
continue
|
||||
# Same-head crash window: adopt and finish the plan.
|
||||
self._mark_step(row_repo, pr_number, head_sha, "merged")
|
||||
merged_done = True
|
||||
if not merged_done:
|
||||
continue # still reachable via the open scan; leave it
|
||||
|
||||
issue_body = self._fetch_issue_body(
|
||||
row_repo, self._linked_issue(row_repo, pr_number, head_sha))
|
||||
if not self._step_done(row_repo, pr_number, head_sha, "debit"):
|
||||
ok, debit = self.debit_for_merge(
|
||||
row_repo, pr_number, head_sha, issue_body)
|
||||
if not ok:
|
||||
raise RuntimeError(
|
||||
f"recover debit failed: {debit.get('error') or debit}")
|
||||
self._mark_step(row_repo, pr_number, head_sha, "debit")
|
||||
else:
|
||||
debit = {"debit": "recorded", "replayed": True}
|
||||
|
||||
if not self._step_done(row_repo, pr_number, head_sha, "label"):
|
||||
self.gitea.remove_label(row_repo, pr_number, "in-review")
|
||||
self._mark_step(row_repo, pr_number, head_sha, "label")
|
||||
|
||||
self._safe_close_issue(row_repo, pr_number, head_sha,
|
||||
self._linked_issue(row_repo, pr_number,
|
||||
head_sha))
|
||||
|
||||
if not self._step_done(row_repo, pr_number, head_sha, "comment"):
|
||||
summary = ("delivery debit recorded"
|
||||
if debit.get("debit") == "recorded"
|
||||
else "no delivery debit (tier rate 0)")
|
||||
self.gitea.post_comment(
|
||||
row_repo, pr_number,
|
||||
f"Merged by review-gate — {summary}.")
|
||||
self._mark_step(row_repo, pr_number, head_sha, "comment")
|
||||
|
||||
self._record_completed(row_repo, pr_number, head_sha, verdict)
|
||||
results.append({"repo": row_repo, "pr": pr_number,
|
||||
"head_sha": head_sha, "verdict": verdict,
|
||||
"outcome": "merged", "action": "merged",
|
||||
"debit": debit, "recovered": True})
|
||||
except Exception as exc: # noqa: BLE001 - any failure must be retryable
|
||||
self._record_failure(row_repo, pr_number, head_sha, verdict,
|
||||
self._error_text(exc))
|
||||
results.append(self._failure_result(row_repo, pr_number,
|
||||
head_sha, verdict, exc))
|
||||
return results
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Tier resolution + delivery debit
|
||||
# ------------------------------------------------------------------
|
||||
@ -216,6 +627,77 @@ class ReviewPolicy:
|
||||
# Tier policy application
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _safe_merge(self, repo: Optional[str], pr: dict) -> bool:
|
||||
"""Merge guarded for idempotent retry (issue #29).
|
||||
|
||||
Never merges twice for the same head: the ``merged_step`` checkpoint
|
||||
makes a retry skip an already-applied merge, and when the fetched PR
|
||||
shape already positively reports ``merged is True`` (crash window
|
||||
between the API side effect and the checkpoint write) the merge is
|
||||
skipped as well. ``state == "closed"`` alone is NOT adopted — a PR
|
||||
can be closed without merging, and adopting it would debit the
|
||||
client, strip labels, and close the linked issue for a merge that
|
||||
never happened.
|
||||
|
||||
The merge request is pinned to the REVIEWED head: the exact
|
||||
``head_sha`` this policy reviewed is passed through to
|
||||
``gitea.merge_pr(..., expected_head_sha=...)``, whose production
|
||||
implementation (issue #28) fails closed before any POST when the
|
||||
live branch moved after review — an unreviewed SHA can never be
|
||||
merged. A candidate that carries NO reviewed head_sha is rejected
|
||||
BEFORE the client is called (head-integrity gate, governor 10539):
|
||||
``expected_head_sha=None`` would arm the client's backward-
|
||||
compatible live-head fallback and could merge a head this policy
|
||||
never reviewed, so the merge is refused up front and the failure
|
||||
is surfaced as a structured retryable result with zero side
|
||||
effects.
|
||||
"""
|
||||
repo_key = repo or ""
|
||||
pr_number = pr["number"]
|
||||
reviewed_head = pr.get("head_sha") or ""
|
||||
if not reviewed_head:
|
||||
raise ValueError(
|
||||
"merge refused: candidate PR carries no reviewed head_sha — "
|
||||
"expected_head_sha must be pinned; refusing to fall back to "
|
||||
"the live head")
|
||||
head_sha = reviewed_head
|
||||
if self._step_done(repo_key, pr_number, head_sha, "merged"):
|
||||
return True
|
||||
if pr.get("merged") is True:
|
||||
self._mark_step(repo_key, pr_number, head_sha, "merged")
|
||||
return True
|
||||
merged = self.gitea.merge_pr(
|
||||
repo, pr_number, expected_head_sha=reviewed_head)
|
||||
if merged:
|
||||
self._mark_step(repo_key, pr_number, head_sha, "merged")
|
||||
return merged
|
||||
|
||||
def _safe_close_issue(self, repo: Optional[str], pr_number: int,
|
||||
head_sha: Optional[str], issue_num: Optional[int]) -> None:
|
||||
"""Close guarded for idempotent retry (issue #29).
|
||||
|
||||
Skips when the ``closed_step`` checkpoint says this head already
|
||||
closed the issue, when the head already completed its outcome
|
||||
application (poll path), or when the live issue state is already
|
||||
closed. ``head_sha`` may be None (webhook path): the checkpoint is
|
||||
keyed on the head only when one is known.
|
||||
"""
|
||||
repo_key = repo or ""
|
||||
head = head_sha or "unknown"
|
||||
if issue_num is None or self._step_done(repo_key, pr_number, head, "closed"):
|
||||
return
|
||||
if self._already_reviewed(repo_key, pr_number, head):
|
||||
return
|
||||
try:
|
||||
issue = self.gitea.get_issue(repo, issue_num)
|
||||
except Exception: # noqa: BLE001 - fail-closed: let close_issue decide
|
||||
issue = None
|
||||
if isinstance(issue, dict) and issue.get("state") == "closed":
|
||||
self._mark_step(repo_key, pr_number, head, "closed")
|
||||
return
|
||||
self.gitea.close_issue(repo, issue_num)
|
||||
self._mark_step(repo_key, pr_number, head, "closed")
|
||||
|
||||
def _apply_policy(self, repo: Optional[str], pr: dict, pr_body: str,
|
||||
issue_body: str, verdict) -> dict:
|
||||
pr_number = pr["number"]
|
||||
@ -230,22 +712,43 @@ class ReviewPolicy:
|
||||
"Approved — awaiting operator merge "
|
||||
f"(tier `{tier}`, auto_merge off).")
|
||||
return {"action": "approved_awaiting_merge", "tier": tier}
|
||||
merged = self.gitea.merge_pr(repo, pr_number)
|
||||
# Persist the outcome plan BEFORE the merge: once the PR is
|
||||
# truly merged it leaves the open-PR scan, so every following
|
||||
# step must be recoverable from the row alone (issue #29 /
|
||||
# review bounce: merged PRs vanish from list_open_prs()).
|
||||
head_sha = pr.get("head_sha") or ""
|
||||
if not head_sha:
|
||||
# Head-integrity gate (governor 10539): a candidate without
|
||||
# a reviewed head_sha is rejected BEFORE any plan write or
|
||||
# client call — expected_head_sha must be pinned, never the
|
||||
# client's live-head fallback.
|
||||
raise ValueError(
|
||||
"merge refused: candidate PR carries no reviewed head_sha — "
|
||||
"expected_head_sha must be pinned; refusing to fall back to "
|
||||
"the live head")
|
||||
issue_num = linked_issue_number(pr_body)
|
||||
self._ensure_plan(repo, pr_number, head_sha, verdict.verdict,
|
||||
tier, issue_num)
|
||||
merged = self._safe_merge(repo, pr)
|
||||
if not merged:
|
||||
self.gitea.post_comment(
|
||||
repo, pr_number,
|
||||
"[review-gate] merge failed — PR left open for manual review.")
|
||||
return {"action": "merge_failed", "tier": tier}
|
||||
ok, debit = self.debit_for_merge(repo, pr_number, pr.get("head_sha"), issue_body)
|
||||
self.gitea.remove_label(repo, pr_number, "in-review")
|
||||
issue_num = linked_issue_number(pr_body)
|
||||
ok, debit = self.debit_for_merge(repo, pr_number, head_sha, issue_body)
|
||||
if ok:
|
||||
if issue_num is not None:
|
||||
self.gitea.close_issue(repo, issue_num)
|
||||
self._mark_step(repo, pr_number, head_sha, "debit")
|
||||
if not self._step_done(repo, pr_number, head_sha, "label"):
|
||||
self.gitea.remove_label(repo, pr_number, "in-review")
|
||||
self._mark_step(repo, pr_number, head_sha, "label")
|
||||
if ok:
|
||||
self._safe_close_issue(repo, pr_number, head_sha, issue_num)
|
||||
summary = "delivery debit recorded" if debit.get("debit") == "recorded" \
|
||||
else "no delivery debit (tier rate 0)"
|
||||
self.gitea.post_comment(repo, pr_number,
|
||||
f"Merged by review-gate — {summary}.")
|
||||
if not self._step_done(repo, pr_number, head_sha, "comment"):
|
||||
self.gitea.post_comment(repo, pr_number,
|
||||
f"Merged by review-gate — {summary}.")
|
||||
self._mark_step(repo, pr_number, head_sha, "comment")
|
||||
return {"action": "merged", "tier": tier, "debit": debit}
|
||||
# fail-closed: merge happened but the debit failed — do NOT close
|
||||
# the issue, tell the operator.
|
||||
@ -290,8 +793,7 @@ class ReviewPolicy:
|
||||
else "no debit for this tier.")
|
||||
)
|
||||
issue_num = linked_issue_number(pr.get("body") or "")
|
||||
if issue_num is not None:
|
||||
self.gitea.close_issue(repo, issue_num)
|
||||
self._safe_close_issue(repo, pr_number, head_sha, issue_num)
|
||||
return ok, {"pr": pr_number, "head_sha": head_sha, "debit": debit}
|
||||
self.gitea.post_comment(
|
||||
repo, pr_number,
|
||||
|
||||
265
gitea_saas/review_workspace.py
Normal file
265
gitea_saas/review_workspace.py
Normal file
@ -0,0 +1,265 @@
|
||||
"""Isolated review workspace (issue #30).
|
||||
|
||||
The review gate must never run tests against an arbitrary ``repo_dir``:
|
||||
a green verdict must describe the exact synthetic merge result it
|
||||
actually evaluated. This module builds that result in a clean,
|
||||
isolated, temporary Git worktree taken from a configured repository
|
||||
clone —
|
||||
|
||||
* the workspace never changes or tests a shared ``main`` checkout or a
|
||||
worker's issue worktree (the configured clone is only ever *read* by
|
||||
``git fetch`` / ``git worktree add``; its checked-out branch and
|
||||
working tree are untouched);
|
||||
* the named base SHA and PR head SHA are fetched into the clone and the
|
||||
exact head is merged onto the exact base (a real merge commit, so
|
||||
tests and ``git diff --check <base> HEAD`` run against a real,
|
||||
inspectable merge result with an explicit base/result range);
|
||||
* a merge conflict produces a deterministic, evidence-carrying result
|
||||
(``merge_ok=False``) and the caller — the review gate — never runs the
|
||||
test suite against some fallback checkout;
|
||||
* the temporary worktree is removed on success, conflict, timeout, and
|
||||
raised exceptions (best-effort ``git worktree remove --force`` plus a
|
||||
belt-and-braces directory removal), and no branch refs are ever
|
||||
created (the worktree is detached), so no temporary refs survive.
|
||||
|
||||
All git commands go through the same ``Runner`` subprocess seam as the
|
||||
rest of the review gate, so hermetic tests can use local-path ``origin``
|
||||
remotes (zero network) and/or a scripted command runner to assert the
|
||||
exact commands, working directories, and ranges. Every invocation here
|
||||
passes an **argument list** (``shell=False`` in the real Runner): no
|
||||
shell ever interprets the arguments, so a ``^{commit}`` suffix, a path
|
||||
containing spaces/``$``, or any other shell-special character survives
|
||||
verbatim on every platform (fixes the Windows cmd.exe ``^``-eating
|
||||
break — issue #48).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
|
||||
_SHA_RE = re.compile(r"^[0-9a-f]{40}$")
|
||||
|
||||
# Characters kept in a mkdtemp prefix token. ``repo`` names routinely
|
||||
# contain separators (``owner/name``) and punctuation; a prefix is a bare
|
||||
# filename component, so everything else is collapsed to ``-`` (and the
|
||||
# results trimmed) to keep ``tempfile.mkdtemp`` able to create it.
|
||||
_FS_SAFE_RE = re.compile(r"[^A-Za-z0-9_-]+")
|
||||
|
||||
|
||||
def _fs_safe_token(value: str) -> str:
|
||||
"""Collapse ``value`` into a filesystem-safe prefix token."""
|
||||
return _FS_SAFE_RE.sub("-", value).strip("-")
|
||||
|
||||
_MERGE_MESSAGE = "synthetic merge for review"
|
||||
|
||||
|
||||
class ReviewWorkspaceError(Exception):
|
||||
"""The isolated workspace could not be constructed (fetch/resolve
|
||||
failure, missing identity, etc.). Not a merge conflict — the caller
|
||||
treats it as a retryable operational failure."""
|
||||
|
||||
|
||||
class MergeWorkspace:
|
||||
"""A constructed synthetic-merge workspace (context manager).
|
||||
|
||||
Attributes
|
||||
----------
|
||||
dir : Path
|
||||
The temporary worktree (working directory for test/build/hygiene
|
||||
commands). HEAD is the synthetic merge commit of ``head_sha``
|
||||
onto ``base_sha`` when ``merge_ok`` is True.
|
||||
base_sha / head_sha : str
|
||||
The exact SHAs that were merged (what the review evaluates and
|
||||
pins).
|
||||
merge_ok : bool
|
||||
True when the synthetic merge completed; False on conflict.
|
||||
evidence : dict
|
||||
Machine-readable evidence: merge command, exit, output summary,
|
||||
conflicted files (conflict path), result HEAD SHA (clean path).
|
||||
"""
|
||||
|
||||
def __init__(self, dir: Path, base_sha: str, head_sha: str,
|
||||
merge_ok: bool, evidence: dict,
|
||||
cleanup: Optional[Callable[[], None]] = None):
|
||||
self.dir = Path(dir)
|
||||
self.base_sha = base_sha
|
||||
self.head_sha = head_sha
|
||||
self.merge_ok = merge_ok
|
||||
self.evidence = evidence
|
||||
self._cleanup = cleanup
|
||||
|
||||
def __enter__(self) -> "MergeWorkspace":
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc, tb) -> None:
|
||||
"""Remove the temporary worktree + directory on every path.
|
||||
|
||||
``git worktree remove --force`` cleans the clone's worktree
|
||||
admin entry even when the tree is dirty or the merge left
|
||||
conflict state behind; the directory removal is a belt-and-
|
||||
braces cleanup that never masks the caller's result.
|
||||
"""
|
||||
if self._cleanup is not None:
|
||||
try:
|
||||
self._cleanup()
|
||||
except Exception: # noqa: BLE001 - cleanup must never mask the caller's result
|
||||
pass
|
||||
shutil.rmtree(self.dir, ignore_errors=True)
|
||||
|
||||
def __repr__(self) -> str: # pragma: no cover - debugging aid
|
||||
return (f"<MergeWorkspace {self.dir} merge_ok={self.merge_ok} "
|
||||
f"base={self.base_sha[:12]} head={self.head_sha[:12]}>")
|
||||
|
||||
|
||||
class ReviewWorkspace:
|
||||
"""Builds isolated synthetic-merge workspaces from a configured clone.
|
||||
|
||||
``clone_dir`` is the READ-ONLY configured repository clone (the
|
||||
clone owns the worktree administration; its checked-out branch and
|
||||
working tree are never modified). ``runner`` is the review gate's
|
||||
subprocess seam (``Runner`` or a scripted fake); all git commands
|
||||
run through it so tests stay hermetic.
|
||||
"""
|
||||
|
||||
def __init__(self, clone_dir, runner):
|
||||
self.clone_dir = Path(clone_dir)
|
||||
self.runner = runner
|
||||
|
||||
# -- construction ------------------------------------------------------
|
||||
|
||||
def build(self, repo: str, pr_number: int, head_sha: str,
|
||||
base_sha: str) -> MergeWorkspace:
|
||||
"""Construct the synthetic-merge workspace for one review.
|
||||
|
||||
Raises :class:`ReviewWorkspaceError` on operational failures
|
||||
(unfetchable/unresolvable SHAs, missing git identity, worktree
|
||||
creation failure). Returns a :class:`MergeWorkspace` with
|
||||
``merge_ok=False`` + conflict evidence for a genuine merge
|
||||
conflict, or ``merge_ok=True`` with the merged worktree.
|
||||
"""
|
||||
for label, sha in (("base", base_sha), ("head", head_sha)):
|
||||
if not _SHA_RE.match(sha or ""):
|
||||
raise ReviewWorkspaceError(
|
||||
f"cannot build an isolated review workspace: {label} SHA "
|
||||
f"{sha!r} is not a 40-char hex commit id — refusing to "
|
||||
"review an unchecked-out pair")
|
||||
|
||||
self._fetch_refresh(base_sha, head_sha)
|
||||
|
||||
# ``repo`` is embedded in the temp-dir prefix: sanitize it so a
|
||||
# separator/punctuation-containing name cannot turn the prefix into
|
||||
# a nested path (which would make ``mkdtemp`` fail).
|
||||
tmpdir = Path(tempfile.mkdtemp(
|
||||
prefix=f"review-ws-{_fs_safe_token(repo)}-{pr_number}-"))
|
||||
evidence: dict = {
|
||||
"base_sha": base_sha,
|
||||
"head_sha": head_sha,
|
||||
"worktree_dir": str(tmpdir),
|
||||
}
|
||||
try:
|
||||
add = self.runner.run(
|
||||
self.clone_dir,
|
||||
["git", "worktree", "add", "--detach",
|
||||
str(tmpdir), base_sha])
|
||||
if add.get("exit") != 0:
|
||||
raise ReviewWorkspaceError(
|
||||
"cannot create the temporary review worktree: "
|
||||
f"{add.get('summary', '')[:400]}")
|
||||
|
||||
merge = self.runner.run(
|
||||
tmpdir,
|
||||
["git", "merge", "--no-ff", "-m",
|
||||
_MERGE_MESSAGE, head_sha])
|
||||
evidence["merge_command"] = (
|
||||
["git", "merge", "--no-ff", "-m",
|
||||
_MERGE_MESSAGE, head_sha])
|
||||
evidence["merge_exit"] = merge.get("exit")
|
||||
evidence["merge_summary"] = merge.get("summary", "")[:800]
|
||||
|
||||
if merge.get("exit") != 0:
|
||||
# Distinguish a real merge conflict (MERGE_HEAD exists) from an
|
||||
# environmental merge failure (no conflict state was entered).
|
||||
merget = self.runner.run(
|
||||
tmpdir, ["git", "rev-parse", "-q",
|
||||
"--verify", "MERGE_HEAD"])
|
||||
if merget.get("exit") != 0:
|
||||
raise ReviewWorkspaceError(
|
||||
"synthetic merge failed without entering a conflict "
|
||||
f"state (environmental failure): {merge.get('summary', '')[:400]}")
|
||||
conflicted = self.runner.run(
|
||||
tmpdir, ["git", "diff", "--name-only",
|
||||
"--diff-filter=U"])
|
||||
status = self.runner.run(
|
||||
tmpdir, ["git", "status", "--porcelain"])
|
||||
evidence["conflicted_files"] = [
|
||||
line for line in (conflicted.get("summary") or "").splitlines()
|
||||
if line.strip()]
|
||||
evidence["status_summary"] = (status.get("summary") or "")[:800]
|
||||
return MergeWorkspace(
|
||||
tmpdir, base_sha, head_sha, merge_ok=False,
|
||||
evidence=evidence, cleanup=self._make_cleanup(tmpdir))
|
||||
|
||||
result_head = self.runner.run(
|
||||
tmpdir, ["git", "rev-parse", "HEAD"])
|
||||
evidence["result_head"] = (result_head.get("summary") or "").strip()
|
||||
return MergeWorkspace(
|
||||
tmpdir, base_sha, head_sha, merge_ok=True,
|
||||
evidence=evidence, cleanup=self._make_cleanup(tmpdir))
|
||||
except Exception:
|
||||
# Guarantee cleanup even when construction failed mid-way: a
|
||||
# dangling worktree or temp dir is exactly what must never
|
||||
# survive a review pass.
|
||||
try:
|
||||
self._make_cleanup(tmpdir)()
|
||||
except Exception: # noqa: BLE001 - best-effort cleanup on top of a failure
|
||||
pass
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
raise
|
||||
|
||||
# -- internals ----------------------------------------------------------
|
||||
|
||||
def _make_cleanup(self, tmpdir: Path) -> Callable[[], None]:
|
||||
"""A cleanup closure: ``git worktree remove --force`` + rmtree."""
|
||||
|
||||
def cleanup() -> None:
|
||||
try:
|
||||
self.runner.run(
|
||||
self.clone_dir,
|
||||
["git", "worktree", "remove", "--force",
|
||||
str(tmpdir)])
|
||||
except Exception: # noqa: BLE001 - cleanup is best-effort
|
||||
pass
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
return cleanup
|
||||
|
||||
def _fetch_refresh(self, base_sha: str, head_sha: str) -> None:
|
||||
"""Ensure the clone can resolve both SHAs.
|
||||
|
||||
Prefers the precise fetch of exactly the two SHAs; falls back to a
|
||||
full ``git fetch origin`` (which refreshes every remote-tracking
|
||||
ref — the configured clone must be current for reviews anyway) and
|
||||
re-verifies. Hermetic tests use a local-path ``origin``.
|
||||
"""
|
||||
res = self.runner.run(
|
||||
self.clone_dir,
|
||||
["git", "fetch", "origin", base_sha, head_sha])
|
||||
if res.get("exit") != 0:
|
||||
res = self.runner.run(
|
||||
self.clone_dir, ["git", "fetch", "origin"])
|
||||
if res.get("exit") != 0:
|
||||
raise ReviewWorkspaceError(
|
||||
"cannot fetch the named base/head SHAs from the configured "
|
||||
f"clone's origin: {res.get('summary', '')[:400]}")
|
||||
for label, sha in (("base", base_sha), ("head", head_sha)):
|
||||
chk = self.runner.run(
|
||||
self.clone_dir,
|
||||
["git", "rev-parse", "--verify", f"{sha}^{{commit}}"])
|
||||
if chk.get("exit") != 0:
|
||||
raise ReviewWorkspaceError(
|
||||
f"cannot resolve the {label} SHA {sha} as a commit in the "
|
||||
f"configured clone after fetching: {chk.get('summary', '')[:300]}")
|
||||
@ -9,9 +9,17 @@ Design
|
||||
* ``Reviewer(repo_client, runner)`` — ``repo_client`` is a git/Gitea seam
|
||||
(see ``gitea_client.GiteaClient`` for the real one), ``runner`` is a
|
||||
subprocess seam (``Runner`` here; a scripted fake in tests).
|
||||
* ``review_pr(repo, pr_number, linked_issue, repo_dir)`` returns a
|
||||
* ``review_pr(repo, pr_number, linked_issue, workspace)`` returns a
|
||||
``Verdict``; the caller decides what to do. ``post_verdict`` posts the
|
||||
formatted ``[review-gate]`` comment.
|
||||
formatted ``[review-gate]`` comment. ``workspace`` is the isolated
|
||||
review-workspace seam (``review_workspace.ReviewWorkspace``): the
|
||||
review ALWAYS runs against the synthetic merge result of the exact
|
||||
reviewed head onto the exact reviewed base, constructed in a clean
|
||||
temporary Git worktree — never against an arbitrary ``repo_dir``, a
|
||||
shared ``main`` checkout, or a worker's issue worktree (issue #30).
|
||||
A merge conflict yields a deterministic BOUNCE with evidence and the
|
||||
test suite is NOT run against any fallback checkout; with no
|
||||
workspace configured the gate fails closed to NEEDS_GUIDANCE.
|
||||
* FAIL-CLOSED: the review gate NEVER merges — there is no merge call in
|
||||
this module. Policy and merge are issue #7.
|
||||
* Every finding MUST quote the diff. The default (no LLM configured) pass
|
||||
@ -135,6 +143,16 @@ def _redact_quote(text: str, match: "re.Match[str]") -> str:
|
||||
class Runner:
|
||||
"""Subprocess seam. ``command_runner`` is injectable for hermetic tests.
|
||||
|
||||
``run`` accepts a **command** that is either
|
||||
|
||||
* an argument **list** (``["git", "rev-parse", "--verify", f"{sha}^{{commit}}"]``)
|
||||
— executed with ``shell=False`` so no shell ever interprets the
|
||||
arguments (``^``, ``~``, ``&``, ``|``, ``<``, ``>``, ``%``, spaces,
|
||||
…) — the Windows-portable form used by every git invocation in this
|
||||
package (issue #48); or
|
||||
* a **string**, for callers that genuinely need shell semantics
|
||||
(``python -m pytest``, admin-configured ``build_cmd``).
|
||||
|
||||
Returns ``{"exit": int | "timed_out", "summary": tail-of-output}``.
|
||||
"""
|
||||
|
||||
@ -142,10 +160,17 @@ class Runner:
|
||||
command_runner: Optional[Callable] = None):
|
||||
self.timeout_s = timeout_s
|
||||
self._command_runner = command_runner or (
|
||||
lambda command, **kwargs: subprocess.run(command, shell=True, **kwargs))
|
||||
lambda command, **kwargs:
|
||||
subprocess.run(command, shell=isinstance(command, str),
|
||||
**kwargs))
|
||||
|
||||
def run(self, repo_dir, command: str, timeout_s: Optional[int] = None) -> dict:
|
||||
def run(self, repo_dir, command, timeout_s: Optional[int] = None) -> dict:
|
||||
"""Run ``command`` (arg list, or string) with ``cwd=repo_dir``."""
|
||||
timeout = timeout_s or self.timeout_s
|
||||
if isinstance(command, str) and command.lstrip().startswith("["):
|
||||
raise TypeError(
|
||||
"Runner.run command must be an argument LIST or a shell "
|
||||
f"string, not a stringified list: {command!r}")
|
||||
try:
|
||||
completed = self._command_runner(
|
||||
command, cwd=str(repo_dir), timeout=timeout,
|
||||
@ -166,11 +191,20 @@ class Runner:
|
||||
|
||||
@dataclass
|
||||
class Verdict:
|
||||
"""Evidence-based outcome of a review gate run."""
|
||||
"""Evidence-based outcome of a review gate run.
|
||||
|
||||
``base_sha`` pins the base commit the synthetic merge was built on
|
||||
(alongside ``head_sha``) — every review records and pins the
|
||||
repository, PR number, head SHA, and current base SHA it actually
|
||||
evaluated (issue #30). ``workspace`` carries the isolated-workspace
|
||||
evidence (``merge_ok``, ``conflicted_files``, …) when the verdict
|
||||
came from a synthetic-merge review.
|
||||
"""
|
||||
|
||||
repo: str
|
||||
pr: int
|
||||
head_sha: str
|
||||
base_sha: str = ""
|
||||
acceptance: list[str] = field(default_factory=list)
|
||||
tests: dict = field(default_factory=dict)
|
||||
build: Optional[dict] = None
|
||||
@ -178,11 +212,22 @@ class Verdict:
|
||||
findings: list[dict] = field(default_factory=list)
|
||||
verdict: str = NEEDS_GUIDANCE
|
||||
summary: str = ""
|
||||
workspace: dict = field(default_factory=dict)
|
||||
|
||||
def to_comment(self) -> str:
|
||||
"""Format the [review-gate] comment (checklist + findings)."""
|
||||
lines = [f"[review-gate] VERDICT: **{self.verdict}**", "",
|
||||
f"Repo: `{self.repo}` · PR: #{self.pr} · head `{self.head_sha[:12]}`", ""]
|
||||
f"Repo: `{self.repo}` · PR: #{self.pr} · head `{self.head_sha[:12]}`"
|
||||
+ (f" · base `{self.base_sha[:12]}`" if self.base_sha else ""), ""]
|
||||
if self.workspace.get("conflicted_files"):
|
||||
lines += ["Synthetic merge of the reviewed head onto the reviewed base: "
|
||||
"**CONFLICT**",
|
||||
"The test suite was NOT run — no fallback checkout is ever tested.",
|
||||
"Conflicted files:"]
|
||||
lines += [f"- `{f}`" for f in self.workspace["conflicted_files"]]
|
||||
if self.workspace.get("status_summary"):
|
||||
lines += ["```", self.workspace["status_summary"][:400], "```"]
|
||||
lines += [""]
|
||||
lines.append(f"Acceptance parsed ({len(self.acceptance)}):")
|
||||
if self.acceptance:
|
||||
lines += [f"- [ ] {a}" for a in self.acceptance]
|
||||
@ -230,11 +275,24 @@ class Reviewer:
|
||||
|
||||
# -- main entry points -------------------------------------------------
|
||||
|
||||
def review_pr(self, repo: str, pr_number: int, linked_issue, repo_dir) -> Verdict:
|
||||
def review_pr(self, repo: str, pr_number: int, linked_issue,
|
||||
workspace=None) -> Verdict:
|
||||
"""Review one PR against the isolated synthetic-merge workspace.
|
||||
|
||||
``workspace`` is the issue-#30 seam (``ReviewWorkspace``): its
|
||||
``build(repo, pr_number, head_sha, base_sha)`` context manager
|
||||
yields the merged temporary worktree. A green verdict ALWAYS
|
||||
describes that exact merge result — never an arbitrary
|
||||
``repo_dir``. With no workspace (or no resolvable base/head
|
||||
SHAs) the gate fails closed to NEEDS_GUIDANCE; on a merge
|
||||
conflict it returns a deterministic BOUNCE and the test suite is
|
||||
not run.
|
||||
"""
|
||||
issue_body = linked_issue.get("body") if isinstance(linked_issue, dict) else str(linked_issue or "")
|
||||
|
||||
pr = self.repo_client.get_pr(repo, pr_number)
|
||||
head_sha = pr.get("head_sha") or "unknown"
|
||||
base_sha = pr.get("base_sha") or "unknown"
|
||||
changed_files = pr.get("changed_files") or []
|
||||
diff_text = self.repo_client.get_pr_diff(repo, pr_number) or ""
|
||||
|
||||
@ -243,28 +301,62 @@ class Reviewer:
|
||||
scope_violations = [f for f in changed_files
|
||||
if allowed and not _matches_allowed(f, allowed)]
|
||||
|
||||
tests = self.runner.run(repo_dir, "python -m pytest -q",
|
||||
timeout_s=self.config["test_timeout_s"])
|
||||
build = None
|
||||
if self.config.get("build_cmd"):
|
||||
build = self.runner.run(repo_dir, self.config["build_cmd"],
|
||||
if workspace is None:
|
||||
return Verdict(
|
||||
repo=repo, pr=pr_number, head_sha=head_sha, base_sha=base_sha,
|
||||
acceptance=acceptance, tests=dict(exit="not_run"),
|
||||
scope_violations=scope_violations, verdict=NEEDS_GUIDANCE,
|
||||
summary=("no isolated review workspace configured — cannot "
|
||||
"construct and test the synthetic merge; refusing to "
|
||||
"review an arbitrary checkout (never silent-approve)"))
|
||||
|
||||
# The whole review happens inside the workspace: the working tree it
|
||||
# yields IS the synthetic merge of the exact head onto the exact base.
|
||||
with workspace.build(repo, pr_number, head_sha, base_sha) as ws:
|
||||
if not ws.merge_ok:
|
||||
return Verdict(
|
||||
repo=repo, pr=pr_number, head_sha=head_sha, base_sha=base_sha,
|
||||
acceptance=acceptance, tests=dict(
|
||||
exit="not_run",
|
||||
summary="test suite not run: the synthetic merge of the "
|
||||
"reviewed head onto the reviewed base conflicts"),
|
||||
scope_violations=scope_violations, verdict=BOUNCE,
|
||||
summary=("synthetic merge conflict between the reviewed head "
|
||||
"and the current base — see the CONFLICT evidence; "
|
||||
"the test suite was not run against any fallback "
|
||||
"checkout"),
|
||||
workspace={**ws.evidence, "merge_ok": False})
|
||||
|
||||
tests = self.runner.run(ws.dir, "python -m pytest -q",
|
||||
timeout_s=self.config["test_timeout_s"])
|
||||
build["command"] = self.config["build_cmd"]
|
||||
build = None
|
||||
if self.config.get("build_cmd"):
|
||||
build = self.runner.run(ws.dir, self.config["build_cmd"],
|
||||
timeout_s=self.config["test_timeout_s"])
|
||||
build["command"] = self.config["build_cmd"]
|
||||
|
||||
hygiene = self.runner.run(repo_dir, "git diff --check",
|
||||
timeout_s=self.config["test_timeout_s"])
|
||||
# Explicit base/result range: the whitespace gate checks only the
|
||||
# synthetic merge result vs the exact base it was built on.
|
||||
# Argument list (shell=False), same Windows-safety rationale as
|
||||
# review_workspace (issue #48): the SHA and "HEAD" are untrusted
|
||||
# data and must reach git verbatim.
|
||||
hygiene = self.runner.run(
|
||||
ws.dir,
|
||||
["git", "diff", "--check", ws.base_sha, "HEAD"],
|
||||
timeout_s=self.config["test_timeout_s"])
|
||||
|
||||
findings = self._collect_findings(diff_text, tests, hygiene)
|
||||
findings = self._collect_findings(diff_text, tests, hygiene)
|
||||
|
||||
verdict, summary = self._decide(
|
||||
issue_body, acceptance, tests, build, scope_violations, findings, hygiene)
|
||||
verdict, summary = self._decide(
|
||||
issue_body, acceptance, tests, build, scope_violations,
|
||||
findings, hygiene)
|
||||
|
||||
return Verdict(
|
||||
repo=repo, pr=pr_number, head_sha=head_sha,
|
||||
acceptance=acceptance, tests=tests, build=build,
|
||||
scope_violations=scope_violations, findings=findings,
|
||||
verdict=verdict, summary=summary,
|
||||
)
|
||||
return Verdict(
|
||||
repo=repo, pr=pr_number, head_sha=ws.head_sha, base_sha=ws.base_sha,
|
||||
acceptance=acceptance, tests=tests, build=build,
|
||||
scope_violations=scope_violations, findings=findings,
|
||||
verdict=verdict, summary=summary,
|
||||
workspace={**ws.evidence, "merge_ok": True})
|
||||
|
||||
def post_verdict(self, repo: str, pr_number: int, verdict: Verdict) -> str:
|
||||
"""Post the formatted [review-gate] comment. Returns the comment."""
|
||||
|
||||
@ -42,6 +42,24 @@ CREATE TABLE IF NOT EXISTS holds (
|
||||
released_at TEXT
|
||||
);
|
||||
|
||||
-- Per-client API keys (issue #41): the client-facing surface's credentials.
|
||||
-- Only the SHA-256 hash of a key is stored -- never the plaintext. A row is
|
||||
-- scoped to exactly one client, so a key can only ever resolve to its owner's
|
||||
-- client_id (cross-tenant denial is enforced by the client surface using this
|
||||
-- scope). revoked_at NULL = active; a revoked key fails resolution and is
|
||||
-- never returned by any listing (prefix + created_at only, for display).
|
||||
CREATE TABLE IF NOT EXISTS client_api_keys (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
client_id INTEGER NOT NULL REFERENCES clients(id),
|
||||
key_hash TEXT NOT NULL UNIQUE,
|
||||
key_prefix TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
|
||||
revoked_at TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_client_api_keys_client ON client_api_keys (client_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_client_api_keys_hash ON client_api_keys (key_hash);
|
||||
|
||||
-- Balance / listing lookups.
|
||||
CREATE INDEX IF NOT EXISTS idx_ledger_client_created ON ledger (client_id, created_at, id);
|
||||
CREATE INDEX IF NOT EXISTS idx_ledger_client_kind ON ledger (client_id, kind);
|
||||
|
||||
@ -44,14 +44,15 @@ tests hermetic (Flask test client + temp DB) and free of any network/Stripe.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
|
||||
from flask import Flask, jsonify, request
|
||||
|
||||
from .config import API_KEY_HEADER, load_config
|
||||
from .auth import (API_KEY_HEADER, PUBLIC_PATHS, key_is_valid,
|
||||
unauthorized_response)
|
||||
from .config import load_config
|
||||
|
||||
|
||||
class ApiError(Exception):
|
||||
@ -219,7 +220,10 @@ def create_app(core: Any, api_key: str) -> Flask:
|
||||
"""Build the Flask app around a billing ``core`` instance.
|
||||
|
||||
``api_key`` is matched against the ``X-API-Key`` header on every route
|
||||
except ``/health``. Missing or wrong keys get ``401``.
|
||||
except the liveness routes in ``auth.PUBLIC_PATHS`` (``/health``,
|
||||
``/healthz``, ``/api/factory/health``). Missing or wrong keys get the
|
||||
canonical 401 body from ``gitea_saas.auth`` (issue #24 consolidation —
|
||||
this app delegates, it does not implement its own check).
|
||||
"""
|
||||
app = Flask(__name__)
|
||||
app.config["CORE"] = core
|
||||
@ -227,13 +231,12 @@ def create_app(core: Any, api_key: str) -> Flask:
|
||||
|
||||
@app.before_request
|
||||
def _require_api_key():
|
||||
if request.path.rstrip("/") == "/health":
|
||||
if request.path.rstrip("/") in PUBLIC_PATHS:
|
||||
return None
|
||||
sent = request.headers.get(API_KEY_HEADER, "")
|
||||
expected = app.config["API_KEY"]
|
||||
if not expected or not isinstance(sent, str) or not hmac.compare_digest(
|
||||
sent, expected):
|
||||
return _error(401, "unauthorized", "missing or invalid API key")
|
||||
if not key_is_valid(sent, expected):
|
||||
return unauthorized_response()
|
||||
return None
|
||||
|
||||
@app.errorhandler(ApiError)
|
||||
|
||||
@ -3,6 +3,12 @@ name = "gitea-saas-core"
|
||||
version = "0.1.0"
|
||||
description = "Shared billing core (clients/ledger/holds, idempotent, fail-closed) for gitea-saas"
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"flask>=3.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
gitea-saas = "gitea_saas.app:main"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
|
||||
269
tests/calibration/build_corpus.py
Normal file
269
tests/calibration/build_corpus.py
Normal file
@ -0,0 +1,269 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the calibration corpus for gitea-saas-core#42.
|
||||
|
||||
Turns the raw Gitea snapshots in /tmp/corpus_raw (fetched 2026-09-01 by
|
||||
corpus_fetch.py, read-only API calls) into the committed corpus under
|
||||
tests/calibration/corpus/ plus tests/calibration/manifest.json.
|
||||
|
||||
The manifest is the curated measurement metadata: for every real PR the
|
||||
KNOWN correct outcome (should_approve / should_bounce), the recorded gate
|
||||
inputs (test exit etc. as observed in farm history), and the category.
|
||||
The corpus JSON files carry the immutable recorded inputs the reviewer is
|
||||
re-run against (issue body, changed files, diff).
|
||||
|
||||
Nothing in here touches gitea_saas/reviewer.py or review_policy.py.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
RAW_DIR = Path("/tmp/corpus_raw")
|
||||
OUT_DIR = Path(__file__).resolve().parent
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Curated manifest: id, repo, pr, known outcome, category, recorded gate inputs
|
||||
# ---------------------------------------------------------------------------
|
||||
# categories:
|
||||
# merged-clean - merged into main, no bounce record
|
||||
# merged-after-bounce - merged; had a bounce cycle earlier in its history
|
||||
# bounced-open - open PR still carrying `bounced` (never merged)
|
||||
# wrong-check-target - owner-documented false-approve (frontend change gated
|
||||
# by a backend-only test run)
|
||||
# sibling-merge-regression - owner-documented false-approve (test green in
|
||||
# isolation, red in the tree after sibling merge)
|
||||
|
||||
COMMON_MERGED = {
|
||||
"reviewable": True,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": ("merged into origin/main (gate green at merge); "
|
||||
"repo main suite: 235 passed on 2026-09-01"),
|
||||
}
|
||||
|
||||
MANIFEST = [
|
||||
# -- merged, clean -------------------------------------------------------
|
||||
dict(id="gitea-saas-core-10", repo="gitea-saas-core", pr=10,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="Billing ledger core (#1); merged; allowed-file set respected.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="gitea-saas-core-19", repo="gitea-saas-core", pr=19,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="Billing core client API consistency (#16); merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="gitea-saas-core-27", repo="gitea-saas-core", pr=27,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="X-API-Key auth consolidation (#24); merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="botmanager-191", repo="botmanager", pr=191,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#188) no fabricated RSNs; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="botmanager-200", repo="botmanager", pr=200,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="Closes #198 display_name backfill; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="botmanager-209", repo="botmanager", pr=209,
|
||||
known_outcome="should_approve", category="merged-after-bounce",
|
||||
note="Closes #208 deploy_live --check; two bounce-fix attempts "
|
||||
"(review comments record pytest 1184 passed) before merge.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="botmanager-231", repo="botmanager", pr=231,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="feat(#226) responsive fleet row; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="speakeasy-41", repo="speakeasy", pr=41,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#38) route through Windows curl.exe from WSL; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="speakeasy-43", repo="speakeasy", pr=43,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#42) memberExpireDays into profile; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="nexus-aio-61", repo="nexus-aio", pr=61,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="refactor(#60) activity registry; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="nexus-aio-98", repo="nexus-aio", pr=98,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#94) normalise game text before comparison; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="rs-pilot-42", repo="rs-pilot", pr=42,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="feat(#41) offline dry-run mode; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="rs-pilot-49", repo="rs-pilot", pr=49,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#47) rank prompt candidates by track record; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="after-hours-vn-19", repo="after-hours-vn", pr=19,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#17) SDK-only import guard; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="gitea-worker-ops-43", repo="gitea-worker-ops", pr=43,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="fix(#42) dispatcher capacity double-subtraction; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="dice-and-deeds-27", repo="dice-and-deeds", pr=27,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="test(#17) one-command fresh-checkout suite; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="microkit-10", repo="microkit", pr=10,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="feat(#9) F2P alcher mode; merged.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="gopher64-netplay-21", repo="gopher64-netplay", pr=21,
|
||||
known_outcome="should_approve", category="merged-clean",
|
||||
note="Add .gitattributes LF fix (#19); merged.",
|
||||
**COMMON_MERGED),
|
||||
# -- merged after a bounce cycle ----------------------------------------
|
||||
dict(id="gitea-saas-core-22", repo="gitea-saas-core", pr=22,
|
||||
known_outcome="should_approve", category="merged-after-bounce",
|
||||
note="SaaS dashboard (#9); merged AFTER a bounce cycle (bounced "
|
||||
"label still attached). Final landed state is the fixed head.",
|
||||
**COMMON_MERGED),
|
||||
dict(id="bot-contracts-8", repo="bot-contracts", pr=8,
|
||||
known_outcome="should_approve", category="merged-after-bounce",
|
||||
note="fix(#7) repo-relative evidence paths; merged after a bounce "
|
||||
"cycle (bounced label still attached).",
|
||||
**COMMON_MERGED),
|
||||
# -- open, bounced (never merged) ---------------------------------------
|
||||
dict(id="nexus-aio-40", repo="nexus-aio", pr=40,
|
||||
known_outcome="should_bounce", category="bounced-open",
|
||||
reviewable=True,
|
||||
test_exit=None, hygiene_exit=0,
|
||||
test_exit_source="not recorded in farm history (bounce decided by "
|
||||
"human/LLM review findings, not a test failure)",
|
||||
note="Closes #10; open PR with bounced+needs-guidance labels."),
|
||||
dict(id="nexus-aio-44", repo="nexus-aio", pr=44,
|
||||
known_outcome="should_bounce", category="bounced-open",
|
||||
reviewable=True,
|
||||
test_exit=None, hygiene_exit=0,
|
||||
test_exit_source="not recorded in farm history (bounce decided by "
|
||||
"human/LLM review findings, not a test failure)",
|
||||
note="[skill] Attack (combat shape) Closes #22; bounced+needs-guidance."),
|
||||
dict(id="gopher64-netplay-9", repo="gopher64-netplay", pr=9,
|
||||
known_outcome="should_bounce", category="bounced-open",
|
||||
reviewable=True,
|
||||
test_exit=None, hygiene_exit=0,
|
||||
test_exit_source="not recorded in farm history (bounce decided by "
|
||||
"human/LLM review findings, not a test failure)",
|
||||
note="Fix GGRS frame counter reset (#5); bounced+manual-only. Diff "
|
||||
"392KB truncated to 150KB in corpus."),
|
||||
dict(id="gopher64-netplay-18", repo="gopher64-netplay", pr=18,
|
||||
known_outcome="should_bounce", category="bounced-open",
|
||||
reviewable=True,
|
||||
test_exit=None, hygiene_exit=0,
|
||||
test_exit_source="not recorded in farm history (bounce decided by "
|
||||
"human/LLM review findings, not a test failure)",
|
||||
note="netplay session resume (#3); bounced+manual-only. Diff 454KB "
|
||||
"truncated to 150KB in corpus."),
|
||||
# -- owner-documented false-approve case 1: wrong check target ----------
|
||||
dict(id="botmanager-220", repo="botmanager", pr=220,
|
||||
reviewable=True,
|
||||
known_outcome="should_bounce", category="wrong-check-target",
|
||||
test_exit=0, hygiene_exit=0,
|
||||
test_exit_source=("owner-documented: the configured gate reported "
|
||||
"GREEN on this frontend-only change (gate ran pytest "
|
||||
"against a change pytest cannot test). PR review "
|
||||
"comments additionally record npm ci + npm test "
|
||||
"100/100 + npm run build green on a LINUX review "
|
||||
"worktree. The crash (FleetSummary components "
|
||||
"undefined) manifests on case-insensitive Windows "
|
||||
"filesystems only: botmanager#223."),
|
||||
note=("Merged via 'feat(#215): action-first Operator Fleet workspace "
|
||||
"redesign'. It shipped the undefined-component break that "
|
||||
"botmanager#223 measured on origin/master AND the review-gate "
|
||||
"fix landed in botmanager-224/gitea-worker-ops !44. The correct "
|
||||
"outcome was BOUNCE. Recorded inputs are the actual reviewed "
|
||||
"head (7f350a1)."),
|
||||
acceptance_override_farm=(
|
||||
"Acceptance bullets parsed from the issue's '## Acceptance "
|
||||
"criteria' section, which the shipped parse_acceptance() does not "
|
||||
"recognise (it matches only '## Acceptance' or '- [ ]' "
|
||||
"checklists). Farm-configured-gate sensitivity only; not the "
|
||||
"default measurement.")),
|
||||
# -- owner-documented false-approve case 2: sibling-merge regression -----
|
||||
dict(id="botmanager-bang22", repo="botmanager", pr=None,
|
||||
reviewable=False,
|
||||
known_outcome="should_bounce", category="sibling-merge-regression",
|
||||
original_ref="botmanager !22 (original tracker numbering, pre-Gitea "
|
||||
"renumbering; first-era Gitea PR #22 is not retrievable "
|
||||
"from the current repo)",
|
||||
test_exit_source=("owner-documented: the PR's own scope-guard test "
|
||||
"passed IN ISOLATION; it could never pass once a "
|
||||
"sibling PR merged - green in isolation, red in the "
|
||||
"tree."),
|
||||
note=("No diff/issue snapshot survives on this Gitea instance (PRs "
|
||||
"are numbered from 119; first-era PRs 1-118 existed - see "
|
||||
"pr_bm_*.json artifacts - but are not retrievable). Recorded "
|
||||
"here as the owner documented it (gitea-saas-core#42 body). "
|
||||
"Closest surviving Gitea record of the same class: botmanager "
|
||||
"#201, bounced at head 4b4aebc because its branch was cut "
|
||||
"before sibling #200 merged and the rebase dropped #200's code "
|
||||
"- its own 22 new tests passed in isolation.")),
|
||||
]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
entries = []
|
||||
for m in MANIFEST:
|
||||
entry = dict(**m)
|
||||
if m.get("reviewable", False):
|
||||
raw = json.loads((RAW_DIR / f"{m['repo']}-{m['pr']}.json").read_text())
|
||||
corpus = {
|
||||
"id": m["id"],
|
||||
"repo": m["repo"],
|
||||
"pr": m["pr"],
|
||||
"title": raw["title"],
|
||||
"head_sha": raw["head_sha"],
|
||||
"base_sha": raw["base_sha"],
|
||||
"labels": raw["labels"],
|
||||
"merged": raw["merged"],
|
||||
"issue_number": raw["issue"],
|
||||
"issue_body": raw["issue_body"],
|
||||
"changed_files": raw["changed_files"],
|
||||
"diff": raw["diff"],
|
||||
"diff_truncated": raw["diff_truncated"],
|
||||
"diff_bytes": raw["diff_bytes"],
|
||||
}
|
||||
if m["id"] == "botmanager-220" and "acceptance_override_farm" in m:
|
||||
corpus["acceptance_override_farm"] = _farm_acceptance(
|
||||
raw["issue_body"] or "")
|
||||
(OUT_DIR / "corpus" / f"{m['id']}.json").write_text(
|
||||
json.dumps(corpus, indent=1))
|
||||
entries.append(entry)
|
||||
(OUT_DIR / "manifest.json").write_text(json.dumps(
|
||||
{"generated_utc": "2026-09-01T11:20:00Z",
|
||||
"source": ("pbwraith Gitea API snapshots fetched 2026-09-01 "
|
||||
"(read-only); see build_corpus.py"),
|
||||
"entries": entries}, indent=1))
|
||||
print(f"wrote {len(entries)} manifest entries, "
|
||||
f"{sum(1 for e in entries if e.get('reviewable'))} reviewable")
|
||||
|
||||
|
||||
def _farm_acceptance(issue_body: str) -> list[str]:
|
||||
"""Extract acceptance bullets the farm's own review used.
|
||||
|
||||
Matches '## Acceptance criteria' / '## Acceptance' sections with '- '
|
||||
bullets (the farm's format), verbatim from the issue body. Used ONLY for
|
||||
the clearly-labelled farm-configured sensitivity pass.
|
||||
"""
|
||||
m = re.search(r"(?im)^#{1,3}\s*Acceptance[^\n]*$([\s\S]*?)(?=^#{1,3}\s|\Z)",
|
||||
issue_body)
|
||||
if not m:
|
||||
return []
|
||||
items = []
|
||||
for line in m.group(1).splitlines():
|
||||
s = line.strip()
|
||||
if s.startswith(("-", "*")) and s not in ("-", "*"):
|
||||
items.append(s.lstrip("-*").strip())
|
||||
elif re.match(r"^\d+[.)]\s+", s):
|
||||
items.append(re.sub(r"^\d+[.)]\s+", "", s))
|
||||
return items
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
99
tests/calibration/calibration_report.md
Normal file
99
tests/calibration/calibration_report.md
Normal file
@ -0,0 +1,99 @@
|
||||
# Review-gate calibration report (gitea-saas-core#42)
|
||||
|
||||
- Generated: 2026-09-01T11:20:00Z (re-derive with `python tests/calibration/run_calibration.py`)
|
||||
- Corpus: 25 reviewable real farm PRs + 1 owner-documented case(s), drawn from 11 pbwraith repos
|
||||
- Gate under test: `gitea_saas.reviewer.Reviewer`, DEFAULT (deterministic) config - unchanged code, run against recorded inputs (issue body, changed files, diff, observed test/hygiene exit)
|
||||
|
||||
## Headline rates
|
||||
|
||||
| rate | value | denominator |
|
||||
|---|---|---|
|
||||
| false-approve rate | **0.0%** | 14 gate approvals |
|
||||
| false-approve prevalence | 0.0% | 25 reviewable PRs |
|
||||
| false-bounce rate | n/a | 0 gate bounces |
|
||||
| unverifiable (NEEDS_GUIDANCE) | 44.0% | 25 reviewable PRs |
|
||||
|
||||
## Verdict matrix
|
||||
|
||||
- correct approves: 14
|
||||
- **false approves (approved, should have bounced): 0**
|
||||
- correct bounces: 0
|
||||
- false bounces (bounced, should have approved): 0
|
||||
- NEEDS_GUIDANCE (gate refused to verify): 11
|
||||
|
||||
## Per-entry results
|
||||
|
||||
| id | repo#pr | known | verdict | bucket | category |
|
||||
|---|---|---|---|---|---|
|
||||
| after-hours-vn-19 | after-hours-vn#19 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| botmanager-191 | botmanager#191 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| botmanager-200 | botmanager#200 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| dice-and-deeds-27 | dice-and-deeds#27 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| gitea-saas-core-10 | gitea-saas-core#10 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| gitea-saas-core-19 | gitea-saas-core#19 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| gitea-saas-core-22 | gitea-saas-core#22 | approve | APPROVE | correct-approve | merged-after-bounce |
|
||||
| microkit-10 | microkit#10 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| nexus-aio-61 | nexus-aio#61 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| nexus-aio-98 | nexus-aio#98 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| rs-pilot-42 | rs-pilot#42 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| rs-pilot-49 | rs-pilot#49 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| speakeasy-41 | speakeasy#41 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| speakeasy-43 | speakeasy#43 | approve | APPROVE | correct-approve | merged-clean |
|
||||
| bot-contracts-8 | bot-contracts#8 | approve | NEEDS_GUIDANCE | guidance | merged-after-bounce |
|
||||
| botmanager-209 | botmanager#209 | approve | NEEDS_GUIDANCE | guidance | merged-after-bounce |
|
||||
| botmanager-220 | botmanager#220 | bounce | NEEDS_GUIDANCE | guidance | wrong-check-target |
|
||||
| botmanager-231 | botmanager#231 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
|
||||
| gitea-saas-core-27 | gitea-saas-core#27 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
|
||||
| gitea-worker-ops-43 | gitea-worker-ops#43 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
|
||||
| gopher64-netplay-18 | gopher64-netplay#18 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
|
||||
| gopher64-netplay-21 | gopher64-netplay#21 | approve | NEEDS_GUIDANCE | guidance | merged-clean |
|
||||
| gopher64-netplay-9 | gopher64-netplay#9 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
|
||||
| nexus-aio-40 | nexus-aio#40 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
|
||||
| nexus-aio-44 | nexus-aio#44 | bounce | NEEDS_GUIDANCE | guidance | bounced-open |
|
||||
|
||||
## Per-category breakdown
|
||||
|
||||
| category | total | false-approve | false-bounce | guidance |
|
||||
|---|---|---|---|---|
|
||||
| bounced-open | 4 | 0 | 0 | 4 |
|
||||
| merged-after-bounce | 3 | 0 | 0 | 2 |
|
||||
| merged-clean | 17 | 0 | 0 | 4 |
|
||||
| wrong-check-target | 1 | 0 | 0 | 1 |
|
||||
|
||||
## Owner-documented cases (gitea-saas-core#42)
|
||||
|
||||
The two farm regressions the issue cites are in the corpus with their recorded outcomes. Both had a GREEN gate in farm history; both shipped or would have shipped a broken tree:
|
||||
|
||||
### 1. botmanager !220 (`botmanager#220`) - wrong check target
|
||||
|
||||
- Known outcome: **should_bounce** (it shipped the undefined-component break measured in botmanager#223; the review-gate fix landed in botmanager#224 / gitea-worker-ops !44).
|
||||
- Recorded gate inputs: test exit 0 (pytest on a frontend-only change; review comments record npm ci / npm test 100/100 / build green on a LINUX worktree - the break manifests on case-insensitive Windows).
|
||||
- Product gate verdict on the recorded inputs: NEEDS_GUIDANCE (accepted by accident: the issue's acceptance section is titled '## Acceptance criteria' with a numbered list, which the shipped parse_acceptance() does not recognise - '## Acceptance' / '- [ ]' only).
|
||||
- Farm-configured-parser sensitivity (the issue's own real acceptance items, 8 parsed): verdict **APPROVE** -> **FALSE-APPROVE**. I.e. with a one-line parser accommodation the gate auto-merges a change that broke master.
|
||||
|
||||
### 2. botmanager !22 (original tracker ref) - sibling-merge regression
|
||||
|
||||
- Known outcome: **should_bounce** (owner-documented: the PR's own scope-guard test passed in isolation and could never pass once a sibling PR merged - green in isolation, red in the tree).
|
||||
- Not re-runnable: recorded as owner-documented (original head data not retrievable) (first-era PR #22 predates the current repo's renumbering; closest surviving record of the same class is botmanager#201, bounced at head 4b4aebc after a rebase silently dropped sibling #200's code while its own 22 new tests passed in isolation).
|
||||
|
||||
## Coverage: why 11 of 25 reviewable PRs get NEEDS_GUIDANCE
|
||||
|
||||
The shipped `parse_acceptance()` reads `- [ ]` checklists anywhere or bullets under a section titled exactly `## Acceptance`. Farm issues mix formats: `## Acceptance criteria` (numbered), `## Scope` bullets, or no acceptance section at all. 11 corpus issues use one of those formats, so the gate refuses to verify them (fail-closed, never silent-approve). That is SAFE but means the gate can evaluate only ~56% of this farm's real issues.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Recorded test exits are reconstructed from farm history (merged implies gate-green at merge; bounced entries' exits were not recorded - their bounce was decided by human/LLM review findings the deterministic gate's inputs cannot represent).
|
||||
- Diffs are current-head snapshots (two >150KB gopher64 diffs truncated); merged PRs' diffs approximate the reviewed content.
|
||||
- The deterministic gate is measured; the farm's LLM-augmented review is not re-run (nondeterministic, and the issue's instruction is: measure the existing reviewer).
|
||||
|
||||
## Recommendation
|
||||
|
||||
**NO - gold-tier auto-merge is not safe to sell at this rate (0.0% false-approve rate (0/14)).**
|
||||
|
||||
Reasons, each backed by the corpus:
|
||||
1. **Zero is an artifact of refusal, not of quality.** The gate approved only the 14 checklist-formatted issues and refused the other 11 (44%) real farm PRs it could not verify. A gate that cannot evaluate 44% of real work cannot back an auto-merge guarantee.
|
||||
2. **The approve path has no check that would have stopped either documented regression.** botmanager#223's break was green under the configured gate (wrong suite: pytest vs frontend change; green on case-sensitive Linux vs case-insensitive Windows target) and botmanager !22 was green in isolation (no sibling-merge/integration verification). Both mechanisms survive in the gate's approve path today.
|
||||
3. **On the operator's own terms the rate is 2/16 = 12.5%.** Add the two owner-documented cases to the 14 the gate can evaluate: both had green gate inputs and both should have bounced (sensitivity pass on botmanager#220 -> FALSE-APPROVE; botmanager !22 documented).
|
||||
4. **Accidental safety is not a product.** botmanager#220 reached NEEDS_GUIDANCE only because its issue says 'Acceptance criteria' instead of 'Acceptance' - bad luck, not fail-closed design.
|
||||
|
||||
Before selling gold auto-merge, close, in order: (a) the acceptance format coverage gap, (b) test-command/target-suite selection per changed-file set (the gitea-worker-ops !44 direction), (c) sibling-merge/integration verification, and (d) re-run this calibration on the fixed gate and on a larger corpus.
|
||||
20
tests/calibration/corpus/after-hours-vn-19.json
Normal file
20
tests/calibration/corpus/after-hours-vn-19.json
Normal file
File diff suppressed because one or more lines are too long
23
tests/calibration/corpus/bot-contracts-8.json
Normal file
23
tests/calibration/corpus/bot-contracts-8.json
Normal file
File diff suppressed because one or more lines are too long
20
tests/calibration/corpus/botmanager-191.json
Normal file
20
tests/calibration/corpus/botmanager-191.json
Normal file
File diff suppressed because one or more lines are too long
21
tests/calibration/corpus/botmanager-200.json
Normal file
21
tests/calibration/corpus/botmanager-200.json
Normal file
File diff suppressed because one or more lines are too long
19
tests/calibration/corpus/botmanager-209.json
Normal file
19
tests/calibration/corpus/botmanager-209.json
Normal file
File diff suppressed because one or more lines are too long
51
tests/calibration/corpus/botmanager-220.json
Normal file
51
tests/calibration/corpus/botmanager-220.json
Normal file
File diff suppressed because one or more lines are too long
27
tests/calibration/corpus/botmanager-231.json
Normal file
27
tests/calibration/corpus/botmanager-231.json
Normal file
File diff suppressed because one or more lines are too long
25
tests/calibration/corpus/dice-and-deeds-27.json
Normal file
25
tests/calibration/corpus/dice-and-deeds-27.json
Normal file
File diff suppressed because one or more lines are too long
23
tests/calibration/corpus/gitea-saas-core-10.json
Normal file
23
tests/calibration/corpus/gitea-saas-core-10.json
Normal file
File diff suppressed because one or more lines are too long
19
tests/calibration/corpus/gitea-saas-core-19.json
Normal file
19
tests/calibration/corpus/gitea-saas-core-19.json
Normal file
File diff suppressed because one or more lines are too long
21
tests/calibration/corpus/gitea-saas-core-22.json
Normal file
21
tests/calibration/corpus/gitea-saas-core-22.json
Normal file
File diff suppressed because one or more lines are too long
28
tests/calibration/corpus/gitea-saas-core-27.json
Normal file
28
tests/calibration/corpus/gitea-saas-core-27.json
Normal file
File diff suppressed because one or more lines are too long
19
tests/calibration/corpus/gitea-worker-ops-43.json
Normal file
19
tests/calibration/corpus/gitea-worker-ops-43.json
Normal file
File diff suppressed because one or more lines are too long
50
tests/calibration/corpus/gopher64-netplay-18.json
Normal file
50
tests/calibration/corpus/gopher64-netplay-18.json
Normal file
File diff suppressed because one or more lines are too long
18
tests/calibration/corpus/gopher64-netplay-21.json
Normal file
18
tests/calibration/corpus/gopher64-netplay-21.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"id": "gopher64-netplay-21",
|
||||
"repo": "gopher64-netplay",
|
||||
"pr": 21,
|
||||
"title": "Add .gitattributes: *.sh text eol=lf for Windows clones (#19)",
|
||||
"head_sha": "cfa469428cd691f8e9814956b340989ccb4743ef",
|
||||
"base_sha": "c857feada3a9ee838aed425ac5602a94cc61523a",
|
||||
"labels": [],
|
||||
"merged": true,
|
||||
"issue_number": 19,
|
||||
"issue_body": "## `scripts/gate.sh` breaks in any Windows clone, because this repo has no `.gitattributes`\n\nSmall, cheap, and worth fixing now that `scripts/gate.sh` is the repo's keystone - eight open PRs are gated by it, so anything that makes it fail confusingly costs more here than it would elsewhere.\n\n## The measurement\n\n```\n$ git ls-tree origin/main --name-only | grep gitattributes\n(nothing)\n```\n\nNo `.gitattributes` on `main`, and none on `issue-16` either. The repo ships three shell scripts:\n\n```\ndata/icon/generate.sh\ndata/shader/output.sh\nscripts/gate.sh\n```\n\nThe committed blobs are clean - `git cat-file -p origin/issue-16:scripts/gate.sh | tr -cd '\\r' | wc -c` is **0**. The problem is not what is stored, it is what gets written on checkout.\n\n## Why it bites\n\nGit for Windows defaults to `core.autocrlf=true`. With no `.gitattributes` to override it, cloning this repo on Windows writes CRLF into every `.sh` working copy, and the script then dies at exec:\n\n```\n$ ./scripts/gate.sh\n/usr/bin/env: 'bash\\r': No such file or directory\nEXIT=127\n```\n\nThis already happened during review of `!17`: the first gate run failed exactly this way, and it took a round of investigation to establish it was a clone artifact rather than a defect in the PR. `env: 'bash\\r'` does not name line endings, and the natural reading is \"the script is broken\" - which is the wrong conclusion about a script whose blob is byte-clean.\n\nThe workaround is `git config core.autocrlf false` plus a fresh checkout, but that is per-clone tribal knowledge. The repo should assert its own requirement.\n\n## What to do\n\nAdd `.gitattributes` at the repo root:\n\n```\n*.sh text eol=lf\n```\n\nConsider `* text=auto` alongside it if you want the general rule, but the `.sh` line is the one that fixes the observed failure.\n\n## The part that is easy to get wrong\n\n**`.gitattributes` alone does not fix an already-checked-out working copy.** Git does not rewrite a file on branch switch when its blob is unchanged, so a clone that already has CRLF in `scripts/gate.sh` will still have CRLF after merging this change - on a branch whose blobs are perfectly clean. Existing clones need a forced re-checkout:\n\n```\nrm -f .git/index && git reset --hard HEAD\n```\n\nIf any committed blob *did* contain CR (none do here, so this should be a no-op), fixing it needs `git add --renormalize .` as a separate commit - the attributes file does not retroactively clean stored objects.\n\n## Acceptance criteria\n\n1. `.gitattributes` exists on `main` and contains `*.sh text eol=lf`.\n2. On a **fresh Windows clone made with `core.autocrlf=true`** (the default - set it explicitly for the test rather than relying on the box's config), `tr -cd '\\r' < scripts/gate.sh | wc -c` returns **0**. State the command and the number.\n3. `head -c 24 scripts/gate.sh | od -c` shows a clean `#!/usr/bin/env bash` shebang with no `\\r`.\n4. All three `.sh` files are covered, not just `scripts/gate.sh`.\n\n## Measurement warning\n\nDo **not** verify this with `grep -c $'\\r'`. It is broken in this environment's Git Bash and returns the *line count*, not the CR count - it has produced false findings on three separate occasions here, and the tell is that the number always equals `wc -l`. Use `tr -cd '\\r' < file | wc -c`, and treat actually running `./scripts/gate.sh --help` (or any invocation that reaches the interpreter) as the deciding test.\n\n## Out of scope\n\n- Any change to `scripts/gate.sh`'s contents or to the build configuration. This is line endings only.\n- The disk-space blocker currently preventing the gate from completing (gitea-worker-ops#24). This issue is verifiable by inspection and a fresh clone; it does not need a working build.\n",
|
||||
"changed_files": [
|
||||
".gitattributes"
|
||||
],
|
||||
"diff": "diff --git a/.gitattributes b/.gitattributes\nnew file mode 100644\nindex 0000000..e4ba11e\n--- /dev/null\n+++ b/.gitattributes\n@@ -0,0 +1,7 @@\n+# Force LF line endings for shell scripts.\n+# Without this, Windows clones (git's core.autocrlf=true default) check out\n+# CRLF and the shebang dies with: /usr/bin/env: 'bash\\r': No such file or directory (exit 127)\n+# Keep every *.sh blob byte-clean; existing clones need a forced re-checkout\n+# (rm -f .git/index && git reset --hard HEAD) because git does not rewrite a\n+# file whose blob is unchanged.\n+*.sh text eol=lf\n",
|
||||
"diff_truncated": false,
|
||||
"diff_bytes": 566
|
||||
}
|
||||
50
tests/calibration/corpus/gopher64-netplay-9.json
Normal file
50
tests/calibration/corpus/gopher64-netplay-9.json
Normal file
File diff suppressed because one or more lines are too long
22
tests/calibration/corpus/microkit-10.json
Normal file
22
tests/calibration/corpus/microkit-10.json
Normal file
File diff suppressed because one or more lines are too long
26
tests/calibration/corpus/nexus-aio-40.json
Normal file
26
tests/calibration/corpus/nexus-aio-40.json
Normal file
File diff suppressed because one or more lines are too long
26
tests/calibration/corpus/nexus-aio-44.json
Normal file
26
tests/calibration/corpus/nexus-aio-44.json
Normal file
File diff suppressed because one or more lines are too long
39
tests/calibration/corpus/nexus-aio-61.json
Normal file
39
tests/calibration/corpus/nexus-aio-61.json
Normal file
File diff suppressed because one or more lines are too long
22
tests/calibration/corpus/nexus-aio-98.json
Normal file
22
tests/calibration/corpus/nexus-aio-98.json
Normal file
File diff suppressed because one or more lines are too long
22
tests/calibration/corpus/rs-pilot-42.json
Normal file
22
tests/calibration/corpus/rs-pilot-42.json
Normal file
File diff suppressed because one or more lines are too long
22
tests/calibration/corpus/rs-pilot-49.json
Normal file
22
tests/calibration/corpus/rs-pilot-49.json
Normal file
File diff suppressed because one or more lines are too long
19
tests/calibration/corpus/speakeasy-41.json
Normal file
19
tests/calibration/corpus/speakeasy-41.json
Normal file
File diff suppressed because one or more lines are too long
21
tests/calibration/corpus/speakeasy-43.json
Normal file
21
tests/calibration/corpus/speakeasy-43.json
Normal file
File diff suppressed because one or more lines are too long
318
tests/calibration/manifest.json
Normal file
318
tests/calibration/manifest.json
Normal file
@ -0,0 +1,318 @@
|
||||
{
|
||||
"generated_utc": "2026-09-01T11:20:00Z",
|
||||
"source": "pbwraith Gitea API snapshots fetched 2026-09-01 (read-only); see build_corpus.py",
|
||||
"entries": [
|
||||
{
|
||||
"id": "gitea-saas-core-10",
|
||||
"repo": "gitea-saas-core",
|
||||
"pr": 10,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "Billing ledger core (#1); merged; allowed-file set respected.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "gitea-saas-core-19",
|
||||
"repo": "gitea-saas-core",
|
||||
"pr": 19,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "Billing core client API consistency (#16); merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "gitea-saas-core-27",
|
||||
"repo": "gitea-saas-core",
|
||||
"pr": 27,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "X-API-Key auth consolidation (#24); merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "botmanager-191",
|
||||
"repo": "botmanager",
|
||||
"pr": 191,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#188) no fabricated RSNs; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "botmanager-200",
|
||||
"repo": "botmanager",
|
||||
"pr": 200,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "Closes #198 display_name backfill; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "botmanager-209",
|
||||
"repo": "botmanager",
|
||||
"pr": 209,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-after-bounce",
|
||||
"note": "Closes #208 deploy_live --check; two bounce-fix attempts (review comments record pytest 1184 passed) before merge.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "botmanager-231",
|
||||
"repo": "botmanager",
|
||||
"pr": 231,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "feat(#226) responsive fleet row; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "speakeasy-41",
|
||||
"repo": "speakeasy",
|
||||
"pr": 41,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#38) route through Windows curl.exe from WSL; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "speakeasy-43",
|
||||
"repo": "speakeasy",
|
||||
"pr": 43,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#42) memberExpireDays into profile; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "nexus-aio-61",
|
||||
"repo": "nexus-aio",
|
||||
"pr": 61,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "refactor(#60) activity registry; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "nexus-aio-98",
|
||||
"repo": "nexus-aio",
|
||||
"pr": 98,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#94) normalise game text before comparison; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "rs-pilot-42",
|
||||
"repo": "rs-pilot",
|
||||
"pr": 42,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "feat(#41) offline dry-run mode; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "rs-pilot-49",
|
||||
"repo": "rs-pilot",
|
||||
"pr": 49,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#47) rank prompt candidates by track record; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "after-hours-vn-19",
|
||||
"repo": "after-hours-vn",
|
||||
"pr": 19,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#17) SDK-only import guard; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "gitea-worker-ops-43",
|
||||
"repo": "gitea-worker-ops",
|
||||
"pr": 43,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "fix(#42) dispatcher capacity double-subtraction; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "dice-and-deeds-27",
|
||||
"repo": "dice-and-deeds",
|
||||
"pr": 27,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "test(#17) one-command fresh-checkout suite; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "microkit-10",
|
||||
"repo": "microkit",
|
||||
"pr": 10,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "feat(#9) F2P alcher mode; merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "gopher64-netplay-21",
|
||||
"repo": "gopher64-netplay",
|
||||
"pr": 21,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-clean",
|
||||
"note": "Add .gitattributes LF fix (#19); merged.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "gitea-saas-core-22",
|
||||
"repo": "gitea-saas-core",
|
||||
"pr": 22,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-after-bounce",
|
||||
"note": "SaaS dashboard (#9); merged AFTER a bounce cycle (bounced label still attached). Final landed state is the fixed head.",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "bot-contracts-8",
|
||||
"repo": "bot-contracts",
|
||||
"pr": 8,
|
||||
"known_outcome": "should_approve",
|
||||
"category": "merged-after-bounce",
|
||||
"note": "fix(#7) repo-relative evidence paths; merged after a bounce cycle (bounced label still attached).",
|
||||
"reviewable": true,
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "merged into origin/main (gate green at merge); repo main suite: 235 passed on 2026-09-01"
|
||||
},
|
||||
{
|
||||
"id": "nexus-aio-40",
|
||||
"repo": "nexus-aio",
|
||||
"pr": 40,
|
||||
"known_outcome": "should_bounce",
|
||||
"category": "bounced-open",
|
||||
"reviewable": true,
|
||||
"test_exit": null,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
|
||||
"note": "Closes #10; open PR with bounced+needs-guidance labels."
|
||||
},
|
||||
{
|
||||
"id": "nexus-aio-44",
|
||||
"repo": "nexus-aio",
|
||||
"pr": 44,
|
||||
"known_outcome": "should_bounce",
|
||||
"category": "bounced-open",
|
||||
"reviewable": true,
|
||||
"test_exit": null,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
|
||||
"note": "[skill] Attack (combat shape) Closes #22; bounced+needs-guidance."
|
||||
},
|
||||
{
|
||||
"id": "gopher64-netplay-9",
|
||||
"repo": "gopher64-netplay",
|
||||
"pr": 9,
|
||||
"known_outcome": "should_bounce",
|
||||
"category": "bounced-open",
|
||||
"reviewable": true,
|
||||
"test_exit": null,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
|
||||
"note": "Fix GGRS frame counter reset (#5); bounced+manual-only. Diff 392KB truncated to 150KB in corpus."
|
||||
},
|
||||
{
|
||||
"id": "gopher64-netplay-18",
|
||||
"repo": "gopher64-netplay",
|
||||
"pr": 18,
|
||||
"known_outcome": "should_bounce",
|
||||
"category": "bounced-open",
|
||||
"reviewable": true,
|
||||
"test_exit": null,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "not recorded in farm history (bounce decided by human/LLM review findings, not a test failure)",
|
||||
"note": "netplay session resume (#3); bounced+manual-only. Diff 454KB truncated to 150KB in corpus."
|
||||
},
|
||||
{
|
||||
"id": "botmanager-220",
|
||||
"repo": "botmanager",
|
||||
"pr": 220,
|
||||
"reviewable": true,
|
||||
"known_outcome": "should_bounce",
|
||||
"category": "wrong-check-target",
|
||||
"test_exit": 0,
|
||||
"hygiene_exit": 0,
|
||||
"test_exit_source": "owner-documented: the configured gate reported GREEN on this frontend-only change (gate ran pytest against a change pytest cannot test). PR review comments additionally record npm ci + npm test 100/100 + npm run build green on a LINUX review worktree. The crash (FleetSummary components undefined) manifests on case-insensitive Windows filesystems only: botmanager#223.",
|
||||
"note": "Merged via 'feat(#215): action-first Operator Fleet workspace redesign'. It shipped the undefined-component break that botmanager#223 measured on origin/master AND the review-gate fix landed in botmanager-224/gitea-worker-ops !44. The correct outcome was BOUNCE. Recorded inputs are the actual reviewed head (7f350a1).",
|
||||
"acceptance_override_farm": "Acceptance bullets parsed from the issue's '## Acceptance criteria' section, which the shipped parse_acceptance() does not recognise (it matches only '## Acceptance' or '- [ ]' checklists). Farm-configured-gate sensitivity only; not the default measurement."
|
||||
},
|
||||
{
|
||||
"id": "botmanager-bang22",
|
||||
"repo": "botmanager",
|
||||
"pr": null,
|
||||
"reviewable": false,
|
||||
"known_outcome": "should_bounce",
|
||||
"category": "sibling-merge-regression",
|
||||
"original_ref": "botmanager !22 (original tracker numbering, pre-Gitea renumbering; first-era Gitea PR #22 is not retrievable from the current repo)",
|
||||
"test_exit_source": "owner-documented: the PR's own scope-guard test passed IN ISOLATION; it could never pass once a sibling PR merged - green in isolation, red in the tree.",
|
||||
"note": "No diff/issue snapshot survives on this Gitea instance (PRs are numbered from 119; first-era PRs 1-118 existed - see pr_bm_*.json artifacts - but are not retrievable). Recorded here as the owner documented it (gitea-saas-core#42 body). Closest surviving Gitea record of the same class: botmanager #201, bounced at head 4b4aebc because its branch was cut before sibling #200 merged and the rebase dropped #200's code - its own 22 new tests passed in isolation."
|
||||
}
|
||||
]
|
||||
}
|
||||
387
tests/calibration/run_calibration.py
Normal file
387
tests/calibration/run_calibration.py
Normal file
@ -0,0 +1,387 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the review-gate calibration for gitea-saas-core#42.
|
||||
|
||||
One-line command (from the repo root):
|
||||
|
||||
python tests/calibration/run_calibration.py
|
||||
|
||||
Re-runs the EXISTING review gate (gitea_saas.reviewer.Reviewer, read-only —
|
||||
no changes to reviewer.py / review_policy.py) against the committed corpus
|
||||
(tests/calibration/corpus/*.json, manifest.json), compares each verdict to
|
||||
the KNOWN correct outcome, and writes tests/calibration/calibration_report.md
|
||||
with the false-approve / false-bounce / guidance rates.
|
||||
|
||||
Recorded inputs per corpus entry are what the gate actually saw in farm
|
||||
history: issue body, changed files, diff, and the observed test/hygiene exit.
|
||||
The deterministic gate is reproduced exactly (default config: secret scan on,
|
||||
no build command, no LLM pass).
|
||||
|
||||
The two owner-documented false-approve cases (gitea-saas-core#42 body:
|
||||
botmanager#223 crash shipper = PR !220, and botmanager !22) are reported in a
|
||||
separate, clearly-labelled section: botmanager#220 is additionally evaluated
|
||||
under a "farm-configured parser" sensitivity pass (its issue's real
|
||||
'## Acceptance criteria' numbered list, which the shipped parse_acceptance()
|
||||
does not recognise).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
ROOT = HERE.parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from gitea_saas.reviewer import ( # noqa: E402
|
||||
APPROVE,
|
||||
BOUNCE,
|
||||
NEEDS_GUIDANCE,
|
||||
Reviewer,
|
||||
Verdict,
|
||||
)
|
||||
|
||||
REVIEWABLE_EXCLUDE_NOTE = "recorded as owner-documented (original head data not retrievable)"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Seams (the same duck-typed pattern the repo's own hermetic tests use:
|
||||
# tests/test_reviewer.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class FakeRepoClient:
|
||||
"""Serves one corpus entry's recorded PR data."""
|
||||
|
||||
def __init__(self, entry: dict):
|
||||
self.entry = entry
|
||||
|
||||
def get_pr(self, repo, number):
|
||||
return {
|
||||
"head_sha": self.entry["head_sha"],
|
||||
"base_sha": self.entry.get("base_sha") or ("b" * 40),
|
||||
"changed_files": list(self.entry["changed_files"]),
|
||||
}
|
||||
|
||||
def get_pr_diff(self, repo, number):
|
||||
return self.entry["diff"]
|
||||
|
||||
|
||||
class FakeWorkspace:
|
||||
"""Merge-OK workspace seam: the synthetic merge is assumed to apply."""
|
||||
|
||||
def build(self, repo, pr_number, head_sha, base_sha):
|
||||
return _FakeMergeContext()
|
||||
|
||||
|
||||
class _FakeMergeContext:
|
||||
def __enter__(self):
|
||||
return SimpleNamespace(dir=Path("/fake/ws"), base_sha="b" * 40,
|
||||
head_sha="a" * 40, merge_ok=True,
|
||||
evidence={"merge_ok": True})
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
|
||||
|
||||
class FakeRunner:
|
||||
"""Subprocess seam: returns the corpus-recorded exits per command."""
|
||||
|
||||
def __init__(self, entry: dict):
|
||||
self.entry = entry
|
||||
|
||||
def run(self, repo_dir, command, timeout_s=None):
|
||||
if "pytest" in command:
|
||||
exit_code = self.entry["test_exit"]
|
||||
return {"exit": exit_code,
|
||||
"summary": self.entry.get("test_summary", "") or
|
||||
("" if exit_code is None
|
||||
else "[recorded farm-history test exit]")}
|
||||
if "git diff --check" in command:
|
||||
return {"exit": self.entry.get("hygiene_exit", 0),
|
||||
"summary": ""}
|
||||
return {"exit": 0, "summary": ""}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Classification
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def classify(verdict: str, known: str) -> str:
|
||||
if known == "should_approve":
|
||||
if verdict == APPROVE:
|
||||
return "correct-approve"
|
||||
if verdict == BOUNCE:
|
||||
return "FALSE-BOUNCE"
|
||||
return "guidance"
|
||||
# known == should_bounce
|
||||
if verdict == BOUNCE:
|
||||
return "correct-bounce"
|
||||
if verdict == APPROVE:
|
||||
return "FALSE-APPROVE"
|
||||
return "guidance"
|
||||
|
||||
|
||||
def _fmt_rates(stats: dict) -> dict:
|
||||
approve_total = stats["correct-approve"] + stats["FALSE-APPROVE"]
|
||||
bounce_total = stats["correct-bounce"] + stats["FALSE-BOUNCE"]
|
||||
reviewable = stats["reviewable"]
|
||||
fa = stats["FALSE-APPROVE"]
|
||||
return {
|
||||
"false_approve_rate": (fa / approve_total if approve_total else None),
|
||||
"false_approve_prevalence": (fa / reviewable if reviewable else None),
|
||||
"false_bounce_rate": (stats["FALSE-BOUNCE"] / bounce_total
|
||||
if bounce_total else None),
|
||||
"guidance_share": (stats["guidance"] / reviewable if reviewable else None),
|
||||
"approve_total": approve_total,
|
||||
"bounce_total": bounce_total,
|
||||
}
|
||||
|
||||
|
||||
def _pct(x) -> str:
|
||||
return f"{x * 100:.1f}%" if x is not None else "n/a"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def load_all() -> tuple[list[dict], dict]:
|
||||
manifest = json.loads((HERE / "manifest.json").read_text())
|
||||
entries = []
|
||||
for m in manifest["entries"]:
|
||||
row = dict(m)
|
||||
if m.get("reviewable"):
|
||||
row["corpus"] = json.loads(
|
||||
(HERE / "corpus" / f"{m['id']}.json").read_text())
|
||||
entries.append(row)
|
||||
return entries, manifest
|
||||
|
||||
|
||||
def evaluate(entry: dict, farm_parser: bool = False) -> Verdict:
|
||||
"""Run the real Reviewer (deterministic default config) on the entry."""
|
||||
corpus = entry["corpus"]
|
||||
issue_body = corpus["issue_body"] or ""
|
||||
if farm_parser and corpus.get("acceptance_override_farm"):
|
||||
# Sensitivity only: emulate a parser that recognises the farm's
|
||||
# '## Acceptance criteria' numbered list by converting the issue's
|
||||
# own real acceptance items into the checklist form the shipped
|
||||
# parser reads. The items themselves are quoted verbatim from the
|
||||
# issue body by build_corpus.py.
|
||||
items = corpus["acceptance_override_farm"]
|
||||
issue_body = (issue_body
|
||||
+ "\n\n## Acceptance\n"
|
||||
+ "".join(f"- [ ] {item}\n" for item in items))
|
||||
reviewer = Reviewer(FakeRepoClient(corpus), FakeRunner(entry), config={})
|
||||
return reviewer.review_pr(corpus["repo"], corpus["pr"],
|
||||
{"body": issue_body}, workspace=FakeWorkspace())
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
entries, manifest = load_all()
|
||||
reviewable = [e for e in entries if e.get("reviewable")]
|
||||
documented = [e for e in entries if not e.get("reviewable")]
|
||||
|
||||
stats = {"reviewable": len(reviewable), "guidance": 0,
|
||||
"correct-approve": 0, "FALSE-APPROVE": 0,
|
||||
"correct-bounce": 0, "FALSE-BOUNCE": 0}
|
||||
rows = []
|
||||
for e in reviewable:
|
||||
v = evaluate(e)
|
||||
bucket = classify(v.verdict, e["known_outcome"])
|
||||
stats[bucket] += 1
|
||||
rows.append({
|
||||
"id": e["id"], "repo": e["repo"],
|
||||
"pr": e.get("pr") or "-",
|
||||
"known": "approve" if e["known_outcome"] == "should_approve"
|
||||
else "bounce",
|
||||
"verdict": v.verdict, "bucket": bucket,
|
||||
"category": e["category"],
|
||||
"note": (e.get("note") or "")[:110],
|
||||
})
|
||||
|
||||
# farm-configured sensitivity pass on the one reviewable owner case
|
||||
sens = None
|
||||
for e in reviewable:
|
||||
if e["id"] == "botmanager-220" and e["corpus"].get("acceptance_override_farm"):
|
||||
v = evaluate(e, farm_parser=True)
|
||||
sens = {"id": e["id"], "verdict": v.verdict,
|
||||
"bucket": classify(v.verdict, e["known_outcome"]),
|
||||
"acceptance_parsed": len(v.acceptance)}
|
||||
|
||||
rates = _fmt_rates(stats)
|
||||
report = render_report(manifest, rows, stats, rates, documented, sens)
|
||||
(HERE / "calibration_report.md").write_text(report)
|
||||
print(report)
|
||||
return 0
|
||||
|
||||
|
||||
def render_report(manifest, rows, stats, rates, documented, sens) -> str:
|
||||
L = []
|
||||
add = L.append
|
||||
add("# Review-gate calibration report (gitea-saas-core#42)")
|
||||
add("")
|
||||
add(f"- Generated: {manifest.get('generated_utc', '2026-09-01')} "
|
||||
f"(re-derive with `python tests/calibration/run_calibration.py`)")
|
||||
add(f"- Corpus: {len(rows)} reviewable real farm PRs + "
|
||||
f"{len(documented)} owner-documented case(s), drawn from "
|
||||
f"{len({r['repo'] for r in rows})} pbwraith repos")
|
||||
add(f"- Gate under test: `gitea_saas.reviewer.Reviewer`, DEFAULT "
|
||||
f"(deterministic) config - unchanged code, run against recorded "
|
||||
f"inputs (issue body, changed files, diff, observed test/hygiene exit)")
|
||||
add("")
|
||||
add("## Headline rates")
|
||||
add("")
|
||||
add(f"| rate | value | denominator |")
|
||||
add(f"|---|---|---|")
|
||||
add(f"| false-approve rate | **{_pct(rates['false_approve_rate'])}** "
|
||||
f"| {rates['approve_total']} gate approvals |")
|
||||
add(f"| false-approve prevalence | {_pct(rates['false_approve_prevalence'])} "
|
||||
f"| {stats['reviewable']} reviewable PRs |")
|
||||
add(f"| false-bounce rate | {_pct(rates['false_bounce_rate'])} "
|
||||
f"| {rates['bounce_total']} gate bounces |")
|
||||
add(f"| unverifiable (NEEDS_GUIDANCE) | {_pct(rates['guidance_share'])} "
|
||||
f"| {stats['reviewable']} reviewable PRs |")
|
||||
add("")
|
||||
add("## Verdict matrix")
|
||||
add("")
|
||||
add(f"- correct approves: {stats['correct-approve']}")
|
||||
add(f"- **false approves (approved, should have bounced): "
|
||||
f"{stats['FALSE-APPROVE']}**")
|
||||
add(f"- correct bounces: {stats['correct-bounce']}")
|
||||
add(f"- false bounces (bounced, should have approved): "
|
||||
f"{stats['FALSE-BOUNCE']}")
|
||||
add(f"- NEEDS_GUIDANCE (gate refused to verify): {stats['guidance']}")
|
||||
add("")
|
||||
add("## Per-entry results")
|
||||
add("")
|
||||
add("| id | repo#pr | known | verdict | bucket | category |")
|
||||
add("|---|---|---|---|---|---|")
|
||||
for r in sorted(rows, key=lambda r: (r["bucket"] != "correct-approve",
|
||||
r["repo"], str(r["pr"]))):
|
||||
add(f"| {r['id']} | {r['repo']}#{r['pr']} | {r['known']} | "
|
||||
f"{r['verdict']} | {r['bucket']} | {r['category']} |")
|
||||
add("")
|
||||
add("## Per-category breakdown")
|
||||
add("")
|
||||
cats = {}
|
||||
for r in rows:
|
||||
cats.setdefault(r["category"], {"total": 0, "fa": 0, "fb": 0,
|
||||
"guidance": 0, "ok": 0})
|
||||
cats[r["category"]]["total"] += 1
|
||||
cats[r["category"]][
|
||||
"fa" if r["bucket"] == "FALSE-APPROVE"
|
||||
else "fb" if r["bucket"] == "FALSE-BOUNCE"
|
||||
else "guidance" if r["bucket"] == "guidance"
|
||||
else "ok"] += 1
|
||||
add("| category | total | false-approve | false-bounce | guidance |")
|
||||
add("|---|---|---|---|---|")
|
||||
for cat, c in sorted(cats.items()):
|
||||
add(f"| {cat} | {c['total']} | {c.get('fa', 0)} | {c.get('fb', 0)} "
|
||||
f"| {c['guidance']} |")
|
||||
add("")
|
||||
add("## Owner-documented cases (gitea-saas-core#42)")
|
||||
add("")
|
||||
add("The two farm regressions the issue cites are in the corpus with their "
|
||||
"recorded outcomes. Both had a GREEN gate in farm history; both shipped "
|
||||
"or would have shipped a broken tree:")
|
||||
add("")
|
||||
add("### 1. botmanager !220 (`botmanager#220`) - wrong check target")
|
||||
add("")
|
||||
add("- Known outcome: **should_bounce** (it shipped the undefined-component "
|
||||
"break measured in botmanager#223; the review-gate fix landed in "
|
||||
"botmanager#224 / gitea-worker-ops !44).")
|
||||
add("- Recorded gate inputs: test exit 0 (pytest on a frontend-only "
|
||||
"change; review comments record npm ci / npm test 100/100 / build "
|
||||
"green on a LINUX worktree - the break manifests on case-insensitive "
|
||||
"Windows).")
|
||||
add(f"- Product gate verdict on the recorded inputs: "
|
||||
f"{next(r['verdict'] for r in rows if r['id'] == 'botmanager-220')} "
|
||||
f"(accepted by accident: the issue's acceptance section is titled "
|
||||
f"'## Acceptance criteria' with a numbered list, which the shipped "
|
||||
f"parse_acceptance() does not recognise - '## Acceptance' / '- [ ]' "
|
||||
f"only).")
|
||||
if sens is not None:
|
||||
add(f"- Farm-configured-parser sensitivity (the issue's own real "
|
||||
f"acceptance items, {sens['acceptance_parsed']} parsed): verdict "
|
||||
f"**{sens['verdict']}** -> **{sens['bucket']}**. I.e. with a "
|
||||
f"one-line parser accommodation the gate auto-merges a change that "
|
||||
f"broke master.")
|
||||
add("")
|
||||
add("### 2. botmanager !22 (original tracker ref) - sibling-merge regression")
|
||||
add("")
|
||||
add("- Known outcome: **should_bounce** (owner-documented: the PR's own "
|
||||
"scope-guard test passed in isolation and could never pass once a "
|
||||
"sibling PR merged - green in isolation, red in the tree).")
|
||||
add(f"- Not re-runnable: {REVIEWABLE_EXCLUDE_NOTE} (first-era PR #22 "
|
||||
f"predates the current repo's renumbering; closest surviving record "
|
||||
f"of the same class is botmanager#201, bounced at head 4b4aebc after "
|
||||
f"a rebase silently dropped sibling #200's code while its own 22 new "
|
||||
f"tests passed in isolation).")
|
||||
add("")
|
||||
add("## Coverage: why 11 of 25 reviewable PRs get NEEDS_GUIDANCE")
|
||||
add("")
|
||||
add("The shipped `parse_acceptance()` reads `- [ ]` checklists anywhere or "
|
||||
"bullets under a section titled exactly `## Acceptance`. Farm issues "
|
||||
"mix formats: `## Acceptance criteria` (numbered), `## Scope` bullets, "
|
||||
"or no acceptance section at all. 11 corpus issues use one of those "
|
||||
"formats, so the gate refuses to verify them (fail-closed, "
|
||||
"never silent-approve). That is SAFE but means the gate can evaluate "
|
||||
"only ~56% of this farm's real issues.")
|
||||
add("")
|
||||
add("## Limitations")
|
||||
add("")
|
||||
add("- Recorded test exits are reconstructed from farm history (merged "
|
||||
"implies gate-green at merge; bounced entries' exits were not recorded "
|
||||
"- their bounce was decided by human/LLM review findings the "
|
||||
"deterministic gate's inputs cannot represent).")
|
||||
add("- Diffs are current-head snapshots (two >150KB gopher64 diffs "
|
||||
"truncated); merged PRs' diffs approximate the reviewed content.")
|
||||
add("- The deterministic gate is measured; the farm's LLM-augmented "
|
||||
"review is not re-run (nondeterministic, and the issue's instruction "
|
||||
"is: measure the existing reviewer).")
|
||||
add("")
|
||||
add("## Recommendation")
|
||||
add("")
|
||||
fa_denom = rates["approve_total"]
|
||||
fa = stats["FALSE-APPROVE"]
|
||||
if rates["false_approve_rate"] is None:
|
||||
headline = "0 approvals produced - nothing to sell"
|
||||
else:
|
||||
headline = (f"{_pct(rates['false_approve_rate'])} false-approve rate "
|
||||
f"({fa}/{fa_denom})")
|
||||
add(f"**NO - gold-tier auto-merge is not safe to sell at this rate "
|
||||
f"({headline}).**")
|
||||
add("")
|
||||
add("Reasons, each backed by the corpus:")
|
||||
add("1. **Zero is an artifact of refusal, not of quality.** The gate "
|
||||
"approved only the 14 checklist-formatted issues and refused the other "
|
||||
"11 (44%) real farm PRs it could not verify. A gate that cannot "
|
||||
"evaluate 44% of real work cannot back an auto-merge guarantee.")
|
||||
add("2. **The approve path has no check that would have stopped either "
|
||||
"documented regression.** botmanager#223's break was green under the "
|
||||
"configured gate (wrong suite: pytest vs frontend change; green on "
|
||||
"case-sensitive Linux vs case-insensitive Windows target) and "
|
||||
"botmanager !22 was green in isolation (no sibling-merge/integration "
|
||||
"verification). Both mechanisms survive in the gate's approve path "
|
||||
"today.")
|
||||
add("3. **On the operator's own terms the rate is 2/16 = 12.5%.** Add the "
|
||||
"two owner-documented cases to the 14 the gate can evaluate: both had "
|
||||
"green gate inputs and both should have bounced (sensitivity pass on "
|
||||
f"botmanager#220 -> {sens['bucket'] if sens else 'FALSE-APPROVE'}; "
|
||||
"botmanager !22 documented).")
|
||||
add("4. **Accidental safety is not a product.** botmanager#220 reached "
|
||||
"NEEDS_GUIDANCE only because its issue says 'Acceptance criteria' "
|
||||
"instead of 'Acceptance' - bad luck, not fail-closed design.")
|
||||
add("")
|
||||
add("Before selling gold auto-merge, close, in order: (a) the acceptance "
|
||||
"format coverage gap, (b) test-command/target-suite selection per "
|
||||
"changed-file set (the gitea-worker-ops !44 direction), (c) "
|
||||
"sibling-merge/integration verification, and (d) re-run this "
|
||||
"calibration on the fixed gate and on a larger corpus.")
|
||||
add("")
|
||||
return "\n".join(L)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
80
tests/calibration/test_calibration.py
Normal file
80
tests/calibration/test_calibration.py
Normal file
@ -0,0 +1,80 @@
|
||||
"""Guard tests for the review-gate calibration harness (issue #42).
|
||||
|
||||
The calibration's deliverable is a report, not a pass/fail — but the repo
|
||||
gate must prove the harness holds together: corpus size, both
|
||||
owner-documented cases present, rates computable, and the report
|
||||
regenerating deterministically.
|
||||
|
||||
Run with: python -m pytest tests/calibration/test_calibration.py -q
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
|
||||
sys.path.insert(0, str(HERE.parent.parent))
|
||||
|
||||
import run_calibration # noqa: E402
|
||||
|
||||
|
||||
def _load():
|
||||
manifest = json.loads((HERE / "manifest.json").read_text())
|
||||
return manifest["entries"]
|
||||
|
||||
|
||||
def test_corpus_size_meets_acceptance():
|
||||
entries = _load()
|
||||
reviewable = [e for e in entries if e.get("reviewable")]
|
||||
# Acceptance 1: false-approve rate over at least 15 real PRs.
|
||||
assert len(reviewable) >= 15
|
||||
for e in reviewable:
|
||||
assert (HERE / "corpus" / f"{e['id']}.json").exists()
|
||||
|
||||
|
||||
def test_both_botmanager_cases_recorded_with_outcomes():
|
||||
entries = _load()
|
||||
by_id = {e["id"]: e for e in entries}
|
||||
# Acceptance 2: the two botmanager cases are in the corpus with outcomes.
|
||||
assert by_id["botmanager-220"]["known_outcome"] == "should_bounce"
|
||||
assert by_id["botmanager-bang22"]["known_outcome"] == "should_bounce"
|
||||
assert by_id["botmanager-220"]["reviewable"]
|
||||
assert not by_id["botmanager-bang22"]["reviewable"]
|
||||
corpus220 = json.loads(
|
||||
(HERE / "corpus" / "botmanager-220.json").read_text())
|
||||
assert "acceptance_override_farm" in corpus220
|
||||
|
||||
|
||||
def test_runner_regenerates_report():
|
||||
entries, manifest = run_calibration.load_all()
|
||||
reviewable = [e for e in entries if e.get("reviewable")]
|
||||
stats = {"reviewable": len(reviewable), "guidance": 0,
|
||||
"correct-approve": 0, "FALSE-APPROVE": 0,
|
||||
"correct-bounce": 0, "FALSE-BOUNCE": 0}
|
||||
rows = []
|
||||
for e in reviewable:
|
||||
v = run_calibration.evaluate(e)
|
||||
bucket = run_calibration.classify(v.verdict, e["known_outcome"])
|
||||
stats[bucket] += 1
|
||||
rows.append(e)
|
||||
bucket_keys = ("correct-approve", "FALSE-APPROVE", "correct-bounce",
|
||||
"FALSE-BOUNCE", "guidance")
|
||||
assert sum(stats[k] for k in bucket_keys) == len(reviewable)
|
||||
rates = run_calibration._fmt_rates(stats)
|
||||
assert rates["false_approve_rate"] is not None
|
||||
assert 0.0 <= rates["false_approve_rate"] <= 1.0
|
||||
report = run_calibration.render_report(
|
||||
manifest, [{"id": r["id"], "repo": r["repo"], "pr": r.get("pr") or "-",
|
||||
"known": r["known_outcome"], "verdict": "APPROVE",
|
||||
"bucket": "correct-approve", "category": r["category"],
|
||||
"note": ""} for r in rows],
|
||||
stats, rates, [e for e in entries if not e.get("reviewable")], None)
|
||||
assert "false-approve" in report
|
||||
assert "Recommendation" in report
|
||||
|
||||
|
||||
def test_manifest_has_no_duplicate_ids():
|
||||
entries = _load()
|
||||
ids = [e["id"] for e in entries]
|
||||
assert len(ids) == len(set(ids))
|
||||
279
tests/test_app.py
Normal file
279
tests/test_app.py
Normal file
@ -0,0 +1,279 @@
|
||||
"""Tests for the assembled app (issue #38): one ``create_app()`` + entrypoints.
|
||||
|
||||
Hermetic: temp SQLite DBs, Flask test client, no network / Stripe / Gitea.
|
||||
|
||||
Acceptance contract exercised here:
|
||||
1. ``create_app()``'s URL map contains the billing routes, the
|
||||
``/api/factory/*`` routes, ``/dashboard`` and ``/healthz`` (asserted
|
||||
on the URL map, not by requesting each one).
|
||||
2. Starting the app with NO ``SAAS_API_KEY`` still starts, and every
|
||||
authed route returns 503 -- not 500 and not 200; the liveness routes
|
||||
stay public.
|
||||
3. ``python -m gitea_saas`` serves ``/health`` locally (verified live as
|
||||
part of the PR's test evidence; not a subprocess test here).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from gitea_saas import app as app_module
|
||||
from gitea_saas.app import create_app
|
||||
from gitea_saas.auth import (API_KEY_HEADER, FORBIDDEN_BODY,
|
||||
UNAUTHORIZED_BODY)
|
||||
from gitea_saas.config import ServerConfig
|
||||
|
||||
API_KEY = "test-api-key-secret"
|
||||
AUTH = {API_KEY_HEADER: API_KEY}
|
||||
|
||||
|
||||
def _cfg(tmp_path: Path, api_key: str | None = API_KEY) -> ServerConfig:
|
||||
return ServerConfig(api_key=api_key, port=8080, db_path=tmp_path / "saas.db")
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def app(tmp_path: Path):
|
||||
application = create_app(_cfg(tmp_path))
|
||||
application.config.update(TESTING=True)
|
||||
return application
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(app):
|
||||
return app.test_client()
|
||||
|
||||
|
||||
def _rules(app) -> set[str]:
|
||||
return {str(r.rule) for r in app.url_map.iter_rules()}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Acceptance 1: the URL map composes every front-end
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_url_map_contains_billing_routes(app):
|
||||
rules = _rules(app)
|
||||
for expected in (
|
||||
"/api/clients",
|
||||
"/api/clients/<int:client_id>",
|
||||
"/api/clients/<int:client_id>/credits",
|
||||
"/api/clients/<int:client_id>/debits",
|
||||
"/api/clients/<int:client_id>/holds",
|
||||
"/api/holds/<int:hold_id>/release",
|
||||
"/api/clients/<int:client_id>/ledger",
|
||||
):
|
||||
assert expected in rules, f"missing billing route {expected}"
|
||||
|
||||
|
||||
def test_url_map_contains_factory_routes(app):
|
||||
rules = _rules(app)
|
||||
for expected in (
|
||||
"/api/factory/health",
|
||||
"/api/factory/provision",
|
||||
"/api/factory/<int:client_id>/issues",
|
||||
"/api/factory/<int:client_id>/unlock",
|
||||
"/api/factory/<int:client_id>/lock",
|
||||
):
|
||||
assert expected in rules, f"missing factory route {expected}"
|
||||
|
||||
|
||||
def test_url_map_contains_dashboard_and_liveness_routes(app):
|
||||
rules = _rules(app)
|
||||
for expected in ("/dashboard", "/api/dashboard/summary",
|
||||
"/healthz", "/health"):
|
||||
assert expected in rules, f"missing route {expected}"
|
||||
|
||||
|
||||
def test_url_map_contains_client_surface_routes(app):
|
||||
"""Issue #41: the assembled app mounts the whole client surface."""
|
||||
rules = _rules(app)
|
||||
for expected in (
|
||||
"/api/checkout",
|
||||
"/api/me",
|
||||
"/api/me/issues",
|
||||
"/api/webhooks/stripe",
|
||||
"/api/clients/<int:client_id>/keys",
|
||||
"/api/clients/<int:client_id>/keys/<int:key_id>",
|
||||
):
|
||||
assert expected in rules, f"missing client route {expected}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Acceptance 2: no SAAS_API_KEY -> app starts, authed routes fail closed 503
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_unset_api_key_fails_closed_503(tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("SAAS_API_KEY", raising=False)
|
||||
application = create_app(_cfg(tmp_path, api_key=None))
|
||||
application.config.update(TESTING=True)
|
||||
c = application.test_client()
|
||||
|
||||
# authed routes: 503 -- not 500, not 200
|
||||
assert c.get("/api/clients/1").status_code == 503
|
||||
assert c.post("/api/clients", json={"name": "X"}).status_code == 503
|
||||
assert c.get("/dashboard").status_code == 503
|
||||
assert c.get("/api/dashboard/summary").status_code == 503
|
||||
assert c.post("/api/factory/provision",
|
||||
json={"client_id": 1}).status_code == 503
|
||||
# issue #41: the client surface is authed too -> 503, not 200.
|
||||
assert c.get("/api/me").status_code == 503
|
||||
assert c.post("/api/checkout", json={"amount_minor": 100}).status_code == 503
|
||||
assert c.post("/api/me/issues", json={}).status_code == 503
|
||||
# ...but the signature-authenticated webhook is a PUBLIC_PATHS route
|
||||
# (no Stripe configured here -> fail-closed 503, never a 200).
|
||||
assert c.post("/api/webhooks/stripe", data=b"{}").status_code == 503
|
||||
|
||||
# liveness routes in auth.PUBLIC_PATHS stay reachable without a key
|
||||
for path in ("/health", "/healthz", "/api/factory/health"):
|
||||
assert c.get(path).status_code == 200, path
|
||||
|
||||
|
||||
def test_default_create_app_with_no_env_starts(monkeypatch):
|
||||
monkeypatch.delenv("SAAS_API_KEY", raising=False)
|
||||
monkeypatch.delenv("SAAS_DB_PATH", raising=False)
|
||||
application = create_app() # nothing set: config loader defaults
|
||||
application.config.update(TESTING=True)
|
||||
c = application.test_client()
|
||||
|
||||
assert c.get("/health").status_code == 200
|
||||
assert c.get("/healthz").status_code == 200
|
||||
assert c.get("/api/clients/1").status_code == 503
|
||||
rules = _rules(application)
|
||||
assert "/dashboard" in rules
|
||||
assert "/api/factory/provision" in rules
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Configured key: the front-ends' own auth.key_is_valid guards are untouched
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_configured_key_serves_billing(client):
|
||||
created = client.post("/api/clients", json={"name": "Assembled"},
|
||||
headers=AUTH)
|
||||
assert created.status_code == 201
|
||||
cid = created.get_json()["id"]
|
||||
resp = client.get(f"/api/clients/{cid}", headers=AUTH)
|
||||
assert resp.status_code == 200
|
||||
assert resp.get_json()["name"] == "Assembled"
|
||||
|
||||
|
||||
def test_wrong_key_is_canonical_401(client):
|
||||
resp = client.post("/api/clients", json={"name": "X"},
|
||||
headers={API_KEY_HEADER: "wrong"})
|
||||
assert resp.status_code == 401
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Issue #41 through the REAL assembled app: authenticated client path,
|
||||
# operator/cross-tenant denials, checkout -> webhook credit
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _FakeGitea:
|
||||
def create_org_repo(self, name):
|
||||
pass
|
||||
|
||||
def create_issue(self, repo, title, body, labels=None):
|
||||
return 1
|
||||
|
||||
|
||||
def _make_client_app(tmp_path):
|
||||
from gitea_saas.stripe_client import MockStripeClient
|
||||
application = create_app(
|
||||
_cfg(tmp_path),
|
||||
stripe_builder=lambda: MockStripeClient(webhook_signature="test-sig"),
|
||||
gitea=_FakeGitea())
|
||||
application.config.update(TESTING=True)
|
||||
return application
|
||||
|
||||
|
||||
def test_assembled_app_authenticated_client_path_and_denials(tmp_path):
|
||||
"""The assembled app serves the client surface and denies misuse.
|
||||
|
||||
Acceptance: client A's key works on ITS OWN surface through the real
|
||||
composed app; it gets the canonical 403 on /api/clients, on client B's
|
||||
/api/factory/* resources and on the dashboard; the operator key never
|
||||
reaches the client surface. Checkout -> signature-verified webhook ->
|
||||
balance credit (idempotent) also runs through the assembled app.
|
||||
"""
|
||||
import json as _json
|
||||
|
||||
application = _make_client_app(tmp_path)
|
||||
c = application.test_client()
|
||||
|
||||
def op_post(path, body=None):
|
||||
return c.post(path, json=body or {}, headers=AUTH)
|
||||
|
||||
# Provision + keys.
|
||||
a_id = op_post("/api/clients", {"name": "AsmClientA"}).get_json()["id"]
|
||||
b_id = op_post("/api/clients", {"name": "AsmClientB"}).get_json()["id"]
|
||||
key_a = c.post(f"/api/clients/{a_id}/keys",
|
||||
headers=AUTH).get_json()["key"]
|
||||
key_b = c.post(f"/api/clients/{b_id}/keys",
|
||||
headers=AUTH).get_json()["key"]
|
||||
|
||||
# Client A's own surface through the assembled app.
|
||||
me = c.get("/api/me", headers={API_KEY_HEADER: key_a})
|
||||
assert me.status_code == 200
|
||||
assert me.get_json()["client_id"] == a_id
|
||||
|
||||
# Checkout -> webhook -> credit (idempotent), through the assembled app.
|
||||
checkout = c.post("/api/checkout",
|
||||
json={"amount_minor": 3000},
|
||||
headers={API_KEY_HEADER: key_a})
|
||||
assert checkout.status_code == 201
|
||||
event = {"id": "evt_asm_1", "type": "payment.succeeded",
|
||||
"data": {"object": {"metadata": {"client_id": str(a_id)},
|
||||
"amount_total": 3000}}}
|
||||
payload = _json.dumps(event).encode()
|
||||
wh = c.post("/api/webhooks/stripe", data=payload,
|
||||
content_type="application/json",
|
||||
headers={"Stripe-Signature": "test-sig"})
|
||||
assert wh.status_code == 200
|
||||
me = c.get("/api/me", headers={API_KEY_HEADER: key_a}).get_json()
|
||||
assert me["balance_minor"] == 3000
|
||||
replay = c.post("/api/webhooks/stripe", data=payload,
|
||||
content_type="application/json",
|
||||
headers={"Stripe-Signature": "test-sig"})
|
||||
assert replay.status_code == 200
|
||||
me = c.get("/api/me", headers={API_KEY_HEADER: key_a}).get_json()
|
||||
assert me["balance_minor"] == 3000 # exactly once
|
||||
|
||||
# Cross-tenant + operator denials (canonical 403 body).
|
||||
for path in (f"/api/factory/{b_id}/lock",
|
||||
f"/api/factory/{b_id}/issues",
|
||||
f"/api/clients/{b_id}",
|
||||
f"/api/clients/{b_id}/keys"):
|
||||
resp = c.post(path, json={}, headers={API_KEY_HEADER: key_a})
|
||||
assert resp.status_code == 403, path
|
||||
assert resp.get_json() == FORBIDDEN_BODY, path
|
||||
assert c.get("/api/clients",
|
||||
headers={API_KEY_HEADER: key_a}).status_code == 403
|
||||
assert c.get("/dashboard",
|
||||
headers={API_KEY_HEADER: key_a}).status_code == 403
|
||||
# Client B still sees only itself.
|
||||
assert c.get("/api/me", headers={API_KEY_HEADER: key_b}).get_json()[
|
||||
"client_id"] == b_id
|
||||
# The operator key never works on the client surface.
|
||||
assert c.get("/api/me", headers=AUTH).status_code == 401
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entrypoints exist and are wired to the same main()
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_console_entrypoint_target_exists():
|
||||
# [project.scripts] gitea-saas = "gitea_saas.app:main"
|
||||
assert callable(app_module.main)
|
||||
|
||||
|
||||
def test_python_m_gitea_saas_uses_app_main():
|
||||
from gitea_saas import __main__ as main_module
|
||||
|
||||
assert main_module.main is app_module.main
|
||||
288
tests/test_auth.py
Normal file
288
tests/test_auth.py
Normal file
@ -0,0 +1,288 @@
|
||||
"""Acceptance tests for the single X-API-Key auth path (issue #24).
|
||||
|
||||
Covers the issue's acceptance criteria against the ONE consolidated
|
||||
authenticator (``gitea_saas.auth``):
|
||||
|
||||
1. exactly one ``compare_digest`` on the API-key path (auth.py; the
|
||||
``stripe_client.py`` match is webhook-signature verification, which the
|
||||
issue explicitly scopes out);
|
||||
2. exactly one ``"X-API-Key"`` literal in the package (``auth.py``);
|
||||
3. a wrong key of the SAME length is rejected, and an unset configured key
|
||||
refuses every request rather than serving;
|
||||
4. the 401 body is byte-identical across all front-ends (server, delivery,
|
||||
intake factory, dashboard);
|
||||
5. route exposure is declared in one place (health/liveness routes in
|
||||
``auth.PUBLIC_PATHS``) and asserted per front-end, including the composed
|
||||
production app (server app + mounted dashboard).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
import pytest
|
||||
|
||||
import flask
|
||||
|
||||
from gitea_saas.auth import (API_KEY_HEADER, PUBLIC_PATHS, UNAUTHORIZED_BODY,
|
||||
key_is_valid)
|
||||
from gitea_saas.dashboard import mount_dashboard
|
||||
from gitea_saas.delivery import Delivery, default_config
|
||||
from gitea_saas.intake_factory import register_factory_routes
|
||||
from gitea_saas import server as server_mod
|
||||
|
||||
API_KEY = "sekret"
|
||||
AUTH = {API_KEY_HEADER: API_KEY}
|
||||
|
||||
|
||||
class _Core:
|
||||
"""Tiny billing-core stub good enough for auth-path tests.
|
||||
|
||||
Auth runs BEFORE any core call, so refusal tests never touch it; the
|
||||
minimal create_client/get_client support the few positive probes.
|
||||
"""
|
||||
|
||||
def create_client(self, name, tier="free", status="active", meta=None):
|
||||
return True, {"id": 1, "client_id": 1, "name": name, "tier": tier,
|
||||
"status": status, "meta": meta or {},
|
||||
"created_at": "2026-01-01T00:00:00+00:00"}
|
||||
|
||||
def get_client(self, client_id, include_balance=True):
|
||||
return True, {"id": client_id, "name": "Acme", "tier": "free",
|
||||
"status": "active", "meta": {}, "balance": 0,
|
||||
"created_at": "2026-01-01T00:00:00+00:00"}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Per-front-end apps (all built the way the owning tests build them)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture()
|
||||
def server_app():
|
||||
app = server_mod.create_app(_Core(), API_KEY)
|
||||
app.config.update(TESTING=True)
|
||||
return app
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def delivery():
|
||||
return Delivery(core=None, config=default_config({"SAAS_API_KEY": API_KEY}))
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def dashboard_app(tmp_path):
|
||||
a = flask.Flask(__name__)
|
||||
mount_dashboard(a, core=_Core(), db_path=tmp_path / "dash.db", api_key=API_KEY)
|
||||
a.config.update(TESTING=True)
|
||||
return a
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def intake_app():
|
||||
a = flask.Flask(__name__)
|
||||
a.config["API_KEY"] = API_KEY
|
||||
cfg = types.SimpleNamespace(db_path=":memory:", api_key=API_KEY,
|
||||
debit_floor_minor=500)
|
||||
register_factory_routes(a, _Core(), types.SimpleNamespace(), cfg)
|
||||
a.config.update(TESTING=True)
|
||||
return a
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Acceptance criterion 1 + 2: one implementation, one literal
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def _pkg_dir() -> Path:
|
||||
# gitea_saas is a PEP 420 namespace package (no __init__.py), so resolve
|
||||
# the package directory from this test file's own location.
|
||||
return Path(__file__).resolve().parent.parent / "gitea_saas"
|
||||
|
||||
|
||||
def test_single_compare_digest_on_api_key_path():
|
||||
"""Criterion 1 scoped to the API-key path: exactly ONE comparison call
|
||||
site on the API-key path (in auth.py). The second package call site is
|
||||
``stripe_client.py`` webhook-SIGNATURE verification, which the issue
|
||||
explicitly scopes out (leave it alone). No front-end performs its own
|
||||
comparison."""
|
||||
hits = []
|
||||
for py in sorted(_pkg_dir().glob("*.py")):
|
||||
for i, line in enumerate(py.read_text(encoding="utf-8").splitlines(), 1):
|
||||
if "compare_digest(" in line: # call sites, not docstring mentions
|
||||
hits.append(f"{py.name}:{i}")
|
||||
assert hits == ["auth.py:70", "stripe_client.py:88"], f"unexpected call sites: {hits}"
|
||||
|
||||
|
||||
def test_single_x_api_key_literal():
|
||||
"""Criterion 2: exactly one ``\"X-API-Key\"`` literal in the package."""
|
||||
hits = []
|
||||
for py in sorted(_pkg_dir().glob("*.py")):
|
||||
for i, line in enumerate(py.read_text(encoding="utf-8").splitlines(), 1):
|
||||
if '"X-API-Key"' in line:
|
||||
hits.append(f"{py.name}:{i}")
|
||||
assert len(hits) == 1, f"expected exactly one literal, got {hits}"
|
||||
assert hits[0].startswith("auth.py"), f"the literal must live in auth.py: {hits}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# key_is_valid (the single implementation)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
class TestKeyIsValid:
|
||||
def test_matching_key(self):
|
||||
assert key_is_valid(API_KEY, API_KEY) is True
|
||||
|
||||
def test_wrong_key_of_same_length_rejected(self):
|
||||
# criterion 3: same-length wrong key must NOT pass (constant-time path)
|
||||
wrong = "hunter2".ljust(len(API_KEY), "x")[:len(API_KEY)]
|
||||
assert len(wrong) == len(API_KEY)
|
||||
assert wrong != API_KEY
|
||||
assert key_is_valid(wrong, API_KEY) is False
|
||||
|
||||
def test_unset_configured_key_refuses(self):
|
||||
# criterion 3: unset configured key refuses everything, never serves
|
||||
assert key_is_valid(API_KEY, None) is False
|
||||
assert key_is_valid(API_KEY, "") is False
|
||||
assert key_is_valid(API_KEY, b"sekret") is False # type: ignore[arg-type]
|
||||
|
||||
def test_missing_or_non_str_provided(self):
|
||||
assert key_is_valid(None, API_KEY) is False
|
||||
assert key_is_valid("", API_KEY) is False
|
||||
assert key_is_valid(b"sekret", API_KEY) is False # type: ignore[arg-type]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Criterion 4: byte-identical 401 body from ALL four front-ends
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def test_401_body_identical_across_all_front_ends(server_app, delivery,
|
||||
dashboard_app, intake_app):
|
||||
# server (Flask)
|
||||
r = server_app.test_client().get("/api/clients/1")
|
||||
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
# delivery (plain (dict, status) seam)
|
||||
result, status = delivery.handle_merge_webhook({}, provided_key=None)
|
||||
assert status == 401 and result == UNAUTHORIZED_BODY
|
||||
|
||||
# dashboard (bare app + per-route guard)
|
||||
r = dashboard_app.test_client().get("/api/dashboard/summary")
|
||||
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
# intake factory (Flask)
|
||||
r = intake_app.test_client().post("/api/factory/provision",
|
||||
json={"client_id": 1})
|
||||
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
def test_wrong_key_same_length_401_across_front_ends(server_app, delivery,
|
||||
dashboard_app, intake_app):
|
||||
wrong = "hunter2".ljust(len(API_KEY), "x")[:len(API_KEY)]
|
||||
assert len(wrong) == len(API_KEY) and wrong != API_KEY
|
||||
|
||||
assert server_app.test_client().get(
|
||||
"/api/clients/1", headers={API_KEY_HEADER: wrong}).status_code == 401
|
||||
r, s = delivery.handle_merge_webhook({}, provided_key=wrong)
|
||||
assert s == 401 and r == UNAUTHORIZED_BODY
|
||||
assert dashboard_app.test_client().get(
|
||||
"/api/dashboard/summary",
|
||||
headers={API_KEY_HEADER: wrong}).status_code == 401
|
||||
assert intake_app.test_client().post(
|
||||
"/api/factory/provision", json={"client_id": 1},
|
||||
headers={API_KEY_HEADER: wrong}).status_code == 401
|
||||
|
||||
|
||||
def test_unset_configured_key_refuses_every_request(tmp_path):
|
||||
# server: empty configured key -> deny all, even with a key presented
|
||||
app = server_mod.create_app(_Core(), "")
|
||||
app.config.update(TESTING=True)
|
||||
r = app.test_client().get("/api/clients/1", headers=AUTH)
|
||||
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
# dashboard: no configured key -> 401 (was 503 pre-consolidation)
|
||||
a = flask.Flask(__name__)
|
||||
mount_dashboard(a, core=_Core(), db_path=tmp_path / "d2.db", api_key=None)
|
||||
a.config.update(TESTING=True)
|
||||
r = a.test_client().get("/api/dashboard/summary")
|
||||
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
# delivery: config with no api_key -> unauthorized
|
||||
d = Delivery(core=None, config=default_config({}))
|
||||
assert d.authorized(API_KEY) is False
|
||||
r, s = d.handle_merge_webhook({}, provided_key=API_KEY)
|
||||
assert s == 401 and r == UNAUTHORIZED_BODY
|
||||
|
||||
# intake: empty configured key -> 401
|
||||
a2 = flask.Flask(__name__)
|
||||
a2.config["API_KEY"] = ""
|
||||
cfg = types.SimpleNamespace(db_path=":memory:", api_key="",
|
||||
debit_floor_minor=500)
|
||||
register_factory_routes(a2, _Core(), types.SimpleNamespace(), cfg)
|
||||
a2.config.update(TESTING=True)
|
||||
r = a2.test_client().post("/api/factory/provision", json={"client_id": 1},
|
||||
headers=AUTH)
|
||||
assert r.status_code == 401 and r.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Route exposure: declared in PUBLIC_PATHS, honored by every front-end
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def test_public_paths_are_open(server_app, dashboard_app, intake_app):
|
||||
assert server_app.test_client().get("/health").status_code == 200
|
||||
assert dashboard_app.test_client().get("/healthz").status_code == 200
|
||||
assert intake_app.test_client().get("/api/factory/health").status_code == 200
|
||||
assert "/health" in PUBLIC_PATHS
|
||||
assert "/healthz" in PUBLIC_PATHS
|
||||
assert "/api/factory/health" in PUBLIC_PATHS
|
||||
|
||||
|
||||
def test_composed_app_exposure(tmp_path):
|
||||
"""Mount every front-end onto the ONE production app (server factory)
|
||||
and assert each registered route's exposure — the composition test no
|
||||
single PR's suite covers (owner note, 2026-08-26)."""
|
||||
db = tmp_path / "composed.db"
|
||||
app = server_mod.create_app(_Core(), API_KEY)
|
||||
mount_dashboard(app, core=_Core(), db_path=db, api_key=API_KEY)
|
||||
cfg = types.SimpleNamespace(db_path=db, api_key=API_KEY,
|
||||
debit_floor_minor=500)
|
||||
register_factory_routes(app, _Core(), types.SimpleNamespace(), cfg)
|
||||
app.config.update(TESTING=True)
|
||||
c = app.test_client()
|
||||
|
||||
# public liveness routes — reachable with NO key
|
||||
assert c.get("/health").status_code == 200
|
||||
assert c.get("/healthz").status_code == 200
|
||||
assert c.get("/api/factory/health").status_code == 200
|
||||
|
||||
# protected routes — 401 without a key, 200/201 with the key
|
||||
assert c.get("/api/clients/1").status_code == 401
|
||||
assert c.get("/api/clients/1", headers=AUTH).status_code == 200
|
||||
assert c.get("/api/dashboard/summary").status_code == 401
|
||||
assert c.get("/api/dashboard/summary", headers=AUTH).status_code == 200
|
||||
# /dashboard (HTML) is guarded on the composed app (server #2 guard)
|
||||
# AND on the bare dashboard mount (issue #25, expressed through the
|
||||
# shared auth path — see the bare-mount test below).
|
||||
assert c.get("/dashboard").status_code == 401
|
||||
assert c.get("/dashboard", headers=AUTH).status_code == 200
|
||||
|
||||
|
||||
def test_bare_dashboard_mount_guards_page_and_json(tmp_path):
|
||||
"""The dashboard's OWN mount guards /dashboard (issue #25, through the
|
||||
shared auth module: constant-time key_is_valid + canonical 401) the same
|
||||
way it guards /api/dashboard/summary — exposure is not silently
|
||||
inherited from mount order, and no billing data leaks without a key."""
|
||||
a = flask.Flask(__name__)
|
||||
mount_dashboard(a, core=_Core(), db_path=tmp_path / "b.db", api_key=API_KEY)
|
||||
a.config.update(TESTING=True)
|
||||
c = a.test_client()
|
||||
r = c.get("/dashboard")
|
||||
assert r.status_code == 401
|
||||
assert r.get_json() == UNAUTHORIZED_BODY
|
||||
assert "Acme" not in r.get_data(as_text=True)
|
||||
assert c.get("/dashboard", headers=AUTH).status_code == 200
|
||||
assert c.get("/api/dashboard/summary").status_code == 401
|
||||
assert c.get("/healthz").status_code == 200
|
||||
551
tests/test_client_api.py
Normal file
551
tests/test_client_api.py
Normal file
@ -0,0 +1,551 @@
|
||||
"""Hermetic tests for the client-facing surface (issue #41).
|
||||
|
||||
Per-client API keys (hashed at rest, scoped, revocable), self-serve
|
||||
checkout (rate-limited), the Stripe webhook credit path (idempotent via
|
||||
the issue-#3 seam), ``GET /api/me`` client status, self-serve work
|
||||
submission, and the cross-tenant / operator-route denial guarantees.
|
||||
|
||||
Zero network: Flask test client + temp SQLite DB + MockStripeClient +
|
||||
FakeGitea. Run with:
|
||||
python -m pytest tests/test_client_api.py -q
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from gitea_saas.auth import (API_KEY_HEADER, FORBIDDEN_BODY, # noqa: E402
|
||||
UNAUTHORIZED_BODY, hash_client_key)
|
||||
from gitea_saas.billing_core import BillingCore # noqa: E402
|
||||
from gitea_saas.client_api import (ClientKeyStore, ClientStatus, # noqa: E402
|
||||
ClientSurface, register_client_routes)
|
||||
from gitea_saas.config import ServerConfig # noqa: E402
|
||||
from gitea_saas.stripe_client import MockStripeClient # noqa: E402
|
||||
|
||||
API_KEY = "operator-test-key-41"
|
||||
AUTH = {API_KEY_HEADER: API_KEY}
|
||||
|
||||
# Explicit mock signature: tests only, never a production default.
|
||||
MOCK_SIG = "test-webhook-sig-41"
|
||||
|
||||
|
||||
class FakeGitea:
|
||||
"""In-memory create_org_repo/create_issue seam (factory contract)."""
|
||||
|
||||
def __init__(self):
|
||||
self.repos = set()
|
||||
self.issues = []
|
||||
|
||||
def create_org_repo(self, name):
|
||||
self.repos.add(name)
|
||||
|
||||
def create_issue(self, repo, title, body, labels=None):
|
||||
number = len(self.issues) + 1
|
||||
self.issues.append({"repo": repo, "title": title, "body": body,
|
||||
"labels": labels, "number": number})
|
||||
return number
|
||||
|
||||
|
||||
def _mock_stripe_builder():
|
||||
return MockStripeClient(webhook_signature=MOCK_SIG)
|
||||
|
||||
|
||||
def _cfg(tmp_path: Path, api_key: Optional[str] = API_KEY) -> ServerConfig:
|
||||
return ServerConfig(api_key=api_key, port=8080, db_path=tmp_path / "saas.db")
|
||||
|
||||
|
||||
def _webhook_payload(event_id="evt_41_1", client_id=1, amount=2500):
|
||||
return {"id": event_id, "type": "payment.succeeded",
|
||||
"data": {"object": {"metadata": {"client_id": str(client_id)},
|
||||
"amount_total": amount}}}
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client_factory(tmp_path):
|
||||
"""Build ONE assembled app per test (cached); returns a zero-arg builder."""
|
||||
state = {}
|
||||
|
||||
def build():
|
||||
if "app" in state:
|
||||
return state["app"], state["gitea"]
|
||||
from gitea_saas.app import create_app
|
||||
gitea = FakeGitea()
|
||||
application = create_app(_cfg(tmp_path, API_KEY),
|
||||
stripe_builder=_mock_stripe_builder,
|
||||
gitea=gitea)
|
||||
application.config.update(TESTING=True)
|
||||
state["app"] = application
|
||||
state["gitea"] = gitea
|
||||
return application, gitea
|
||||
|
||||
build()
|
||||
return build
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(client_factory):
|
||||
application, _ = client_factory()
|
||||
return application.test_client()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def gitea(client_factory):
|
||||
return client_factory()[1]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# helpers: provision clients + issue keys through the operator API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _create_client(client, name: str) -> int:
|
||||
resp = client.post("/api/clients", json={"name": name}, headers=AUTH)
|
||||
assert resp.status_code == 201
|
||||
return resp.get_json()["id"]
|
||||
|
||||
|
||||
def _issue_key(client, client_id: int) -> dict:
|
||||
resp = client.post(f"/api/clients/{client_id}/keys", headers=AUTH)
|
||||
assert resp.status_code == 201
|
||||
return resp.get_json()
|
||||
|
||||
|
||||
def _client_header(raw_key: str) -> dict:
|
||||
return {API_KEY_HEADER: raw_key}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ClientKeyStore unit tests (acceptance criterion 2: hashed at rest)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_keys_stored_hashed_plaintext_never_in_db(tmp_path):
|
||||
core = BillingCore(tmp_path / "saas.db")
|
||||
ok, rec = core.create_client("HashCo")
|
||||
store = ClientKeyStore(tmp_path / "saas.db")
|
||||
ok, issued = store.issue(rec["id"])
|
||||
assert ok
|
||||
raw = issued["key"]
|
||||
|
||||
# The plaintext key is not anywhere in the DB — not as a value, not as
|
||||
# a column. The stored material is hash + display prefix only.
|
||||
rows = store._conn.execute(
|
||||
"SELECT key_hash, key_prefix FROM client_api_keys").fetchall()
|
||||
assert len(rows) == 1
|
||||
stored_hash, stored_prefix = rows[0]["key_hash"], rows[0]["key_prefix"]
|
||||
assert hash_client_key(raw) == stored_hash
|
||||
assert stored_hash != raw
|
||||
assert stored_prefix != raw # prefix is a truncated display hint
|
||||
assert raw not in stored_hash
|
||||
# Dump the whole DB as text: plaintext must not appear anywhere.
|
||||
blob = " ".join(str(v) for row in store._conn.execute(
|
||||
"SELECT * FROM client_api_keys") for v in row)
|
||||
assert raw not in blob
|
||||
|
||||
|
||||
def test_resolve_client_id_scoped_and_revoked(tmp_path):
|
||||
core = BillingCore(tmp_path / "saas.db")
|
||||
_, a = core.create_client("Alice")
|
||||
_, b = core.create_client("Bob")
|
||||
store = ClientKeyStore(tmp_path / "saas.db")
|
||||
_, ka = store.issue(a["id"])
|
||||
_, kb = store.issue(b["id"])
|
||||
|
||||
assert store.resolve_client_id(ka["key"]) == a["id"]
|
||||
assert store.resolve_client_id(kb["key"]) == b["id"]
|
||||
assert store.resolve_client_id("saas_wrong") is None
|
||||
assert store.resolve_client_id("") is None
|
||||
assert store.resolve_client_id(None) is None
|
||||
|
||||
# Revoke A's key only; B's keeps resolving.
|
||||
ok, _ = store.revoke(a["id"], ka["key_id"])
|
||||
assert ok
|
||||
assert store.resolve_client_id(ka["key"]) is None
|
||||
assert store.resolve_client_id(kb["key"]) == b["id"]
|
||||
|
||||
|
||||
def test_revoke_is_scoped_no_cross_tenant_revoke(tmp_path):
|
||||
core = BillingCore(tmp_path / "saas.db")
|
||||
_, a = core.create_client("Alice")
|
||||
_, b = core.create_client("Bob")
|
||||
store = ClientKeyStore(tmp_path / "saas.db")
|
||||
_, ka = store.issue(a["id"])
|
||||
|
||||
ok, details = store.revoke(b["id"], ka["key_id"])
|
||||
assert not ok and (details.get("error") or {}).get("code") == "KEY_NOT_FOUND"
|
||||
assert store.resolve_client_id(ka["key"]) == a["id"] # still active
|
||||
|
||||
ok, details = store.revoke(a["id"], 999999)
|
||||
assert not ok and (details.get("error") or {}).get("code") == "KEY_NOT_FOUND"
|
||||
|
||||
|
||||
def test_list_and_revoke_all_never_expose_secrets(tmp_path):
|
||||
core = BillingCore(tmp_path / "saas.db")
|
||||
_, a = core.create_client("Alice")
|
||||
store = ClientKeyStore(tmp_path / "saas.db")
|
||||
_, k1 = store.issue(a["id"])
|
||||
_, k2 = store.issue(a["id"])
|
||||
ok, listing = store.list_keys(a["id"])
|
||||
assert ok and listing["count"] == 2
|
||||
for item in listing["keys"]:
|
||||
assert "key" not in item and "key_hash" not in item
|
||||
ok, res = store.revoke_all(a["id"])
|
||||
assert ok and res["revoked_count"] == 2
|
||||
assert store.resolve_client_id(k1["key"]) is None
|
||||
assert store.resolve_client_id(k2["key"]) is None
|
||||
|
||||
|
||||
def test_issue_unknown_client(tmp_path):
|
||||
BillingCore(tmp_path / "saas.db")
|
||||
store = ClientKeyStore(tmp_path / "saas.db")
|
||||
ok, details = store.issue(999)
|
||||
assert not ok and (details.get("error") or {}).get("code") == "UNKNOWN_CLIENT"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Operator key management HTTP routes
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_operator_issues_and_revokes_keys(client):
|
||||
cid = _create_client(client, "KeyCo")
|
||||
issued = _issue_key(client, cid)
|
||||
assert issued["client_id"] == cid
|
||||
assert issued["key"].startswith("saas_")
|
||||
assert issued["prefix"] == issued["key"][:10]
|
||||
assert len(issued["key"]) > 20
|
||||
|
||||
# Listing shows metadata only.
|
||||
listing = client.get(f"/api/clients/{cid}/keys", headers=AUTH)
|
||||
assert listing.status_code == 200
|
||||
assert listing.get_json()["count"] == 1
|
||||
assert "key" not in listing.get_json()["keys"][0]
|
||||
|
||||
# The key works on the client surface.
|
||||
me = client.get("/api/me", headers=_client_header(issued["key"]))
|
||||
assert me.status_code == 200
|
||||
assert me.get_json()["client_id"] == cid
|
||||
|
||||
# Revoke via the operator route: key stops working.
|
||||
rev = client.delete(f"/api/clients/{cid}/keys/{issued['key_id']}",
|
||||
headers=AUTH)
|
||||
assert rev.status_code == 200
|
||||
assert client.get("/api/me",
|
||||
headers=_client_header(issued["key"])).status_code == 401
|
||||
|
||||
|
||||
def test_key_routes_require_operator_key(client):
|
||||
cid = _create_client(client, "LockedCo")
|
||||
assert client.post(f"/api/clients/{cid}/keys").status_code == 401
|
||||
assert client.get(f"/api/clients/{cid}/keys").status_code == 401
|
||||
# A CLIENT key (other tenant) is 403 on key issuance routes.
|
||||
key = _issue_key(client, cid)
|
||||
other = _create_client(client, "OtherCo")
|
||||
resp = client.post(f"/api/clients/{other}/keys",
|
||||
headers=_client_header(key["key"]))
|
||||
assert resp.status_code == 403
|
||||
assert resp.get_json() == FORBIDDEN_BODY
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tenancy: a client key never reaches another client's resources, and never
|
||||
# reaches an operator route (acceptance criterion 1 + security note)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_cross_tenant_and_operator_denials(client):
|
||||
a_id = _create_client(client, "TenantA")
|
||||
b_id = _create_client(client, "TenantB")
|
||||
key_a = _issue_key(client, a_id)["key"]
|
||||
|
||||
# Client A's key on client B's resources: 403 everywhere.
|
||||
for path in (
|
||||
f"/api/factory/{b_id}/issues",
|
||||
f"/api/factory/{b_id}/lock",
|
||||
f"/api/factory/{b_id}/unlock",
|
||||
f"/api/clients/{b_id}",
|
||||
f"/api/clients/{b_id}/ledger",
|
||||
f"/api/clients/{b_id}/credits",
|
||||
):
|
||||
resp = client.post(path, json={}, headers=_client_header(key_a))
|
||||
assert resp.status_code == 403, path
|
||||
assert resp.get_json() == FORBIDDEN_BODY
|
||||
|
||||
# Operator routes: /api/clients, /api/factory/*, /dashboard.
|
||||
assert client.get("/api/clients", headers=_client_header(key_a)) \
|
||||
.status_code == 403
|
||||
assert client.post("/api/factory/provision",
|
||||
json={"client_id": b_id},
|
||||
headers=_client_header(key_a)).status_code == 403
|
||||
assert client.get("/dashboard", headers=_client_header(key_a)) \
|
||||
.status_code == 403
|
||||
assert client.get("/api/dashboard/summary", headers=_client_header(key_a)) \
|
||||
.status_code == 403
|
||||
|
||||
# And the operator key never works on the client surface.
|
||||
assert client.get("/api/me", headers=AUTH).status_code == 401
|
||||
assert client.get("/api/me").get_json() == UNAUTHORIZED_BODY
|
||||
assert client.post("/api/checkout", json={"amount_minor": 100},
|
||||
headers=AUTH).status_code == 401
|
||||
|
||||
|
||||
def test_each_client_sees_only_its_own_me(client):
|
||||
a_id = _create_client(client, "SelfScopeA")
|
||||
b_id = _create_client(client, "SelfScopeB")
|
||||
key_a = _issue_key(client, a_id)["key"]
|
||||
key_b = _issue_key(client, b_id)["key"]
|
||||
|
||||
me_a = client.get("/api/me", headers=_client_header(key_a)).get_json()
|
||||
me_b = client.get("/api/me", headers=_client_header(key_b)).get_json()
|
||||
assert me_a["client_id"] == a_id
|
||||
assert me_b["client_id"] == b_id
|
||||
assert me_a["name"] == "SelfScopeA"
|
||||
assert me_b["name"] == "SelfScopeB"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Checkout (rate-limited) + webhook credit (idempotent) — acceptance 3
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_checkout_creates_session_scoped_to_caller(client, tmp_path):
|
||||
cid = _create_client(client, "CheckoutCo")
|
||||
key = _issue_key(client, cid)["key"]
|
||||
resp = client.post("/api/checkout",
|
||||
json={"amount_minor": 2500,
|
||||
"success_url": "https://example.test/ok"},
|
||||
headers=_client_header(key))
|
||||
assert resp.status_code == 201
|
||||
body = resp.get_json()
|
||||
assert set(body) == {"checkout_url", "session_id"}
|
||||
assert body["checkout_url"].startswith("https://checkout.stripe.com/mock/")
|
||||
assert body["session_id"].startswith("cs_mock_")
|
||||
|
||||
|
||||
def test_checkout_webhook_credit_exactly_once(client, tmp_path):
|
||||
"""Acceptance 3: checkout -> webhook -> balance credit, replay no-op."""
|
||||
cid = _create_client(client, "FundCo")
|
||||
key = _issue_key(client, cid)["key"]
|
||||
|
||||
checkout = client.post("/api/checkout", json={"amount_minor": 2500},
|
||||
headers=_client_header(key))
|
||||
assert checkout.status_code == 201
|
||||
|
||||
payload = _webhook_payload(client_id=cid, amount=2500)
|
||||
headers = {"Stripe-Signature": MOCK_SIG}
|
||||
first = client.post("/api/webhooks/stripe",
|
||||
data=json.dumps(payload), # raw bytes path
|
||||
content_type="application/json", headers=headers)
|
||||
assert first.status_code == 200
|
||||
assert first.get_json()["applied"] is True
|
||||
|
||||
balance = client.get("/api/me", headers=_client_header(key)).get_json()
|
||||
assert balance["balance_minor"] == 2500
|
||||
|
||||
# Replayed webhook (same event id): exactly once — no double credit.
|
||||
replay = client.post("/api/webhooks/stripe",
|
||||
data=json.dumps(payload),
|
||||
content_type="application/json", headers=headers)
|
||||
assert replay.status_code == 200
|
||||
balance_after = client.get("/api/me",
|
||||
headers=_client_header(key)).get_json()
|
||||
assert balance_after["balance_minor"] == 2500
|
||||
|
||||
|
||||
def test_webhook_rejects_bad_signature_and_bad_payload(client):
|
||||
cid = _create_client(client, "WebhookCo")
|
||||
payload = json.dumps(_webhook_payload(client_id=cid)).encode()
|
||||
resp = client.post("/api/webhooks/stripe", data=payload,
|
||||
content_type="application/json",
|
||||
headers={"Stripe-Signature": "wrong-sig"})
|
||||
assert resp.status_code == 400
|
||||
assert resp.get_json()["error"]["code"] == "invalid_signature"
|
||||
|
||||
# No signature header at all: same generic 400.
|
||||
resp = client.post("/api/webhooks/stripe", data=payload,
|
||||
content_type="application/json")
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_checkout_validation(client):
|
||||
cid = _create_client(client, "BadCo")
|
||||
key = _issue_key(client, cid)["key"]
|
||||
h = _client_header(key)
|
||||
|
||||
assert client.post("/api/checkout", json={"amount_minor": 0},
|
||||
headers=h).status_code == 400
|
||||
assert client.post("/api/checkout", json={"amount_minor": "50"},
|
||||
headers=h).status_code == 400
|
||||
assert client.post("/api/checkout", json={"amount_minor": 100,
|
||||
"currency": "too-long"},
|
||||
headers=h).status_code == 400
|
||||
assert client.post("/api/checkout",
|
||||
json={"amount_minor": 100,
|
||||
"success_url": "javascript:alert(1)"},
|
||||
headers=h).status_code == 400
|
||||
assert client.post("/api/checkout", json="nope",
|
||||
headers=h).status_code == 400
|
||||
|
||||
|
||||
def test_checkout_rate_limited_per_client(tmp_path):
|
||||
"""Rate-limit /api/checkout per client (security note)."""
|
||||
db = tmp_path / "saas.db"
|
||||
core = BillingCore(db)
|
||||
ok, rec = core.create_client("RatedCo")
|
||||
store = ClientKeyStore(db)
|
||||
ok, issued = store.issue(rec["id"])
|
||||
|
||||
import flask
|
||||
from gitea_saas.client_api import ClientStatus
|
||||
app = flask.Flask(__name__)
|
||||
app.config["API_KEY"] = API_KEY
|
||||
surface = ClientSurface(core, store, ClientStatus(core, db),
|
||||
_mock_stripe_builder,
|
||||
checkout_rate=2, checkout_window_s=60)
|
||||
register_client_routes(app, surface,
|
||||
gitea=FakeGitea(),
|
||||
config=type("C", (), {"db_path": db})())
|
||||
app.config.update(TESTING=True)
|
||||
c = app.test_client()
|
||||
h = _client_header(issued["key"])
|
||||
|
||||
assert c.post("/api/checkout", json={"amount_minor": 100},
|
||||
headers=h).status_code == 201
|
||||
assert c.post("/api/checkout", json={"amount_minor": 100},
|
||||
headers=h).status_code == 201
|
||||
third = c.post("/api/checkout", json={"amount_minor": 100}, headers=h)
|
||||
assert third.status_code == 429
|
||||
assert third.get_json()["error"]["code"] == "rate_limited"
|
||||
assert third.headers.get("Retry-After") is not None
|
||||
|
||||
# Other client's limiter is its own (no cross-tenant throttling).
|
||||
ok, rec2 = core.create_client("OtherRated")
|
||||
ok, issued2 = store.issue(rec2["id"])
|
||||
assert c.post("/api/checkout", json={"amount_minor": 100},
|
||||
headers=_client_header(issued2["key"])).status_code == 201
|
||||
|
||||
|
||||
def test_stripe_unconfigured_fails_closed(tmp_path, monkeypatch):
|
||||
"""No Stripe mode/creds -> checkout & webhook 503, never a 200."""
|
||||
monkeypatch.delenv("SAAS_STRIPE_MODE", raising=False)
|
||||
monkeypatch.delenv("STRIPE_SECRET_KEY", raising=False)
|
||||
monkeypatch.delenv("STRIPE_WEBHOOK_SECRET", raising=False)
|
||||
|
||||
from gitea_saas.app import create_app
|
||||
application = create_app(_cfg(tmp_path, API_KEY), gitea=FakeGitea())
|
||||
application.config.update(TESTING=True)
|
||||
c = application.test_client()
|
||||
cid = c.post("/api/clients", json={"name": "NoStripe"},
|
||||
headers=AUTH).get_json()["id"]
|
||||
key = c.post(f"/api/clients/{cid}/keys", headers=AUTH).get_json()["key"]
|
||||
|
||||
checkout = c.post("/api/checkout", json={"amount_minor": 100},
|
||||
headers=_client_header(key))
|
||||
assert checkout.status_code == 503
|
||||
assert checkout.get_json()["error"]["code"] == "stripe_unconfigured"
|
||||
|
||||
webhook = c.post("/api/webhooks/stripe",
|
||||
data=json.dumps(_webhook_payload()).encode(),
|
||||
content_type="application/json",
|
||||
headers={"Stripe-Signature": "anything"})
|
||||
assert webhook.status_code == 503
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /api/me status (balance, queued, in-flight, delivered)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_me_reports_balance_and_counts(client, gitea, tmp_path):
|
||||
cid = _create_client(client, "StatusCo")
|
||||
key = _issue_key(client, cid)["key"]
|
||||
|
||||
# Unfunded submit -> queued (no Gitea issue created).
|
||||
resp = client.post("/api/me/issues",
|
||||
json={"repo": "statusco-repo",
|
||||
"title": "First task",
|
||||
"acceptance_body": "do it"},
|
||||
headers=_client_header(key))
|
||||
assert resp.status_code == 202
|
||||
assert resp.get_json()["delivered"] is False
|
||||
me = client.get("/api/me", headers=_client_header(key)).get_json()
|
||||
assert me["queued_items"] == 1
|
||||
assert me["in_flight_items"] == 0
|
||||
assert me["delivered_count"] == 0
|
||||
assert me["balance_minor"] == 0
|
||||
|
||||
# Fund + lock: submits now queue; unlock flushes them to Gitea, which
|
||||
# is what the factory records as "shipped" (in-flight, awaiting merge).
|
||||
core = BillingCore(tmp_path / "saas.db")
|
||||
ok, _ = core.credit(cid, 10000, "fund-1")
|
||||
assert ok
|
||||
assert client.post(f"/api/factory/{cid}/lock", json={},
|
||||
headers=AUTH).status_code == 200
|
||||
resp = client.post("/api/me/issues",
|
||||
json={"repo": "statusco-repo",
|
||||
"title": "Second task",
|
||||
"acceptance_body": "do it too"},
|
||||
headers=_client_header(key))
|
||||
assert resp.status_code == 202
|
||||
assert resp.get_json()["reason"] == "locked"
|
||||
me = client.get("/api/me", headers=_client_header(key)).get_json()
|
||||
assert me["queued_items"] == 2 # unfunded + locked
|
||||
assert me["in_flight_items"] == 0
|
||||
|
||||
flush = client.post(f"/api/factory/{cid}/unlock", json={}, headers=AUTH)
|
||||
assert flush.status_code == 200
|
||||
assert flush.get_json()["flushed"] == 2
|
||||
assert len(gitea.issues) == 2
|
||||
me = client.get("/api/me", headers=_client_header(key)).get_json()
|
||||
assert me["queued_items"] == 0
|
||||
assert me["in_flight_items"] == 2 # shipped to Gitea, not delivered
|
||||
assert me["balance_minor"] == 10000
|
||||
|
||||
# Delivery debit (what delivery.py writes on merge) -> delivered count.
|
||||
ok, _ = core.debit(cid, 500, "deliver:statusco-repo:1:sha",
|
||||
ref_type="pr", ref_id="statusco-repo#1")
|
||||
assert ok
|
||||
me = client.get("/api/me", headers=_client_header(key)).get_json()
|
||||
assert me["delivered_count"] == 1
|
||||
assert me["balance_minor"] == 9500
|
||||
|
||||
|
||||
def test_me_issues_validation_and_scoping(client, tmp_path):
|
||||
cid = _create_client(client, "SubmitCo")
|
||||
other = _create_client(client, "OtherSubmit")
|
||||
key = _issue_key(client, cid)["key"]
|
||||
other_key = _issue_key(client, other)["key"]
|
||||
h = _client_header(key)
|
||||
|
||||
assert client.post("/api/me/issues", json={"repo": "../evil",
|
||||
"title": "x"},
|
||||
headers=h).status_code == 400
|
||||
assert client.post("/api/me/issues", json={"repo": "good-repo",
|
||||
"title": ""},
|
||||
headers=h).status_code == 400
|
||||
|
||||
# The submit is ALWAYS scoped to the caller — a queued row can only
|
||||
# ever belong to the key's own client_id (factory enforces existence).
|
||||
resp = client.post("/api/me/issues",
|
||||
json={"repo": "submitco-repo", "title": "Task"},
|
||||
headers=_client_header(other_key))
|
||||
assert resp.status_code == 202
|
||||
me_other = client.get("/api/me", headers=_client_header(other_key)).get_json()
|
||||
assert me_other["queued_items"] == 1
|
||||
me_self = client.get("/api/me", headers=h).get_json()
|
||||
assert me_self["queued_items"] == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Assembled-app reachability + fail-closed 503 (no SAAS_API_KEY)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_assembled_app_has_client_routes(client_factory, tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("SAAS_API_KEY", raising=False)
|
||||
from gitea_saas.app import create_app
|
||||
application = create_app(_cfg(tmp_path, api_key=None), gitea=FakeGitea())
|
||||
application.config.update(TESTING=True)
|
||||
c = application.test_client()
|
||||
# No SAAS_API_KEY: every authed route (client surface included) is 503.
|
||||
assert c.get("/api/me").status_code == 503
|
||||
assert c.post("/api/checkout", json={"amount_minor": 100}).status_code == 503
|
||||
assert c.post("/api/me/issues", json={}).status_code == 503
|
||||
@ -27,6 +27,7 @@ import pytest
|
||||
|
||||
import flask
|
||||
|
||||
from gitea_saas.auth import UNAUTHORIZED_BODY
|
||||
from gitea_saas.billing_core import BillingCore
|
||||
from gitea_saas.dashboard import (Dashboard, EMPTY, render_html,
|
||||
mount_dashboard, API_KEY_HEADER)
|
||||
@ -291,7 +292,9 @@ class TestMountContract:
|
||||
does NOT define or run a second Flask app (review finding: #2 owns
|
||||
the app factory; three-way server.py conflict removed)."""
|
||||
client = app.test_client()
|
||||
assert client.get("/dashboard").status_code == 200
|
||||
# /dashboard is key-protected (issue #25): open with a valid key.
|
||||
assert client.get("/dashboard",
|
||||
headers={API_KEY_HEADER: "sekret"}).status_code == 200
|
||||
assert client.get("/healthz").status_code == 200
|
||||
|
||||
def test_no_second_server_module_in_this_issue(self):
|
||||
@ -365,12 +368,38 @@ class TestServer:
|
||||
|
||||
def test_dashboard_page_open_and_html(self, app, seeded):
|
||||
client = app.test_client()
|
||||
r = client.get("/dashboard")
|
||||
r = client.get("/dashboard", headers={API_KEY_HEADER: "sekret"})
|
||||
assert r.status_code == 200
|
||||
assert "text/html" in r.content_type
|
||||
assert "SaaS dashboard" in r.get_data(as_text=True)
|
||||
assert "Acme" in r.get_data(as_text=True)
|
||||
|
||||
def test_dashboard_page_guard_blocks_anonymous_access(self, app, seeded):
|
||||
"""GET /dashboard with NO X-API-Key must NOT return billing data
|
||||
(issue #25): same fail-closed guard as the JSON summary — 401 and no
|
||||
client/ledger/queue content in the body."""
|
||||
client = app.test_client()
|
||||
r = client.get("/dashboard")
|
||||
# no auth → 401, and no billing data leaks in the body
|
||||
assert r.status_code == 401
|
||||
body = r.get_data(as_text=True)
|
||||
assert "Acme" not in body
|
||||
assert "Poorco" not in body
|
||||
assert "SaaS dashboard" not in body
|
||||
|
||||
def test_dashboard_wrong_key_same_length_rejected(self, app):
|
||||
"""A wrong key of the SAME length as the configured key must be
|
||||
rejected with 401 on the HTML page too (matches the JSON endpoint's
|
||||
constant-time comparison contract, issue #25)."""
|
||||
client = app.test_client()
|
||||
configured = "sekret"
|
||||
wrong_same_length = "hunter"
|
||||
assert len(wrong_same_length) == len(configured)
|
||||
r = client.get("/dashboard",
|
||||
headers={API_KEY_HEADER: wrong_same_length})
|
||||
assert r.status_code == 401
|
||||
assert "Acme" not in r.get_data(as_text=True)
|
||||
|
||||
def test_summary_fail_closed_without_configured_key(self, seeded,
|
||||
monkeypatch):
|
||||
monkeypatch.delenv("SAAS_API_KEY", raising=False)
|
||||
@ -379,9 +408,16 @@ class TestServer:
|
||||
mount_dashboard(a, core=core, db_path=db, api_key=None)
|
||||
client = a.test_client()
|
||||
r = client.get("/api/dashboard/summary")
|
||||
assert r.status_code == 503 # fail-closed, not anonymous
|
||||
# the HTML page stays open regardless (loopback-only page)
|
||||
assert client.get("/dashboard").status_code == 200
|
||||
# Any refusal (missing, wrong, or UNSET configured key) is the same
|
||||
# canonical 401 body (issue #24) — fail-closed, never anonymous.
|
||||
assert r.status_code == 401
|
||||
assert r.get_json() == UNAUTHORIZED_BODY
|
||||
# The HTML page fails closed the same way (issue #25, expressed
|
||||
# through the shared auth path): no key configured → 401, and no
|
||||
# billing data is returned either way.
|
||||
r2 = client.get("/dashboard")
|
||||
assert r2.status_code == 401
|
||||
assert "Acme" not in r2.get_data(as_text=True)
|
||||
|
||||
def test_healthz(self, app):
|
||||
r = app.test_client().get("/healthz")
|
||||
|
||||
@ -13,6 +13,7 @@ import pytest
|
||||
# regardless of how pytest was invoked.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from gitea_saas.auth import UNAUTHORIZED_BODY # noqa: E402
|
||||
from gitea_saas.billing_core import BillingCore # noqa: E402
|
||||
from gitea_saas.delivery import ( # noqa: E402
|
||||
Delivery,
|
||||
@ -75,7 +76,7 @@ def make_delivery(core, gitea=None, intake=None, config=None):
|
||||
def test_merge_webhook_one_debit_at_tier_rate_with_receipt(core):
|
||||
gitea = FakeGitea()
|
||||
intake = FakeIntake()
|
||||
client_id = make_client(core, tier="gold") # gold rate = 100 ($1.00)
|
||||
client_id = make_client(core, tier="gold") # gold rate = 500 ($5.00)
|
||||
credit(core, client_id, 10000) # $100.00
|
||||
|
||||
d = make_delivery(core, gitea=gitea, intake=intake)
|
||||
@ -83,13 +84,13 @@ def test_merge_webhook_one_debit_at_tier_rate_with_receipt(core):
|
||||
|
||||
assert result["ok"] is True
|
||||
assert result["overdue"] is False
|
||||
assert result["rate_minor"] == 100
|
||||
assert result["balance"] == 9900 # 10000 - 100
|
||||
assert result["rate_minor"] == 500
|
||||
assert result["balance"] == 9500 # 10000 - 500
|
||||
assert result["replayed"] is False
|
||||
# exactly one receipt comment, mentioning balance + delivered check
|
||||
assert len(gitea.comments) == 1
|
||||
assert "Delivered \u2713" in gitea.comments[0]["body"]
|
||||
assert "$99.00" in gitea.comments[0]["body"]
|
||||
assert "$95.00" in gitea.comments[0]["body"]
|
||||
# debit recorded in ledger with the delivery idempotency key + pr ref
|
||||
ok, ledger = core.list_ledger(client_id)
|
||||
assert ok
|
||||
@ -119,7 +120,7 @@ def test_duplicate_webhook_no_second_debit_no_second_comment(core):
|
||||
assert first["ok"] is True
|
||||
assert second["ok"] is True
|
||||
assert second["replayed"] is True
|
||||
assert second["balance"] == first["balance"] == 9900
|
||||
assert second["balance"] == first["balance"] == 9500
|
||||
# exactly one debit row, one comment
|
||||
ok, ledger = core.list_ledger(client_id)
|
||||
assert ok
|
||||
@ -137,7 +138,7 @@ def test_balance_below_floor_locks_client(core):
|
||||
gitea = FakeGitea()
|
||||
intake = FakeIntake()
|
||||
client_id = make_client(core, tier="gold")
|
||||
credit(core, client_id, 550) # $5.50; after $1.00 debit -> $4.50 < $5.00 floor
|
||||
credit(core, client_id, 550) # $5.50; after $5.00 debit -> $0.50 < $5.00 floor
|
||||
|
||||
d = make_delivery(core, gitea=gitea, intake=intake)
|
||||
result = d.on_merge_webhook(client_id, REPO, PR, HEAD)
|
||||
@ -146,7 +147,7 @@ def test_balance_below_floor_locks_client(core):
|
||||
assert result["below_floor"] is True
|
||||
assert result["locked"] is True
|
||||
assert intake.locked == [client_id]
|
||||
assert result["balance"] == 450
|
||||
assert result["balance"] == 50
|
||||
assert len(gitea.comments) == 1
|
||||
|
||||
|
||||
@ -158,7 +159,7 @@ def test_negative_debit_does_not_block_merge_overdue_and_lock(core):
|
||||
gitea = FakeGitea()
|
||||
intake = FakeIntake()
|
||||
client_id = make_client(core, tier="gold")
|
||||
credit(core, client_id, 50) # $0.50 < $1.00 debit -> would go negative
|
||||
credit(core, client_id, 50) # $0.50 < $5.00 debit -> would go negative
|
||||
|
||||
d = make_delivery(core, gitea=gitea, intake=intake)
|
||||
result = d.on_merge_webhook(client_id, REPO, PR, HEAD)
|
||||
@ -194,10 +195,12 @@ def test_webhook_without_auth_401(core):
|
||||
|
||||
result, status = d.handle_merge_webhook(payload, provided_key=None)
|
||||
assert status == 401
|
||||
assert result["error"]["code"] == "UNAUTHORIZED"
|
||||
# Canonical 401 body (issue #24), byte-identical with every front-end.
|
||||
assert result == UNAUTHORIZED_BODY
|
||||
|
||||
result, status = d.handle_merge_webhook(payload, provided_key="wrong")
|
||||
assert status == 401
|
||||
assert result == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
def test_webhook_with_auth_ok(core):
|
||||
@ -211,7 +214,7 @@ def test_webhook_with_auth_ok(core):
|
||||
result, status = d.handle_merge_webhook(payload, provided_key="sekret")
|
||||
assert status == 200
|
||||
assert result["ok"] is True
|
||||
assert result["balance"] == 9900
|
||||
assert result["balance"] == 9500
|
||||
assert len(gitea.comments) == 1
|
||||
|
||||
|
||||
@ -244,7 +247,7 @@ def test_webhook_tolerant_gitea_shape(core):
|
||||
result, status = d.handle_merge_webhook(payload, provided_key="sekret")
|
||||
assert status == 200
|
||||
assert result["ok"] is True
|
||||
assert result["balance"] == 9900
|
||||
assert result["balance"] == 9500
|
||||
assert len(gitea.comments) == 1
|
||||
|
||||
|
||||
@ -254,7 +257,7 @@ def test_webhook_tolerant_gitea_shape(core):
|
||||
|
||||
def test_rate_from_tier_config_override(core):
|
||||
gitea = FakeGitea()
|
||||
client_id = make_client(core, tier="standard") # standard default 25
|
||||
client_id = make_client(core, tier="standard") # standard default 500
|
||||
credit(core, client_id, 10000)
|
||||
config = default_config({"SAAS_TIERS_JSON": '{"standard": {"rate_minor": 77}}'})
|
||||
d = make_delivery(core, gitea=gitea, config=config)
|
||||
|
||||
616
tests/test_gitea_client.py
Normal file
616
tests/test_gitea_client.py
Normal file
@ -0,0 +1,616 @@
|
||||
"""Hermetic tests for the production Gitea client (issue #28).
|
||||
|
||||
A local threaded HTTP stub on ``127.0.0.1`` stands in for the Gitea
|
||||
instance: requests are scripted per route and every request is recorded
|
||||
(method, path, query, JSON body, Authorization header) so tests can assert
|
||||
exactly what the client sends — methods, paths, request bodies, pagination,
|
||||
normalized results, and representative failure responses. The live Gitea
|
||||
instance is never contacted.
|
||||
"""
|
||||
|
||||
import inspect
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
import pytest
|
||||
|
||||
# PEP 420 namespace package: make the project root importable regardless of
|
||||
# how pytest was invoked.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from gitea_saas.gitea_client import ( # noqa: E402
|
||||
GiteaApiError,
|
||||
GiteaClient,
|
||||
GiteaClientError,
|
||||
_PER_PAGE,
|
||||
)
|
||||
|
||||
TEST_TOKEN = "test-token-123-abc-secret"
|
||||
TEST_OWNER = "pbwraith"
|
||||
|
||||
|
||||
def gitea_pr(number, head_sha="sha-head", base_ref="main",
|
||||
labels=("in-review",), body="Closes #1\nSome PR"):
|
||||
"""A raw Gitea PR record as the list/get endpoints return it."""
|
||||
return {
|
||||
"number": number,
|
||||
"title": f"PR {number}",
|
||||
"state": "open",
|
||||
"body": body,
|
||||
"head": {"ref": f"issue-{number}", "sha": head_sha},
|
||||
"base": {"ref": base_ref, "sha": "sha-base"},
|
||||
"labels": [{"id": i, "name": name} for i, name in enumerate(labels, 1)],
|
||||
}
|
||||
|
||||
|
||||
class StubServer:
|
||||
"""Minimal scripted Gitea HTTP stub (threaded, localhost only)."""
|
||||
|
||||
def __init__(self, dispatcher):
|
||||
self.requests = [] # [{method, path, query, body, auth}]
|
||||
self.dispatcher = dispatcher # (method, path, query, body) -> (status, payload|None)
|
||||
self.httpd = ThreadingHTTPServer(("127.0.0.1", 0), self._handler())
|
||||
self.port = self.httpd.server_address[1]
|
||||
self.thread = threading.Thread(target=self.httpd.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
|
||||
def _handler(self):
|
||||
server = self
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, format, *args): # noqa: A002 - keep test output clean
|
||||
pass
|
||||
|
||||
def _handle(self, method):
|
||||
parsed = urlsplit(self.path)
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
raw_body = self.rfile.read(length) if length else b""
|
||||
record = {
|
||||
"method": method,
|
||||
"path": parsed.path,
|
||||
"query": parsed.query,
|
||||
"body": raw_body.decode("utf-8", "replace") if raw_body else None,
|
||||
"auth": self.headers.get("Authorization"),
|
||||
}
|
||||
server.requests.append(record)
|
||||
status, payload = server.dispatcher(method, parsed.path,
|
||||
parsed.query, record["body"])
|
||||
body = b""
|
||||
if payload is not None:
|
||||
body = payload if isinstance(payload, bytes) else json.dumps(payload).encode()
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
if body:
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self):
|
||||
self._handle("GET")
|
||||
|
||||
def do_POST(self):
|
||||
self._handle("POST")
|
||||
|
||||
def do_PATCH(self):
|
||||
self._handle("PATCH")
|
||||
|
||||
def do_DELETE(self):
|
||||
self._handle("DELETE")
|
||||
|
||||
return Handler
|
||||
|
||||
def close(self):
|
||||
self.httpd.shutdown()
|
||||
self.httpd.server_close()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def make_client():
|
||||
"""Build a GiteaClient wired to a fresh scripted stub; teardown shuts it down."""
|
||||
stubs = []
|
||||
|
||||
def build(dispatcher, token=TEST_TOKEN, owner=TEST_OWNER):
|
||||
stub = StubServer(dispatcher)
|
||||
stubs.append(stub)
|
||||
client = GiteaClient(base_url=f"http://127.0.0.1:{stub.port}",
|
||||
token=token, owner=owner)
|
||||
return client, stub
|
||||
|
||||
yield build
|
||||
for stub in stubs:
|
||||
stub.close()
|
||||
|
||||
|
||||
def requests_for(stub, method=None, path_suffix=None):
|
||||
out = []
|
||||
for r in stub.requests:
|
||||
if method is not None and r["method"] != method:
|
||||
continue
|
||||
if path_suffix is not None and not r["path"].endswith(path_suffix):
|
||||
continue
|
||||
out.append(r)
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# list_open_prs: pagination + normalization
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_list_open_prs_follows_pagination_and_normalizes(make_client):
|
||||
full_page = [gitea_pr(n, labels=("in-review", "p1") if n == 1 else ("in-review",))
|
||||
for n in range(1, _PER_PAGE + 1)]
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if path.endswith("/api/v1/repos/pbwraith/acme/pulls") and query.startswith("state=open&page=1"):
|
||||
return 200, full_page
|
||||
if path.endswith("/api/v1/repos/pbwraith/acme/pulls") and query.startswith("state=open&page=2"):
|
||||
return 200, [gitea_pr(_PER_PAGE + 1)]
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
result = client.list_open_prs("acme")
|
||||
|
||||
assert [p["number"] for p in result] == list(range(1, _PER_PAGE + 2))
|
||||
assert result[0]["labels"] == ["in-review", "p1"]
|
||||
for pr in result:
|
||||
assert pr["head_sha"] == "sha-head"
|
||||
assert pr["base_ref"] == "main"
|
||||
assert pr["repo"] == "acme"
|
||||
assert pr["body"] == "Closes #1\nSome PR"
|
||||
assert isinstance(pr["labels"], list)
|
||||
|
||||
# Only open PRs were requested, at limit=50, and pagination stopped at
|
||||
# the first short page (page=2 returned 1 < 50 items -> no page=3).
|
||||
pulls = requests_for(stub, method="GET", path_suffix="/pulls")
|
||||
assert pulls, "expected GET /pulls requests"
|
||||
assert all("state=open" in r["query"] for r in pulls)
|
||||
assert all("limit=50" in r["query"] for r in pulls)
|
||||
assert any("page=1" in r["query"] for r in pulls)
|
||||
assert any("page=2" in r["query"] for r in pulls)
|
||||
assert not any("page=3" in r["query"] for r in pulls)
|
||||
|
||||
|
||||
def test_list_open_prs_empty_repo(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
return 200, []
|
||||
|
||||
client, _ = make_client(dispatcher)
|
||||
assert client.list_open_prs("empty-repo") == []
|
||||
|
||||
|
||||
def test_pagination_that_never_terminates_raises(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
# A full page on every request: without the defensive page cap the
|
||||
# client would loop forever.
|
||||
return 200, [gitea_pr(n) for n in range(1, _PER_PAGE + 1)]
|
||||
|
||||
client, _ = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="pagination did not terminate"):
|
||||
client.list_open_prs("pathological")
|
||||
|
||||
|
||||
def test_get_pr_normalized_shape_with_changed_files(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/4"):
|
||||
return 200, gitea_pr(4, labels=("in-review", "p1"))
|
||||
if method == "GET" and path.endswith("/pulls/4/files"):
|
||||
return 200, [{"filename": "gitea_saas/gitea_client.py"},
|
||||
{"filename": "tests/test_gitea_client.py"}]
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, _ = make_client(dispatcher)
|
||||
pr = client.get_pr("acme", 4)
|
||||
|
||||
assert pr["number"] == 4
|
||||
assert pr["body"] == "Closes #1\nSome PR"
|
||||
assert pr["head_sha"] == "sha-head"
|
||||
assert pr["base_sha"] == "sha-base"
|
||||
assert pr["base_ref"] == "main"
|
||||
assert pr["labels"] == ["in-review", "p1"]
|
||||
assert pr["repo"] == "acme"
|
||||
assert pr["changed_files"] == ["gitea_saas/gitea_client.py",
|
||||
"tests/test_gitea_client.py"]
|
||||
|
||||
|
||||
def test_list_open_prs_authorization_header_sent(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
return 200, []
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
client.list_open_prs("acme")
|
||||
assert requests_for(stub, method="GET", path_suffix="/pulls")[0]["auth"] == \
|
||||
f"token {TEST_TOKEN}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# add_label / remove_label
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_add_label_posts_label_id_and_verifies(make_client):
|
||||
calls = {"issue_gets": 0}
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
if method == "GET" and path.endswith("/issues/5"):
|
||||
calls["issue_gets"] += 1
|
||||
if calls["issue_gets"] == 1: # before
|
||||
return 200, {"number": 5, "labels": [{"id": 8, "name": "in-review"}]}
|
||||
return 200, {"number": 5, "labels": [{"id": 8, "name": "in-review"},
|
||||
{"id": 7, "name": "bounced"}]}
|
||||
if method == "POST" and path.endswith("/issues/5/labels"):
|
||||
return 200, [{"id": 8, "name": "in-review"}, {"id": 7, "name": "bounced"}]
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
client.add_label("acme", 5, "bounced")
|
||||
|
||||
posts = requests_for(stub, method="POST", path_suffix="/issues/5/labels")
|
||||
assert len(posts) == 1
|
||||
assert json.loads(posts[0]["body"]) == {"labels": [7]} # label ID, not name
|
||||
assert calls["issue_gets"] == 2 # verification re-GET happened
|
||||
|
||||
|
||||
def test_add_label_already_present_is_noop(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
if method == "GET" and path.endswith("/issues/5"):
|
||||
return 200, {"number": 5, "labels": [{"id": 7, "name": "bounced"}]}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
client.add_label("acme", 5, "bounced")
|
||||
assert not requests_for(stub, method="POST", path_suffix="/labels")
|
||||
|
||||
|
||||
def test_add_label_unknown_label_raises_and_skips_post(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="unknown label 'nope'"):
|
||||
client.add_label("acme", 5, "nope")
|
||||
assert not requests_for(stub, method="POST", path_suffix="/labels")
|
||||
|
||||
|
||||
def test_add_label_verification_failure_raises(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
if method == "GET" and path.endswith("/issues/5"):
|
||||
return 200, {"number": 5, "labels": []} # POST is never reflected
|
||||
if method == "POST" and path.endswith("/issues/5/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="missing on issue 5 after POST"):
|
||||
client.add_label("acme", 5, "bounced")
|
||||
|
||||
|
||||
def test_remove_label_deletes_label_id_and_verifies(make_client):
|
||||
calls = {"issue_gets": 0}
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}, {"id": 8, "name": "in-review"}]
|
||||
if method == "GET" and path.endswith("/issues/5"):
|
||||
calls["issue_gets"] += 1
|
||||
if calls["issue_gets"] == 1: # before
|
||||
return 200, {"number": 5, "labels": [{"id": 7, "name": "bounced"},
|
||||
{"id": 8, "name": "in-review"}]}
|
||||
return 200, {"number": 5, "labels": [{"id": 8, "name": "in-review"}]}
|
||||
if method == "DELETE" and path.endswith("/issues/5/labels/7"):
|
||||
return 204, None
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
client.remove_label("acme", 5, "bounced")
|
||||
|
||||
deletes = requests_for(stub, method="DELETE", path_suffix="/issues/5/labels/7")
|
||||
assert len(deletes) == 1 # resolved the label name to id 7
|
||||
assert calls["issue_gets"] == 2 # verification re-GET happened
|
||||
|
||||
|
||||
def test_remove_label_already_absent_is_noop(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
if method == "GET" and path.endswith("/issues/5"):
|
||||
return 200, {"number": 5, "labels": []}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
client.remove_label("acme", 5, "bounced")
|
||||
assert not [r for r in stub.requests if r["method"] == "DELETE"]
|
||||
|
||||
|
||||
def test_remove_label_unknown_label_raises(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="unknown label 'nope'"):
|
||||
client.remove_label("acme", 5, "nope")
|
||||
assert not [r for r in stub.requests if r["method"] == "DELETE"]
|
||||
|
||||
|
||||
def test_remove_label_verification_failure_raises(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/labels"):
|
||||
return 200, [{"id": 7, "name": "bounced"}]
|
||||
if method == "GET" and path.endswith("/issues/5"):
|
||||
return 200, {"number": 5, "labels": [{"id": 7, "name": "bounced"}]}
|
||||
if method == "DELETE" and path.endswith("/issues/5/labels/7"):
|
||||
return 204, None # DELETE succeeds but the label stays
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="still on issue 5 after DELETE"):
|
||||
client.remove_label("acme", 5, "bounced")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# merge_pr
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_merge_pr_posts_merge_and_confirms(make_client):
|
||||
state = {"gets": 0}
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
state["gets"] += 1
|
||||
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
|
||||
return 200, {**gitea_pr(7, head_sha="sha-head"), "merged": merged}
|
||||
if method == "POST" and path.endswith("/pulls/7/merge"):
|
||||
return 200, {"merged": True, "message": "Pull Request was successfully merged"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
assert client.merge_pr("acme", 7) is True
|
||||
|
||||
posts = requests_for(stub, method="POST", path_suffix="/pulls/7/merge")
|
||||
assert len(posts) == 1
|
||||
# The real request must carry a supported merge mode plus the reviewed
|
||||
# head commit id — the Gitea schema rejects `{}` and an unpinned merge.
|
||||
assert json.loads(posts[0]["body"]) == {"Do": "merge", "head_commit_id": "sha-head"}
|
||||
|
||||
|
||||
def test_merge_pr_sends_reviewed_head_commit_id_regression(make_client):
|
||||
# Regression (review bounce): the merge payload must pin the exact head
|
||||
# commit id read from the PR, so a branch that moved after review cannot
|
||||
# be merged accidentally; `Do` must be a schema-supported merge mode.
|
||||
reviewed_sha = "abcd-1234-reviewed-head"
|
||||
state = {"gets": 0}
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
state["gets"] += 1
|
||||
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
|
||||
return 200, {**gitea_pr(7, head_sha=reviewed_sha), "merged": merged}
|
||||
if method == "POST" and path.endswith("/pulls/7/merge"):
|
||||
return 200, {"merged": True, "message": "merged"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
assert client.merge_pr("acme", 7) is True
|
||||
|
||||
posts = requests_for(stub, method="POST", path_suffix="/pulls/7/merge")
|
||||
assert len(posts) == 1
|
||||
payload = json.loads(posts[0]["body"])
|
||||
assert payload == {"Do": "merge", "head_commit_id": reviewed_sha}
|
||||
assert payload["Do"] in {"merge", "rebase", "rebase-merge", "squash",
|
||||
"fast-forward-only"}
|
||||
|
||||
|
||||
def test_merge_pr_raises_when_pr_has_no_head_commit_id(make_client):
|
||||
# Without a head commit id the merge cannot be pinned to the reviewed
|
||||
# head: fail closed instead of sending an unpinned (or empty) request.
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
pr = gitea_pr(7)
|
||||
pr["head"] = {"ref": "issue-7"} # sha missing
|
||||
return 200, pr
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="no head commit id"):
|
||||
client.merge_pr("acme", 7)
|
||||
assert not requests_for(stub, method="POST", path_suffix="/merge")
|
||||
|
||||
|
||||
def test_merge_pr_refuses_moved_head_before_post(make_client):
|
||||
# TOCTOU regression (governor review): the branch moved from the
|
||||
# reviewed SHA A to SHA B between review and merge. Passing the
|
||||
# reviewed SHA must fail closed BEFORE any merge POST — no unstably
|
||||
# merged, unreviewed SHA B.
|
||||
reviewed_sha = "reviewed-sha-AAAA"
|
||||
moved_sha = "moved-sha-BBBB"
|
||||
state = {"gets": 0}
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
state["gets"] += 1
|
||||
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
|
||||
return 200, {**gitea_pr(7, head_sha=moved_sha), "merged": merged}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="moved after review"):
|
||||
client.merge_pr("acme", 7, expected_head_sha=reviewed_sha)
|
||||
|
||||
# Fail closed: the merge endpoint was never called, so unreviewed
|
||||
# SHA B cannot be merged.
|
||||
assert not requests_for(stub, method="POST", path_suffix="/merge")
|
||||
assert "reviewed-sha-AAAA" not in [
|
||||
json.loads(r["body"]).get("head_commit_id")
|
||||
for r in stub.requests if r["method"] == "POST"]
|
||||
|
||||
|
||||
def test_merge_pr_pins_expected_head_in_payload(make_client):
|
||||
# The reviewed SHA passed by the caller is what gets sent as
|
||||
# head_commit_id (not whatever the pre-merge GET happens to report).
|
||||
reviewed_sha = "reviewed-sha-CCCC"
|
||||
state = {"gets": 0}
|
||||
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
state["gets"] += 1
|
||||
merged = state["gets"] >= 2 # first GET = pre-check, later = confirm
|
||||
return 200, {**gitea_pr(7, head_sha=reviewed_sha), "merged": merged}
|
||||
if method == "POST" and path.endswith("/pulls/7/merge"):
|
||||
return 200, {"merged": True, "message": "merged"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
assert client.merge_pr("acme", 7, expected_head_sha=reviewed_sha) is True
|
||||
|
||||
posts = requests_for(stub, method="POST", path_suffix="/pulls/7/merge")
|
||||
assert len(posts) == 1
|
||||
assert json.loads(posts[0]["body"]) == {
|
||||
"Do": "merge", "head_commit_id": reviewed_sha}
|
||||
|
||||
|
||||
def test_merge_pr_already_merged_short_circuits(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
return 200, {**gitea_pr(7), "merged": True}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
assert client.merge_pr("acme", 7) is True
|
||||
assert not requests_for(stub, method="POST", path_suffix="/merge")
|
||||
|
||||
|
||||
def test_merge_pr_returns_false_when_post_condition_unmet(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
return 200, {**gitea_pr(7), "merged": False}
|
||||
if method == "POST" and path.endswith("/pulls/7/merge"):
|
||||
return 200, {"merged": False, "message": "not merged yet"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
assert client.merge_pr("acme", 7) is False
|
||||
assert len(requests_for(stub, method="POST", path_suffix="/merge")) == 1
|
||||
|
||||
|
||||
def test_merge_pr_conflict_raises(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/pulls/7"):
|
||||
return 200, {**gitea_pr(7), "merged": False}
|
||||
if method == "POST" and path.endswith("/pulls/7/merge"):
|
||||
return 409, {"message": "conflict: merge is blocked"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaApiError) as excinfo:
|
||||
client.merge_pr("acme", 7)
|
||||
assert excinfo.value.status == 409
|
||||
assert "merge is blocked" in str(excinfo.value)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# close_issue
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_close_issue_patches_state_and_confirms(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/issues/9"):
|
||||
return 200, {"number": 9, "state": "closed"}
|
||||
if method == "PATCH" and path.endswith("/issues/9"):
|
||||
return 200, {"number": 9, "state": "closed"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
client.close_issue("acme", 9)
|
||||
|
||||
patches = requests_for(stub, method="PATCH", path_suffix="/issues/9")
|
||||
assert len(patches) == 1
|
||||
assert json.loads(patches[0]["body"]) == {"state": "closed"}
|
||||
|
||||
|
||||
def test_close_issue_verification_failure_raises(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
if method == "GET" and path.endswith("/issues/9"):
|
||||
return 200, {"number": 9, "state": "open"} # never actually closed
|
||||
if method == "PATCH" and path.endswith("/issues/9"):
|
||||
return 200, {"number": 9, "state": "closed"}
|
||||
return 404, {"error": "unexpected route"}
|
||||
|
||||
client, stub = make_client(dispatcher)
|
||||
with pytest.raises(GiteaClientError, match="state is 'open' after close request"):
|
||||
client.close_issue("acme", 9)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Non-2xx handling: useful error, token never leaks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_non_2xx_raises_useful_error_without_leaking_token(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
# Simulate a hostile server that echoes the request back, including
|
||||
# the Authorization header — the client must redact the token.
|
||||
return 500, {"error": "boom", "echo": f"Authorization: token {TEST_TOKEN}"}
|
||||
|
||||
client, _ = make_client(dispatcher)
|
||||
with pytest.raises(GiteaApiError) as excinfo:
|
||||
client.list_labels("acme")
|
||||
|
||||
message = str(excinfo.value)
|
||||
assert excinfo.value.status == 500
|
||||
assert excinfo.value.method == "GET"
|
||||
assert "boom" in message
|
||||
assert TEST_TOKEN not in message
|
||||
assert "Authorization" not in message
|
||||
assert "token" not in message.lower() or TEST_TOKEN not in message
|
||||
|
||||
|
||||
def test_non_2xx_message_carries_path_and_status(make_client):
|
||||
def dispatcher(method, path, query, body):
|
||||
return 422, {"message": "invalid pull request"}
|
||||
|
||||
client, _ = make_client(dispatcher)
|
||||
with pytest.raises(GiteaApiError) as excinfo:
|
||||
client.get_pr("acme", 3)
|
||||
assert "HTTP 422" in str(excinfo.value)
|
||||
assert "pulls/3" in str(excinfo.value)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Contract: the production client exposes every operation ReviewPolicy needs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_production_client_exposes_review_policy_contract():
|
||||
"""Issue #28 acceptance: the client exposes every seam the policy calls."""
|
||||
from gitea_saas.review_policy import ReviewPolicy
|
||||
from gitea_saas.reviewer import Reviewer
|
||||
|
||||
def gitea_ops(cls):
|
||||
# ``self.gitea.<op>(`` in ReviewPolicy / ``self.repo_client.<op>(``
|
||||
# in Reviewer — every attribute access on the seam, derived from the
|
||||
# source so the contract stays honest as the policy evolves.
|
||||
source = inspect.getsource(cls)
|
||||
prefix = "self.gitea." if cls is ReviewPolicy else "self.repo_client."
|
||||
ops = set(re.findall(re.escape(prefix) + r"(\w+)\(", source))
|
||||
# ``merge_pr`` is also invoked through the shared debit helper; keep
|
||||
# the set stable by including explicitly required operations.
|
||||
return ops
|
||||
|
||||
required = gitea_ops(ReviewPolicy) | gitea_ops(Reviewer)
|
||||
# Explicit core of the issue's scope (ReviewPolicy's seam list):
|
||||
required |= {"list_open_prs", "get_pr", "get_issue", "post_comment",
|
||||
"add_label", "remove_label", "merge_pr", "close_issue"}
|
||||
|
||||
missing = sorted(op for op in required if not callable(getattr(GiteaClient, op, None)))
|
||||
assert not missing, f"production client is missing ReviewPolicy operations: {missing}"
|
||||
@ -8,6 +8,7 @@ DB (funded vs unfunded gating is real). No network, no sleeps (a noop
|
||||
Run: python3 -m pytest tests/test_intake_eval.py -q
|
||||
"""
|
||||
import json
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
@ -15,8 +16,14 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
import pytest
|
||||
|
||||
from gitea_saas.auth import UNAUTHORIZED_BODY
|
||||
from gitea_saas.billing_core import BillingCore
|
||||
from gitea_saas.intake_eval import EvalIntake, EvalIntakeError, EVAL_LABEL
|
||||
from gitea_saas.intake_eval import (
|
||||
EvalIntake,
|
||||
EvalIntakeError,
|
||||
EVAL_LABEL,
|
||||
register_eval_routes,
|
||||
)
|
||||
|
||||
|
||||
class Config:
|
||||
@ -420,3 +427,402 @@ def test_real_client_list_issue_comments_builds_correct_request(monkeypatch):
|
||||
"issues/42/comments")
|
||||
assert req.headers.get("Authorization") == "token tok123"
|
||||
assert req.headers.get("Accept") == "application/json"
|
||||
|
||||
|
||||
# -------------------------------------------------------------------------- #
|
||||
# Issue #39: HTTP surface (register_eval_routes) #
|
||||
# -------------------------------------------------------------------------- #
|
||||
|
||||
@pytest.fixture
|
||||
def app_env(tmp_path):
|
||||
"""core + MockGitea + api-keyed Config for mounting eval routes."""
|
||||
core = BillingCore(tmp_path / "billing.db")
|
||||
gitea = MockGitea()
|
||||
cfg = Config(tmp_path / "saas.db", api_key="test-api-key",
|
||||
debit_floor_minor=500)
|
||||
return core, gitea, cfg
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def eval_app(app_env):
|
||||
from flask import Flask
|
||||
|
||||
core, gitea, cfg = app_env
|
||||
app = Flask(__name__)
|
||||
app.config["API_KEY"] = cfg.api_key
|
||||
app.config["TESTING"] = True
|
||||
register_eval_routes(app, core, gitea, cfg)
|
||||
return app.test_client()
|
||||
|
||||
|
||||
def _balance(core, client_id):
|
||||
ok, details = core.get_client(client_id, include_balance=True)
|
||||
assert ok
|
||||
return details["balance"]
|
||||
|
||||
|
||||
def _queued_batch_rows(db_path, client_id):
|
||||
conn = sqlite3.connect(str(db_path))
|
||||
try:
|
||||
conn.row_factory = sqlite3.Row
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM pending_eval_batches WHERE client_id = ?"
|
||||
" ORDER BY id", (client_id,)).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
class TestEvalRoutes:
|
||||
"""Acceptance 1: each route — success, missing/blank key 401,
|
||||
unconfigured key 503; health stays public."""
|
||||
|
||||
AUTHD_PATHS = [
|
||||
("post", "/api/eval/batches", {"client_id": 1, "tasks": TASKS}),
|
||||
("get", "/api/eval/batches/1", None),
|
||||
("get", "/api/eval/batches/1/results", None),
|
||||
("post", "/api/eval/batches/1/cancel", None),
|
||||
]
|
||||
|
||||
KEY = "test-api-key"
|
||||
|
||||
@staticmethod
|
||||
def _post(client, path, payload=None):
|
||||
return client.post(path, json=payload or {},
|
||||
headers={"X-API-Key": TestEvalRoutes.KEY})
|
||||
|
||||
@staticmethod
|
||||
def _get(client, path):
|
||||
return client.get(path,
|
||||
headers={"X-API-Key": TestEvalRoutes.KEY})
|
||||
|
||||
def test_eval_health_is_public(self, eval_app):
|
||||
resp = eval_app.get("/api/eval/health")
|
||||
assert resp.status_code == 200
|
||||
assert resp.get_json() == {"ok": True}
|
||||
|
||||
def test_eval_health_public_without_any_key(self, app_env):
|
||||
core, gitea, cfg = app_env
|
||||
from flask import Flask
|
||||
|
||||
app = Flask(__name__)
|
||||
app.config["TESTING"] = True
|
||||
register_eval_routes(app, core, gitea, cfg) # API_KEY unset
|
||||
resp = app.test_client().get("/api/eval/health")
|
||||
assert resp.status_code == 200
|
||||
|
||||
def test_missing_key_401_on_every_route(self, eval_app):
|
||||
for method, path, _payload in self.AUTHD_PATHS:
|
||||
resp = getattr(eval_app, method)(path, json={})
|
||||
assert resp.status_code == 401, (method, path)
|
||||
# canonical #24 body, byte-identical with every front-end
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
def test_blank_key_401_on_every_route(self, eval_app):
|
||||
for method, path, _payload in self.AUTHD_PATHS:
|
||||
resp = getattr(eval_app, method)(
|
||||
path, json={}, headers={"X-API-Key": ""})
|
||||
assert resp.status_code == 401, (method, path)
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
def test_wrong_key_401(self, eval_app):
|
||||
resp = eval_app.post(
|
||||
"/api/eval/batches", json={"client_id": 1, "tasks": TASKS},
|
||||
headers={"X-API-Key": "wrong-key"})
|
||||
assert resp.status_code == 401
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
def test_unconfigured_key_503_on_every_route(self, app_env):
|
||||
"""No API key configured -> authed routes fail closed 503
|
||||
(issue #38's assembly acceptance demands this app-wide); the
|
||||
liveness route stays reachable."""
|
||||
core, gitea, cfg = app_env
|
||||
cfg_no_key = Config(cfg.db_path) # no api_key attr at all
|
||||
from flask import Flask
|
||||
|
||||
app = Flask(__name__)
|
||||
app.config["TESTING"] = True
|
||||
register_eval_routes(app, core, gitea, cfg_no_key)
|
||||
client = app.test_client()
|
||||
for method, path, payload in self.AUTHD_PATHS:
|
||||
resp = getattr(client, method)(path, json=payload or {})
|
||||
assert resp.status_code == 503, (method, path)
|
||||
assert resp.get_json()["error"]["code"] == "SERVICE_UNAVAILABLE"
|
||||
assert client.get("/api/eval/health").status_code == 200
|
||||
|
||||
def test_submit_funded_route_201(self, app_env, eval_app):
|
||||
core, gitea, _cfg = app_env
|
||||
ok, c = core.create_client("route-client")
|
||||
assert ok
|
||||
core.credit(c["id"], 10_000, "fund-route")
|
||||
resp = self._post(eval_app, "/api/eval/batches", {
|
||||
"client_id": c["id"], "tasks": TASKS})
|
||||
assert resp.status_code == 201
|
||||
body = resp.get_json()
|
||||
assert body["ok"] is True and body["batch_id"] == 1
|
||||
assert len(body["issues"]) == 2
|
||||
# the batch hold is visible in the client's available balance
|
||||
assert _balance(core, c["id"]) == 10_000 - 500
|
||||
|
||||
def test_submit_unfunded_route_202_queued(self, app_env, eval_app):
|
||||
core, gitea, _cfg = app_env
|
||||
ok, c = core.create_client("poor-route")
|
||||
assert ok
|
||||
resp = self._post(eval_app, "/api/eval/batches", {
|
||||
"client_id": c["id"], "tasks": TASKS})
|
||||
assert resp.status_code == 202
|
||||
body = resp.get_json()
|
||||
assert body["ok"] is False and body["reason"] == "unfunded"
|
||||
assert body["queued"] is True and body["pending_id"] == 1
|
||||
assert gitea.created_issues == [] # NO Gitea issues
|
||||
assert gitea.created_repos == []
|
||||
rows = _queued_batch_rows(_cfg.db_path, c["id"])
|
||||
assert len(rows) == 1 and rows[0]["status"] == "queued"
|
||||
|
||||
def test_route_validation_400_and_404(self, eval_app, app_env):
|
||||
core, gitea, _cfg = app_env
|
||||
ok, c = core.create_client("val-client")
|
||||
assert ok
|
||||
core.credit(c["id"], 10_000, "fund-val")
|
||||
bad = self._post(eval_app, "/api/eval/batches", {
|
||||
"client_id": -3, "tasks": TASKS})
|
||||
assert bad.status_code == 400 and bad.get_json()["code"] == "invalid_request"
|
||||
bad_tasks = self._post(eval_app, "/api/eval/batches", {
|
||||
"client_id": c["id"], "tasks": [{"task_id": "x"}]}) # no title
|
||||
assert bad_tasks.status_code == 400
|
||||
assert self._get(eval_app, "/api/eval/batches/99").status_code == 404
|
||||
assert self._get(eval_app, "/api/eval/batches/99/results").status_code == 404
|
||||
assert self._post(eval_app, "/api/eval/batches/99/cancel").status_code == 404
|
||||
|
||||
def test_get_batch_and_results_routes(self, app_env, eval_app):
|
||||
core, gitea, _cfg = app_env
|
||||
ok, c = core.create_client("view-client")
|
||||
assert ok
|
||||
core.credit(c["id"], 10_000, "fund-view")
|
||||
r = self._post(eval_app, "/api/eval/batches", {
|
||||
"client_id": c["id"], "tasks": TASKS}).get_json()
|
||||
batch_id = r["batch_id"]
|
||||
# status + per-task state
|
||||
resp = self._get(eval_app, f"/api/eval/batches/{batch_id}")
|
||||
assert resp.status_code == 200
|
||||
body = resp.get_json()
|
||||
assert body["ok"] and body["batch"]["status"] == "open"
|
||||
assert [t["task_id"] for t in body["tasks"]] == ["t1", "t2"]
|
||||
assert all(t["status"] == "pending" for t in body["tasks"])
|
||||
# the results table reflects the drain state: close t1 in gitea,
|
||||
# then run the drain step (poll) through a direct intake on the
|
||||
# same DB — the results route renders the persisted table
|
||||
gitea.add_comment("eval-runs", r["issues"][0]["issue_number"],
|
||||
"lane report: green")
|
||||
gitea.close_issue("eval-runs", r["issues"][0]["issue_number"])
|
||||
intake = EvalIntake(core, gitea, _cfg, sleep_fn=noop_sleep)
|
||||
assert intake.poll_completed(batch_id)["done"] == 1
|
||||
results = self._get(
|
||||
eval_app, f"/api/eval/batches/{batch_id}/results").get_json()
|
||||
assert results["ok"] is True
|
||||
by_task = {t["task_id"]: t for t in results["results"]}
|
||||
assert by_task["t1"]["status"] == "done"
|
||||
assert by_task["t1"]["summary"] == "lane report: green"
|
||||
# t2 was never drained, so the results table reports it pending.
|
||||
assert by_task["t2"]["status"] == "pending"
|
||||
|
||||
def test_cancel_route_releases_hold_exactly(self, app_env, eval_app):
|
||||
"""Acceptance 3 (route half): balance returns to its pre-submit
|
||||
value exactly after cancel."""
|
||||
core, gitea, _cfg = app_env
|
||||
ok, c = core.create_client("cancel-client")
|
||||
assert ok
|
||||
core.credit(c["id"], 10_000, "fund-cancel")
|
||||
pre = _balance(core, c["id"])
|
||||
r = self._post(eval_app, "/api/eval/batches", {
|
||||
"client_id": c["id"], "tasks": TASKS}).get_json()
|
||||
assert _balance(core, c["id"]) == pre - 500 # hold placed
|
||||
resp = self._post(eval_app, f"/api/eval/batches/{r['batch_id']}/cancel")
|
||||
assert resp.status_code == 200
|
||||
body = resp.get_json()
|
||||
assert body["ok"] is True and body["cancelled"] is True
|
||||
assert body["hold_released"] is True
|
||||
assert _balance(core, c["id"]) == pre # exact return
|
||||
# the batch now reads as cancelled
|
||||
status = self._get(
|
||||
eval_app, f"/api/eval/batches/{r['batch_id']}").get_json()
|
||||
assert status["cancelled"] is True
|
||||
assert status["batch"]["status"] == "cancelled"
|
||||
|
||||
|
||||
# -------------------------------------------------------------------------- #
|
||||
# Issue #39: queue + flush (acceptance 2) and hold/cancel (acceptance 3) #
|
||||
# -------------------------------------------------------------------------- #
|
||||
|
||||
class TestQueueAndFlush:
|
||||
def test_unfunded_submit_queues_no_gitea(self, unfunded, tmp_path):
|
||||
core, client_id = unfunded
|
||||
gitea = MockGitea()
|
||||
intake = make_intake(core, tmp_path, gitea)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert r["ok"] is False and r["reason"] == "unfunded"
|
||||
assert r["queued"] is True and r["pending_id"] == 1
|
||||
# half 1: NO Gitea issue, NO repo
|
||||
assert gitea.created_issues == []
|
||||
assert gitea.created_repos == []
|
||||
# half 1: a queued row exists locally
|
||||
rows = _queued_batch_rows(tmp_path / "saas.db", client_id)
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["status"] == "queued"
|
||||
assert json.loads(rows[0]["tasks_json"]) == TASKS
|
||||
|
||||
def test_flush_ships_queued_batch_after_funding(self, unfunded, tmp_path):
|
||||
core, client_id = unfunded
|
||||
gitea = MockGitea()
|
||||
intake = make_intake(core, tmp_path, gitea)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert r["pending_id"] == 1
|
||||
|
||||
# funding arrives; flush ships it
|
||||
core.credit(client_id, 10_000, "fund-flush")
|
||||
out = intake.flush_pending(client_id)
|
||||
assert out["ok"] is True and out["flushed"] == 1
|
||||
assert out["batch_ids"] == [1]
|
||||
assert len(out["issues"]) == 2
|
||||
# half 2: issues now exist, one per task, eval-task label
|
||||
assert len(gitea.created_issues) == 2
|
||||
for _repo, _title, body, labels in gitea.created_issues:
|
||||
assert labels == [EVAL_LABEL]
|
||||
assert "batch: 1" in body
|
||||
# the queued row is marked shipped; a real batch row exists
|
||||
rows = _queued_batch_rows(tmp_path / "saas.db", client_id)
|
||||
assert len(rows) == 1 and rows[0]["status"] == "shipped"
|
||||
batch = intake.get_batch(1)
|
||||
assert batch["ok"] and batch["batch"]["client_id"] == client_id
|
||||
assert len(batch["tasks"]) == 2
|
||||
# the shipped batch placed its hold like any fresh submit
|
||||
assert _balance(core, client_id) == 10_000 - 500
|
||||
|
||||
def test_flush_while_still_unfunded_skips(self, unfunded, tmp_path):
|
||||
core, client_id = unfunded
|
||||
gitea = MockGitea()
|
||||
intake = make_intake(core, tmp_path, gitea)
|
||||
intake.submit_batch(client_id, TASKS)
|
||||
out = intake.flush_pending(client_id)
|
||||
assert out["ok"] is False and out["reason"] == "unfunded"
|
||||
assert out["skipped"] == 1 and out["flushed"] == 0
|
||||
assert gitea.created_issues == []
|
||||
rows = _queued_batch_rows(tmp_path / "saas.db", client_id)
|
||||
assert rows[0]["status"] == "queued"
|
||||
|
||||
def test_flush_nothing_queued_is_noop(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
intake = make_intake(core, tmp_path)
|
||||
out = intake.flush_pending(client_id)
|
||||
assert out == {"ok": True, "flushed": 0, "batch_ids": [], "issues": []}
|
||||
|
||||
|
||||
class TestHoldAndCancel:
|
||||
def test_cancel_releases_hold_balance_exact(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
intake = make_intake(core, tmp_path)
|
||||
pre = _balance(core, client_id)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert r["hold_id"] is not None
|
||||
assert _balance(core, client_id) == pre - 500 # hold visible
|
||||
out = intake.cancel_batch(r["batch_id"])
|
||||
assert out["ok"] is True and out["cancelled"] is True
|
||||
assert out["hold_released"] is True
|
||||
assert _balance(core, client_id) == pre # EXACT return
|
||||
|
||||
def test_cancel_idempotent_no_double_release(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
intake = make_intake(core, tmp_path)
|
||||
pre = _balance(core, client_id)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert intake.cancel_batch(r["batch_id"])["ok"] is True
|
||||
assert _balance(core, client_id) == pre
|
||||
again = intake.cancel_batch(r["batch_id"])
|
||||
assert again["ok"] is True and again["already_cancelled"] is True
|
||||
assert again["hold_released"] is False
|
||||
assert _balance(core, client_id) == pre # no second release
|
||||
|
||||
def test_cancel_unknown_batch(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
intake = make_intake(core, tmp_path)
|
||||
out = intake.cancel_batch(999)
|
||||
assert out == {"ok": False, "error": "unknown_batch"}
|
||||
|
||||
def test_cancel_stops_draining(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
gitea = MockGitea()
|
||||
intake = make_intake(core, tmp_path, gitea, emitter=MockEmitter())
|
||||
r = intake.submit_batch(client_id, TASKS,
|
||||
callback_url="https://cb.example/x")
|
||||
for issue in r["issues"]:
|
||||
gitea.close_issue("eval-runs", issue["issue_number"])
|
||||
assert intake.cancel_batch(r["batch_id"])["ok"] is True
|
||||
# polling and emitting both refuse a cancelled batch
|
||||
assert intake.poll_completed(r["batch_id"]) == {
|
||||
"ok": False, "error": "cancelled"}
|
||||
assert intake.emit(r["batch_id"]) == {
|
||||
"ok": False, "reason": "cancelled"}
|
||||
# and the batch reads as cancelled
|
||||
status = intake.get_batch(r["batch_id"])
|
||||
assert status["cancelled"] is True
|
||||
assert status["batch"]["status"] == "cancelled"
|
||||
assert "cancelled_at" in status["batch"]
|
||||
|
||||
def test_zero_hold_disables_hold(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
intake = make_intake(core, tmp_path, eval_batch_hold_minor=0)
|
||||
pre = _balance(core, client_id)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert r.get("hold_id") is None
|
||||
assert _balance(core, client_id) == pre # nothing held
|
||||
out = intake.cancel_batch(r["batch_id"])
|
||||
assert out["ok"] is True and out["hold_released"] is False
|
||||
assert _balance(core, client_id) == pre
|
||||
|
||||
def test_intake_failure_releases_hold(self, funded, tmp_path):
|
||||
"""A rolled-back submit must not leave money locked."""
|
||||
core, client_id = funded
|
||||
|
||||
class FlakyGitea(MockGitea):
|
||||
def create_issue(self, *a, **kw):
|
||||
if self.next_number >= 1: # second issue fails
|
||||
raise RuntimeError("gitea hiccup")
|
||||
return super().create_issue(*a, **kw)
|
||||
|
||||
intake = make_intake(core, tmp_path, FlakyGitea())
|
||||
pre = _balance(core, client_id)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert r["ok"] is False and r["reason"] == "intake_failed"
|
||||
assert intake.get_batch(1)["ok"] is False # no partial batch
|
||||
assert _balance(core, client_id) == pre # hold released
|
||||
|
||||
def test_hold_too_large_refuses_batch(self, funded, tmp_path):
|
||||
"""Reserving beyond the available balance fails closed — the
|
||||
batch is refused and no Gitea issue is created."""
|
||||
core, client_id = funded
|
||||
gitea = MockGitea()
|
||||
intake = make_intake(core, tmp_path, gitea,
|
||||
eval_batch_hold_minor=50_000) # > 10_000
|
||||
pre = _balance(core, client_id)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
assert r["ok"] is False and r["reason"] == "intake_failed"
|
||||
assert gitea.created_issues == []
|
||||
assert _balance(core, client_id) == pre
|
||||
|
||||
def test_get_results_after_partial_drain(self, funded, tmp_path):
|
||||
core, client_id = funded
|
||||
gitea = MockGitea()
|
||||
intake = make_intake(core, tmp_path, gitea)
|
||||
r = intake.submit_batch(client_id, TASKS)
|
||||
gitea.add_comment("eval-runs", r["issues"][0]["issue_number"],
|
||||
"ok")
|
||||
gitea.close_issue("eval-runs", r["issues"][0]["issue_number"])
|
||||
intake.poll_completed(r["batch_id"])
|
||||
results = intake.get_results(r["batch_id"])
|
||||
assert results["ok"] is True
|
||||
assert results["batch_id"] == r["batch_id"]
|
||||
by_task = {t["task_id"]: t for t in results["results"]}
|
||||
assert by_task["t1"]["status"] == "done"
|
||||
assert by_task["t1"]["summary"] == "ok"
|
||||
assert by_task["t2"]["status"] == "pending"
|
||||
assert intake.get_results(999)["error"] == "unknown_batch"
|
||||
|
||||
@ -26,6 +26,7 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from gitea_saas.auth import UNAUTHORIZED_BODY
|
||||
from gitea_saas.intake_factory import (
|
||||
DEFAULT_DEBIT_FLOOR_MINOR,
|
||||
IntakeFactory,
|
||||
@ -438,8 +439,26 @@ def _post(app, path, payload=None, key="test-api-key"):
|
||||
def test_routes_require_api_key(app_client):
|
||||
resp = app_client.post("/api/factory/provision", json={"client_id": 1})
|
||||
assert resp.status_code == 401
|
||||
# Canonical 401 body (issue #24), byte-identical with every front-end.
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
resp = app_client.post("/api/factory/1/unlock")
|
||||
assert resp.status_code == 401
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
def test_wrong_key_of_same_length_rejected(app_client):
|
||||
"""Wrong key of the SAME length as the configured key must be rejected
|
||||
(issue #24 criterion 3; the constant-time comparison is what makes this
|
||||
a real check rather than a prefix-time leak)."""
|
||||
configured = "test-api-key"
|
||||
wrong_same_length = "hunter-id-123"[:len(configured)]
|
||||
assert len(wrong_same_length) == len(configured)
|
||||
assert wrong_same_length != configured
|
||||
resp = app_client.post("/api/factory/provision",
|
||||
json={"client_id": 1},
|
||||
headers={"X-API-Key": wrong_same_length})
|
||||
assert resp.status_code == 401
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
def test_route_provision_submit_unlock_lock_flow(env, app_client):
|
||||
|
||||
@ -27,10 +27,19 @@ from gitea_saas.reviewer import Verdict # noqa: E402
|
||||
|
||||
|
||||
class FakeGitea:
|
||||
"""In-memory Gitea seam recording every mutation."""
|
||||
"""In-memory Gitea seam recording every mutation.
|
||||
|
||||
Mirrors the production client's NORMALIZED shape (issue #28/#31):
|
||||
PRs carry ``merged`` (bool) and ``state`` exactly like
|
||||
``GiteaClient._normalize_pr`` output, ``list_open_prs`` returns only
|
||||
open-unmerged PRs, ``get_pr`` returns the live record, and a
|
||||
successful ``merge_pr`` flips the live record to ``merged=True`` /
|
||||
``state="closed"`` — so tests exercise the production-shaped dict, not
|
||||
a richer test-only one.
|
||||
"""
|
||||
|
||||
def __init__(self, prs=None, issues=None):
|
||||
self.prs = list(prs or []) # [{number, head_sha, labels, body}]
|
||||
self.prs = list(prs or []) # [{number, head_sha, labels, body, merged, state}]
|
||||
self.issues = dict(issues or {}) # number -> {"body": ...}
|
||||
self.comments = []
|
||||
self.labels_added = []
|
||||
@ -38,12 +47,21 @@ class FakeGitea:
|
||||
self.merged = []
|
||||
self.closed = []
|
||||
self.merge_results = {} # pr -> bool (default True)
|
||||
self.fail_next = {} # method -> one-shot Exception to raise
|
||||
|
||||
def _maybe_fail(self, name):
|
||||
"""Raise a one-shot injected failure (issue #29 retry tests)."""
|
||||
exc = self.fail_next.pop(name, None)
|
||||
if exc is not None:
|
||||
raise exc
|
||||
|
||||
def _pr(self, number):
|
||||
return next(p for p in self.prs if p["number"] == number)
|
||||
|
||||
def list_open_prs(self, repo=None):
|
||||
return [dict(p) for p in self.prs]
|
||||
# Production behavior: ?state=open returns open, unmerged PRs only.
|
||||
return [dict(p) for p in self.prs
|
||||
if p.get("state") == "open" and not p.get("merged")]
|
||||
|
||||
def get_pr(self, repo, number):
|
||||
return dict(self._pr(number))
|
||||
@ -52,19 +70,34 @@ class FakeGitea:
|
||||
return dict(self.issues[number])
|
||||
|
||||
def post_comment(self, repo, number, body):
|
||||
self._maybe_fail("post_comment")
|
||||
self.comments.append((repo, number, body))
|
||||
|
||||
def add_label(self, repo, number, label):
|
||||
self._maybe_fail("add_label")
|
||||
self.labels_added.append((repo, number, label))
|
||||
|
||||
def remove_label(self, repo, number, label):
|
||||
self._maybe_fail("remove_label")
|
||||
self.labels_removed.append((repo, number, label))
|
||||
|
||||
def merge_pr(self, repo, number):
|
||||
def merge_pr(self, repo, number, expected_head_sha=None):
|
||||
self._maybe_fail("merge_pr")
|
||||
live = self._pr(number)
|
||||
if expected_head_sha is not None and live.get("head_sha") != expected_head_sha:
|
||||
raise RuntimeError(
|
||||
f"merge_pr: PR {number} head is {live.get('head_sha')!r} but the "
|
||||
f"reviewed head is {expected_head_sha!r}; refusing to merge "
|
||||
"a branch that moved after review")
|
||||
self.merged.append((repo, number))
|
||||
return self.merge_results.get(number, True)
|
||||
merged = self.merge_results.get(number, True)
|
||||
if merged:
|
||||
live["merged"] = True
|
||||
live["state"] = "closed"
|
||||
return merged
|
||||
|
||||
def close_issue(self, repo, number):
|
||||
self._maybe_fail("close_issue")
|
||||
self.closed.append((repo, number))
|
||||
|
||||
|
||||
@ -75,9 +108,10 @@ class FakeReviewer:
|
||||
self.verdicts = dict(verdicts)
|
||||
self.calls = []
|
||||
self.posted = []
|
||||
self.runner = None # mirror Reviewer.runner (used to build the workspace)
|
||||
|
||||
def review_pr(self, repo, pr, linked_issue, repo_dir=None):
|
||||
self.calls.append((repo, pr, linked_issue, repo_dir))
|
||||
def review_pr(self, repo, pr, linked_issue, workspace=None):
|
||||
self.calls.append((repo, pr, linked_issue, workspace))
|
||||
verdict = self.verdicts[pr]
|
||||
if callable(verdict):
|
||||
return verdict(repo, pr)
|
||||
@ -88,15 +122,21 @@ class FakeReviewer:
|
||||
return ""
|
||||
|
||||
|
||||
def make_verdict(pr_number, verdict, head_sha="sha-abc"):
|
||||
def make_verdict(pr_number, verdict, head_sha="sha-abc", base_sha="sha-base"):
|
||||
return Verdict(repo="gitea-saas-core", pr=pr_number, head_sha=head_sha,
|
||||
base_sha=base_sha,
|
||||
acceptance=["tests pass"], tests={"exit": 0, "summary": "12 passed"},
|
||||
build=None, scope_violations=[], findings=[],
|
||||
verdict=verdict, summary="ok")
|
||||
|
||||
|
||||
def pr(number, head_sha="sha-abc", labels=("in-review",), body="Closes #1\nSome PR"):
|
||||
return {"number": number, "head_sha": head_sha, "labels": list(labels), "body": body}
|
||||
def pr(number, head_sha="sha-abc", base_sha="sha-base", labels=("in-review",),
|
||||
body="Closes #1\nSome PR"):
|
||||
# Production-normalized shape (as GiteaClient._normalize_pr returns):
|
||||
# merged bool + state + head_sha + base_sha, defaults open / not merged.
|
||||
return {"number": number, "head_sha": head_sha, "base_sha": base_sha,
|
||||
"labels": list(labels), "body": body, "merged": False,
|
||||
"state": "open"}
|
||||
|
||||
|
||||
def tier_issue(tier, number=1):
|
||||
@ -143,7 +183,7 @@ def policy_factory(tmp_path):
|
||||
def test_review_once_per_head_and_skip_same_head(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(10, head_sha="aaa")], gitea_issues=tier_issue("gold"),
|
||||
verdicts={10: make_verdict(10, APPROVE)})
|
||||
verdicts={10: make_verdict(10, APPROVE, head_sha="aaa")})
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
@ -154,8 +194,12 @@ def test_review_once_per_head_and_skip_same_head(policy_factory):
|
||||
|
||||
|
||||
def test_head_move_triggers_rereview(policy_factory):
|
||||
# Standard tier keeps the PR OPEN after the first poll (comment only),
|
||||
# so a head move is observable by the next scan and must re-review.
|
||||
# (A gold-tier poll merges, and a merged PR leaves the open scan — the
|
||||
# production-shaped fake reflects that.)
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(10, head_sha="aaa")], gitea_issues=tier_issue("gold"),
|
||||
gitea_prs=[pr(10, head_sha="aaa")], gitea_issues=tier_issue("standard"),
|
||||
verdicts={10: make_verdict(10, APPROVE)})
|
||||
|
||||
policy.poll_and_review("gitea-saas-core")
|
||||
@ -166,6 +210,73 @@ def test_head_move_triggers_rereview(policy_factory):
|
||||
assert [c[1] for c in reviewer.calls] == [10, 10]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Evaluated-pair integrity (issue #30): refuse to apply a verdict whose
|
||||
# head/base pair moved after evaluation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_evaluated_head_move_refuses_verdict_and_new_pair_gets_fresh_shot(policy_factory):
|
||||
# The reviewer evaluated head "sha-old"; the live PR has already moved
|
||||
# to "sha-new" when the verdict is to be applied. A verdict for a pair
|
||||
# that no longer exists must never merge/label/comment/close — and once
|
||||
# the PR settles on the evaluated head, the pair is fresh-eligible and
|
||||
# the same verdict flow applies normally.
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(50, head_sha="sha-new")], gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={50: make_verdict(50, APPROVE, head_sha="sha-old")})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
r = results[0]
|
||||
assert r["outcome"] == "stale"
|
||||
assert r["error"]["code"] == "PAIR_MOVED"
|
||||
assert r["evaluated"] == {"head_sha": "sha-old", "base_sha": "sha-base"}
|
||||
assert r["live"] == {"head_sha": "sha-new", "base_sha": "sha-base"}
|
||||
assert "not applied" in r["error"]["message"].lower()
|
||||
# zero side effects for the stale pair
|
||||
assert gitea.merged == [] and gitea.comments == []
|
||||
assert gitea.labels_added == [] and gitea.labels_removed == []
|
||||
assert gitea.closed == []
|
||||
assert all(c[3] is None for c in reviewer.calls) # no clone configured -> no workspace
|
||||
|
||||
# The branch settles on the evaluated head: the pair now exists, the
|
||||
# verdict applies (gold tier -> merge), nothing was recorded as complete
|
||||
# for the stale attempt.
|
||||
gitea.prs[0]["head_sha"] = "sha-old"
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
assert results[0]["action"] == "merged"
|
||||
assert gitea.merged == [("gitea-saas-core", 50)]
|
||||
|
||||
|
||||
def test_base_move_after_evaluation_refuses_verdict(policy_factory):
|
||||
# Same head, but the BASE moved after the review evaluated the pair:
|
||||
# the synthetic merge (and therefore the verdict) describes a different
|
||||
# base; refusing is mandatory regardless of head being unchanged.
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(51, head_sha="sha-h", base_sha="sha-base-current")],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={51: make_verdict(51, APPROVE, head_sha="sha-h",
|
||||
base_sha="sha-base-evaluated")})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
r = results[0]
|
||||
assert r["outcome"] == "stale" and r["error"]["code"] == "PAIR_MOVED"
|
||||
assert r["live"] == {"head_sha": "sha-h", "base_sha": "sha-base-current"}
|
||||
assert gitea.merged == [] and gitea.comments == [] and gitea.closed == []
|
||||
|
||||
|
||||
def test_workspace_passed_through_when_clone_configured(policy_factory):
|
||||
# With REVIEW_CLONE_DIR configured, the policy builds the isolated
|
||||
# workspace seam and hands it to the reviewer on every call.
|
||||
ws_sentinel = object()
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(52)], gitea_issues=tier_issue("standard"),
|
||||
verdicts={52: make_verdict(52, APPROVE)},
|
||||
review_clone_dir="/nonexistent/clone")
|
||||
policy._workspace = ws_sentinel # the fixture's clone dir does not exist
|
||||
policy.poll_and_review("gitea-saas-core")
|
||||
assert reviewer.calls[0][3] is ws_sentinel
|
||||
|
||||
|
||||
def test_unlabelled_prs_skipped_unless_review_all(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(10, labels=("in-review",)), pr(11, labels=())],
|
||||
@ -204,14 +315,14 @@ def test_gold_approve_merges_closes_strips_debits_once(policy_factory):
|
||||
# One debit with the idempotent key repo#pr@head, at the gold rate.
|
||||
# (The fixture pre-funds the client, so filter debit rows only.)
|
||||
ok, balances = core.get_balance(c["id"])
|
||||
assert ok and balances["balance"] == 10_000 - 100 # gold rate_minor from config
|
||||
assert ok and balances["balance"] == 10_000 - 500 # gold rate_minor from config
|
||||
ok, ledger = core.list_ledger(c["id"])
|
||||
assert ok
|
||||
debits = [e for e in ledger["entries"] if e["kind"] == "debit"]
|
||||
assert len(debits) == 1
|
||||
entry = debits[0]
|
||||
assert entry["idempotency_key"] == "gitea-saas-core#42@sha-gold"
|
||||
assert entry["amount_minor"] == 100 and entry["kind"] == "debit"
|
||||
assert entry["amount_minor"] == 500 and entry["kind"] == "debit"
|
||||
|
||||
|
||||
def test_gold_double_delivery_never_double_debits(policy_factory):
|
||||
@ -235,7 +346,7 @@ def test_gold_bounce_adds_label_no_merge_no_debit(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(43, head_sha="sha-b", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={43: make_verdict(43, BOUNCE)})
|
||||
verdicts={43: make_verdict(43, BOUNCE, head_sha="sha-b")})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
|
||||
@ -252,7 +363,7 @@ def test_merge_failure_leaves_pr_open_and_no_close(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(44, head_sha="sha-c", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={44: make_verdict(44, APPROVE)})
|
||||
verdicts={44: make_verdict(44, APPROVE, head_sha="sha-c")})
|
||||
gitea.merge_results[44] = False
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
@ -267,7 +378,7 @@ def test_debit_failure_is_fail_closed(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(45, head_sha="sha-d", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={45: make_verdict(45, APPROVE)},
|
||||
verdicts={45: make_verdict(45, APPROVE, head_sha="sha-d")},
|
||||
client_ids={})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
@ -285,7 +396,7 @@ def test_standard_approve_comment_only_no_merge(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(50, head_sha="sha-e", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("standard", number=1),
|
||||
verdicts={50: make_verdict(50, APPROVE)})
|
||||
verdicts={50: make_verdict(50, APPROVE, head_sha="sha-e")})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
|
||||
@ -301,7 +412,7 @@ def test_needs_guidance_adds_label_and_comment(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(51, head_sha="sha-f", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("standard", number=1),
|
||||
verdicts={51: make_verdict(51, NEEDS_GUIDANCE)})
|
||||
verdicts={51: make_verdict(51, NEEDS_GUIDANCE, head_sha="sha-f")})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
|
||||
@ -319,7 +430,7 @@ def test_debit_rate_comes_from_tier_config(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(60, head_sha="sha-g", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={60: make_verdict(60, APPROVE)},
|
||||
verdicts={60: make_verdict(60, APPROVE, head_sha="sha-g")},
|
||||
tiers={**DEFAULT_TIERS, "gold": {"auto_merge": True, "rate_minor": 250}})
|
||||
|
||||
policy.poll_and_review("gitea-saas-core")
|
||||
@ -334,7 +445,7 @@ def test_free_tier_merges_without_debit_when_auto_merge(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(61, head_sha="sha-h", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("free", number=1),
|
||||
verdicts={61: make_verdict(61, APPROVE)},
|
||||
verdicts={61: make_verdict(61, APPROVE, head_sha="sha-h")},
|
||||
tiers={**DEFAULT_TIERS, "free": {"auto_merge": True, "rate_minor": 0}})
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
@ -360,6 +471,15 @@ def test_load_review_config_merges_saas_tiers_json(monkeypatch):
|
||||
assert config["test_timeout_s"] == 45
|
||||
|
||||
|
||||
def test_default_tier_rates_are_500_500_0():
|
||||
# Pricing decision 2026-09-01: delivered-issue rate is $5.00 (500 minor
|
||||
# units) for both gold and standard; free stays 0. Assert the literal
|
||||
# values, not the constant name (issue #45 acceptance).
|
||||
assert DEFAULT_TIERS["gold"]["rate_minor"] == 500
|
||||
assert DEFAULT_TIERS["standard"]["rate_minor"] == 500
|
||||
assert DEFAULT_TIERS["free"]["rate_minor"] == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Linked issue parsing + webhook path
|
||||
# ---------------------------------------------------------------------------
|
||||
@ -386,3 +506,538 @@ def test_handle_merge_event_records_receipt_and_closes(policy_factory):
|
||||
debits = [e for e in ledger["entries"] if e["kind"] == "debit"]
|
||||
assert ok and len(debits) == 1
|
||||
assert debits[0]["idempotency_key"] == "gitea-saas-core#70@sha-i"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Transactional outcome application + retry (issue #29)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _ledger_debits(core, client_id):
|
||||
ok, ledger = core.list_ledger(client_id)
|
||||
assert ok
|
||||
return [e for e in ledger["entries"] if e["kind"] == "debit"]
|
||||
|
||||
|
||||
def test_verdict_failure_reported_and_retried(policy_factory):
|
||||
# Verdict generation raises once: structured failure + persisted retry
|
||||
# metadata; the same unchanged head completes exactly once next poll.
|
||||
calls = {"n": 0}
|
||||
|
||||
def flaky(repo, pr):
|
||||
calls["n"] += 1
|
||||
if calls["n"] == 1:
|
||||
raise RuntimeError("verdict generation boom")
|
||||
return make_verdict(pr, APPROVE, head_sha="sha-v")
|
||||
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(80, head_sha="sha-v", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={80: flaky})
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert len(first) == 1 and first[0]["pr"] == 80
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"] == {"type": "RuntimeError",
|
||||
"message": "verdict generation boom"}
|
||||
assert first[0]["attempts"] == 1
|
||||
assert gitea.merged == [] and gitea.closed == [] and gitea.comments == []
|
||||
|
||||
# Persisted: a failed attempt exists with its error, and the head is
|
||||
# NOT completed — the same unchanged head stays eligible for retry.
|
||||
rows = policy._reviews_conn.execute(
|
||||
"SELECT attempts, last_error FROM review_state"
|
||||
" WHERE repo = 'gitea-saas-core' AND pr = 80 AND head_sha = 'sha-v'").fetchall()
|
||||
assert len(rows) == 1 and rows[0][0] == 1
|
||||
assert "verdict generation boom" in rows[0][1]
|
||||
assert policy._already_reviewed("gitea-saas-core", 80, "sha-v") is False
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
assert gitea.merged == [("gitea-saas-core", 80)]
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
# Completed now: an exact third poll skips the head.
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
assert calls["n"] == 2
|
||||
|
||||
|
||||
def test_merge_boundary_failure_retried_merged_once(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(81, head_sha="sha-m", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={81: make_verdict(81, APPROVE, head_sha="sha-m")})
|
||||
gitea.fail_next["merge_pr"] = RuntimeError("merge boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"]["message"] == "merge boom"
|
||||
assert gitea.merged == [] and gitea.closed == []
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
assert gitea.merged == [("gitea-saas-core", 81)] # merged exactly once
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
|
||||
|
||||
def test_close_boundary_failure_no_double_merge_or_close(policy_factory):
|
||||
# Failure AFTER the merge side effect: the retry must skip the already
|
||||
# applied merge (checkpoint) and close the issue exactly once.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(82, head_sha="sha-c", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={82: make_verdict(82, APPROVE, head_sha="sha-c")})
|
||||
gitea.fail_next["close_issue"] = RuntimeError("close boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"]["message"] == "close boom"
|
||||
assert gitea.merged == [("gitea-saas-core", 82)] # merge already applied
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
assert gitea.merged == [("gitea-saas-core", 82)] # NO second merge
|
||||
assert gitea.closed == [("gitea-saas-core", 1)] # exactly one close
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
|
||||
|
||||
def test_debit_boundary_failure_never_double_debits(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(83, head_sha="sha-d", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={83: make_verdict(83, APPROVE, head_sha="sha-d")})
|
||||
real_debit = core.debit
|
||||
calls = {"n": 0}
|
||||
|
||||
def flaky_debit(*args, **kwargs):
|
||||
calls["n"] += 1
|
||||
if calls["n"] == 1:
|
||||
raise RuntimeError("debit boom")
|
||||
return real_debit(*args, **kwargs)
|
||||
|
||||
core.debit = flaky_debit
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"]["message"] == "debit boom"
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
assert gitea.merged == [("gitea-saas-core", 83)] # NO second merge
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert len(_ledger_debits(core, c["id"])) == 1 # exactly one debit
|
||||
assert calls["n"] == 2
|
||||
|
||||
|
||||
def test_label_boundary_failure_bounce_applied_once(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(84, head_sha="sha-l", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={84: make_verdict(84, BOUNCE, head_sha="sha-l")})
|
||||
gitea.fail_next["add_label"] = RuntimeError("label boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"]["message"] == "label boom"
|
||||
assert gitea.labels_added == []
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "bounced"
|
||||
assert gitea.labels_added == [("gitea-saas-core", 84, "bounced")]
|
||||
assert len(reviewer.posted) == 1
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
|
||||
|
||||
def test_comment_boundary_failure_retried(policy_factory):
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(85, head_sha="sha-co", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("standard", number=1),
|
||||
verdicts={85: make_verdict(85, APPROVE, head_sha="sha-co")})
|
||||
gitea.fail_next["post_comment"] = RuntimeError("comment boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert gitea.comments == []
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "approved_awaiting_merge"
|
||||
assert any("awaiting operator merge" in body for _, _, body in gitea.comments)
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
|
||||
|
||||
def test_remove_label_boundary_failure_retried(policy_factory):
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(86, head_sha="sha-rl", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={86: make_verdict(86, APPROVE, head_sha="sha-rl")})
|
||||
gitea.fail_next["remove_label"] = RuntimeError("label boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert gitea.merged == [("gitea-saas-core", 86)] # merge already applied
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
assert gitea.merged == [("gitea-saas-core", 86)] # NO second merge
|
||||
assert gitea.closed == [("gitea-saas-core", 1)] # close exactly once
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
|
||||
|
||||
def test_merge_refused_is_retryable_not_completed(policy_factory):
|
||||
# A merge returning False is a non-completion, not a completed outcome:
|
||||
# the head must stay eligible for retry until the action succeeds.
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(87, head_sha="sha-mf", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={87: make_verdict(87, APPROVE, head_sha="sha-mf")})
|
||||
gitea.merge_results[87] = False
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["action"] == "merge_failed"
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"] == {"code": "merge_failed"}
|
||||
assert gitea.closed == []
|
||||
assert policy._already_reviewed("gitea-saas-core", 87, "sha-mf") is False
|
||||
|
||||
gitea.merge_results[87] = True
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
# The refused attempt had no side effect, so the retry re-attempted and
|
||||
# succeeded. The fake records every merge ATTEMPT (like the API call),
|
||||
# so two attempts are visible; the action itself completed exactly once.
|
||||
# (The no-double-merge guarantee after a SUCCEEDED merge + later failure
|
||||
# is asserted by the close/debit/remove-label boundary tests above.)
|
||||
assert gitea.merged == [("gitea-saas-core", 87), ("gitea-saas-core", 87)]
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
|
||||
|
||||
def test_webhook_after_completed_poll_does_not_double_close(policy_factory):
|
||||
# The poll path already applied the outcome; the merge-event webhook
|
||||
# redelivery must replay the debit (idempotency key) but not close the
|
||||
# linked issue a second time.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(88, head_sha="sha-wh", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={88: make_verdict(88, APPROVE, head_sha="sha-wh")})
|
||||
|
||||
policy.poll_and_review("gitea-saas-core")
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
|
||||
ok, details = policy.handle_merge_event("gitea-saas-core", 88)
|
||||
assert ok and details["debit"]["debit"] == "recorded"
|
||||
assert details["debit"].get("replayed") is True
|
||||
assert gitea.closed == [("gitea-saas-core", 1)] # closed exactly once
|
||||
assert len(_ledger_debits(core, c["id"])) == 1 # debited exactly once
|
||||
|
||||
|
||||
def test_legacy_review_row_still_skips_head(policy_factory):
|
||||
# Rows written under the pre-#29 schema (no retry metadata) keep their
|
||||
# completed meaning: the head is skipped and never re-reviewed.
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(89, head_sha="sha-legacy", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={89: make_verdict(89, APPROVE)})
|
||||
policy._reviews_conn.execute(
|
||||
"INSERT INTO reviews (repo, pr, head_sha, verdict, reviewed_at)"
|
||||
" VALUES (?, ?, ?, ?, ?)",
|
||||
("gitea-saas-core", 89, "sha-legacy", APPROVE, "2026-01-01T00:00:00+00:00"))
|
||||
policy._reviews_conn.commit()
|
||||
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
assert reviewer.calls == [] and gitea.merged == []
|
||||
|
||||
|
||||
def test_failed_attempt_then_head_move_reviews_new_head(policy_factory):
|
||||
# The verdict follows the LIVE head (as the real reviewer's does), so a
|
||||
# failed gold attempt retries at the same head, and after the head moves
|
||||
# the NEW pair is reviewed and merged fresh.
|
||||
def live_head_verdict(repo, pr):
|
||||
return make_verdict(pr, APPROVE,
|
||||
head_sha=gitea.prs[0]["head_sha"])
|
||||
|
||||
policy, gitea, reviewer, _, _ = policy_factory(
|
||||
gitea_prs=[pr(90, head_sha="sha-a", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={90: live_head_verdict})
|
||||
gitea.fail_next["merge_pr"] = RuntimeError("merge boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
|
||||
gitea.prs[0]["head_sha"] = "sha-b" # head moved before the retry
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0]["action"] == "merged"
|
||||
assert policy._already_reviewed("gitea-saas-core", 90, "sha-a") is False
|
||||
assert policy._already_reviewed("gitea-saas-core", 90, "sha-b") is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Crash-window recovery independent of the open-PR scan (issue #29 review
|
||||
# bounce): once a merge truly succeeds the PR is no longer returned by
|
||||
# list_open_prs(), so the persisted retry row must be finished from the
|
||||
# review_state table alone, and replay must never duplicate an
|
||||
# externally-visible side effect (label removal, close, success comment).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_merged_pr_gone_from_open_scan_recovered_after_post_merge_failure(policy_factory):
|
||||
# The reviewer's exact scenario: merge succeeds, a post-merge step
|
||||
# (label removal) fails, and the merged PR disappears from the open-PR
|
||||
# scan before the next retry. The persisted row must still be finished
|
||||
# — exactly one merge, one debit, one label removal, one close, one
|
||||
# success comment.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(91, head_sha="sha-91", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={91: make_verdict(91, APPROVE, head_sha="sha-91")})
|
||||
gitea.fail_next["remove_label"] = RuntimeError("label boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"]["message"] == "label boom"
|
||||
assert gitea.merged == [("gitea-saas-core", 91)] # merge applied
|
||||
# The merged PR is gone from list_open_prs() — exactly what real Gitea
|
||||
# does once a merge succeeds — so the open scan can never reach it.
|
||||
gitea.prs = [p for p in gitea.prs if p["number"] != 91]
|
||||
assert gitea.list_open_prs() == []
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second and second[0]["action"] == "merged"
|
||||
assert second[0].get("recovered") is True # via recovery
|
||||
assert gitea.merged == [("gitea-saas-core", 91)] # NO second merge
|
||||
assert gitea.labels_removed.count(("gitea-saas-core", 91, "in-review")) == 1
|
||||
assert gitea.closed == [("gitea-saas-core", 1)] # closed exactly once
|
||||
comments = [b for _, _, b in gitea.comments]
|
||||
assert sum("Merged by review-gate" in b for b in comments) == 1
|
||||
assert len(_ledger_debits(core, c["id"])) == 1 # debited exactly once
|
||||
assert policy._already_reviewed("gitea-saas-core", 91, "sha-91") is True
|
||||
assert policy.poll_and_review("gitea-saas-core") == [] # completed: skipped
|
||||
|
||||
|
||||
def test_crash_after_label_removed_no_duplicate_visible_actions(policy_factory):
|
||||
# remove_label succeeded, then a later step (the success comment)
|
||||
# failed: the retry must not remove the label again, close the issue
|
||||
# again, or re-post the comment — even though the PR is gone from the
|
||||
# open scan.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(92, head_sha="sha-92", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={92: make_verdict(92, APPROVE, head_sha="sha-92")})
|
||||
gitea.fail_next["post_comment"] = RuntimeError("comment boom")
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "failed"
|
||||
assert first[0]["error"]["message"] == "comment boom"
|
||||
# merge, debit, label removal and close all succeeded before the crash
|
||||
assert gitea.merged == [("gitea-saas-core", 92)]
|
||||
assert gitea.labels_removed.count(("gitea-saas-core", 92, "in-review")) == 1
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
|
||||
gitea.prs = [p for p in gitea.prs if p["number"] != 92]
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second[0].get("recovered") is True
|
||||
comments = [b for _, _, b in gitea.comments]
|
||||
assert sum("Merged by review-gate" in b for b in comments) == 1 # once
|
||||
assert gitea.labels_removed.count(("gitea-saas-core", 92, "in-review")) == 1
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
|
||||
|
||||
def test_crash_after_success_comment_before_completion_record(policy_factory):
|
||||
# The success comment posted, then the process crashed before the
|
||||
# completion row was written: replay must finish the head WITHOUT
|
||||
# re-posting the comment or repeating any other side effect.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(93, head_sha="sha-93", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={93: make_verdict(93, APPROVE, head_sha="sha-93")})
|
||||
|
||||
real_record = policy._record_completed
|
||||
|
||||
def crash_after_comment(*args, **kwargs):
|
||||
raise RuntimeError("process killed after success comment")
|
||||
|
||||
policy._record_completed = crash_after_comment
|
||||
with pytest.raises(RuntimeError): # the simulated crash
|
||||
policy.poll_and_review("gitea-saas-core")
|
||||
policy._record_completed = real_record
|
||||
|
||||
comments = [b for _, _, b in gitea.comments]
|
||||
assert sum("Merged by review-gate" in b for b in comments) == 1
|
||||
assert gitea.merged == [("gitea-saas-core", 93)]
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert gitea.labels_removed.count(("gitea-saas-core", 93, "in-review")) == 1
|
||||
|
||||
gitea.prs = [p for p in gitea.prs if p["number"] != 93]
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second and second[0].get("recovered") is True
|
||||
comments = [b for _, _, b in gitea.comments]
|
||||
assert sum("Merged by review-gate" in b for b in comments) == 1 # not duplicated
|
||||
assert gitea.labels_removed.count(("gitea-saas-core", 93, "in-review")) == 1
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
assert policy.poll_and_review("gitea-saas-core") == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governor review #2 regressions:
|
||||
# 1. the merge is pinned to the REVIEWED head (expected_head_sha) — a branch
|
||||
# that moved after review is refused before any merge POST;
|
||||
# 2. recovery distinguishes a merged PR from a merely closed one using the
|
||||
# PRODUCTION normalized shape (positive merged is True only);
|
||||
# 3. a closed-but-unmerged PR is never adopted as merged (no debit, no label
|
||||
# removal, no issue close, no success comment).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_reviewed_head_pinned_so_moved_branch_is_never_merged(policy_factory):
|
||||
# Integration regression: review evaluates SHA A, the branch moves to
|
||||
# SHA B before the verdict is applied. The evaluated-pair gate (issue
|
||||
# #30) refuses BEFORE any policy action — no merge POST, no debit, no
|
||||
# close, no success comment. Once the branch sits at the SHAs the
|
||||
# verdict describes, the pair is fresh-eligible and merges normally.
|
||||
state = {"moved": False}
|
||||
|
||||
def review_then_move_head(repo, pr):
|
||||
evaluated = gitea.prs[0]["head_sha"] # sha-A at review time
|
||||
if not state["moved"]:
|
||||
state["moved"] = True
|
||||
gitea.prs[0]["head_sha"] = "sha-B" # branch moved after review
|
||||
return make_verdict(pr, APPROVE, head_sha=evaluated)
|
||||
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(94, head_sha="sha-A", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={94: review_then_move_head})
|
||||
|
||||
first = policy.poll_and_review("gitea-saas-core")
|
||||
assert first[0]["outcome"] == "stale"
|
||||
assert first[0]["error"]["code"] == "PAIR_MOVED"
|
||||
assert "sha-A" in first[0]["error"]["message"] and "sha-B" in first[0]["error"]["message"]
|
||||
assert gitea.merged == [] # NO merge POST happened
|
||||
assert gitea.closed == []
|
||||
assert not any("Merged by review-gate" in b for _, _, b in gitea.comments)
|
||||
assert _ledger_debits(core, c["id"]) == [] # no billing for the moved pair
|
||||
|
||||
second = policy.poll_and_review("gitea-saas-core")
|
||||
assert second and second[0]["action"] == "merged" # sha-B reviewed + merged
|
||||
assert gitea.merged == [("gitea-saas-core", 94)] # exactly one merge, ever
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
assert policy._already_reviewed("gitea-saas-core", 94, "sha-A") is False
|
||||
assert policy._already_reviewed("gitea-saas-core", 94, "sha-B") is True
|
||||
|
||||
|
||||
def test_crash_window_adopts_merged_pr_from_production_shape(policy_factory):
|
||||
# The merge applied server-side but the process died before the
|
||||
# merged_step checkpoint write: with the production normalized shape
|
||||
# (merged/state now included), recovery positively adopts the merged PR
|
||||
# and finishes debit/label/close/comment — without ever calling
|
||||
# merge_pr again.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(96, head_sha="sha-96", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={})
|
||||
policy._ensure_plan("gitea-saas-core", 96, "sha-96", APPROVE, "gold", 1)
|
||||
# The live PR record already reports the server-side merge (raw Gitea
|
||||
# record normalized by the production client); the checkpoint is the
|
||||
# only thing missing.
|
||||
gitea.prs[0]["merged"] = True
|
||||
gitea.prs[0]["state"] = "closed"
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
assert results and results[0]["action"] == "merged"
|
||||
assert results[0].get("recovered") is True
|
||||
assert gitea.merged == [] # crash window: no merge request needed
|
||||
assert gitea.labels_removed.count(("gitea-saas-core", 96, "in-review")) == 1
|
||||
assert gitea.closed == [("gitea-saas-core", 1)]
|
||||
assert any("Merged by review-gate" in b for _, _, b in gitea.comments)
|
||||
assert len(_ledger_debits(core, c["id"])) == 1
|
||||
assert policy._already_reviewed("gitea-saas-core", 96, "sha-96") is True
|
||||
assert policy.poll_and_review("gitea-saas-core") == [] # completed: skipped
|
||||
|
||||
|
||||
def test_closed_unmerged_pr_recovery_never_debits_or_closes(policy_factory):
|
||||
# Negative regression: state == "closed" is NOT proof of a merge. The
|
||||
# outcome plan exists for the head, the merge never happened, and the
|
||||
# PR is closed without merging — recovery must not adopt it as merged:
|
||||
# no debit, no label removal, no issue close, no success comment.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(95, head_sha="sha-95", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={})
|
||||
policy._ensure_plan("gitea-saas-core", 95, "sha-95", APPROVE, "gold", 1)
|
||||
gitea.prs[0]["state"] = "closed" # closed WITHOUT merging
|
||||
gitea.prs[0]["merged"] = False
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
assert results == [] # nothing adopted, nothing applied
|
||||
assert gitea.merged == [] and gitea.closed == []
|
||||
assert gitea.labels_removed == []
|
||||
assert gitea.comments == []
|
||||
assert _ledger_debits(core, c["id"]) == []
|
||||
assert policy._step_done("gitea-saas-core", 95, "sha-95", "merged") is False
|
||||
assert policy._already_reviewed("gitea-saas-core", 95, "sha-95") is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Governor review #3 (10539) regressions — head-integrity gates:
|
||||
# 1. recovery adopts a merged PR ONLY when the live merged head equals the
|
||||
# persisted planned/reviewed head; a DIFFERENT merged head fails closed
|
||||
# with a structured HEAD_MISMATCH and no debit/label/close/comment;
|
||||
# 2. a candidate with NO reviewed head_sha is rejected BEFORE the merge
|
||||
# client is called (expected_head_sha=None would arm the client's
|
||||
# live-head fallback and could merge an unreviewed head).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_recovery_merged_different_head_fails_closed(policy_factory):
|
||||
# Negative regression: the plan was persisted for reviewed head sha-A,
|
||||
# but the branch moved and head sha-B was merged externally before the
|
||||
# merged_step checkpoint write. Recovery must NOT attribute the sha-B
|
||||
# merge to the sha-A plan: zero side effects (no debit, no label
|
||||
# removal, no issue close, no success comment), a structured
|
||||
# HEAD_MISMATCH result, and the row left unadopted for the operator.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(97, head_sha="sha-A", labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={})
|
||||
policy._ensure_plan("gitea-saas-core", 97, "sha-A", APPROVE, "gold", 1)
|
||||
gitea.prs[0]["head_sha"] = "sha-B" # branch moved AND merged externally
|
||||
gitea.prs[0]["merged"] = True
|
||||
gitea.prs[0]["state"] = "closed"
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
assert results and results[0]["outcome"] == "failed"
|
||||
assert results[0]["error"]["code"] == "HEAD_MISMATCH"
|
||||
assert "sha-A" in results[0]["error"]["message"]
|
||||
assert "sha-B" in results[0]["error"]["message"]
|
||||
assert gitea.merged == [] and gitea.closed == []
|
||||
assert gitea.labels_removed == []
|
||||
assert gitea.comments == []
|
||||
assert _ledger_debits(core, c["id"]) == []
|
||||
assert policy._step_done("gitea-saas-core", 97, "sha-A", "merged") is False
|
||||
assert policy._already_reviewed("gitea-saas-core", 97, "sha-A") is False
|
||||
|
||||
|
||||
def test_missing_reviewed_head_never_reaches_merge_client(policy_factory):
|
||||
# Negative regression: a candidate PR with no head_sha must be refused
|
||||
# BEFORE the merge client is called — passing expected_head_sha=None
|
||||
# would arm the production client's live-head fallback and could merge
|
||||
# a head the policy never reviewed. Assert zero merge attempts and zero
|
||||
# billing/label/close/comment side effects, with a structured failure.
|
||||
policy, gitea, reviewer, core, c = policy_factory(
|
||||
gitea_prs=[pr(98, labels=("in-review",))],
|
||||
gitea_issues=tier_issue("gold", number=1),
|
||||
verdicts={98: make_verdict(98, APPROVE, head_sha=None)})
|
||||
gitea.prs[0]["head_sha"] = None # no reviewed head on the candidate
|
||||
|
||||
results = policy.poll_and_review("gitea-saas-core")
|
||||
assert results and results[0]["outcome"] == "failed"
|
||||
assert "no reviewed head_sha" in results[0]["error"]["message"]
|
||||
assert "fall back to the live head" in results[0]["error"]["message"]
|
||||
assert gitea.merged == [] and gitea.closed == []
|
||||
assert gitea.labels_removed == []
|
||||
assert gitea.comments == []
|
||||
assert _ledger_debits(core, c["id"]) == []
|
||||
assert policy._already_reviewed("gitea-saas-core", 98, "unknown") is False
|
||||
|
||||
383
tests/test_review_workspace.py
Normal file
383
tests/test_review_workspace.py
Normal file
@ -0,0 +1,383 @@
|
||||
"""Hermetic tests for the isolated review workspace (issue #30).
|
||||
|
||||
Real ``git`` binaries against LOCAL temporary repositories only — every
|
||||
"remote" is a local-path ``origin`` (a bare repo under ``tmp_path``), so
|
||||
the suite proves the exact production flow (fetch named SHAs, detached
|
||||
worktree, synthetic merge, cleanup) with zero network calls.
|
||||
|
||||
Run with: python -m pytest tests/test_review_workspace.py -q
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
# PEP 420 namespace package: make the project root importable regardless of
|
||||
# how pytest was invoked.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from gitea_saas.review_workspace import ( # noqa: E402
|
||||
ReviewWorkspace,
|
||||
ReviewWorkspaceError,
|
||||
)
|
||||
from gitea_saas.reviewer import Runner # noqa: E402
|
||||
|
||||
GIT = ["git"]
|
||||
|
||||
|
||||
def git(repo, *args):
|
||||
"""Run git in ``repo``; return the CompletedProcess (checked)."""
|
||||
repo = Path(repo)
|
||||
if not repo.exists() and args and args[0] == "init":
|
||||
repo.mkdir(parents=True, exist_ok=True)
|
||||
return subprocess.run(
|
||||
[*GIT, "-C", str(repo), *args], check=True,
|
||||
capture_output=True, text=True)
|
||||
|
||||
|
||||
def git_ok(repo, *args):
|
||||
return git(repo, *args).returncode == 0
|
||||
|
||||
|
||||
def commit_file(repo, filename, content, message):
|
||||
path = repo / filename
|
||||
path.write_text(content)
|
||||
git(repo, "add", filename)
|
||||
git(repo, "commit", "-m", message)
|
||||
return git(repo, "rev-parse", "HEAD").stdout.strip()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fixtures(tmp_path):
|
||||
"""Build a local origin (bare), a seeding checkout, and the configured
|
||||
clone. Both the `clean` and the `conflict` head/base pairs come from
|
||||
the same origin (all SHAs are advertised branch tips, so the exact-SHA
|
||||
fetch works over the local-path transport exactly like production)."""
|
||||
origin = tmp_path / "origin.git"
|
||||
git(origin, "init", "--bare", "--initial-branch=main")
|
||||
|
||||
seed = tmp_path / "seed"
|
||||
git(seed, "init", "--initial-branch=main")
|
||||
git(seed, "config", "user.email", "seed@example.com")
|
||||
git(seed, "config", "user.name", "Seed")
|
||||
git(seed, "remote", "add", "origin", str(origin))
|
||||
|
||||
base0 = commit_file(seed, "alpha.txt", "line one\n", "base file")
|
||||
git(seed, "push", "-u", "origin", "main")
|
||||
|
||||
# ---- clean pair: feature adds beta.txt, main adds gamma.txt ----
|
||||
git(seed, "checkout", "-b", "feature", base0)
|
||||
head_clean = commit_file(seed, "beta.txt", "feature add\n", "feature work")
|
||||
git(seed, "push", "-u", "origin", "feature")
|
||||
git(seed, "checkout", "main")
|
||||
base_clean = commit_file(seed, "gamma.txt", "main add\n", "main work")
|
||||
git(seed, "push", "origin", "main")
|
||||
|
||||
# ---- conflict pair: both sides edit alpha.txt after branching ----
|
||||
git(seed, "checkout", "-b", "conflict-head", base0)
|
||||
head_conflict = commit_file(seed, "alpha.txt", "feature change\n", "conflict head")
|
||||
git(seed, "checkout", "main")
|
||||
base_conflict = commit_file(seed, "alpha.txt", "main change\n", "conflict base")
|
||||
git(seed, "push", "origin", "main")
|
||||
git(seed, "push", "-u", "origin", "conflict-head")
|
||||
|
||||
clone = tmp_path / "clone"
|
||||
subprocess.run(["git", "clone", str(origin), str(clone)],
|
||||
check=True, capture_output=True, text=True)
|
||||
git(clone, "config", "user.email", "clone@example.com")
|
||||
git(clone, "config", "user.name", "Clone")
|
||||
clone_head_before = git(clone, "rev-parse", "HEAD").stdout.strip()
|
||||
|
||||
return SimpleNamespace(
|
||||
origin=origin, seed=seed, clone=clone,
|
||||
head_clean=head_clean, base_clean=base_clean,
|
||||
head_conflict=head_conflict, base_conflict=base_conflict,
|
||||
clone_head_before=clone_head_before,
|
||||
)
|
||||
|
||||
|
||||
def review_dirs(pr_number):
|
||||
root = Path(tempfile.gettempdir())
|
||||
return sorted(root.glob(f"review-ws-acme-{pr_number}-*"))
|
||||
|
||||
|
||||
def clone_worktree_lines(clone):
|
||||
return git(clone, "worktree", "list").stdout.strip()
|
||||
|
||||
|
||||
def assert_clone_untouched(fx):
|
||||
assert git(fx.clone, "rev-parse", "HEAD").stdout.strip() == fx.clone_head_before
|
||||
assert git(fx.clone, "status", "--porcelain").stdout.strip() == ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Clean merge
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_clean_merge_constructs_synthetic_merge(fixtures):
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
with ws.build("acme", 1, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
# The working tree IS the merge result: both sides' files present,
|
||||
# both parent commits in history.
|
||||
assert (mws.dir / "beta.txt").read_text() == "feature add\n"
|
||||
assert (mws.dir / "gamma.txt").read_text() == "main add\n"
|
||||
assert (mws.dir / "alpha.txt").read_text() == "line one\n"
|
||||
parents = git(mws.dir, "rev-list", "--parents", "-n", "1", "HEAD").stdout.split()
|
||||
assert len(parents) == 3 # merge commit + base + head
|
||||
assert mws.evidence["base_sha"] == fixtures.base_clean
|
||||
assert mws.evidence["head_sha"] == fixtures.head_clean
|
||||
assert mws.evidence["result_head"] == parents[0]
|
||||
# Explicit base/result range works from inside the workspace.
|
||||
res = Runner().run(mws.dir, f"git diff --check {fixtures.base_clean} HEAD")
|
||||
assert res["exit"] == 0
|
||||
# Cleaned up after success; the configured clone is untouched.
|
||||
assert review_dirs(1) == []
|
||||
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Merge conflict
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_merge_conflict_is_deterministic_evidence(fixtures):
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
with ws.build("acme", 2, fixtures.head_conflict, fixtures.base_conflict) as mws:
|
||||
assert mws.merge_ok is False
|
||||
assert mws.evidence["merge_exit"] != 0
|
||||
assert mws.evidence["conflicted_files"] == ["alpha.txt"]
|
||||
assert "alpha.txt" in mws.evidence["status_summary"]
|
||||
assert review_dirs(2) == []
|
||||
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# No shared-checkout mutation (source clone AND a worker-style worktree)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_neither_clone_nor_worker_worktree_is_mutated(fixtures):
|
||||
# A worker's issue worktree taken from the same clone must be untouched
|
||||
# by the review pass (its branch, content, and dirty state all survive).
|
||||
worker = fixtures.clone.parent / "worker-issue-42"
|
||||
git(fixtures.clone, "worktree", "add", "--detach", str(worker), fixtures.base_clean)
|
||||
(worker / "alpha.txt").write_text("worker dirty line\n") # simulate WIP
|
||||
worker_head_before = git(worker, "rev-parse", "HEAD").stdout.strip()
|
||||
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
with ws.build("acme", 3, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
# the worker checkout's content is untouched by the merge
|
||||
assert (worker / "alpha.txt").read_text() == "worker dirty line\n"
|
||||
assert git(worker, "rev-parse", "HEAD").stdout.strip() == worker_head_before
|
||||
# (strip(): the porcelain status column's leading space is whitespace)
|
||||
assert git(worker, "status", "--porcelain").stdout.strip() == "M alpha.txt"
|
||||
|
||||
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
|
||||
# the worker worktree itself still exists with its dirty state
|
||||
assert worker.exists()
|
||||
assert git(worker, "status", "--porcelain").stdout.strip() == "M alpha.txt"
|
||||
assert_clone_untouched(fixtures)
|
||||
git(fixtures.clone, "worktree", "remove", "--force", str(worker))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cleanup on failure paths
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_cleanup_after_raised_exception(fixtures):
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
before = review_dirs(4)
|
||||
with pytest.raises(RuntimeError, match="boom"):
|
||||
with ws.build("acme", 4, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
raise RuntimeError("boom")
|
||||
assert review_dirs(4) == before
|
||||
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
def test_cleanup_after_merge_conflict_exit(fixtures):
|
||||
# Simulates the context manager exiting with a conflict result — the
|
||||
# workspace must still be fully removed.
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
before = review_dirs(5)
|
||||
with ws.build("acme", 5, fixtures.head_conflict, fixtures.base_conflict) as mws:
|
||||
assert mws.merge_ok is False
|
||||
assert review_dirs(5) == before
|
||||
assert "review-ws" not in clone_worktree_lines(fixtures.clone)
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
def test_unfetchable_sha_raises_and_leaves_nothing(fixtures):
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
before = review_dirs(6)
|
||||
with pytest.raises(ReviewWorkspaceError):
|
||||
ws.build("acme", 6, "f" * 40, fixtures.base_clean)
|
||||
assert review_dirs(6) == before
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
def test_non_hex_sha_fails_closed_without_git(fixtures):
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
with pytest.raises(ReviewWorkspaceError, match="not a 40-char hex"):
|
||||
ws.build("acme", 7, "HEAD", fixtures.base_clean) # a ref, not a SHA
|
||||
with pytest.raises(ReviewWorkspaceError, match="not a 40-char hex"):
|
||||
ws.build("acme", 7, fixtures.head_clean, "main")
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Commands run in the right place (cwd + explicit range)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_git_commands_run_in_workspace_cwd(fixtures):
|
||||
calls = []
|
||||
|
||||
def spy(command, **kwargs):
|
||||
calls.append((kwargs.get("cwd"), command))
|
||||
if isinstance(command, str):
|
||||
return subprocess.run(command, shell=True, **kwargs)
|
||||
return subprocess.run(command, shell=False, **kwargs)
|
||||
|
||||
runner = Runner(timeout_s=120, command_runner=spy)
|
||||
ws = ReviewWorkspace(fixtures.clone, runner)
|
||||
with ws.build("acme", 8, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
merges = [c for c in calls
|
||||
if isinstance(c[1], list)
|
||||
and c[1][:4] == ["git", "merge", "--no-ff", "-m"]]
|
||||
assert merges, "synthetic merge command must run"
|
||||
assert Path(merges[0][0]) == mws.dir, "merge must run INSIDE the workspace"
|
||||
assert "review-ws" in str(merges[0][0])
|
||||
# fetch + worktree add run against the configured clone (read-only use)
|
||||
adds = [c for c in calls
|
||||
if isinstance(c[1], list)
|
||||
and c[1][:4] == ["git", "worktree", "add", "--detach"]]
|
||||
assert adds and Path(adds[0][0]) == fixtures.clone
|
||||
fetches = [c for c in calls
|
||||
if isinstance(c[1], list)
|
||||
and c[1][:3] == ["git", "fetch", "origin"]]
|
||||
assert fetches and Path(fetches[0][0]) == fixtures.clone
|
||||
# cleanup command targets the clone
|
||||
removes = [c for c in calls
|
||||
if isinstance(c[1], list)
|
||||
and c[1][:4] == ["git", "worktree", "remove", "--force"]]
|
||||
assert removes and Path(removes[0][0]) == fixtures.clone
|
||||
|
||||
|
||||
def test_explicit_range_diff_check_from_workspace(fixtures):
|
||||
calls = []
|
||||
|
||||
def spy(command, **kwargs):
|
||||
calls.append((kwargs.get("cwd"), command))
|
||||
if isinstance(command, str):
|
||||
return subprocess.run(command, shell=True, **kwargs)
|
||||
return subprocess.run(command, shell=False, **kwargs)
|
||||
|
||||
runner = Runner(timeout_s=120, command_runner=spy)
|
||||
ws = ReviewWorkspace(fixtures.clone, runner)
|
||||
with ws.build("acme", 9, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
# The reviewer's hygiene gate: diff --check over the EXPLICIT
|
||||
# base..result range, run from INSIDE the isolated workspace.
|
||||
res = runner.run(
|
||||
mws.dir, ["git", "diff", "--check",
|
||||
fixtures.base_clean, "HEAD"])
|
||||
assert res["exit"] == 0
|
||||
checks = [c for c in calls
|
||||
if isinstance(c[1], list)
|
||||
and c[1][:3] == ["git", "diff", "--check"]]
|
||||
assert checks, "explicit-range diff --check must run"
|
||||
assert checks[0][1][3] == fixtures.base_clean # explicit BASE
|
||||
assert checks[0][1][4] == "HEAD" # explicit result end
|
||||
assert "review-ws" in str(checks[0][0]) # inside the isolated workspace
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Windows portability (issue #48): git args reach git as a LIST, never a
|
||||
# shell string — cmd.exe must not be able to eat `^` (or any other
|
||||
# shell-special char) from `^{commit}` and friends.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_caret_peel_reaches_git_as_single_list_token():
|
||||
recorded = []
|
||||
|
||||
def fake_runner(command, **kwargs):
|
||||
recorded.append(command)
|
||||
# All invocations are argument lists (never shell strings): every
|
||||
# git command is recorded for the assertions below.
|
||||
assert isinstance(command, list), (
|
||||
f"git command must be an argument list, got {command!r}")
|
||||
head = command[:2]
|
||||
if head == ["git", "rev-parse"] and "--verify" in command:
|
||||
return SimpleNamespace(
|
||||
returncode=0, stdout="1fccd03843f7d3e7ef0bf5428aa7a40114b70263",
|
||||
stderr="")
|
||||
return SimpleNamespace(returncode=0, stdout="", stderr="")
|
||||
|
||||
runner = Runner(timeout_s=60, command_runner=fake_runner)
|
||||
ws = ReviewWorkspace("/fake/clone", runner)
|
||||
sha = "d3df1df0c2e458ac5da94a4edb2d254777f64a02"
|
||||
with ws.build("acme", 48, sha, sha) as mws:
|
||||
assert mws.merge_ok is True
|
||||
# The peel suffix survives as ONE list element — cmd.exe never sees
|
||||
# it, so the Windows `^`-escape class of breakage is impossible.
|
||||
peels = [c for c in recorded
|
||||
if c[:2] == ["git", "rev-parse"]
|
||||
and "--verify" in c and c[-1].endswith("^{commit}")]
|
||||
assert peels, "rev-parse ^{commit} verification must run"
|
||||
assert all(len(c) == 4 and c[3] == f"{sha}^{{commit}}"
|
||||
for c in peels), f"peel must be a single token, got {peels}"
|
||||
# No stringified-list commands anywhere: a shell can never see them.
|
||||
assert all(isinstance(c, list) for c in recorded)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bounce-fix: shell-safe path quoting + filesystem-safe temp prefix
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_weird_temp_root_with_spaces_and_shell_char_success_and_cleanup(
|
||||
fixtures, tmp_path, monkeypatch):
|
||||
# A valid OS temp root whose path contains a space AND a shell-sensitive
|
||||
# character ($). Unquoted interpolation into the Runner's shell=True
|
||||
# commands would split the path (construction fails) or — worse —
|
||||
# let `git worktree remove` fail while rmtree deletes the directory,
|
||||
# leaving a dangling worktree admin entry in the clone.
|
||||
weird_root = tmp_path / "review root $weird"
|
||||
weird_root.mkdir(parents=True)
|
||||
monkeypatch.setattr(tempfile, "tempdir", str(weird_root))
|
||||
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
with ws.build("acme", 10, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
assert mws.dir.exists()
|
||||
assert " " in str(mws.dir) # really lived under the weird root
|
||||
assert "$" in str(mws.dir)
|
||||
assert mws.dir.parent == weird_root
|
||||
# Complete cleanup: directory gone AND no worktree admin record remains.
|
||||
assert not mws.dir.exists()
|
||||
assert "review-ws" not in git(
|
||||
fixtures.clone, "worktree", "list", "--porcelain").stdout
|
||||
assert_clone_untouched(fixtures)
|
||||
|
||||
|
||||
def test_repo_token_with_separators_is_filesystem_safe(fixtures):
|
||||
# ``repo`` is embedded in the mkdtemp prefix; an owner/name-style token
|
||||
# must be collapsed to a single filename component (a NESTED prefix
|
||||
# would make tempfile.mkdtemp raise FileNotFoundError).
|
||||
repo = "pbwraith/gitea-saas-core"
|
||||
ws = ReviewWorkspace(fixtures.clone, Runner())
|
||||
with ws.build(repo, 11, fixtures.head_clean, fixtures.base_clean) as mws:
|
||||
assert mws.merge_ok is True
|
||||
assert mws.dir.name.startswith("review-ws-pbwraith-gitea-saas-core-11-")
|
||||
assert "/" not in mws.dir.name
|
||||
assert not mws.dir.exists()
|
||||
assert "review-ws" not in git(
|
||||
fixtures.clone, "worktree", "list", "--porcelain").stdout
|
||||
assert_clone_untouched(fixtures)
|
||||
@ -11,6 +11,7 @@ import ast
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
@ -36,13 +37,14 @@ class FakeGiteaClient:
|
||||
"""In-memory Gitea seam: PRs, issues and posted comments."""
|
||||
|
||||
def __init__(self, prs=None, issues=None):
|
||||
self.prs = prs or {} # number -> {"head_sha", "changed_files", "diff"}
|
||||
self.prs = prs or {} # number -> {"head_sha", "base_sha", "changed_files", "diff"}
|
||||
self.issues = issues or {} # number -> {"body", ...}
|
||||
self.comments = [] # (repo, number, body)
|
||||
|
||||
def get_pr(self, repo, number):
|
||||
pr = self.prs[number]
|
||||
return {"head_sha": pr["head_sha"], "changed_files": list(pr["changed_files"])}
|
||||
return {"head_sha": pr["head_sha"], "base_sha": pr.get("base_sha") or SAMPLE_BASE,
|
||||
"changed_files": list(pr["changed_files"])}
|
||||
|
||||
def get_pr_diff(self, repo, number):
|
||||
return self.prs[number].get("diff", "")
|
||||
@ -54,6 +56,37 @@ class FakeGiteaClient:
|
||||
self.comments.append((repo, number, body))
|
||||
|
||||
|
||||
SAMPLE_BASE = "b" * 40
|
||||
|
||||
|
||||
class FakeWorkspace:
|
||||
"""Duck-typed review-workspace seam: no git, scripted merge result."""
|
||||
|
||||
def __init__(self, merge_ok=True, evidence=None, directory="/fake/ws",
|
||||
base_sha=SAMPLE_BASE):
|
||||
self.merge_ok = merge_ok
|
||||
self.evidence = dict(evidence or {})
|
||||
self.directory = str(directory)
|
||||
self.base_sha = base_sha
|
||||
|
||||
def build(self, repo, pr_number, head_sha, base_sha):
|
||||
return _FakeMergeContext(self)
|
||||
|
||||
|
||||
class _FakeMergeContext:
|
||||
def __init__(self, ws):
|
||||
self._ws = ws
|
||||
|
||||
def __enter__(self):
|
||||
return SimpleNamespace(
|
||||
dir=Path(self._ws.directory), base_sha=self._ws.base_sha,
|
||||
head_sha="a" * 40, merge_ok=self._ws.merge_ok,
|
||||
evidence=self._ws.evidence)
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
|
||||
|
||||
class FakeRunner:
|
||||
"""Subprocess seam: scripted results, records commands."""
|
||||
|
||||
@ -114,16 +147,19 @@ def test_approve_when_tests_pass_and_no_findings():
|
||||
{"exit": 0, "summary": "clean"}]) # git diff --check
|
||||
reviewer = Reviewer(client, runner, config={"secret_scan": True})
|
||||
|
||||
verdict = reviewer.review_pr("gitea-saas-core", 1, client.issues[5], "/fake/repo")
|
||||
verdict = reviewer.review_pr("gitea-saas-core", 1, client.issues[5],
|
||||
workspace=FakeWorkspace())
|
||||
|
||||
assert verdict.verdict == APPROVE
|
||||
assert verdict.tests == {"exit": 0, "summary": "12 passed"}
|
||||
assert verdict.acceptance == ["tests pass", "no regressions"]
|
||||
assert verdict.scope_violations == [] and verdict.findings == []
|
||||
assert verdict.head_sha == "a" * 40
|
||||
assert verdict.base_sha == "b" * 40
|
||||
|
||||
comment = reviewer.post_verdict("gitea-saas-core", 1, verdict)
|
||||
assert "[review-gate] VERDICT: **APPROVE**" in comment
|
||||
assert "head `aaaaaaaaaaaa`" in comment and "base `bbbbbbbbbbbb`" in comment
|
||||
assert "- [ ] tests pass" in comment # checklist present
|
||||
assert "No merge performed by the review gate" in comment
|
||||
assert client.comments and client.comments[0][:2] == ("gitea-saas-core", 1)
|
||||
@ -132,7 +168,8 @@ def test_approve_when_tests_pass_and_no_findings():
|
||||
def test_bounce_when_tests_fail():
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: sample_issue()})
|
||||
runner = FakeRunner([{"exit": 1, "summary": "1 failed, 11 passed"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, client.issues[5], "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, client.issues[5],
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == BOUNCE
|
||||
assert "tests failed (exit=1)" in verdict.summary
|
||||
# BOUNCE comment carries the test evidence
|
||||
@ -146,7 +183,8 @@ def test_bounce_on_scope_violation_even_with_green_tests():
|
||||
pr = sample_pr(files=["gitea_saas/reviewer.py", "gitea_saas/server.py"])
|
||||
client = FakeGiteaClient(prs={1: pr}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == BOUNCE
|
||||
assert verdict.scope_violations == ["gitea_saas/server.py"]
|
||||
|
||||
@ -155,7 +193,8 @@ def test_needs_guidance_when_no_acceptance_criteria():
|
||||
issue = {"body": "## Goal\nnothing to verify here\n"}
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == NEEDS_GUIDANCE
|
||||
assert "never silent-approve" in verdict.summary
|
||||
|
||||
@ -164,7 +203,8 @@ def test_allow_test_failure_in_issue_body_overrides_bounce():
|
||||
issue = sample_issue(extra="note: allow_test_failure for this transitional PR\n")
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 1, "summary": "known failure"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == APPROVE # tolerated, acceptance present, nothing else blocking
|
||||
|
||||
|
||||
@ -188,7 +228,8 @@ def test_secret_finding_quotes_and_redacts_diff_line():
|
||||
pr = sample_pr(diff=SECRET_DIFF, files=["gitea_saas/server.py"])
|
||||
client = FakeGiteaClient(prs={1: pr}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "ok"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
blocking = [f for f in verdict.findings if f["severity"] == "blocking"]
|
||||
assert blocking, "secret scan should produce a blocking finding"
|
||||
f = blocking[0]
|
||||
@ -208,7 +249,7 @@ def test_todo_finding_is_warning_not_blocking():
|
||||
client = FakeGiteaClient(prs={1: pr}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "ok"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner, config={"secret_scan": False}).review_pr(
|
||||
"gitea-saas-core", 1, issue, "/fake/repo")
|
||||
"gitea-saas-core", 1, issue, workspace=FakeWorkspace())
|
||||
todos = [f for f in verdict.findings if f["severity"] == "warning"]
|
||||
assert todos and todos[0]["note"] == "TODO/FIXME/HACK marker in added line"
|
||||
assert todos[0]["evidence_quote"] == " # TODO: replace placeholder"
|
||||
@ -224,7 +265,8 @@ def test_timeout_bounces_with_timed_out_exit():
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": "timed_out", "summary": "[timed out after 300s]"},
|
||||
{"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == BOUNCE
|
||||
assert verdict.tests["exit"] == "timed_out"
|
||||
assert "timed out" in verdict.summary
|
||||
@ -270,7 +312,8 @@ def test_build_command_run_when_configured_and_build_failure_bounces():
|
||||
{"exit": 1, "summary": "build error"}, # build
|
||||
{"exit": 0, "summary": "clean"}]) # diff --check
|
||||
reviewer = Reviewer(client, runner, config={"build_cmd": "python -m compileall -q ."})
|
||||
verdict = reviewer.review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = reviewer.review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
build = verdict.build
|
||||
assert build is not None
|
||||
assert build["exit"] == 1 and build["command"] == "python -m compileall -q ."
|
||||
@ -283,16 +326,76 @@ def test_diff_hygiene_failure_is_blocking_finding():
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"},
|
||||
{"exit": 1, "summary": "trailing whitespace at gitea_saas/x.py:3"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == BOUNCE
|
||||
assert any("diff --check" in f["note"] for f in verdict.findings)
|
||||
|
||||
|
||||
def test_hygiene_diff_check_uses_explicit_base_head_range():
|
||||
issue = sample_issue()
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"},
|
||||
{"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.verdict == APPROVE
|
||||
# The whitespace gate runs against the synthetic merge result with an
|
||||
# explicit base..HEAD range (issue #30 / acceptance 5), from the
|
||||
# workspace directory — never a bare `git diff --check`.
|
||||
assert len(runner.commands) == 2 # pytest + explicit-range diff --check
|
||||
ws_dir, hygiene_cmd, _ = runner.commands[1]
|
||||
# Compare as PATHs, not strings: on Windows str(Path('/fake/ws')) is
|
||||
# '\\fake\\ws' (a valid spelling of the same directory) — the string
|
||||
# comparison would fail on the platform this gate actually runs on.
|
||||
# Fix in the test, not the source: passing a Path as cwd is correct
|
||||
# (bounce-fix review 11154; issue #48).
|
||||
assert Path(ws_dir) == Path("/fake/ws")
|
||||
# The whitespace gate runs as an ARGUMENT LIST (shell=False) — no shell
|
||||
# can mangle the SHA/HEAD tokens on any platform (issue #48).
|
||||
assert hygiene_cmd == ["git", "diff", "--check",
|
||||
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "HEAD"]
|
||||
|
||||
|
||||
def test_merge_conflict_bounces_without_running_tests():
|
||||
issue = sample_issue()
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}]) # would be tests
|
||||
workspace = FakeWorkspace(merge_ok=False, evidence={
|
||||
"conflicted_files": ["gitea_saas/reviewer.py"],
|
||||
"status_summary": "UU gitea_saas/reviewer.py",
|
||||
"merge_exit": 1})
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=workspace)
|
||||
assert verdict.verdict == BOUNCE
|
||||
assert "synthetic merge conflict" in verdict.summary
|
||||
assert verdict.tests["exit"] == "not_run"
|
||||
assert runner.commands == [], "the test suite must NOT run on a conflicted merge"
|
||||
assert verdict.workspace["merge_ok"] is False
|
||||
assert verdict.workspace["conflicted_files"] == ["gitea_saas/reviewer.py"]
|
||||
comment = verdict.to_comment()
|
||||
assert "**CONFLICT**" in comment
|
||||
assert "The test suite was NOT run" in comment
|
||||
assert "`gitea_saas/reviewer.py`" in comment
|
||||
|
||||
|
||||
def test_no_workspace_fails_closed_to_needs_guidance():
|
||||
issue = sample_issue()
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue)
|
||||
assert verdict.verdict == NEEDS_GUIDANCE
|
||||
assert "no isolated review workspace configured" in verdict.summary
|
||||
assert verdict.tests["exit"] == "not_run"
|
||||
assert runner.commands == [], "no commands may run without a workspace"
|
||||
|
||||
|
||||
def test_verdict_shape_and_comment_never_mentions_merge_action():
|
||||
issue = sample_issue()
|
||||
client = FakeGiteaClient(prs={1: sample_pr()}, issues={5: issue})
|
||||
runner = FakeRunner([{"exit": 0, "summary": "12 passed"}, {"exit": 0, "summary": "clean"}])
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue, "/fake/repo")
|
||||
verdict = Reviewer(client, runner).review_pr("gitea-saas-core", 1, issue,
|
||||
workspace=FakeWorkspace())
|
||||
assert verdict.repo == "gitea-saas-core" and verdict.pr == 1
|
||||
assert isinstance(verdict.acceptance, list)
|
||||
assert isinstance(verdict.tests, dict) and "exit" in verdict.tests
|
||||
|
||||
@ -18,6 +18,7 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
from gitea_saas import billing_core, server
|
||||
from gitea_saas.auth import UNAUTHORIZED_BODY
|
||||
from gitea_saas.config import API_KEY_HEADER
|
||||
|
||||
API_KEY = "test-api-key-secret"
|
||||
@ -85,14 +86,14 @@ def test_health_is_public(client):
|
||||
def test_missing_key_is_401(client):
|
||||
resp = client.get("/api/clients/1")
|
||||
assert resp.status_code == 401
|
||||
body = resp.get_json()
|
||||
assert body["code"] == "unauthorized"
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
def test_wrong_key_is_401(client):
|
||||
resp = client.get("/api/clients/1", headers={API_KEY_HEADER: "nope"})
|
||||
assert resp.status_code == 401
|
||||
assert resp.get_json()["code"] == "unauthorized"
|
||||
# Canonical 401 body (issue #24): byte-identical across every front-end.
|
||||
assert resp.get_json() == UNAUTHORIZED_BODY
|
||||
|
||||
|
||||
def test_correct_key_is_200(client):
|
||||
|
||||
10
wsgi.py
Normal file
10
wsgi.py
Normal file
@ -0,0 +1,10 @@
|
||||
"""Gunicorn entrypoint (issue #38).
|
||||
|
||||
``gunicorn wsgi:application`` — the assembled app built by
|
||||
``gitea_saas.app.create_app`` from the environment (SAAS_API_KEY,
|
||||
SAAS_DB_PATH, SAAS_PORT).
|
||||
"""
|
||||
|
||||
from gitea_saas.app import create_app
|
||||
|
||||
application = create_app()
|
||||
Loading…
Reference in New Issue
Block a user