Review gate: automated PR reviewer (acceptance parse, tests, verdicts) (#5) #12
No reviewers
Labels
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: pbwraith/gitea-saas-core#12
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "issue-5"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #5
Review gate: the automated PR reviewer. No dependencies — built directly on main (docs-only baseline).
Files (only the allowed set):
gitea_saas/gitea_client.py— Gitea API seam (urllib, token from env, never on a command line):get_issue,get_pr(head_sha + changed_files),get_pr_diff,post_comment. Real HTTP only when a token exists; tests use an in-memory fake.gitea_saas/reviewer.py—Reviewer(repo_client, runner)withreview_pr(repo, pr, linked_issue, repo_dir) -> Verdictandpost_verdict(...)(posts[review-gate] VERDICT: ...comment with the acceptance checklist + findings).tests/test_reviewer.py— 16 hermetic tests.Pipeline per review: fetch PR head sha / files / diff -> parse acceptance checklist (
- [ ]/- [x], or## Acceptancesection) -> parse Files allowed scope -> runpython -m pytest -q(REVIEW_TEST_TIMEOUT_S, default 300) -> optional build cmd (REVIEW_BUILD_CMD) ->git diff --check-> findings pass (deterministic secret/TODO scan by default; configurable LLM pass validated so unquoted findings are dropped) -> verdict.Verdict rules: tests non-zero -> BOUNCE (unless
allow_test_failurein the issue body); timeout -> BOUNCE withtests.exit = "timed_out"; scope violation -> BOUNCE; blocking finding -> BOUNCE; empty acceptance -> NEEDS_GUIDANCE (never silent-approve); else APPROVE.Fail-closed guarantees:
evidence_quote), redacting leaked secret values; the no-LLM default is fully deterministic.Verification:
python -m pytest tests/test_reviewer.py -q-> 16 passed;git diff --checkclean; no network, no real subprocesses beyond injected fakes.[worker-lane3] Implemented on branch
issue-5(base: main, no dependencies); PR #12 open for review (Closes #5).Commit:
df72b39— "gitea-saas-core#5: review gate — automated PR reviewer (acceptance parse, tests, verdicts)"Test evidence:
python -m pytest tests/test_reviewer.py -q→ 16 passed;git diff --checkclean; hermetic (no network, no real subprocesses — FakeGiteaClient + FakeRunner seams, Runner timeout path tested via injected command runner). Acceptance coverage: APPROVE on green PR with comment checklist; BOUNCE on failing tests (evidence in verdict); BOUNCE with scope_violation outside Files allowed; NEEDS_GUIDANCE on issue without acceptance (never silent-approve); findings quote the diff (secret values redacted, lines anchored to real added lines); no merge call anywhere (AST-checked); timeout → BOUNCE with tests.exit="timed_out". Verdict rules + config via REVIEW_BUILD_CMD / REVIEW_TEST_TIMEOUT_S / REVIEW_SECRET_SCAN (+ optional llm callable, validated).Not merged — escalated to the owner, no defect found.
Verified so the owner reviews a known-good PR rather than an unknown one.
Rebased on the current
main(with #1 merged): 0 conflicts, 42 passed.I looked specifically at the credential handling, since that is what puts this
on the escalation list:
gitea_client.py:24—tokencomes from theGITEA_TOKENenv var or anexplicit argument, defaulting to
"".:30-37— it is used only as anAuthorization: token ...header on aurllibrequest. It is never logged, never interpolated into a message, andnever placed on a command line, which matches the module docstring's claim.
rejected by the server. There is no path where a missing credential grants
more access than a present one — which is exactly the failure mode its sibling
!11 does have, so it is worth stating that this one does not.
So this is sound as far as I can verify it. The only reason it is not merged is
the standing rule that this sweep does not merge changes touching auth or
credentials — that is a policy boundary, not a criticism of the code.
Deliberately leaving it unlabelled:
bouncedwould queue it for rework, andthere is nothing here to rework.
Superseded by !15 -- recommend closing rather than merging
git merge-base --is-ancestor origin/issue-5 origin/issue-6returns true, andthe
gitea_saas/gitea_client.pyblobs on the two branches hash identically(
9698050b6f2ebcb4). Merging !15 therefore lands this PR's work in full.Measured directly: applying !15 first takes the tree to 84 passed, and
applying !12 on top of that leaves it at 84 passed -- this PR contributes
zero additional tests once !15 is in.
Nothing here is wrong; it is just already contained in !15. Flagging it so
nobody spends a review cycle on it twice, and so it is not merged as a
third source of
gitea_client.py.Verified composition of all five non-bounced PRs (measured, not inferred)
Every one of these reports
mergeable=true; that flag is computed againstmainin isolation and says nothing about whether they coexist. Merged forreal into a throwaway worktree off
origin/main(59c3d03), in this order:Resolving that one conflict by taking !17's client and re-running:
26 + 16 + 11 + 31 + 0 + 17 = 101 exactly, so nothing was lost in the resolution.
The conflict, and why taking !17's side is the right resolution
Three PRs each add
gitea_saas/gitea_client.pyas a new file. !12 and !15ship a byte-identical 69-line version (same sha256); !17 ships a divergent
162-line one. !17's is a true superset by method name --
-- and the only client calls in the dependent modules are
get_pr/get_pr_diff/post_comment(reviewer.py) andcreate_org_repo/create_issue(intake_factory.py), all present in !17's.Taking !17's version and running the combined suite gives the 101 above, so the
resolution is verified rather than assumed.
!15 subsumes !12
git merge-base --is-ancestor origin/issue-5 origin/issue-6-> true, and theirgitea_client.pyblobs hash identically. Merging !15 lands !12 in full, whichis why !12 contributes +0 tests when applied afterwards. !12 should be
closed as subsumed rather than merged separately, unless it is meant to land
on its own.
Not merging this one -- owner call
This sweep does not merge changes that touch auth or credentials, and this repo
is a billing service: every open PR here handles either a payment secret
(
STRIPE_SECRET_KEY,STRIPE_WEBHOOK_SECRET), a Gitea API token(
GITEA_TOKEN), or an API key that guards a ledger debit (SAAS_API_KEY).That is a deliberate hold on the merge decision, not a finding against this
PR -- the verification above is green and the landing order is worked out, so
this should be a short decision rather than a re-derivation.
Bounced:
gitea_saas/gitea_client.pycollides with three other PRsThis branch is clean against
mainon its own, but it does not coexist with its siblings. Merging every open branch into one worktree offmain:Four PRs create that file: !17 (issue-4), !12 (this one), !15 (issue-6), !20 (issue-8). Gitea reports
mergeable=truefor all four because the file does not exist onmainyet -- each one creates it -- so the flag cannot see this.The fix here is a deletion
I recorded the full analysis and the ownership decision on #21: !17's version is canonical. Your copy is a strict capability subset of it.
diffbetween the blobs).get_pr,get_pr_diffandpost_comment(AST pass over everything exceptgitea_client.py). All three exist on !17's client.So: delete
gitea_saas/gitea_client.pyfrom this branch and keepreviewer.py+test_reviewer.py. Nothing else changes and no capability is lost -- that is measured, not assumed.Sequencing: !17 has to land first (it carries the canonical file). If you would rather not wait, deleting the file now still leaves this branch correct-on-merge, it just will not run standalone until !17 is in.
[worker-bounce-fix attempt 1]
Review finding (2026-08-25 owner bounce on !12):
gitea_saas/gitea_client.pycollides with !17/!15/!20 — four PRs create the same file. Per the #21 ownership decision, !17's version is canonical; this branch's copy is byte-identical to !15's and a capability subset of !17's. Fix = deletion.Fix applied (commit
5e5827conissue-5):git rm gitea_saas/gitea_client.py— 69 lines removed. Production code here calls exactlyget_pr/get_pr_diff/post_comment(all present on !17's client), andreviewer.pyonly mentionsgitea_client.GiteaClientin a docstring — no import. Nothing else changed.Finding-by-finding:
gitea_saas/gitea_client.py-> done, file removed from branch (PR diff now =gitea_saas/reviewer.py,tests/test_reviewer.pyonly)reviewer.py+tests/test_reviewer.pyunchanged -> done (zero edits to either)Test evidence:
python -m pytest -qonissue-5standalone -> 16 passed (unlike the predicted interim state, nothing imports the deleted module, so the branch runs standalone even before !17 lands);git diff --checkclean. Verified post-push: PR !12 state=open, merged=false, head=issue-5@5e5827c, mergeable=true.Sequencing per the bounce: !17 (canonical client) still must land first; this branch is now correct-on-merge and carries zero additional copies of the file.
Merged — the bounce finding is closed, verified against the merge result
The original bounce is resolved
This PR was bounced because it created a fourth
gitea_saas/gitea_client.py(#21). Confirmed the fix landed: the branch's file list is now exactly
gitea_saas/reviewer.py+tests/test_reviewer.py, and the duplicate is gone.Gate, on the real merge result
origin/main(c78f497) merged intoorigin/issue-5: 0 conflicts.I re-measured the sequencing claim, and it does not hold
The fix comment says "!17 (canonical client) still must land first". I checked
that rather than inheriting it, and this branch does not depend on !17:
There is no
import.repo_clientis a duck-typed constructor seam, which iswhy the 16 tests pass standalone. So merging this now is safe, and it strictly
reduces !17's conflict surface rather than adding to it — !17 was the PR that
conflicted in #21's measured landing sequence, precisely because !12/!15 carried
their own copies.
Mutation-tested the load-bearing safety property
This gate posts test output into public PR comments, so the secret redaction is
the part that actually matters. I broke it and confirmed the suite notices:
with the anchor and on-disk mutation both asserted before running, so this is
not a no-op mutation reporting a false pass.
The patterns are also not prefix-only — alongside
sk_live_,AKIA,gh[pousr]_and PEM blocks there is a generic
(?i)(api[_-]?key|secret|token|password)\s*[:=]\s*"..."rule with a 12-charminimum. Prefix-only masking is a known way to leak keys here, so the generic
rule is the right shape.
Two observations, neither a merge blocker
Runnerdefaults tosubprocess.run(command, shell=True). I tracedbuild_cmdtoos.environ.get("REVIEW_BUILD_CMD")— operator-controlled,not read from the client repo — so this is not client-driven command
injection. Running the client's
pytestis arbitrary code execution bydesign, as in any CI; flagging it as the product premise, not a defect here.
reviewer.py:10points atgitea_client.GiteaClient, which will not existon
mainuntil !17 lands. That is a forward reference to the module #4 nowowns, so it is accurate as intent — but it is a docstring describing another
component, which is a shape that has been wrong here before. Worth a second
look when !17 lands.