Repository navigation
π review: only a security finding parks an untrusted PR during CI/Review - #15641
Conversation
Owner rule: no decisions by a human while a PR is in CI/Review unless the review found something. The aggregate honoured a reviewer decision marker blindly on non-hive-authored PRs; it now parks only when the question is backed (duplicate, risk class, high-risk marker, finding at threshold, risk-escalation term) and records the finding as decision_finding, which the 085 comment, verdict-route log and tracker print. Unbacked questions are advisory and the PR moves on to the merge gate's human-merge rule. Fixes #15640 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Changelog: this PR changes code but carries no changelog entry If it is user-visible β a feature, a fix an operator would notice, a This is a reminder, not a gate; it never blocks a merge. |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
@clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
There was a problem hiding this comment.
Conclusion: the aggregate logic reads correctly to me, but two things in this PR make CI red and both are this diff's own doing. Fix those and this looks ready.
1. pkg/github test still asserts the old behaviour β test (rest 3/4) fails.
src/pkg/github/review_verdict_relay_test.go:134-162 (TestRecordReviewVerdictPromotesDecisionMarker) feeds a non-hive-authored (HiveAuthoredKnown:true, HiveAuthored:false) verdict with the question "@clubanderson should decide whether this groundwork lands before its consumer." and then requires Collect to return requires_human + DecisionRequested. That question is exactly the one review_test.go now classifies as unbacked ("bare product question keeps moving"), so the new decisionQuestionsBacked branch in src/pkg/review/review.go:489-520 resolves it to approve/MergeEligible with the advisory reason β which is what the CI log shows: aggregate = {... Verdict:approve MergeEligible:true ... DecisionRequested:false ...}, want decision request. The first half of the test (recorded[0].DecisionRequested, the relay-side promotion) still holds; only the Collect assertion is stale. Either switch the question to a backed one (as you did for TestAggregateReportsCarriesDecisionQuestion) or assert the advisory outcome. Severity: high (red required check).
2. Changelog fragment is under src/changelog.d/, so the guard fails and it would never reach CHANGELOG.md.
src/changelog.d/fixed-contributor-decision-needs-finding.md β src/scripts/check-changelog-fragment.sh:44-51 rejects this location explicitly ("compile-changelog.sh only reads the repo-root changelog.d/") and the job output says: misplaced fragment β not autofixing. Run: git mv src/changelog.d/fixed-contributor-decision-needs-finding.md changelog.d/fixed-contributor-decision-needs-finding.md. Severity: medium.
What I verified as correct: the gate is only taken for !agg.HiveAuthored and when SelfAuthoredDecisionPolicy != "escalate", so hive-authored PRs keep the existing resolveSelfAuthoredDecisionQuestions path and the escalate switch restores park-on-marker (covered by TestAggregateReportsContributorDecisionEscalatePolicyKeepsQuestion). The fix-cap branch inside the new block mirrors the existing one below it. DecisionFinding is threaded end-to-end: review.Aggregate β contributorLaneVerdictFromArtifact (cmd/hive/merge_eligibility.go:237) β prstage.ReviewVerdict β humanEvidence (automerge_stage_human.go:224) β stagelabels.Evidence/RouteHuman (human.go:359) and the tracker blocker (stage_tracker_decision.go:838). src/docs/review-swarm.md and hive-merge.md describe the new rule and match the code.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Check report (hive scanner, comment-only β human-authored branch). Two PR-local failures:
π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
Owner refinements (#15640): the only early exit from hands-free CI/Review for a contributor PR is something fishy or insecure β one predicate, review.IsSecuritySensitiveFinding (reviewer high-risk marker, security perspective hold, security-class question, finding on a sentinel or security.sensitive_paths path, insecure behaviour flagged). Duplicates, nits, ordinary findings go back to the clanker or are advisory. The operator has the last word on untrusted merges: a green, approved, soaked untrusted PR parks at 090, never auto-merges (test). Sweep tests assert no CI/Review stage is human-owned and every tracker CI/Review row names its mover. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
There was a problem hiding this comment.
Conclusion: the mechanism is sound, but the code backs a decision question on a narrower set than the PR body says it does, and the fragment is in the wrong directory (that is the red changelog-fragment-guard). Two concrete items; the rest reads correctly to me.
1. Body vs. diff β duplicates and non-security findings do not "back" a question.
The body lists five backings: hive-redundant-of / duplicate wording, <!-- hive-decision-risk: high -->, a risk-class question, an in-scope finding at the requires_human threshold, or a risk-escalation term. The gate actually consulted is securityFindingAcross β IsSecuritySensitiveFinding (src/pkg/review/security_finding.go), which accepts only: DecisionRiskHigh, security perspective at requires_human, a question whose risk class is in securitySensitiveClasses, an in-scope finding on a sensitive path, or a securityFindingTerms match. Neither RedundantOf nor a threshold-severity non-security finding appears anywhere in it, and review.go:495 / :559 call nothing else.
Consequence, traced through contributorHandsFreeAggregate: a contributor PR whose reviewer wrote <!-- hive-redundant-of: #N --> with no findings becomes approve + MergeEligible (duplicate proceeds to the merge gate); with an info "duplicate of #N" finding it becomes changes_requested + FixCycle β a fixer is handed a no-op merge it cannot fix, and it only reaches a person after maxFix attempts. Either the body's list should shrink to "security-sensitive", or IsSecuritySensitiveFinding/securityFindingAcross should also treat r.RedundantOf != 0 and threshold-severity in-scope findings as backing. Which one is your call β the owner rule quoted in the body ("unless the review found something") reads like the broader set.
2. Changelog fragment is under src/. src/changelog.d/fixed-contributor-decision-needs-finding.md β check-changelog-fragment.sh:44-51 rejects this because compile-changelog.sh only reads repo-root changelog.d/; the fix is git mv to changelog.d/. (#15645 has it in the right place.)
Verified as correct: markRequiresHumanPerspectivesAsApprove only runs on the no-fix path; contributorHandsFree keeps hive-authored PRs and self_authored_decision_policy: escalate on the old behaviour; SecuritySensitivePaths always includes sentinel.DefaultSensitivePaths so an empty operator list cannot disable the path check.
Test checks are re-running on the new head; only the fragment guard is red right now.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
1b8b470
βΉοΈ Sentinel notice β informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
|
@clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
stage 080-awaiting-merge-admission β 071-needs-author: reviewer-verdict:changes_requested |
A bare correctness requires_human or product decision marker on a contributor PR aggregates to advisory approve; the security perspective's hold and a sensitive-paths question survive with DecisionFinding set. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Fixes #15640
Owner rules (verbatim)
Root cause
Not a missing verdict: the review swarm ran. The reviewer's
decision_requestedmarker forces every perspective torequires_humanandaggregateReportshonoured it blindly for non-hive-authored PRs (hive-authored PRs already pass theDecisionRiskClassesdenylist). Nothing checked whether the question was a security finding, and nothing carried what was found into the verdict record, the 085 comment, or the tracker β so a decision was owed with no visible reason, and the PR had no mover.Fix
review.IsSecuritySensitiveFinding(pkg/review/security_finding.go) is the only early exit from hands-free CI/Review for a PR the hive did not author. Triggers: the reviewer's<!-- hive-decision-risk: high -->marker; the security perspective withholding approval; a decision question in a security-class risk class (security,permissions-credentials,sensitive-paths: workflows, CODEOWNERS, deploy, Dockerfiles, dependency manifests, β¦); an in-scope finding on a sensitive path (sentinel.DefaultSensitivePaths+ newsecurity.sensitive_pathsglobs); an in-scope finding or summary flagging insecure/suspicious behaviour (injection, exfiltration, secrets, unpinned actions, supply chain, β¦). Lane-side equivalents (sentinel-blocked,workflow-paths,tests-removed, reporter-trust hold) already park the same way.requires_humanon contributor PRs. Backed β 085 withAggregate.DecisionFindingset. Not backed β ordinary findings route to the clanker aschanges_requested(fix-cycle cap still ends at 085 with "fix cycle cap reached"); a bare question is advisory βapprove, reasonno security finding β the PR keeps moving; a person has the last word at the merge gate; recorded as advisory: β¦.self_authored_decision_policy: escalaterestores park-on-every-marker.DecisionFindingplumbedprstage.ReviewVerdictβcmd/hive/merge_eligibility.goβstagelabels.Evidenceβ 085 comment (the review found: β¦), thereviewer verdict routelog (decision_finding), and the tracker decision popover blocker β the operator is told WHAT was found.contributor-lane:claim-tier-untrusted/untrusted-author-rolerefuse the merge and the stage engine parks at090-ready-for-human-mergefor the operator's merge-now / queue-merge. The hive never auto-merges an untrusted author's PR, green + approved + soaked or not.Write-back gate audit
The 085 label was withheld because the
stage_labelwrite-back kind requires thehive-reviewopt-in at L5 for non-hive-authored items (pkg/writeback/kinds.go). That gate is correct and unchanged. The hive did post the "Needs a decision" comment on each PR; the issue label stayed at 040 and the tracker read the suppressed-label record as "PR in progress" β #15593 (Record.LabelsSuppressed, not yet rolled) fixes that display. Only the stage computation and the missing finding text are fixed here.What the 12 example PRs do post-roll
The aggregate is recomputed every tick from the retained review-report files, so records recover without manual action. Against the real 55 reports from the hosted pod:
hive-decision-risk: highthe review found: the reviewer marked this change high-risksensitive-pathsquestionthe review found: sensitive-paths question from the β¦ reviewapprove; lanecontributor-lane:sentinel-blockedThe three duplicates now flow to the merge queue where the operator decides (merge or close); the trust-surface eight stay a decision but say why. No contributor PR with nothing security-relevant behind its question parks during CI/Review again.
Tests
pkg/review:TestAggregateReportsContributorDecisionNeedsBacking(12 subcases incl.security.sensitive_paths),β¦ContributorRequiresHumanNeedsSecurityFinding,β¦EscalatePolicyKeepsQuestion,β¦UnbackedDecisionWithChangesRoutesToFixer.pkg/stageengine:TestClassifyStageContributorLaneNoDecisionDuringCIReview(a/b/c),TestContributorLaneCIReviewNeverParksOnAHuman(sweep).pkg/github/automerge:TestContributorLaneUntrustedGreenApprovedParksForHumanMerge.pkg/stagelabels: 085 comment printsthe review found: β¦.pkg/dashboard:TestStageTrackerCIReviewCaptionAlwaysNamesAMover.go vet+go test -count=1onpkg/review pkg/config pkg/stageengine/... pkg/stagelabels pkg/github/automerge pkg/dashboard cmd/hive: all pass (config golden regenerated for the newsecurityblock; settings-parity allowlist entry added).