Skip to content

πŸ› review: only a security finding parks an untrusted PR during CI/Review - #15641

Merged
clubanderson merged 4 commits into
v5from
fix/no-decision-in-ci-review
Oct 30, 2026
Merged

clubanderson merged 4 commits into
v5from
fix/no-decision-in-ci-review

Conversation

@clubanderson

@clubanderson clubanderson commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Fixes #15640

Owner rules (verbatim)

there should be no decisions by a human while these are in ci/review β€” unless the review found something.
unless you find something fishy and insecure happening at the pr in progress level, then we can let things go and then decide in merge queue, if needed.
we dont want to create a log jam during pr in progress.
i always have the last word when it comes to merge of untrusted.

Root cause

Not a missing verdict: the review swarm ran. The reviewer's decision_requested marker forces every perspective to requires_human and aggregateReports honoured it blindly for non-hive-authored PRs (hive-authored PRs already pass the DecisionRiskClasses denylist). Nothing checked whether the question was a security finding, and nothing carried what was found into the verdict record, the 085 comment, or the tracker β€” so a decision was owed with no visible reason, and the PR had no mover.

Fix

  • One predicate β€” review.IsSecuritySensitiveFinding (pkg/review/security_finding.go) is the only early exit from hands-free CI/Review for a PR the hive did not author. Triggers: the reviewer's <!-- hive-decision-risk: high --> marker; the security perspective withholding approval; a decision question in a security-class risk class (security, permissions-credentials, sensitive-paths: workflows, CODEOWNERS, deploy, Dockerfiles, dependency manifests, …); an in-scope finding on a sensitive path (sentinel.DefaultSensitivePaths + new security.sensitive_paths globs); an in-scope finding or summary flagging insecure/suspicious behaviour (injection, exfiltration, secrets, unpinned actions, supply chain, …). Lane-side equivalents (sentinel-blocked, workflow-paths, tests-removed, reporter-trust hold) already park the same way.
  • Applied to both the decision-marker path and bare requires_human on contributor PRs. Backed β†’ 085 with Aggregate.DecisionFinding set. Not backed β†’ ordinary findings route to the clanker as changes_requested (fix-cycle cap still ends at 085 with "fix cycle cap reached"); a bare question is advisory β†’ approve, reason no security finding β€” the PR keeps moving; a person has the last word at the merge gate; recorded as advisory: …. self_authored_decision_policy: escalate restores park-on-every-marker.
  • DecisionFinding plumbed prstage.ReviewVerdict β†’ cmd/hive/merge_eligibility.go β†’ stagelabels.Evidence β†’ 085 comment (the review found: …), the reviewer verdict route log (decision_finding), and the tracker decision popover blocker β€” the operator is told WHAT was found.
  • Untrusted merge invariant β€” unchanged hooks, now pinned: contributor-lane:claim-tier-untrusted / untrusted-author-role refuse the merge and the stage engine parks at 090-ready-for-human-merge for the operator's merge-now / queue-merge. The hive never auto-merges an untrusted author's PR, green + approved + soaked or not.
  • No log jam β€” sweep test over every contributor lane reason Γ— verdict shape asserts no CI/Review stage is human-owned and 085 is reached only through a security lane reason, a verdict naming its finding, or an operator-fault reason (no reviewer configured / App login unknown / files API refused). Tracker test asserts every CI/Review row names a non-maintainer mover (review swarm / CI / hive / author).

Write-back gate audit

The 085 label was withheld because the stage_label write-back kind requires the hive-review opt-in at L5 for non-hive-authored items (pkg/writeback/kinds.go). That gate is correct and unchanged. The hive did post the "Needs a decision" comment on each PR; the issue label stayed at 040 and the tracker read the suppressed-label record as "PR in progress" β€” #15593 (Record.LabelsSuppressed, not yet rolled) fixes that display. Only the stage computation and the missing finding text are fixed here.

What the 12 example PRs do post-roll

The aggregate is recomputed every tick from the retained review-report files, so records recover without manual action. Against the real 55 reports from the hosted pod:

PR Reviewer question Post-roll
hive#15312 (#15273) duplicate of #15309 un-parks: advisory approve β†’ merge gate; author below lane trust β†’ 090 awaiting operator merge (or close)
hive#15324 (#15320) already landed via #15301 same
hive#15343 (#15338) byte-identical to #15341 same
hive#15410 (#15340) trust surface, reviewer hive-decision-risk: high stays 085 β€” the review found: the reviewer marked this change high-risk
hive#15433 (#15432) same same
hive#15482 (#15437) same same
hive#15487 (#15438) same same
hive#15489 (#15439) same same
docs#514 same same
docs#518 sensitive-paths question stays 085 β€” the review found: sensitive-paths question from the … review
crux#53 aggregate approve; lane contributor-lane:sentinel-blocked unchanged: parked by the sentinel alert (a security finding); operator clears the label / merges at the gate

The three duplicates now flow to the merge queue where the operator decides (merge or close); the trust-surface eight stay a decision but say why. No contributor PR with nothing security-relevant behind its question parks during CI/Review again.

Tests

  • pkg/review: TestAggregateReportsContributorDecisionNeedsBacking (12 subcases incl. security.sensitive_paths), …ContributorRequiresHumanNeedsSecurityFinding, …EscalatePolicyKeepsQuestion, …UnbackedDecisionWithChangesRoutesToFixer.
  • pkg/stageengine: TestClassifyStageContributorLaneNoDecisionDuringCIReview (a/b/c), TestContributorLaneCIReviewNeverParksOnAHuman (sweep).
  • pkg/github/automerge: TestContributorLaneUntrustedGreenApprovedParksForHumanMerge.
  • pkg/stagelabels: 085 comment prints the review found: …. pkg/dashboard: TestStageTrackerCIReviewCaptionAlwaysNamesAMover.
  • go vet + go test -count=1 on pkg/review pkg/config pkg/stageengine/... pkg/stagelabels pkg/github/automerge pkg/dashboard cmd/hive: all pass (config golden regenerated for the new security block; settings-parity allowlist entry added).

Owner rule: no decisions by a human while a PR is in CI/Review unless the
review found something. The aggregate honoured a reviewer decision marker
blindly on non-hive-authored PRs; it now parks only when the question is
backed (duplicate, risk class, high-risk marker, finding at threshold,
risk-escalation term) and records the finding as decision_finding, which
the 085 comment, verdict-route log and tracker print. Unbacked questions
are advisory and the PR moves on to the merge gate's human-merge rule.

Fixes #15640

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 30, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Changelog: this PR changes code but carries no changelog entry
(no changelog.d/ fragment and no CHANGELOG.md edit).

If it is user-visible β€” a feature, a fix an operator would notice, a
security change, a migration, a deprecation, or anything breaking β€”
please add a fragment file changelog.d/<category>-<pr-or-slug>.md
containing exactly your entry (see changelog.d/README.md; the
category is one of added/changed/deprecated/fixed/security). Please
do not append to CHANGELOG.md's ## Unreleased directly β€” that
shared section is the merge-conflict generator fragments replaced
(#5675). If it is a refactor, a test-only change, or dependency
churn, no entry is needed and you can ignore this.

This is a reminder, not a gate; it never blocks a merge.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#15664 πŸ› tracker: a PR waiting on its author never leads with Close as superseded same files src/pkg/dashboard/stage_tracker_decision.go
#15633 🌱 refactor: split stage tracker UI tests same files changelog.d/fixed-contributor-decision-needs-finding.md
src/cmd/hive/main.go
src/cmd/hive/merge_eligibility.go
src/docs/hive-merge.md
src/docs/review-swarm.md
src/pkg/config/config.go
src/pkg/config/security.go
src/pkg/config/testdata/agent_runtime_golden.json
src/pkg/dashboard/settings_parity_allowlist.txt
src/pkg/dashboard/stage_tracker_ci_review_mover_test.go
src/pkg/dashboard/stage_tracker_decision.go
src/pkg/github/automerge/automerge_contributor_lane_test.go
src/pkg/github/automerge/automerge_stage_human.go
src/pkg/github/automerge/automerge_stage_labels.go
src/pkg/github/review_verdict_relay_test.go
src/pkg/review/plan_match_test.go
src/pkg/review/review.go
src/pkg/review/review_test.go
src/pkg/review/security_finding.go
src/pkg/stageengine/derive_pr_no_decision_in_ci_test.go
src/pkg/stageengine/prstage/types.go
src/pkg/stagelabels/human.go
src/pkg/stagelabels/human_test.go
#15582 πŸ› fix: warn once per PR on review dispatch promotion same files src/cmd/hive/main.go
#15308 ✨ feature: enqueue eligible PRs into GitHub native merge queues same files src/pkg/github/automerge/automerge_contributor_lane_test.go
#15673 πŸ› issues the hive files carry the filing agent in the title same files src/docs/review-swarm.md
#15671 πŸ› review: the hive closes its own already-landed PRs instead of asking a person same files src/docs/hive-merge.md
src/pkg/github/automerge/automerge_stage_human.go
src/pkg/stagelabels/human.go
src/pkg/stagelabels/human_test.go
#15617 πŸ› review: split the grounding budget across fan-out sub-agents same files changelog.d/fixed-contributor-decision-needs-finding.md
src/cmd/hive/main.go
src/cmd/hive/merge_eligibility.go
src/docs/hive-merge.md
src/docs/review-swarm.md
src/pkg/config/config.go
src/pkg/config/security.go
src/pkg/config/testdata/agent_runtime_golden.json
src/pkg/dashboard/settings_parity_allowlist.txt
src/pkg/dashboard/stage_tracker_ci_review_mover_test.go
src/pkg/dashboard/stage_tracker_decision.go
src/pkg/github/automerge/automerge_contributor_lane_test.go
src/pkg/github/automerge/automerge_stage_human.go
src/pkg/github/automerge/automerge_stage_labels.go
src/pkg/github/review_verdict_relay_test.go
src/pkg/review/plan_match_test.go
src/pkg/review/review.go
src/pkg/review/review_test.go
src/pkg/review/security_finding.go
src/pkg/stageengine/derive_pr_no_decision_in_ci_test.go
src/pkg/stageengine/prstage/types.go
src/pkg/stagelabels/human.go
src/pkg/stagelabels/human_test.go
#15680 πŸ› review: one backlog issue per finding; carried-over notes never refile same files src/docs/review-swarm.md
#15668 πŸ› Lead 090 merge-gate action is Approve and let hive merge same files src/docs/hive-merge.md
src/pkg/dashboard/stage_tracker_decision.go

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • test (rest 3/4)

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conclusion: the aggregate logic reads correctly to me, but two things in this PR make CI red and both are this diff's own doing. Fix those and this looks ready.

1. pkg/github test still asserts the old behaviour β€” test (rest 3/4) fails.
src/pkg/github/review_verdict_relay_test.go:134-162 (TestRecordReviewVerdictPromotesDecisionMarker) feeds a non-hive-authored (HiveAuthoredKnown:true, HiveAuthored:false) verdict with the question "@clubanderson should decide whether this groundwork lands before its consumer." and then requires Collect to return requires_human + DecisionRequested. That question is exactly the one review_test.go now classifies as unbacked ("bare product question keeps moving"), so the new decisionQuestionsBacked branch in src/pkg/review/review.go:489-520 resolves it to approve/MergeEligible with the advisory reason β€” which is what the CI log shows: aggregate = {... Verdict:approve MergeEligible:true ... DecisionRequested:false ...}, want decision request. The first half of the test (recorded[0].DecisionRequested, the relay-side promotion) still holds; only the Collect assertion is stale. Either switch the question to a backed one (as you did for TestAggregateReportsCarriesDecisionQuestion) or assert the advisory outcome. Severity: high (red required check).

2. Changelog fragment is under src/changelog.d/, so the guard fails and it would never reach CHANGELOG.md.
src/changelog.d/fixed-contributor-decision-needs-finding.md β€” src/scripts/check-changelog-fragment.sh:44-51 rejects this location explicitly ("compile-changelog.sh only reads the repo-root changelog.d/") and the job output says: misplaced fragment β€” not autofixing. Run: git mv src/changelog.d/fixed-contributor-decision-needs-finding.md changelog.d/fixed-contributor-decision-needs-finding.md. Severity: medium.

What I verified as correct: the gate is only taken for !agg.HiveAuthored and when SelfAuthoredDecisionPolicy != "escalate", so hive-authored PRs keep the existing resolveSelfAuthoredDecisionQuestions path and the escalate switch restores park-on-marker (covered by TestAggregateReportsContributorDecisionEscalatePolicyKeepsQuestion). The fix-cap branch inside the new block mirrors the existing one below it. DecisionFinding is threaded end-to-end: review.Aggregate β†’ contributorLaneVerdictFromArtifact (cmd/hive/merge_eligibility.go:237) β†’ prstage.ReviewVerdict β†’ humanEvidence (automerge_stage_human.go:224) β†’ stagelabels.Evidence/RouteHuman (human.go:359) and the tracker blocker (stage_tracker_decision.go:838). src/docs/review-swarm.md and hive-merge.md describe the new rule and match the code.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Check report (hive scanner, comment-only β€” human-authored branch). Two PR-local failures:

  1. changelog-fragment-guard β€” the fragment is under src/changelog.d/; fragments live at the repo root:

    git mv src/changelog.d/fixed-contributor-decision-needs-finding.md changelog.d/fixed-contributor-decision-needs-finding.md
    
  2. test (rest 3/4) β€” pkg/github TestRecordReviewVerdictPromotesDecisionMarker (review_verdict_relay_test.go:161). The test's fixture is a bare decision question on a contributor PR with no finding, no risk-class wording and no hive-decision-risk: high marker β€” exactly the case this PR now downgrades to advisory, so the aggregate comes back approve with Reasons:[decision question not backed by a finding β€” the PR keeps moving; …] and the test's want decision request no longer holds.

    Fix in the test (it exercises marker promotion, not the finding gate): add the explicit high-risk marker to the body so the question still parks:

    body := "Looks correct, but this is a maintainer decision.\n\n<!-- hive-verdict: needs-decision -->\n<!-- hive-decision-risk: high -->\n<!-- hive-decision-question: @clubanderson should decide whether this groundwork lands before its consumer. -->"

    (or, if the intent is to pin the new behaviour, flip the assertion to expect VerdictApprove / DecisionRequested=false and a DecisionFinding == "").


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: ed091b4d8

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Owner refinements (#15640): the only early exit from hands-free CI/Review
for a contributor PR is something fishy or insecure β€” one predicate,
review.IsSecuritySensitiveFinding (reviewer high-risk marker, security
perspective hold, security-class question, finding on a sentinel or
security.sensitive_paths path, insecure behaviour flagged). Duplicates,
nits, ordinary findings go back to the clanker or are advisory. The
operator has the last word on untrusted merges: a green, approved, soaked
untrusted PR parks at 090, never auto-merges (test). Sweep tests assert no
CI/Review stage is human-owned and every tracker CI/Review row names its
mover.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 30, 2026
@clubanderson clubanderson changed the title πŸ› review: contributor decision questions park at 085 only on a finding πŸ› review: only a security finding parks an untrusted PR during CI/Review Oct 30, 2026
Signed-off-by: Andrew Anderson <andy@clubanderson.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conclusion: the mechanism is sound, but the code backs a decision question on a narrower set than the PR body says it does, and the fragment is in the wrong directory (that is the red changelog-fragment-guard). Two concrete items; the rest reads correctly to me.

1. Body vs. diff β€” duplicates and non-security findings do not "back" a question.
The body lists five backings: hive-redundant-of / duplicate wording, <!-- hive-decision-risk: high -->, a risk-class question, an in-scope finding at the requires_human threshold, or a risk-escalation term. The gate actually consulted is securityFindingAcross β†’ IsSecuritySensitiveFinding (src/pkg/review/security_finding.go), which accepts only: DecisionRiskHigh, security perspective at requires_human, a question whose risk class is in securitySensitiveClasses, an in-scope finding on a sensitive path, or a securityFindingTerms match. Neither RedundantOf nor a threshold-severity non-security finding appears anywhere in it, and review.go:495 / :559 call nothing else.

Consequence, traced through contributorHandsFreeAggregate: a contributor PR whose reviewer wrote <!-- hive-redundant-of: #N --> with no findings becomes approve + MergeEligible (duplicate proceeds to the merge gate); with an info "duplicate of #N" finding it becomes changes_requested + FixCycle β€” a fixer is handed a no-op merge it cannot fix, and it only reaches a person after maxFix attempts. Either the body's list should shrink to "security-sensitive", or IsSecuritySensitiveFinding/securityFindingAcross should also treat r.RedundantOf != 0 and threshold-severity in-scope findings as backing. Which one is your call β€” the owner rule quoted in the body ("unless the review found something") reads like the broader set.

2. Changelog fragment is under src/. src/changelog.d/fixed-contributor-decision-needs-finding.md β€” check-changelog-fragment.sh:44-51 rejects this because compile-changelog.sh only reads repo-root changelog.d/; the fix is git mv to changelog.d/. (#15645 has it in the right place.)

Verified as correct: markRequiresHumanPerspectivesAsApprove only runs on the no-fix path; contributorHandsFree keeps hive-authored PRs and self_authored_decision_policy: escalate on the old behaviour; SecuritySensitivePaths always includes sentinel.DefaultSensitivePaths so an empty operator list cannot disable the path check.

Test checks are re-running on the new head; only the fragment guard is red right now.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor
1b8b470

ℹ️ Sentinel notice β€” informational; author is trusted (owner), merge is not blocked

Hive flagged this PR (author @clubanderson, head 1b8b470368c7) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation β€” a maintainer should confirm the change is intended before it merges.

  • remote_code_execution β€” adds a remote-fetch-and-execute, decoded payload, reverse shell or embedded credential (Adds curl|sh, eval of encoded payloads, reverse shells or embedded private keys)
    • src/pkg/review/security_finding.go

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the review swarm requested changes

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

stage 080-awaiting-merge-admission β†’ 071-needs-author: reviewer-verdict:changes_requested

A bare correctness requires_human or product decision marker on a
contributor PR aggregates to advisory approve; the security perspective's
hold and a sensitive-paths question survive with DecisionFinding set.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Contributor-lane PRs park at 085 needs-decision during CI/Review with no finding surfaced

1 participant