Repository navigation
✨ Auto-decide reviewer scope questions on hive-authored green PRs - #16598
Conversation
A review perspective that ends with a question rather than a finding parked hive-authored, green, finding-free PRs at 085 on wording alone (docs#618, hive#16456). Classify the ask: policy sign-off stays a person's; a scope / follow-up question runs the existing Let-hive-decide pipeline once per head (approve -> merge queue + follow-up issue; direction -> fixer; owner-only -> park with the recommendation). Config governor.review.auto_decide_questions (default on); review_questions_autodecided_1h on /api/status. Fixes #16597 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. @clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. Review findings from Read 1. Fail-open when the verdict aggregate is unavailable (medium). 2. Open question, not a finding: CI currently shows |
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
There was a problem hiding this comment.
Read review_question_autodecide.go, decision_question_class.go, the candidate filter's inputs in stage_tracker_build.go, and EnrichCIStatus in pkg/github/client.go at this head. The classifier and once-per-head ledger are sound; the CI-green precondition is not actually enforced, which matters because an approve verdict admits the PR to the merge queue and files a follow-up issue.
Medium — "CI green" is asserted, not checked (src/pkg/dashboard/review_question_autodecide.go:333, correctness)
autoDecideReviewQuestionhardcodesCIGreen: trueand relies onreviewQuestionCandidateshaving excluded red/running CI viait.ciRed || it.ciChecking(line 225).it.ciCheckingispending && pr.CIChecksRunning(stage_tracker_build.go:382). ButCIChecksRunningis assigned in exactly one place,client.go:2737, inside thehasFailbranch — i.e. only whenCIStatusis alreadyfailure, which makesciRedtrue andciCheckingfalse. In the plain pending case (client.go:2742, checks still running, none failed) and the no-check-runs case (client.go:2727)CIChecksRunningstays false, sociCheckingis false and the row passes the filter.- Net effect: a hive-authored PR whose checks are still running, or has no check runs yet, is classified with
CIGreen: true, consumes one of the 3 paid decisions per sweep, is marked once-per-head (so it is not revisited when CI actually settles on that head), and on approve gets a follow-up issue filed. The comment onreviewQuestionCandidates("CI neither red nor still running"), theCIGreenfield doc, and the config doc ("own green, finding-free PR") all promise otherwise. - Suggested fix: derive
CIGreenfrom the PR'sCIStatus == "success"(orCIRealCheckRuns > 0 && allDone) at the candidate stage instead of inferring it fromciChecking, and add a test row withCIStatus: pending, CIChecksRunning: falsethat must be excluded. (TheciCheckingdead-flag is pre-existing; the new feature is the first thing that depends on its negative.)
Low — marker written before the job is known to start (review_question_autodecide.go:354-363)
reviewQuestionAuto.mark runs before delegatedJobs.start; when start returns !ok (a delegate already running for the key) the function returns without deciding, but the head is now seen and is never retried. Moving mark after a successful start keeps the once-per-head guarantee without burning the head on a race.
For awareness, not a defect: .github/workflows/** is only a sign-off path when FromFork is true (decision_question_class.go:115). A hive-branch PR that edits a workflow with a scope question auto-decides. That is deliberate per the comments; stating it so the owner confirms it is intended.
— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
CI triage for Failure: Cause: the fixture path Fix (test only): use a YAML path that is not in (Any Not pushing to this branch since it is human-authored. 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 5db2829. |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Fixes #16597
Why
docs#618 (hive-authored, CI 4/4, Review 5/5, no findings) parked at 085 "Needs human decision" because one perspective ended with a question — "@clubanderson should decide whether to apply the hive-docs-refresh.yml opt-in … and whether this PR should merge ahead of that workflow edit" — and the word workflow tripped the self-authored risk class. Same shape as hive#16456. Owner: "this is auto-merge material, not needs-human".
What
pkg/review/decision_question_class.go— pureClassifyDecisionQuestion: policy sign-off (workflow/action edits from a fork, OWNERS / CODEOWNERS / SECURITY / rulesets / secrets / key material,auto_merge/human_merge_paths/self_authored_decision_policyconfig edits, owner-only guard wording: sentinel, GitHub/App permissions, unauthorized, tokens/secrets/credentials, branch protection) vs scope question (hive-authored, CI green, no unresolved finding) vs ineligible. Table test with 17 cases including both real asks verbatim.pkg/dashboard/review_question_autodecide.go— paced sweep fromUpdateStatusIfFresh: picks 085 PR rows a reviewer's question parked (not on hold, CI not red / running), reads fork + files from GitHub, classifies, and for a scope question runs the existingrunDelegatedDecisionpipeline with the hive as requester (pendinglet-hive-decideledger row, same job registry as the button). Approve → the pipeline's/lgtm/approve+ afollow-up: <question>issue (writeback-gated); direction → fixer; owner-only / recommend-only → park stays and the row already shows "Recommendation recorded" / "needs your sign-off" (🐛 Activity Monitor: say whose sign-off a parked review needs and lead with Approve #16578). Once per head via/data/review-question-autodecide.json; one INFO linereviewer question auto-decidedwith class + reason;review_questions_autodecided_1hon/api/status.governor.review.auto_decide_questions(defaulttrue), documented indocs/operator-reference/sweeps-and-operations.mdwith the two classes; golden updated.Tests
pkg/review:TestClassifyDecisionQuestion(17 cases),TestClassifyDecisionQuestionRealAsksMatchRiskClassByWording.pkg/dashboard: candidate selection, unresolved-finding read, once-per-head marker + 1h counter, follow-up title.