Repository navigation
✨ review: add review.shadow to record would_have_held for the review gate - #12158
mrbobbytables wants to merge 5 commits into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch:
Conflicts are checked with every PR merged onto the current tip of
|
|
CI triage (comment-only — head lives in a fork):
🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 |
|
CI triage for head of this PR (run 38787988867 / 38787988855): PR-local — needs a push from the fork:
Not yours — infra / shared, will clear on rerun:
Head is in a fork, so no push from the hive; merging 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
|
Check report for head
After rebase + (1) + (2), the remaining red should clear; 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
…gate Hive-Run: hivecommons#12104 Hive-Plan: RFC hivecommons#12062 / hivecommons#12068 shadow-mode gates Hive-Spec: review-shadow#would-have-held Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> (cherry picked from commit a187caa) Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…adow write path Adds a review.shadow reference in the review config handler so the Settings parity test sees the new toggle, and a round-trip test for recordReviewShadow through MergeReviewEvidenceShadow. Hive-Fix-Attempt: 1/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…logging Review follow-ups for hivecommons#12158 (review.shadow, hivecommons#12104): - Branch the review-verdicts.json load warning on require_approval so a shadow-only run no longer claims merge eligibility will fail closed. - Extract writeShadowEvidence, shared by intent.shadow and review.shadow, so the two recorders cannot drift again; recordReviewShadow now gets the logger==nil fallback and logs confidence like the intent gate. - When a signing key is configured but unusable, skip the shadow write and Warn instead of rewriting a signed bundle as unsigned. - Record both shadow observations from one eligible block with a shared evaluated_at. - Move the review shadow code to review_shadow.go beside its test, and add the dashboard PUT/GET + Settings toggle parity coverage for review.shadow. Hive-Fix-Attempt: 2/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…w signing-key change Add review.shadow to the /api/config/review GET response and PUT request schemas in dashboard/openapi.json so the spec matches what handleReviewConfigGet/Put emit and accept. Add a changed- changelog fragment for writeIntentShadow now skipping (Warn) instead of writing unsigned when a configured signing key fails to load. Hive-Fix-Attempt: 3/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…op the inline style LoadSigningKey treats a missing key file as 'unsigned', so the signing-key-unusable test never reached the guard; write a malformed key instead. Use the existing .mt-3 utility on the new review.shadow toggle so the style ratchet stays at baseline. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
d6e14a0 to
77d5118
Compare
…gate (#12337) * ✨ review: add review.shadow to record would_have_held for the review gate Hive-Run: #12104 Hive-Plan: RFC #12062 / #12068 shadow-mode gates Hive-Spec: review-shadow#would-have-held Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> (cherry picked from commit a187caa) Signed-off-by: Andy Anderson <clubanderson@users.noreply.github.com> * 🧪 review: reference review.shadow for settings parity and test the shadow write path Adds a review.shadow reference in the review config handler so the Settings parity test sees the new toggle, and a round-trip test for recordReviewShadow through MergeReviewEvidenceShadow. Hive-Fix-Attempt: 1/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Signed-off-by: Andy Anderson <clubanderson@users.noreply.github.com> * 🧹 review: share the shadow evidence writer and tighten review.shadow logging Review follow-ups for #12158 (review.shadow, #12104): - Branch the review-verdicts.json load warning on require_approval so a shadow-only run no longer claims merge eligibility will fail closed. - Extract writeShadowEvidence, shared by intent.shadow and review.shadow, so the two recorders cannot drift again; recordReviewShadow now gets the logger==nil fallback and logs confidence like the intent gate. - When a signing key is configured but unusable, skip the shadow write and Warn instead of rewriting a signed bundle as unsigned. - Record both shadow observations from one eligible block with a shared evaluated_at. - Move the review shadow code to review_shadow.go beside its test, and add the dashboard PUT/GET + Settings toggle parity coverage for review.shadow. Hive-Fix-Attempt: 2/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Signed-off-by: Andy Anderson <clubanderson@users.noreply.github.com> * 📝 review: document review.shadow in openapi.json and the intent.shadow signing-key change Add review.shadow to the /api/config/review GET response and PUT request schemas in dashboard/openapi.json so the spec matches what handleReviewConfigGet/Put emit and accept. Add a changed- changelog fragment for writeIntentShadow now skipping (Warn) instead of writing unsigned when a configured signing key fails to load. Hive-Fix-Attempt: 3/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Signed-off-by: Andy Anderson <clubanderson@users.noreply.github.com> * 🧪 review: make the unusable-signing-key test use a malformed key and drop the toggle's inline style - TestRecordReviewShadowSigningKeyUnusable pointed SigningKeyFile at a missing file, which LoadSigningKey treats as "no key" (records unsigned by design). Write a malformed key so the test exercises the guard that skips the write with a Warn. - The new Settings toggle added an inline style="margin-top:10px", pushing operator index.html over the style-ratchet baseline; .config-toggle already spaces stacked toggles via margin-bottom, so drop it. - Document the signing-key caveat in review-swarm.md / intent-verification.md and list review.shadow in hive.yaml.example (swarm docs-currency notes). Signed-off-by: Andy Anderson <clubanderson@users.noreply.github.com> --------- Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Signed-off-by: Andy Anderson <clubanderson@users.noreply.github.com> Signed-off-by: scanner <scanner@hive.kubestellar.io> Co-authored-by: mrbobbytables <mrbobbytables@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: scanner <scanner@hive.kubestellar.io> Co-authored-by: Andy Anderson <clubanderson@users.noreply.github.com>
There was a problem hiding this comment.
Reviewed (intent-alignment) — diff satisfies #12104; one scope note, non-blocking.
- Disclosed scope creep:
intent.shadowbehavior change (src/cmd/hive/shadow_gate.go:25-28,src/cmd/hive/intent_shadow.go:50). #12104 only asks forreview.shadow; foldingwriteIntentShadowinto the sharedwriteShadowEvidencehelper changes the existing intent path from "write unsigned on key-load error" (oldkey = nilfallback) to "skip with Warn". This is a sensible fix and is disclosed in the PR body andchangelog.d/changed-12104-intent-shadow-signing-key.md, but it is a behavior change to a different feature riding on this issue — flagging so a maintainer consciously accepts it rather than discovering it later.
Verified against the issue: classifyMergeEligibility review gate still keyed only on requireReviewApproval (merge_eligibility.go:462), so shadow cannot alter a bucket; shadowReview is forced off when require_approval is on (merge_eligibility.go:843); artifact-unavailable case mirrored (review_shadow.go:21-22); dashboard PUT/GET + toggle and table tests present.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
docs-currency (read at head 77d5118)
-
Changelog names a config key that does not exist.
changelog.d/changed-12104-intent-shadow-signing-key.md:1says the new skip-on-bad-key behaviour triggers whenevidence.signing_key_pathpoints at an unusable key. The hive.yaml key isevidence.signing_key_file(src/pkg/config/evidence.go:17;src/docs/operator-reference.md:164;src/docs/review-evidence.md:104).SigningKeyPath()is only the Go accessor (evidence.go:26). An operator grepping their config forsigning_key_pathfinds nothing. The PR body repeats the same name. -
Operator docs now contradict shadow-write behaviour.
src/docs/operator-reference.md:164states that when the signing key is "empty, absent or unusable" bundles are written with"signed": falseand a warning logged. After this PR,src/cmd/hive/shadow_gate.go:26-30skips the write entirely for an unusable key — for bothreview.shadowand (changed behaviour)intent.shadow. Neither that row nor theintent.shadowsection (src/docs/intent-verification.md:76, "Recording needsevidenceenabled and an existing bundle for the head") nor the newreview.shadowparagraph (src/docs/review-swarm.md:224) mentions that a loadable-but-invalid key suppresses shadow recording. The fact lives only in the changelog fragment; one sentence in the operator-reference row (or the two shadow sections) would close the gap.
— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Style/maintainability review (read at head 77d5118)
Shadow gate duplicates the enforce predicate instead of sharing it — src/cmd/hive/review_shadow.go:19-25 re-implements the require_approval decision (!g.reviewLoaded → "review approval required, but review-verdicts.json is unavailable"; !HasAggregateApproval(...) → "awaiting review approval") as a second copy of the branch in classifyMergeEligibility at src/cmd/hive/merge_eligibility.go:462-468, including both reason strings verbatim. The contract in #12104 is that shadow evaluates the gate exactly as require_approval would; with two hand-synced copies, any later change to the enforce branch (an extra condition, a reworded reason) silently makes would_have_held diverge from what the real gate does — which is the one thing this signal must not do. The sibling intent.shadow avoids this by reusing the same predicate the enforce path uses (intentShadowGate → v.BlocksMerge(true), src/cmd/hive/intent_shadow.go:23-24). Suggest extracting a small reviewGateHold(g mergeGates, fullRepo string, pr github.PullRequest) (held bool, reason string) in merge_eligibility.go and calling it from both classifyMergeEligibility and reviewShadowGate; TestReviewShadowGate would then cover parity for free.
Everything else here (shared writeShadowEvidence extraction, dashboard PUT pointer-field pattern, Settings toggle markup, changelog fragment naming, docs placement) follows the existing idioms.
— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Landed on v5 via #12337 (your commits preserved as author, plus the two CI fixes — malformed-key test fixture and the style-ratchet inline style). Closing as superseded; #12104 is closed by that merge. Thank you @mrbobbytables! |
Closes #12104
Adds
review.shadow(default false). When on, the review-approval gate is evaluated asrequire_approvalwould (including the artifact-unavailable case) but the PR stays merge-eligible;shadow.review_gate.would_have_held+ reason/confidence is recorded on the evidence bundle and an audit log line is emitted.require_approval: truestill blocks. Nothing here can block a merge.Behavior note (intent.shadow):
writeIntentShadownow uses the sharedwriteShadowEvidencehelper. Whenevidence.signing_key_pathpoints at a key that exists but fails to load, the shadow observation is skipped with aWarninstead of being written unsigned (previously it fell back tokey = nil), so a misconfigured key can't downgrade a signed bundle to"signed": false. An absent key path/missing key file still records unsigned as before. Covered bychangelog.d/changed-12104-intent-shadow-signing-key.md.Includes dashboard PUT + Settings toggle, docs, changelog fragment, and a table test (shadow on / off / require_approval on). Not run locally per AGENTS.md; CI is the verdict.
— hive: backend=copilot model=claude-sonnet-5.5
🐝 Hive Agent:
contributor| SHA:ccd1e5722