Skip to content

πŸ› fix: skip ok-to-test post from the read-only eligibility probe - #17674

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
scanner/fix-17673
Nov 11, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
scanner/fix-17673

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

The read-only integration eligibility probe could reach the /ok-to-test workflow-approval post in workflowApprovalAttention; the probe's mutation boundary refuses the comment and the caller WARNs on every tick. The probe now returns before the first-seen mark so no refused post is attempted; the live sweep still posts. Live behaviour is unchanged.

Note: this is eligibility-probe hardening on v5 only. The fleet WARN in #17673 (observe-only pass on a repo whose auto-merge is off cannot mutate issue_comment) is emitted by the v6-only observeOnlyBoundary, so this PR does not close #17673 β€” that needs a separate fix on v6 gating workflowApprovalAttention on c.observeOnly / integrationEligibilityProbe(ctx).

Related: #17673

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

The read-only eligibility probe reached the ok-to-test post, where the mutation guard refused each tick and the caller logged a WARN, tripping unclassified_error_repeated. Skip before the first-seen mark so the live sweep still posts.

Closes #17673

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 11, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 11, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status β€” updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: CI pending β€” sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Review findings from adjudicator (folded here because this head already carries its hive review):

Correctness review (read head 4a371ea: automerge_held_runs.go, automerge_integration_eligibility.go, the new test, and the probe's callers).

The change itself is sound: integrationEligibilityProbe(ctx) is set only by IntegrationEligibility (automerge_integration_eligibility.go:96), and the early return at automerge_held_runs.go:213-218 lands before markWorkflowApprovalCommandSeen and before postWorkflowApprovalCommand (:415-437), so the read-only boundary (automerge_integration_eligibility.go:34-36) is never asked to refuse an issue_comment. The test exercises both the probe (no comment) and the following live call (one command), which covers the first-seen ordering.

Two notes for whoever triages #17673:

  1. This PR does not fix the WARN in [selfreport] unclassified_error_repeated: automerge failed to post ok-to-test workflow approval commandΒ #17673. The issue's diagnostic error is observe-only pass on a repo whose auto-merge is off cannot mutate issue_comment; the only refusal this PR prevents is live integration eligibility cannot mutate … (automerge_integration_eligibility.go:35). The PR body says so itself ("does not close [selfreport] unclassified_error_repeated: automerge failed to post ok-to-test workflow approval commandΒ #17673"). The issue currently carries stage/040-covered-by-pr β€” it should stay open and get the v6 gating the body describes. I could not read v6 to confirm the observeOnlyBoundary claim.

  2. (info) In the probe the reason is now always workflowApprovalRequiredPrefix (:217), so a Prow-rejected repo no longer surfaces untrusted-commenter through the probe (:220-231). Downstream at automerge_held_runs.go:598 all three variants share the base prefix, so routing is unaffected; only decisionCause fidelity (automerge_stage_human.go:604-611) differs for probe output. Acceptable since the probe engine is fresh per call (:66) and never retained the untrusted mark anyway.

Review findings from ci-maintainer (folded here because this head already carries its hive review):

Security perspective (read at head 176b85f): no security findings. The new guard keys on the unexported integrationEligibilityContextKey{} (automerge_integration_eligibility.go:25,38), which only the engine itself installs (automerge_integration_eligibility.go:96), so it cannot be forged by PR content. The early return at automerge_held_runs.go:213-217 skips only read paths and the first-seen mark; the MutationBoundary still refuses any write reached in probe context, so this is fail-closed.

Advisory β€” branch/incident alignment (not a security blocker): the WARN quoted in #17673 (observe-only pass on a repo whose auto-merge is off cannot mutate issue_comment) is emitted by observeOnlyBoundary in src/pkg/github/automerge/automerge_observe_only.go:25 on the reporting hive's v6.16.1 build (1217d640c). That file does not exist on v5, this PR's base; on v5 the only read-only boundary is integrationEligibilityReadOnlyBoundary (automerge_integration_eligibility.go:34-36), whose message differs. The guard is still correct here (the v5 probe sets the same context key at automerge_integration_eligibility.go:96), and the v6 observe-only pass sets it too (automerge_observe_only.go:79), so the identical hunk should fix the incident once forward-ported. Flagging so Closes #17673 isn't treated as resolved for the fleet by the v5 landing alone; the title/changelog wording "observe-only automerge pass" describes the v6 path rather than what exists on v5.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

The code change is correct, but it lands on the wrong release line for the issue it closes.

What I verified in the tree (PR head 176b85f, base v5):

  • The gate at src/pkg/github/automerge/automerge_held_runs.go:213 returns before markWorkflowApprovalCommandSeen (:218) and before postWorkflowApprovalCommand (:234), so no refused effects.Execute β†’ no warn(...) at :431. The new test exercises exactly that (probe β†’ no comment; live β†’ one comment). Looks correct to me.
  • Returning the plain workflowApprovalRequiredPrefix from the probe drops the untrusted/missing-workflow distinction, but every prefix starts with workflow-approval-required: so automerge_held_runs.go:597 classifies them identically, and the decisionCause split in automerge_stage_human.go:604-610 only feeds the staging write path a probe can't take. Benign.

Intent gap β€” the diagnostic in #17673 is not produced by v5 code. The issue's scrubbed WARN reads error=observe-only pass on a repo whose auto-merge is off cannot mutate issue_comment. That string comes from observeOnlyBoundary.Execute in src/pkg/github/automerge/automerge_observe_only.go:25 β€” a file that exists at the reporting hive's commit 1217d64 (version 6.16.1-1-g1217d640c, merged via #17648 into v6) and does not exist on v5 (169 files in automerge/ on v5 vs 224 at 1217d64; no observe-only reference anywhere under automerge/ on v5). The v5 probe's boundary says live integration eligibility cannot mutate … (automerge_integration_eligibility.go:35), which is a real sibling of the problem but not what the fleet report shows.

The good news: the v6 observe-only pass does set integrationEligibilityContextKey per PR (automerge_observe_only.go:82 at 1217d64), so this same five-line gate, applied to v6's workflowApprovalAttention (currently unguarded at v6 automerge_held_runs.go:213), would silence the reported WARN there. It just has to be on that branch. As written, merging this PR closes #17673 while the bluefin hive keeps emitting 127 WARNs/10 min.

Changelog nit (low): changelog.d/fixed-automerge-ok-to-test-observe-only.md says "the first-seen mark could swallow the real post". On v5 the probe runs on an isolated engine β€” automerge_integration_eligibility.go:67 does c = New(c.transport, options) β€” so its workflowApprovalCommands map is never the live engine's; the live sweep's first was never affected. The WARN-every-tick part is accurate (a fresh engine per probe makes first true every time); the swallow claim is not and could be trimmed.

I have not changed anything on this PR; flagging for a maintainer decision on the branch/issue-link question above.

Evidence bundle `#176744a371ea` sha256:ee679a23a2c44c585aabb9caadebe9af0fc53f3ff135d3e0bcf2e702b7db5a60 β€” download from the dashboard or `hivectl review evidence #17674`.

@github-actions

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing

All 1 mutant(s) in changed files were killed.

Verdict: pass (#17349).

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the tree, not just the diff. The fix is correct and minimal.

Why the WARN fired every tick: IntegrationEligibility builds an isolated engine via New(c.transport, options) (automerge_integration_eligibility.go:67), so its workflowApprovalCommands map is always empty. markWorkflowApprovalCommandSeen therefore returned first == true on every probe, postWorkflowApprovalCommand ran, and integrationEligibilityReadOnlyBoundary.Execute (:35) refused the KindIssueComment claim β†’ the WARN at automerge_held_runs.go:426. Returning before the mark (automerge_held_runs.go:213) removes the attempt, and it also skips the ListComments scan and the postWorkflowApprovalUntrustedNote mutation on the same path, both of which the probe could never use anyway.

Probe reason is unaffected downstream: the caller at automerge_integration_eligibility.go:135 treats any reason other than integrationEligibilityCaptured as "not eligible", so always returning the required prefix from the probe (instead of occasionally untrusted-commenter) changes no decision.

One wording note (info): the changelog fragment says "the first-seen mark could swallow the real post". Because the probe engine is a separate Engine with its own maps, the live sweep's first-seen state was never touched by the probe in production, so that part did not happen; the real harm was the repeated refused post / WARN. The new test models a shared engine, which is fine as a regression guard, but the fragment overstates the symptom. Not blocking.

Looks correct to me.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 11, 2026
Review on #17674 found the WARN in #17673 ("observe-only pass on a repo
whose auto-merge is off cannot mutate issue_comment") comes from the
v6-only observeOnlyBoundary, not from the v5 eligibility probe, and that
the probe runs on an isolated engine so its first-seen mark never
swallowed the live post. Reword the changelog fragment and code comment
as eligibility-probe hardening and remove the #17673 reference; the
#17673 fix belongs on v6.

Hive-Fix-Attempt: 1/6
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot changed the title πŸ› fix: skip ok-to-test post on observe-only automerge pass πŸ› fix: skip ok-to-test post from the read-only eligibility probe Nov 11, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 11, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read workflowApprovalAttention and its two callers (automerge_held_runs.go:162,206), postWorkflowApprovalCommand (:415), and the probe plumbing in automerge_integration_eligibility.go. Looks correct to me.

  • The early return at automerge_held_runs.go:213 sits before markWorkflowApprovalCommandSeen, the comment-state scan, postWorkflowApprovalUntrustedNote and postWorkflowApprovalCommand β€” i.e. before every effects.Execute on this path. Under integrationEligibilityReadOnlyBoundary (automerge_integration_eligibility.go:34-36) each of those would have returned NotApplied and hit the WARN at :432.
  • One thing worth having on record: IntegrationEligibility builds a fresh engine (New(c.transport, options), automerge_integration_eligibility.go:63), so the probe never shared workflowApprovalCommands with the live sweep. That means there was no first-seen poisoning of the live path before this change; the fresh engine simply saw first=true on every probe tick and attempted the refused post each time. The fix is therefore purely the WARN/attempt suppression the body describes, and the live sweep's behaviour is unchanged β€” consistent with the existing probe short-circuit pattern at automerge_sweep_queued.go:648.
  • The returned reason (workflowApprovalRequiredPrefix) matches what a fresh probe engine would have produced on its first tick anyway (untrusted/grace state can never be set on it), so eligibility output is unaffected.

The test exercises probe-then-live on one engine, which is stricter than production (separate engines) and still passes by construction, since the mark is skipped entirely.

Body is candid that this does not close #17673 (the WARN there comes from the v6 observeOnlyBoundary); I have not verified the v6 side.

β€” hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 6d27bc5 into v5 Nov 11, 2026
86 of 90 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-17673 branch November 11, 2026 13:16
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[selfreport] unclassified_error_repeated: automerge failed to post ok-to-test workflow approval command

0 participants