Skip to content

fix(ci): correct required-workflow-bootstrap job-scope check boundary - #1528

Closed
seonghobae wants to merge 1 commit into
mainfrom
fix/strix-quick-gate-bootstrap-scope
Closed

fix(ci): correct required-workflow-bootstrap job-scope check boundary#1528
seonghobae wants to merge 1 commit into
mainfrom
fix/strix-quick-gate-bootstrap-scope

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Root cause

scripts/ci/test_strix_quick_gate.sh's assert_opencode_review_uses_codegraph_and_contextual_orchestrator check used:

awk '/^  required-workflow-bootstrap:$/,/^[^ ]/' "$bootstrap_file" | grep -q '^[[:space:]]*if:'

to assert that the required-workflow-bootstrap job in .github/workflows/opencode-review.yml has no if: gating (it must run unconditionally so branch protection always sees the required context). The end pattern /^[^ ]/ only matches a fully unindented line, but job headers in this workflow are 2-space indented — so the awk range never actually stops at the next job boundary and instead sweeps every subsequent job (coverage-source-tree, coverage-evidence, opencode-review-target) all the way to end of file.

This was latent (the swept-in jobs had no if: of their own) until #1497 ("require substantive agent verdicts") added a genuine step-level if: github.event.action != 'closed' inside opencode-review-target. The over-broad range now sweeps that line in and misattributes it to required-workflow-bootstrap, which has no if: at all and was always compliant — failing exact-head-path-policy/test_strix_quick_gate on every open PR against current main, not just the one where I first hit it (#1527).

Confirmed reproducing identically against a clean origin/main checkout (1cbb6aaf), independent of any PR's own diff.

Fix

Corrected the awk range to track an explicit found flag and exit as soon as it reaches the next 2-space-indented job header, so it captures only the required-workflow-bootstrap job's own body:

awk '
	/^  required-workflow-bootstrap:$/ { found=1; print; next }
	found && /^  [A-Za-z0-9_-]+:$/ { exit }
	found { print }
' "$bootstrap_file" | grep -q '^[[:space:]]*if:'

Verified the corrected range now extracts only required-workflow-bootstrap's own steps (stopping right before coverage-source-tree:), and still correctly flags a real violation if one were reintroduced.

Verification

  • PYTHONPATH=. coverage run -m pytest tests && coverage report && interrogate: all tests pass, 100% statement/branch coverage, 100% docstrings (unchanged from baseline — this is a test-script-only fix, no scripts/ci/*.py touched).
  • bash scripts/ci/test_strix_quick_gate.sh: previously failed with FAIL: opencode required workflow bootstrap must not depend on required-workflow event payload fields; now reports test_strix_quick_gate: PASS.

Developer experience

Restores exact-head-path-policy/test_strix_quick_gate to a correct, narrowly-scoped check instead of a false positive that blocks every open PR once opencode-review.yml gains any step-level if: anywhere after the bootstrap job.

User experience

No user-facing change — CI-only fix.


Generated by Claude Code

scripts/ci/test_strix_quick_gate.sh's awk range for the
required-workflow-bootstrap job used /^[^ ]/ as its end pattern, which only
matches a fully-unindented line, so it never stopped at the next
2-space-indented job header and instead swept every later job in
opencode-review.yml (coverage-source-tree, coverage-evidence,
opencode-review-target) into the check.

This was latent until #1497 ("require substantive agent verdicts") added a
genuine step-level `if: github.event.action != 'closed'` inside
opencode-review-target, which the over-broad range then misattributed to
required-workflow-bootstrap (which has no `if:` at all and was always
compliant) -- failing exact-head-path-policy on every open PR in this
repository, confirmed reproducing on a clean origin/main checkout.

Corrected the range to exit as soon as it reaches the next 2-space-indented
job header.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 40 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 63be1b71-396e-465b-849b-fc9d35d6b472

📥 Commits

Reviewing files that changed from the base of the PR and between 1cbb6aa and b62e75f.

📒 Files selected for processing (2)
  • CHANGELOG.md
  • scripts/ci/test_strix_quick_gate.sh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Copy link
Copy Markdown
Contributor Author

Closing as a duplicate writer of canonical existing owner #1506.

Live exact-head comparison:

  • fix(ci): bound required-workflow-bootstrap awk extraction to its own job #1506 c6305c9732faceb8a21561db11dc4717296a9a1c is 0 behind protected main and already owns scripts/ci/test_strix_quick_gate.sh for this exact job-boundary defect.
  • It contains the same stateful next-job boundary plus the still-required pipefail/SIGPIPE-safe consumer and executable regressions for a real job-level if: and a 20,000-line block.
  • Its exact-head path-policy and generated Security/SAST/CodeQL/Python/OSV/Secret/Scorecard/SBOM workflows are terminal-success; required OpenCode/full Strix/Noema/formal approval remain non-passing, so no merge or evidence transfer is implied.

#1528 b62e75f3dd28788b0a37aef2e9532451359f24b0 is a narrower later implementation of the same boundary. Keeping it open would violate the single-writer owner rule and spend review/scanner capacity on a competing branch. #834 remains blocked on protected integration of #1506 and fresh exact-head evidence.

@seonghobae seonghobae closed this Aug 31, 2026
seonghobae added a commit that referenced this pull request Sep 2, 2026
Add real, non-vacuous regression tests to
tests/test_pr1669_cancel_stale_opencode_runs.py for the exact incident PR
#1669 exists to fix: ContextualWisdomLab/naruon PR #1528's Strix run
33581213829 (head cf472cf77fb93325858f485a22e967449d7c387a) was
force-cancelled while it was the PR's sole, unchanged current head, because
stale_pr_run_ids() and active_review_run_refs() computed the expected head as
str(pr.get("headRefOid") or "").lower() instead of validating it.

The branch's tip (c06b625) already carries the real fix -- both functions now
validate the snapshot headRefOid via validate_git_sha() and fail safe (empty
result) when it is missing or malformed, matching the idiom used elsewhere in
this file -- and the six temp_pr1669_*/-_temp_pr1669_* debris files from an
earlier abandoned self-repair attempt are already gone from this branch.
What was missing was direct regression coverage naming the incident: the
prior test file only covered cancel_stale_opencode_runs()'s revalidation
plumbing, and the PR description claimed three tests
(test_stale_pr_run_ids_preserves_current_head_run_when_head_ref_oid_missing,
test_active_review_run_refs_preserves_current_head_run_when_head_ref_oid_missing,
test_cancel_stale_pr_runs_issues_no_cancel_call_when_head_ref_oid_missing)
that did not exist anywhere in the repository.

Each new test uses the real naruon PR #1528 / run 33581213829 identifiers,
was verified by temporarily reverting the validate_git_sha() guard to fail
against the pre-fix code, and passes against the current fix.

Full suite: 2604 passed, 1 skipped, 21 subtests passed. 100% coverage on
scripts/ci. 100% docstring coverage (interrogate).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
seonghobae added a commit that referenced this pull request Sep 2, 2026
…l-failure fix

The org's own automated PR review/fix loop had already reconciled this
branch with main once (commit c946c7b), but that reconciliation silently
regressed the actual headRefOid fix back to the pre-fix buggy pattern
(str(pr.get("headRefOid") or "").lower()) in stale_pr_run_ids() and
active_review_run_refs(), and lost the entire live-revalidation safety net
(_direct_pr_run_still_superseded, _review_run_still_superseded,
_cancel_revalidated_review_run_refs). Its own attempt to push a corrected
commit then failed closed (correctly) when its configured push credential
was unavailable, leaving the branch stuck in the regressed state.

This commit resolves a fresh merge of current main (with #1707/#1702/#1704/
#1711/#1712 all applied) directly against a37a428 -- this branch's last
verified-good commit (100% coverage, 2614 tests, real regression tests
reproducing the naruon PR #1528 incident) -- rather than building on top of
the already-regressed c946c7b.

Combines both fixes at every cancellation call site (cancel_stale_pr_runs,
cancel_stale_opencode_runs, _cancel_revalidated_review_run_refs): PR #1669's
live revalidation immediately before each destructive cancel call (closing
the TOCTOU gap a snapshot-only headRefOid check can't), and #1712's check
of force_cancel_workflow_runs's actual per-run failure result (so a run
proven stale but whose cancel API call GitHub itself rejected is never
reported as cancelled). #1712's simpler force_cancel_workflow_run_refs
wrapper is removed as dead code now that its call sites all use the more
thorough per-caller revalidation; its own tests were adapted to target the
functions that actually carry its safety guarantee forward, not deleted.

Verified: PYTHONPATH=. coverage run -m pytest tests -- 2621 passed, 1
skipped, 21 subtests; coverage report -- 100% on scripts/ci; interrogate --
100% docstrings.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
seonghobae added a commit that referenced this pull request Sep 2, 2026
#1669)

* fix(scheduler): never let a falsy headRefOid cancel every run for a PR

stale_pr_run_ids() and active_review_run_refs() in
scripts/ci/pr_review_merge_scheduler.py computed the PR's expected current
head as str(pr.get("headRefOid") or "").lower(), unlike every other
head-comparison call site in this file, which validates via
validate_git_sha(). A falsy headRefOid (missing/None/empty -- plausible on a
PR inspected moments after it opens) silently coerced to "", which never
equals a real run head_sha, so every active run for that PR -- including one
for its true, unchanged current head -- was misclassified as stale and
force-cancelled with no further check by cancel_stale_pr_runs() /
cancel_stale_opencode_runs(). This is called unconditionally from inspect_pr()
by both the per-PR scan-pr-queue job and the hourly org-queue-sweep job, so it
is an org-wide exposure, not repo-specific.

This reproduces the 2026-09-02 incident where naruon PR #1528's Strix run
(33581213829) was cancelled while it was the PR's sole, still-current head.
It is the same bug class docs/doctoring/queue-hygiene-live-ref-race.md already
fixed for the sibling bash "Queue hygiene" cancellation path (which
revalidates every candidate via revalidate_queue_cancellation.sh before
cancelling) -- that fix never touched this earlier-running, revalidation-free
path in the same inspect_pr() pass.

Both functions now fail safe on a falsy headRefOid: log a warning and return
no stale/cancellable runs for that PR, instead of treating an unresolved head
as "matches nothing, so everything is stale."

Adds three regression tests reproducing the incident with the real run id,
head SHA, and PR number, each failing against the pre-fix code and passing
after the guard. Full suite: 2603 passed, 1 skipped; 100% coverage and
docstrings hold on scripts/ci. Doctoring note:
docs/doctoring/scheduler-stale-headrefoid-cancellation.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test(scheduler): stage live-head cancellation race repair

* ci(scheduler): run bounded live-head cancellation repair

* fix(scheduler): repair exact-head cancellation race findings

* fix(scheduler): run exact-head live cancellation repair v2

* fix(scheduler): adapt legacy synthetic SHA fixtures

* fix(scheduler): publish validated repair with branch token

* test(scheduler): preserve docstring gate in generated race repair

* ci(scheduler): verify v4 live-head race repair

* fix(ci): isolate PR1669 writer credential to publish step

* docs: make stale-cancellation incident references linkable

* ci: retrigger exact-head PR1669 guarded repair

* test(scheduler): cover live cancellation revalidation branches

* fix(actions): publish guarded scheduler repair with branch token

* fix(actions): isolate scheduler repair write authority

* fix(scheduler): make PR1669 repair current-main aware

* fix(actions): run current-main-aware PR1669 publisher

* test(scheduler): cover parallel live stale-run cancellation

* test(actions): rerun PR1669 with parallel cancellation coverage

* test(scheduler): refresh legacy cancellation fixtures

* ci(pr1669): verify refreshed legacy fixtures

* test(pr1669): close live revalidation coverage gaps

* test(pr1669): cover direct revalidation fail-closed branch

* test(scheduler): cover stale OpenCode cancellation owner path

* refactor(pr1669): retire dead pre-revalidation batch helper

* fix(scheduler): cover exact-head cancellation path

* test(scheduler): align PR1669 fixtures with live revalidation

* ci(scheduler): run PR1669 v8 fixture repair

* fix(ci): provision PR1669 publisher verification toolchain

* fix(ci): publish PR1669 successor with scoped workflow token

* ci: preserve PR1669 successor check triggering

* test(scheduler): prove draft retry cancellation race

* ci: include PR1669 draft-review RED repair

* fix(ci): strip PR1669 regression EOF whitespace

* fix(ci): align PR1669 draft cancellation fixture

* fix(ci): make PR1669 fixture reconciliation indentation-safe

* ci(scheduler): coalesce PR1669 repair onto one runner

* ci(scheduler): publish verified one-shot repair with scoped token

* fix(scheduler): revalidate live state before cancellation

* test(scheduler): preserve failed stale-run cancellations

* test(scheduler): cover dispatch cancellation failure

* ci(scheduler): repair failed-cancellation result semantics

* ci: add one-shot failed-cancellation repair

* ci: retire weaker PR1669 repair writer

* ci: repair PR1669 cancellation-result fixtures

* test(scheduler): make central run revalidation credential RED

* repair(scheduler): bind stale-run reads to causal credential

* test(pr1669): reproduce the naruon headRefOid incident directly

Add real, non-vacuous regression tests to
tests/test_pr1669_cancel_stale_opencode_runs.py for the exact incident PR
#1669 exists to fix: ContextualWisdomLab/naruon PR #1528's Strix run
33581213829 (head cf472cf77fb93325858f485a22e967449d7c387a) was
force-cancelled while it was the PR's sole, unchanged current head, because
stale_pr_run_ids() and active_review_run_refs() computed the expected head as
str(pr.get("headRefOid") or "").lower() instead of validating it.

The branch's tip (c06b625) already carries the real fix -- both functions now
validate the snapshot headRefOid via validate_git_sha() and fail safe (empty
result) when it is missing or malformed, matching the idiom used elsewhere in
this file -- and the six temp_pr1669_*/-_temp_pr1669_* debris files from an
earlier abandoned self-repair attempt are already gone from this branch.
What was missing was direct regression coverage naming the incident: the
prior test file only covered cancel_stale_opencode_runs()'s revalidation
plumbing, and the PR description claimed three tests
(test_stale_pr_run_ids_preserves_current_head_run_when_head_ref_oid_missing,
test_active_review_run_refs_preserves_current_head_run_when_head_ref_oid_missing,
test_cancel_stale_pr_runs_issues_no_cancel_call_when_head_ref_oid_missing)
that did not exist anywhere in the repository.

Each new test uses the real naruon PR #1528 / run 33581213829 identifiers,
was verified by temporarily reverting the validate_git_sha() guard to fail
against the pre-fix code, and passes against the current fix.

Full suite: 2604 passed, 1 skipped, 21 subtests passed. 100% coverage on
scripts/ci. 100% docstring coverage (interrogate).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* ci(pr1669): remove a second, out-of-scope self-repair attempt

While rebasing onto this branch's latest remote tip, a second in-flight
self-repair attempt (unrelated to the headRefOid bug this PR fixes) was
found already pushed here: a push-triggered, contents:write one-shot
workflow (.github/workflows/_temp_pr1669_failed_cancel_result_repair.yml)
plus three failing regression tests for a different bug -- a rejected
force_cancel_workflow_runs() API call being silently reported as a
successful cancellation.

That bug looks real, but it is not the headRefOid bug this PR exists to
fix, and the still-armed workflow (watching pushes to this exact branch,
touching tests/test_pr1669_cancel_stale_opencode_runs.py among its
trigger paths) would have reacted to this very push and self-modified the
branch again. Remove both rather than let another self-modifying
generator loop run here or leave unlanded RED tests behind; the finding
has been routed to a separate, properly scoped follow-up instead of being
fixed or left dangling in this PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* ci(pr1669): remove a third, out-of-scope RED test file

tests/test_pr1669_central_run_revalidation_credential.py (added by a
concurrent, still-running self-repair pass) asserts that
_fresh_active_run_for_cancellation() must read a central repository_dispatch
run through the dispatch credential (gh_api_json_via_dispatch_token) rather
than the target-repo read credential (gh_api_json). One of its two tests
currently fails against this file's actual implementation, which always
uses gh_api_json() regardless of which repository is being read -- a
real-looking credential-boundary gap, but in the defense-in-depth
revalidate-before-cancel machinery, not the headRefOid bug this PR exists
to fix, and no corresponding code change had landed for it (diff against
c06b625 confirms scripts/ci/pr_review_merge_scheduler.py is unchanged).

Removed for the same reason as the prior cleanup commit: keep this PR's
diff scoped to the headRefOid fix it was opened for, and genuinely green,
rather than carrying an unlanded RED test for an unrelated, still-being-
worked-on finding.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
seonghobae pushed a commit that referenced this pull request Sep 3, 2026
…13 audit

- CodeRabbit: "with no run at all" contradicted the same sentence's own
  status: queued evidence -- a run existed, its job just never started.
  Reworded to "with no job started" in both the doctoring record and the
  gap-baseline entry.
- Devin (analysis): qualify the bare naruon PR #1528 references with the
  ContextualWisdomLab/ owner prefix so they resolve as cross-repo links from
  this repo, consistent with this cycle's established convention.
- Devin (bug): soften the unverified claim that the five pull_request_target
  firings on one unchanged SHA were "near-certainly labeled/unlabeled" --
  only synchronize was ruled out; the specific event types were not actually
  checked. Say so.
- Devin (bug, most substantive): the audit's noema-review.yml row claimed
  unconditional reliability for native cancel-in-progress, but that
  mechanism cancels whichever run most recently entered the concurrency
  group -- keyed on run-creation order, not head-SHA recency. An older
  push's synchronize event processed after a newer one's could in principle
  cancel the current run instead of the stale one; existing tests
  (tests/test_noema_review_gate.py) prove the *explicit* cancellation step
  can't do this, but don't cover native cancel-in-progress itself. No
  evidence this has ever happened, and no workflow change is made here --
  recorded as an open, unverified risk (footnoted in the table, and as a new
  bullet in "What this resolves, and what it does not") rather than silently
  dismissed or asserted as a confirmed bug, consistent with this record's
  own practice for its other open leads.

Grepped tests/ for the exact strings touched here; only
tests/test_product_technical_gap_baseline.py pins this file's content, and
it still passes. Full suite: 2630 passed, 1 skipped (two pre-existing
Python-3.12-only test files not collectible under this sandbox's 3.11).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants