Skip to content

ci(strix): 메타데이터 작업을 단일 job으로 통합 - #1962

Open
seonghobae wants to merge 16 commits into
mainfrom
codex/strix-metadata-single-job
Open

ci(strix): 메타데이터 작업을 단일 job으로 통합#1962
seonghobae wants to merge 16 commits into
mainfrom
codex/strix-metadata-single-job

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

변경

Strix의 changed-scopeadmit-current-head를 읽기 전용 메타데이터 job 하나로 합칩니다. admission이 먼저 실행되며 현재 PR과 일치할 때만 파일을 분류합니다. scanner는 code=trueadmitted=true를 모두 요구합니다.

  • 메타데이터 job: 2개 → 1개, 50% 감소.
  • 코드 변경 PR의 opened/reopened/ready_for_review 경로: 3개 → 2개, 33.3% 감소.
  • synchronize 및 exact dispatch 경로: 각각 4개 → 3개, 25% 감소.

이는 소스에서 확인한 job 구조의 변화입니다. 조직 전체 적체나 실제 대기시간이 그 비율만큼 줄었다는 뜻은 아닙니다. admission·파일 분류 shell, workflow concurrency, cleanup, scanner 본문은 기존 내용을 보존했습니다. 메타데이터 각 step은 5분, 합친 job은 10분이며 모델 실행 시간 제한은 추가하지 않았습니다. 서로 다른 권한이 필요한 cleanup·scan·status publication은 합치지 않습니다.

최신 검증 — c31aa02

현재 head c31aa0234dcae6bf1b7e27c55be7bdc87d4fe507은 이전 3ae7c80d에 main dd0b96feded94f66ecf59b25a5a9b58cfc8b4f69를 충돌 없이 일반 merge한 결과입니다. 원래 핵심 5파일 patch-id ecd42b181d1c8926c0b2b6b75f655fed93700db1을 보존했고 정상 push 뒤 원격 SHA를 재확인했습니다. Ready·미병합입니다.

  • 집중 시험: 104 passed, 32.40초, -W error, exit 0.
  • 전체 pytest: 2969 passed, 1 LLVM 19 도구 부재 skip, 21 subtests, 554.72초, exit 0. stderr 0바이트. JUnit SHA-256 c49237b86c807cf70cc160f8598d38d810f06bcfaf0e19371b8431169875e98b. coverage 측정은 포함하지 않았습니다.
  • Strix workflow: 격리 actionlint 011a6d15e749bb3f2d771eed9c7aa0e7e3e10ee7로 exit 0. 기본 Homebrew 1.7.12는 macOS 교착 정황을 확인하고 해당 검사 프로세스만 종료했으며 통과로 계산하지 않습니다.
  • 상속된 OpenCode dispatch의 ShellCheck 포함 actionlint는 29건 진단, exit 1입니다. 해당 파일 blob은 main과 이 head 모두 d86497b3f43bebbabbb4f504eb5132cdf3b7b293입니다. Strix 통합에서 새로 발생한 진단은 아니지만 전체 lint GREEN으로 주장하지 않습니다. 기존 fix(ci): protect current review evidence from stale runs #1899 수리와 범위를 분리합니다.
  • 이전 3ae7c80d quick-gate의 관련 source blob은 동일하지만 재실행하지 않았으므로 아래 PASS를 새 head 실행 증거로 옮기지 않습니다. 시각 검수도 아래 명시한 이전 head 문서 렌더링 범위입니다.
  • 새 head의 hosted Checks·독립 승인·보호 병합은 별도 확인 대상입니다. 이전 Devin COMMENTED는 APPROVED가 아니며 새 head 승인으로 승계하지 않습니다.

아래는 이전 head의 검증 이력입니다. 아래의 ‘현재 head’ 표현은 당시 3ae7c80d를 가리킵니다.

이전 검증 — 3ae7c80

현재 검증 head: 3ae7c80d36b42fa6f8d93bbd4b9620da13032b29, 일반 merge로 반영한 main: c232ca03ef891644e5397313b87849cf9e5a25ac. 원래 8개 파일 delta와 주요 5개 파일 patch-id를 보존했습니다.

  • RED: 통합 전 새 계약 3 failed, 65 passed.
  • 이전 head 726b601e 재검증: tests/test_docs_only_pr_runner_admission.py, tests/test_required_workflow_queue_contract.py, tests/test_required_review_runner_image_contract.py, tests/test_required_security_runner_image_contract.py — 84 passed, 7.33초, 종료 코드 0, -W error.
  • actionlint .github/workflows/strix.yml: 출력 없음, 종료 코드 0.
  • git diff --check: 통과.
  • 별도 읽기 전용 검토: 유효 finding 0. GitHub 승인으로 사용하지 않습니다.
  • 현재 head의 위 4개 시험 및 tests/test_strix_recovered_transient_sanitizer.py, tests/test_contextual_orchestrator_review_runtime_preflight.py: 192 passed, 11.76초, 종료 코드 0, -W error. actionlint와 diff 검사도 현재 head에서 통과했습니다.
  • 이전 head 726b601e의 작업자 scripts/ci/test_strix_quick_gate.sh: 최종 PASS와 종료 코드 0을 확인했습니다. 별도 실행도 최종 PASS·종료 코드 0으로 끝났습니다. 두 결과 모두 이전 head의 증거이며 새 head의 전체 suite나 hosted 검증으로 승계하지 않습니다.

남은 검증과 경계

초기 Draft에서 로컬 검증을 마친 뒤 리뷰 가능 상태로 전환합니다. main c232ca03를 충돌 없이 일반 merge하고 현재 head 시험을 완료했습니다. 현재 head 전체 pytest는 2966 passed, 1 LLVM 19 도구 부재 skip, 21 subtests passed, 475.10초, 종료 코드 0입니다. JUnit SHA-256: 342674c7eefb059bec8d99eec51ee9bda0d20ce3307381848334c69ac96ca27a. 이 plain pytest 명령에는 coverage 측정이 포함되지 않습니다. 현재 head의 긴 Strix quickgate는 현행 CI와 같은 테스트 전용 3초/5초 fixture 설정으로 약 34분 후 최종 PASS, 종료 코드 0으로 끝났습니다. 로그 SHA-256: 0ff47ce48d78d188d029e98adb65691f4548f209ae0f5fe4fe1ae9c7635ae447. 거부 반례의 NO_CONCLUSION/Reason 출력은 예상 fixture 진단이며 실제 scanner 성공 로그로 사용하지 않습니다. 현재 head의 hosted agent-review-runtime-quality는 SUCCESS입니다. Strix·Noema·CodeQL 등 필수 검사에는 아직 queued 항목이 있으며 독립 GitHub 승인은 없습니다. 현재 .github/mainnaruon/develop의 classic required contexts에는 별도 admission 이름이 없음을 조회했지만, 조직 전체 ruleset은 조회 권한 부족으로 확인하지 못했습니다. required strix 이름은 유지합니다.

Project API에는 조회 권한이 없지만, 브라우저를 다시 사용할 수 있게 된 뒤 Codex가 2026-09-06 KST에 실제 UI에서 naruon Platform Roadmap에 등록하고 Status: In Progress를 확인했습니다. Edge에서 PR 본문과 현재 head 고정 결정 문서의 위·아래 전체 구간을 시각 검수했습니다. 제목·본문·인라인 코드·인용 링크에 관찰된 잘림·겹침은 없었습니다. 이번 범위는 데스크톱 문서 렌더링이며 모바일·제품 UI·접근성 전체 검증은 아닙니다. 문서의 초기 Project 미확인 기록은 당시 상태이고 이 PR 본문이 후속 증거입니다. 병합·보호 규칙 우회·provider 변경·credential 범위 확대는 하지 않았습니다.

G-03/G-04와 목표 16·17에 연결합니다. 결정·권한·순차 실행 지연의 대가는 docs/doctoring/strix-metadata-admission-consolidation.md에 기록했습니다. GitHub 공식 workflow 문서와 Granite 논문을 인용했으며, arXiv의 CC BY 4.0 표시를 확인한 원본 PDF와 라이선스·SHA-256을 함께 보존했습니다. #1938의 별도 concurrency 변경은 이 PR에 복사하지 않습니다.


리베이스 및 후속 (2026-09-06)

origin/main 을 병합했습니다. 충돌 1건, tests/test_required_workflow_queue_contract.py 에서 두 변경이 같은 단언에서 교차했습니다 — main 은 그것들을 workflow_level_concurrency_group 헬퍼로 옮겼고(#1975), 이 브랜치는 잡 병합에 따라 의존 이름을 바꿨습니다. 둘 다 살려서 해소했습니다.

assert "needs: [changed-scope]" in strix_job
assert "needs.changed-scope.outputs.admitted == 'true'" in strix_job
assert "strix-security-scan-${{" in group_value          # 원시 슬라이스가 아니라 헬퍼

표시 이름을 구체화했습니다

잡 id 는 changed-scope유지합니다 — CLAUDE.md:165 가 그 id 를 required workflow 스킵 패턴의 이름으로 지목하므로, 바꾸면 다음 사람이 문서에서 찾다가 못 찾습니다.

대신 표시 이름Detect changed scope and admit the current head 로 바꿨습니다. peer 1 이 실측해 보니 세 워크플로가 이미 같은 문자열로 표시되고 있어서 이 충돌은 이 PR 이 만드는 게 아니라 이미 있습니다. 다만 이 PR 은 strix 의 잡이 하는 일을 늘리므로, 자기가 만드는 몫만 해결합니다. 체크 목록이 그 모호성이 실제로 무는 표면이라 주석이 아니라 거기에 적었습니다.

세 워크플로가 같은 표시 이름을 쓰는 것 자체는 저장소 전역 관례 문제이고 이 PR 범위가 아닙니다.

점유에 대한 절제를 수치로 보강합니다

본문의 "개수 변화가 조직 적체나 대기시간 감소를 뜻하지 않는다" 는 절제를 peer 1 의 측정이 뒷받침합니다.

범위: strix.yml 안에서, .github 저장소, run 20건 / 잡 100건, runner_id 기준
  strix 잡          개수 20%  ·  러너 시간 98.7%
  게이트·보조 4잡   개수 80%  ·  러너 시간  1.3%

메타데이터 잡을 줄여도 점유는 사실상 안 변합니다. 줄어드는 것은 슬롯 획득 횟수뿐이고, 그게 정확히 이 PR 이 주장하는 바입니다.

이 수치의 범위를 그대로 적습니다 — org 전역이 아니고 strix.yml 한 워크플로 안의 분포이며, 다른 워크플로에 일반화되지 않습니다.

게이트 (병합 트리)

2988 passed · coverage 100% 0 miss · interrogate 100%

테스트 수 예측은 여기서 쓸 수 없습니다 — 파라미터화 4개가 추가돼 def 개수와 passed 가 어긋납니다. 그래서 그 예측이 원래 잡으려던 것(중복)을 직접 검사했습니다: 파일 내 중복 테스트 이름 0, 수집된 중복 node-id 0.

인용된 논문에 대해

docs/papers/…granite….pdf 는 이 저장소의 첫 바이너리 PDF 라 조직 규약(허용 라이선스만, 추가 전 SPDX 확인)에 걸리는지 확인했습니다. doctoring 기록이 이미 CC BY 4.0, SHA-256, DOI 를 명시하고 있고 재배포를 허용하는 라이선스입니다. 그리고 그 기록이 논문은 이 변경의 큐 지표를 입증하지 않는다고 스스로 범위를 긋고 있습니다.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 23 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 00b38f8a-fde7-4e96-85e4-b7e1871516ff

📥 Commits

Reviewing files that changed from the base of the PR and between dd0b96f and 402aca1.

⛔ Files ignored due to path filters (1)
  • docs/papers/granite-granular-runtime-enforcement-github-actions-permissions-v1.pdf is excluded by !**/*.pdf
📒 Files selected for processing (28)
  • .github/workflows/agent-mention-noema-dispatch.yml
  • .github/workflows/agent-mention-opencode-dispatch.yml
  • .github/workflows/audit-central-ruleset.yml
  • .github/workflows/strix.yml
  • CHANGELOG.md
  • docs/doctoring/strix-metadata-admission-consolidation.md
  • docs/product-technical-gap-baseline.md
  • scripts/ci/audit_org_codeql_coverage.py
  • scripts/ci/pr_review_merge_scheduler_core.py
  • scripts/ci/test_strix_quick_gate.sh
  • tests/test_agent_mention_downstream_idempotency.py
  • tests/test_agent_mention_queue_isolation.py
  • tests/test_agent_review_runtime_quality_consolidation.py
  • tests/test_audit_org_codeql_coverage.py
  • tests/test_bootstrap_codeql_pull_requests.py
  • tests/test_central_required_workflow_ruleset_audit.py
  • tests/test_close_empty_pr_queue_pressure.py
  • tests/test_codeql_scan_dispatch_workflow_contract.py
  • tests/test_docs_only_pr_runner_admission.py
  • tests/test_exact_artifact_quality_single_runner.py
  • tests/test_noema_review_gate.py
  • tests/test_noema_token_lifetime_stale_run_contract.py
  • tests/test_opencode_required_rerun_capacity.py
  • tests/test_opencode_required_verdict_regression.py
  • tests/test_pr_review_fix_scheduler_source_pin.py
  • tests/test_pr_review_merge_scheduler.py
  • tests/test_repository_metadata_workflow_pages.py
  • tests/test_required_workflow_queue_contract.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

현재 head 3ae7c80d36b42fa6f8d93bbd4b9620da13032b29의 수동 리뷰를 요청합니다. 메타데이터 job 통합의 admission 순서, stale-head 차단, output 전달, 권한 분리와 기존 scanner 동작 보존을 확인해 주세요. 전체 pytest 2966 passed/1 LLVM 도구 부재 skip/exit 0, 관련 192개 -W error 시험과 actionlint는 통과했습니다. 현재-head 긴 Strix quickgate는 아직 실행 중이며, hosted 성공이나 병합 준비 완료를 주장하지 않습니다. Draft 및 보호 규칙은 유지합니다.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '== HEAD and merge base =='
git rev-parse HEAD
base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main)"
printf 'merge-base=%s\n' "$base"

echo '== Changed files =='
git diff --name-status "$base"...HEAD

echo '== Workflow structural map =='
ast-grep outline .github/workflows/strix.yml --items all 2>/dev/null || true

echo '== Workflow: job declarations, dependencies, conditions, permissions, and outputs =='
rg -n -C 4 '^(name:|on:|permissions:|concurrency:|  (changed-scope|admit-current-head|metadata|scanner|scan)[A-Za-z0-9_-]*:|    needs:|    if:|    permissions:|    outputs:|      (code|admitted|head|sha)[A-Za-z0-9_-]*:|      id:|      run:)' .github/workflows/strix.yml

echo '== Workflow diff =='
git diff --unified=80 "$base"...HEAD -- .github/workflows/strix.yml

echo '== Changed test and documentation maps =='
fd -t f . tests | rg '(docs_only_pr_runner_admission|required_workflow_queue_contract|required_review_runner_image_contract|required_security_runner_image_contract|strix_recovered_transient_sanitizer|contextual_orchestrator_review_runtime_preflight|strix-metadata-admission-consolidation)'

Length of output: 30633


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '== Combined metadata job =='
sed -n '90,215p' .github/workflows/strix.yml

echo '== Scanner declaration and first steps =='
sed -n '330,455p' .github/workflows/strix.yml

echo '== All consolidated-output references =='
rg -n -C 3 'needs\.(changed-scope|admit-current-head)\.outputs|steps\.(admission|scope)\.outputs|target_repository|pr_number|admitted|deps' .github/workflows/strix.yml

echo '== Scanner job diff only =='
base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main)"
git diff --unified=12 "$base"...HEAD -- .github/workflows/strix.yml |
  sed -n '/^   strix:/,/^   publish-manual-pr-evidence-status:/p'

echo '== Changed admission and queue-contract tests =='
git diff --unified=100 "$base"...HEAD -- \
  tests/test_docs_only_pr_runner_admission.py \
  tests/test_required_workflow_queue_contract.py

echo '== Relevant test definition map =='
ast-grep outline tests/test_docs_only_pr_runner_admission.py --items all 2>/dev/null || true
ast-grep outline tests/test_required_workflow_queue_contract.py --items all 2>/dev/null || true

Length of output: 74190


@seonghobae 수동 검토를 완료했습니다. 유효한 finding은 없습니다.

확인한 항목은 다음과 같습니다.

  • changed-scopeVerify event metadata against the live pull request를 먼저 실행합니다.
  • Classify changed pathssteps.admission.outputs.admitted == 'true'일 때만 실행합니다.
  • stale head는 admitted=false를 유지하고 파일 분류를 실행하지 않습니다.
  • strixcode == 'true'admitted == 'true'를 모두 요구합니다.
  • 통합 job은 contents: readpull-requests: read만 사용합니다.
  • strix job은 OIDC와 상태 게시 권한을 별도로 유지합니다.
  • scanner 변경은 needs와 admission output 참조를 통합 job으로 옮긴 부분뿐입니다.
  • 추가된 계약 시험은 native PR, exact dispatch, stale head, fail-open 파일 목록 처리, output 전달, 실행 순서를 다룹니다.

Hosted Checks와 실행 중인 긴 Strix quick gate는 이 수동 검토의 범위에 포함하지 않았습니다. Draft 상태와 보호 규칙 유지도 확인했습니다.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@seonghobae
seonghobae marked this pull request as ready for review September 6, 2026 12:59

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@seonghobae

Copy link
Copy Markdown
Contributor Author

현재 head c31aa0234dcae6bf1b7e27c55be7bdc87d4fe507의 갱신된 PR 본문을 실제 Edge 데스크톱 화면에서 검수했습니다. 최신 검증 제목·목록·긴 SHA 줄바꿈·이전 검증 이력 구분에 관찰된 잘림이나 겹침은 없습니다. Project 카드의 In Progress도 확인했습니다. 본문의 Project #1이 저장소 PR #1로 자동 링크되는 문제는 실제 프로젝트 URL로 수정했습니다.

CodeRabbit의 녹색 상태 옆 실제 설명은 Review rate limited이며 승인으로 사용하지 않습니다. 새 head의 필수 Checks는 대기 중입니다. 이번 검수는 PR 본문의 데스크톱 렌더링에 한정되며 모바일·제품 UI·접근성 전범위나 보호 병합 증거가 아닙니다.

seonghobae and others added 5 commits September 6, 2026 23:44
)

The contract asserted that expressions appear in the concurrency block, which
the block's own documentation satisfies while the key says something else. Slice
to the group's value with comments stripped so the assertion tests the key.

Author: separate session. Verified independently: mutant controls 7/7 caught on
the branch, 7/7 missed on main, adversarial helper inputs leak no comments.

Merger verification (this session): head matched the verified SHA exactly, 0
behind main, merge tree identical to the branch tree, tests-only (2 files, no
path outside tests/). Own discriminating control -- collapse the group key to
the repository alone while moving the expressions into the comment beside it:
main 58 passed (misses it), branch 1 failed (catches it). Full gate on the merge
tree: 2958 passed, 1 skipped, coverage 100%, interrogate 100%.

Merged under the standing chicken-and-egg authorization: the required contexts
CodeQL compatibility analysis (actions)/(python) cannot be produced for a
code-touching pull request in this repository -- codeql-scan-dispatch.yml has
never succeeded (0 of 1931) because its actor allowlist admits no identity that
dispatches it. See #1929.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#1964)

Move the concurrency group from job level to workflow level in both agent-mention
dispatch workflows so a superseded mention is coalesced while it is still queued,
instead of holding its queue slot until a runner frees up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ck (#1975)

The helper sliced from `permissions:`, so it raised IndexError on the two
workflows that declare permissions first, and it returned a folded key's raw
newlines rather than the value YAML produces. Nine of twenty-nine
workflow-level keys are folded, including every required review workflow.

Author and verifier were separate sessions. Merger verification (this session,
independent runs): head matched GitHub exactly, 0 behind main, merge tree
identical to the branch tree, tests-only. Exact-match against a yaml oracle
across every workflow: 29 match, 0 mismatch, 0 exception. Seven adversarial
inputs pass, three of them designed here rather than reused -- a comment
quoting `group:` before the real key, a job-level concurrency block appearing
first in the file, and literal `|`/`|-` scalars, which are refused rather
than silently folded into a value YAML never produces. Two-way control on the
live hole: flipping noema-review.yml's cancel-in-progress to false behind a
comment passes on main (2961) and fails here. pr-review-autofix.yml's
deliberate cancel-in-progress: false is preserved. Gate: 2964 passed, 1
skipped, coverage 100%, interrogate 100%.

Bypass basis stated plainly: this change is tests-only and does not itself
unblock anything, so it does not meet the narrower 'the PR's own diff edits
review-pipeline files' reading recorded in
docs/product-technical-gap-baseline.md. It is merged under this session's
standing instruction to fix the queue, which names verifying
`cancel-in-progress: true` as part of that work. The required CodeQL contexts
remain unreachable for any code-touching PR here (#1929).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ot materialize (#1973)

Materializing the PR merge tree is a precondition of coverage-source-tree, so a
conflicting head can only produce a failed dispatch. The guard returns before
review_dispatch_admitted, preserving the bounded admission budget for a PR a
review could actually finish. UNKNOWN is deliberately not blocked.

Authored, verified and merged by three separate sessions.

Merger verification, my own runs: head matched GitHub, 0 behind main, merge tree
identical to the branch tree. Against current main the diff is 2 files, 206
insertions, 0 deletions; the production change is 29 added lines and nothing
removed. The guard sits at line 3691 and review_dispatch_admitted at 3706, so
the budget is preserved; 6 call sites handle the new return value. Negative
control, removing only the 15 guard lines: exactly 2 tests fail --
test_review_dispatch_skips_a_head_whose_merge_tree_cannot_materialize and
test_review_dispatch_reads_the_rest_merge_state_not_only_graphql -- asserting
merge_conflict against a received dispatched. Gate: 2968 passed, 1 skipped,
coverage 100% (13196 statements, 0 missed), interrogate 100%.

The cited measurement was corrected before merge. It read '20 dispatches across
80.5 hours'; two sessions independently recounted .github#1529 as 27 dispatches
across 100.8 hours with zero successes (20 cancelled, 7 failed). The original
figure came from a run window that silently truncated before the pull request
existed. The comment now carries the corrected numbers and the window.

Authorization is this session's standing instruction to clear the queue, which
lives outside this repository's text -- as docs/product-technical-gap-baseline.md
itself records after a 2026-09-01 correction, and that file is annotated '(not
merge authorization)' at every entry point. Corroborating rather than
authorizing: this PR's own diff edits scripts/ci/ review-pipeline code, so it
cannot validate itself across the pull_request_target trust boundary, which is
the conservative condition that document records one earlier pass imposing on
itself. On this head all 12 required contexts are unsatisfiable: 9 never
reported, 3 queued, 0 passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…fails (#1979)

strix.yml's workflow-level cancel-in-progress was guarded by nothing: its own
test asserted the string as a substring, and the list in
test_required_pull_request_workflows_cancel_superseded_runs did not include it.

Authored and verified by separate sessions. Merger verification, my own runs:
head agreed across three paths (local ref, ls-remote, PR head) after the author
hit a push/PR-creation mismatch on this branch; 0 behind main; merge tree
identical to the branch tree; 13 files, all under tests/. Assertion forms: 24
helper calls, 4 line-anchored regexes, 19 assertions replaced.

Two-way control, comment out strix.yml's flag and set it false: main reports
2968 passed and does not catch it; this branch fails 2 tests. Gate on the branch
tree: 2968 passed, 1 skipped, coverage 100% (13196 statements, 0 missed),
interrogate 100%.

Four workflows are deliberately left uncontracted because no test states their
intended value, and writing one would invent policy. One of them,
scheduled-security-scan.yml, does produce the required context 'Detect CodeQL
languages' -- which strengthens rather than weakens that choice: its
cancel-in-progress: true with a github.ref-shared group is what starved it to
228 cancellations and 0 completions over 2026-09-01..09-05, so the correct value
is an open policy question on #1800, not a contract to fix here.

Authorization is this session's standing instruction to clear the queue, which
lives outside this repository's text. Corroborating, not authorizing: on this
head none of the 12 required contexts can pass -- the CodeQL pair is
unreachable while codeql-scan-dispatch has never succeeded (#1929).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode reviewed the current-head mergeability evidence and changed-file flow before approval, then found merge conflicts on the affected path.

Findings

1. HIGH Merge Conflict Guidance - Resolve the PR branch against the latest base branch

  • Problem: GitHub reports mergeStateStatus DIRTY for this pull request.
  • Root cause: Branch codex/strix-metadata-single-job cannot be merged cleanly into main; the changed-file flow below shows which review/runtime path is blocked by the conflict.
  • Fix: Merge or rebase the latest main into codex/strix-metadata-single-job, resolve conflict markers in the PR branch, rerun the focused checks, and push the same branch.
  • Repair commands:
gh pr checkout 1962 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main  # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:codex/strix-metadata-single-job
# rebase path only: git push --force-with-lease origin HEAD:codex/strix-metadata-single-job
  • Regression test: Keep OpenCode approval gated on mergeability so model-output failures cannot approve a conflicted PR.

Merge Conflict Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: strix.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["actionlint plus required checks"]
  Evidence --> S2["Repository file: CHANGELOG.md"]
  S2 --> I2["repository behavior"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["required checks"]
  Evidence --> S3["Docs: strix-metadata-admission-consolidation.md (3 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V3["docs review"]
  Evidence --> S4["CI script: test_strix_quick_gate.sh"]
  S4 --> I4["review and security gate shell path"]
  I4 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V4["bash -n plus Strix self-test"]
  Evidence --> S5["Test: test_docs_only_pr_runner_admission.py (2 files)"]
  S5 --> I5["regression suite"]
  I5 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V5["targeted test run"]
Loading
  • Result: REQUEST_CHANGES
  • Reason: mergeStateStatus is DIRTY; mergeable is CONFLICTING.
  • Head SHA: c31aa0234dcae6bf1b7e27c55be7bdc87d4fe507
  • Workflow run: 34038910229
  • Workflow attempt: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: strix.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["actionlint plus required checks"]
  Evidence --> S2["Repository file: CHANGELOG.md"]
  S2 --> I2["repository behavior"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["required checks"]
  Evidence --> S3["Docs: strix-metadata-admission-consolidation.md (3 files)"]
  S3 --> I3["operator or user guidance"]
  I3 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V3["docs review"]
  Evidence --> S4["CI script: test_strix_quick_gate.sh"]
  S4 --> I4["review and security gate shell path"]
  I4 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V4["bash -n plus Strix self-test"]
  Evidence --> S5["Test: test_docs_only_pr_runner_admission.py (2 files)"]
  S5 --> I5["regression suite"]
  I5 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V5["targeted test run"]
Loading

@opencode-agent

opencode-agent Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

seonghobae and others added 6 commits September 7, 2026 03:16
…an workflows (#1980)

python-security.yml and sast-semgrep.yml had only the presence of
cancel-in-progress asserted, so flipping it to false passed the whole suite.
This pins the value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…review runs (#1983)

`active_review_run_refs` matched a workflow run's `name` exactly against the
review workflow aliases. But eight workflows in this repository define
`run-name:`, and that set contains every workflow whose runs this matcher looks
for -- `opencode-review.yml` ("Required OpenCode Review"),
`opencode-review-dispatch.yml` ("OpenCode Review Dispatch") and `strix.yml`
("Strix Security Scan"). For such a workflow GitHub reports the *rendered* run
name in `name` -- the same string as `display_title`, e.g.

    OpenCode Review Dispatch #834e748ee6...

Sampled 2026-09-07: 100 of 100 opencode-review-dispatch runs carry that form
and none carries the bare workflow name. So the exact match dropped every
production dispatch run at this line, before the `event == "repository_dispatch"`
branch immediately below that exists to read them. Two consequences:

  * `already_running` never suppressed a same-head repeat. .github#1529 took 27
    dispatches on one unchanged head over 100.8 hours; each new run's creation
    preceded the previous run's cancellation by about three seconds, so the
    previous run was demonstrably still active when the check ran and did not
    see it.
  * `stale` never populated, so older-head central runs were never cancelled.

A first count of the live queue said "20 duplicates of 45 active runs" and was
wrong: it grouped by repository and PR without the workflow, so runs of
different dispatch workflows on one PR were counted as duplicates of each other.
Regrouped by (workflow, repository, PR):

    active repository_dispatch runs, queued + in_progress   33
      codeql-scan-dispatch.yml      25 runs / 12 keys / 13 same-head duplicates
      opencode-review-dispatch.yml   5 runs /  5 keys /  0 duplicates
      pr-review-autofix.yml          3 runs /  3 keys /  0 duplicates

So the workflows this matcher governs show no live duplication at this instant.
The harm this fix addresses is the historical chain on .github#1529 and a
suppression that has never once fired, not a backlog visible right now. The 13
duplicates all belong to CodeQL Scan Dispatch, which this matcher does not
govern; that workflow also defines `run-name:`, which makes it a separate lead
rather than evidence for this change. Reviving stale cancellation is separately
safe: of 163 non-terminal central runs, 49 are review or dispatch
kind and 4 become cancellable, all of them subjects that no longer exist (3
closed or merged PRs, 1 moved head).

The fix is confined to the run comparison. `OPENCODE_WORKFLOW_NAMES` is
unchanged, because its other consumer compares a *workflow* object's name,
which is genuinely bare. `active_review_run_refs` has exactly two call sites,
OpenCode's and Strix's, so both are fixed here; the Strix side is pinned by its
own test so a later narrowing to the OpenCode aliases cannot silently reopen
half of it.

This is one instance of a class, and the file already contains the stable form.
`run.name` is compared as an identifier at four places -- `:1250`, `:3198`,
`:3254` (this one) and `:3783` -- while `:3060` keys on
`run.get("workflow_id") or run.get("path") or run.get("name")`, which cannot be
rewritten by a `run-name:`. `:1250` in particular feeds the REST fallback's
workflow-level policy boundary and would see a rendered title where it expects a
workflow name. Fixing the whole class means moving the callers from display
names to paths, which also touches how `dispatch_title_prefixes` is built, so it
is deliberately left out of this change; .github#1941 is the same root seen from
the `display_title` side. Recorded here so the next reader does not rediscover
it as a fifth instance.

Note the new behaviour this enables: while a same-head central run is active,
a repeat is now suppressed. A run that never terminates would therefore hold
the PR, where before the check simply never fired.

The existing fixture sets a bare `name` beside a rendered `display_title`, a
payload GitHub never emits for a `run-name:` workflow, which is why 100 percent
line coverage of that branch never revealed that production could not reach it.

Developer experience: the scheduler's same-head suppression and stale-run
cancellation work against real payloads instead of a shape only the tests
produce.
User experience: a pull request stops accumulating duplicate concurrent review
runs that cancel each other, so a review that starts can finish.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ree cannot materialize (#1973)" (#1985)

This reverts commit ad0779b.

I wrote that guard and its justification is false. It blocked every OpenCode
dispatch on a DIRTY/CONFLICTING head on the grounds that "a conflicting head can
only produce a failed dispatch". The run object does conclude failure, but the
review is published anyway: the reviewer reads the pull request diff, not a
merge tree -- "Coverage is a separate gate", in the review's own words -- and
only `coverage-source-tree` needs the merge commit.

Measured on .github#1529, the PR that motivated the guard: the last of its 27
dispatches published a 2404-character review at 2026-09-05T19:40:58Z. That
review body carries its own run id, 33969161561, which concluded failure at the
receipt gate nine seconds later. It is the only OpenCode review that head has
(4 reviews total, 1 by opencode-agent[bot]) and it is why the PR now reads as
reviewed. The guard would have discarded it. The pattern is not unique to
#1529: .github#1555 is CONFLICTING right now and its current head carries a
2019-character CHANGES_REQUESTED review from the same reviewer.

I reached "produced nothing" by reading run conclusions, which are roll-ups
that cannot name what a run did.

The cost the guard claimed to protect is also wrong by two orders of magnitude.
Across all 27 dispatches of that head, 109 jobs: 27 were allocated a runner for
0.24 h in total, 82 never were, and the rest of the elapsed time was queue
residency. Blocking the whole chain would have saved fourteen minutes of runner
time.

The repeat itself is a real problem, and it is fixed at its cause rather than
here. `active_review_run_refs` matched a run's `name` exactly against the
review workflow aliases, but the central review workflows define `run-name:`,
so GitHub sends the rendered title in that field and every dispatch run was
filtered out before the check could see it -- `already_running` never fired.
With that repaired, a conflicting head receives one dispatch and the next is
suppressed while it runs, which is the outcome this guard was reaching for
without discarding the review.

Developer experience: the scheduler no longer prints a skip reason that asserts
an outcome contradicted by the runs it cites.
User experience: a conflicting pull request receives a review and repair
guidance, instead of repair guidance alone.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…etector (#1987)

Two unrelated audits share one job in audit-central-ruleset.yml, and the
ruleset step runs first. It has exited 1 since at least 2026-09-04 on
owner-configured governance drift:

    ERROR: exactly two approving reviews are not required
    ERROR: last-push approval protection is disabled
    FAIL: ruleset 18156473 has 2 governance drift reason(s)

Live values on ruleset 18156473 today are `required_approving_review_count: 1`
and `require_last_push_approval: false`, against the 2 and true the audit
asserts. Both are owner-configured settings, and this change does not touch
either them or the audit's expectations: with every session sharing one GitHub
identity and unable to approve another's pull request, a two-approval
requirement may well have been relaxed deliberately, in which case the stale
side is the assertion rather than the configuration. Deciding that is an owner
call.

What is not an owner call is the collateral damage. Because the failure exits a
shared job, the two steps below it never ran: the CodeQL coverage detector and
the backlog-38 bootstrap that opens CodeQL setup pull requests. So the detector
that would have reported a coverage gap has been dead for days, and its
workflow was red the whole time for an unrelated reason -- red status, wrong
subject, and no signal about coverage either way.

The coverage step now carries `if: always()`. It builds its own repository list
into its own temp file and the step above exports nothing to GITHUB_ENV or
GITHUB_OUTPUT, so it has no data dependency to lose; the job still fails
overall. The bootstrap step deliberately does not get the same guard, because
it opens pull requests and running a mutation after an unexplained upstream
failure is a different decision from running a read-only detector. A contract
test pins both halves.

The detector also needed a correction of its own. It accepted
`default_setup_state == "configured"` as coverage, but a repository can report
`configured` with an **empty** `languages` list, which scans nothing. Measured
2026-09-07: life-os, aFIPC and inkspan all report that shape, and life-os has
zero CodeQL analyses of any language while codeql-pr.yml still runs on every
pull request head. The control holds in both directions -- html4tree, naruon
and wardnet have non-empty language lists and do have `dynamic/` analyses for
exactly those languages. The audit workflow now collects `languages` alongside
`state`, the predicate requires a non-empty list, and a payload missing the new
key fails closed rather than falling back to the state alone.

Gaps of the two kinds are reported as different sentences, because they need
different fixes: enable languages on an existing setup, versus set coverage up
at all.

A reviewer then asked whether the newly-unblocked step could run and audit zero
repositories while still passing, and one layer below the fix it could:

    $ echo '[]' | python3 scripts/ci/audit_org_codeql_coverage.py
    PASS: all 0 repositories have real CodeQL coverage        exit 0

The calling workflow already refuses that -- its sentinel check requires known
private repositories to appear in the enumeration, which an empty list fails --
but the script is directly runnable against a JSON path or stdin, so the guard
did not cover every entry point. `main` now refuses an empty payload. This is
the same vacuous-pass shape as the `configured`-with-no-languages case above,
one level down, which is where it was found: a pass that examines nothing is
not a pass.

The step's independence from the failing step above it is established
mechanically rather than by reading the YAML. Steps can only share state
through `GITHUB_ENV`, `GITHUB_OUTPUT`, `GITHUB_PATH` or files, since each `run:`
is a separate shell. The ruleset step uses none of those channels, and the two
steps' `$RUNNER_TEMP` paths are disjoint: `central-required-workflow-*`,
`ruleset-probe-*` and `stacked-opencode-ruleset.*` against `codeql-coverage-*`,
`codeql-analysis-*` and `codeql-default-setup-*`.

Developer experience: a governance-drift failure no longer hides an unrelated
security-coverage result, and the audit says which of the two problems a
repository has.
User experience: a repository whose CodeQL setup is switched on but scanning
nothing is reported as uncovered instead of counted as covered.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…1989)

#1987 taught the CodeQL coverage audit to refuse an empty payload, because
"PASS: all 0 repositories have real CodeQL coverage" reads as success over a
run that examined nothing. Reviewing that change, host 2 fed it a payload of a
single *archived* repository:

    PASS: all 1 repositories have real CodeQL coverage        exit 0

Non-empty, so it clears the new guard, and archived repositories are then
legitimately skipped by the coverage loop. The subject set is empty again, by a
different route -- the same defect the guard was added to close, one input shape
sideways from the one it checks.

The count that matters is what the audit examined. `auditable_repositories()`
is now the single place that decides which repositories are in scope, shared by
the loop and by the guard, so the two cannot drift apart when the archived rule
changes. The guard refuses an empty examined set however it became empty, and
the PASS line reports the examined count rather than the supplied one, so an
organization of nothing but archived repositories can no longer be reported as
fully covered.

Developer experience: the pass line's number is the number of repositories the
run actually checked.
User experience: an audit that examined nothing says so instead of passing.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
One conflict, in `tests/test_required_workflow_queue_contract.py`, where two
changes crossed in the same assertions: main moved them onto
`workflow_level_concurrency_group` (#1975) while this branch renamed the job
dependency after merging `admit-current-head` into `changed-scope`. Resolved by
keeping both — the helper reads the real key, and the assertions name the merged
job:

    assert "needs: [changed-scope]" in strix_job
    assert "needs.changed-scope.outputs.admitted == 'true'" in strix_job
    assert "strix-security-scan-${{" in group_value          # helper, not the raw slice

Also specialises the merged job's display name. The id stays `changed-scope`
because CLAUDE.md:165 names that id as the required-workflow skip pattern, but
sast-semgrep.yml and security-scan.yml publish a job displayed identically that
does NOT admit, and telling them apart from a check list alone cost real time
today. The check list is where that ambiguity bites, so the divergence is stated
there rather than in a comment only a file-opener sees. peer 1 proposed the
narrowed form after measuring that the three workflows already share the display
string, so the collision predates this change.

Gates on the merge tree: 2988 passed, coverage 100%, interrogate 100%. The test
count is not a useful prediction here (4 parametrize decorators added), so
duplicates were checked directly: 0 duplicate test names per file, 0 duplicate
collected node ids.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant