From a14946739b3ac4efd57ce1b0cfd16ad17c38e2a6 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 7 Sep 2026 00:43:40 +0900 Subject: [PATCH 1/3] docs(claude): record the review-dispatch handshake and two measurement traps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three gotchas found while diagnosing #1923's failing checks, none of which were discoverable from the repository without re-deriving them: A red `CodeQL compatibility analysis (…)` or `opencode-review` check is half a handshake rather than a finding. Those jobs dispatch their work and fail deliberately in ~8 seconds to release the runner, expecting to be rerun after the dispatch publishes a verdict; the annotations state this verbatim. Since both dispatches are currently rejected by the identity allowlist in #1929, the state is expected on every pull request and is not fixable from the pull request. `repository_dispatch` runs pin `head_sha` to the default branch, so per-pull-request fan-out cannot be computed from it -- "N runs, one distinct head_sha" means `main` stood still. Same class as `pull.head.repo.pushed_at` describing the repository rather than the branch. A failing step's name is a roll-up: `codeql-scan-dispatch` reported the same step 3 across a template parse error (pre-#1926) and an authorization rejection (post-#1926), so splitting failures by step name merges causes and hides a landed fix. Split by annotation message. Also notes that `opencode-review` names jobs in two different workflows. No CHANGELOG entry: this is documentation only, and every entry lands at line 1 where it conflicts with every other open pull request carrying one (measured in #1961 -- eight collisions in nineteen hours on one branch). Gates: 2961 passed / 1 skipped / 21 subtests. No test reads CLAUDE.md. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 30db1fc23b..b27569d7da 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -211,6 +211,35 @@ repeatable compile command. to the branch at all — so re-run the merge yourself immediately before resolving and treat any earlier measurement, including your own from minutes ago, as expired. Resolving against a stale smaller scope silently leaves conflicts unhandled. +- **A red `CodeQL compatibility analysis (…)` or `opencode-review` is half a handshake, not a + finding.** These jobs hand their work to a dispatch workflow and then fail *on purpose* — in about + 8 seconds — to release the runner instead of holding it idle, expecting to be rerun once the + dispatch publishes a verdict. The annotations say so verbatim: `CodeQL scan dispatched. The + dispatch workflow will rerun this exact failed CodeQL job after publishing its terminal verdict`, + and `No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. The dispatch + workflow will rerun this failed job…`. So a red check here means *the second half did not happen*, + which is almost never fixable from the pull request. Read the annotation before changing any code: + a completion measured in seconds is a handshake, and a scan that really ran takes minutes. Both + dispatches are currently rejected by the identity allowlist tracked in #1929, so this state is + expected on every pull request until that is decided. +- **`repository_dispatch` runs pin `head_sha` to the default branch, so it cannot identify the pull + request they belong to.** Every dispatch created while `main` sits at one commit shares that + commit, which makes "N runs, 1 distinct `head_sha`" look like runaway amplification when it only + means `main` stood still. Do not compute per-pull-request fan-out from `head_sha` on these runs. + This is the same class as `pull.head.repo.pushed_at` being the *repository's* last push rather + than the branch's — a nested field that describes the container, not the subject. A cheap tell for + the whole class: if several distinct objects carry an identical second-resolution value, the field + belongs to something above them. +- **A failing step's *name* is a roll-up, exactly like a run conclusion.** `codeql-scan-dispatch` has + reported `3: Bind workflow inputs to live organization pull request metadata` continuously across + two completely different causes — a workflow template parse error before #1926 (`The template is + not valid … (Line: 149, Col: 28): A sequence was not expected`) and an authorization rejection + after it (`repository_dispatch authorization rejected actor=…`). Splitting a failure population by + step name therefore merges causes and hides a fix that already landed. Split by the annotation + message. Relatedly, one check-run **name** can come from two different workflows: `opencode-review` + is both the required fail-closed gate in `opencode-review.yml` and a job in + `opencode-review-dispatch.yml`, so a census keyed on the name alone mixes them — narrow by + `run_id` or workflow file first. - **No test parses fenced code blocks.** The doc-contract tests match exact prose in specific files; none of them check Markdown structure, and `ARCHITECTURE.md` (five mermaid diagrams) is read by no test at all. A conflict resolution that splits a fenced block into two fragments therefore ships From 96309d833d1da136c3c7549119ae8b213c02505a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 7 Sep 2026 02:38:30 +0900 Subject: [PATCH 2/3] docs(claude): the CodeQL handshake is scope-gated, not unconditional My own pull request refuted the entry as first written. It claimed the CodeQL compatibility jobs always dispatch and then fail deliberately; on this CLAUDE.md-only branch both language jobs went green in 2 s and 22 s. Reading the steps shows why: `Request current-head CodeQL scan dispatch` and `Release runner or enforce current-head CodeQL verdict` both skip when nothing analyzable changed, and the job reports success. On #1923, which carries a .py change, step 2 succeeds and step 3 fails -- the handshake this entry describes. So the useful rule is sharper than the original: a red check means the scope gate opened and the handshake began; a fast green one means it never started. Recorded that distinction rather than the unconditional claim, which would have taught a future session to expect red where green is correct. Gates: 2961 passed / 1 skipped / 21 subtests. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index b27569d7da..4faf00de38 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -214,7 +214,13 @@ repeatable compile command. - **A red `CodeQL compatibility analysis (…)` or `opencode-review` is half a handshake, not a finding.** These jobs hand their work to a dispatch workflow and then fail *on purpose* — in about 8 seconds — to release the runner instead of holding it idle, expecting to be rerun once the - dispatch publishes a verdict. The annotations say so verbatim: `CodeQL scan dispatched. The + dispatch publishes a verdict. **This only happens when the change is in scope:** on a pull request + that touches nothing analyzable, `Request current-head CodeQL scan dispatch` and `Release runner or + enforce current-head CodeQL verdict` both *skip* and the job reports **success** in a couple of + seconds — verified on a `CLAUDE.md`-only pull request, where both language jobs went green in 2 s + and 22 s while the same jobs failed on a pull request carrying a `.py` change. So a **red** check + means the scope gate opened and the handshake began; a fast green one means it never started. The + annotations on the red path say so verbatim: `CodeQL scan dispatched. The dispatch workflow will rerun this exact failed CodeQL job after publishing its terminal verdict`, and `No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. The dispatch workflow will rerun this failed job…`. So a red check here means *the second half did not happen*, From f57d64e6789bf15810c6757ac80a0269d598abd6 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 7 Sep 2026 03:07:56 +0900 Subject: [PATCH 3/3] docs(claude): separate the scope-gated check from the one that is not The previous revision fixed the CodeQL claim but left the qualifier attached to a sentence covering both checks, which reads as though `opencode-review` were scope-gated too. It is not. Read from source rather than waiting for CI: `opencode-review-target` is gated only on `needs.admit-current-head.outputs.admitted == 'true'`, and its `Fail closed without a current-head OpenCode verdict` step carries no `if:` at all -- so it fails on every admitted head, documentation-only included. `codeql-pr.yml` skips both inner steps when nothing analyzable changed and reports success. The entry now states the expectation directly: on a docs pull request, CodeQL green and `opencode-review` red. That is a falsifiable prediction about this branch, which is the point. Gates: 2961 passed / 1 skipped / 21 subtests. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4faf00de38..41c55ff115 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -214,13 +214,17 @@ repeatable compile command. - **A red `CodeQL compatibility analysis (…)` or `opencode-review` is half a handshake, not a finding.** These jobs hand their work to a dispatch workflow and then fail *on purpose* — in about 8 seconds — to release the runner instead of holding it idle, expecting to be rerun once the - dispatch publishes a verdict. **This only happens when the change is in scope:** on a pull request - that touches nothing analyzable, `Request current-head CodeQL scan dispatch` and `Release runner or - enforce current-head CodeQL verdict` both *skip* and the job reports **success** in a couple of - seconds — verified on a `CLAUDE.md`-only pull request, where both language jobs went green in 2 s - and 22 s while the same jobs failed on a pull request carrying a `.py` change. So a **red** check - means the scope gate opened and the handshake began; a fast green one means it never started. The - annotations on the red path say so verbatim: `CodeQL scan dispatched. The + dispatch publishes a verdict. **The two differ in when they reach that point, and the difference is + worth knowing before you debug either.** `codeql-pr.yml` is *scope-gated*: on a pull request that + touches nothing analyzable, `Request current-head CodeQL scan dispatch` and `Release runner or + enforce current-head CodeQL verdict` both **skip** and the job reports success in seconds — measured + on a `CLAUDE.md`-only pull request, where both language jobs went green in 2 s and 22 s while the + same jobs failed on a branch carrying a `.py` change. `opencode-review.yml` is **not** scope-gated: + `opencode-review-target` is gated only on `needs.admit-current-head.outputs.admitted == 'true'`, and + its `Fail closed without a current-head OpenCode verdict` step carries no `if:` at all, so it fails + on every admitted head including a documentation-only one. Expect CodeQL green and `opencode-review` + red on a docs pull request; a red CodeQL check means its scope gate opened. The annotations say so + verbatim: `CodeQL scan dispatched. The dispatch workflow will rerun this exact failed CodeQL job after publishing its terminal verdict`, and `No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head. The dispatch workflow will rerun this failed job…`. So a red check here means *the second half did not happen*,