docs(claude): record the review-dispatch handshake and two measurement traps

[seonghobae]

Three gotchas found while diagnosing #1923's failing checks. Each cost real time to derive and none is recoverable from the repository without re-deriving it.

1. A red CodeQL or opencode-review check is half a handshake, not a finding

CodeQL compatibility analysis (…) and opencode-review hand their work to a dispatch workflow, then fail on purpose — in about 8 seconds — to release the runner rather than hold it idle, expecting a rerun once the dispatch publishes a verdict. The annotations say so:

CodeQL scan dispatched. The dispatch workflow will rerun this exact
failed CodeQL job after publishing its terminal verdict.

No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head.
The dispatch workflow will rerun this failed job after publishing an
authenticated exact-head verdict.

So a red check here means the second half did not happen. Both dispatches are currently rejected by the identity allowlist in #1929, which makes this state expected on every pull request and not fixable from one. The tell is duration: a completion measured in seconds is a handshake; a scan that really ran takes minutes.

2. repository_dispatch runs pin head_sha to the default branch

Every dispatch created while main sits at one commit carries that commit, so "N runs, one distinct head_sha" looks like runaway amplification when it only means main stood still. I nearly published an amplification figure from exactly that.

Same class as pull.head.repo.pushed_at being the repository's last push rather than the branch's — a nested field describing the container, not the subject. Cheap tell for the whole class: if distinct objects share an identical second-resolution value, the field belongs to something above them.

3. A failing step's name is a roll-up, like a run conclusion

codeql-scan-dispatch has reported the same 3: Bind workflow inputs to live organization pull request metadata across two unrelated causes:

before #1926   The template is not valid … (Line: 149, Col: 28): A sequence was not expected
after  #1926   repository_dispatch authorization rejected actor=…

Splitting failures by step name merges those and hides a fix that already landed. Split by annotation message. The entry also records that opencode-review names jobs in two workflows (opencode-review.yml's required gate and opencode-review-dispatch.yml), so a census keyed on check-run name alone mixes populations.

Notes

No CHANGELOG entry, deliberately. This is documentation only, and every entry is prepended at line 1, where it conflicts with every other open pull request carrying one. #1961 measures that cost: eight collisions in nineteen hours on a single branch, with resolution slower than the interval between invalidations.

Gates: 2961 passed / 1 skipped / 21 subtests. No test reads CLAUDE.md, so there is no doc-contract exposure.

The failing checks on this pull request will be the same handshake described in item 1 — which is, unavoidably, its own illustration.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 21 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 2beed2d0-d80b-4e19-a65d-900a0daa6c08

📥 Commits

Reviewing files that changed from the base of the PR and between 6e014c9 and f57d64e.

📒 Files selected for processing (1)

• CLAUDE.md

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

My own prediction in this pull request's description was wrong, and correcting it improved the entry.

I wrote that this branch's checks would fail with the handshake described in item 1 — "unavoidably, its own illustration." They did not:

CodeQL compatibility analysis (actions)   success   2s
CodeQL compatibility analysis (python)    success  22s

The step list explains it. On this branch, which touches only CLAUDE.md:

2. skipped   Request current-head CodeQL scan dispatch
3. skipped   Release runner or enforce current-head CodeQL verdict
   -> job success

On #1923, which carries a .py change:

2. success   Request current-head CodeQL scan dispatch
3. failure   Release runner or enforce current-head CodeQL verdict
   -> job failure

The handshake is scope-gated. My entry stated it unconditionally, which would have taught a future session to expect a red check where green is correct — the opposite of the confusion the entry exists to prevent.

Amended in 8f3a… to record the sharper rule: a red check means the scope gate opened and the handshake began; a fast green one means it never started. Gates re-run: 2961 passed / 1 skipped.

Worth noting how this surfaced: the entry was falsifiable, I stated the prediction publicly, and the pull request's own CI refuted it within the hour. Had I written the claim without the prediction, it would have merged as written.

[seonghobae]

Prediction confirmed on this branch's own CI. The entry stated, before these ran, that a documentation-only pull request should show CodeQL green and opencode-review red:

CodeQL compatibility analysis (actions)   success     2s
CodeQL compatibility analysis (python)    success     2s
opencode-review                           failure     5s
noema-review                              failure   670s

Both halves hold, and the durations independently confirm the mechanism the entry describes:

• CodeQL green in 2 s — the scope gate stayed shut, both inner steps skipped, nothing dispatched.
• opencode-review red in 5 s — not scope-gated; its Fail closed without a current-head OpenCode verdict step carries no if:, so it runs on every admitted head and fails in seconds. That is the handshake, not a review of this change.
• noema-review red at 670 s — eleven minutes is not a handshake. That one genuinely tried and failed upstream, which is the contrast the entry uses to make the duration tell usable: seconds means the fail-fast path, minutes means something actually ran.

How this entry reached correctness

Worth recording, because the process mattered more than the content:

v1  handshake is unconditional            -> refuted by this PR's own green CodeQL checks
v2  fixed CodeQL, implied opencode too    -> refuted by reading opencode-review.yml
v3  each check's gating stated separately -> confirmed above

Two of the three revisions came from the claim being falsifiable and publicly predicted. Had the behaviour simply been described without a prediction attached, v1 would have merged and been wrong in the repository permanently — teaching every future session to expect red where green is correct.

The second refutation is worth noting separately: it came from reading the workflow source, not from waiting for CI. When the feedback loop is a saturated queue measured in hours, the workflow file answers the same question in seconds.

Gates on the current head: 2961 passed / 1 skipped / 21 subtests.