fix(noema): upload the sidecar stderr and preflight report when the verdict phase fails

[seonghobae]

Observability follow-up to #1935 / #1939, lane (b) of the split agreed with the orchestrator lane: ship the sidecar's diagnostics when a Noema verdict fails.

What changes

• .github/workflows/noema-review.yml: one step, right after Prepare Noema model verdict, if: failure() && env.PR_NUMBER != '', uploading strix_runs/contextual-orchestrator-sidecar.stderr.log and strix_runs/contextual-orchestrator-preflight.json as noema-sidecar-evidence (same pinned actions/upload-artifact@043fb46d… v7.0.1 that strix.yml uses, if-no-files-found: ignore, retention-days: 5). No permission change: the job already has what the action needs, and the job has no harden-runner egress allowlist to extend.
• tests/test_noema_orchestrator_workflow_contract.py: test_noema_review_uploads_sidecar_evidence_on_failure — step exists, condition, pin equal to Strix's, both paths, if-no-files-found: ignore, ordering Prepare < Upload < Refresh, and exactly one upload-artifact in the workflow. Negative control: with the step removed the test fails.
• CHANGELOG.md entry.

Why

A failed Noema run currently leaves artifacts=0. Run 33981136873 (#1940, post-#1939 sidecar) spent 3122 s walking the six ready routes twice each (≈ 548 s per candidate: orchestrator retry × client retry × 90 s per-recv timeout) and ended in HTTP Error 502, and the only diagnosis in the job log is the caller's one-line summary — the exact gap behind the "why 900 s / why failed" complaint (item 4). The sidecar's stderr and the preflight report were written to strix_runs/ and discarded with the runner.

Strix already publishes the same two files in strix-reports (verified on #1812's failed run: contextual-orchestrator-sidecar.stderr.log 2,279 B, contextual-orchestrator-preflight.json 3,607 B), so this adds no new exposure. The stderr file is the output of scripts/ci/sanitize_contextual_orchestrator_sidecar_stream.py, an allowlist sanitizer: only request_failed …, provider_discovery_failed …, preflight_route_rejected … lines with constrained field charsets and a few fixed diagnostic prefixes pass; everything else is counted as omitted_unstructured_lines.

Not claimed

• This does not add per-attempt route outcomes to the file. Those need a structured INFO line from the orchestrator (lane (a), CO side) and a matching allowlist entry in the sanitizer; the format is being agreed and the sanitizer change follows in this lane.
• Strix is untouched (already covered). The scheduler, the dispatch variable (opencode-review-dispatch rejects the current dispatcher identity: allowlist says github-actions[bot], sender is opencode-agent[bot] #1929) and the first-byte-timeout policy are separate.

Verification

Full gate on a300d82f with GITHUB_ACTIONS=true: 2897 passed / 1 skipped, coverage report --fail-under=100 → 100%, interrogate → 100%; the noema and queue contract suites (69 tests) pass; negative control: with main's noema-review.yml swapped in the new test fails (1 failed), restored it passes. YAML parsed with an independent parser (step sits between Prepare Noema model verdict and Refresh repository-scoped Noema GitHub App token for publication); actionlint reports nothing new.

Chicken-and-egg note: noema-review.yml is a required pull_request_target workflow, so this PR's own noema-review context runs the base branch's file and cannot exercise the change; same category as #1937.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 6 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 0a1faa25-9f79-4125-aa7e-504ac5970a4f

📥 Commits

Reviewing files that changed from the base of the PR and between 972b74b and c1f822b.

📒 Files selected for processing (3)

• .github/workflows/noema-review.yml
• CHANGELOG.md
• tests/test_noema_orchestrator_workflow_contract.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Companions: #1943 enables DEBUG logging in the sidecar process with an asctime LEVEL name message formatter, and a follow-up PR extends sanitize_contextual_orchestrator_sidecar_stream.py so the orchestrator's provider_*/circuit_* events survive into the file this step uploads. Any merge order is safe.

[seonghobae]

Third companion in this lane: #1945 — the sidecar sanitizer now admits the orchestrator's provider_*/circuit_* templates (bounded fields only, error_message cut, both log prefixes, timestamp kept). Any merge order stays safe.

[seonghobae]

Merged as fe827e13 (squash, bypass over REST) — author session peer1, merger a separate session.

Verified by the merging session's own run, not relayed. Because main had moved since this PR's head was verified, the merged tree was gated, not just the head: git merge-tree of 972b74be × c1f822b8 — no conflicts, no markers; vs its base: .github/workflows/noema-review.yml (one 11-line step), CHANGELOG, tests/test_noema_orchestrator_workflow_contract.py; head c1f822b8 is the author's merge of main@972b74be with the changelog entry placed above #1945's; full gate on that merged tree: 2904 passed, 1 skipped, 21 subtests passed in 113.63s (0:01:53); coverage 100%; RESULT: PASSED (minimum: 100.0%, actual: 100.0%). After the squash, main's tree 1e666659ce3e is byte-identical to the gated merge tree 1e666659ce3e. Negative control on the head: on a300d82f (noema-review.yml byte-identical to this head: yes; its contract test unchanged: yes): with main's workflow file test_noema_review_uploads_sidecar_evidence_on_failure fails (1 failed / 11 passed); with the PR's, 12 pass; ruby YAML load ok, actionlint 0 findings. noema-review.yml gains exactly one step, if: failure() && env.PR_NUMBER != '', actions/upload-artifact@043fb46d… (byte-identical pin to strix.yml:1047), uploading strix_runs/contextual-orchestrator-sidecar.stderr.log and strix_runs/contextual-orchestrator-preflight.json (the paths contextual_orchestrator_review_sidecar.sh:28/:231/:233 write) as noema-sidecar-evidence, if-no-files-found: ignore, 5-day retention; the contract test locks the pin to strix's. With #1943 (DEBUG attempt lines) and #1945 (sanitizer admits them) already on main, this closes the observability trio: the next failing Noema run's artifact should show provider_attempt → provider_attempt_failed → provider_exhausted per route with timestamps.

Authorization, cited at the act from the standing directive: "60-job ceiling에 막혀 있을 거라 일을 하기 어렵다면 60-job ceiling을 만드는 workflow issues를 추적해서 해소하세요. 이 상황은 Chicken-and-eggs 상황이므로 Bypass merge가 허용됩니다." This PR is the tracing half of that grant — it makes the ≈548 s-per-candidate slot burn behind today's noema/strix failures readable as a per-route timeline in the run artifact — and it structurally cannot validate itself (its own required run executes main's copy of the file it changes). Review-pipeline files under scripts/ci/ or .github/workflows/ only; no pinned or policy file; not dirty. Of the day's bypasses this is the weakest case (instrumentation, not a fix), stated as such.

[seonghobae]

Verified in production: the first three post-fe827e13 Noema failures (runs 33997442892, 33997237253, 33996977334) each shipped noema-sidecar-evidence with a per-route timeline — 12 agent ids, timestamps, zero omitted_unstructured_lines. Three failures turned out to have three different causes (gateway walk with permanent rejections of two OpenRouter free models; a verdict failing local schema validation; the gh pr diff 300-file limit on #1555). Table on #1935.