Skip to content
1 change: 0 additions & 1 deletion .github/workflows/opencode-review-dispatch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5302,7 +5302,6 @@ jobs:
--run-attempt "$RUN_ATTEMPT" \
--coverage-result "${COVERAGE_EVIDENCE_RESULT:-unknown}" \
>"$body_file"
printf '\n%s\n\n%s\n' "## Review outcome" "Coverage is a gate, not the review. This body reviews the changed product files." >>"$body_file"
create_pull_review "$event" "$(cat "$body_file")"
# create_pull_review COMMENT rewrites the status comment to Gate
# result: COMMENT. Restore the coverage gate so a miss never looks
Expand Down
3 changes: 3 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# AGENTS.md — ContextualWisdomLab .github

<!-- CWL-ENTRY -->
Fallback file inventories must not claim a completed model review or observed
runtime flow. See [the reproduction](docs/doctoring/review-fallback-quality-boundary.md).

> **Agents: read the master context FIRST.** Before any work, read [`docs/CWL-MASTER-CONTEXT.md`](docs/CWL-MASTER-CONTEXT.md) (mission · naruon-as-platform + inter-component UML · cross-cutting disciplines · conventions · roadmap · current state), the live **GitHub Project #1** <https://github.com/orgs/ContextualWisdomLab/projects/1> (work/roadmap source of truth), the full spec **ContextualWisdomLab/naruon#974**, the live gap snapshot [`docs/product-technical-gap-baseline.md`](docs/product-technical-gap-baseline.md) (not merge authorization; Figma File ID for this repo is N/A per [`docs/adr/0002-product-technical-gap-baseline.md`](docs/adr/0002-product-technical-gap-baseline.md)), and operate the Project per [`docs/agent-github-project-protocol.md`](docs/agent-github-project-protocol.md). The repo/Project — not any private agent memory — is the source of truth. The standing autonomous operating directive for the continuous PR review→fix→merge→develop loop across the ecosystem is [`docs/product-goal-directive.md`](docs/product-goal-directive.md) — a `/goal` session's 4000-character pointer refers to it; read the full directive before running or configuring any such loop.

Materialize accepts only exact SHA-256 pins, a bounded relative `-r` include
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
### OpenCode fallback output states its review limits

- Fallback comments identify the changed-file inventory and the next action,
instead of claiming completed model review. Remove the duplicate generated
diagram and identify inspected public symbols without asserting an API change.

### Failed-check finding names the Strix sandbox instead of the gateway

- `opencode-review-dispatch.yml`'s `emit_strix_provider_failure_finding` rendered one fixed finding for every `STRIX_PROVIDER_UNAVAILABLE` line, whose Root cause read "The contextual-orchestrator gateway or its discovered provider pool was unavailable for this run". `#1953` had just given the Strix sandbox bootstrap failure its own second verdict token (`STRIX_SANDBOX_UNAVAILABLE`) precisely because that attribution is wrong for it -- the sandbox container never reaches its Caido proxy, so the run dies before the gateway serves anything -- and this consumer re-applied the wrong attribution one step downstream, into the review findings and the failure census. The emitter now branches on the second token: a sandbox verdict gets a finding that names Strix's sandbox, says the verdict does not name the gateway, and tells the reader not to change gateway or provider configuration on its strength. A `STRIX_PROVIDER_UNAVAILABLE` line without the token keeps its existing text verbatim, so the gateway class has no regression surface. No test covered this finding text at all before (`gateway or its discovered provider pool` matched nothing under `tests/`); `tests/test_opencode_dispatch_strix_sandbox_finding.py` now runs the production emitter from the published run block and pins both directions plus the no-signal case. Refs #1953, #1935.
Expand Down
3 changes: 3 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Read first

Keep fallback inventories distinct from completed model reviews; reproduce the
publication contract using [this record](docs/doctoring/review-fallback-quality-boundary.md).

`AGENTS.md` is the canonical agent entry point. Per its instructions, before any work read
[`docs/CWL-MASTER-CONTEXT.md`](docs/CWL-MASTER-CONTEXT.md) (mission, ecosystem UML, cross-cutting
disciplines CP-1..CP-5/G6/SEAM, binding engineering conventions in §7, roadmap), the live
Expand Down
83 changes: 83 additions & 0 deletions docs/doctoring/review-fallback-quality-boundary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Review fallback quality boundary

## Observed failure

On 2026-09-08, the latest five Noema/OpenCode formal review objects found in the
20 newest open `.github` PRs were all fallback/status-derived OpenCode output.
There were 11 matching review objects in that bounded collection. The selected
five are not a representative sample of model quality or the organization.

| PR | Review ID | Reviewed commit | Matched head at collection |
| --- | --- | --- | --- |
| #2029 | 5136651994 | eb79481bc1696c63273b6c2ca22b5e34f68d0208 | No |
| #2021 | 5134455762 | 4dec9c6a2e03d2ff3e62b186a47fe99910cfcd63 | No |
| #2019 | 5134382662 | 100f1e1b0554b48ba2f843c965d414993cf5588f | Yes |
| #2020 | 5134381479 | 29d9d5ec6a02f5c23b315b70f8315a582dac5d1b | No |
| #2016 | 5134310293 | 5baffac6fdb754e6444c4dfad6a56e4af89d9fe7 | Yes |

All five described a gate limitation and contained Mermaid, but none supplied
a behavioral summary, a concrete file responsibility beyond its category, or
a hypothesis/counterexample/evidence chain. Their diagrams were generic file
inventories. These observations do not establish an outage cause.

The [#2029 review](https://github.com/ContextualWisdomLab/.github/pull/2029#pullrequestreview-5136651994)
claimed that OpenCode reviewed the product diff, then displayed two generated
file maps. `build_fallback_review` created the first map and completion claim;
`publish_fallback_diff_review` appended another completion claim, and the
common publisher appended its own map. File classification is not source review.

## Repair and measurement

Label the fallback as an inventory with a next action, remove its duplicate
diagram, and stop the caller from appending a completed-review claim. Public
symbols read from supplied files are identified as such, not as changed APIs.
The common publisher retains its evidence map and gate handling. No inference
result, approval, or finding is synthesized.

English/Korean regression baseline: 0/2 outputs correctly identify the fallback
limit on main `7fd571dbcdbae6acf29d8f4ee704d7ba6297e4db`. Reproduce with:

```sh
uv run --no-project .venv/bin/python -m pytest tests/test_opencode_review_surfaces.py tests/test_opencode_review_comment_helpers.py --cov=scripts.ci.opencode_review_surfaces --cov-report=term-missing --cov-fail-under=100 -q
```

After repair: 2/2 language cases pass; all 43 surface/helper tests pass with
100% statement and branch coverage of `opencode_review_surfaces`. Docstrings
also reach 100%. Actionlint passes with its optional ShellCheck/Pyflakes
integrations disabled; the full invocation was interrupted after remaining
silent for several minutes. It is not recorded as a full lint pass.

## Model availability dependency

[Noema run 34128415567](https://github.com/ContextualWisdomLab/.github/actions/runs/34128415567/job/101776606311)
failed after 1,976.2 seconds with one caller attempt and terminal HTTP 502.
Its [sidecar artifact](https://github.com/ContextualWisdomLab/.github/actions/runs/34128415567/artifacts/10025654264)
records `provider_connection_error`, 26 review-phase attempt log entries,
including 19 timeouts and three HTTP errors. Six routes passing preflight
did not establish success for the real review request.

The installed owner revision was `414f22973658c4ddc3d4320fcf7acd9b4e8ba991`,
also the consumer pin observed during this investigation.
[ContextualWisdomLab/contextual-orchestrator#1094](https://github.com/ContextualWisdomLab/contextual-orchestrator/pull/1094)
already contains a candidate for structured-response synthesizer failover on
retryable 502/429/timeout errors. At head
`1c61eff2da012382255bf8b4e1aa6dd0d6dd05ca`, it remains open with six nonterminal
checks, nine unresolved threads, and no exact-head formal review.
Triage that owner candidate before duplicating repair in a consumer. Its scope
matches the failure, but this historical log does not prove the fix works.
Adoption requires protected owner merge, immutable release, consumer contract
verification, and a real Noema replay. Do not add caller retries or paid routes.

This repair improves truthful publication, not model availability. A subsequent
real model review must still demonstrate behavioral explanation, source-bound
findings, counterexamples, verification limits, and useful next actions. Use the
[Greptile reference](https://github.com/BerriAI/litellm/pull/20421#issuecomment-3847832704)
for concrete behavior/flow explanations, not an unsupported numerical score.

## References

GitHub. (n.d.). *REST API endpoints for pull request reviews*. Retrieved
September 8, 2026, from https://docs.github.com/en/rest/pulls/reviews

Greptile. (2026, February 4). *Greptile overview* [Pull request comment]. GitHub.
https://github.com/BerriAI/litellm/pull/20421#issuecomment-3847832704
15 changes: 13 additions & 2 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
# Product and Technical Gap Baseline

작성 기준일: **2026-08-26 10:35 KST**
## 2026-09-08 review-quality follow-up — fallback output and availability

Proposed: fallback inventories no longer claim completed model review and no
longer generate a duplicate diagram. English/Korean evidence-limit cases improve
from 0/2 to 2/2; 43 surface/helper tests pass with 100% module statement/branch
coverage. This does not establish real model review quality or deployment.
The observed Noema HTTP 502 failure points to the already-open owner candidate
ContextualWisdomLab/contextual-orchestrator#1094; triage its exact-head review
threads before immutable adoption and live replay. See
[the bounded sample, RCA, and validation limits](doctoring/review-fallback-quality-boundary.md).

작성 기준일: **2026-09-08 13:51 KST**
대상: **ContextualWisdomLab/.github** 중앙 거버넌스·자동화 레포지터리와 이를 소비하는 naruon 생태계
현재 보호된 `main`: `826b92394c63deb6981c3a8d16a724d71f85a0d7`
현재 보호된 `main`: `7fd571dbcdbae6acf29d8f4ee704d7ba6297e4db`
현재 열린 PR 수: **107** (아래 표에 이 스냅샷의 전체 목록 포함; live API 재수집)

이 문서는 제품·기술·운영 Gap을 현재 문서와 현재 GitHub 상태에 묶어 두는 기준선이다. 새 작업은 먼저 이 문서의 Gap ID를 PR 설명과 테스트 증거에 연결하고, PR의 정확한 exact HEAD·Checks·리뷰를 다시 수집한 뒤 구현한다. 표의 상태는 작성 시점의 관측값이므로, 병합 판단에는 재사용하지 않는다. 이 인벤토리는 스냅샷이며 merge authorization이 아니다.
Expand Down
18 changes: 11 additions & 7 deletions scripts/ci/opencode_review_surfaces.py
Original file line number Diff line number Diff line change
Expand Up @@ -514,24 +514,28 @@ def build_fallback_review(
language: str = "english",
coverage_result: str = "success",
) -> str:
"""Build a source-backed formal review of the actual changed product files."""
"""List supplied changed files without claiming a completed model review."""
paths = [posix_path(path) for path in changed_files if str(path).strip()]
korean = _language(language) == "korean"
overview = "Pull request overview" if not korean else "Pull request 개요"
walkthrough = "Changed files" if not korean else "변경 파일"
diagram = "Changed behavior" if not korean else "변경 동작"
findings = "Findings" if not korean else "발견 사항"
intro = (
"OpenCode reviewed the current-head product diff. Coverage is a separate gate."
"This fallback is a changed-file inventory, not a completed model review. "
"Inspect the workflow failure and rerun the review after resolving it. "
"File categories and suggested checks below are not execution evidence."
Comment on lines +524 to +526

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the fallback evidence map inventory-only

When this fallback contains a Rust path for which rust_api_symbols finds no recognized public item, the published body contradicts this disclaimer: create_pull_review still calls ensure_review_body_has_change_graph, whose emit_mermaid branch renders Caller->>Crate: changed public API and Tests->>Crate: regression coverage without evidence that either interaction occurred. Suppress that graph for fallback reviews or render only file/category nodes so the fallback does not continue asserting an unobserved API and test flow.

AGENTS.md reference: AGENTS.md:L4-L5

Useful? React with 👍 / 👎.

if not korean
else "OpenCode가 현재 head의 제품 diff를 리뷰했습니다. 커버리지는 별도 게이트입니다."
else "이 대체 출력은 변경 파일 목록이며, 모델 리뷰가 완료됐다는 뜻은 아닙니다. "
"워크플로 실패 원인을 해결한 뒤 리뷰를 다시 실행하세요. "
"아래 파일 분류와 권장 검사는 실행 증거가 아닙니다."
)
if not paths:
intro = (
empty_path_notice = (
"OpenCode could not list changed product files for this head."
if not korean
else "OpenCode가 이 head의 변경 제품 파일을 나열하지 못했습니다."
)
intro = f"{empty_path_notice} {intro}"
lines = [
f"## {overview}",
"",
Expand All @@ -544,10 +548,10 @@ def build_fallback_review(
lines.extend(f"- {_file_role(path)}" for path in paths)
else:
lines.append("- No changed product files were supplied to the fallback review.")
lines.extend(["", f"## {diagram}", "", emit_mermaid(paths, source_root=source_root).rstrip(), ""])
lines.append("")
symbols = rust_api_symbols(source_root, paths)
if symbols:
api_heading = "Changed API" if not korean else "변경 API"
api_heading = "Public symbols in supplied files" if not korean else "제공된 파일의 공개 심볼"
lines.extend([f"## {api_heading}", ""])
lines.extend(f"- `{symbol}`" for symbol in symbols)
lines.append("")
Expand Down
3 changes: 2 additions & 1 deletion tests/test_opencode_agent_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -1994,7 +1994,8 @@ def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
assert "needs.coverage-evidence.result != 'cancelled'" in workflow
assert "request_changes_for_coverage_evidence_failure" in workflow
assert "implementation_completeness_scan.py" in workflow
assert '"## Review outcome"' in workflow
assert "opencode_review_surfaces.py build-fallback-review" in workflow
assert "This body reviews the changed product files." not in workflow
assert '"## Check outcome"' not in workflow
assert 'update_review_overview "COVERAGE_BLOCKED"' in workflow
assert "record coverage-evidence blocker states" in workflow
Expand Down
52 changes: 47 additions & 5 deletions tests/test_opencode_review_surfaces.py
Original file line number Diff line number Diff line change
Expand Up @@ -360,23 +360,64 @@ def test_central_workflow_in_diff_is_a_workflow_surface_not_line_one_finding() -
)


def test_fallback_review_empty_file_list() -> None:
"""Missing changed-file evidence still produces a distinct review body."""
@pytest.mark.parametrize("language,notice", [
("english", "This fallback is a changed-file inventory, not a completed model review."),
("korean", "이 대체 출력은 변경 파일 목록이며, 모델 리뷰가 완료됐다는 뜻은 아닙니다."),
])
def test_fallback_labels_its_evidence_limit(language: str, notice: str) -> None:
"""A file inventory cannot claim completed review or observed behavior."""
review = surfaces.build_fallback_review(
changed_files=["src/example.py"], head_sha=HEAD,
run_id="1", run_attempt="1", language=language,
)
assert notice in review
assert "reviewed the current-head product diff" not in review
assert "제품 diff를 리뷰했습니다" not in review
assert "## Changed behavior" not in review
assert "## 변경 동작" not in review
assert "```mermaid" not in review # The publisher appends its single evidence map.


@pytest.mark.parametrize(
("language", "notice", "action"),
(
(
"english",
"not a completed model review",
"Inspect the workflow failure and rerun the review after resolving it.",
),
(
"korean",
"모델 리뷰가 완료됐다는 뜻은 아닙니다.",
"워크플로 실패 원인을 해결한 뒤 리뷰를 다시 실행하세요.",
),
),
)
def test_fallback_review_empty_file_list(
language: str, notice: str, action: str
) -> None:
"""Missing changed-file evidence keeps the fallback boundary and next action."""
review = surfaces.build_fallback_review(
changed_files=[],
head_sha=HEAD,
run_id="1",
run_attempt="1",
language=language,
)
assert notice in review
assert action in review
assert (
"No changed product files" in review
or "변경 제품 파일을 나열하지 못했습니다" in review
)
assert "No changed product files" in review
assert ".github/workflows/opencode-review.yml:1" not in review


def test_fallback_review_rejects_accidental_central_workflow_citation(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""A synthesized body may not mention the central workflow unless it changed."""
monkeypatch.setattr(surfaces, "CENTRAL_WORKFLOW_ANCHOR", "Coverage is a separate gate")
monkeypatch.setattr(surfaces, "_file_role", lambda path: surfaces.CENTRAL_WORKFLOW_ANCHOR)
with pytest.raises(ValueError, match="must not cite"):
surfaces.build_fallback_review(
changed_files=ORIGINWEAVE_47_FILES,
Expand Down Expand Up @@ -466,7 +507,7 @@ def test_surfaces_cover_remaining_review_branches(tmp_path: Path) -> None:
source_root=tmp_path,
language="korean",
)
assert "변경 API" in review
assert "제공된 파일의 공개 심볼" in review
assert "`Once`" in review


Expand Down Expand Up @@ -793,6 +834,7 @@ def test_publisher_workflow_cannot_replace_review_with_coverage_finding(
fallback_fn = fallback_fn.split("request_changes_for_coverage_evidence_failure()", 1)[0]
assert "create_pull_review" in fallback_fn
assert "request_changes_for_coverage_evidence_failure" in fallback_fn
assert "This body reviews the changed product files" not in fallback_fn
assert fallback_fn.index("create_pull_review") < fallback_fn.index(
"request_changes_for_coverage_evidence_failure"
)
Expand Down
2 changes: 1 addition & 1 deletion tests/test_pr_review_autofix_nvidia_nim_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
DOCTORING_RECORD = Path("docs/doctoring/hourly-nvidia-nim-autofix.md")
CHANGELOG = Path("CHANGELOG.md")
REVIEW_DISPATCH_WORKFLOW = Path(".github/workflows/opencode-review-dispatch.yml")
REVIEW_DISPATCH_BLOB_SHA = "d86497b3f43bebbabbb4f504eb5132cdf3b7b293"
REVIEW_DISPATCH_BLOB_SHA = "1004b82a336fd455d995233fdb437249280886fc"


def _workflow_text(path: Path) -> str:
Expand Down
Loading