docs(gap-baseline): record #1532 fix and owner's 2026-09-01 review-quality directive

docs/product-technical-gap-baseline.md

@@ -93,6 +93,10 @@ flowchart LR
 | G-13 | hourly scheduler는 존재하지만 no-op/credential unavailable/queued Checks의 customer next action을 모든 caller가 동일한 receipt로 내는지 미확인이다 | 자동화가 실패해도 운영자가 무엇을 고쳐야 하는지 알 수 없다 | `skipped_credential_unavailable` receipt와 다음 행동 문구를 exact-head Checks로 검증하고, bounded receipt schema, retry floor, single-flight, no secret fallback을 모든 caller contract test로 고정한다 |
 | G-14 | release/changelog/version 증거가 각 PR에 분산되고 현재 central repo 보호 main의 release candidate가 명확하지 않다 | 운영자는 어떤 기능이 supportable release인지 확인할 수 없다 | merge 후 release readiness ledger, CHANGELOG, semantic version/tag, rollback/operability evidence를 함께 갱신한다 |
 | G-15 | 첨부파일 처리 경계가 제품별로 다르고, 1MB 상한은 업무 데이터와 맞지 않으며 미지원 MIME/컨테이너가 parser registry에서 명시적으로 pending/quarantine 되는지 확인되지 않았다. 현재 20MB 초과 파일 가능성과 PDF/HWP/HWPX·이미지·압축파일의 parse/sidecar 흐름을 하나의 exact contract로 묶지 못했다 | 큰 업무 첨부를 거부하거나 파싱 실패를 조용히 잃으면 고객의 메일·문서 업무가 중단된다 | naruon/newsdom-api 소유 PR에서 streaming upload, configurable bounded limit above 20MB, MIME sniffing, parser capability registry, quarantine/retry, source-position provenance, and ADR를 추가하고 size/unsupported-type/zip-bomb tests를 required evidence로 만든다 |
+| G-19 | OpenCode/Noema의 실제 리뷰 산출물 깊이·정확도가 CodeRabbit/Devin과 대등한지 측정된 적이 없다 (owner directive, 2026-09-01) | 자동 리뷰가 형식적으로만 통과하고 실제 결함은 사람이나 외부 봇에만 의존해 발견된다 | 동일 PR 집합에 대한 finding 수/정밀도/심각도 비교 방법론을 정의하고, `opencode.jsonc`/`ci-review-prompt.md`/`code-reviewer-prompt.md`/`scripts/ci/noema_review_gate.py`의 프롬프트·설정을 감사한 뒤 parity 기준에 맞춰 조정한다 |
+| G-20 | Strix의 현재 보안 스캔 범위가 diff-only인지 전체 코드베이스인지 `strix.yml` 설정 기준으로 확인되지 않았다 (owner directive, 2026-09-01: "보안 리뷰는 전체 코드로 수행") | diff만 스캔하면 기존 코드에 남아 있는 취약점이 영구히 미탐지 상태로 남는다 | `strix.yml`의 실제 스캔 대상 설정을 확인하고 diff-only이면 전체 저장소로 확장하며, 실행시간·timeout 예산(§ 2026-09-01 owner directive의 3시간 하한과 연동)을 함께 재산정한다 |
+| G-21 | contextual-orchestrator의 리뷰용 모델 선택이 free-tier 후보를 대체로 동등하게 취급해 12개 후보 순차 preflight 같은 지연이 발생한다 (owner directive, 2026-09-01: "빠르면서 능력이 좋은 모델에 요청을 보내어 시간을 당기시오") | 리뷰 1건이 수 시간씩 걸려 병합 루프 전체가 느려진다 | `model_discovery.py`의 provider별 응답 시간 데이터(있다면) 또는 신규 latency 신호를 정의하고, free+ZDR 제약 내에서 속도·능력 가중 랭킹을 구현한다 |
+| G-22 | `.github` 저장소의 `Required OpenCode Review` 실행이 장시간 `queued` 상태로 정체되는 패턴이 관측된다 (2026-09-01 실측: 동시에 45개 run이 `queued`, 일부는 약 2시간째 시작조차 못함). 러너 용량 제약과 일치하는 관측이지만, 조직의 실제 Actions 동시성 한도·과금 데이터로 직접 확인되지는 않았다 (Devin 리뷰 지적, #1535) | 개별 poll-budget/dispatch-worker 정합성을 고쳐도(#1532) job 자체가 러너를 못 얻으면 리뷰가 여전히 지연되거나 시간 내 완료되지 못한다; PR이 많고 각 PR이 재검증마다 main을 반복적으로 restack하면서 스스로 대기열 부하를 키운다 | `opencode-review.yml`의 `concurrency` 그룹(레포+PR 번호 스코프, `cancel-in-progress: true`)은 정상 작동을 확인함 — 오래된 head의 stale run이 아니라 대기열 정체가 원인. 다음 단계: 조직 Actions 동시성 한도/과금 데이터를 직접 확인(관리자 권한 필요, 아직 미확보), 열린 PR 총량 축소를 통한 처리 요구량 감소, 또는 PR별 재검증 트리거의 배치·디바운스 — 아직 미착수 |
 
 ## 4. 열린 PR live inventory
 
@@ -1715,6 +1719,89 @@ string, a bare number) confirmed to fail against the pre-fix script (`KeyError:
 signature as the original round-4 bug) before passing after the fix. 1930 tests pass; 100% coverage and
 100% docstring coverage on `scripts/ci/`.
 
+## 2026-09-01 opencode-review verdict-poll/dispatch-worker budget mismatch fixed; three follow-up product gaps opened
+
+- **Root-caused and fixed the systemic `opencode-review` blocker** that had left `.github#1500`, `#1506`,
+  `#1527`, `#1529`, and `ContextualWisdomLab/contextual-orchestrator#968`/
+  `ContextualWisdomLab/contextual-orchestrator#946` all stuck: `opencode-review.yml`'s verdict
+  poll job (`timeout-minutes: 100`, 90 minutes of polling) was structurally shorter than the dispatched
+  `opencode-review-dispatch.yml` worker job's own ceiling (`timeout-minutes: 325`, 205 minutes reserved
+  for its model-pool step alone, needed because the contextual-orchestrator sidecar preflight probes ~12
+  free-tier candidate models sequentially with real `TimeoutError`s and stale-model 404s). The dispatch
+  always succeeded; the poll could never outlast a worker job 3.6x its own budget. Confirmed with
+  concrete timestamps: on `.github#1500`, dispatch fired `11:37:27Z`, the poll gave up `13:08:30Z`, but
+  the `scan-pr-queue` pass touching that exact PR still had not produced an approval as of `13:55:04Z` —
+  47 minutes after the poll had already failed closed. Fixed in `.github#1532` (merged via
+  owner-authorized admin bypass — the fix edits `opencode-review.yml` itself, the exact required-check
+  pipeline it depends on, making normal review of this specific PR structurally impossible under the
+  pre-fix budget): raised the poll job to `timeout-minutes: 340` / 660 attempts (~330m), and added
+  `test_verdict_poll_budget_covers_the_dispatched_review_jobs_own_ceiling` so the two workflows' budgets
+  cannot silently drift out of alignment again. `contextual-orchestrator` is a contributing factor (its
+  free-tier model pool's real latency/flakiness is why worker timeouts run in the hundreds of minutes)
+  but not a new regression; open PR `#971` there is unrelated to this failure mode.
+- **Owner directive, 2026-09-01**, broadcast across `#1500`/`#1502`/`#1503`/`#1527`: any timeout on the
+  review path should be at minimum 3 hours ("120초 같은 건 당황스럽군요" — something like 120 seconds is
+  embarrassing), OpenCode/Noema should review as thoroughly as CodeRabbit/Devin, Strix's security review
+  should cover the entire codebase (not just the diff), and contextual-orchestrator should route review
+  requests to fast, capable models to cut wall-clock latency. Grepped `main` for remaining short
+  (≤120s) timeouts on the review path: two remain, both with fixes already in flight and not duplicated
+  here — `scripts/ci/noema_review_gate.py:656`'s `opener.open(request, timeout=120)` is addressed by
+  `#1508` (removes the per-request LLM timeout entirely so Noema reviews run for the enclosing job's
+  lifetime; currently `mergeable_state: dirty`, a real conflict against current `main`, not yet
+  resolved), and `scripts/ci/contextual_orchestrator_review_sidecar.sh:484`'s `curl -sS --max-time 120`
+  gateway-preflight call is being reworked in `#1415` — its latest iteration raises the bound to one
+  hour per attempt with caller-specific job ceilings, which is progress over the 120-second value but
+  **does not yet meet this directive's 3-hour floor**; still under active iteration against fresh Devin
+  findings (including this exact gap, independently flagged there), not merged, and not counted as
+  resolved here.
+- **Three new tracked product gaps opened by this directive (register entries G-19/G-20/G-21 above —
+  numbered past `#1348`'s pending `G-16` allocation for its own queue-hygiene gap, still open and
+  unmerged as of this writing, to avoid a future collision),
+  not yet scoped or started:**
+  - **G-19 (G-Review-Depth)**: OpenCode/Noema review output quality/thoroughness has no measured parity target
+    against CodeRabbit/Devin. Needs: a comparison methodology (same PRs, same findings categories),
+    review prompt/config audit (`opencode.jsonc`, `ci-review-prompt.md`, `code-reviewer-prompt.md`,
+    `scripts/ci/noema_review_gate.py`'s prompt construction), and a decision on what "parity" concretely
+    means (finding count, finding precision, severity calibration, or some combination) before any
+    prompt/config change is made.
+  - **G-20 (G-Strix-FullScope)**: Strix's current security-review scope (diff-only vs. whole-repository) needs
+    to be confirmed against `strix.yml`'s actual scan configuration, then widened to full-codebase if it
+    is currently diff-scoped, with a cost/runtime-budget analysis (a whole-repo scan changes the timeout
+    arithmetic this same directive also touches).
+  - **G-21 (G-FastModelRouting)**: `contextual-orchestrator`'s model selection for review workloads currently
+    treats free-tier candidates largely interchangeably (see the 12-candidate sequential preflight cost
+    above); the directive asks for latency-aware ranking toward fast, capable models within the
+    free+ZDR-constrained pool, which needs a defined speed/capability signal (e.g. `model_discovery.py`'s
+    existing per-provider timing data, if any) before implementation — not attempted blind this pass.
+- **A fourth, distinct systemic pattern found while verifying `.github#1500`'s fix actually let its review
+  pipeline run (register entry G-22 above): a sustained `Required OpenCode Review` queue backlog, not a
+  dispatch-logic bug.** `#1532`'s poll-budget fix and `#1500`'s own legacy-review-suppression fix both
+  address correctness of the review *state machine*; neither helps if the underlying `Required OpenCode
+  Review` job never gets a runner. Measured directly: `gh`/GraphQL listing of `opencode-review.yml` runs
+  filtered to `status=queued` on `ContextualWisdomLab/.github` alone returned 45 simultaneous queued runs
+  spanning nearly two hours of creation timestamps, one job (`coverage-source-tree` on `#1535`) still
+  queued 1h47m after its run was created. Ruled out a cancellation bug first, on the actual affected
+  workflow: `.github#1500`'s own `opencode-review.yml` run for head `99eb4bab` (created `03:21:58Z`) shows
+  `status: completed, conclusion: cancelled` once the next push (`9d370d59`, `03:26:21Z`) superseded it —
+  `concurrency.group` is scoped to `repo+PR-number` (not head SHA) with `cancel-in-progress: true`, and a
+  superseded push to the same PR does correctly cancel its predecessor's `Required OpenCode Review` run
+  specifically. So the 45 queued runs are each a distinct, still-live PR's *current* head competing for
+  runner capacity, not stale zombies left behind by the cancellation path. **What this does not establish**
+  (Devin review finding, `#1535`): a large, long-waiting queue is consistent with runner-capacity
+  constraints, but is not itself proof of a specific organization-level Actions concurrency ceiling being
+  hit — that would need the org's actual concurrency limit/billing data, which this investigation did not
+  have access to and did not attempt to obtain. This repo's own operating pattern is a plausible contributing
+  factor either way: dozens of open PRs each restack onto `main` every time `main` moves (visible in commit
+  messages like "merge(main): restack ... on current control plane"), and each restack re-queues a full
+  ~15-30-job fan-out (SBOM, CodeQL, Semgrep, Trivy, OSV-scan, Scorecard, dependency-review, gitleaks, two
+  Python quality matrices, plus the review dispatch itself). Not fixed this pass — next step is confirming
+  the org's actual Actions concurrency/billing limits (needs admin access not available here) before
+  treating either remedy below as authoritative: an account/billing-level Actions concurrency increase
+  (not a code change) or reducing open-PR volume/restack frequency (the org's own stated PR-to-zero merge
+  goal already pushes this direction, but is not sufficient alone while dozens of PRs are open at once).
+  Left as an open observation under G-22 rather than attempting an unreviewed job-fan-out reduction that
+  could weaken a required security gate.
+
 ## 5. 실행 루프와 고객의 다음 행동
 
 각 hourly pass는 아래 순서를 유지한다.