Skip to content

fix(strix): report 경고와 provider 장애 진단 분리 - #1994

Draft
seonghobae wants to merge 1 commit into
fix/strix-free-only-model-normalization-20260902from
codex/strix-report-failure-class
Draft

fix(strix): report 경고와 provider 장애 진단 분리#1994
seonghobae wants to merge 1 commit into
fix/strix-free-only-model-normalization-20260902from
codex/strix-report-failure-class

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

범위

#1668 exact 82b19c4144d10550fb35145b2dc24cdb2db6f27a 위의 후속 개발 stack입니다. 기존 #1668 branch는 수정하지 않았습니다. 소비자 배포나 미출시 의존성 채택이 아닙니다.

report-only 경고를 provider 장애로 표시하던 남은 진단 경로만 분리합니다. Warn/Fatal/Denied/Timeout 패턴, 좁은 기존 예외, free-only 모델 경계, 실패 종료코드는 유지합니다. 기존 console/provider-report 분류기를 재사용하며 독립된 provider 오류가 함께 있으면 기존 provider 진단을 보존합니다. 실제 스캔, 경고 무시, API key 추가, 유료 우회는 없습니다.

실제 hosted 근거와 한계

ContextualWisdomLab/naruon#1244의 run 34039160285, job 101508385806, artifact 9993763655의 trusted gate는 dd0b96feded94f66ecf59b25a5a9b58cfc8b4f69였습니다. strix.log:442의 web_search 설정 WARNING이 실제 실패 패턴에 매칭됐습니다. 내부 재시도 뒤 completed/scan_completed=true, SARIF 0 results, child rc0 기록이 있어도 이 경고는 계속 FAIL입니다. 이 RCA는 provider pool 최종 소진을 입증하지 않으며, 이 PR은 공급자 복구를 포함하지 않습니다. 전체 보고서의 보안 분석 정확성을 승인하지도 않습니다.

로컬 검증

  • RED: 원본 production에서 fake Strix 1회, rc0, report 보존 및 gate FAIL은 기대대로이며 진단 assertion 2개 실패. /tmp/cwl-strix-report-red2.log (그 앞의 fixture 누락 실행은 제품 RED로 세지 않음).
  • GREEN report-only: terminal handle 65346, chunk e6eb6e, exit=0; 로그 /tmp/cwl-strix-report-green.log.
  • GREEN mixed provider: terminal handle 72072, chunk 537a5d, exit=0; 로그 /tmp/cwl-strix-report-mixed.log.
  • 종료코드는 terminal tool receipt에만 보존됐으며 위 로그에 별도 exit footer는 없습니다. 로그 끝의 NO_CONCLUSION은 기존 음성 검사의 출력입니다.
  • 관련 pytest 6파일 normal 35 passed / 20.13s, Actions -W error 35 passed / 21.83s, 모두 exit 0. 로그/JUnit /tmp/cwl-strix-report-{normal,ci}.{log,xml}.
  • main 독립 focused: session 36184, chunk 0c6e31, exit 0; /tmp/cwl-strix-main-independent-report.log (coordinator 확인).
  • actionlint .github/workflows/strix.yml, bash 구문 및 diff-check exit 0.

pytest 대상은 tests/test_strix_orchestrator_free_only.py, tests/test_strix_no_heuristic_retry_contract.py, tests/test_strix_no_heuristic_severity_contract.py, tests/test_strix_model_behavior_error.py, tests/test_strix_caido_bootstrap_timing_retry.py, tests/test_strix_nvidia_nim_not_found_fallback.py입니다. python -m pytest -q -W error로 실행했습니다.

전체 suite는 미실행입니다. 검증은 커밋 직전 동결된 동일 3파일에서 수행했고 커밋 뒤 재실행하지 않았습니다. 실제 hosted 새 head 검증 및 보호 병합 증거는 아직 없습니다. 로컬 경로는 검증한 장비의 receipt이며 GitHub artifact가 아닙니다.

변경은 gate, 기존 shell harness, 짧은 doctoring 3파일뿐입니다.

Keep unknown report warnings fail-closed without inferring a provider outage.
Preserve independent provider signals and the parent free-only contract.

Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Seongho Bae <me@seonghobae.me>
@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae added bug Something isn't working priority: medium Normal-priority or P2 work status: draft Draft pull request type: bug Defect or incorrect behavior labels Sep 6, 2026 — with ChatGPT Codex Connector

Copy link
Copy Markdown
Contributor Author

Adjacent instance in the same theme, from CI-failure triage (lane:jan) — not a request to widen this PR, and I am not opening a change for it.

#1953 gave the Strix sandbox bootstrap failure its own token, and opencode-review-dispatch.yml:5648 reads it correctly. One layer up, strix.yml does not. The gate emits

STRIX_PROVIDER_UNAVAILABLE: STRIX_SANDBOX_UNAVAILABLE: the last Strix attempt ended in the sandbox bootstrap …

(scripts/ci/strix_quick_gate.sh:4393 on main), and backend_unavailable_signal at strix.yml:951 matches on the leading token, so the check annotation at strix.yml:1008 fires with

Strix could not complete authoritative vulnerability analysis because its provider/backend was unavailable (rate limit, token cap, connection, warm-up, or model-behavior failure).

For a sandbox-bootstrap run none of those five causes applies — the container never reaches its Caido proxy, so nothing is asked of the gateway. That is the attribution error #1953's changelog describes, in the one place a reader looks first.

The bucketing itself is right and should stay: a finding-free sandbox outage is incomplete infrastructure evidence, which is why the gate deliberately keeps the leading STRIX_PROVIDER_UNAVAILABLE token. Only the annotation's message asserts a cause it has not established.

Two related measurements from the same triage, in case they are useful here:

  • Classifying on "which token the line starts with" misfiles every sandbox run, because the sandbox verdict is one composite line that also starts with STRIX_PROVIDER_UNAVAILABLE:. The test has to be a substring search.
  • On runs created before #1953 merged (2026-09-06T07:43:39Z) the token cannot be present at all. ContextualWisdomLab/contextual-orchestrator#983's strix job 100891196281 (created 2026-09-04T02:36:16Z) died three-for-three in loginAsGuest failed after 10 attempts: curl exit 7: … 127.0.0.1 port 48080, the loop printed Retrying model 'orchestrator/free' due to Caido sandbox bootstrap timing (attempt 3/3), and the terminal verdict was still the bare gateway sentence. For such runs the faithful signal is that retry-reason line (strix_quick_gate.sh:3137-3141).

Both are recorded in docs/doctoring/ci-failure-signature-triage.md (#1913). Scoping the annotation fix is yours to decide — it may well belong in a separate change rather than here.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority: medium Normal-priority or P2 work status: draft Draft pull request type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant