perf(routing): index psychometric context positions once - #1058
seonghobae wants to merge 3 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (6)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthrough심리측정 라우팅의 컨텍스트 위치 조회를 최적화했습니다. Python 3.12 전용 벤치마크를 추가했습니다. 정확도와 지연 시간을 함께 평가하는 문서 기준과 런타임 테스트를 추가했습니다. Changes심리측정 라우팅 평가
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to This change speeds psychometric routing preparation without changing routing policy, APIs, or item order. No current merge-blocking risk is identified. Sequence Diagram(s)sequenceDiagram
participant Benchmark
participant PsychometricRoutingEvidence
participant fast_mlsirm
Benchmark->>PsychometricRoutingEvidence: 512개 관측값 기록
Benchmark->>fast_mlsirm: 피팅 및 확률 예측 패치
Benchmark->>PsychometricRoutingEvidence: ranked_evidence 9회 호출
PsychometricRoutingEvidence->>fast_mlsirm: 고정 피팅 및 예측 수행
Benchmark-->>Benchmark: 지연 시간 중앙값과 샘플 결과 출력
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 3 files. (3 skipped: 3 unsupported.)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2f90116b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Seongho Bae <me@seonghobae.me>
PR #1067 has exactly the same tree as #1058, already an ancestor of #1064. Preserve both histories and the complete #1064 tree without reapplying identical cherry-picked changes. The successor update remains a fast-forward. Source tree: 8735f95 Predecessors: #1058, #1059, #1061, #1062, #1064. No predecessor is closed before protected delivery and delta verification. Signed-off-by: Seongho Bae <me@seonghobae.me>
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head product diff. Coverage is a separate gate.
Changed files
contextual_orchestrator/psychometric_routing.py— Python module behaviordocs/doctoring/measured-routing-evidence.md— operator or user guidancedocs/papers/README.md— operator or user guidancedocs/product-technical-gap-baseline.md— operator or user guidancescripts/benchmark_psychometric_routing.py— Python module behaviortests/test_psychometric_routing.py— regression suite
Changed behavior
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Python: psychometric_routing.py (2 files)"]
S1 --> I1["Python module behavior"]
I1 --> R1["Review risk: Python: psychometric_routing.py (2 files)"]
R1 --> V1["pytest plus coverage"]
Evidence --> S2["Docs: measured-routing-evidence.md (3 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs: measured-routing-evidence.md (3 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test: test_psychometric_routing.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_psychometric_routing.py"]
R3 --> V3["targeted test run"]
Findings
No source-backed product finding is synthesized from the coverage gate. A coverage miss belongs in the status comment.
- Head SHA:
3c22bf31c33520b128cb2c449d03e3762e1f4c75 - Workflow run: 33897183732
- Workflow attempt: 1
- Coverage gate:
failure
Review outcome
Coverage is a gate, not the review. This body reviews the changed product files.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Python: psychometric_routing.py (2 files)"]
S1 --> I1["Python module behavior"]
I1 --> R1["Review risk: Python: psychometric_routing.py (2 files)"]
R1 --> V1["pytest plus coverage"]
Evidence --> S2["Docs: measured-routing-evidence.md (3 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs: measured-routing-evidence.md (3 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test: test_psychometric_routing.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_psychometric_routing.py"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Coverage evidence did not pass, so approval is blocked. The formal pull-request review is the source-backed diff review, not this status comment. |
seonghobae
added
maintenance
priority: medium
ChatGPT Codex Connector
1 participant
Successor tracking
The complete delta at this PR's head
3c22bf31c33520b128cb2c449d03e3762e1f4c75is preserved in trusted-branch successor #1067 at1481c595dc1d16e7bf4b65addaf0bd30322cf2b8. Commit ancestry and tree equality are recorded indocs/product-technical-gap-baseline.md. Review and required-workflow integration continue there. This PR remains open until protected delivery and a fresh full-delta audit; later changes here must also be carried forward.Outcome
Reduces deterministic Python-side preparation before the Rust-backed psychometric fit without changing scores, item order, or the fail-closed fit contract. Adds a reproducible accuracy-time KPI contract and research traceability.
Root cause
PsychometricRoutingEvidence._fit_locked()repeatedly scanned the complete context list while sorting item keys, making preparation quadratic at the 512-context ceiling.Minimal change
Build the context-position mapping once. The benchmark now fails with an actionable message below Python 3.12 and the documented command selects Python 3.12, matching the locked NumPy/fast-mlsirm markers while preserving product support for Python 3.10/3.11.
Autoresearch KPI
Same host/source, 512 contexts, 4 models, 2 dichotomous items/context, 9 samples:
main@2e414d15: median 2.448167 ms0561c9b8: median 1.100583 msThis isolates gateway preparation; it is not an answer-accuracy claim.
Exact-head verification (
3c22bf31)uv run --python 3.12 python scripts/benchmark_psychometric_routing.pygit diff --checkThe Python-runtime review finding is fixed and its thread is resolved.
ruffis not present in the locked environment, so no unpinned tool was installed.Summary by CodeRabbit
성능 개선
벤치마크
테스트
문서