fix(discovery): union Models.dev metadata with provider evidence, not clobber - #1007
fix(discovery): union Models.dev metadata with provider evidence, not clobber#1007seonghobae wants to merge 8 commits into
Conversation
… clobber _merge_models_dev_metadata unconditionally overwrote a matched row's architecture/max_output_tokens/context_window with Models.dev's values even when Models.dev had no modalities/limit data for that model, silently discarding the provider's own already-discovered catalog evidence for nothing. Reproduced live with a synthetic partial record, then fixed: these capacity/modality fields are now a field-level union (Models.dev's value wins only when Models.dev actually reports one; the provider's own value survives otherwise). Cost (pricing/is_free) is intentionally left untouched and stays exclusively Models.dev-sourced per ADR 0041's cost-safety argument -- a compromised provider must never be able to self-report "free". Also records live verification of two adjacent gaps that were already correct on main and needed only regression coverage, not a fix: - OpenRouter is already a live PROVIDER_MODEL_SOURCES entry feeding orchestrator/free with no provider-specific carve-out; verified live against the real OpenRouter API (566 models discovered, 10 correctly classified free). - TaskOrchestrator._zdr_agent_allowed's `not zdr_required or has_zdr_tag` boolean makes it mathematically impossible for a non-ZDR-requiring pool to exclude a ZDR-capable agent; added an end-to-end regression test through bootstrap activation covering ZDR-capable free models from two provider families at once. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reachedNext included review available in 43 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (4)
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (5)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughModels.dev 메타데이터 병합이 필드별 보존 방식으로 변경되었습니다. Models.dev가 값을 보고하지 않으면 제공자 카탈로그 값을 유지합니다. 무료 모델 풀의 ZDR 선택 동작에 대한 회귀 테스트와 라이브 검증 문서도 추가되었습니다. Changes모델 메타데이터 병합
무료 모델 풀의 ZDR 검증
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to This change prevents incomplete catalog metadata from erasing valid provider capability and capacity information while preserving existing pricing and privacy behavior. No actionable merge-blocking risk remains beyond normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files. (2 skipped: 2 unsupported.) ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
|
Routine staleness triage: this PR's base was behind current Verification before push:
No source changes beyond the merge commit itself; fresh exact-head CI evidence is still required before merge per this repo's governance. Generated by Claude Code |
Remove OpenRouter and ZDR evidence already superseded by protected main.\n\nCommit-Message-Assisted-by: Codex Signed-off-by: Seongho Bae <me@seonghobae.me>
Commit-Message-Assisted-by: Codex Signed-off-by: Seongho Bae <me@seonghobae.me>
…etadata-union Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Adjudication evidence (host 1 session, 2026-09-06 KST; full report with commands in #1080). Nothing here closes, flips, or retargets anything — the decision is the opener's. Contained in #1031. No closure needed if this lands first — #1007 → #1031 is merge-tree clean and is the order the discovery cluster assumes. If #1031 lands first, this PR becomes superseded (re-check: |
2 participants
Summary
architecture,context_window, andmax_output_tokenswhen a matching Models.dev record omits the corresponding field.Successor reconciliation
Protected
main@60c562defc81fb1897fa97ebdb5bf8f69eae0c55includes PR #1047 (17095dad706199a2e108ea6c20d7e6f1f5ac3b89) and PR #1048 (60c562defc81fb1897fa97ebdb5bf8f69eae0c55). This branch merged that main without force-push.PR #1047 already supplies OpenRouter plus Models.dev composition and its privacy regression coverage. The earlier OpenRouter/ZDR status prose and bootstrap test were therefore removed from this PR. The remaining delta is limited to the independent metadata-union implementation, its focused regression test, and the changelog fragment.
Verification
uv run --group dev pytest -q tests/test_model_discovery.py tests/test_provider_bootstrap.py— 153 passeduv run --group dev python -m compileall -q contextual_orchestrator/model_discovery.py— passedgit diff --check— passedCurrent exact head:
ef530306onmain@60c562de. Hosted exact-head checks and fresh robot review remain required before merge.