fix(routing): select concrete free model groups - #971
Conversation
Exclude OpenRouter's aggregate free router while retaining discovered concrete free models. Group exact model identities across provider accounts and remove fixed inference deadlines, including readiness races. Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthrough모델 검색과 부트스트랩이 모델 그룹 및 fingerprint ID를 사용합니다. 공급자 호출은 선택적 타임아웃과 명시적 취소를 지원합니다. OpenRouter ZDR 라우팅, 임베딩 실패 기록, readiness 동시성, 레거시 에이전트 동기화 및 관련 회귀 검증을 갱신합니다. Changes모델 검색 및 부트스트랩
공급자 취소 및 readiness
ZDR 라우팅 및 임베딩
계약 및 운영 지원
Estimated code review effort: 5 (Critical) | ~120 minutes Merge Risk: 🔴 Critical · up to The current head still relies on temporary source-rewriting machinery instead of committed production fixes, while provider discovery can wait indefinitely or leak background work and embedding and failover paths retain correctness and privacy risks. These issues can cause hangs, resource exhaustion, incorrect routing, or loss of privacy guarantees, so the PR is not merge-ready and should remain blocked. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 52.20% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 341 functions across 44 files. (3 skipped: 3 unsupported.)
✨ Finishing Touches 💡 2📝 Generate docstrings 💡
⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Routing contract: provider family is not a product grouping. Selection and measurement use |
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Fresh consumer evidence that should be carried by this owner lane rather than worked around in leaves:
Owner RED/GREEN acceptance to add before release: a long-lived |
Fresh cross-repo consumer evidence on PR #971 (keyverse#132, EgressWeave#235) independently hit the same owner-lane gap: no canonical immutable release means consumers vendor a raw source SHA, and no owner-side resumable/ checkpointed execution model means every leaf re-invents its own wall-clock timeout wrapper around the timeout=null inference contract. Records the gap, why it's owner-lane (two independent consumers, not one), scoped action items (SemVer release; a dedicated ADR for resumability with distinguishable terminal states), and the owner's own stated RED/GREEN acceptance criteria, per this repo's standing directive to keep the gap baseline current from live PR/consumer evidence. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
|
Acknowledged — recorded this evidence and the two scoped action items in This is genuinely two separable, architecturally significant pieces of work, not a small local fix, so I'm not folding an implementation into this PR (which is already close to mergeable on its own narrower scope: model-group timeout/thread-pool-shutdown fixes). Proposing to track them as their own dedicated tracks:
Will pick these up as the next highest-leverage gap items once the current PR review/fix backlog is exhausted, per the standing loop. Flag if you'd rather see either scoped differently or prioritized sooner. 🤖 Generated with Claude Code Generated by Claude Code |
Two real citation errors in the 2026-09-02 canonical-release/resumable- execution entry: docs/CWL-MASTER-CONTEXT.md does not exist in this repo (it is ContextualWisdomLab/.github's org-wide doc) and this repo's AGENTS.md carries no such rule directly -- corrected to cite it as the .github repo's document. The no-fixed-timeout policy citation named "ADR-0005", which is the .github repo's numbering for a related policy; this repo's own equivalent is docs/planning/adrs/0032-model-group-cost-aware-discovery.md -- corrected to cite that file directly with its exact text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
"RED to GREEN owner repair" failure — not a defect in this PR's diff, already self-resolvedInvestigated the failing required-looking check What it actually isThat job belongs to workflow Why it failedThe failed step is its own safety guard ( This run was queued for the push at Confirms it isn't live
ConclusionNot a defect introduced by this PR's diff — it's the same known org-wide GitHub Actions queue-congestion pattern hitting a now-already-deleted one-shot repair workflow's own stale-head guard. No source change is needed or applicable here; the machinery that produced this failing run is already gone from the branch. Current-head required checks (Security, CodeQL, Full unit and contract suite, etc.) remain the authoritative gate and are tracked separately. Generated by Claude Code |
|
Scheduled review-feedback autofix for this PR head.
|
|
Scheduled review-feedback autofix for this PR head.
|
|
@jules Fresh exact-head review on This matters with the fleet's current measured consumer evidence: same-day model-backed Actions stalls have grown from roughly 649.5s to 1332.6s, 1462.9s, and 2161.9s while holding scarce job capacity. Do not 'fix' that by reintroducing a global model wall-clock timeout: #1053 correctly moves the default application model timeout to Please repair this same branch by normal descendant after re-reading live main/head. RED/GREEN acceptance:
Also update this PR's stale 'current exact head' prose after the causal repair; predecessor tests/reviews do not transfer. Keep Draft until the unchanged repaired head has exact tests/security/review/SBOM/provenance. |
|
Scheduled review-feedback autofix for this PR head.
|
|
Fresh fleet revalidation on current exact head The last exact-head hosted full suite is also a real source/test RED, not merely queue starvation: Tests run Required GREEN: first reconcile all 12 protected descendants with a normal non-force descendant while preserving this PR's valid routing/endpoint/bootstrap deltas; then re-run the seven concrete failures and repair any residual current-tree defects causally; only after that regenerate the full exact-head required protection set ( |
|
Scheduled review-feedback autofix for this PR head.
|
Preserve both histories with a normal two-parent merge against protected main@414f22973658c4ddc3d4320fcf7acd9b4e8ba991. Six overlapping files were conflict-free under exact merge-base 212ff43; main-only security, test, documentation, and removed legacy workflow deltas are retained. Repair the seven hosted test failures without weakening production fail-closed behavior: inherit main's json/shutil/subprocess admin-test imports, inject the existing deterministic authoritative embedding counter into four synthetic OpenRouter ZDR fixtures, and durably upsert each legacy bootstrap identity before restart. TaskOrchestrator construction is intentionally not a persistence side effect. No production token-count fallback, identity heuristic, timeout, retry, provider/model selection, or check threshold is changed.
The exact-head push run 34065942588 failed at workflow startup with zero jobs because the restack left one three-way conflict marker block around setup-node. Retain the protected-main setup-node v7 immutable pin and the branch's bounded non-model setup step. No model, gateway, routing, timeout, or permission behavior is changed.
The review thread identified a vacuous direct-check path: invoking either new identity regression file as documented for neighboring checks ran zero tests and returned success. Add pytest entry points to both bootstrap and catalog companions and register both commands in README. Production discovery, persistence, routing, and admission behavior is unchanged.
Exact-head Tests run 34066167599, job 101575237781 failed with 325 collection errors sharing one cause: contextual_orchestrator/orchestrator.py contained NUL bytes and was only 600,060 bytes after connector-side blob truncation. Restore the complete 792,692-byte conflict-free three-way merge result. No source semantics are otherwise changed.
RED: durable provider bootstrap currently alphabetizes enabled agent IDs instead of preserving the selector's cost/model-group order.
Keep resolved durable IDs in the selector's cost/model-group order while retaining set-based membership and identity-collision checks. Documents the exact external-review regression and acceptance state.
The ordered durable identity accumulator now owns report order; retain only the membership set used by collision and withdrawal checks.
|
Canonical-owner finding to integrate after the current exact-head writer completes (verified 2026-09-07): |
|
Scheduled review-feedback autofix for this PR head.
|
Current owner state
This remains the broad trusted owner lane for concrete
orchestrator/freegrouping, provider discovery/routing, endpoint lifecycle, durable bootstrap identity and related routing behavior. It is Draft and not merge-ready.414f22973658c4ddc3d4320fcf7acd9b4e8ba99150b0c869571f361865efd22c7c596fda978679e82026-09-07 durable selection-order repair
External thread
PRRT_kwDOTB3CTs6elTYCwas causal: durable bootstrap converted the selector result to a set and returned alphabetically sorted persisted IDs, while the ephemeral report preserved cost/model-group order. RED commit98aed1a811cb894881b4c9aeb20de4f0b00fb634adds a two-model durable-pool oracle whose selected order is deliberately opposite lexical agent-ID order. Successord469085df5cf6db4cf57cd6da43569f77920c463keeps set membership for fail-closed collision/withdrawal checks but separately records, activates, and reports resolved persisted IDs in selector order. It adds no ranking, provider quota, timeout, or dependency.Exact-head Security and Quality run 34068287994 is terminal GREEN: unchanged full suite/package quality, Hypothesis/Atheris fuzzing, CodeQL, dependency audit and SBOM all succeeded. Code Quality
34068286556is also GREEN. Remaining SAST, compatibility CodeQL, Security Scan, Noema, OpenCode and Strix lanes are current-head but still queued, so this PR remains Draft.Restack and hosted-RED repair
The previous head
df587e1704ecd12e355b4d1a70facb387a4648cbdiverged from 13 protected descendants. The successor preserves both histories. The six paths changed on both sides were merged with exact base212ff437dc297613289dba2e6064ade9942e07d8: five were conflict-free, while the setup-node block in.github/workflows/opencode-hourly-loop.ymlrequired the explicit resolution recorded below. All 29 main-only path deltas, including security/test/docs changes and deletion of obsoleteci.yml/fuzz.yml, were inherited from protected main.Tests run
33667736989, job100373515693had reported 7 failed, 3439 passed, 2 skipped. This successor repairs those failures without weakening production behavior:tests/test_admin_contract.pyinherits protected main'sjson,shutil, andsubprocessimports._ExactTestCounter; production remains fail-closed when authoritative token sizing is unavailable.sync_discovered_agentsbefore restart.TaskOrchestratorconstruction intentionally remains free of an implicit persistence side effect.Restack commit
8b05b831…initially produced push run34065942588with failure and zero jobs because one unresolved three-way marker remained aroundactions/setup-node;521e613…removed that artifact while preserving the immutable setup-node v7 pin and bounded non-model setup timeout.6ddc6b4…then closed the vacuous direct-check path by giving both bootstrap identity fixtures real pytest entry points and registering both in README. Its hosted Tests run34066167599, job101575237781exposed a single publication defect behind 325 collection errors:orchestrator.pyhad been truncated to 600,060 bytes and contained NUL bytes. Current successor1e59d4f…restores the complete 792,692-byte merged blob; the GitHub content API reports no NUL characters. Fresh SAST34066349130, CodeQL34066349112, merge scheduler34066348028, Strix34066348040, OpenCode34066348094, Noema34066348025, Security/Quality34066349029, Security Scan34066349074, and Code Quality34066347141runs materialized for this exact head. Code Quality is terminal GREEN; the Security/Quality CodeQL+supply-chain+SBOM job is terminal GREEN; its unchanged full-suite and coverage-guided fuzz jobs are still executing. The other exact-head lanes remain queued/pending. Predecessor results do not authorize this head.2026-09-07 ambiguous bootstrap-admission repair
External thread
PRRT_kwDOTB3CTs6eh3BQremained causal: both bootstrapselectors let lexical provider/model identity decide a bounded cutoff when
price evidence was equal or incomplete. Test-only RED
eeb9cc1bafe579032ab48778fa08c24e0b3f0aa1produced exact-head Security andQuality run
34071330949, job101589111271: the two new contracts failedwith
DID NOT RAISE, and the unchanged remainder was3490 passed, 2 skipped.GREEN implementation
575148b9df50b4e3840bc25f4fba1367d8f6a654retains explicitprovider/model-group availability constraints but rejects a selected/excluded
capacity boundary sharing the same comparable-cost or unknown-price state.
The error tells operators to supply comparable price evidence or include the
entire tied class. It adds no score, weight, quota, fuzzy identity, or learned
quality claim. The doctoring record now distinguishes direct provider
bootstrap's model-group contract from the discovery CLI's additional
provider-spread contract; ADR 0032 is demoted to Proposed while protected
delivery remains pending. Successor
50b0c869571f361865efd22c7c596fda978679e8adds the corresponding equal-known-price edge cases without changing production. Exact-head Security/Quality34072366669, SAST34072366670, CodeQL34072366652, and Security Scan34072366665have materialized but are not yet terminal. Both related review threads remain open until current-head focused/full evidence succeeds.Preserved valid direction
The branch still owns valid repaired behavior that must survive review: purpose-complete self-modifying source-fix machinery stays absent; endpoint-race/process-exit cleanup contracts; bounded provider-embedding worker shutdown; durable bootstrap/catalog identity; provider discovery and free-pool evidence boundaries; default inference timeout
null; separation of user cancellation, provider termination, audited administrative timeout and infrastructure loss.GitHub Actions consumers request only
orchestrator/free. Provider/model/group/credential discovery, free/privacy/capability admission, serving order and failover remain contextual-orchestrator authority. OpenAI integration may remain globally available while OpenAI-derived candidates remain excluded from the free pool under the current policy. Private candidates require ZDR admission.Still-live owner RED
Promotion contract
Before Ready or ordinary protected merge:
No force push, destructive rebase, stale-tree overlay, source-neutral retrigger, predecessor-evidence transfer, self-approval, administrator bypass or gate weakening is acceptance evidence.