Skip to content

fix(routing): select concrete free model groups - #971

Draft
seonghobae wants to merge 132 commits into
mainfrom
fix/model-group-timeout-openrouter
Draft

fix(routing): select concrete free model groups#971
seonghobae wants to merge 132 commits into
mainfrom
fix/model-group-timeout-openrouter

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Current owner state

This remains the broad trusted owner lane for concrete orchestrator/free grouping, provider discovery/routing, endpoint lifecycle, durable bootstrap identity and related routing behavior. It is Draft and not merge-ready.

  • protected main: 414f22973658c4ddc3d4320fcf7acd9b4e8ba991
  • current exact head: 50b0c869571f361865efd22c7c596fda978679e8
  • merge base: protected main itself
  • ancestry: ahead 126 / behind 0
  • GitHub mergeability: mergeable
  • publication: normal non-force two-parent merge; no rebase or force push

2026-09-07 durable selection-order repair

External thread PRRT_kwDOTB3CTs6elTYC was causal: durable bootstrap converted the selector result to a set and returned alphabetically sorted persisted IDs, while the ephemeral report preserved cost/model-group order. RED commit 98aed1a811cb894881b4c9aeb20de4f0b00fb634 adds a two-model durable-pool oracle whose selected order is deliberately opposite lexical agent-ID order. Successor d469085df5cf6db4cf57cd6da43569f77920c463 keeps set membership for fail-closed collision/withdrawal checks but separately records, activates, and reports resolved persisted IDs in selector order. It adds no ranking, provider quota, timeout, or dependency.

Exact-head Security and Quality run 34068287994 is terminal GREEN: unchanged full suite/package quality, Hypothesis/Atheris fuzzing, CodeQL, dependency audit and SBOM all succeeded. Code Quality 34068286556 is also GREEN. Remaining SAST, compatibility CodeQL, Security Scan, Noema, OpenCode and Strix lanes are current-head but still queued, so this PR remains Draft.

Restack and hosted-RED repair

The previous head df587e1704ecd12e355b4d1a70facb387a4648cb diverged from 13 protected descendants. The successor preserves both histories. The six paths changed on both sides were merged with exact base 212ff437dc297613289dba2e6064ade9942e07d8: five were conflict-free, while the setup-node block in .github/workflows/opencode-hourly-loop.yml required the explicit resolution recorded below. All 29 main-only path deltas, including security/test/docs changes and deletion of obsolete ci.yml/fuzz.yml, were inherited from protected main.

Tests run 33667736989, job 100373515693 had reported 7 failed, 3439 passed, 2 skipped. This successor repairs those failures without weakening production behavior:

  1. tests/test_admin_contract.py inherits protected main's json, shutil, and subprocess imports.
  2. Four synthetic OpenRouter ZDR batch fixtures inject the already-defined deterministic _ExactTestCounter; production remains fail-closed when authoritative token sizing is unavailable.
  3. The two durable bootstrap identity fixtures now call sync_discovered_agents before restart. TaskOrchestrator construction intentionally remains free of an implicit persistence side effect.

Restack commit 8b05b831… initially produced push run 34065942588 with failure and zero jobs because one unresolved three-way marker remained around actions/setup-node; 521e613… removed that artifact while preserving the immutable setup-node v7 pin and bounded non-model setup timeout. 6ddc6b4… then closed the vacuous direct-check path by giving both bootstrap identity fixtures real pytest entry points and registering both in README. Its hosted Tests run 34066167599, job 101575237781 exposed a single publication defect behind 325 collection errors: orchestrator.py had been truncated to 600,060 bytes and contained NUL bytes. Current successor 1e59d4f… restores the complete 792,692-byte merged blob; the GitHub content API reports no NUL characters. Fresh SAST 34066349130, CodeQL 34066349112, merge scheduler 34066348028, Strix 34066348040, OpenCode 34066348094, Noema 34066348025, Security/Quality 34066349029, Security Scan 34066349074, and Code Quality 34066347141 runs materialized for this exact head. Code Quality is terminal GREEN; the Security/Quality CodeQL+supply-chain+SBOM job is terminal GREEN; its unchanged full-suite and coverage-guided fuzz jobs are still executing. The other exact-head lanes remain queued/pending. Predecessor results do not authorize this head.

2026-09-07 ambiguous bootstrap-admission repair

External thread PRRT_kwDOTB3CTs6eh3BQ remained causal: both bootstrap
selectors let lexical provider/model identity decide a bounded cutoff when
price evidence was equal or incomplete. Test-only RED
eeb9cc1bafe579032ab48778fa08c24e0b3f0aa1 produced exact-head Security and
Quality run 34071330949, job 101589111271: the two new contracts failed
with DID NOT RAISE, and the unchanged remainder was 3490 passed, 2 skipped.

GREEN implementation 575148b9df50b4e3840bc25f4fba1367d8f6a654 retains explicit
provider/model-group availability constraints but rejects a selected/excluded
capacity boundary sharing the same comparable-cost or unknown-price state.
The error tells operators to supply comparable price evidence or include the
entire tied class. It adds no score, weight, quota, fuzzy identity, or learned
quality claim. The doctoring record now distinguishes direct provider
bootstrap's model-group contract from the discovery CLI's additional
provider-spread contract; ADR 0032 is demoted to Proposed while protected
delivery remains pending. Successor 50b0c869571f361865efd22c7c596fda978679e8 adds the corresponding equal-known-price edge cases without changing production. Exact-head Security/Quality 34072366669, SAST 34072366670, CodeQL 34072366652, and Security Scan 34072366665 have materialized but are not yet terminal. Both related review threads remain open until current-head focused/full evidence succeeds.

Preserved valid direction

The branch still owns valid repaired behavior that must survive review: purpose-complete self-modifying source-fix machinery stays absent; endpoint-race/process-exit cleanup contracts; bounded provider-embedding worker shutdown; durable bootstrap/catalog identity; provider discovery and free-pool evidence boundaries; default inference timeout null; separation of user cancellation, provider termination, audited administrative timeout and infrastructure loss.

GitHub Actions consumers request only orchestrator/free. Provider/model/group/credential discovery, free/privacy/capability admission, serving order and failover remain contextual-orchestrator authority. OpenAI integration may remain globally available while OpenAI-derived candidates remain excluded from the free pool under the current policy. Private candidates require ZDR admission.

Still-live owner RED

  • Bootstrap admission's equal/incomplete-evidence cutoff has a RED and minimal GREEN candidate on the current head; keep the manual-admission and migration threads open until exact-head full verification proves it.
  • Hourly model-backed work still needs durable/resumable execution semantics. A leaf finite wall-clock model timeout is not a substitute for checkpoint/re-dispatch.
  • Provider/model-group diversity and migration semantics need exact-head revalidation.
  • Three valid unresolved review threads remain: hourly resumability/queueing, diversity-migration semantics, and manual admission rules. Durable selection order is repaired and exact-head full-suite/fuzz verified. Outdated location alone is not resolution.

Promotion contract

Before Ready or ordinary protected merge:

  1. rerun the seven focused regressions on this exact head and repair any residual causal defect;
  2. run the complete unchanged-head suite plus docstring/test/edge contracts;
  3. require every live protected context on this SHA, including Hypothesis, Atheris, CodeQL, Python supply chain, dependency review, OSV, Trivy, Scorecard, coverage evidence, OpenCode review, Strix and PR-queue scan;
  4. resolve only findings proven obsolete or repaired on the current source and obtain qualifying independent review;
  5. keep ADR/PRD/ARCHITECTURE/CHANGELOG/product-gap evidence aligned with the resulting tree.

No force push, destructive rebase, stale-tree overlay, source-neutral retrigger, predecessor-evidence transfer, self-approval, administrator bypass or gate weakening is acceptance evidence.

Exclude OpenRouter's aggregate free router while retaining discovered concrete free models. Group exact model identities across provider accounts and remove fixed inference deadlines, including readiness races.

Signed-off-by: Seongho Bae <me@seonghobae.me>
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

모델 검색과 부트스트랩이 모델 그룹 및 fingerprint ID를 사용합니다. 공급자 호출은 선택적 타임아웃과 명시적 취소를 지원합니다. OpenRouter ZDR 라우팅, 임베딩 실패 기록, readiness 동시성, 레거시 에이전트 동기화 및 관련 회귀 검증을 갱신합니다.

Changes

모델 검색 및 부트스트랩

Layer / File(s) Summary
모델 그룹 식별 및 검색 제한
contextual_orchestrator/model_discovery.py, contextual_orchestrator/conventions.py, contextual_orchestrator/provider_bootstrap.py
모델 그룹과 SHA-256 기반 agent ID를 추가했습니다. 레거시 ID 재사용을 지원합니다. provider별 discovery deadline과 openrouter/free 제외를 적용합니다.
부트스트랩 선택 및 동기화
contextual_orchestrator/provider_bootstrap.py, contextual_orchestrator/provider_catalog_bootstrap.py, contextual_orchestrator/__main__.py, tests/test_*bootstrap*, tests/test_discover_models_cli.py
provider 및 model group 다양성을 순서대로 적용합니다. 운영자 관리 에이전트와의 ID 충돌을 거부합니다. 선택된 모델을 동기화 후 다시 매칭합니다.

공급자 취소 및 readiness

Layer / File(s) Summary
취소 가능한 공급자 호출
contextual_orchestrator/orchestrator.py, contextual_orchestrator/endpoint_race.py, tests/test_endpoint_race.py, tests/test_local_mlx.py
DNS, TCP, 소켓 종료까지 공급자 취소를 확장했습니다. race loser 결과를 세분화해 기록합니다.
무기한 probe 및 readiness 갱신
contextual_orchestrator/orchestrator.py, README.md, contextual_orchestrator/api_contract.py, tests/test_local_mlx.py
기본 probe 및 연결 타임아웃을 선택 사항으로 변경했습니다. 동시 refresh에는 refresh_in_progress를 반환합니다.

ZDR 라우팅 및 임베딩

Layer / File(s) Summary
OpenRouter ZDR 요청 보정
contextual_orchestrator/orchestrator.py, contextual_orchestrator/batch_routing.py, contextual_orchestrator/cost_router.py, tests/test_orchestrator_client_boundaries.py
OpenRouter의 실제 base_url을 기준으로 provider를 식별합니다. zdr_only 범위에서 모든 주요 전송 경로와 Batch JSONL에 provider.zdr=true를 적용합니다.
임베딩 실행 및 실패 상태
contextual_orchestrator/server.py, contextual_orchestrator/cost_router.py, contextual_orchestrator/batch_routing.py, tests/test_batch_embeddings.py, tests/test_provider_embedding_batch_backend.py
무기한 실행과 고정 claim lease를 분리합니다. 불완전한 결과와 terminal batch 상태를 embedding failure로 기록합니다. 복구된 ZDR batch의 현재 태그를 재검증합니다.

계약 및 운영 지원

Layer / File(s) Summary
재시도·운영 문서 및 workflow
docs/adr/*, docs/planning/adrs/*, CHANGELOG.md, CHANGELOG.d/*, docs/product-technical-gap-baseline.md, .github/workflows/*, docs/kv-credentials.md
기본 transport retry를 0으로 기록합니다. OpenRouter evidence 및 ZDR 정책을 갱신합니다. workflow 단계별 제한과 gateway 프로세스 상태 확인을 추가합니다.
회귀 계약
tests/test_pr971_review_quality_regressions.py, tests/test_provider_gateway_resilience.py, tests/test_openrouter_uptime.py, tests/test_embeddings_model_pool_http_honesty.py, tests/test_provider_error_taxonomy.py
discovery deadline, provider 오류 분류, uptime timeout, embedding failure, ZDR 정책 복원을 검증합니다.
임시 수리 자동화
.github/workflows/source-fix-971-live-review-quality.yml, scripts/ci/pr971_live_review_quality_repair.py
RED 테스트 실행, 소스 수리, GREEN 검증, traceability 갱신 및 수리 파일 삭제 순서를 정의합니다.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🔴 Critical · up to 03b62

The current head still relies on temporary source-rewriting machinery instead of committed production fixes, while provider discovery can wait indefinitely or leak background work and embedding and failover paths retain correctness and privacy risks. These issues can cause hangs, resource exhaustion, incorrect routing, or loss of privacy guarantees, so the PR is not merge-ready and should remain blocked.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 52.20% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 341 functions across 44 files. (3 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 구체적인 무료 모델 그룹을 라우팅 대상으로 선택하는 실제 변경을 정확히 설명합니다. 전체 변경 범위를 모두 포함하지 않지만, 제목은 주요 변경 사항의 일부를 간결하게 요약할 수 있습니다.
Full details: Docstring Coverage

Explanation

Docstring coverage is 52.20% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 341 functions across 44 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
⚔️ Resolve merge conflicts 💡
  • Resolve merge conflict in branch fix/model-group-timeout-openrouter
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/model-group-timeout-openrouter

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Routing contract: provider family is not a product grouping. Selection and measurement use model_group only. OpenRouter discovery retains concrete free model IDs such as vendor/model:free; the aggregate openrouter/free meta-router is excluded because contextual-orchestrator itself owns concrete-model selection, RPM/TPM measurement, and failover.

devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Seongho Bae <me@seonghobae.me>
@seonghobae
seonghobae enabled auto-merge (squash) August 31, 2026 12:16
@opencode-agent
opencode-agent Bot disabled auto-merge August 31, 2026 13:18
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

Signed-off-by: Seongho Bae <me@seonghobae.me>
devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Seongho Bae <me@seonghobae.me>
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae
seonghobae enabled auto-merge (squash) August 31, 2026 23:01
Signed-off-by: Seongho Bae <me@seonghobae.me>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

Devin Review

Copy link
Copy Markdown
Contributor Author

Fresh consumer evidence that should be carried by this owner lane rather than worked around in leaves:

  • ContextualWisdomLab/keyverse#132 is migrating its hourly model-backed workflow to orchestrator/free, but it still has to vendor/pin contextual-orchestrator source revision 464da4715b495b5eaaa593eba3796e2d976ee0c9. The owner repository currently has no GitHub latest release endpoint (/releases/latest returns 404). Under the CWL boundary, a consumer must not treat a sibling source commit as the durable released client/schema contract; this owner lane needs to land a canonical immutable release/client revision and provide an upgrade target, after which Keyverse can bump to that released contract.
  • Keyverse also inherits OPENCODE_RUN_TIMEOUT_SECONDS=2100; EgressWeave#235 currently has a 45-minute Actions job timeout around the same gateway-backed OpenCode pattern. Both demonstrate the same owner/runtime gap already described in this PR body: slow reasoning/streaming/tool-call work cannot be classified as failed solely from elapsed time, but a hung external execution also must not serialize the hourly lane forever. Please preserve the timeout=null inference contract and solve resumability/checkpoint/re-dispatch as an owner runtime concern, with explicit user/provider/admin/infrastructure terminal states rather than leaf wall-clock wrappers.

Owner RED/GREEN acceptance to add before release: a long-lived orchestrator/free request survives beyond the former leaf timeout without synthetic model failure; explicit user cancellation/provider termination/audited admin timeout/infrastructure loss remain distinguishable; an interrupted maintenance execution can resume/re-dispatch with exact-head/checkpoint identity; and the resulting released API/client/schema is immutable enough for consumers to pin without vendoring this repository's source. No paid/provider-specific fallback should be required.

Fresh cross-repo consumer evidence on PR #971 (keyverse#132, EgressWeave#235)
independently hit the same owner-lane gap: no canonical immutable release
means consumers vendor a raw source SHA, and no owner-side resumable/
checkpointed execution model means every leaf re-invents its own wall-clock
timeout wrapper around the timeout=null inference contract. Records the
gap, why it's owner-lane (two independent consumers, not one), scoped action
items (SemVer release; a dedicated ADR for resumability with distinguishable
terminal states), and the owner's own stated RED/GREEN acceptance criteria,
per this repo's standing directive to keep the gap baseline current from
live PR/consumer evidence.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4

Copy link
Copy Markdown
Contributor Author

Acknowledged — recorded this evidence and the two scoped action items in docs/product-technical-gap-baseline.md (pushed as 2d8e233a).

This is genuinely two separable, architecturally significant pieces of work, not a small local fix, so I'm not folding an implementation into this PR (which is already close to mergeable on its own narrower scope: model-group timeout/thread-pool-shutdown fixes). Proposing to track them as their own dedicated tracks:

  1. Canonical immutable release — publish a tagged SemVer GitHub Release for this repo so consumers (keyverse and others) can pin /releases/latest instead of vendoring a source SHA. This is the more bounded of the two — no design questions, just release process.
  2. Resumable long-running execution — needs its own ADR before any code: distinguishable terminal states (user cancellation / provider termination / audited admin timeout / infrastructure loss), checkpoint/exact-head re-dispatch, while strictly preserving the existing timeout=null inference contract. This is the wider one and deserves design review before implementation, per this repo's own Ponytail gate.

Will pick these up as the next highest-leverage gap items once the current PR review/fix backlog is exhausted, per the standing loop. Flag if you'd rather see either scoped differently or prioritized sooner.


🤖 Generated with Claude Code


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

Two real citation errors in the 2026-09-02 canonical-release/resumable-
execution entry: docs/CWL-MASTER-CONTEXT.md does not exist in this repo (it
is ContextualWisdomLab/.github's org-wide doc) and this repo's AGENTS.md
carries no such rule directly -- corrected to cite it as the .github repo's
document. The no-fixed-timeout policy citation named "ADR-0005", which is
the .github repo's numbering for a related policy; this repo's own
equivalent is docs/planning/adrs/0032-model-group-cost-aware-discovery.md
-- corrected to cite that file directly with its exact text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 0 new potential issues.

Devin Review

Copy link
Copy Markdown
Contributor Author

"RED to GREEN owner repair" failure — not a defect in this PR's diff, already self-resolved

Investigated the failing required-looking check RED to GREEN owner repair (run 33632367429, job 100254717794, conclusion failure).

What it actually is

That job belongs to workflow .github/workflows/source-fix-971-live-review-quality.yml (display name "PR 971 exact-head review-quality repair" / "PR 971 live review quality repair") — one of this PR's own one-shot, branch-scoped, self-modifying source-fix workflows, same class already documented and retired earlier today (see the 06:19 / 08:18 / 09:21 UTC comments above about source-fix-971-default-retry-policy and source-fix-971-exact-head-review/source-fix-971-runtime-context-green).

Why it failed

The failed step is its own safety guard (Guard exact writer head):

remote_head="$(git ls-remote origin "refs/heads/${GITHUB_REF_NAME}" | cut -f1)"
test "$remote_head" = "$GITHUB_SHA"
test -e scripts/ci/pr971_live_review_quality_repair.py

This run was queued for the push at fe13068c (created_at 12:52:16Z) but, under the same org-wide Actions capacity congestion already called out earlier on this PR, didn't actually get a runner until 23:32:23Z — over 10.5 hours later. By then the branch tip had moved well past fe13068c (through this session's own 2d8e233a / df587e17, currently the PR head), so the guard's exact-head check failed closed exactly as designed, rather than letting a stale one-shot workflow mutate source against an out-of-date checkout.

Confirms it isn't live

  • .github/workflows/source-fix-971-live-review-quality.yml and scripts/ci/pr971_live_review_quality_repair.py no longer exist on the current head (df587e17) — this workflow has already been removed per the standing self-removal convention for this repair machinery.
  • All 7 runs of this workflow (workflow_id 348344002) are completed (this failure plus prior cancelled/failure attempts); none are queued. There is nothing left to retrigger — the workflow file that would need re-running is gone from the branch, and this job does not appear among the 24 check runs on the current head, so it is not gating the current head.

Conclusion

Not a defect introduced by this PR's diff — it's the same known org-wide GitHub Actions queue-congestion pattern hitting a now-already-deleted one-shot repair workflow's own stale-head guard. No source change is needed or applicable here; the machinery that produced this failing run is already gone from the branch. Current-head required checks (Security, CodeQL, Full unit and contract suite, etc.) remain the authoritative gate and are tracked separately.


Generated by Claude Code

@opencode-agent

opencode-agent Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: df587e1704ecd12e355b4d1a70facb387a4648cb

@opencode-agent

opencode-agent Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: df587e1704ecd12e355b4d1a70facb387a4648cb

@seonghobae
seonghobae marked this pull request as draft September 4, 2026 10:19

Copy link
Copy Markdown
Contributor Author

@jules Fresh exact-head review on df587e1704ecd12e355b4d1a70facb387a4648cb found a still-live availability contradiction in the canonical free-pool owner lane. The current code has a circuit breaker, but _eligible_for_role returns healthy or eligible; when every eligible member is still circuit-open it deliberately re-admits the known-open members. The same current branch hard-codes circuit_failure_threshold = 3 and circuit_reset_seconds = 30.0. That can make an all-saturated orchestrator/free request serially pay known-bad provider work again, and the two numeric decisions are hand-authored policy of the same class this PR already rejects for default retries/admission ordering.

This matters with the fleet's current measured consumer evidence: same-day model-backed Actions stalls have grown from roughly 649.5s to 1332.6s, 1462.9s, and 2161.9s while holding scarce job capacity. Do not 'fix' that by reintroducing a global model wall-clock timeout: #1053 correctly moves the default application model timeout to None, so active reasoning/streaming/tool work remains unbounded unless the caller/user/provider/admin terminates it. The repair belongs in provider-health/admission, not elapsed inference time.

Please repair this same branch by normal descendant after re-reading live main/head. RED/GREEN acceptance:

  • if one or more eligible members are currently healthy, no circuit-open member may be invoked;
  • if all eligible members are still open from fresh source-backed provider failures, fail closed with a typed pool-temporarily-unavailable outcome carrying only safe retry/eligibility evidence instead of probing every known-open member in series;
  • after the health observation's governed expiry/recovery condition, the member becomes probe-eligible again, and a successful probe clears its failure state;
  • keep provider 429/Retry-After, quota/auth/permanent refusal, transport failure, provider termination, and model-content/reasoning duration as distinct evidence classes; do not count explicit race-loser cancellation as provider failure;
  • do not embed provider/model/group names or a paid fallback in consumers; orchestrator/free remains the only GitHub Actions selector;
  • replace the unexplained 3/30 literals with explicit configurable/evidence-governed policy or document an accepted standard/experiment that justifies them; configuration must preserve fail-closed bounds and deterministic tests;
  • add deterministic concurrent-request/herd regressions so a fresh open state prevents duplicate known-bad attempts, all-open requests do not fan out or serialize through the open set, and recovery does not permanently starve a provider.

Also update this PR's stale 'current exact head' prose after the causal repair; predecessor tests/reviews do not transfer. Keep Draft until the unchanged repaired head has exact tests/security/review/SBOM/provenance.

@opencode-agent

opencode-agent Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: df587e1704ecd12e355b4d1a70facb387a4648cb

Copy link
Copy Markdown
Contributor Author

Fresh fleet revalidation on current exact head df587e1704ecd12e355b4d1a70facb387a4648cb supersedes the stale head/base text in the PR body. GitHub now reports this PR as Draft. Current protected main is a080297d2546bb61e89520d637cabc202db331ec; comparison is diverged (ahead_by=122, behind_by=12, merge base 212ff437dc297613289dba2e6064ade9942e07d8). The 12 protected descendants are not discardable: they overlap this lane in model_discovery.py, orchestrator.py, the hourly OpenCode workflow/contracts, and tests/test_admin_contract.py, among other owner paths. Promotion therefore requires a conflict-aware normal descendant restack, not a stale-tree overlay or force rebase.

The last exact-head hosted full suite is also a real source/test RED, not merely queue starvation: Tests run 33667736989, job 100373515693 checked out the exact PR merge and finished 7 failed, 3439 passed, 2 skipped. One failure is a deterministic missing-import defect in tests/test_admin_contract.py (json is used without import; the same file later uses shutil and subprocess). Current protected main already contains all three imports, so that protected delta must be adopted during restack rather than independently reimplemented. Four embedding-batch tests fail closed because no authoritative tokenizer is available for text-embedding-3-small, and the two durable-bootstrap identity regressions still report generated IDs instead of the persisted legacy ID on this exact head. NIM coverage/docstrings/package smoke passed; Fuzz, OSV, Security Scan, SAST and Scorecard passed, while Security failed and CodeQL ended startup_failure.

Required GREEN: first reconcile all 12 protected descendants with a normal non-force descendant while preserving this PR's valid routing/endpoint/bootstrap deltas; then re-run the seven concrete failures and repair any residual current-tree defects causally; only after that regenerate the full exact-head required protection set (Hypothesis, Atheris, CodeQL, supply-chain/dependency/OSV/Trivy/Scorecard/coverage/OpenCode/Strix/scan-pr-queue) and current independent review. No predecessor result, stale merge commit, no-op retrigger, source overlay, self-approval, or gate weakening is acceptance evidence.

@opencode-agent

opencode-agent Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: df587e1704ecd12e355b4d1a70facb387a4648cb

Preserve both histories with a normal two-parent merge against protected main@414f22973658c4ddc3d4320fcf7acd9b4e8ba991. Six overlapping files were conflict-free under exact merge-base 212ff43; main-only security, test, documentation, and removed legacy workflow deltas are retained.

Repair the seven hosted test failures without weakening production fail-closed behavior: inherit main's json/shutil/subprocess admin-test imports, inject the existing deterministic authoritative embedding counter into four synthetic OpenRouter ZDR fixtures, and durably upsert each legacy bootstrap identity before restart. TaskOrchestrator construction is intentionally not a persistence side effect.

No production token-count fallback, identity heuristic, timeout, retry, provider/model selection, or check threshold is changed.
The exact-head push run 34065942588 failed at workflow startup with zero jobs because the restack left one three-way conflict marker block around setup-node. Retain the protected-main setup-node v7 immutable pin and the branch's bounded non-model setup step. No model, gateway, routing, timeout, or permission behavior is changed.
The review thread identified a vacuous direct-check path: invoking either new identity regression file as documented for neighboring checks ran zero tests and returned success. Add pytest entry points to both bootstrap and catalog companions and register both commands in README. Production discovery, persistence, routing, and admission behavior is unchanged.
Exact-head Tests run 34066167599, job 101575237781 failed with 325 collection errors sharing one cause: contextual_orchestrator/orchestrator.py contained NUL bytes and was only 600,060 bytes after connector-side blob truncation. Restore the complete 792,692-byte conflict-free three-way merge result. No source semantics are otherwise changed.
RED: durable provider bootstrap currently alphabetizes enabled agent IDs instead of preserving the selector's cost/model-group order.
Keep resolved durable IDs in the selector's cost/model-group order while retaining set-based membership and identity-collision checks. Documents the exact external-review regression and acceptance state.
The ordered durable identity accumulator now owns report order; retain only the membership set used by collision and withdrawal checks.

Copy link
Copy Markdown
Contributor Author

Canonical-owner finding to integrate after the current exact-head writer completes (verified 2026-09-07): #972@876f1679cd44d7e9691b95344118bfe324e31287 admits supports_parallel_tool_calls=None/untagged agents into blind orchestrator/free, and its tests explicitly preserve [multi, unknown]. That conflicts with the binding missing-capability fail-closed rule. Preserve unknown rows in discovery/price inventory and non-tool/request-scoped routes, but require affirmative request-relevant capability evidence for the blind free pool. The same probe authors timeout=30.0, max_tokens=32, and temperature=0.0 as admission-affecting compute constants without cited standard/experiment/caller provenance. Full exact-head evidence and integration direction are recorded on #972 in comment 5563657095. Add RED contracts before source repair; do not replace these values with different constants or disturb the current 50b0c869571f361865efd22c7c596fda978679e8 verification run.

@opencode-agent

opencode-agent Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Scheduled review-feedback autofix for this PR head.

  • Head SHA: 50b0c869571f361865efd22c7c596fda978679e8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority: high status: draft type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants