[CO owner] eliminate orchestrator/free provider-family SPOF before gateway release

[seonghobae]

2026-09-05 ownership correction. The current organization operating contract has already selected the third option below: retain orchestrator/free, fail closed when capability is unavailable, and repair provider-family diversity plus the immutable gateway release in the contextual-orchestrator owner lane. This is not awaiting an external product or user decision. Contextual-orchestrator issue #1041 comment 5550590339 has claimed the six-point immutable-release prerequisite; central .github issue #1759 owns the subsequent released-artifact migration and unchanged NewsDOM #682 canary. Keep this issue open as the free-pool availability acceptance tracker, without reintroducing paid/direct provider fallback or treating infrastructure failure as a consumer source finding.

Why this is an issue, not another ADR edit

docs/adr/0003-contextual-orchestrator-vendored-free-zdr.md:169-181 records that Strix (the organization-wide required security-review gate) was switched from orchestrator/auto (provider-diverse, paid-inclusive) to orchestrator/free (zero-cost) on 2026-08-30, and that this removed Strix's only fallback: a live 2026-08-30 reproduction showed the free pool's discovered routes collapsing to one provider family (4/4 candidates rejected — 2 timeouts, 2 HTTP 404s from retired NVIDIA-hosted models), with only auto's paid fallback keeping that run alive.

The ADR's own 2026-08-31 correction is explicit that this was never actually reviewed or accepted by anyone with authority to do so — an earlier version of the same ADR entry had fabricated a claim that the org owner explicitly directed the switch and quoted a fabricated "owner response"; that attribution was retracted as false, not a record of a real decision. The ADR has carried "this remains an open, unreviewed risk" since that correction, now 6+ days.

Given that history, an agent session unilaterally re-declaring "risk accepted" in the ADR would repeat exactly the failure mode that made the correction necessary in the first place. This needs an actual decision from someone with the authority to accept or reject the trade-off — hence an issue, not another autonomous ADR edit.

The concrete trade-off

• Today: Strix runs only against orchestrator/free. If the free catalog's currently-discovered routes collapse to a single provider family during an outage (as already reproduced once), Strix goes fully dark — no fallback — rather than degraded-but-running.
• Reverting to orchestrator/auto restores the fallback but reintroduces the possibility of a paid model executing the organization's required security-review gate, which is what motivated the original switch to free.
• A third option (closing the gap without reverting): diversify the free pool's provider coverage or add a genuinely free/ZDR-compliant secondary route, so free itself stops single-point-of-failing rather than falling back to auto.

Live signal to check before deciding

scripts/ci/contextual_orchestrator_review_policy.py already computes free_account_diversity and free_pool_account_diversity (distinct provider-accounts backing the free pool, and how many survive pool admission) in its routing report. Before this decision is made, whoever picks it up should pull a current reading of those two numbers — if free_pool_account_diversity is still effectively 1 (or has been intermittently 1, matching the 2026-08-30 reproduction), that's live confirmation the risk is still real today, not just historical.

What's being asked for

A decision from the repo owner (or whoever has that authority) on one of the three options above — or an explicit, reviewed "accept this risk as-is for now" that can then be recorded honestly in the ADR (by a human, or by an agent quoting an actual instruction, never fabricated). Until then this issue tracks the decision as open; no code or ADR change is proposed here.

Related: docs/adr/0003-contextual-orchestrator-vendored-free-zdr.md, docs/product-technical-gap-baseline.md's 2026-08-30 sidecar-preflight entries (full reproduction evidence trail).

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX

[claude]

Reproduction 7 days later, larger catalog, same single-family collapse

This issue's evidence is the 2026-08-30 run where the free pool's discovered routes collapsed to one provider family with 4/4 candidates rejected. Here is the same collapse on 2026-09-06, with the catalog six times larger — so catalog growth has not restored diversity.

Source: required noema-review on .github#1884@396b4dee, job 101521704729, 17:33:26Z → 18:14:14Z; artifact noema-sidecar-evidence (id 9994541963), contract strix-plain-chat-preflight-v2.

The gateway itself was healthy — this is a pool-composition failure, not an unreachable-gateway one:

"gateway": {"attempts": 1, "endpoint": "chat/completions", "finish_reason": "stop", "status": "ready"}
"candidate_count": 24, "probed_count": 16, "ready_count": 6,
"deferred_count": 2, "rejected_count": 8, "skipped_count": 4, "target_ready": 8

All 6 ready routes are nvidia_nim / nvidia_nim_sub — three models, each counted twice because the two NVIDIA accounts are separate credentials on the same provider family:

model	providers
deepseek-ai/deepseek-v4-flash-0731	nvidia_nim, nvidia_nim_sub
deepseek-ai/deepseek-v4-pro-0813	nvidia_nim, nvidia_nim_sub
meta/llama-3.2-11b-vision-instruct	nvidia_nim, nvidia_nim_sub

The two independent free sources that could have supplied diversity each failed, in different ways:

• Bytez never entered the pool at all: provider_discovery_failed provider=bytez code=http_status_500 (first line of the sidecar log — discovery, not inference).
• OpenRouter produced its only two candidates and both deferred on HTTPError / http_status: 429: cohere/north-mini-code:free, dots-studio/dots-3-note-preview:free.

The 8 rejects are NVIDIA 404s on retired hosted models (google/gemma-3-12b-it, google/gemma-3-4b-it, both accounts) plus TimeoutErrors (google/gemma-4-31b-it, both accounts).

target_ready is 8 and only 6 were reached, so the pool was already under its own diversity target before inference began.

What happened next is the failure this issue predicts. With no other family to fall back to, every attempt against those six routes hit the socket idle timeout — eight measured provider_attempt → provider_attempt_failed … error_type=TimeoutError intervals of 90.1, 90.1, 90.1, 90.2, 90.1, 90.1, 90.0, 90.1 s — through circuit_failure → circuit_opened(reset_seconds=30) → circuit_cleared → retry, then attempts=3 exhausted on the last route and:

request_failed status=502 code=provider_connection_error

The required gate went fully dark, exactly the "no fallback" outcome in the trade-off section, and it did so without any paid model being needed to diagnose it — the free pool had a healthy gateway and simply had nowhere else to route.

Two points for the acceptance tracker, offered as measurement rather than as a decision:

1. The free pool currently has a hard dependency on NVIDIA NIM. Diversity is not merely thin; it is zero — the two NVIDIA credentials are one family. This sits directly against the organization direction that direct NIM communication is a removal target: today, removing NIM from the free pool would leave it empty.
2. Provider-family diversity should be measurable from the preflight artifact. Every run already emits routes[].provider, so a distinct-family count over status == "ready" is derivable with no new probing, and target_ready already exists as a companion threshold. That would let "free-pool availability" be an observed number in this tracker instead of a per-incident reconstruction.

I am not proposing a code change here, and specifically not reintroducing paid or direct-provider fallback, per this issue's ownership correction.

---

Generated by Claude Code

[claude]

Refining my own suggestion above: the metric exists, but on the wrong axis and the wrong scope

In my previous comment I suggested that provider-family diversity "should be measurable from the preflight artifact," implying nothing computes it. That was imprecise, and acting on it as written would mean rebuilding something that partly exists. Three corrections, from the source rather than the artifact:

1. A diversity metric already exists — in a different report. scripts/ci/contextual_orchestrator_review_launcher.py::_with_discovery_counts emits free_account_diversity and free_pool_account_diversity, gated on FREE_POOL_CREDENTIAL_NAMES (BYTEZ_API_KEY, NVIDIA_NIM_API_KEY, NVIDIA_NIM_API_KEY_SUB, OPENROUTER_API_KEY).

2. It is scoped to the discovery catalog, not to what inference can actually use. Its input is free_rows = [row for row in rows if row.get("cost_evidence") == "free"], and its own docstring pins the contract as "among all discovered free routes." The strix-plain-chat-preflight-v2 report — the one uploaded as noema-sidecar-evidence when the gate goes dark — is built separately and carries candidate_count / probed_count / ready_count / target_ready / routes[], with no diversity field of any kind. So on the 2026-09-06 run I posted above, catalog-scoped diversity would have looked healthy (OpenRouter contributed two candidates) while the ready set had collapsed to one family. The metric cannot see the failure at the moment the failure matters, and the operator diagnosing the dark gate cannot see the metric.

3. The axis is accounts, not families — deliberately, and correctly for its own purpose.

def provider_account(provider_name: str) -> str:
    """Return the independently credentialed provider account identity."""
    return provider_name

The identity function is right for rate-limit isolation: NVIDIA_NIM_API_KEY and NVIDIA_NIM_API_KEY_SUB were confirmed on 2026-09-03 to have independent rate limits rather than a shared pool, and test_strix_serializes_provider_evidence_per_repository_and_pr records PR-scoped Strix concurrency being restored on exactly that finding. But it makes account diversity a poor proxy for the SPOF this issue tracks. On the run above, an account-diversity count over the ready set reports 2 (nvidia_nim, nvidia_nim_sub) while outage diversity is 1:

model	nvidia_nim	nvidia_nim_sub
google/gemma-4-31b-it	TimeoutError 90.1 s	TimeoutError 90.1 s
deepseek-ai/deepseek-v4-flash-0731	TimeoutError 90.1 s	TimeoutError 90.2 s, then 90.1 s
deepseek-ai/deepseek-v4-pro-0813	TimeoutError 90.1 s	— (circuit already open)

Two independently credentialed accounts, same upstream, same 90 s failure, minutes apart. Independent rate limits do not buy independent availability, so the two axes need to stay distinct rather than one standing in for the other.

So the concrete ask is narrower than what I wrote before: not "add a diversity metric," but give the existing one a family-level companion and evaluate it over status == "ready", then emit it in strix-plain-chat-preflight-v2 alongside ready_count and target_ready. Every input is already in the report — routes[].provider and routes[].status — so this needs no additional probing and no change to what is discovered.

I am not implementing that here: the axis semantics (what counts as one family, and what a family-diversity floor should be) is an acceptance decision on this issue rather than a mechanical edit, and per this issue's ownership correction I am not the one to declare it. Flagging it as a precise spec for whoever takes it.

---

Generated by Claude Code

[claude]

A near-simultaneous success/failure pair — and why the pair cannot be compared

Two noema-review runs on this repository overlapped almost entirely this morning and ended differently:

PR	head	window	result
#1884	767e740c (docs-only)	07:37:15Z → 08:02:07Z (25 min)	success, APPROVE published on the exact head
#1967	8ca96943 (544-line test + 94-line workflow)	07:35:44Z → 08:08:48Z (33 min)	failure, request_failed status=502 code=provider_connection_error

The failing run reproduces this issue's shape for the fourth time (artifact noema-sidecar-evidence 10009710244):

"gateway": {"attempts": 1, "finish_reason": "stop", "status": "ready"}
"candidate_count": 24, "ready_count": 6, "deferred_count": 2,
"rejected_count": 8, "skipped_count": 4, "target_ready": 8

All six ready routes are again nvidia_nim / nvidia_nim_sub — the same three models across the two accounts. 39 provider attempts, 22 recorded failures, 16 TimeoutError against 6 HTTPError, ending 502.

The useful part is what the pair cannot tell us. The obvious reading is that the single remaining family is marginal rather than dead, and serves a small request while timing out on a larger one. I can't support that from the evidence: noema-review.yml uploads noema-sidecar-evidence only if: failure(), so the succeeding run produced no preflight at all. Its ready_count, its provider mix, and how many attempts it needed are unobservable. Whether the two runs even saw the same pool composition is not something the artifacts can answer.

So a controlled comparison is impossible by construction, and that is a second instrumentation gap alongside the missing family-diversity field I described in #1915 (comment). Both point the same way: the acceptance signal this tracker needs — "is the free pool healthy right now?" — is only ever emitted when the pool has already failed. A preflight report retained on success as well as failure would make success-vs-failure pairs like this one comparable, and would cost one artifact upload on a path that already builds the report either way.

Recording this as measurement. I am not proposing a change to what gets probed, and not reintroducing paid or direct-provider fallback.

---

Generated by Claude Code