Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d16d54e
fix(discovery): stop treating OpenRouter as a whole-account ZDR exclu…
claude Aug 31, 2026
994d03a
Merge remote-tracking branch 'origin/main' into fix/openrouter-not-ev…
claude Aug 31, 2026
2691d84
fix(batch): enforce OpenRouter ZDR pin
seonghobae Aug 31, 2026
4466e3c
Revert "fix(batch): enforce OpenRouter ZDR pin"
seonghobae Aug 31, 2026
82d4a18
fix(batch): pin OpenRouter ZDR in JSONL
seonghobae Aug 31, 2026
9073a35
docs(zdr): correct stale batch-out-of-scope claim after ZDR pin landed
Aug 31, 2026
eae7330
fix(zdr): pin OpenRouter speech requests
seonghobae Aug 31, 2026
98f0109
fix(api): validate capability provider routing
seonghobae Aug 31, 2026
bc0076a
fix(zdr): validate provider shape in the shared OpenRouter ZDR pin
claude Aug 31, 2026
4bc6059
fix(embeddings): pin OpenRouter ZDR routing
seonghobae Aug 31, 2026
37f2e08
fix(embeddings): retain atomic target routing metadata
seonghobae Aug 31, 2026
944485c
fix(privacy): infer OpenRouter for legacy embeddings
seonghobae Aug 31, 2026
df97709
fix(privacy): derive OpenRouter identity from base_url in the ZDR pin
claude Aug 31, 2026
2f42488
fix(privacy): trust exact OpenRouter destination for ZDR
seonghobae Aug 31, 2026
bbbc051
fix(privacy): close the mistyped provider_name gap in cost_router.py too
claude Aug 31, 2026
f771ad4
fix(privacy): stack OpenRouter ZDR enforcement
seonghobae Sep 1, 2026
74e880b
Merge remote-tracking branch 'origin/fix/model-group-timeout-openrout…
seonghobae Sep 1, 2026
39dff20
Merge remote-tracking branch 'origin/fix/model-group-timeout-openrout…
seonghobae Sep 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.d/openrouter-not-evidence-only.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Stopped treating OpenRouter as a whole-account, ZDR-motivated serving exclusion. `PROVIDER_MODEL_SOURCES`'s `openrouter` entry no longer sets `evidence_only=True`: ZDR eligibility is a route/model-level property, never grounds to block an entire provider account from serving, and OpenRouter was the one provider source with genuinely reliable native pricing/`is_free` evidence, so excluding it directly caused `orchestrator/free`'s previously-documented structural emptiness (ADR 0041). Also fixed a backwards side effect of the old flag: `_apply_discovered_model_evidence` could never mark OpenRouter's own rows `zdr_capable=True` even when they exactly matched OpenRouter's own declared ZDR feed. The provider-neutral evidence-application contract from PR #901 (OpenRouter's feed also crediting matching rows from every other provider) is unchanged. Since OpenRouter can multiplex one model id across several backing providers, `ModelClient` now pins every OpenRouter request made under an active `zdr_only` scope with OpenRouter's own documented `"provider": {"zdr": true}` request-time enforcement, applied at the shared `_send`/`_stream_send`/`_send_raw` transport chokepoints, and at the async Batch API path (`_batch_run`) via the same helper on each uploaded JSONL request body.
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,19 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
retried as if it were a network blip. Fixes the shared classifier itself
(not just the discovery retry call site), so every current and future
caller of `is_transient_error` benefits.
- (Devin review on #953) `_pin_openrouter_zdr` no longer raises a bare
`TypeError` from `dict()` when a caller-supplied `provider` field is
present but not an object (an int, bool, list, or string) under an active
`zdr_only` scope. It now validates the field and raises a named `ValueError`
("provider must be an object with optional OpenRouter routing keys")
instead, matching this codebase's existing convention for malformed
caller-input fields. Every call site sharing this one choke point (chat,
streaming, tools/binary-media passthrough, and the batch JSONL path)
benefits; a valid `provider` object or an absent/`None` one keep their
existing behavior unchanged.
- OpenRouter embedding Batch JSONL now carries the same request-scoped
`provider.zdr=true` enforcement when `zdr_only` selects an attested
OpenRouter embedding agent.

### Added

Expand Down
7 changes: 5 additions & 2 deletions contextual_orchestrator/batch_routing.py
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,7 @@ class EmbeddingBatchRequest:
token_count: int = 0
zdr_only: bool = False
agent_id: Optional[str] = None
provider_routing: Optional[Dict[str, Any]] = None
Comment thread
seonghobae marked this conversation as resolved.

def wire_custom_id(self) -> str:
"""Return a provider-safe id while retaining the internal request mapping.
Expand All @@ -476,14 +477,16 @@ def wire_custom_id(self) -> str:

def to_jsonl_line(self, endpoint: str = "/v1/embeddings") -> Dict[str, Any]:
"""Render this request as an OpenAI Batch API embeddings JSONL line."""
body: Dict[str, Any] = {"model": self.model, "input": self.input_text}
if self.provider_routing is not None:
body["provider"] = dict(self.provider_routing)
return {
# The provider body stays OpenAI-compatible; the backend's tracked
# request map carries the immutable route identity separately.
"custom_id": self.wire_custom_id(),
"method": "POST",
"url": endpoint,
# ``zdr_only`` is enforced before this provider JSONL is built.
"body": {"model": self.model, "input": self.input_text},
"body": body,
}


Expand Down
44 changes: 39 additions & 5 deletions contextual_orchestrator/cost_router.py
Original file line number Diff line number Diff line change
Expand Up @@ -780,14 +780,20 @@ def submit_embeddings_batch(
raise TypeError("zdr_only must be a boolean")
if agent_id is not None and (not isinstance(agent_id, str) or not agent_id):
raise TypeError("agent_id must be a non-empty string when provided")
resolved_model, resolved_agent_id = self._resolve_embedding_target(model, zdr_only, agent_id)
resolved_model, resolved_agent_id, resolved_provider = self._resolve_embedding_target(
model, zdr_only, agent_id
)
provider_routing = (
{"zdr": True} if zdr_only and resolved_provider == "openrouter" else None
)
shared_attribution = dict(attribution or {})
requests, part_counts, part_limits = self._build_embedding_requests(
inputs,
model=resolved_model,
attribution=shared_attribution,
zdr_only=zdr_only,
agent_id=resolved_agent_id,
provider_routing=provider_routing,
)
job = self.embedding_batch_backend.submit(requests, metadata=metadata)
self._embedding_jobs[job.job_id] = job
Expand All @@ -800,10 +806,10 @@ def submit_embeddings_batch(

def _resolve_embedding_target(
self, model: str, zdr_only: bool, agent_id: Optional[str]
) -> tuple[str, Optional[str]]:
) -> tuple[str, Optional[str], Optional[str]]:
"""Resolve one embedding member without losing a caller's member choice."""
if agent_id is None and not zdr_only:
return model, None
return model, None, None
selection_model = (
None
if model in {"contextual-orchestrator", getattr(self.orchestrator, "AUTO_MODEL", "")}
Expand All @@ -812,10 +818,11 @@ def _resolve_embedding_target(
with self.orchestrator.request_policy(zdr_only):
candidates = self.orchestrator._capability_agents("embedding", selection_model)
if agent_id is None:
return candidates[0].model, candidates[0].id
selected = candidates[0]
return selected.model, selected.id, _resolved_provider_name(selected)
for candidate in candidates:
if candidate.id == agent_id:
return candidate.model, candidate.id
return candidate.model, candidate.id, _resolved_provider_name(candidate)
raise RuntimeError(f"embedding agent {agent_id!r} is not eligible for this request")

def _build_embedding_requests(
Expand All @@ -826,6 +833,7 @@ def _build_embedding_requests(
attribution: Dict[str, Any],
zdr_only: bool,
agent_id: Optional[str],
provider_routing: Optional[Dict[str, Any]],
) -> tuple[List[EmbeddingBatchRequest], List[int], Dict[str, int]]:
"""Map original embedding inputs into token-budgeted provider parts."""
max_tokens, max_chars = self._embedding_request_limits()
Expand All @@ -850,6 +858,7 @@ def _build_embedding_requests(
token_count=token_count,
zdr_only=zdr_only,
agent_id=agent_id,
provider_routing=provider_routing,
)
)
return requests, part_counts, {
Expand Down Expand Up @@ -1156,6 +1165,31 @@ def _provider_from_base_url(base_url: str) -> str:
return host


def _resolved_provider_name(agent: Any) -> str:
"""Return a canonical provider name for one selected agent snapshot.

``base_url`` is what actually decides an outbound HTTP destination;
``provider_name`` is a free-text label unvalidated at ``ModelAgent``
construction, so it can be empty *or* nonempty-but-wrong (a typo, a
stale copy-paste). Trusting a nonempty ``provider_name`` unconditionally
— the previous ``agent.provider_name or ...`` short-circuit — let an
agent whose ``base_url`` is OpenRouter's own endpoint report a different
provider identity, which made ``submit_embeddings_batch``'s ZDR pin
(``provider_routing = {"zdr": True} if resolved_provider == "openrouter"
...``) silently skip OpenRouter requests under an active ``zdr_only``
scope. The exact destination hostname is checked first and is
authoritative whenever it is OpenRouter's, mirroring
``orchestrator._resolved_openrouter_provider`` so both ZDR-pin choke
points (the embedding-batch path here and the chat/streaming/raw/batch
JSONL path there) share one normalization rule (CodeRabbit review on
#953, discussion_r3898471887 / discussion_r3898659143).
"""
host = _provider_from_base_url(agent.base_url)
if host == "openrouter.ai":
return "openrouter"
return agent.provider_name or host


def _positive_int(value: Any, default: int) -> int:
"""Return ``value`` as a positive int, or ``default`` when invalid."""
try:
Expand Down
5 changes: 5 additions & 0 deletions contextual_orchestrator/model_discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -1334,6 +1334,11 @@ def discover_all_models(
# OpenRouter's authenticated catalog supplies routable account-model rows;
# its public ZDR endpoint adds route-specific privacy evidence without
# turning the whole provider account into either ZDR-only or non-serving.
# Request-time ZDR enforcement for OpenRouter specifically is a runtime
# concern (see ModelClient's `provider: {"zdr": true}` pin), not a
# discovery-time exclusion: OpenRouter can multiplex a model across several
# backing providers, so a stale discovery-time snapshot cannot by itself
# guarantee which provider serves a given request.
routed = _apply_discovered_model_evidence(
_deduplicate_discovered_models(discovered),
_openrouter_zdr_model_ids(timeout=timeout),
Expand Down
68 changes: 66 additions & 2 deletions contextual_orchestrator/orchestrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,66 @@ def _cost_usd_decimal(output_tokens: int, price_per_million: float) -> Decimal:
)
_REQUEST_ZDR_ONLY: ContextVar[bool] = ContextVar("request_zdr_only", default=False)


def _resolved_openrouter_provider(agent: ModelAgent) -> str:
"""Canonical provider identity for the ZDR-pin decision, base_url-first.

``ModelAgent.provider_name`` is free-text and unvalidated at construction
(hand-authored JSON, ``model_discovery.py`` auto-discovery, or KV-driven
config can all leave it empty or typo'd). Trusting it verbatim here would
let an agent whose ``base_url`` is OpenRouter's own endpoint silently skip
the ``provider.zdr=true`` enforcement pin under an explicit ``zdr_only``
scope while still routing bytes to OpenRouter (base_url decides where the
request goes; this function only decides whether the pin is applied) —
a silent ZDR-policy bypass, not a crash (CodeRabbit review on #953,
discussion_r3898471887). Treating the exact OpenRouter hostname as
authoritative also covers a nonempty typo in that free-text field. Every
call site that funnels through this shared choke point (chat, streaming,
raw, binary media, and non-embedding batch JSONL) therefore gets the same
protection the embedding batch path already has.
"""
host = urlparse(agent.base_url).hostname or ""
if host == "openrouter.ai":
return "openrouter"
return agent.provider_name or host


def _pin_openrouter_zdr(agent: ModelAgent, payload: dict[str, Any]) -> dict[str, Any]:
"""Force OpenRouter to enforce zero-data-retention at request time.

OpenRouter can multiplex one model id across several backing providers;
a discovery-time ZDR feed snapshot proves a route was ZDR-attested when
it was fetched, not which provider actually serves a later request. Their
documented ``provider: {"zdr": true}`` request field is OpenRouter's own
server-side enforcement (https://openrouter.ai/docs/features/provider-routing)
and is authoritative for the request being sent right now, so it is
applied here rather than trusted to have been decided correctly upstream.
A caller-supplied ``provider`` object (e.g. explicit routing preferences)
is preserved and only gains the ``zdr`` key.

``provider`` is an optional caller passthrough field reaching this shared
choke point unvalidated from every call site (chat, streaming, tools and
binary-media passthrough, and the batch JSONL path). A malformed truthy
non-mapping value (an int, bool, list, or string) must fail with a named,
caller-actionable validation error here rather than an opaque ``TypeError``
from ``dict()`` deep inside provider-transport code (Devin review on #953).

The "is this agent OpenRouter" check itself goes through
``_resolved_openrouter_provider`` rather than a bare ``agent.provider_name``
comparison, so a misconfigured agent (empty/wrong ``provider_name`` but a
``base_url`` that is actually OpenRouter's) still gets pinned instead of
silently bypassing ZDR enforcement (CodeRabbit review on #953).
"""
if not _REQUEST_ZDR_ONLY.get() or _resolved_openrouter_provider(agent) != "openrouter":
return payload
provider_routing = payload.get("provider")
if provider_routing is not None and not isinstance(provider_routing, dict):
raise ValueError("provider must be an object with optional OpenRouter routing keys")
provider_routing = dict(provider_routing or {})
provider_routing["zdr"] = True
return {**payload, "provider": provider_routing}
Comment thread
seonghobae marked this conversation as resolved.


SECRET_PATTERNS = (
re.compile(r"(?i)(api[_-]?key|token|secret|password)(['\"]?\s*[:=]\s*['\"]?)[A-Za-z0-9._~+/=-]{12,}"),
re.compile(r"(?i)(bearer\s+)[A-Za-z0-9._~+/=-]{12,}"),
Expand Down Expand Up @@ -1732,6 +1792,7 @@ def _send(
timeout: float | None = None,
) -> str:
"""Perform one provider HTTP request (isolated so retry/backoff stays testable)."""
payload = _pin_openrouter_zdr(agent, payload)
api_key = _provider_credential(agent)
headers = {"content-type": "application/json"}
if api_key:
Expand Down Expand Up @@ -2002,6 +2063,7 @@ def _stream_send(
):
"""Stream content deltas from a provider SSE response (real transport, testable)."""
self._local.usage = None
payload = _pin_openrouter_zdr(agent, payload)
api_key = _provider_credential(agent)
headers = {"content-type": "application/json", "accept": "text/event-stream"}
if api_key:
Expand Down Expand Up @@ -2178,6 +2240,7 @@ def proxy_send_bytes(
self, agent: ModelAgent, endpoint: str, payload: dict[str, Any]
) -> tuple[bytes, str]:
"""Passthrough a provider response whose body is binary media."""
payload = _pin_openrouter_zdr(agent, payload)
Comment thread
seonghobae marked this conversation as resolved.
if agent.base_url.startswith("mock://"):
return b"mock audio", "audio/mpeg"
api_key = _provider_credential(agent) # pragma: no cover
Expand Down Expand Up @@ -2330,6 +2393,7 @@ def _send_raw(
destination: ProviderDestination | None = None,
) -> dict[str, Any]: # pragma: no cover
"""One provider HTTP request returning the FULL provider JSON (for passthrough)."""
payload = _pin_openrouter_zdr(agent, payload)
api_key = _provider_credential(agent)
headers = {"content-type": "application/json"}
if api_key:
Expand Down Expand Up @@ -2566,12 +2630,12 @@ def _batch_run(
"custom_id": custom_id,
"method": "POST",
"url": "/v1/chat/completions",
"body": self.apply_effort_profile(agent, {
"body": _pin_openrouter_zdr(agent, self.apply_effort_profile(agent, {
"model": agent.model,
"messages": messages,
"temperature": settings["temperature"] if temperature is None else temperature,
"max_tokens": settings["max_output_tokens"],
}, effort_profile),
}, effort_profile)),
}, ensure_ascii=False)
for custom_id, messages in requests.items()
]
Expand Down
2 changes: 2 additions & 0 deletions contextual_orchestrator/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -2293,6 +2293,8 @@ def _validate_mode(mode: Any) -> str:

def _validate_capability_request(path: str, body: dict[str, Any]) -> None:
"""Validate the required trust-boundary fields for media/rerank passthrough."""
if "provider" in body and not isinstance(body["provider"], dict):
raise RequestError(400, "invalid_provider", "provider must be an object")
if "model" in body:
model = body["model"]
if not isinstance(model, str):
Expand Down
6 changes: 6 additions & 0 deletions docs/planning/adrs/0032-model-group-cost-aware-discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ does not exclude non-ZDR routes from ordinary requests. Missing or failed ZDR
evidence therefore fails closed only for `zdr_only` selection, not for general
inference.

Every OpenRouter wire transport applies the provider's documented
`provider.zdr=true` request-time enforcement inside that policy scope. This
includes JSON chat, streaming, structured passthrough, chat and embedding
batch JSONL, and the binary-response speech transport; the response media
type does not weaken the privacy contract of its JSON request body.

OpenRouter discovery retains the concrete free-model list returned by its model
catalog, including exact `vendor/model:free` identifiers and any row whose
complete structured monetary price is zero. The aggregate `openrouter/free`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,29 @@ nothing about the retry can turn a paid model free. Motivated by the
`orchestrator/free` review-sidecar reliability gap in
`ContextualWisdomLab/.github` PR #1433.

## Amendment (2026-08-31): OpenRouter is no longer `evidence_only`

This ADR's Context section characterized OpenRouter's `evidence_only=True`
(commit `952996ec`) as settled, deliberate ZDR-privacy hardening "that stays
untouched." That characterization was false; it was reversed on direct
review this pass: ZDR eligibility is a route/model-level property
(`is_zdr_model`, exact feed matching), never grounds to block an entire
provider account from serving. OpenRouter's `ProviderModelSource` entry no
longer sets `evidence_only=True`.

This directly closes the gap this ADR's Context section itself identified
("only OpenRouter's own API ever reports real pricing... and OpenRouter...
never serves inference... `orchestrator/free` was therefore structurally
empty in practice"): OpenRouter can now serve like every other discovered
provider, independent of and in addition to this ADR's Models.dev join for
the other five sources. The provider-neutral ZDR-evidence-application
contract (OpenRouter's feed also crediting matching rows from *other*
providers, insisted on during PR #901's review) is unchanged; what changed
is that OpenRouter's own rows are no longer the one arbitrary exception to
it. See `docs/product-technical-gap-baseline.md`'s 2026-08-31 entry for the
full mechanism, the request-time ZDR-pinning enforcement this required, and
its stated scope limits.

## References

Models.dev. (2026). *Models.dev API*. https://models.dev/api.json
Expand Down
Loading