Description
When I configure preferred_models via PUT /v1/routing with model IDs that don't have a configured provider key, the router silently falls back to the full eligible set instead of surfacing the misconfiguration. This makes it really hard to debug why my preferred tier isn't being honored.
Steps to Reproduce
- Start Lite with only
OPENAI_API_KEY set (no Anthropic key)
PUT /v1/routing with {"strategy": "quality", "preferred_models": ["claude-opus-4-7", "gpt-5.5"]}
- Send
POST /v1/chat/completions with model="auto"
Expected vs Actual
Expected: Either a 422 saying "preferred model claude-opus-4-7 has no deployable provider" or at minimum an x-orca-warning header indicating the preferred set was empty after filtering.
Actual: The request succeeds, routes to the cheapest available OpenAI model, and the response looks identical to a cheapest strategy call. The dashboard shows the workspace strategy as "quality" which is misleading.
Environment
- OrcaRouter Lite
main (commit ac46b84)
- Docker on Ubuntu 24.04
- Python 3.12.7
The narrowing logic in app/auto_routing.py:choose_auto_model (around line 230) does:
if preferred_models:
preferred_set = set(preferred_models)
narrowed = [m for m in eligible if m.id in preferred_set]
if narrowed:
eligible = narrowed
The if narrowed guard means when the intersection is empty, the full eligible set passes through silently. I think this should at least log a warning, or better yet, return metadata that the caller can surface.
Description
When I configure
preferred_modelsviaPUT /v1/routingwith model IDs that don't have a configured provider key, the router silently falls back to the full eligible set instead of surfacing the misconfiguration. This makes it really hard to debug why my preferred tier isn't being honored.Steps to Reproduce
OPENAI_API_KEYset (no Anthropic key)PUT /v1/routingwith{"strategy": "quality", "preferred_models": ["claude-opus-4-7", "gpt-5.5"]}POST /v1/chat/completionswithmodel="auto"Expected vs Actual
Expected: Either a 422 saying "preferred model claude-opus-4-7 has no deployable provider" or at minimum an
x-orca-warningheader indicating the preferred set was empty after filtering.Actual: The request succeeds, routes to the cheapest available OpenAI model, and the response looks identical to a
cheapeststrategy call. The dashboard shows the workspace strategy as "quality" which is misleading.Environment
main(commitac46b84)The narrowing logic in
app/auto_routing.py:choose_auto_model(around line 230) does:The
if narrowedguard means when the intersection is empty, the full eligible set passes through silently. I think this should at least log a warning, or better yet, return metadata that the caller can surface.