Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,28 @@ Requirements and caveats:
- Per-seat persona prompts from the earlier bundled worker are not available;
the panel answers the caller's prompt directly.

## Jev Router

A `jev-router` alias uses
[OpenRouter's Jev Router](https://openrouter.ai/typesafe/jev-router)
as one model-selection strategy under Cerberus policy. Cerberus owns the pool:
the alias's candidates, exact registry slugs under one OpenRouter credential,
filtered before every request by cost, provider health, cooldowns and credential
presence. Only what survives is sent; Jev picks one model and its reasoning
effort from it. Cerberus then checks the router's own metadata and withholds any
response that left the pool — including OpenRouter's `models_ignored` fallback,
which routes over its whole pool — while still recording the billed usage.

Requirements and caveats:

- An OpenRouter API key is required; the pool must share one credential.
- Callers may send generation fields and function tools only. Model lists,
`plugins`, `provider`, server tools, `reasoning_effort` and unknown fields are
refused with 400; Jev chooses the reasoning effort.
- Not streamed: the decision is verified before the answer is returned.
- A withheld response has already been billed by OpenRouter.
- The hosted router cannot choose local or non-OpenRouter providers.

## Dispatch

Dispatch resolves a stable Cerberus alias through a validated, identity-scoped
Expand Down
16 changes: 16 additions & 0 deletions config/cerberus.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ providers:
models:
openrouter/free: { cost_tier: free, capabilities: [chat, tools] }
anthropic/claude-sonnet: { cost_tier: paid, capabilities: [chat, tools, coding] }
google/gemini-pro: { cost_tier: paid, capabilities: [chat, tools, coding] }

google:
protocol: openai
Expand Down Expand Up @@ -61,6 +62,21 @@ aliases:
judge: { provider: openrouter, credential: primary, model: openrouter/free }
require_human_review: true

cerberus/auto:
mode: jev-router
# Jev as one model-selection strategy under Cerberus policy. Before every
# request Cerberus filters this pool by cost, health, cooldown and
# credential; OpenRouter's Jev Router then picks one model (and its
# reasoning effort) from what remains. Exact registry slugs only, all under
# one OpenRouter credential. A response from outside the pool is withheld.
candidates: # the pool; no preference order
- { provider: openrouter, credential: primary, model: anthropic/claude-sonnet }
- { provider: openrouter, credential: primary, model: google/gemini-pro }
jev_router:
backend: openrouter
timeout_seconds: 120
allow_paid_pool: true # Jev often picks a paid model; opt in explicitly

identities:
example-reader:
credential_env: CB_KEY_EXAMPLE_READER # holds a cb- client key
Expand Down
49 changes: 49 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ client → cerberus-api → identity + alias authorization + cost policy
→ Fusion backend → OpenRouter /chat/completions
→ openrouter:fusion server tool → panel models
→ analyst → final response
→ filtered pool → Jev Router → OpenRouter /chat/completions
→ typesafe/jev-router chooses one pool model
→ decision verified against the pool → response
```

Cerberus owns routing policy, identity authorization, configured free/paid
Expand All @@ -24,6 +27,52 @@ the alias names. A caller cannot select the panel, the analyst, or the tool
surface of a fusion request, and cannot reach the backend except through a
fusion alias it is allowed to use. Fusion-mode deliberation is not streamed.

## Jev Router

A `jev-router` alias makes Jev one model-selection strategy beside dedicated
paths and Fusion; it does not hand Cerberus's policy to it. Cerberus answers
"what may run?" and Jev answers "which of those should run?" for one request.

The pool is the alias's candidates, each an exact registry slug under one
OpenRouter credential. Before every request `router/pool.py` filters it in the
failover loop's own order — cost (a paid member needs `allow_paid_pool`), then
provider health, the most specific active cooldown, and credential presence —
and only the survivors are sent, as the `jev-router` plugin's `models` list on a
`typesafe/jev-router` call. An empty pool is never sent: the plugin treats an
include list that matches nothing as no list and routes over OpenRouter's whole
pool, so Cerberus answers 503 instead. Patterns (`*`, `~family`) are refused at
validation for the same reason.

Every call asks for router metadata, and the response is accepted only when the
`jev-router` stage is present, reports no `list_fallback`, and names a served
model and resolved models that are all in the pool sent (a slug also matches its
`-YYYYMMDD` revisions, as the plugin documents). Anything else — `models_ignored`
above all — is withheld with 502 and recorded as `out_of_policy`. The call has
already run and been billed, so its usage and reported cost are still recorded.
The metadata is removed from the payload; the router's decision is reported in
the `cerberus` envelope and the `jev_router` telemetry record instead.

A jev-router request accepts an allowlist of caller fields, not a denylist:
messages, sampling and output controls, and function tools. Anything else is
refused with 400 before any upstream call, including model fallback lists,
`route`, `plugins`, `preset`, `provider`, server tools and any field this
release does not know. `reasoning_effort` and `reasoning` are refused because
Jev chooses the effort; a per-candidate reasoning budget is refused at
validation for the same reason. Jev Router requests are not streamed, because
the decision has to be checked before any of the answer reaches the caller.

`/admin/routes` projects the alias as a pool with no preference order: each
member is `eligible` (in the pool the next request would send) or `excluded`
with the first filter that refused it, from the same `resolve_pool` dispatch
calls. Readiness is the backend's presence and a non-empty pool.

The hosted router can only choose among models one OpenRouter credential
reaches; it cannot place a request on a local or non-OpenRouter provider. A
native strategy in which Cerberus asks Jev for a decision over every eligible
backend and executes the winner itself is the intended next step, and would
reuse the same pool filter. There is no per-request cost ceiling: Cerberus does
not hold model prices, and a ceiling checked after a routed call is too late.

## Dispatch control plane

A Cerberus alias is a stable, policy-bearing address. The active SQLite revision
Expand Down
2 changes: 1 addition & 1 deletion docs/persistence.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Persistence review for 0.01

SQLite stores two state classes. Revisioned control state contains service
aliases, bindings, ordered paths, Fusion members, identity policy and credential
aliases, bindings, ordered paths, Fusion and jev-router pool members, identity policy and credential
references plus one atomic active-revision pointer. Operational state contains
provider health, cooldowns, routing events, usage projections and audit records;
with OIDC enabled it also includes pending login and session state. Credential
Expand Down
35 changes: 31 additions & 4 deletions src/cerberus/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
from cerberus.fusion import FusionBackend, OpenRouterFusionBackend, fusion_dispatch, fusion_status
from cerberus.identity.session_store import InMemorySessionStore, SqliteSessionStore
from cerberus.identity.sso import AdminSSO
from cerberus.jev_router import JevRouterBackend, OpenRouterJevRouterBackend, jev_router_dispatch
from cerberus.registry import CerberusConfig, ConfigDocument, load_config_document
from cerberus.registry.schema import Alias, Candidate
from cerberus.router.dispatch import dispatch, shadow_decision_event, unauthorized_event
Expand Down Expand Up @@ -162,6 +163,7 @@ def create_app(
jwks_transport: httpx.AsyncBaseTransport | None = None,
fusion_transport: httpx.AsyncBaseTransport | None = None,
sso_transport: httpx.AsyncBaseTransport | None = None,
jev_router_transport: httpx.AsyncBaseTransport | None = None,
) -> FastAPI:
if config is None:
document = load_config_document()
Expand Down Expand Up @@ -210,6 +212,11 @@ def create_app(
# Managed deliberation backends, keyed by FusionPolicy.backend. Fixed at boot
# like the telemetry sink; an alias whose backend is absent fails closed.
fusion_backends: dict[str, FusionBackend] = {"openrouter": OpenRouterFusionBackend(fusion_transport)}
# Managed model-selection routers, keyed by JevRouterPolicy.backend; fixed at
# boot the same way.
jev_router_backends: dict[str, JevRouterBackend] = {
"openrouter": OpenRouterJevRouterBackend(jev_router_transport)
}
if boot_config.admin_sso is not None:
# sessions + pending logins share the state DB when one is configured, so
# they survive restarts and are consistent across workers; else in-memory
Expand All @@ -229,6 +236,8 @@ async def lifespan(app: FastAPI):
await app.state.http_client.aclose()
for backend in fusion_backends.values():
await backend.aclose()
for router in jev_router_backends.values():
await router.aclose()
if verifier is not None:
await verifier.aclose()
if admin_sso is not None:
Expand Down Expand Up @@ -644,7 +653,8 @@ async def admin_routes(request: Request) -> Response:
with the real one. This answers with the router's own verdicts: the same
cost gate, the same health/cooldown/credential order the failover loop
walks, the same authorization predicate the inference path applies, and
the same three gates fusion dispatch refuses on.
the same three gates fusion dispatch refuses on, and the same filtered
pool a jev-router request would send.

Read-only, and behind the same boundary as /admin/status: the projection
names every provider, model and credential reference in the revision, so
Expand All @@ -661,6 +671,7 @@ async def admin_routes(request: Request) -> Response:
control_plane=control_plane,
store=store,
backend_names=fusion_backends.keys(),
jev_router_backend_names=jev_router_backends.keys(),
),
headers=_ADMIN_UI_HEADERS,
)
Expand Down Expand Up @@ -923,9 +934,11 @@ async def chat_completions(request: Request) -> JSONResponse | StreamingResponse
status_code=403,
content={"error": {"message": f"Identity not authorized for {alias_name!r}", "reason": denial}},
)
if alias.mode != "fusion":
# fusion rejects caller tools outright (fusion_dispatch); elsewhere
# only function tools are forwarded
if alias.mode not in ("fusion", "jev-router"):
# fusion rejects caller tools outright (fusion_dispatch) and
# jev-router refuses anything outside its own allowlist
# (jev_router_dispatch), each with a routing event; elsewhere only
# function tools are forwarded
server_tools = non_function_tools(body)
if server_tools:
return JSONResponse(
Expand Down Expand Up @@ -960,6 +973,20 @@ async def chat_completions(request: Request) -> JSONResponse | StreamingResponse
telemetry=telemetry,
control_plane=control_plane,
)
if alias.mode == "jev-router":
# Cerberus filters the pool; Jev picks within it. A router outage
# fails only jev-router aliases.
return await jev_router_dispatch(
body=body,
alias_name=alias_name,
alias=alias,
identity_name=context.name if context else None,
document=document,
backends=jev_router_backends,
store=store,
telemetry=telemetry,
control_plane=control_plane,
)
return await dispatch(
body=body,
alias_name=alias_name,
Expand Down
59 changes: 59 additions & 0 deletions src/cerberus/control/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,20 @@
runtime_exclusion() health, then cooldown, then credential presence
authorization_error() whether a configured identity may use the alias
fusion_readiness() backend, credential and provider gates for fusion
resolve_pool() the filtered pool a jev-router request would send
jev_router_readiness() backend presence and a non-empty pool

Fusion is projected differently on purpose. Its panel never enters the failover
loop — Cerberus resolves one backend, one credential and one provider and sends
a single deliberation request — so panel members carry no eligibility, standby
or cooldown state. Calling them "eligible" would describe a loop that does not
run. They are a service chain with one readiness verdict over the whole alias.

A jev-router alias is a pool, not a ladder: Jev chooses among its members, so
none is preferred over another and there is no standby. Each member is either in
the pool the next request would send (eligible) or filtered out before Jev sees
it (excluded, with the first filter that refused it).

Read-only: no upstream connection, no cooldown applied, no lifecycle state, and
no secret value or credential locator — a path names its credential, never where
that credential is read from.
Expand All @@ -29,10 +36,13 @@

from cerberus.fusion.dispatch import fusion_readiness
from cerberus.identity.auth import IdentityContext, authorization_error
from cerberus.jev_router.backend import JEV_ROUTER_MODEL
from cerberus.jev_router.dispatch import jev_router_readiness
from cerberus.registry.loader import ConfigDocument
from cerberus.registry.schema import Alias, CerberusConfig, Candidate
from cerberus.router.availability import runtime_exclusion
from cerberus.router.engine import Target, cost_eligible, ordered_targets
from cerberus.router.pool import resolve_pool

# a path is eligible when the router would attempt it first, standby when it is
# attemptable but a later preference, excluded when the router would skip it
Expand Down Expand Up @@ -155,12 +165,51 @@ def _fusion(
}


def _jev_router(
config: CerberusConfig,
alias_name: str,
alias: Alias,
*,
backend_names: Collection[str],
control_plane: Any | None,
store: Any,
now: float | None,
) -> dict[str, Any]:
assert alias.jev_router is not None
policy = alias.jev_router
members = resolve_pool(
config, alias_name, allow_paid=policy.allow_paid_pool, control_plane=control_plane, store=store, now=now
)
pool = [
_path(
ordinal,
member.target,
STATE_EXCLUDED if member.exclusion is not None else STATE_ELIGIBLE,
(
{"reason": member.exclusion.reason, "scope": member.exclusion.scope, "retry_at": member.exclusion.retry_at}
if member.exclusion is not None
else None
),
)
for ordinal, member in enumerate(members)
]
return {
"backend": policy.backend,
"router_model": JEV_ROUTER_MODEL,
"allow_paid_pool": policy.allow_paid_pool,
"timeout_seconds": policy.timeout_seconds,
"pool": pool,
"readiness": jev_router_readiness(members, policy, backend_names=backend_names),
}


def route_projection(
document: ConfigDocument,
*,
control_plane: Any | None,
store: Any,
backend_names: Collection[str] = (),
jev_router_backend_names: Collection[str] = (),
now: float | None = None,
) -> dict[str, Any]:
"""The active revision's routes, with every state decided server-side."""
Expand All @@ -177,6 +226,16 @@ def route_projection(
projected["fusion"] = _fusion(
config, alias, backend_names=backend_names, control_plane=control_plane
)
elif alias.mode == "jev-router":
projected["jev_router"] = _jev_router(
config,
alias_name,
alias,
backend_names=jev_router_backend_names,
control_plane=control_plane,
store=store,
now=now,
)
else:
projected["allow_paid_fallback"] = alias.allow_paid_fallback
projected["paths"] = _paths(
Expand Down
16 changes: 12 additions & 4 deletions src/cerberus/control/sqlite.py
Original file line number Diff line number Diff line change
Expand Up @@ -309,18 +309,26 @@ def _materialize(self, document: ConfigDocument) -> None:
)
self._connection.execute(
"INSERT INTO alias_bindings VALUES (?, ?, ?)",
(revision, alias_name, "fusion" if alias.mode == "fusion" else "dedicated"),
(revision, alias_name, alias.mode if alias.mode in ("fusion", "jev-router") else "dedicated"),
)
for ordinal, candidate in enumerate(alias.candidates):
model = config.resolve_model(candidate, context=f"alias {alias_name!r} candidate")
self._connection.execute(
"INSERT INTO route_paths VALUES (?, ?, ?, ?, ?, ?, ?)",
(revision, alias_name, ordinal, candidate.provider, candidate.credential, candidate.model, model.cost_tier),
)
if alias.mode == "fusion":
if alias.mode in ("fusion", "jev-router"):
self._connection.execute(
"INSERT INTO route_members VALUES (?, ?, 'panel', ?, ?, ?, ?)",
(revision, alias_name, ordinal, candidate.provider, candidate.credential, candidate.model),
"INSERT INTO route_members VALUES (?, ?, ?, ?, ?, ?, ?)",
(
revision,
alias_name,
"panel" if alias.mode == "fusion" else "pool",
ordinal,
candidate.provider,
candidate.credential,
candidate.model,
),
)
if alias.fusion is not None:
for role, candidate in (("analyst", alias.fusion.judge), ("outer", alias.fusion.outer_model)):
Expand Down
30 changes: 30 additions & 0 deletions src/cerberus/jev_router/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""jev-router mode: OpenRouter's hosted Jev Router as one strategy under Cerberus policy.

Cerberus decides which models may serve the alias and filters them by cost,
health, cooldown and credential before every request; Jev picks one model from
what survives. Cerberus verifies the router's decision against the pool it sent
and withholds any response that left it. Jev chooses; Cerberus governs.
"""

from cerberus.jev_router.backend import (
JEV_ROUTER_MODEL,
JevRouterBackend,
JevRouterError,
JevRouterRequest,
JevRouterResult,
OpenRouterJevRouterBackend,
RouterDecision,
)
from cerberus.jev_router.dispatch import jev_router_dispatch, jev_router_readiness

__all__ = [
"JEV_ROUTER_MODEL",
"JevRouterBackend",
"JevRouterError",
"JevRouterRequest",
"JevRouterResult",
"OpenRouterJevRouterBackend",
"RouterDecision",
"jev_router_dispatch",
"jev_router_readiness",
]
Loading
Loading