Add jev: Jev decides which model, Cerberus executes it

README.md

@@ -95,6 +95,24 @@ Requirements and caveats:
 - A withheld response has already been billed by OpenRouter.
 - The hosted router cannot choose local or non-OpenRouter providers.
 
+## Jev
+
+A `jev` alias makes Jev Cerberus's model-selection intelligence while Cerberus
+keeps execution. The pool may span every provider Cerberus knows, local hosts
+included. Before each request Cerberus filters it, asks Jev through OpenRouter's
+Decisions API which remaining model is the cheapest one strong enough, and runs
+that model first through its own failover loop, the rest in configured order.
+
+- Each pool model needs a registry `strength` (`basic`, `standard`, `strong`,
+  `frontier`); Jev weighs it against cost tier.
+- Jev reads the latest user message by default (`jev.input`), or the whole
+  conversation, or request metadata only. Provider names never leave.
+- `typesafe/jev-1.13` is pinned by default so an upstream model update cannot
+  silently change routing.
+- If no usable decision comes back, the request runs in configured order; a
+  decision outage never takes the alias down.
+- Streams, fails over and cools down like any dispatch alias.
+
 ## Dispatch
 
 Dispatch resolves a stable Cerberus alias through a validated, identity-scoped

config/cerberus.example.yaml

@@ -18,7 +18,8 @@ providers:
       primary: { api_key_env: OPENROUTER_API_KEY }
     models:
       openrouter/free: { cost_tier: free, capabilities: [chat, tools] }
-      anthropic/claude-sonnet: { cost_tier: paid, capabilities: [chat, tools, coding] }
+      # strength is read only by jev aliases: Jev weighs it against cost
+      anthropic/claude-sonnet: { cost_tier: paid, strength: frontier, capabilities: [chat, tools, coding] }
       google/gemini-pro: { cost_tier: paid, capabilities: [chat, tools, coding] }
 
   google:
@@ -27,7 +28,7 @@ providers:
     credentials:
       free-1: { api_key_env: GEMINI_KEY_1 }
     models:
-      gemini-flash: { cost_tier: free, capabilities: [chat, tools] }
+      gemini-flash: { cost_tier: free, strength: standard, capabilities: [chat, tools] }
     quota_cooldown_seconds: 14400   # daily free-tier quota
     transport_cooldown_seconds: 30  # network blips never brick a provider for hours
     quota_scope: model              # credential = one 429 cools every model on that key
@@ -77,6 +78,24 @@ aliases:
       timeout_seconds: 120
       allow_paid_pool: true         # Jev often picks a paid model; opt in explicitly
 
+  cerberus/smart:
+    mode: jev
+    # Jev as Cerberus's model-selection intelligence; Cerberus executes. The
+    # pool may span any providers (local hosts included). Before each request
+    # Cerberus filters it, asks Jev through OpenRouter's Decisions API which
+    # remaining model is the cheapest one strong enough, then runs that model
+    # first through its own failover loop. No usable answer -> configured order.
+    candidates:                     # configured order is the fallback order
+      - { provider: google, credential: free-1, model: gemini-flash }
+      - { provider: openrouter, credential: primary, model: anthropic/claude-sonnet }
+    jev:
+      decider: { provider: openrouter, credential: primary }
+      model: typesafe/jev-1.13      # pinned; bump deliberately after testing
+      input: last_user_message      # or full_conversation, or metadata (no text)
+      max_input_chars: 4000
+      timeout_seconds: 5            # decision deadline; then configured order
+      allow_paid_pool: true
+
 identities:
   example-reader:
     credential_env: CB_KEY_EXAMPLE_READER    # holds a cb- client key

docs/architecture.md

@@ -9,6 +9,8 @@ client → cerberus-api → identity + alias authorization + cost policy
                      → filtered pool → Jev Router → OpenRouter /chat/completions
                                       → typesafe/jev-router chooses one pool model
                                       → decision verified against the pool → response
+                     → filtered pool → Jev decision → OpenRouter /api/alpha/decisions
+                                      → chosen model first → Dispatch → any provider
 ```
 
 Cerberus owns routing policy, identity authorization, configured free/paid
@@ -68,10 +70,51 @@ calls. Readiness is the backend's presence and a non-empty pool.
 
 The hosted router can only choose among models one OpenRouter credential
 reaches; it cannot place a request on a local or non-OpenRouter provider. A
-native strategy in which Cerberus asks Jev for a decision over every eligible
-backend and executes the winner itself is the intended next step, and would
-reuse the same pool filter. There is no per-request cost ceiling: Cerberus does
-not hold model prices, and a ceiling checked after a routed call is too late.
+`jev` alias does. There is no per-request cost ceiling: Cerberus does not hold
+model prices, and a ceiling checked after a routed call is too late.
+
+## Jev
+
+A `jev` alias uses Jev as Cerberus's model-selection intelligence and keeps
+execution in Cerberus. Three layers, three questions: Cerberus answers "what may
+run?", Jev answers "what should run?", and the provider runs it.
+
+The pool is the alias's candidates on any provider — a local llama.cpp host,
+Cloudflare, OpenRouter — each a registry model with a `strength`. Before every
+request `router/pool.py` filters it exactly as it does for jev-router (cost, then
+health, cooldown, credential). The distinct models that survive are offered to
+Jev under opaque ids (`m1`, `m2`, …) and described by registry facts only: model
+id, strength, cost tier, context window, capabilities, description. Provider and
+credential names never leave. One `choice` question goes to OpenRouter's
+Decisions API (`POST /api/alpha/decisions`, model `typesafe/jev-1.13` pinned by
+default) under the decider credential the policy names: which candidate is the
+least costly one still strong enough for this request.
+
+The decision service is a third party even when the winner is local, so what it
+reads is policy (`jev.input`): the latest user message (default), the whole
+conversation, or request metadata with no prompt text at all — always truncated
+to `max_input_chars`, always with the request's shape (turns, size, tools,
+response format, requested output length). With one model or none left there is
+nothing to choose, and no decision is asked, so no text leaves.
+
+Jev's answer can only reorder: the chosen model goes first, the rest of the
+cost-admitted pool follows in configured order, and the ordinary failover loop
+executes that order — health, cooldown and credential gates, 429 cooldowns,
+streaming and telemetry included. An answer that is not exactly an offered id is
+never interpreted. Every way of not getting a usable answer — no decider backend
+or credential, the decider's provider marked down, timeout, transport or HTTP
+error, an unreadable response, an absent or unknown choice — runs the request in
+configured order instead, which every pool member already satisfies; the alias
+stays routable while no decision can be had.
+
+Each routing event carries a `jev` record: options offered, whether the decision
+was `chosen`, `skipped` or a `fallback`, the reason, the choice and its
+probability, decision latency, status and usage. It never carries request text.
+The confirmed Decisions API contract covers the endpoint, the model, the
+`model`/`state`/`questions` request and `answers` keyed by question id; the
+fields inside one question and one answer are not yet confirmed, so the answer
+is read tolerantly and a misread degrades to configured order rather than to an
+unvetted model.
 
 ## Dispatch control plane
 

docs/persistence.md

@@ -1,7 +1,7 @@
 # Persistence review for 0.01
 
 SQLite stores two state classes. Revisioned control state contains service
-aliases, bindings, ordered paths, Fusion and jev-router pool members, identity policy and credential
+aliases, bindings, ordered paths, Fusion, jev-router and jev pool members, identity policy and credential
 references plus one atomic active-revision pointer. Operational state contains
 provider health, cooldowns, routing events, usage projections and audit records;
 with OIDC enabled it also includes pending login and session state. Credential

src/cerberus/app.py

@@ -48,6 +48,7 @@
 from cerberus.fusion import FusionBackend, OpenRouterFusionBackend, fusion_dispatch, fusion_status
 from cerberus.identity.session_store import InMemorySessionStore, SqliteSessionStore
 from cerberus.identity.sso import AdminSSO
+from cerberus.jev import DecisionBackend, OpenRouterJevDecider, jev_dispatch
 from cerberus.jev_router import JevRouterBackend, OpenRouterJevRouterBackend, jev_router_dispatch
 from cerberus.registry import CerberusConfig, ConfigDocument, load_config_document
 from cerberus.registry.schema import Alias, Candidate
@@ -164,6 +165,7 @@ def create_app(
     fusion_transport: httpx.AsyncBaseTransport | None = None,
     sso_transport: httpx.AsyncBaseTransport | None = None,
     jev_router_transport: httpx.AsyncBaseTransport | None = None,
+    jev_transport: httpx.AsyncBaseTransport | None = None,
 ) -> FastAPI:
     if config is None:
         document = load_config_document()
@@ -217,6 +219,9 @@ def create_app(
     jev_router_backends: dict[str, JevRouterBackend] = {
         "openrouter": OpenRouterJevRouterBackend(jev_router_transport)
     }
+    # Decision services a jev alias asks which pool model to run, keyed by
+    # JevPolicy.backend; fixed at boot the same way.
+    jev_deciders: dict[str, DecisionBackend] = {"openrouter": OpenRouterJevDecider(jev_transport)}
     if boot_config.admin_sso is not None:
         # sessions + pending logins share the state DB when one is configured, so
         # they survive restarts and are consistent across workers; else in-memory
@@ -238,6 +243,8 @@ async def lifespan(app: FastAPI):
                 await backend.aclose()
             for router in jev_router_backends.values():
                 await router.aclose()
+            for decider in jev_deciders.values():
+                await decider.aclose()
             if verifier is not None:
                 await verifier.aclose()
             if admin_sso is not None:
@@ -654,7 +661,7 @@ async def admin_routes(request: Request) -> Response:
         cost gate, the same health/cooldown/credential order the failover loop
         walks, the same authorization predicate the inference path applies, and
         the same three gates fusion dispatch refuses on, and the same filtered
-        pool a jev-router request would send.
+        pool a jev-router request would send or a jev decision would be offered.
 
         Read-only, and behind the same boundary as /admin/status: the projection
         names every provider, model and credential reference in the revision, so
@@ -672,6 +679,7 @@ async def admin_routes(request: Request) -> Response:
                 store=store,
                 backend_names=fusion_backends.keys(),
                 jev_router_backend_names=jev_router_backends.keys(),
+                jev_decider_names=jev_deciders.keys(),
             ),
             headers=_ADMIN_UI_HEADERS,
         )
@@ -987,6 +995,21 @@ async def chat_completions(request: Request) -> JSONResponse | StreamingResponse
                 telemetry=telemetry,
                 control_plane=control_plane,
             )
+        if alias.mode == "jev":
+            # Jev chooses the order; Cerberus's own loop executes it, so jev
+            # aliases stream, fail over and cool down like any dispatch alias.
+            return await jev_dispatch(
+                body=body,
+                alias_name=alias_name,
+                alias=alias,
+                identity_name=context.name if context else None,
+                document=document,
+                backends=jev_deciders,
+                store=store,
+                client=request.app.state.http_client,
+                telemetry=telemetry,
+                control_plane=control_plane,
+            )
         return await dispatch(
             body=body,
             alias_name=alias_name,

src/cerberus/control/routes.py

@@ -10,8 +10,10 @@
     runtime_exclusion()    health, then cooldown, then credential presence
     authorization_error()  whether a configured identity may use the alias
     fusion_readiness()     backend, credential and provider gates for fusion
-    resolve_pool()         the filtered pool a jev-router request would send
+    resolve_pool()         the filtered pool a jev-router request would send,
+                           or a jev decision would be offered
     jev_router_readiness() backend presence and a non-empty pool
+    decider_readiness()    whether a jev alias could ask for a decision now
 
 Fusion is projected differently on purpose. Its panel never enters the failover
 loop — Cerberus resolves one backend, one credential and one provider and sends
@@ -24,6 +26,10 @@
 the pool the next request would send (eligible) or filtered out before Jev sees
 it (excluded, with the first filter that refused it).
 
+A jev alias is the same kind of pool, across any providers, plus a decider. The
+alias stays routable while no decision can be had — it then runs in configured
+order — so its readiness and its decider's are reported separately.
+
 Read-only: no upstream connection, no cooldown applied, no lifecycle state, and
 no secret value or credential locator — a path names its credential, never where
 that credential is read from.
@@ -36,6 +42,7 @@
 
 from cerberus.fusion.dispatch import fusion_readiness
 from cerberus.identity.auth import IdentityContext, authorization_error
+from cerberus.jev.dispatch import decider_readiness
 from cerberus.jev_router.backend import JEV_ROUTER_MODEL
 from cerberus.jev_router.dispatch import jev_router_readiness
 from cerberus.registry.loader import ConfigDocument
@@ -203,13 +210,62 @@ def _jev_router(
     }
 
 
+def _jev(
+    config: CerberusConfig,
+    alias_name: str,
+    alias: Alias,
+    *,
+    decider_names: Collection[str],
+    control_plane: Any | None,
+    store: Any,
+    now: float | None,
+) -> dict[str, Any]:
+    assert alias.jev is not None
+    policy = alias.jev
+    members = resolve_pool(
+        config, alias_name, allow_paid=policy.allow_paid_pool, control_plane=control_plane, store=store, now=now
+    )
+    pool = [
+        _path(
+            ordinal,
+            member.target,
+            STATE_EXCLUDED if member.exclusion is not None else STATE_ELIGIBLE,
+            (
+                {"reason": member.exclusion.reason, "scope": member.exclusion.scope, "retry_at": member.exclusion.retry_at}
+                if member.exclusion is not None
+                else None
+            ),
+        )
+        for ordinal, member in enumerate(members)
+    ]
+    attemptable = sum(1 for member in members if member.exclusion is None)
+    return {
+        "decider": {
+            "backend": policy.backend,
+            "provider": policy.decider.provider,
+            "credential_ref": policy.decider.credential,
+            "model": policy.model,
+            "input": policy.input,
+            "max_input_chars": policy.max_input_chars,
+            "timeout_seconds": policy.timeout_seconds,
+            "readiness": decider_readiness(
+                config, alias, backend_names=decider_names, control_plane=control_plane
+            ),
+        },
+        "allow_paid_pool": policy.allow_paid_pool,
+        "pool": pool,
+        "readiness": {"pool_available": attemptable, "pool_size": len(members), "available": attemptable > 0},
+    }
+
+
 def route_projection(
     document: ConfigDocument,
     *,
     control_plane: Any | None,
     store: Any,
     backend_names: Collection[str] = (),
     jev_router_backend_names: Collection[str] = (),
+    jev_decider_names: Collection[str] = (),
     now: float | None = None,
 ) -> dict[str, Any]:
     """The active revision's routes, with every state decided server-side."""
@@ -236,6 +292,16 @@ def route_projection(
                 store=store,
                 now=now,
             )
+        elif alias.mode == "jev":
+            projected["jev"] = _jev(
+                config,
+                alias_name,
+                alias,
+                decider_names=jev_decider_names,
+                control_plane=control_plane,
+                store=store,
+                now=now,
+            )
         else:
             projected["allow_paid_fallback"] = alias.allow_paid_fallback
             projected["paths"] = _paths(

src/cerberus/control/sqlite.py

@@ -309,15 +309,15 @@ def _materialize(self, document: ConfigDocument) -> None:
             )
             self._connection.execute(
                 "INSERT INTO alias_bindings VALUES (?, ?, ?)",
-                (revision, alias_name, alias.mode if alias.mode in ("fusion", "jev-router") else "dedicated"),
+                (revision, alias_name, alias.mode if alias.mode in ("fusion", "jev-router", "jev") else "dedicated"),
             )
             for ordinal, candidate in enumerate(alias.candidates):
                 model = config.resolve_model(candidate, context=f"alias {alias_name!r} candidate")
                 self._connection.execute(
                     "INSERT INTO route_paths VALUES (?, ?, ?, ?, ?, ?, ?)",
                     (revision, alias_name, ordinal, candidate.provider, candidate.credential, candidate.model, model.cost_tier),
                 )
-                if alias.mode in ("fusion", "jev-router"):
+                if alias.mode in ("fusion", "jev-router", "jev"):
                     self._connection.execute(
                         "INSERT INTO route_members VALUES (?, ?, ?, ?, ?, ?, ?)",
                         (

src/cerberus/jev/__init__.py

@@ -0,0 +1,27 @@
+"""jev mode: Jev as Cerberus's model-selection intelligence, Cerberus as executor.
+
+Cerberus filters a pool that may span every provider it knows — local hosts,
+Cloudflare, OpenRouter — and asks Jev, through OpenRouter's Decisions API, one
+question: which surviving model is the cheapest one strong enough for this
+request. Cerberus then runs that model through its own failover loop. Jev
+decides; Cerberus governs and executes.
+"""
+
+from cerberus.jev.backend import (
+    Decision,
+    DecisionBackend,
+    DecisionError,
+    DecisionRequest,
+    OpenRouterJevDecider,
+)
+from cerberus.jev.dispatch import decider_readiness, jev_dispatch
+
+__all__ = [
+    "Decision",
+    "DecisionBackend",
+    "DecisionError",
+    "DecisionRequest",
+    "OpenRouterJevDecider",
+    "decider_readiness",
+    "jev_dispatch",
+]