Skip to content

Ask Jev for reasoning effort as a second decision question - #30

Merged
Deep-Sixed merged 1 commit into
mainfrom
claude/zealous-edison-ik9ez1
Oct 5, 2026
Merged

Deep-Sixed merged 1 commit into
mainfrom
claude/zealous-edison-ik9ez1

Conversation

@Deep-Sixed

Copy link
Copy Markdown
Owner

Summary

Follow-up to Deep-Sixed/Cerberus#29. A jev alias can now ask Jev, in the same Decisions API call, how much reasoning effort a request needs, as well as which model should serve it.

cerberus/smart:
  mode: jev
  candidates:
    - { provider: google,     credential: free-1,  model: gemini-flash }
    - { provider: openrouter, credential: primary, model: anthropic/claude-sonnet, reasoning_effort: medium }
  jev:
    decider: { provider: openrouter, credential: primary }
    reasoning_efforts: [low, medium, high]   # ask, and choose only among these

Where the answer applies

  • It replaces the configured reasoning_effort of every planned candidate that sets one, and is never added to a candidate that sets none. Cerberus already configures effort per route, and some providers reject the parameter (OpenAI rejects it for non-reasoning models), so adding it everywhere could turn a working route into a 400.
  • The reasoning_efforts list keeps Jev to values every such route accepts. For example, leave out minimal if one of those routes rejects it.
  • The existing rule for callers is unchanged: a caller's own reasoning_effort wins unless the candidate sets reasoning_effort_override.

Each answer is checked on its own

  • An effort outside the list, or a missing effort answer, leaves the configured effort in place. The model choice still stands.
  • When only one model is left, the call is still made, for the effort question alone. When there's nothing to ask, no call is made and no request text leaves.
  • A failed call (timeout, HTTP error, …) records the same reason for both answers. The request runs in configured order with configured efforts.
  • DispatchPlan may change the effort of a target that already sets one, and nothing else about it. dispatch() refuses any other difference from a configured target.

Validation

jev.reasoning_efforts is refused when it:

  • has fewer than two values
  • repeats a value
  • contains a value outside minimal / low / medium / high
  • has no pool candidate setting reasoning_effort for the answer to replace, since it would never be applied

Wire and telemetry

  • The decision adapter now sends a list of questions and returns one answer per question:
    • route: unchanged
    • effort: {"id": "effort", "type": "choice", "question": …, "options": [...]}
  • The jev record gains effort, effort_confidence and effort_reason. The reason is one of not_requested, not_applicable, no_choice, choice_outside_options, or the reason the call failed.
  • The event's existing reasoning_effort field shows the effort that was applied.
  • /admin/routes and the console show which efforts Jev may choose.

Test plan

  • ./scripts/ci.sh: ruff clean, 431 passed. 12 new tests in tests/test_jev_dispatch.py and 1 new schema test.
  • Mutation checks:
    • applying the effort to every candidate fails 5 tests
    • accepting any effort answer fails 2
    • letting a plan add an effort fails 1
  • node --check src/cerberus/ui/static/app.js
  • python scripts/check-public-history.py: 0 findings.
  • Live probe against the Decisions API. This shares Add jev: Jev decides which model, Cerberus executes it #29's open question about the field names inside a question and an answer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01P4Nm4MVSMmG6JyZi8WumtK


Generated by Claude Code

A jev alias with `jev.reasoning_efforts` set asks a second choice
question in the same Decisions API call: how much reasoning effort the
request needs, answered from that list only.

- The answer replaces the configured reasoning_effort of every planned
  candidate that sets one, and is never added to one that sets none: a
  route without an effort may not accept the parameter at all. The list
  keeps Jev to values every such route accepts. A caller's own stated
  effort still wins unless the candidate sets reasoning_effort_override.
- Each answer is vetted on its own: an effort outside the list leaves
  the configured effort standing whatever the model answer was. With
  one model left the call is still made, for the effort alone; with
  neither question to ask, no call and no request text leaves.
- DispatchPlan may change the effort of a target that sets one and
  nothing else; dispatch refuses any other difference from a configured
  target.
- Validation refuses an efforts list with fewer than two values, a
  repeat, or no pool candidate whose effort it could replace.
- The decision adapter now carries a list of questions and returns one
  answer per question. The jev telemetry record gains effort,
  effort_confidence and effort_reason; /admin/routes and the console
  show which efforts Jev may choose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P4Nm4MVSMmG6JyZi8WumtK
@Deep-Sixed
Deep-Sixed marked this pull request as ready for review October 5, 2026 18:48
@Deep-Sixed
Deep-Sixed merged commit 1ae1146 into main Oct 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants