Skip to content

Select the session model and fail closed on a mismatch - #907

Open
orveth wants to merge 1 commit into
mainfrom
feat/785-model-select-readback
Open

Select the session model and fail closed on a mismatch#907
orveth wants to merge 1 commit into
mainfrom
feat/785-model-select-readback

Conversation

@orveth

@orveth orveth commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Implements the write route for #785 phase 2. Read the limitations section — a composed model request refuses today, by design.

The problem

A job may name a model. Nothing could select one, so a seat served whatever the harness defaulted to. Verified by enumerating every send_request call site in driver/acp_driver.rs rather than grepping for a name: the driver's entire outbound surface was four JSON-RPC methods — initialize, session/new, session/prompt, session/cancel. No config-option write existed.

The core design: the setter's success is not the check

This is the whole point of the change, and it comes from reading both adapters' sources.

codex-acp accepts an unrecognised model verbatim. src/thread.rs:3080-3091:

        let preset = presets.iter().find(|p| p.id.as_str() == &*model_id);

        let model_to_use = preset
            .map(|p| p.model.clone())
            .unwrap_or_else(|| model_id.to_string());

        if model_to_use.is_empty() {
            return Err(Error::invalid_params().data("No model selected"));
        }

The only rejection is the empty string; :3106 says the intent outright — "If the user selected a raw model string (not a known preset), don't invent a default."

claude-agent-acp does the opposite. It throws on unknown, but first fuzzy-resolves aliases (main :5010-5031) and then substitutes the canonical value, :5037: "Use the canonical option value so downstream code always receives the model ID rather than the caller-supplied alias."

Both return OK having bound something other than what was named. A caller trusting the return runs a job on a model the buyer did not ask for.

Two guards, and neither subsumes the other

A reader seeing two guards will assume one is redundant. They are not — they see different failures, and each is the only guard with access to its own.

1. Membership, pre-write, in the driver (DriverError::ModelNotOffered). Refuses a model the selector does not offer, before writing anything. This is the only guard that can see codex's echo: there the read-back returns the request verbatim, so request == bound and any comparison passes. The offered set is options[].value plus currentValue — the current value is settable even when the picker omits it (a session resumed onto an allowlist-excluded model reports exactly that), so options alone is not the settable set.

2. Exact comparison, post-write, in the engine (verify_bound_model). The only guard that can see claude's canonicalisation. It is a pure function and lives outside the driver deliberately: one place, so a driver cannot bypass it, and no harness needed to test the policy.

an_echoed_unknown_model_is_not_caught_here_and_that_is_deliberate pins the boundary in a test rather than a comment, so the division of labour cannot be mistaken for redundancy.

No aliasing forgiveness on our side. seller_agents.rs states the rule this inherits: a named request is exact or nothing, with no nearest-match fallback. A harness that canonicalises opus to claude-opus-4-6 has bound a different string than the one signed, and the buyer filtered and paid on the string.

Addressing the option

The setter takes configId, and ACP leaves id agent-chosen. The id is therefore discovered from the same model-category entry the read path keys on, sharing one lookup (first_model_config_option). A hardcoded "model" would repeat exactly the defect the read path was corrected for in #898 — and sharing the lookup is what guarantees we write to the selector we then read back.

The wire method

session/set_config_option, confirmed from source, not from a doc summary. agent-client-protocol v0.14.0 (the pinned dependency of codex-acp), sha256 5efba6592048ef8a9ac97de8d79b2d9933d8ac4d94f7a2de102348fed0c61103, src/schema/client_to_agent/requests.rs:

impl_jsonrpc_request!(NewSessionRequest, NewSessionResponse, "session/new");
impl_jsonrpc_request!(PromptRequest, PromptResponse, "session/prompt");
...
impl_jsonrpc_request!(
    SetSessionConfigOptionRequest,
    SetSessionConfigOptionResponse,
    "session/set_config_option"
);

The positive control is the adjacency: the same macro table binds the two methods this driver already speaks successfully. Params {sessionId, configId, value}, confirmed from claude-agent-acp's handler reading them off the JSON-RPC params. The result is the full updated configOptions, so the setter's own response is the read-back channel.

Where the refusal lands

Between start_session and prompt. The session exists, no prompt has been sent, so no compute is spent and no delivery exists. This is the seam #788 names for the same reason.

Limitations — please read rather than discover

A composed request refuses today. gpt-5.6-sol[low] is model × reasoning-effort in one string. #785 states explicitly that whether the settable surface is two orthogonal axes or 25 enumerated composites is a hypothesis, not to be assumed, so this change does not parse model[effort] — parsing would bake in an unverified format. Consequence: a composed request fails the exact comparison and is refused. Refusing is the safe direction, and it means composed model requests do not work yet.

Effort is never written, which is why a model write cannot clobber it. Both adapters rebuild configOptions on a model change and can silently clamp effort (claude main :5716-5745; codex :3094-3110), so a future change that writes effort must write model first.

Nothing supplies a requested model yet. RunParams.requested_model is None at all three call sites, so today's behaviour is byte-for-byte unchanged. The seam is marked in seller_exec.rs with the constraint in code: the value must come from the stored signed offer, never award params, or a seat would run on a value the buyer never signed. That plumbing waits on #900.

Not verified against a running harness. No live handshake was made. Unproven: that the call is reachable on our transport, that currentModelId follows the setter, whether the settable surface is axes or composites, and what codex does downstream with an accepted-but-unknown model. That last one is money-relevant and is the probe I would run first.

Red-proof

Both mutations applied one at a time, restoring a cmp-verified pristine copy between each.

  1. Lenient comparison (trim + case-fold + substring) → 2 failed, including a_fuzzily_resolved_alias_is_refused: opusclaude-opus-4-6 gets accepted. That is the substitution this change exists to refuse.
  2. Drop currentValue from the offered set2 failed, left ["m-ok"] vs right ["m-ok", "m-real"]: the out-of-picker value is lost and a legitimate resumed session would be refused.

Suites

Tree /srv/forge/workspaces/mobee-785, commands copied from .github/workflows/ci.yml, cargo's own exit code read unpiped.

cargo test -p maxplayer-core --features acp --locked
  -> ok. 399 passed; 0 failed  (+1 in another target; 400 total)
  -> exit 0

cargo test -p maxplayer-core --release \
  --features acp,gateway,git-delivery,wallet --locked -j 6
  -> ok. 1235 passed; 0 failed; 2 ignored  (lib target)
  -> 1245 passed; 0 failed across all 8 targets; zero failures sections
  -> exit 0

10 tests added, and all 10 are confirmed present and ... ok in the money-path row
itself
, matched by name against a list extracted from the diff rather than hand-typed.
#894's known flake passed.

One thing a reviewer re-running this suite should know

seller_node::run::tests::a_losing_open_pool_claimant_releases_its_slot_when_it_sees_the_award
is load-fragile, and it is not this change. Measured, tree held constant:

my tree fe19768 @ loadavg 29.50 -> FAIL · FAIL · ok
my tree fe19768 @ loadavg  5.35 -> ok · ok · ok
pristine main a29f8a8 @ loadavg 5.10 -> ok · ok · ok

At matched load, main and this branch are identical. The first two rows alone would have
been confounded — load varied with the tree — so the third row is the one that
separates "this diff perturbs timing" from "the test is load-fragile". Filed separately;
it is a property of the suite, not of this PR.

A job may name a model. Nothing could select one, so a seat served
whatever the harness defaulted to.

The setter's success is not the check, and this is the whole design.
Read at claude-agent-acp v0.62.0 and codex-acp main: codex accepts an
unrecognised id verbatim and forwards it, rejecting only the empty
string; claude-agent-acp resolves aliases like `opus` onto a canonical
id and substitutes it deliberately. Both return OK having bound
something other than what was named.

So two guards catch two different substitutions, and neither
subsumes the other.

Membership, pre-write, in the driver: refuse a model the selector does
not offer. This is the only guard that can see codex's echo, because
there the read-back returns the request verbatim and any comparison
passes. The offered set is `options[].value` plus `currentValue` — the
current value is settable even when the picker omits it, so `options`
alone is not the settable set.

Exact comparison, post-write, in the engine: read back what the
harness reports as bound and refuse anything but the requested string.
This is the only guard that can see claude's canonicalisation. It
lives outside the driver so a driver cannot bypass it, and it is a
pure function so the policy is testable without a harness.

The write addresses the option by `configId`, and ACP leaves `id`
agent-chosen, so the id is discovered from the same model-category
entry the read path uses. A hardcoded "model" would repeat the defect
the read path exists to avoid, and sharing the lookup is what
guarantees we write to the selector we then read.

Refusal lands before the prompt, so no compute is spent and no
delivery exists.

NOT wired: nothing supplies a requested model yet. The seam is marked
in seller_exec, and the value must come from the stored signed offer,
never award params. That waits on #900's contract.

NOT supported: a composed `model[effort]` id. #785 states that whether
the settable surface is two axes or enumerated composites is a
hypothesis, so parsing the composition would bake in an unverified
format. A composed request fails the exact comparison and refuses.

NOT verified: any of this against a running harness. The wire method
`session/set_config_option` is confirmed from agent-client-protocol
v0.14.0, but no live handshake was made.
@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
mobee Ready Ready Preview Aug 25, 2026 5:57pm

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant