Repository navigation
[7/8] feat: the remaining upstream model kinds - #255
Open
cunninghamcard-bit wants to merge 26 commits into
Open
cunninghamcard-bit wants to merge 26 commits into
cunninghamcard-bit wants to merge 26 commits into
Conversation
…upstream packages - xAI speech, transcription and video models. - Google and Mistral speech and transcription models. - Groq transcription model. - customProvider, ported from upstream. - New models respect the provider boundary rules. OpenAI Chat replay of non-object tool input is pinned to upstream, and the ported Mistral tests name their pinned upstream source. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
6 of 12 tasks
9 of 12 tasks
…maining-upstream-model-kinds Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
Brings [6/8]'s vendor alignment (user-agents from the pinned versions, SigV4 per upstream, Vertex MaaS and xAI, Azure token honours omitted credentials, the restored registry base URL test) together with what [6/8] merged from [5/8] and [1/8]. No conflicts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
…V4 release @ai-sdk/provider 4.0.22 renamed the experimental EvaluationModelV4 to DecisionModelV4 (src/decision-model/v4/decision-model-v4.ts:6-17) and EvaluationUnsupportedQuestionTypeError to DecisionUnsupportedQuestionTypeError. The decision model follows that release, so the baseline moves to it. - VERSIONS.json pins 4.0.22 with its integrity. - The recorder's package.json and package-lock.json name the same version (regenerated with npm install --package-lock-only --ignore-scripts; the pinned vendor packages keep their own 4.0.21 copies). - aisdk_provider_pin_test fails when the pin drifts from 4.0.22. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
TranscriptionModelV4.doStream is optional (@ai-sdk/provider@4.0.22 src/transcription-model/v4/transcription-model-v4.ts:44-46), and experimental_streamTranscribe rejects a model without it before any I/O (ai@7.0.127 src/transcribe/stream-transcribe.ts:125-138). aimux had a default method that failed with UnsupportedFunctionality, so a missing method could only be found by calling it. - TranscriptionModel::do_stream returns Option<BoxFuture<..>>, None by default, as the alignment doc (§3.1, D22) specifies. - stream_transcribe reports None as UnsupportedFunctionality with the upstream message. - OpenAI, Azure, Google, Vertex, xAI and ElevenLabs return Some; per-model checks (a model ID that cannot stream) stay inside the future, as in the upstream doStream bodies. Groq, Mistral and the other batch-only models return None. - Tests: transcription_model::tests::a_model_without_do_stream_cannot_stream, groq_transcription_test::the_model_has_no_streaming_method. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
A custom provider lost the fallback provider's model discovery, so a registry entry built with custom_provider could no longer list models. Discovery is an aimux extension; it now follows upstream's files/skills forwarding (ai@7.0.127 src/registry/custom-provider.ts:268-283): the configured CustomProviderOptions::discovery, else the fallback's. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
SpeechResult gains usage: Option<JsonObject>, as SpeechModelV4Result's usage?: JSONObject (@ai-sdk/provider@4.0.22 src/speech-model/v4/speech-model-v4-result.ts:23-27). Google speech passes its usageMetadata through unchanged and leaves usage unset when the response has none or null (@ai-sdk/google@4.0.87 src/google-speech-model.ts:356-358, google-speech-api.ts:35). The other vendors report none. Telemetry of speech usage is not part of this change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
Supported question types are a property of the model, as upstream DecisionModelV4.supportedQuestionTypes (@ai-sdk/provider@4.0.22 src/decision-model/v4/decision-model-v4.ts:11), so the recording input no longer stores input.decision_capabilities and replay no longer reads it. The live recording fixture drops the field; the replay mock accepts every question type and size and takes rounding and the probability source from the recorded result. Recording schema stays 3. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
…ader rules Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
aimux's Decision API had its own shape: a question list with ids, Choice
options and Score levels, a capability struct, refusals and skips as
answers, and extension fields on the call and the result. The AI SDK now
has DecisionModelV4 (@ai-sdk/provider@4.0.22 src/decision-model/v4/) and
decide (ai@7.0.128 src/decide/decide.ts). This commit makes aimux's
Decision that contract, in one step, with no transitional code (D31). It
folds N5, N6, N7, C29 and N8 of layer 7.
Model and operation (N5, C29):
- One DecisionModel per decision-model-v4.ts:6-16: specification_version
("v4"), provider, model_id, supported_question_types and
do_decide(&DecisionCallOptions). Call options, questions, answers and the
result follow decision-model-v4-call-options.ts:7-14,
decision-model-v4-question.ts and decision-model-v4-result.ts:7-56.
- decide follows decide.ts:76-185: validate the input
(validate-decision.ts:50-125), reject unsupported question types before
any I/O with the DecisionUnsupportedQuestionTypeError message
(decision-unsupported-question-type-error.ts:18-31), retry (default 2),
validate the answers once after the retries (validate-decision.ts:176-298,
tolerance 1e-6 and rounding decimals 0..15), complete total_tokens only
when both counts are known and default the response timestamp and model.
Provider lookup (N7):
- Provider::evaluation_model and the EvaluationModel trait are removed.
Provider::decision_model returns Option<Result<Arc<dyn DecisionModel>>>,
None by default. The registry and custom_provider have one decision
lookup and report None as NoSuchModel("decisionModel")
(provider-registry.ts:447-461, custom-provider.ts:241-258). The
deprecated evaluationModel / doEvaluate aliases are not ported.
- EvaluationLanguageModel is DecisionLanguageModel, with the default
provider "{language provider}.decision" and the response format name
"decision" (provider-utils 5.0.55 decision-language-model.ts:24, :158).
Google's decision_model is that adapter ("google.evaluation",
google 4.0.87 google-provider.ts:438-441); OpenAI's stays the native
Decisions API, and OpenAIProvider::evaluation builds the Responses
adapter ("openai.evaluation", openai 4.0.83 openai-provider.ts:355-358).
API (N6):
- No capability struct. Limits are checked in do_decide before HTTP.
- Request extensions move to provider_options.<provider>: images, OpenAI
choice_values (typed native values) and safety_identifier, runtime
protocol. Result extensions move to provider_metadata.<provider>:
probability_source, per-answer confidence, OpenAI typed values, Laya
abstentions, latency_ms.
- An OpenAI refusal or a vLLM skipped question fails the decision; a V4
decision answers every question.
- Recordings hold the redacted DecideOptions and the DecideResult, with no
capability snapshot; mock replay returns the recorded result.
- FFI: aimux_decide takes DecideOptions and returns DecideResult;
aimux_decision_capabilities keeps its export and returns the model's V4
properties. Bindings: capabilities() is describe() (Node
describeDecisionModel, Python describe_decision_model).
Fixtures (N8): the decision fixtures and the live Jev recordings were
converted to the V4 shape by a deterministic script. Re-decoding both live
Jev wire responses through the new adapter gives the converted results
(decision_recording_test).
Tests pinned to upstream: decision_test (questions_have_the_v4_shape,
invalid_input_is_rejected_before_the_model_is_called,
unsupported_question_types_fail_before_the_model_is_called,
transient_failures_are_retried_twice_by_default,
answers_are_validated_once_after_the_retries,
the_result_completes_usage_and_response,
invalid_answers_report_the_upstream_messages, ...); custom_provider_test
(decision_model::*, decision_*, registry_reports_missing_decision_models);
decision_language_model_test
(provider_defaults_to_the_language_provider_with_a_decision_suffix,
errors_use_the_upstream_decision_messages,
the_language_model_call_joins_the_decision_recording); provider tests for
OpenAI, Google, Jev and the runtimes; FFI, Node and Python tests.
Docs: docs/api/decision.md describes the V4 API; the alignment doc §3.1,
§3.5, §5 and §9.1 (decide telemetry is a known gap); revision lines on
RFC-0037 and RFC-0038; CHANGELOG Breaking entries.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
…settings, factories) fx3/p6 at 3f389ba: layer 6 after its rounds 3 and 4, and the merge of layer 5, which carries layers 2 to 4. Conflicts resolved: - provider-utils lib.rs: layer 7's decision_language_model module and layer 6's download_blob; the download_guard re-export goes, as in layer 5. - decision_support.rs: layer 7's provider_metadata and response_info, built on layer 3's IndexMap SharedProviderMetadata. - jev.rs: layer 6's jev_failed_response_handler and layer 7's rounding doc. - openai/mod.rs imports: layer 7's DecisionLanguageModel, without the load_optional_setting import layer 5 dropped. - Recorder pins: package.json pins @ai-sdk/provider 4.0.22 (layer 7) and provider-utils 5.0.55 (layer 2). package-lock.json was regenerated from layer 6's lock with npm install --package-lock-only --ignore-scripts: the only change is the top-level provider moving to 4.0.22, with each vendor keeping its own nested copy. VERSIONS.json merged cleanly (18 entries). - CHANGELOG: both sides' Added bullets; layer 6's Fixed section. - Generated TS types: layer 6's, regenerated in a follow-up commit. Follow-up commits fix the code that does not compile against the merged layers. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
The merge of [6/8] left layer 7's decision and transcription code on the pre-merge shapes: - indexmap is a workspace dependency since layer 3; the duplicate key in aimux-core is gone and the other crates take the workspace entry. - WrapProvider forwards decision_model with the provider factory's Option<Result<..>> signature. - CallOptions no longer carries call_id (layer 4): the decision language model's exchanges share the decision's recording context, which supplies the call id. - Provider metadata and options are IndexMap namespaces (layer 3). - The google, groq, mistral and xai transcription models refuse AudioInput::Url (layer 5) with AudioInput::url_error(), as every transcription model does; only transcribe downloads a URL. - Tests: SharedProviderOptions literals, download: None in TranscriptionCallOptions, do_stream's Option (C21), and the replay tool's decision_model probe; the two test modules in transcription_model.rs are one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
google, groq, mistral and xai each get url_audio_is_refused_before_any_request: an AudioInput::Url fails with the url error before any request is sent, as layer 5 pins for the transcription models it already had. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
Layer 4 moved max_retries and timeout out of CallOptions into the operation's OperationSettings, which generate_text and stream_text scope around their model calls. decide now scopes its model call the same way, so a language model under DecisionLanguageModel (a router or MoA reading the child retry count, a TraceLayer) sees the decision's settings. A direct do_decide call has none, as a direct do_generate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
… order V4 decision questions and choice criteria are maps, and the OpenAI request lists them in the caller's order. The test re-serialized the fixture's options through org.json's JSONObject, which does not keep key order, so the request listed score before choice and failed the comparison with the fixture. It now passes the options' raw text. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
Layer 4 rewrote the Rust doc of applyToolCallRepair (it repairs the call in the result's steps) without regenerating index.d.ts, so every napi build rewrote the file. This is the napi build's output for the merged source; only that doc comment changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
fx3/p6 at 8bef2c9, which carries fx3/p4 at 4ce9aa0 and fx3/p3 at 35711cb: - the Node reasoning_replay test sends its tool call with the tool result (layer 3's prompt check); - the Node and Python DeepSeek tests read the provider's raw usage from steps[0].usage.raw, since the summed usage carries none (layer 4); - bindings/node/index.d.ts gets layer 4's regeneration, the same hunk as 6aa327d, so it merges without a conflict. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
…order The same order loss as the Kotlin test before efd10d6: the test re-encoded the fixture's options with JSONSerialization, whose dictionaries keep no key order, and compared the request as an NSDictionary, whose arrays are ordered. V4 questions and choice criteria are maps, and the OpenAI request lists them in the caller's order. The test now passes the raw text of the fixture's options object. Not compiled here (no Swift toolchain); the brace matcher was checked against the fixture with an equivalent script. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
fx3/p6 at a382242, which merges fx3/p5 at 9d01eb5 (recording schema 2; recordings written by master load and fail at replay time, naming aimux 0.5 or earlier, where this version cannot replay them). Conflicts, resolved to layer 7's decision shape (N9: no capability snapshot; DecisionModelV4) with layer 5's schema 2: - aimux-core/src/decision_replay.rs: layer 7's MockDecisionReplayModel (no capabilities). A master decision recording (no provider_id) loads; replay_decision_with_model reports its pre-V4 options as written by aimux 0.5 or earlier, and the mock's no-match error counts such recordings. - aimux-providers/src/replay.rs: layer 7's Arc return and NoSuchModel, with ProviderRecord::registry_id. - jev_systemone_live.jsonl: layer 7's converted content at schema 2; the .md note joins both texts. - docs/api/decision.md: layer 7's recording text at schema 2, plus how aimux 0.5 decision recordings behave. Also: master_recording_test's decision test follows layer 7 (the replay error instead of the capability error); CHANGELOG ("schema stays 2", pre-V4 decision options and results) and the RFC-0023 revision line no longer mention input.decision_capabilities or provider.profile. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
The provider-factory bullet still said decision recordings keep the capabilities under input; since decisions follow DecisionModelV4 they carry none, as the DecisionModelV4 entry says. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
rustdoc -D warnings rejects intra-doc links to private items. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
10 of 12 tasks
cunninghamcard-bit
marked this pull request as ready for review
October 14, 2026 03:06
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
cunninghamcard-bit
pushed a commit
that referenced
this pull request
Oct 14, 2026
aimux_stream_text runs the whole stream future on the caller's thread (ffi_block_on). With the test-only override at opt-level 0, one stream through the C ABI needs about 530 KB of stack on the AI SDK branches (about 300 KB on master), more than the 512 KB stack of a Swift cooperative thread: the async Swift tests died with SIGBUS on #255 and #259 once they carried this workflow. At opt-level 1 the same stream needs about 110 KB; the shipping profile needs about 160 KB. The library is also 40% smaller, and the build is still far from the shipping profile's fat LTO with one codegen unit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0112xDERMUyD5JALkFhnsdWS
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
This was referenced Oct 14, 2026
Contributor
Author
|
补充 #233 N8 在本 PR(
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by li
Summary
Before: several model kinds that the upstream packages offer are missing: xAI speech, transcription and video; Google and Mistral speech and transcription; Groq transcription; and
customProvider. Decisions use aimux's ownEvaluationModelnext to aDecisionModelwith a capability struct, a list of questions with ids, and extra fields beside the call options; decision recordings store a capability snapshot. Realtime transcription streaming is a required trait method whose default returns Unsupported, speech results have no usage, andcustom_providerdoes not forward discovery.After: the model kinds are ported from the upstream packages, with their upstream tests, and respect the provider boundary rules. Decision is one
DecisionModelshaped as@ai-sdk/provider4.0.22'sDecisionModelV4, anddecidefollows the AI SDK'sdecidestep by step: input validation, unsupported question types rejected before any I/O, two retries by default, then one answer validation. aimux's extensions (images, OpenAI typed values, runtime protocol; confidence, abstention, latency) live in provider options and provider metadata, provider limits are checked before any request, and recordings carry no capability snapshot. Realtime transcription streaming is an optional method, speech results carry the provider's usage, andcustom_providerforwards discovery to its fallback.Type of change
Checklist
cargo fmt --all -- --checkpassescargo clippy --workspace --all-targets -- -D warningspassescargo test --workspacepasses (cassette playback — no network/keys)README.md,docs/,rfc/,bindings/README.md)Notes for reviewers
@ai-sdk/provider4.0.22decision-model/v4,transcription-model-v4,speech-model-v4-result;ai7.0.128decide/;ai7.0.127registry/custom-provider.ts,registry/provider-registry.ts.decidetelemetry (onStart,onEnd, the tracing span): aimux has no telemetry for operations other than language-model calls yet. Registered as a known gap (§9.1).ai/<VERSION>user-agent suffix: no aimux operation adds it, so it is a difference for all operations, not one for decisions.evaluationModel/doEvaluatealiases and the registry's?? evaluationModelfallback are not ported, so there is no transitional API.Provider::evaluation_modelandEvaluationModelare removed.Provider::decision_modelreturnsOption<Result<Arc<dyn DecisionModel>, _>>, and the registry andcustom_providerreport a missing one asNoSuchModel { model_type: "decisionModel" }.decide(model, DecideOptions) -> DecideResult. Questions are a map keyed by id; answers are{choice, probabilities?},{score, probabilities?}and{probability}. An OpenAI refusal or a vLLM skipped question fails the decision; there is no partial result.decision_model("openai:<id>")is the native OpenAI Decisions API. The Responses adapter that upstream resolvesopenai:<id>to isOpenAIProvider::evaluation(id).aimux_decidetakesDecideOptionsand returnsDecideResultJSON.aimux_decision_capabilitieskeeps its export and signature and returns the model description. A provider without decisions givesAIMUX_E_NO_SUCH_MODEL.capabilities()becomesdescribe()(NodedescribeDecisionModel, Pythondescribe_decision_model);decidetakes the V4 options in every language.decideruns its model call inside the operation settings, asgenerate_textdoes, so a router, MoA or trace layer below it sees the decision's retry count and timeout.TranscriptionModel::do_streamis optional, and speech results haveusage.@ai-sdk/providermoves to 4.0.22, the firstDecisionModelV4release.decidefollowsai7.0.128, whileVERSIONS.jsonstill pinsai7.0.127.cargo test --workspacepass, as doscripts/check_provider_boundaries.sh,scripts/gen_ts_types.py --checkand the pin script tests. The Node (ava), Python (pytest), Go, Java and Kotlin suites pass locally; the Swift and Dart decision tests were updated and run in CI.docs/api/decision.mdrewritten;docs/README.md; alignment doc §0 scope line, §3.1 Decision paragraph, §3.5 decide callbacks, §5 Decision row, §9.1 decide telemetry gap, appendix note; revision lines on RFC-0037 and RFC-0038; the RFC-0023 revision line for aimux 0.5 decision recordings; CHANGELOG.ffi-x86_64-unknown-linux-gnuffi-aarch64-apple-darwinffi-x86_64-pc-windows-msvcnode-x86_64-unknown-linux-gnunode-aarch64-unknown-linux-gnunode-aarch64-apple-darwinnode-x86_64-apple-darwinnode-x86_64-pc-windows-msvcnode-aarch64-pc-windows-msvc