Skip to content

refactor(core)!: model the provider prompt by role, with one file part - #218

Closed
cunninghamcard-bit wants to merge 16 commits into
rfc-0036/provider-options-typesfrom
rfc-0036/provider-prompt-types
Closed

cunninghamcard-bit wants to merge 16 commits into
rfc-0036/provider-options-typesfrom
rfc-0036/provider-prompt-types

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Replace the provider-facing prompt struct with role-specific messages and parts based on LanguageModelV4Prompt and LanguageModelV4Message in @ai-sdk/provider. This removes role/part combinations that providers cannot accept and gives file input and tool output explicit unions. This branch follows rfc-0036/provider-options-types and precedes rfc-0036/provider-result-types; vendor converters, recordings, replay consumers and generated Node types are migrated with it.

Before and after

Concept and upstream source Before After
LanguageModelV4Message in language-model-v4-prompt.ts One LanguageModelPromptMessage struct using the call-layer ContentPart union for every role LanguageModelMessage has system, user, assistant and tool variants; system content is a string and the other roles have distinct part unions
LanguageModelV4FilePart Image, binary file, base64 file, URL file and reference file variants reach provider converters separately One FilePart uses tagged FileData; the call-layer conversion folds the existing file variants into it
LanguageModelV4ToolResultPart and LanguageModelV4ToolResultOutput Free JSON result and error flags Required tool_name and structured output: text, json, execution-denied, error-text, error-json or content; content supports text, file and custom parts
LanguageModelV4ReasoningPart and LanguageModelV4ToolCallPart Dedicated signature fields and a with_signature compatibility path Provider prompt signatures use provider options; the compatibility path is removed and conversion forwards those options without overriding them
Assistant and tool unions in language-model-v4-prompt.ts No reasoning-file, custom or tool-approval-response provider parts Assistant supports reasoning-file and custom; tool messages support tool-approval-response; part tags use upstream's hyphenated names
convert-to-openai-chat-messages.ts and shared chat consumers Non-object OpenAI tool inputs serialized directly; Groq tool output could consume OpenAI cache options; references always used the OpenAI namespace OpenAI non-object tool input becomes {}; cache breakpoints are OpenAI-only; DeepSeek resolves its own references and Groq/Compatible reject references
convert-to-openai-responses-input.ts User references could become empty text; denied approval output could produce another function-call output References become file IDs; execution-denied output carrying an approval ID is skipped
convert-to-google-messages.ts Tool content files could become text; original cloud-storage URLs were ignored Supported Vertex tool content URLs become fileData; gs: URLs preserve original_url; Vertex user references are rejected
Bedrock, Cohere, Mistral and xAI prompt conversion modules Unsupported inputs could be dropped, coerced or sent under the wrong file shape Bedrock emits S3 image locations and rejects unsupported tool-content media types; Cohere and Mistral reject unsupported references and document inputs and resolve partial image media types; xAI rejects inline non-image files

convert_to_language_model_prompt rejects parts that are invalid for their role, non-text system content and tool results without a name or a preceding named call. The call-layer message representation remains separate from these provider types. Recordings now serialize system content as a string and all provider file parts through the single file shape.

The ${string}.${string} constraint on custom kinds and the URL type are TypeScript-level constraints, not runtime validation in the upstream type module. Rust keeps String; JSON tool values remain Value.

Tests

Remove hand-written mock-server tests built around the old prompt struct, the hand-written stream-error and tool-result/reasoning suites, and session-inference tests that pin Rust-specific behavior. Keep and migrate upstream-ported tests and cassette replay tests to the role-specific prompt and structured tool output. Restore the upstream Groq tool-argument and reasoning cases, Google model-path cases and Mistral strict-tool case that had been removed with broader test files. No cassette or fixtures/ file changes are part of this branch.

The Groq file-reference test asserts upstream's error text, file parts with provider references.

Not in this pull request

  • Serde naming and optionality remain pending a maintainer decision: snake_case fields, PascalCase external tags in the existing shared wire types, Option fields serialized as null where omission is not configured, and generated Node fields that are required-nullable. In particular, message-level provider_options remains required-nullable in generated Node types while upstream providerOptions is optional. This branch does not decide the binding-wide wire contract.

  • The call-layer ModelMessage tool-result shape still uses free JSON and flags. Its replacement and the wider call-layer message/stream separation need the maintainer's decision; provider prompt types do not settle that contract.

  • Bedrock user-file media detection and validation are fixed on rfc-0036/vendor-alignment, which owns the broader vendor behavior changes. This branch's tool-content validation does not establish full user-file conversion parity.

  • OpenAI Responses approval-ID mapping in generated output and custom-tool async replay are fixed on rfc-0036/vendor-alignment; those belong to the later vendor result and tool behavior work.

  • A complete upstream test-suite port is outside this prompt migration. Keeping and restoring selected upstream cases does not establish complete coverage of each vendor converter.

  • aimux-providers/tests/anthropic_remaining_test.rs, a translation of upstream Anthropic cases, is deleted here; it comes back adapted and un-ignored on vendor-alignment, which also implements the behaviour its cases cover.

Stack refresh (2026-10-07)

Head df272b11 follows rfc-0036/provider-options-types at c22dcfad. The stack includes master at 0b365b39 and the updated #203 (3e2d9fc7), #204 (b897cb21) and #213 (7c0933b1). Updated parents were merged into their children to preserve the existing public merge history; the PR base and Draft state are unchanged.

Verification

  • Relative to the updated immediate base: 93 files changed, 4177 insertions(+), 12601 deletions(-).
  • Rust formatting and workspace Clippy passed during this layer’s refresh. Final integration at local wire-format 415e37e1 passes cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, and the provider boundary script.
  • Full cargo test --workspace at final wire-format: 2,819 passed, 0 failed, 6 ignored, including doctests. The intermediate factory-source integration also passed 2,434 tests before the final obsolete Decision hook removal. These are integration results, not a claim that the full suite was rerun at every PR head.
  • Generated TypeScript synchronization, Node build:typed, and web production build passed at the wire-format layer.

Not verified

No GitHub check runs are attached to this Draft stack head at the time of this update. Go, Java, Kotlin, Swift and Flutter bindings were not compiled locally in this refresh; Node npm test and Python pytest were not run. Live vendor calls and unported upstream cases remain outside this validation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS

chenhaonan and others added 16 commits October 5, 2026 11:51
…vider prompts by hand

What: deletes 34 test files under aimux-providers/tests. Each one builds
provider prompts from the old message struct and asserts request JSON or
parsed output that a hand-written mock server returns.

Why: the next commit replaces the provider prompt type. These files pin
Rust's own earlier behaviour rather than a recording or an upstream sample,
so they are removed instead of being rewritten. The same files are removed
on the provider factory branch. Recorded-cassette tests, upstream-sample
tests and the remaining suites stay and are converted.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
What: `LanguageModelPrompt` is now `Vec<LanguageModelMessage>`, an enum by
role: a system message holds a string; user, assistant and tool messages
hold their own part unions (`UserPart`, `AssistantPart`, `ToolPart`). The
part structs (`TextPart`, `FilePart`, `ReasoningPart`, `ToolCallPart`,
`ToolResultPart`) are each defined once. A provider sees exactly one file
part, whose `data` is the existing `FileData` union.
`convert_to_language_model_prompt` is the single conversion from user
messages: it folds the five user-facing file variants into `FilePart` and
rejects a part its role does not allow, or a system message that is not
plain text, with `InvalidPrompt`. `LanguageModelPromptMessage` is removed.

Why: this is the shape of upstream's `LanguageModelV4Message`. Before, every
provider received the user-facing part union for every role, so each
converter repeated five file arms and handled role and part combinations
that cannot occur. The user-facing `ModelMessage` and `ContentPart` and
their JSON are unchanged; the tool result fields and the two signature
fields keep their current form and are listed as remaining differences.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
…and tests

What: the vendor converters (OpenAI chat and Responses, Open Responses,
Anthropic, Bedrock, Google, Cohere, Mistral, xAI, Hugging Face), the replay
helpers, the web and replay tools and the remaining tests match on
`LanguageModelMessage` and the per-role parts. The five file arms in each
converter become one arm that matches `FilePart.data`; arms for role and
part combinations the types rule out are deleted. TypeScript types are
regenerated.

Why: follow-up to the core change. Request JSON for a given prompt is
unchanged; the recorded-cassette and upstream-sample tests pass without any
change to cassettes or fixtures.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
The changelog is what binding authors read before upgrading; the prompt type
change alters the provider trait input and the recorded prompt JSON.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
What: Cohere and Mistral decide image versus document by the top-level
media type, so a bare "image" is an image again. Inline text file data is
rejected where upstream rejects it (OpenAI Responses, Mistral, Cohere
images) and Google falls back to text/plain for an incomplete media type.
The provider tool-result part no longer accepts "output" as a second name
for "result".

Why: an independent review found that the move to one file part changed
these three behaviours; the alias gave one field two spellings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
What: brings up the tool, content and provider-option type changes and
converts this branch's code to them. In the provider prompt, a tool result
has a required tool name and a structured output with upstream's six
variants instead of a free JSON value plus flags. Reasoning and tool-call
signatures travel in provider options under the vendor namespace. The
assistant side gains reasoning-file and custom parts and the tool side
gains tool-approval-response. Hand-written tests that no longer compile
are deleted.

Why: a field-by-field comparison with the pinned upstream prompt types
found these differences. The user-level message types are unchanged; the
contract fixture pins them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
What: removes stream_error_semantics_test.rs and
tool_result_and_reasoning_fix_test.rs.

Why: the project keeps cassette replays, upstream samples and contract
fixtures as its tests. These two files asserted behaviour written by hand
against a mock and had been rewritten rather than deleted.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
What: the type tags of provider prompt parts and tool result outputs are
tool-call, tool-result, reasoning-file, tool-approval-response,
execution-denied, error-text, error-json.

Why: these types are new in this stack and no fixture pins them, so they
take upstream's spelling instead of snake_case.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
…contract fixtures

Why: carry up the stack the source union, the removal of the separate
thought-signature field, the deletion of the self-generated contract
fixtures and the changes below this branch. This branch's code is
converted to them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
… files and tool output; drop with_signature

What: OpenAI Responses sends user file references as file ids and skips a
denied tool result after a rejected approval; the shared OpenAI-style
converter sends non-object tool input as `{}`, keeps the cache breakpoint
option OpenAI-only and resolves references per vendor (DeepSeek reads its
own, Groq and the compatible package reject them); Google writes tool
content files as fileData, keeps the original gs: url and rejects user
references on Vertex; Bedrock sends s3 images as s3Location; Cohere and
Mistral handle references and document urls as upstream; xAI rejects
inline non-image files. Removes the with_signature path: signatures
travel only in provider options. Restores six ported tests; deletes the
hand-written session inference tests.

Why: review of the prompt conversion against each pinned convert-to-*
module found these divergences.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
Why: carry up the stack the fixes made after the third independent
review on the branches below; this branch's code is converted to them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
Why: carry up the stack the fixes made after the third independent
review on the branches below; this branch's code is converted to them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
What: the Groq file-reference test asserts the upstream error text,
'file parts with provider references', instead of the earlier
'No provider reference found'.

Why: the converter now rejects references as upstream does; the test
still pinned the old message.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGwanvo9WLU9HWD7sRR8TS
@cunninghamcard-bit

Copy link
Copy Markdown
Contributor Author

Folded into #219 while consolidating the RFC-0036 stack from twelve pull requests into four. This branch is unchanged; its commits are part of the range #219 now shows (base rfc-0036/integration).

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant