Repository navigation
test: port the upstream Open Responses tests - #347
Conversation
@ai-sdk/open-responses 2.0.58 reads `providerOptions.<name>.itemId` and `annotations` off an assistant text part (convert-to-open-responses-input.ts :279-305): the item id becomes the message `id`, `url_citation` annotations go back on the `output_text`, and a change of item id starts a new message. aimux sent the text without either. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 sets `detail` on every user `input_image` from `providerOptions.<name>.imageDetail` (`low`, `high` or `auto`, else `auto`; convert-to-open-responses-input.ts:88-98). aimux only did this for tool-result images; the two now share `image_detail`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 builds the data URL of an inline user file from `resolveFullMediaType` (convert-to-open-responses-input.ts:91-108), which detects a partial type such as `image` from the bytes and throws when it cannot. aimux used the media type as given. The pre-send check that tool results already had now covers user files too. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 throws UnsupportedFunctionalityError for a
user file given as a provider reference ("file parts with provider
references") or as text ("text file parts"),
convert-to-open-responses-input.ts:75-84. aimux dropped the part with a
warning and sent the rest. The pre-send check now fails the call.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 puts an assistant tool call's `providerOptions.<name>.itemId` on the replayed `function_call` as `id` (convert-to-open-responses-input.ts:306-336). aimux sent the call without it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 puts `{ <name>: { itemId } }` on generated and
streamed tool calls, adds `url_citation` annotations on text
(getOutputTextAnnotations), and gives reasoning
createReasoningProviderMetadata: item id, summary, content, encrypted
content (open-responses-language-model.ts:446-505, :757-779, :876-917).
aimux returned every part without provider metadata, so a replayed
response lost its item ids.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 turns a reasoning item without content into one reasoning part whose text is the joined summary (open-responses-language-model.ts:448-473), so summary and encrypted-only items survive a replay. aimux only read `content` and dropped such items. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 turns both `response.reasoning_summary_text.delta` and LM Studio's `response.reasoning_text.delta` into reasoning deltas (open-responses-language-model.ts:878-892). aimux ignored the summary deltas. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 remembers the active reasoning item id and closes it with that id in flush (open-responses-language-model.ts:997-1003). aimux kept only a flag and emitted `reasoning-end` with id `reasoning-0`, which matched no `reasoning-start`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 uses the `response.output_item.done` item's `arguments` unless they are empty, and only then the streamed ones (open-responses-language-model.ts:758-781). aimux preferred the accumulated arguments, which hold the empty string from `response.output_item.added` when no argument events arrive. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 enqueues createOpenResponsesStreamError for a `response.failed` event with `response.error` and for every `error` event (message, code, the event as data), and an `error` event sets the finish reason to error with the code (open-responses-language-model.ts:957-990). aimux only set the finish reason for `response.failed` and ignored `error` events after the first one. The first-event case stays the registered SSE pre-read: do_stream returns it as Err. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 sends `{ type: 'json_object' }` when the JSON
response format has no schema (open-responses-language-model.ts:304-315).
aimux sent `{ type: 'json_schema' }` without a schema, which servers reject.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 warns `provider-defined tool <id>` for every provider tool no extension encodes, and leaves `tool_choice` out when it names such a tool (open-responses-language-model.ts:166-300). aimux has no extensions; it dropped the tools without a warning and sent the choice as a function choice. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 resolves the effort as `openResponsesOptions.reasoningEffort ?? <mapped top-level reasoning>` (open-responses-language-model.ts:330-347), so a provider-native value such as `max` goes out as is. aimux only read `reasoningSummary` from the provider options. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 throws an APICallError with `statusCode: errorMetadata?.statusCode ?? 400` and `data: response.error` when a 200 response carries `error` (open-responses-language-model.ts :411-426). aimux used the HTTP status, 200, and left `data` empty. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/open-responses 2.0.58 throws an APICallError with status 500, `isRetryable: false`, the request and the raw response when the response has no `output` (open-responses-language-model.ts:428-441). aimux returned InvalidResponseData with the same message. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Port @ai-sdk/open-responses 2.0.58 per RFC-0045 into
src/open_responses/tests/: provider.rs and responses/{language_model,
convert_to_input, map_finish_reason}.rs, one file per upstream test file,
with the upstream fixtures and the snapshots as JSON. layout.toml gets the
[open_responses] table: the extension registry and custom tools are
missing (#275), plus 18 sub-feature exclusions and the registered SSE
pre-read.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Old tests in tests/open_responses_test.rs that assert every expectation of an upstream case move, unchanged, into mod trees named after its describe chain and carry its `/// upstream:` marker (36). Tests that cover a case only in part get a `/// partial:` note pointing at the full port (31). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Keeps this PR's API call error for a response without output (master's #290 had turned the old message into InvalidResponseData), and keeps both this PR's tests and master's raw-chunk tests (#285) at the end of open_responses_test.rs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…t/port-the-upstream-open-responses-tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…t-is-an-api-call-error' into test/port-the-upstream-open-responses-tests
…d request The missing output ApiCallError was built in do_generate from the raw request URL and body, so an inline image's base64 went into request_body_values and from there into FFI errors and recordings. The check now runs in open_responses_successful_response_handler, next to the response body error check, and uses the URL and request values the HTTP layer has already sanitized, as the other API call errors do. Test: missing_output_error_does_not_carry_inline_image_data (tests/open_responses_test.rs) fails before the change with the image data in request_body_values. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…missing-output-is-an-api-call-error # Conflicts: # CHANGELOG.md
…t-is-an-api-call-error' into test/port-the-upstream-open-responses-tests # Conflicts: # aimux-providers/layout.toml
…missing-output-is-an-api-call-error # Conflicts: # CHANGELOG.md
… into test/port-the-upstream-open-responses-tests # Conflicts: # aimux-providers/layout.toml
…missing-output-is-an-api-call-error # Conflicts: # CHANGELOG.md
… into test/port-the-upstream-open-responses-tests # Conflicts: # aimux-providers/layout.toml
…missing-output-is-an-api-call-error # Conflicts: # CHANGELOG.md
… into test/port-the-upstream-open-responses-tests # Conflicts: # aimux-providers/layout.toml
…eam-open-responses-tests
eric8810
left a comment
There was a problem hiding this comment.
Verified the post-approval commits on this head: only additive tests/docs or tightened boundary checks; no weakening or behavior regression. Re-approving the current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase is content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; tests and CI unchanged. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; CI 16/16. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master merges and layout.toml conflict resolution; CI 16/16. Re-approving current head.
…eam-open-responses-tests # Conflicts: # aimux-providers/layout.toml
eric8810
left a comment
There was a problem hiding this comment.
Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.
Summary
按 RFC-0045(#281)把
@ai-sdk/open-responses@2.0.58的上游测试迁完,按厂商分批的第九个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303、xAI #312、Anthropic AWS #319、Azure #324)。上游 5 个测试文件共 124 个用例:
tests/open_responses_test.rs里已经完整覆盖的 36 个(finish reason 8 个、输入转换 17 个、语言模型 11 个)直接加上/// upstream: …标记。另有 31 处只覆盖了一部分,加/// partial: …注释指向完整用例。这两类改动单独一个提交,函数体不变。layout.toml的手写排除:sub-feature(Provider 构造与配置:迁移后遗留缺口 #275):customToolId自定义工具 3 个。strictResponseInput2 个。getResponseErrorMetadata2 个。registered(SSE 预读):流的第一个事件就是error时,aimux 的do_stream直接返回 Err,不进流。open-responses-extension.test.ts整个文件。它测的createOpenResponsesExtensionRegistry在 aimux 里没有,layout.toml里这个文件和open-responses-tools.ts都标成missing,挂 Provider 构造与配置:迁移后遗留缺口 #275。迁移中发现 16 处行为差异,都修在 #346 里,一处一个提交(每个提交带一个在 master 上失败的测试),#346 已合入 master:
996eaea1url_citation标注4cfcd02ddetail(上游默认auto,可由 providerOptions 指定)cfac970a53ea2cf69e54ca4a787f45ef3eb0e5f0506be6b7response.reasoning_summary_text.delta被忽略278fb9ec2aab71e80b1433dbresponse.failed和error事件没有变成 error partd3865fa7{ "type": "json_object" }61e808d938892609providerOptions.<name>.reasoningEffortc8afdf3f211a1385位置按新定的布局:新测试全放
src/open_responses/tests/,一个上游测试文件对应一个 Rust 文件。一、体系概览:Open Responses 的测试现在怎么跑(master
b91829af)上游
packages/open-responses/src/对应:二、本 PR 的改动(细节)
1. 新测试目录(
open_responses.rs末尾#[cfg(test)] mod tests;,:1788)2.
aimux-providers/layout.toml(只有[open_responses])3.
tests/open_responses_test.rs:36 个测试加/// upstream: @ai-sdk/open-responses src/<file> › <describe 链>,原来的 11 行/// TS: …:行号换成这个;31 处加/// partial: …; full port in src/open_responses/tests/<file>.rs。harness 上的差别(期望值都没改):
createModel直接 newOpenResponsesLanguageModel,这里用create_open_responses(..).responses(..),差别只在这些用例不比的 URL 和 headers。AiMuxError的变体和字段,不比同一个错误对象;UnsupportedFunctionalityError对应UnsupportedFunctionality(functionality)。undefined的键在 JSON 里省略。Type of change
Checklist
cargo +1.99 fmt -p aimux-providers -- --checkpassescargo +1.99 clippy -p aimux-providers --lib --tests -- -D warningspassescargo +1.99 test -p aimux-providers:lib 192(其中 open_responses 57),129 个测试二进制全过scripts/check_provider_boundaries.shpassesNotes for reviewers
src/test_support/mod.rs、aimux-providers/layout.toml,并改了边界脚本的同两行;这几处在各 PR 里内容相同,后合入的只需合并layout.toml的厂商表。layout.toml的手写排除里,12 个是missing文件里的。map-open-responses-finish-reason.test.ts的 8 个用例全在旧文件里有完整覆盖,所以map_finish_reason.rs只留一段说明,不重复写。strictResponseInput、getResponseErrorMetadata的缺口挂在 Provider 构造与配置:迁移后遗留缺口 #275,行为差异挂在 Core 运行时:与 AI SDK 的剩余差距(对齐文档 §9.1 已知差距) #276。Generated by Claude Code