Repository navigation
test: port the upstream OpenAI-compatible tests - #355
Merged
Merged
Conversation
… error A streamed SSE event whose data is not JSON yielded `JsonParse` with only the parser's message. The AI SDK's `parseJsonEventStream` reads each event with `safeParseJSON`, whose `JSONParseError` reads `JSON parsing failed: Text: <text>.\nError message: <message>` (`@ai-sdk/provider` src/errors/json-parse-error.ts:16-19). Found while porting the `@ai-sdk/openai-compatible` 3.0.62 tests (`doStream › should handle unparsable stream parts`, chat and completion); every stream read through `create_event_source_response_handler` now reports it the same way. Well-formed JSON of the wrong shape stays `InvalidResponseData`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Ports the tests of @ai-sdk/openai-compatible 3.0.62 (RFC-0045) into aimux-providers/src/openai_compatible/tests/, one Rust file per upstream test file, one mod per describe level and an `/// upstream:` marker on every test: 266 tests for 266 of the 268 upstream cases. The other 2 are ts-only in aimux-providers/layout.toml (`[openai_compatible]`): they read the settings handed to mocked model constructors, which aimux's completion, embedding and image models never read. Upstream fixtures are copied unchanged into chat/fixtures/, the four `toMatchSnapshot` values become chat/snapshots/*.json. The model `config` fields (and `CompletionConfig`) are widened to `pub(super)` so the tests can read the configuration a model was built with. Boundary rule 4c lets the in-crate tests name `crate::test_support`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Old tests in openai_compatible_upstream_e2e_test.rs and openai_compatible_factory_test.rs that cover part of an upstream case get a `/// partial:` line naming the case and the file with its full port (20 lines). None of them covers a whole case, so none gets an `/// upstream:` marker. Test bodies are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
This was referenced Oct 20, 2026
…ai-sdk-parse-errors Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…-errors into test/port-the-upstream-openai-compatible-tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
cunninghamcard-bit
marked this pull request as ready for review
October 24, 2026 11:08
6 of 12 tasks
cunninghamcard-bit
changed the base branch from
fix/openai-compatible-unparsable-stream-chunks-are-ai-sdk-parse-errors
to
master
October 25, 2026 08:10
…eam-openai-compatible-tests
layout.toml: master's [mistral] table (#295) and this PR's [openai_compatible] table are both kept. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…eam-openai-compatible-tests # Conflicts: # aimux-providers/layout.toml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
按 RFC-0045(#281)把
@ai-sdk/openai-compatible@3.0.62的上游测试迁完,按厂商分批的第十一个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303、xAI #312、Anthropic AWS #319、Azure #324、Open Responses #347、Google Vertex #352)。上游 10 个测试文件共 268 个用例:
src/openai_compatible/tests/,一个上游测试文件对应一个 Rust 文件(chat 语言模型 107 个、消息转换 50 个、工具 12 个、usage 3 个、content array 2 个、驼峰键 17 个、completion 29 个、嵌入 9 个、图像 26 个、provider 11 个)。layout.toml的ts-only:supportsStructuredOutputs和supportsMultiPartToolContent两个 provider 设置“只传给 chat 模型”。上游查的是 mock 掉的模型构造函数收到的参数;aimux 的 completion、嵌入、图像模型共用 provider 的 chat 设置,但从不读这两个字段,行为上没有可观察的差别。/// upstream:标记;其中 20 处只覆盖了一部分,加了/// partial: …指向完整用例,这部分单独一个提交,函数体不变。迁移中发现 1 处行为差异,修在 #354 里(带一个在 master 上失败的测试)。本 PR 叠在 #354 上:
cb0b764dJsonParse只带 serde 的报错;上游是JSON parsing failed: Text: {unparsable}.\nError message: …。改在 provider-utils 共用的 SSE 解析里,所有走create_event_source_response_handler的厂商一起对齐一、体系概览:OpenAI-compatible 的测试现在怎么跑(master
6c5ff9f6)上游
packages/openai-compatible/src/的 10 个测试文件(用例数):二、本 PR 的改动(细节)
1. 新测试目录(
openai_compatible/mod.rs末尾#[cfg(test)] mod tests;,:368)2. 这些测试动到的进程级状态
新测试编进
aimux-providers的 lib 测试二进制,和其他模块的单元测试同进程并行跑。它们不读也不改环境变量,没有#[serial],没有全局计数器,不写临时文件;每个测试用自己的MockFetch。3.
aimux-providers/layout.toml(只加[openai_compatible],26 个上游源文件)chat/openai-compatible-chat-content-array.test.ts没有同名源文件,用例全迁了,不需要写排除。4. 旧测试的标记(单独一个提交,函数体不变)
harness 上的差别(期望值都没改):
new OpenAICompatibleChatLanguageModel(...)等;这里用create_openai_compatible建,config ›用例用CompatModelConfig加from_config建,读model.config。provider(id)和provider.languageModel(id)返回的模型读不到 config,includeUsage等从它发出的请求体里看。includeUsage: undefined在 Rust 里是 bool,按false断言。request.body是 JSON 字符串,这里是 JSON 值,比之前先解析快照里的字符串。content-type、content-length或cache-control、connection)。ApiCallError.data(上游的原始error对象);抛出的错误比{name, message}。坏 JSON 的报错只比\nError message:之前的部分,后面是 JS 运行时JSON.parse自己的文字。currentDate钩子,只断言时间戳落在调用期间。toThrow("'X' functionality not supported")比UnsupportedFunctionality(X)的 functionality 字符串。ai-sdk-openai-compatible/3.0.62 ai-sdk-provider-utils/5.0.53(上游 mock 成0.0.0-test),单独比。快照里值为undefined的键在 JSON 里省略。Type of change
Checklist
cargo +1.99 fmt --all -- --checkpassescargo +1.99 clippy -p aimux-providers -p aimux-provider-utils -p aimux-ops --lib --tests --all-features -- -D warningspassescargo +1.99 test -p aimux-providers -p aimux-provider-utils -p aimux-ops --all-features:177 个测试二进制,3091 个全过(aimux-providers lib 406,其中 openai_compatible 272:新 266、原有 6)scripts/check_provider_boundaries.shpassesNotes for reviewers
src/test_support/mod.rs、aimux-providers/layout.toml,并改了边界脚本规则 7c、8b 的同两行;这几处在各 PR 里内容相同,后合入的只需合并layout.toml的厂商表。规则 4c 的改动只在本 PR。layout.toml的ts-only里。Generated by Claude Code