Repository navigation
test: port the upstream Google Vertex tests - #352
Merged
Merged
Conversation
…d like the AI SDK A server close before the input audio ended now fails the Gemini Live transcription stream (Google and Vertex) with `InvalidResponseData`: "<label> Live transcription WebSocket closed unexpectedly before finishing (code <code>, reason: <reason>).", the text of the `onClose` handler in @ai-sdk/google 4.0.87 (google-transcription-model.ts:577) and @ai-sdk/google-vertex 5.0.100 (google-vertex-gemini-transcription-model.ts:600). It used to pass on the socket's "websocket closed by peer (code ...)" error. `aimux_provider_utils::ws::peer_close` reads the close frame back from that error, so the format stays in ws.rs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
`VertexImageModel` reports `supports_file_inputs: Some(true)` and `supports_mask_inputs: Some(false)` for the ids in `googleVertexImageModelsWithFileInputSupport` (@ai-sdk/google-vertex 5.0.100, google-vertex-image-model.ts:29-62) and `None` for the rest. It used the trait default, `None`, for every id. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Ports the tests of @ai-sdk/google-vertex 5.0.100 (RFC-0045) into aimux-providers/src/vertex/tests/, one Rust file per upstream test file, one mod per describe level and an `/// upstream:` marker on every test: 199 tests for 199 of the 251 upstream cases. 5 more are covered by old tests, marked in the next commit. The other 47 are in aimux-providers/layout.toml (`[vertex]`): 14 sub-feature (#275) and 17 ts-only exclusions, and the 16 cases of the Cloud Text-to-Speech model, whose source file is `missing` (#275). Upstream fixtures are copied unchanged into fixtures/, `toMatchSnapshot` values become snapshots/*.json. fixtures/aimux-test-rsa-key.pem is a test key: aimux signs the service-account JWT itself where upstream mocks Web Crypto. `GoogleAuth.options` is widened to `pub(super)` so the provider tests can read the options the credential resolver was built with. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Old tests in tests/vertex_location_validation_test.rs that assert every expectation of an upstream case move, unchanged, into mod trees named after its describe chain and carry its `/// upstream:` marker (5). Tests that cover a case only in part get a `/// partial:` note pointing at the full port (the `/// TS:` lines they had are replaced). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
This was referenced Oct 20, 2026
…i-sdk ws.rs and google/transcription.rs resolve to the same tree as the Google fix PR #362 after its master merge: master's ws_error takes the error source, and closed_before_finishing builds InvalidResponseData with AiMuxError::invalid_response_data. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…est/port-the-upstream-google-vertex-tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
cunninghamcard-bit
marked this pull request as ready for review
October 24, 2026 11:23
6 of 12 tasks
cunninghamcard-bit
changed the base branch from
fix/vertex-image-and-live-transcription-match-the-ai-sdk
to
master
October 25, 2026 08:10
…eam-google-vertex-tests # Conflicts: # aimux-providers/tests/google_vertex_image_test.rs
eric's review of #352: four old tests check part of an upstream case without saying so. They now carry `partial:` notes naming the cases and their full ports: should_generate_video (vertex_video_test.rs), and in vertex_upstream_e2e_test.rs the MaaS and xAI sub-provider tests and partner_headers_win_over_per_call_headers. The other unmarked old tests have no upstream test case: the region option check, executable credentials, the partner project and user-agent rules, and the Gemini behaviours that @ai-sdk/google tests rather than @ai-sdk/google-vertex. They stay aimux-own. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…eam-google-vertex-tests
Contributor
Author
Generated by Claude Code Generated by Claude Code |
cunninghamcard-bit
pushed a commit
that referenced
this pull request
Oct 25, 2026
Master now carries the Google Vertex test port (#352, squashed). The merge uses the test branch head 7f719f0 as its base, so only master's own changes since then come in; nothing conflicted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0116HPMoqaLfFr1TyqDq1hrW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
按 RFC-0045(#281)把
@ai-sdk/google-vertex@5.0.100的上游测试迁完,按厂商分批的第十个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303、xAI #312、Anthropic AWS #319、Azure #324、Open Responses #347)。上游 22 个测试文件共 251 个用例:
src/vertex/tests/,一个上游测试文件对应一个 Rust 文件(provider 与鉴权 48 个、Anthropic 30 个、MaaS 24 个、xAI 18 个、嵌入 16 个、图像 10 个、视频 23 个、转写 27 个、location 校验 2 个、语言模型 1 个)。tests/vertex_location_validation_test.rs里已经完整覆盖的 5 个直接加上/// upstream: …标记。另有 37 处旧测试只覆盖了一部分,加/// partial: …注释指向完整用例(其中 4 处是合并 master 后评审要求补的)。这两类改动在单独的提交里,函数体不变。layout.toml的手写排除:sub-feature(Provider 构造与配置:迁移后遗留缺口 #275):GOOGLE_CLIENT_EMAIL/GOOGLE_PRIVATE_KEY/GOOGLE_PRIVATE_KEY_ID读凭据 5 个。tools(Vertex 和 Vertex Anthropic)2 个,工具结果下载上限 1 个。is_retryable1 个,视频操作的 error 状态 1 个。ts-only:new3 个。createOpenAICompatible调用次数或参数 5 个,查 mock 的 base provider 收到的参数 6 个。google-vertex-cloud-tts-speech-model.test.ts整个文件。aimux 没有 Vertex 的 Cloud Text-to-Speech 模型,layout.toml里这个文件和google-vertex-tools.ts都标成missing,挂 Provider 构造与配置:迁移后遗留缺口 #275。迁移中发现 2 处行为差异,都修在 #351 里,一处一个提交(每个提交带一个在 master 上失败的测试)。#351 已合入 master,本 PR 合并了 master(
b7ce78a3,之后又合了一次带 #295 的 master,7f719f0b),不再重复 #351 的提交和测试,那两个测试按 master 的版本保留:4ddae9c2websocket closed by peer (…),上游报… closed unexpectedly before finishing (code 1011, reason: internal error).(Google 和 Vertex 都是)a995f016一、体系概览:Google Vertex 的测试现在怎么跑(master
6c5ff9f6)上游
packages/google-vertex/src/的 22 个测试文件(用例数):二、本 PR 的改动(细节)
1. 新测试目录(
vertex/mod.rs末尾#[cfg(test)] mod tests;,:614)2. 这些测试动到的进程级状态
新测试编进
aimux-providers的 lib 测试二进制,跟其他模块的单元测试同一个进程、并行跑,所以单列出来:3.
aimux-providers/layout.toml(只加[vertex],44 个上游源文件)language model 和 location validation 的测试没有同名源文件,不需要写排除。
4. 旧测试的标记(单独一个提交,函数体不变)
合并 master 后补的 4 处:
vertex_video_test.rs的should_generate_video(对应should return completed with GCS URI video),e2e 里的maas_sub_provider_builds_on_openai_compatible、xai_sub_provider_builds_on_openai_compatible、partner_headers_win_over_per_call_headers(各自对应 MaaS / xAI provider 测试里的几个用例,完整版在src/vertex/tests/maas/、src/vertex/tests/xai/)。e2e 里其余没有标记的测试(transcription 的 region 校验、executable 凭据、partner project、user agent 规则、4 个gemini_*)不对应@ai-sdk/google-vertex的用例,是 aimux 自己的测试(gemini_*的行为在@ai-sdk/google里测)。harness 上的差别(期望值都没改):
vi.fn()查构造参数(provider、baseURL、headers、generateId)。aimux 的模型配置是私有的,这里从模型和它发出的请求上看同样的值:provider()、请求 URL 的前缀、注入的 access token 出现在Authorization头里、自定义generateId出现在 tool call id 里。generateAuthToken返回null时上游的头是Bearer null,这里让生成器返回字符串"null",得到同样的头。TooManyEmbeddingValuesForCall对应InvalidArgument。WsConnector代替上游的 mock WebSocket;TranscriptionResult没有 usage 字段,上游对 usage 的断言没有对应。ai-sdk-google-vertex/5.0.100 ai-sdk-provider-utils/5.0.53(上游 mock 成0.0.0-test),单独比。undefined的键在 JSON 里省略;header 名比小写。Type of change
Checklist
cargo +1.99 fmt --all -- --checkpassescargo +1.99 clippy -p aimux-providers -p aimux-provider-utils -p aimux-ops --lib --tests --all-features -- -D warningspassescargo +1.99 test -p aimux-providers -p aimux-provider-utils -p aimux-ops --all-features:177 个测试二进制,3024 个全过(aimux-providers lib 339,其中 vertex 199)7f719f0b):fmt、clippy、边界检查通过,cargo test -p aimux-providers --lib加*vertex*测试程序共 6 个、462 个测试全过;layout.toml与 master 的[mistral]表合并scripts/check_provider_boundaries.shpassesNotes for reviewers
src/test_support/mod.rs、aimux-providers/layout.toml,并改了边界脚本的同两行;这几处在各 PR 里内容相同,后合入的只需合并layout.toml的厂商表。layout.toml的手写排除里,16 个是missing文件里的。tools、边缘环境变量凭据等缺口挂在 Provider 构造与配置:迁移后遗留缺口 #275,行为差异挂在 Core 运行时:与 AI SDK 的剩余差距(对齐文档 §9.1 已知差距) #276。Generated by Claude Code