Skip to content

test: port the upstream Google tests - #363

Merged
eric8810 merged 11 commits into
masterfrom
test/port-the-upstream-google-tests
Oct 27, 2026
Merged

eric8810 merged 11 commits into
masterfrom
test/port-the-upstream-google-tests

Conversation

@cunninghamcard-bit

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/google@4.0.87 的上游测试迁完,按厂商分批的第十四个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303、xAI #312、Anthropic AWS #319、Azure #324、Open Responses #347、Google Vertex #352、OpenAI-compatible #355、Amazon Bedrock #358、Anthropic #361)。

上游 33 个测试文件共 840 个用例,其中 330 个属于 aimux 没有的模块(下面 missing),在范围内的 510 个:

  • 438 个新写了测试并通过,放在 src/google/tests/,一个上游测试文件对应一个 Rust 文件。google-language-model.test.ts 里有两个同名用例 doGenerate › should pass tools and toolChoice,第二个的标记末尾带 #2(RFC-0045 §4)。
  • 58 个由原有的旧测试完整覆盖,旧测试挪进按 describe 链命名的 mod,加上 /// upstream: 标记,函数名和函数体不变;另有 57 处旧测试只覆盖了一部分,加了 /// partial: … 指向完整用例。这部分单独一个提交。
  • 14 个写进 layout.toml 的 tests 表:
    • 10 个 sub-feature(Provider 构造与配置:迁移后遗留缺口 #275):没有 supportedFunctionResponseUrls(2 个),没有工具结果文件下载 downloadToolResultFiles(1 个),grounding metadata 存成无类型 JSON、不做 schema 校验(1 个),embedding 没有按批次变换 provider options 的钩子(1 个),文件上传没有逐次调用的 headers(1 个),aimux-core 的 ImageResult 没有 isRetryable(1 个),TranscriptionResult 和转写流的 Finish 没有 usage(2 个),没有 Batch API(1 个)。
    • 4 个 ts-only:上传数据是同一个 Uint8Array 视图(对象身份),往 uploadFile 传 ReadableStream(JS 运行时检查),workflow 序列化钩子,mock 构造函数收到的 generateId(上游视频模型从不调用它)。

不在范围内的 330 个:interactions/(194,没有 Interactions 语言模型)、realtime/(84,没有 realtime 模型)、google-batch.ts(27)、speech-translation/(21)、download-tool-result-files.ts(4),文件条目都是 missing,挂 #275,用例不迁。

迁移中发现 1 处行为差异,修在 #362 里;#362 还带上了 #351 的 Live 转写关闭修复(Google 的转写用例也用到,代码在 Google 共用的转写模块里)。本 PR 叠在它上面:

提交 差异
e3fc27aa gemini-2.5-flash-image 等三个图片模型报告支持文件输入、不支持 mask
0cd2bed5 Live 转写在音频结束前被关闭时,报上游的 closed unexpectedly before finishing (code, reason)(与 #351 同一改动)

一、体系概览:Google 的测试现在怎么跑(master 24977c43)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                       google 1 个(utils.rs 里的 #[cfg(test)])
  └─ tests/*.rs
       ├─ google_speech_test.rs                   40 个
       ├─ google_image_test.rs                    19 个(另有 google_vertex_image_test.rs 8 个,属于 Vertex)
       ├─ google_files_test.rs                    17 个   本地 wiremock 服务器
       ├─ google_transcription_test.rs            16 个
       ├─ google_upstream_e2e_test.rs             13 个
       ├─ google_embedding_test.rs、google_evaluation_test.rs   各 7 个
       ├─ google_factory_test.rs                   6 个   回放 fixtures/aisdk/google 的录制
       └─ google_video_test.rs                     4 个
            create_google(GoogleProviderSettings { .. })              google/mod.rs:239(google() :274)
            └ GoogleProvider                                         :284   impl Provider :404
              ├ chat → GoogleModel                                   model.rs:43   impl LanguageModel :154(do_generate :167、do_stream :245)
              │   ├ convert_to_google_messages_for_namespace         convert.rs:86
              │   ├ prepare_tools                                    convert.rs:562
              │   ├ get_google_model_capabilities                    convert.rs:637
              │   ├ GoogleJsonAccumulator(流式工具参数)              utils.rs:121
              │   └ convert_usage / parse_finish_reason              convert.rs:1736 / :1704
              ├ embedding → GoogleEmbeddingModel                     embedding.rs:25
              ├ image → GoogleImageModel                             image.rs:32(本 PR 的基线 #362 上)
              ├ video → GoogleVideoModel                             video.rs:26
              ├ speech → GoogleSpeechModel                           speech.rs:230
              ├ transcription → GoogleTranscriptionModel             transcription.rs:173(Live API 走 WebSocket)
              └ files → GoogleFiles                                  files.rs:137   上传走校验过的下载路径,不用注入的 transport(已登记差异 D26)
            错误:google_failed_response_handler                       mod.rs:95

上游 packages/google/src/ 的 33 个测试文件(用例数):

google-language-model.test.ts                       199
interactions/*.test.ts(9 个文件)                   194   aimux 没有 Interactions 语言模型
realtime/*.test.ts(2 个文件)                        84   aimux 没有 realtime 模型
convert-to-google-messages.test.ts                   66
google-prepare-tools.test.ts                         38
google-speech-model.test.ts                          33
google-json-accumulator.test.ts                      31
google-batch.test.ts                                 27   aimux 没有 Batch API
google-files.test.ts                                 26
google-video-model.test.ts                           23
google-provider.test.ts                              22
speech-translation/google-speech-translation-model.test.ts   21   aimux 没有语音翻译模型
google-embedding-model.test.ts                       17
transcription/google-transcription-model.test.ts     15
google-image-model.test.ts                           14
google-speech-input.test.ts                           7
google-evaluation.test.ts                             7
download-tool-result-files.test.ts                    4   aimux 没有工具结果文件下载
google-supported-file-url.test.ts                     4
get-model-path.test.ts                                3
convert-google-usage.test.ts                          2
sanitize-response-json-schema.test.ts                 1
google-model-capabilities.test.ts                     1
google-error.test.ts                                  1

二、本 PR 的改动(细节)

1. 新测试目录(google/mod.rs 末尾 #[cfg(test)] mod tests;,:519)

cargo test -p aimux-providers --lib google::tests::              438 个
  src/google/tests/mod.rs              每个有新测试的上游测试文件一个 mod
  language_model.rs                196   do_generate(117)/ do_stream(56)/ grounding_metadata_schema(14)/ gemma_model_system_instruction_fix(6)/ url_context_metadata(3)
  convert_to_messages.rs            64   validate_call_options_for_namespace + convert_to_google_messages_for_namespace;Gemma、Gemini 3 的用例经 build_request_body_with_warnings_for_namespace
  prepare_tools.rs                  38   prepare_all_tools;isVertexProvider 的用例建 Namespace::Vertex 的请求体读 tools
  json_accumulator.rs               31   GoogleJsonAccumulator
  video_model.rs                    23
  provider.rs                       20
  embedding_model.rs                14
  image_model.rs、files.rs           各 11
  transcription/transcription_model.rs   8   Live 用例用内存 MockWebSocket,经 GoogleProviderSettings::web_socket 传入
  speech_model.rs                    7
  supported_file_url.rs              4
  get_model_path.rs                  3
  convert_usage.rs、evaluation.rs    各 2
  error.rs、model_capabilities.rs、sanitize_response_json_schema.rs、speech_input.rs   各 1
  fixtures/(15 个)                     上游 __fixtures__ 原样拷贝
  snapshots/(14 个)                    上游 toMatchSnapshot 转成 JSON
  src/test_support/mod.rs                #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;(:408)
可见性                                   convert.rs:86 的 convert_to_google_messages_for_namespace 改成 pub(super)(带 namespace 和 supports_function_response_parts 两个参数的那个)
scripts/check_provider_boundaries.sh     规则 7c、8b 的 find 跳过 */tests/*

2. 这些测试动到的进程级状态

新测试编进 aimux-providers 的 lib 测试二进制,和其他模块的单元测试同进程并行跑。它们不读也不改环境变量,没有 #[serial],不换进程默认 transport,没有全局计数器,不写临时文件。语言模型、图片、视频、语音、转写、embedding 的用例各用自己的 MockFetch(Live 转写用自己的 MockWebSocket),需要 URL 的都显式传 base_url。

files.rs 的 11 个用例例外:文件上传走 aimux 校验过的下载路径,这条路不用注入的 transport(D26),所以每个用例起一个本地 wiremock 服务器(127.0.0.1 随机端口),base_url 指向 {server}/v1beta,用例结束服务器随之关闭。和原有的 google_files_test.rs 用法一样。

3. aimux-providers/layout.toml(只加 [google],66 个上游源文件)

32 个 mapped 到 google/*.rs
interactions/*(17 个)、realtime/*(4 个)、speech-translation/*(2 个)、
google-batch.ts、download-tool-result-files.ts、google-tools.ts、tool/*.ts(7 个)   missing,issue 275
version.ts                                                   not-applicable:版本号钉在 shared/aisdk_pins.rs,user agent 用例比的就是它
convert-to-google-messages.ts  sub-feature 2(supportedFunctionResponseUrls)
google-language-model.ts       sub-feature 2(工具结果文件下载、grounding metadata 校验),ts-only 1(workflow 序列化)
google-files.ts                sub-feature 1(逐次调用的 headers),ts-only 2(Uint8Array 视图、ReadableStream)
google-provider.ts             sub-feature 1(Batch API),ts-only 1(视频模型的 generateId)
transcription/google-transcription-model.ts   sub-feature 2(usage)
google-embedding-model.ts、google-image-model.ts   各 sub-feature 1(按批次的 provider options、isRetryable)

google-speech-model.test.ts 等几个测试文件名和源文件名对不上的,都按源文件登记;.test-d.ts 类型测试不列。

4. 旧测试的标记(单独一个提交,函数名和函数体不变)

tests/google_speech_test.rs           32 个挪进 mod 加 /// upstream:(do_generate、modern_models、get_google_speech_input),7 处 /// partial:
tests/google_files_test.rs            12 个挪进 mod google_files::{constructor, upload_file::{polling, error_handling, provider_options}} 加 /// upstream:,4 处 /// partial:
tests/google_transcription_test.rs     5 个挪进 mod 加 /// upstream:,10 处 /// partial:
tests/google_evaluation_test.rs        5 处 /// TS: 换成 /// upstream:(上游文件没有 describe,不分 mod),2 处 /// partial:
tests/google_image_test.rs             2 个挪进 mod 加 /// upstream:,12 处 /// partial:
tests/google_embedding_test.rs         2 个挪进 mod 加 /// upstream:,5 处 /// partial:
tests/google_upstream_e2e_test.rs     11 处 /// partial:(换掉原来的 /// TS: 行)
tests/google_video_test.rs             5 处 /// partial:
src/google/utils.rs                    1 处 /// partial:

harness 上的差别(期望值都没改,除了下面写明的比法):

  • 上游直接 new GoogleLanguageModel(...);这里用 create_google 加 api_key、base_url、fetch 建。上游 provider 写成 google.vertex.chat 的用例用 GoogleModel::from_config(.., Namespace::Vertex)。
  • 上游对 Vertex provider 传 includeFunctionCallIds: !isVertexProvider(google-language-model.ts:329),aimux 按 Namespace::Vertex 决定(convert.rs:428)。convert-to-google-messages 里 4 个 Vertex 用例直接调转换函数时上游默认带 id,这里期望值经 without_function_call_ids 去掉 id 再比,函数注释里写明了原因。
  • 请求体按 JS 读 JSON 的方式比;mock 响应带上测试服务器的 content-type 和 content-length。toMatchObject 用递归的子集比较。
  • user agent 钉住的是 ai-sdk-google/4.0.87 ai-sdk-provider-utils/5.0.53(上游 mock 成 0.0.0-test),单独比。快照里值为 undefined 的键在 JSON 里省略。
  • Rust 没有 currentDate 钩子:上游比固定时间戳的地方,这里比时间戳在调用前后之间。
  • grounding metadata、url context metadata 没有独立的 schema,用例在响应里带上这段 metadata,看 types::Candidate 能不能解析。
  • JSON accumulator 的原型污染用例:Rust 没有 Object.prototype,只比累积出的对象(自带 __proto__、constructor.prototype 键),和上游断言一致。
  • supported-file-url 的输入先经 url::Url::parse(..).to_string(),和上游 new URL(..) 得到同样的形式。
  • 上游 mock 了模型构造函数的 provider 用例,这里比调用方看得到的东西:provider()、model_id()、max_images_per_call()、supported_urls(),以及模型发出的那一个请求的 URL 和 headers。
  • files 用例按 path 比 URL(wiremock 报出的 URL 不带端口);"ArrayBuffer-backed data" 里"不往 fetch 传 Content-Length"不比,线上客户端总会算出这个头。
  • it.each 的各行在一个函数里循环跑。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt --all -- --check passes
  • cargo +1.99 clippy -p aimux-providers -p aimux-provider-utils -p aimux-ops --lib --tests --all-features -- -D warnings passes
  • cargo +1.99 test -p aimux-providers -p aimux-provider-utils -p aimux-ops --all-features:167 个测试二进制,3266 个测试通过(比 fix: Google image inputs and live transcription match the AI SDK #362 多 438 个)
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers


Generated by Claude Code

cunninghamcard-bit and others added 2 commits October 20, 2026 20:18
Ports the tests of @ai-sdk/google 4.0.87 (RFC-0045) into
aimux-providers/src/google/tests/, one Rust file per upstream test file,
one mod per describe level and an `/// upstream:` marker on every test:
438 new tests. With the 58 old tests marked in the next commit, 496 of the
510 in-scope upstream cases have a test (google-language-model.test.ts has
two cases named `doGenerate › should pass tools and toolChoice`; the
second marker ends in `#2`, RFC-0045 §4). aimux-providers/layout.toml
(`[google]`) lists the rest: interactions/, realtime/, speech-translation/,
google-batch.ts, download-tool-result-files.ts, google-tools.ts and the
tool/*.ts factories are missing (#275), so their 330 cases are not ported;
10 cases are sub-feature (#275: no supportedFunctionResponseUrls, no tool
result file download, untyped grounding metadata, no per-batch embedding
options transformer, no per-call file upload headers, no ImageResult
isRetryable, no transcription usage, no Batch API) and 4 ts-only.

Upstream fixtures are copied unchanged into fixtures/, the
`toMatchSnapshot` values become snapshots/*.json.
`convert_to_google_messages_for_namespace` is widened to `pub(super)` so
the tests can convert for the Vertex namespace; the Vertex-provider cases
expect no function call ids, as upstream passes
`includeFunctionCallIds: !isVertexProvider`. Boundary rules 7c and 8b skip
`tests/` folders. The files tests start local wiremock servers (random
ports): file upload goes through the validated download path, which never
uses an injected transport (registered difference D26). The new tests
touch no environment variable and no process-wide state.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
58 old tests that assert everything an upstream @ai-sdk/google 4.0.87
case asserts get its `/// upstream:` marker and move into mods named after
its describe chain; their names and bodies are unchanged. 57 `/// partial:`
lines point the old tests that cover a case only in part at the full port
in src/google/tests/. Old `/// TS:` lines that the markers replace are
removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
cunninghamcard-bit and others added 2 commits October 24, 2026 09:59
… into test/port-the-upstream-google-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
The abort tests drop the client mid-upload; on a pooled server the late
upload byte reached whichever test borrowed that server next.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 11:11
…-transcription-match-the-ai-sdk' into test/port-the-upstream-google-tests

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Google test port:438 tests、fixtures 逐字节一致、snapshots 精确,无生产行为变化;包含 #362 base 的测试变更。Approve。

cunninghamcard-bit and others added 6 commits October 25, 2026 11:28
…e-ai-sdk' into test/port-the-upstream-google-tests

# Conflicts:
#	aimux-providers/layout.toml
…e-ai-sdk' into test/port-the-upstream-google-tests

# Conflicts:
#	aimux-providers/layout.toml
…e-ai-sdk' into test/port-the-upstream-google-tests

# Conflicts:
#	aimux-providers/layout.toml
…e-ai-sdk' into test/port-the-upstream-google-tests

# Conflicts:
#	aimux-providers/layout.toml
…e-ai-sdk' into test/port-the-upstream-google-tests
…e-ai-sdk' into test/port-the-upstream-google-tests

# Conflicts:
#	aimux-providers/layout.toml
@eric8810
eric8810 changed the base branch from fix/google-image-inputs-and-live-transcription-match-the-ai-sdk to master October 27, 2026 04:37
@eric8810
eric8810 merged commit 911de9e into master Oct 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants