Skip to content

test: port the upstream Groq tests - #301

Merged
eric8810 merged 13 commits into
masterfrom
test/port-the-upstream-groq-tests
Oct 25, 2026
Merged

eric8810 merged 13 commits into
masterfrom
test/port-the-upstream-groq-tests

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/groq@4.0.52 的上游测试迁完,按厂商分批的第四个(Mistral #295、Cohere #296、DeepSeek #299)。

上游 6 个测试文件共 104 个用例:

  • 98 个有完整对应的测试并通过:原有 54 个补上出处标记,新增 44 个(其中 7 个是评审后补的整份快照比较)。
  • 5 个只有部分对应,标 partial: 并写明原因:带 provider reference 的文件部分的报错文本(aimux 的 UnsupportedFunctionality 只带功能名,Provider 构造与配置:迁移后遗留缺口 #275)、流里的 error 部分和无法解析的部分(上游快照的是 JS 的错误对象)、转写模型的两个日期用例(上游靠 _internal.currentDate 注入固定日期,Rust 侧没有这个入口)。
  • 1 个记为 ts-only:只检查 TypeScript 推断出的类型。

迁移中发现一处行为差异,单独开了 fix PR #308(上游 usage.raw 原样保留值为 null 的字段,aimux 会把它丢掉),本 PR 叠在它上面。这个修复最早在本分支上提交过,拆出去之后用一个合并提交把 #308 合进来,所以提交列表里还看得到它,但相对 #308 的 diff 只有测试和 layout.toml。

评审后(dd9c8013):原有测试里凡是只断言了上游用例一部分的,能补全的都改成整份比较,补不全的标 partial::

  • 消息转换、usage、prepareTools、请求体(provider options、service tier、tools/toolChoice、三种 response format、流式请求体)、reasoning 过滤,都改为和上游快照整份比较(整个 messages / tools / 请求体 / usage JSON / warnings);json_object 和 strict false 两个用例的输入 schema 补上了上游的 $schema。
  • 内联的裁剪 fixture(text、tool call、reasoning)换成直接读上游 __fixtures__ 原文件(src/groq/tests/fixtures/)。
  • doGenerate 的 text / tool call / reasoning、doStream 的 text 和两个 tool call 用例、raw chunks 用例:在 src/groq/tests/chat_language_model.rs 新写完整版本(整个结果与 .snap 快照或内联快照比,含 response 的 body 和 headers),tests/ 里的旧版本改标 partial: 指向它们。旧版本用 wiremock,wiremock 会多加响应头,不能逐字比 headers。

位置按新定的布局:Groq 的新测试全放 src/groq/tests/,一个上游测试文件对应一个 Rust 文件。原有的 tests/groq_test.rs、tests/groq_transcription_test.rs 留在原地,由包拆分线程的搬移 PR 合进来。

一、体系概览:Groq 的测试现在怎么跑(master 5743a40a)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                                    Groq 一个都没有(src/groq/*.rs 没有 #[cfg(test)])
  └─ tests/*.rs(126 个独立测试程序)
       ├─ groq_test.rs                    71 个,按 mod package / mod auth 和注释分组,出处写成 /// TS: "…",多数不带 describe 链
       └─ groq_transcription_test.rs      10 个
            create_groq(GroqProviderSettings { .. })                groq/mod.rs:92
            ├ GroqProvider::chat                                    groq/mod.rs:152
            │ └ GroqChatLanguageModel::do_generate / do_stream      groq/model.rs:276 / :347
            │   ├ get_args                                          groq/model.rs:97
            │   │ ├ prepare_tools                                   groq/prepare_tools.rs:22
            │   │ ├ parse_groq_options                              groq/options.rs:85
            │   │ └ convert_to_groq_chat_messages                   groq/convert.rs:112
            │   ├ create_json_response_handler::<GroqChatResponse>  groq/model.rs:284
            │   └ convert_groq_usage(response.usage)                groq/model.rs:335 / 流里取 x_groq.usage :475、:541
            │       └ usage.raw = serde_json::to_value(GroqUsage)   groq/usage.rs:44   (#308 改成保留原始字段)
            └ GroqProvider::transcription                           groq/mod.rs:159
              └ GroqTranscriptionModel::do_generate                 groq/transcription.rs

上游 packages/groq/src/ 对应:

groq-chat-language-model.test.ts             45   createTestServer → provider.chat(...).doGenerate / doStream
groq-prepare-tools.test.ts                   17   直接调 prepareTools
convert-groq-usage.test.ts                   13   直接调 convertGroqUsage
convert-to-groq-chat-messages.test.ts        11   直接调 convertToGroqChatMessages
groq-chat-language-model-options.test.ts     10   直接对 groqLanguageModelChatOptions 这个 zod schema 做 safeParse
groq-transcription-model.test.ts              8   createTestServer → GroqTranscriptionModel.doGenerate

改动前 81 个测试和上游用例对不上号:出处只写了 it 文本或改写过的描述,没有 describe 链,有两个标的文本不是上游的 it,看不出覆盖了哪些用例。

二、本 PR 的改动(细节)

1. 原有 81 个测试补标记(tests/groq_test.rs、tests/groq_transcription_test.rs)

mod convert_to_groq_chat_messages › mod user_messages / mod tool_calls     groq_test.rs:264
mod convert_groq_usage                                                     :487
mod groq_prepare_tools › mod prepare_tools › mod strict_mode_for_function_tools / mod browser_search_tool   :744
mod groq_chat_language_model › mod do_generate › mod text …               :1093
                             › mod do_stream …                             :1710
                             › mod do_stream_with_raw_chunks               :2019
mod groq_transcription_model › mod do_generate                            groq_transcription_test.rs:81
  /// upstream: @ai-sdk/groq src/groq-chat-language-model.test.ts › doGenerate › text › should extract text content

逐个对照场景和断言:66 个对应上游用例,加标记、按 describe 包 mod,其中 54 个与上游断言完全一致(upstream:),12 个只覆盖一部分(partial:,见上);另外 15 个是 aimux 自己的测试(错误体映射、环境变量读取、x_groq usage 等),放在文件末尾不标;#308 加的 1 个 aimux 测试也在末尾。第一个提交只加标记、函数体不变;之后的提交补全断言(见上)。

2. 剩下的上游用例(新目录,groq/mod.rs 末尾 #[cfg(test)] mod tests;)

cargo test -p aimux-providers --lib groq::                           44 个
  src/groq/tests/mod.rs                     mod chat_language_model; mod chat_language_model_options; …
  src/groq/tests/convert_usage.rs                    6 个   null / undefined usage、null 明细、reasoning 超过 completion 时 text 取 0
    convert_groq_usage(..)                                  groq/usage.rs:9
  src/groq/tests/convert_to_chat_messages.rs         5 个   Uint8Array 图片、只有顶层 "image" 时按字节判断子类型、image/* 归一
    convert_to_groq_chat_messages(..)                       groq/convert.rs:112
  src/groq/tests/chat_language_model_options.rs      9 个   reasoningEffort、serviceTier 的接受与拒绝,serde_json::from_value 代替 zod safeParse
    GroqLanguageModelChatOptions                            groq/options.rs:49
  src/groq/tests/prepare_tools.rs                    4 个   tools 为 None / 空、多个工具的 strict、browser search 加 toolChoice
    prepare_tools(..)                                       groq/prepare_tools.rs:22
  src/groq/tests/chat_language_model.rs             20 个   公开 API:create_groq → chat(..).do_generate / do_stream
    mod do_generate                                 10 个   text / tool call / reasoning 快照、response 信息、原始响应头、请求头、structuredOutputs 开关、Kimi K2
    mod do_stream › mod reasoning
      should_stream_reasoning()                                 :471
        /// upstream: @ai-sdk/groq src/groq-chat-language-model.test.ts › doStream › reasoning › should stream reasoning
        include_str!("fixtures/groq-reasoning.chunks.txt") → do_stream → 与 snapshots/do_stream_reasoning.json 比
    mod do_stream                                    9 个   text / tool call / reasoning 快照、tool call 增量和单 chunk、工具调用完成后多一个空 chunk 不重复、响应头、请求头、请求体
    mod do_stream_with_raw_chunks                    1 个   includeRawChunks 的整个流
  src/test_support/mod.rs                   #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;
scripts/check_provider_boundaries.sh        规则 7c、8b 的 find 跳过 */tests/*(同 #[cfg(test)] 块)

3. aimux-providers/layout.toml(只有 [groq])

18 个上游源文件:15 个 mapped(rust 是现在的文件名,搬移 PR 改名时一起改)
  groq-chat-language-model-options.ts    ts-only 1 条:type inference 只检查 TypeScript 类型
groq-tools.ts、tool/browser-search.ts    missing,issue = 275(browserSearch 工具工厂;现在要手写 Tool::Provider id "groq.browser_search")
version.ts                               not-applicable

harness 上的差别(期望值都没改):快照里值为 undefined 的键在 JSON 里省略;user-agent 检查 ai-sdk-groq/4.0.52 而不是上游测试里的 0.0.0-test;SSE 响应带上 TS 测试服务器的 cache-control、connection 头;请求体按紧凑 JSON 文本比较(serde_json 开了 preserve_order,键序不变);Uint8Array 输入用 FileBytes::Binary 构造。上游的 request.body 是 JSON 字符串,aimux 是 JSON 值,快照文件里存解析后的值。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt -p aimux-providers -- --check passes
  • cargo +1.99 clippy -p aimux-providers --lib --tests -- -D warnings passes
  • cargo +1.99 test -p aimux-providers:lib 172(其中 groq 37)、groq 72、groq_transcription 10,以及用到 Groq 的 cassette_full、conformance、non_object_tool_input、package_user_agent、presets,全过
  • 评审后(head dd9c8013):fmt、clippy、边界检查通过,cargo test -p aimux-providers --lib 加 groq* 测试程序共 3 个、268 个测试全过
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers

https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5


Generated by Claude Code

cunninghamcard-bit and others added 6 commits October 20, 2026 05:17
The 81 tests in groq_test.rs and groq_transcription_test.rs now sit in one
module per upstream describe level, and the 66 that port an upstream case
carry a `/// upstream:` marker with the full chain (@ai-sdk/groq 4.0.52).
The other 15 are aimux's own and stay at the top level. Test bodies are
unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
doGenerate › should send request body compares the whole body text, as
upstream's inline snapshot does, not two of its fields. The two
transcription date cases check that the timestamp is the clock's time
during the call; upstream fixes it through _internal.currentDate, which
has no Rust counterpart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
24 upstream cases that call a function directly, in foo/tests.rs next to
the function: convertGroqUsage 6, convertToGroqChatMessages 5,
groqLanguageModelChatOptions 9, prepareTools 4. Two convertGroqUsage
cases fail here (usage.raw drops a null field); the next commit fixes it.
One options case checks only the TypeScript type and is listed as ts-only
in layout.toml.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
convertGroqUsage (convert-groq-usage.ts:53) returns the usage object as
zod parsed it (groq-chat-language-model.ts:564-580): a null field stays
null and unknown keys are dropped. aimux re-serialized the typed struct,
which dropped null fields, so completion_tokens_details: null or
reasoning_tokens: null disappeared from usage.raw. GroqUsage now keeps
the parsed object (known keys only) and convert_groq_usage returns it.

Tests: groq::usage::tests::convert_groq_usage::should_handle_null_reasoning_tokens_in_completion_tokens_details,
should_handle_null_completion_tokens_details.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
13 groq-chat-language-model.test.ts cases in tests/it/groq/, through
create_groq and a mock fetch, with the upstream fixtures and snapshots
beside them: response information, raw headers, headers, structured
output switches, Kimi K2, streamed tool calls and reasoning, the empty
chunk after a finished tool call, and the request body.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
[groq] maps the 18 upstream source files of @ai-sdk/groq 4.0.52: 15 to
their Rust file, the browserSearch tool factory (groq-tools.ts,
tool/browser-search.ts) as missing under #275, version.ts as not
applicable. One options case is ts-only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
All Groq tests of this port now live in one directory, one file per
upstream test file: chat_language_model, chat_language_model_options,
convert_to_chat_messages, convert_usage and prepare_tools, with their
fixtures/ and snapshots/. The src/groq/*/tests.rs files and the
tests/it/ binary are gone; the shared mock fetch is reached through
src/test_support/. The boundary script skips files under tests/ dirs as
it skips #[cfg(test)] blocks. Test bodies and markers are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…m-groq-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 10:59
…elds' into test/port-the-upstream-groq-tests

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Groq test port:多个标为 upstream 的 case 仍是子集断言(text/tool/reasoning/doStream/usage),inline reasoning body 也是裁剪副本;请补完整 snapshots 或收窄“1:1”声明。

Port the doGenerate text, tool-call and reasoning snapshots, the doStream
text snapshot, both tool-call streaming inline snapshots and the raw-chunk
snapshot to src/groq/tests/chat_language_model.rs with whole-value
comparisons. The older tests/ versions of those cases are now marked
partial, as are the error-part, unparsable-part and two transcription date
cases that cannot match upstream exactly.

The remaining tests/ cases now compare the whole request body, usage,
tools and warnings, and read the upstream fixtures instead of trimmed
inline copies.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5

Copy link
Copy Markdown
Contributor Author
eric 的意见(原话) 处理
多个标为 upstream 的 case 仍是子集断言(text/tool/reasoning/doStream/usage) 改在 dd9c8013,分两类处理。补成完整断言:tests/groq_test.rs 里 usage 7 个和 doGenerate usage 4 个改为比较整个 usage JSON;消息转换比较整个 messages;prepareTools 比较整个 tools 和 warnings;provider options、service tier、tools/toolChoice、3 个 response format、流式请求体都比较整段请求体,json_object 和 strict false 的输入 schema 补上了上游的 $schema;reasoning 过滤按上游 toStrictEqual 比较整个事件。旧测试改标 partial:、完整版写在 src/groq/tests/chat_language_model.rs:doGenerate 的 text、tool call、reasoning 和 doStream 的 text(整个结果与 .snap 快照比,含 response 的 body 和 headers),doStream 的两个 tool call 用例和 raw chunks(与内联快照整份比)。旧测试用 wiremock,wiremock 会多加响应头,headers 没法逐字比。
(同上,补不全的部分) 有 5 个用例补不全,只标 partial: 并写明原因:带 provider reference 的文件部分,aimux 的报错只带功能名,缺上游的 message(#275);流里的 error 部分和无法解析的部分,上游快照的是 JS 错误对象;转写的两个日期用例,上游靠 _internal.currentDate 注入,Rust 侧没有这个入口。其中 unparsable 的用例原来多传了 include_raw_chunks,已改成和上游一样不传。
inline reasoning body 也是裁剪副本 text、tool call、reasoning 三个内联副本都删了,改为 include_str! 读上游 __fixtures__ 原文件,放在 src/groq/tests/fixtures/。
请补完整 snapshots 或收窄"1:1"声明。 两样都做了。现在 104 个用例中,98 个有 upstream: 标记(src/ 44 个,原有文件 54 个),5 个只有 partial:,1 个是 ts-only;另有 7 个 partial: 指向有完整测试的用例。PR 描述里「66 个一一对应、函数体不变」和「103 个各有一个标记」两处说法已按这个数字改写。lib + groq* 共 268 个测试全过,fmt、clippy 和边界检查也都通过。

Generated by Claude Code


Generated by Claude Code

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Groq test port 已补全 text/tool/reasoning/stream/usage 的完整 snapshots,fixtures 与上游逐字节一致,98 markers + exclusions 对齐。Approve。

…e-upstream-groq-tests

# Conflicts:
#	aimux-providers/layout.toml
…e-upstream-groq-tests

# Conflicts:
#	aimux-providers/layout.toml
@eric8810
eric8810 changed the base branch from fix/groq-usage-raw-keeps-null-fields to master October 25, 2026 17:20
@eric8810
eric8810 merged commit 5f4d41a into master Oct 25, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants