Skip to content

test: port the upstream DeepSeek tests - #299

Merged
eric8810 merged 14 commits into
masterfrom
test/port-the-upstream-deepseek-tests
Oct 27, 2026
Merged

eric8810 merged 14 commits into
masterfrom
test/port-the-upstream-deepseek-tests

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/deepseek@3.0.56 的上游测试迁完,按厂商分批的第三个(Mistral #295、Cohere #296)。

上游 7 个测试文件共 122 个用例:

  • 98 个有对应测试并通过:原有 75 个补上出处标记,新增 23 个。
  • 4 个记为 ts-only:只有绕过 provider、直接构造模型时才走得到。
  • 20 个是 Files 接口(uploadFile)的用例,aimux 没实现这个接口,layout.toml 记为 missing、挂 Provider 构造与配置:迁移后遗留缺口 #275,测试由实现它的 PR 带进来。

迁移中发现一处行为差异,单独开了 fix PR #309(上游会拒绝 usage 计数不是数字的响应,aimux 把 "prompt_tokens": "100" 读成 0),本 PR 叠在它上面。这个修复最早在本分支上提交过,拆出去之后用一个合并提交把 #309 合进来,所以提交列表里还看得到它,但相对 #309 的 diff 只有测试和 layout.toml。

位置按新定的布局:DeepSeek 的新测试全放 src/deepseek/tests/,保留上游的 chat/ 子目录,一个上游测试文件对应一个 Rust 文件。原有的 tests/deepseek_chat_test.rs 留在原地,由包拆分线程的搬移 PR 合进来。

一、体系概览:DeepSeek 的测试现在怎么跑(master 5743a40a)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                                    DeepSeek 一个都没有(src/deepseek/*.rs 没有 #[cfg(test)])
  └─ tests/*.rs(126 个独立测试程序)
       ├─ deepseek_test.rs                 cassette 回放
       └─ deepseek_chat_test.rs           93 个,按 // ---- describe('…') ---- 分节,注释写法不一,只有 2 个 /// TS:
            create_deepseek(DeepSeekProviderSettings { .. })        deepseek/mod.rs:86
            └ DeepSeekProvider::chat                                deepseek/mod.rs:146
              └ DeepSeekChatLanguageModel::do_generate / do_stream  deepseek/model.rs:364 / :481
                ├ request_body                                      deepseek/model.rs:154
                │ ├ convert_to_deepseek_chat_messages               deepseek/convert.rs:129
                │ └ prepare_tools                                   deepseek/prepare_tools.rs:25
                ├ create_json_response_handler::<ChatResponse>      deepseek/types.rs(usage 原样留作 Value;#309 加校验)
                └ convert_deepseek_usage(raw["usage"])              deepseek/usage.rs:16

上游 packages/deepseek/src/ 对应:

chat/deepseek-chat-language-model.test.ts          61   createTestServer → provider.chat(...).doGenerate / doStream
chat/convert-to-deepseek-chat-messages.test.ts     28   直接调 convertToDeepSeekChatMessages
chat/deepseek-prepare-tools.test.ts                 7   直接调 prepareTools
chat/deepseek-raw-usage.test.ts                     3   doGenerate / doStream 的 usage.raw 和 usage 校验
chat/is-deepseek-v4-model.test.ts                   2   直接调 isDeepSeekV4Model
chat/convert-to-deepseek-usage.test.ts              1   直接调 convertDeepSeekUsage
files/deepseek-files.test.ts                       20   DeepSeekFiles.uploadFile(aimux 没有 deepseek/files)

改动前 93 个测试和上游用例对不上号:函数名多数不是 it 文本,没有一处写完整出处,看不出覆盖了哪些用例。

二、本 PR 的改动(细节)

1. 原有 93 个测试补标记(tests/deepseek_chat_test.rs)

mod deep_seek_chat_language_model                       :529
  mod model_ids / supported_urls / do_generate › text …  :532 起
  mod do_stream …                                        :1271
mod convert_to_deep_seek_chat_messages / mod prepare_tools
  /// upstream: @ai-sdk/deepseek src/chat/deepseek-chat-language-model.test.ts › DeepSeekChatLanguageModel › doGenerate › text › should send message names

逐个对照场景和断言:75 个和上游用例一一对应,加标记、按 describe 包 mod;另外 18 个放在文件末尾不标,其中 11 个是 aimux 自己的测试,7 个用内联数据只断言了快照的一部分,对应的上游用例在第 2 步按快照完整迁。函数名、函数体不变。#309 加的 2 个 aimux 测试也在文件末尾。

2. 剩下的上游用例(新目录,deepseek/mod.rs:196 加 #[cfg(test)] mod tests;)

cargo test -p aimux-providers --lib deepseek::                       23 个
  src/deepseek/tests/mod.rs        mod chat;
  src/deepseek/tests/chat/mod.rs   mod chat_language_model; mod convert_to_chat_messages; mod convert_to_usage; …
  chat/convert_to_chat_messages.rs      6 个   message names、tool result 图片、assistant prefix
    convert_to_deepseek_chat_messages(..)                   deepseek/convert.rs:129
  chat/prepare_tools.rs                 4 个   strict 工具与 beta 端点
    prepare_tools(..)                                       deepseek/prepare_tools.rs:25
  chat/convert_to_usage.rs              1 个   reasoning 超过 completion 时 text 取 0
  chat/is_v4_model.rs                   2 个   it.each,每个函数循环全部行
  chat/chat_language_model.rs           7 个   上游 fixture + toMatchSnapshot 的用例
    mod deep_seek_chat_language_model › mod do_stream › mod reasoning
      should_stream_reasoning()                             :283
        /// upstream: @ai-sdk/deepseek src/chat/deepseek-chat-language-model.test.ts › DeepSeekChatLanguageModel › doStream › reasoning › should stream reasoning
        include_str!("fixtures/deepseek-reasoning.chunks.txt") → do_stream → 与 snapshots/do_stream_reasoning.json 比
  chat/raw_usage.rs                     3 个
    continues_validating_known_usage_field()               :181   三行字符串计数都要报错(靠 #309 的校验)
src/test_support/mod.rs              #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;
scripts/check_provider_boundaries.sh    规则 7c、8b 的 find 跳过 */tests/*(同 #[cfg(test)] 块)

3. aimux-providers/layout.toml(只有 [deepseek])

18 个上游源文件:13 个 mapped(rust 是现在的源文件名,搬移 PR 改名时一起改)
  chat/deepseek-chat-language-model.ts   ts-only 3 条:structured outputs 那组,provider 从不设 supportsStructuredOutputs(deepseek-provider.ts:88-97)
  chat/deepseek-prepare-tools.ts         ts-only 1 条:supportsStrictToolCalls 为 undefined 只出现在直接构造的模型里,provider 总按 base URL 设(:95),aimux 同样
files/deepseek-files-api.ts、deepseek-files-options.ts、deepseek-files.ts   missing,issue = 275
version.ts                                                                not-applicable

harness 上的差别(期望值都没改):快照里值为 undefined 的键在 JSON 里省略;TS 测试服务器的 requestHeaders 不含 user-agent,比较时去掉;Uint8Array 输入用 FileBytes::Binary 构造。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt --all -- --check passes
  • cargo +1.99 clippy -p aimux-providers --lib --tests -- -D warnings passes
  • cargo +1.99 test -p aimux-providers:lib 158(其中 deepseek 23)、deepseek_chat 95、deepseek 4,以及用到 DeepSeek 的 cassette_full、conformance、non_object_tool_input、openai_compatible、package_user_agent、runtime_decision、list_models,全过
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers

🤖 Generated with Claude Code

https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5

cunninghamcard-bit and others added 5 commits October 20, 2026 04:51
75 of the 93 tests in deepseek_chat_test.rs port an upstream case of
@ai-sdk/deepseek@3.0.56 and now carry an RFC-0045 marker, in one mod per
upstream describe. The other 18 are aimux's own or check only part of a
snapshot case and stay unmarked. Bodies and fn names are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
13 cases of convert-to-deepseek-chat-messages, deepseek-prepare-tools,
convert-to-deepseek-usage and is-deepseek-v4-model in foo/tests.rs next to
each source file. One prepare-tools case needs an unset strict-tools
capability that only a directly built model has; it is ts-only in
layout.toml.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
10 cases under tests/it/deepseek/, built as the single tests/it/main.rs
program, with the upstream fixtures and snapshots beside them. The three
structured-outputs cases need a model the provider never builds and are
ts-only. 'continues validating known usage field %s' fails: aimux reads a
string usage count as 0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
tokenUsageSchema (deepseek-chat-api-types.ts:95-113) keeps usage as a loose
object but requires its known counts to be numbers or null, so a response
with "prompt_tokens": "100" is rejected. aimux kept usage as a raw Value
and read such a count as 0. The response and chunk types now validate the
seven counts when they deserialize; the generate path reads usage from the
parsed response. Test: raw_usage::continues_validating_known_usage_field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Maps each upstream source file of @ai-sdk/deepseek@3.0.56 to its Rust file
(RFC-0046 §4.4) with the RFC-0045 tests table. The Files API is missing in
aimux (#275), so its upstream tests are not ported.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
All DeepSeek tests of this port now live in one directory that keeps the
upstream src/chat/ subdirectory, one file per upstream test file:
chat/chat_language_model, convert_to_chat_messages, convert_to_usage,
is_v4_model, prepare_tools and raw_usage, with their fixtures/ and
snapshots/. The src/deepseek/*/tests.rs files and the tests/it/ binary
are gone; the shared mock fetch is reached through src/test_support/.
The boundary script skips files under tests/ dirs as it skips
#[cfg(test)] blocks. Test bodies and markers are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…stream-deepseek-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 10:55
…e-numbers' into test/port-the-upstream-deepseek-tests

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DeepSeek test port:98 个 marker 覆盖 122 个 in-scope case,7 个 fixtures 逐字节一致,无生产代码变化。Approve。

cunninghamcard-bit and others added 5 commits October 25, 2026 11:28
…rt-the-upstream-deepseek-tests

# Conflicts:
#	aimux-providers/layout.toml
…rt-the-upstream-deepseek-tests

# Conflicts:
#	aimux-providers/layout.toml
…rt-the-upstream-deepseek-tests

# Conflicts:
#	aimux-providers/layout.toml
…rt-the-upstream-deepseek-tests

# Conflicts:
#	aimux-providers/layout.toml
@eric8810
eric8810 changed the base branch from fix/deepseek-usage-counts-must-be-numbers to master October 27, 2026 01:21
@eric8810
eric8810 merged commit 8570dbf into master Oct 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants