Skip to content

test: port the upstream Hugging Face tests - #303

Merged
eric8810 merged 29 commits into
masterfrom
test/port-the-upstream-hugging-face-tests
Oct 27, 2026
Merged

eric8810 merged 29 commits into
masterfrom
test/port-the-upstream-hugging-face-tests

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/huggingface@2.0.62 的上游测试迁完,按厂商分批的第五个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301)。

上游 2 个测试文件共 38 个用例:

迁移中发现三处行为差异,各自单独一个 fix PR,本 PR 叠在最后一个上:

这几处修复最早在本分支上提交过,拆出去之后用一个合并提交把 #307 合进来。#307 已合入 master,所以相对 master 的 diff 只有测试和 layout.toml。

原有测试里 18 个只覆盖了上游用例的一部分(或换了输入),完整用例在新文件里重新迁了一遍,原测试留着但不计数,注释指向完整的那个。

位置按新定的布局:一个厂商的测试全放 src/huggingface/tests/,一个上游测试文件对应一个 Rust 文件。原有的 tests/huggingface_responses_test.rs 留在原地,由包拆分线程的搬移 PR 合进来。

一、体系概览:Hugging Face 的测试现在怎么跑(master 5743a40a)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                                   Hugging Face 一个都没有
  └─ tests/*.rs
       └─ huggingface_responses_test.rs     32 个,出处写成 /// TS: "…",不带 describe 链
            create_huggingface(HuggingFaceProviderSettings { .. })        huggingface.rs:106
            ├ Provider::embedding_model / image_model                    huggingface.rs:203 / :207   → NoSuchModel,没有上游的提示文字
            └ HuggingFaceProvider::responses                              huggingface.rs:177
              └ HuggingFaceResponsesModel::do_generate / do_stream        huggingface/responses.rs:163 / :216
                ├ build_request_body_with_warnings                        responses.rs:580
                │ ├ convert_to_huggingface_responses_messages             responses.rs:726
                │ └ prepare_responses_tools                               responses.rs:880
                ├ do_generate → build_generate_content                    responses.rs:933
                │   ├ "function_call"                                     :995    ← 有 output 也不出 tool-result(#306 修)
                │   └ "mcp_call" / "mcp_list_tools"                       :1013   ← dynamic: Some(true)(#305 修)
                └ do_stream → 预读首个事件                                responses.rs:232   首事件是错误就返回 Err(SSE 预读)
                    └ HuggingFaceResponsesStream::transform_event         responses.rs:306
                        ├ huggingface_stream_error(event)                 responses.rs:94    ← 只看顶层 error 字段(#307 修)
                        └ Err(解析失败)                                   :536               ← finish_reason 不变(#307 修)

上游 packages/huggingface/src/ 对应:

huggingface-provider.test.ts                               6   createHuggingFace → provider.responses / embeddingModel / imageModel
responses/huggingface-responses-language-model.test.ts    32   createTestServer → provider.responses(...).doGenerate / doStream

改动前 32 个测试和上游用例对不上号:出处只写了 it 文本,没有 describe 链,18 个只比了部分字段或换了输入。

二、本 PR 的改动(细节)

1. 原有 32 个测试补标记(tests/huggingface_responses_test.rs)

mod hugging_face_responses_language_model › mod do_stream / mod message_conversion / mod structured_output
                                           › mod provider_options / mod tool_preparation / mod top_level_only_media_type_resolution
  /// upstream: @ai-sdk/huggingface src/responses/huggingface-responses-language-model.test.ts › HuggingFaceResponsesLanguageModel › message conversion › should handle assistant messages

12 个和上游用例一一对应,加标记、按 describe 包 mod;18 个只比了部分字段或换了输入,注释写明缺什么、完整用例在哪;2 个是 aimux 自己的测试(坏帧之后流继续、supported_urls);三个 fix PR 另加的 3 个 aimux 测试也在这个文件末尾。函数体不变(逐行比对过)。

2. 剩下的上游用例(新目录,huggingface.rs 末尾 #[cfg(test)] mod tests;)

cargo test -p aimux-providers --lib huggingface::                    22 个
  src/huggingface/tests/mod.rs                 mod provider; mod responses;
  src/huggingface/tests/provider.rs            4 个   默认 / 自定义配置、responses 和 language_model 方法
  src/huggingface/tests/responses/responses_language_model.rs
    mod do_generate / do_stream / message_conversion / tool_calls / structured_output / reasoning   18 个
      /// upstream: … › doGenerate › basic text response › should handle MCP tools with annotations
      MockFetch::json(..) → provider.responses(MODEL).do_generate → 与上游 toMatchInlineSnapshot 的 JSON 比
  src/test_support/mod.rs                      #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;
scripts/check_provider_boundaries.sh           规则 7c、8b 的 find 跳过 */tests/*(同 #[cfg(test)] 块)

3. aimux-providers/layout.toml(只有 [huggingface])

12 个上游源文件:10 个 mapped(rust 是现在的源文件名)
  huggingface-provider.ts                       sub-feature 2 条,issue = 276:NoSuchModelError 的提示文字
  huggingface-responses-language-model.ts       registered 1 条,ref = "SSE 预读":首事件错误
                                                sub-feature 1 条,issue = 276:UnsupportedFunctionalityError 的提示文字
responses/huggingface-responses-settings.ts、version.ts    not-applicable

harness 上的差别(期望值都没改):createTestServer 换成 MockFetch,server.calls[0] 换成它记下的请求体;快照里值为 undefined 的键在 JSON 里省略;typeof provider === 'function' 换成调用 HuggingFaceProvider::call。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt -p aimux-providers -- --check passes
  • cargo +1.99 clippy -p aimux-providers --lib --tests -- -D warnings passes
  • cargo +1.99 test -p aimux-providers:lib 157(其中 huggingface 22)、huggingface_responses 35,以及用到 Hugging Face 的 cassette_full、conformance、package_user_agent、thin_wrapper_config,全过
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers

https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5


Generated by Claude Code

cunninghamcard-bit and others added 6 commits October 20, 2026 06:19
Each test that ports an @ai-sdk/huggingface@2.0.62 case carries the full
describe chain as an `/// upstream:` marker and sits in nested describe
mods. Tests that assert only part of an upstream case say so and point at
the full port; aimux's own tests stay unmarked. Test bodies are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
The upstream cases the old tests do not cover in full go to
src/huggingface/tests/, one file per upstream test file (provider.rs,
responses/responses_language_model.rs). The shared mock fetch is reached
through src/test_support/. The boundary script skips files under tests/
dirs as it skips #[cfg(test)] blocks.

Three cases fail here and are fixed in the next commits: MCP calls are
marked dynamic, a function_call with an output has no tool-result, and a
malformed stream chunk does not make the finish reason error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…ce@2.0.62

doGenerate pushes mcp_call and mcp_list_tools as provider-executed
tool-calls and tool-results without a dynamic flag
(huggingface-responses-language-model.ts:264-301). aimux set
dynamic: true on all four parts. Without a local tool of that name, Core
now returns the call marked invalid, as parseToolCall does; the old test
of that path follows.

Tests: huggingface::tests::responses::responses_language_model::hugging_face_responses_language_model::do_generate::basic_text_response::should_handle_mcp_tools_with_annotations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…ggingface@2.0.62

doGenerate pushes a tool-result after a function_call or mcp_call when
part.output is truthy, with the output string as the result
(huggingface-responses-language-model.ts:273, :312). aimux dropped the
function_call output and, for mcp_call, also gave a tool-result for an
empty string. Both now follow `if (part.output)`, and so does the
stream's function_call. The old function_call test asserted the dropped
output and follows.

Tests: huggingface::tests::responses::responses_language_model::hugging_face_responses_language_model::tool_calls::should_handle_function_call_tool_responses.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…sdk/huggingface@2.0.62

createHuggingFaceResponsesStreamError (huggingface-responses-language-model.ts:562-590)
reads only `error` and `response.failed` events: the details are
response.error for response.failed, else the event's error object or the
event itself, and the message must be a string. The transform then sets
finishReason { unified: 'error', raw: code or type } (:406-419); a chunk
that fails to parse sets { unified: 'error' } (:396-404).

aimux read only a top-level `error` field, so an `error` event with the
message on the event and every `response.failed` event went unnoticed,
and the finish reason stayed other. A first-event error is still returned
by do_stream (registered SSE pre-read), so the mid-stream case gets its
own aimux test.

Tests: huggingface::tests::responses::responses_language_model::hugging_face_responses_language_model::do_stream::should_handle_streaming_errors,
reads_mid_stream_error_events (tests/huggingface_responses_test.rs).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
[huggingface] maps the 12 source files of @ai-sdk/huggingface@2.0.62: 10
mapped, settings.ts and version.ts not-applicable. Four test cases are
excluded by hand: the first-event stream error is the registered SSE
pre-read; the NoSuchModelError and UnsupportedFunctionalityError message
texts come from aimux-core, which #268 changes, and are tracked in #276.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
cunninghamcard-bit and others added 3 commits October 20, 2026 06:46
…ce@2.0.62

doGenerate pushes mcp_call and mcp_list_tools as provider-executed
tool-calls and tool-results without a dynamic flag
(huggingface-responses-language-model.ts:264-301). aimux set
dynamic: true on all four parts. Without a local tool of that name, Core
now returns the call marked invalid, as parseToolCall does.

Tests: should_handle_mcp_tools_with_annotations
(tests/huggingface_responses_test.rs) now expects the invalid call.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…ggingface@2.0.62

doGenerate pushes a tool-result after a function_call or mcp_call when
part.output is truthy, with the output string as the result
(huggingface-responses-language-model.ts:273, :312). aimux dropped the
function_call output and, for mcp_call, also gave a tool-result for an
empty string. Both now follow `if (part.output)`, and so does the
stream's function_call (:489).

Tests: should_handle_function_call_tool_responses now expects the
tool-result; mcp_call_with_empty_output_has_no_tool_result is new
(tests/huggingface_responses_test.rs).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…sdk/huggingface@2.0.62

createHuggingFaceResponsesStreamError (huggingface-responses-language-model.ts:562-590)
reads only `error` and `response.failed` events: the details are
response.error for response.failed, else the event's error object or the
event itself, and the message must be a string. The transform then sets
finishReason { unified: 'error', raw: code or type } (:406-419); a chunk
that fails to parse sets { unified: 'error' } (:396-404).

aimux read only a top-level `error` field, so an `error` event with the
message on the event and every `response.failed` event went unnoticed,
and the finish reason stayed other. A first-event error is still returned
by do_stream (registered SSE pre-read).

Tests: reads_mid_stream_error_events, malformed_chunk_ends_with_an_error_finish
(tests/huggingface_responses_test.rs).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
The three behaviour fixes now live in their own PRs (#305, #306, #307);
this branch stacks on the last one. responses.rs and CHANGELOG.md are the
same on both sides; the old test file keeps the markers and takes the
two aimux tests the fix PRs added.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit changed the base branch from master to fix/hugging-face-stream-errors-end-with-an-error-finish October 20, 2026 06:54
#268 kept the NoSuchModel and UnsupportedFunctionality messages, so the
three cases stay sub-feature; the reasons no longer point at it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
cunninghamcard-bit and others added 2 commits October 24, 2026 09:48
…nish

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…st/port-the-upstream-hugging-face-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 10:40
…nish

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hugging Face test port:34 个 marker + 4 个 layout exclusion 覆盖 38 个上游 case,fixtures 一致,无生产行为变化。Approve。

…nd-with-an-error-finish' into test/port-the-upstream-hugging-face-tests
…ream-errors-end-with-an-error-finish

# Conflicts:
#	CHANGELOG.md
…' into test/port-the-upstream-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml
…ream-errors-end-with-an-error-finish

# Conflicts:
#	CHANGELOG.md
…' into test/port-the-upstream-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml
…ream-errors-end-with-an-error-finish

# Conflicts:
#	CHANGELOG.md
…' into test/port-the-upstream-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml
@cunninghamcard-bit
cunninghamcard-bit changed the base branch from fix/hugging-face-stream-errors-end-with-an-error-finish to master October 25, 2026 20:31
cunninghamcard-bit and others added 3 commits October 25, 2026 20:32
…eam-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase is content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; tests and CI unchanged. Re-approving current head.

…eam-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; CI 16/16. Re-approving current head.

…eam-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-hugging-face-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.

@eric8810
eric8810 merged commit 56c161d into master Oct 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants