Skip to content

test: port the upstream xAI tests - #312

Merged
eric8810 merged 23 commits into
masterfrom
test/port-the-upstream-xai-tests
Oct 27, 2026
Merged

eric8810 merged 23 commits into
masterfrom
test/port-the-upstream-xai-tests

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/xai@5.0.12 的上游测试迁完,按厂商分批的第六个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303)。

上游 15 个测试文件共 367 个用例:

迁移中发现两处行为差异,各自单独一个 fix PR,都已合入 master:

另有一处只是错误类型不同,没有当作差异修:流里的 error、response.failed 事件,上游产出 createProviderStreamError 的对象(带 type、code),aimux 产出 AI_APICallError(providerCode 存上游的 code,type 留在 data 里)。message、statusCode、isRetryable、data 都和上游一致。这 3 个用例经一个视图函数改名后比对,期望值没改。

位置按新定的布局:xAI 的新测试全放 src/xai/tests/,一个上游测试文件对应一个 Rust 文件。原有的 tests/xai_*_test.rs 留在原地,由包拆分线程的搬移 PR 合进来。

一、体系概览:xAI 的测试现在怎么跑(master b91829af)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                                   xAI 一个都没有
  └─ tests/*.rs
       ├─ xai_speech_test.rs           17 个 ┐
       ├─ xai_transcription_test.rs    18 个 ├ 出处写成 /// TS: "…" 或文件头注释,不带 describe 链
       ├─ xai_video_test.rs            34 个 │
       └─ xai_upstream_e2e_test.rs      5 个 ┘ 只比部分字段
            create_xai(XAIProviderSettings { .. })                  xai/mod.rs:319
            ├ XAIProvider::responses / call                         xai/mod.rs:385 / :419
            │ └ XaiResponsesModel::do_generate / do_stream          xai/responses/mod.rs:216 / :506
            │   ├ build_responses_request_body                      responses/convert.rs:717
            │   │ ├ convert_to_xai_responses_input                  responses/convert.rs:425
            │   │ └ prepare_responses_tools                         responses/convert.rs:99
            │   ├ convert_xai_responses_usage                       responses/convert.rs:43
            │   └ transform_event → "error" / "response.failed"     responses/mod.rs:872 / :886
            │       └ xai_stream_error → stream_error_api_call      xai/mod.rs:83   产出 AI_APICallError
            ├ XAIProvider::speech        → XaiSpeechModel::do_generate          xai/speech.rs:157
            ├ XAIProvider::transcription → XaiTranscriptionModel::do_generate   xai/transcription.rs:134
            │                              └ do_stream(WebSocket)             xai/transcription.rs:210(#310、#311 改)
            ├ XAIProvider::video         → XaiVideoModel::do_start / do_status  xai/video.rs:699 / :729
            └ Provider::image_model                                 xai/mod.rs:437   → NoSuchModel

上游 packages/xai/src/ 对应:

responses/xai-responses-language-model.test.ts   101   createTestServer → provider.responses(...).doGenerate / doStream
  createXaiResponsesStreamError                         xai-responses-language-model.ts:57-81 → createProviderStreamError
xai-video-model.test.ts                           76   createTestServer → XaiVideoModel.doStart / doStatus
responses/xai-responses-prepare-tools.test.ts     35   直接调 prepareResponsesTools
files/xai-files.test.ts                           25   XaiFiles(aimux 没有)
xai-image-model.test.ts                           23   XaiImageModel(aimux 没有)
responses/convert-to-xai-responses-input.test.ts  22   直接调 convertToXaiResponsesInput
xai-transcription-model.test.ts                   20   createTestServer / MockWebSocket → doGenerate / doStream
xai-speech-model.test.ts                          17   createTestServer → XaiSpeechModel.doGenerate
xai-provider.test.ts                              16   createXai → 各 model 方法
xai-batch.test.ts                                 16   Batch(本期不做)
responses/convert-xai-responses-usage.test.ts      6   直接调 convertXaiResponsesUsage
其余 4 个文件                                      10   raw usage 3、error 3、supports-reasoning-effort 2、realtime 2

改动前 74 个原有测试里只有 36 个能和上游用例一一对应,出处只写了 it 文本,没有 describe 链。

二、本 PR 的改动(细节)

1. 原有测试补标记(tests/xai_speech_test.rs、xai_transcription_test.rs、xai_video_test.rs、xai_upstream_e2e_test.rs)

mod xai_speech_model › mod do_generate                         xai_speech_test.rs          15 个标记
mod do_generate / mod do_stream                                xai_transcription_test.rs    9 个标记
mod xai_video_model › mod do_start / mod do_status             xai_video_test.rs           12 个标记
  /// upstream: @ai-sdk/xai src/xai-video-model.test.ts › XaiVideoModel › doStatus › should return pending when status is pending
xai_upstream_e2e_test.rs                                       5 个都只比部分字段,改成 /// partial: … 注释

一一对应的加标记、按 describe 包 mod;只覆盖了部分断言或换了输入的不加标记,注释写明完整用例在 src/xai/tests/ 的哪个文件。函数体不变。#310、#311 各加的 1 个 aimux 测试在 xai_transcription_test.rs 末尾。

2. 剩下的上游用例(新目录,xai/mod.rs 末尾 #[cfg(test)] mod tests;)

cargo test -p aimux-providers --lib xai::                            262 个
  src/xai/tests/mod.rs                       mod error; mod provider; mod responses; mod speech_model; …
  src/xai/tests/responses/
    responses_language_model.rs            101 个   公开 API:create_xai → responses(..).do_generate / do_stream
      stream_fetch(events) → run_stream → 与上游 toMatchInlineSnapshot / snapshots/*.json 比      :201 / :230
      stream_provider_error_view(part)                                                        :263
        AI_APICallError 的 providerCode → code、data.type → type,其余字段原样,期望值不动
    responses_prepare_tools.rs              35 个   prepare_responses_tools(..)
    convert_to_responses_input.rs           22 个   convert_to_xai_responses_input(..)
    convert_responses_usage.rs               6 个   convert_xai_responses_usage(..)
    responses_raw_usage.rs                   3 个
    fixtures/ snapshots/                            上游 __fixtures__、__snapshots__ 原样拷贝
  src/xai/tests/video_model.rs              64 个   其中 expired 用例只比 error 文本,不带标记(sub-feature)
  src/xai/tests/provider.rs                 13 个   #[serial],读 XAI_API_KEY
  src/xai/tests/transcription_model.rs      11 个   MockConnector / MockSocket 代替 MockWebSocket   :343 / :279
  src/xai/tests/error.rs、speech_model.rs、supports_reasoning_effort.rs   3 / 2 / 2 个
  src/test_support/mod.rs                    #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;
scripts/check_provider_boundaries.sh         规则 7c、8b 的 find 跳过 */tests/*(同 #[cfg(test)] 块)

3. aimux-providers/layout.toml(只有 [xai])

42 个上游源文件:19 个 mapped(rust 是现在的源文件名)
  xai-provider.ts             sub-feature 3 条,issue = 275:imageModel
  xai-video-model.ts          sub-feature 1 条,issue = 276:错误状态不带 response
files/*(3)、tool/*(9)、xai-image-*(3)          missing,issue = 275
realtime/*(4)、xai-batch.ts、map-xai-finish-reason.ts(只给 batch 用)   not-applicable,对齐文档本期范围外
version.ts、xai-video-settings.ts                    not-applicable

harness 上的差别(期望值都没改):快照里值为 undefined 的键在 JSON 里省略;上游靠 _internal.currentDate 注入固定日期,Rust 侧没有这个入口,改成检查时间戳落在调用前后之间;undici 的 multipart boundary 只比 multipart/form-data; boundary= 前缀;user-agent 比 ai-sdk-xai/5.0.12;result.stream.cancel() 换成丢弃流,await flush() 换成等关闭帧发出。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt -p aimux-providers -- --check passes
  • cargo +1.99 clippy -p aimux-providers --lib --tests -- -D warnings passes
  • cargo +1.99 test -p aimux-providers:lib 397(其中 xai 262)、xai_speech 17、xai_transcription 20、xai_video 34、xai_upstream_e2e 5,以及用到 xAI 的 cassette_full、cassette_multimodal、conformance、data_loss_regression、openai_compatible、package_user_agent、vertex_upstream_e2e,全过
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers


Generated by Claude Code

cunninghamcard-bit and others added 7 commits October 20, 2026 08:12
@ai-sdk/xai 5.0.12 opens the STT WebSocket inside the returned stream
(xai-transcription-model.ts:276-358), so doStream resolves and a connect
failure errors the stream, cancelling the caller's audio. aimux connected
before returning and failed do_stream itself. The connect now runs first
inside the stream; on failure the stream yields the error and ends, which
drops the audio.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
In @ai-sdk/xai 5.0.12, cancelling the transcription stream runs cleanup,
which calls connection.close() (xai-transcription-model.ts:290-298,
:459-462). aimux only sent a close frame after transcript.done or an error
event; a stream dropped earlier released the transport without one. The
socket now lives in a guard that, when dropped before it was closed, sends
the close frame on the runtime.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Upstream's ReadableStream start runs when doStream builds the stream, so
the WebSocket is opened before doStream resolves even if the stream is
never read. Connecting on the stream's first poll would skip the
connection for a stream dropped unread; connect in do_stream instead and
hand the outcome to the stream, which still yields a failure as its
error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…/xai-stt-closes-the-socket-when-the-stream-is-dropped

# Conflicts:
#	aimux-providers/src/xai/transcription.rs
Tests that match an @ai-sdk/xai 5.0.12 case one to one get its marker
and sit in describe-level mods; tests that cover a case only in part keep
a partial: doc line naming the full port in src/xai/tests/. Bodies are
unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
One Rust file per upstream test file under src/xai/tests/, with the
upstream fixtures and snapshots beside them. The error part of a stream
error event is compared through a view that renames providerCode to code
and takes type from the event: aimux has no AI_StreamProviderError, so the
part is an AI_APICallError.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
cunninghamcard-bit and others added 2 commits October 24, 2026 09:48
…opped

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…est/port-the-upstream-xai-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 10:40
cunninghamcard-bit and others added 2 commits October 25, 2026 08:11
…the-socket-when-the-stream-is-dropped

# Conflicts:
#	CHANGELOG.md
…en dropped

The socket guard was built in the first statement of the
async_stream::stream! body, which runs only on the first poll. A stream
that do_stream returned with an open socket but that was dropped before
next() released the transport without a close frame. The guard is now
built in do_stream, right after the connect, and moved into the stream.

The CHANGELOG said the socket is opened inside the stream; do_stream still
awaits ws_connect. The entry now says what changed: a connect failure
arrives as the stream's error.

Test: dropping_an_unpolled_stream_closes_the_socket
(tests/xai_transcription_test.rs) fails without the change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xAI test port:297 个 marker + 4 个 layout case 覆盖 367 个 in-scope case,11 个 fixtures 逐字节一致,无生产变化。Approve。

…en-the-stream-is-dropped' into test/port-the-upstream-xai-tests
@cunninghamcard-bit
cunninghamcard-bit changed the base branch from fix/xai-stt-closes-the-socket-when-the-stream-is-dropped to master October 25, 2026 11:15
cunninghamcard-bit and others added 6 commits October 25, 2026 11:18
…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase is content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; tests and CI unchanged. Re-approving current head.

…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; CI 16/16. Re-approving current head.

…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml

Copy link
Copy Markdown
Contributor Author

Flutter example (android) failed on 0ca5568 while Gradle was still configuring :app: the Android SDK manager's NDK r28c download stalled at 16% and sdkmanager exited 1 after 27 minutes, before any project code was built. This PR only changes Rust tests under aimux-providers and doesn't touch the Flutter binding or the Android toolchain. The same download failure hit #361 this hour and passed on re-run, so I've re-run the failed jobs once here too.


Generated by Claude Code


Generated by Claude Code

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-xai-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.

@eric8810
eric8810 merged commit ecd9c17 into master Oct 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants