Skip to content

test: port the upstream Azure tests - #324

Merged
eric8810 merged 19 commits into
masterfrom
test/port-the-upstream-azure-tests
Oct 27, 2026
Merged

eric8810 merged 19 commits into
masterfrom
test/port-the-upstream-azure-tests

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/azure@4.0.84 的上游测试迁完,按厂商分批的第八个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303、xAI #312、Anthropic AWS #319)。

上游 2 个测试文件共 97 个用例:

  • 88 个有对应测试并通过,全部是新写的:provider 62 个、转写 26 个。原有的 tests/azure_upstream_e2e_test.rs 12 个测试里有 9 个带 /// TS: "…" 出处,但只比部分字段,出处改成 /// partial: … 注释,指向完整用例所在的文件(单独一个提交,函数体不变)。
  • 9 个写进 layout.toml 的手写排除:
    • 8 个 sub-feature(Provider 构造与配置:迁移后遗留缺口 #275):Azure 上的 DeepSeek 模型 4 个、completion() 2 个,aimux 都没移植;embedding 的单次输入字节上限 1 个,aimux 的 EmbeddingModel 只有 max_embeddings_per_call();转写的 workflow 序列化往返 1 个,aimux 的模型不能序列化。
    • 1 个 ts-only:用 vi.spyOn 替换 OpenAITranscriptionModel.prototype.doStream,Rust 模型没有可替换的接缝。

Azure 复用 OpenAI 的模型代码,迁移中发现的 3 处差异都在 OpenAI 共用代码里,原先各自单独一个 fix PR(#323 合并了三处),现在三处都已随 OpenAI 的 fix PR #368 合入 master,#323 已关闭:

位置按新定的布局:新测试全放 src/azure/tests/,一个上游测试文件对应一个 Rust 文件。

一、体系概览:Azure 的测试现在怎么跑(master b91829af)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                                       azure 一个都没有
  └─ tests/*.rs
       ├─ azure_upstream_e2e_test.rs            12 个   出处写成 /// TS: "…",不带 describe 链,只比部分字段
       └─ package_user_agent_test.rs             1 处   ai-sdk-azure/4.0.84
            create_azure(AzureOpenAIProviderSettings { .. })                     azure/mod.rs:177
            ├ AzureOpenAIProvider::chat / responses                             :457 / :469
            │ └ crate::openai 的 Chat / Responses 模型,provider 名 azure.chat / azure.responses
            │   └ OpenAIResponsesLanguageModel → ResponsesStreamTransformer     openai/responses/responses_convert.rs
            │       ├ "response.reasoning_summary_text.delta"                   :1059(#323 改)
            │       └ flush → providerMetadata.azure.responseId                 :1517(#322 改)
            ├ AzureOpenAIProvider::embedding / image / speech                   :482 / :492 / :522   OpenAI 模型
            ├ AzureOpenAIProvider::transcription → AzureTranscriptionModel      :503
            │ └ do_generate:按 providerOptions.azure.api 选 OpenAI 转写或 Azure Speech   azure/transcription.rs:172
            │   do_stream:Speech 直接拒绝,OpenAI 转交                          :364
            ├ AzureOpenAIProvider::call / Provider::language_model              :534 / :544
            └ tools:crate::openai::tools::AzureOpenAITools                     openai/tools.rs:766
                └ file_search(FileSearchArgs { max_num_results: Option<f64> })  :199(#321 改)

上游 packages/azure/src/ 对应:

azure-openai-provider.test.ts       69   createAzure(...).responses / chat / embedding / image / transcription / speech,createTestServer
azure-transcription-model.test.ts   28   AzureTranscriptionModel(定义在 azure-openai-provider.ts),mock fetch + 录制的 MAI-Transcribe-2 响应

改动前 aimux 没有一个测试带上游的完整 describe 链。

二、本 PR 的改动(细节)

1. 新测试目录(azure/mod.rs 末尾 #[cfg(test)] mod tests;)

cargo test -p aimux-providers --lib azure::                            88 个
  src/azure/tests/mod.rs             mod openai_provider; mod transcription_model;
  openai_provider.rs        62 个   mod responses_default_language_model / chat / transcription / speech /
                                    embedding / image / responses / resource_name_validation
    provider(fetch) / azure_with(..) → create_azure(..)                                   :99 / :87
    responses › do_generate / do_stream:fixtures/*.json、*.chunks.txt 回放,
      generate_json / stream_json 与 snapshots/*.json 比(上游 __snapshots__ 转成 JSON)   :283 / :298
      renumber_source_ids:source id 按出现顺序改成 id-0、id-1,对应上游的 mockId      :257
    file_search_tool → provider.tools.file_search(FileSearchArgs { max_num_results: Some(5), .. })   :323
  transcription_model.rs    26 个   mod api_routing / speech_requests / speech_responses /
                                    streaming_and_serialization / recorded_mai_transcribe_2_responses / provider
    setup(settings, body) → RouteFetch 按 URL 分 OpenAI 转写和 Speech                      :110
  fixtures/、snapshots/              上游 __fixtures__ 原样拷贝;快照每个 toMatchSnapshot 一个文件
  src/test_support/mod.rs            #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;
scripts/check_provider_boundaries.sh 规则 7c、8b 的 find 跳过 */tests/*(同 #[cfg(test)] 块)

2. aimux-providers/layout.toml(只有 [azure])

src/azure-openai-provider.ts                   mapped → azure/mod.rs
  sub-feature 8,issue = 275:deepseek ×4、completion ×2、embedding 单次输入上限、workflow 序列化
  ts-only 1:delegates OpenAI streaming without changing call options
  (azure-transcription-model.test.ts 没有同名源文件,它测的 AzureTranscriptionModel 在这个文件里,排除也列在这里)
src/azure-openai-tools.ts                      elsewhere → openai/tools.rs
src/azure-openai-provider-metadata.ts          mapped → azure/options.rs
src/azure-speech-transcription-model{,-options}.ts、azure-transcription-model-options.ts、
  azure-transcription-provider-metadata.ts     mapped → azure/transcription.rs
src/index.ts                                   mapped → azure/mod.rs
src/version.ts                                 not-applicable

3. tests/azure_upstream_e2e_test.rs:9 个测试上的 11 行 /// TS: "…" 改成 /// partial: …; full port in src/azure/tests/<file>.rs。

harness 上的差别(期望值都没改):

  • 上游的 createModel 自己拼 URL 和 headers 建 OpenAIResponsesLanguageModel,这里用 provider.responses(..),差别只在这些用例不比的 URL 和鉴权头。
  • user-agent 比 ai-sdk-azure/4.0.84(钉住的版本),不比上游 mock 的 0.0.0-test;比对发出的 headers 时去掉 user-agent,单独比。
  • InvalidArgument 只带 message,argument: 'resourceName' 改比变体和 message,并确认没有发请求。
  • 中止用例比 AiMuxError::Aborted,不比同一个错误对象;AbortSignal 的同一性改为取消调用方的 signal、看 transport 收到的那份是否跟着取消。
  • 快照里值为 undefined 的键在 JSON 里省略,Date 写成 ISO 字符串。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt -p aimux-providers -- --check passes
  • cargo +1.99 clippy -p aimux-providers --lib --tests -- -D warnings passes
  • cargo +1.99 test -p aimux-providers:lib 223(其中 azure 88),129 个测试二进制全过
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers


Generated by Claude Code

cunninghamcard-bit and others added 5 commits October 20, 2026 09:38
@ai-sdk/openai 4.0.83 takes `maxNumResults` as a JS number and sends it as
is (openai-responses-prepare-tools.ts:219), so `5` reaches the API's integer
field as `5`. aimux typed the field `Option<f64>`, so the typed factory sent
`5.0`. It is an `Option<u32>` now; the other number fields of the tool
arguments that count things (`partial_images`, `output_compression`) are integers already.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
… never started

@ai-sdk/openai 4.0.83 starts the stream's `responseId` at `null`
(openai-responses-language-model.ts:1587) and only `response.created` sets
it, so a stream without that event finishes with `responseId: null`. aimux
wrote an empty string. The generate path already writes `null`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@ai-sdk/openai 4.0.83 puts the reasoning item's id on every
`reasoning-delta` as `providerMetadata.<key>.itemId`
(openai-responses-language-model.ts:2735-2749). aimux sent the deltas
without provider metadata, while its reasoning start and end parts
already carried it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Ports the 97 cases of @ai-sdk/azure 4.0.84 (azure-openai-provider.test.ts,
azure-transcription-model.test.ts) into src/azure/tests/, one Rust file per
upstream test file, each case under a `/// upstream:` marker. 88 are ported
and pass on the three OpenAI fixes below this commit; 9 are listed in
layout.toml (8 sub-feature, 1 ts-only).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
The `/// TS: "…"` lines in azure_upstream_e2e_test.rs named upstream cases
these tests only check in part. They now say `partial:` and point at the
full ports in src/azure/tests/. Test bodies are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5

Copy link
Copy Markdown
Contributor Author

说明里提到的「测试文件没有同名源文件」,check.mjs 该怎么处理,已记到 #278(https://github.com/arcships/aimux/issues/278#issuecomment-6288575903)。本 PR 不变。


Generated by Claude Code

cunninghamcard-bit and others added 2 commits October 24, 2026 09:48
…m-id

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…st/port-the-upstream-azure-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 11:31

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Azure test port:88 tests + 9 exclusions = 97,16 个 snapshots/18 个 fixtures 全部被引用且精确比较,无生产变化。Approve。

@cunninghamcard-bit
cunninghamcard-bit changed the base branch from fix/openai-responses-reasoning-deltas-carry-the-item-id to master October 25, 2026 11:16
cunninghamcard-bit and others added 6 commits October 25, 2026 11:24
…eam-azure-tests

# Conflicts:
#	CHANGELOG.md
#	aimux-providers/layout.toml
#	aimux-providers/tests/azure_upstream_e2e_test.rs
#	aimux-providers/tests/openai_upstream_e2e_test.rs
…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml
…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase is content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; tests and CI unchanged. Re-approving current head.

…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and CHANGELOG/layout conflict resolution; CI 16/16. Re-approving current head.

…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master/parent merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master merges and conflict resolution; CI 16/16. Re-approving current head.

…eam-azure-tests

# Conflicts:
#	aimux-providers/layout.toml

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase verified content-preserving: only master merges and layout.toml conflict resolution; CI 16/16. Re-approving current head.

@eric8810
eric8810 merged commit 5911444 into master Oct 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants