Skip to content

test: port the upstream Anthropic AWS tests - #319

Merged
eric8810 merged 12 commits into
masterfrom
test/port-the-upstream-anthropic-aws-tests
Oct 27, 2026
Merged

eric8810 merged 12 commits into
masterfrom
test/port-the-upstream-anthropic-aws-tests

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

按 RFC-0045(#281)把 @ai-sdk/anthropic-aws@2.0.63 的上游测试迁完,按厂商分批的第七个(Mistral #295、Cohere #296、DeepSeek #299、Groq #301、Hugging Face #303、xAI #312)。

上游 3 个测试文件共 53 个用例:

  • 46 个有对应测试并通过,全部是新写的。原有的 aimux 测试(tests/anthropic_aws_settings_test.rs 等)测的是 aimux 自己的设置规则,和上游用例不一一对应,没有加标记。
  • 7 个写进 layout.toml 的手写排除:
    • 3 个 registered:aimux 也给 GET 和无 body 的请求签名(对齐文档 §9.1 已知差距,list_models 要用);创建模型时不读 region,缺 region 在第一次请求时报错(§5「不可失败的默认 provider 访问」、§3.3 Anthropic-AWS 行)。
    • 2 个 sub-feature:files()、skills(),AWS provider 没有(Provider 构造与配置:迁移后遗留缺口 #275)。
    • 2 个 ts-only:JS 对象 body 被 JSON.stringify;用 new 调 provider 函数。

迁移中发现 5 处行为差异,各自单独一个 fix PR,叠成一串,本 PR 叠在最后一个上:

另有两处没有当作这个包的差异修,标记照留,在用例上写明:

  • NoSuchModel 的文案。上游是 No such embeddingModel: …,aimux-core 统一是 no such model: …。这是 core 的差异,随 core 的测试迁移处理。这 2 个用例比对的是变体和 model_type。
  • call original fetch if init is undefined 这个用例。它的请求是 GET,aimux 会签名(同上面的登记差异),所以只比对请求原样到达原 transport,不比对未签名的 headers。

位置按新定的布局:新测试全放 src/anthropic_aws/tests/,一个上游测试文件对应一个 Rust 文件。

一、体系概览:Anthropic AWS 的测试现在怎么跑(master b91829af)

cargo test -p aimux-providers
  ├─ unittests src/lib.rs                                     anthropic_aws 一个都没有
  └─ tests/*.rs
       ├─ anthropic_aws_settings_test.rs        4 个   region / workspace id / 默认鉴权 / region 覆盖凭证里的 region
       ├─ sigv4_signed_headers_test.rs          2 个   anthropic_aws 的 SignedHeaders
       └─ bedrock_user_agent_test.rs            1 处   anthropic_aws 不是 Bedrock
            create_anthropic_aws(AnthropicAwsProviderSettings { .. })        anthropic_aws/mod.rs:139
            ├ AnthropicAwsProvider::call / messages / Provider::language_model   :242 / :252
            │ └ AnthropicMessagesModel(共享的 Anthropic Messages 模型)
            │   ├ AwsHeaders::resolve           anthropic-version、workspace id、x-api-key   anthropic_aws/model.rs:191
            │   │ └ Auth::environment_credentials / credentials                             :106 / :118
            │   └ AnthropicAwsFetch::fetch      加 user-agent,没有 x-api-key 就签名        :216
            │     └ SigV4Fetch::fetch                                                       aimux-provider-utils/src/sigv4_fetch.rs:422
            ├ Provider::embedding_model / image_model → NoSuchModel                       mod.rs:256
            └ ProviderDiscovery::list_models    GET {base_url}/models,同样签名             :268

上游 packages/anthropic-aws/src/ 对应:

anthropic-aws-provider.test.ts    25   createAnthropicAws(...)('model').doGenerate / doStream,vi.mock('aws4fetch')
anthropic-aws-fetch.test.ts       25   直接调 createSigV4FetchFunction / createApiKeyFetchFunction,mock 签名器和时钟
region-validation.test.ts          3   createAnthropicAws({ region })('m').doGenerate,fetch 抛 'captured'

改动前 aimux 没有一个测试和这 53 个上游用例一一对应。

二、本 PR 的改动(细节)

1. 新测试目录(anthropic_aws/mod.rs 末尾 #[cfg(test)] mod tests;)

cargo test -p aimux-providers --lib anthropic_aws::                    46 个
  src/anthropic_aws/tests/mod.rs        mod fetch; mod provider; mod region_validation;
  fetch.rs                22 个   mod create_sig_v4_fetch_function / mod create_api_key_fetch_function
    sigv4_fetch(credentials, inner) / api_key_fetch → model::authenticated(auth, None, Some(inner)).1      :75 / :86
    改 API key 的用例走 create_anthropic_aws + list_models(key 在 provider headers 里加,不在 transport 里)
    默认 transport 的 2 个用例:FakeProxy + set_default_transport,#[serial],drop 时还原     :243 / :312
  provider.rs             21 个   create_anthropic_aws → call(..).do_generate / do_stream,MockFetch
  region_validation.rs     3 个   mod anthropic_aws_region_validation(上游 describe 是 `${name} region validation`)
    CapturingFetch:记 URL 后报错,对应上游的 fetch 抛 'captured'                       :49
  src/test_support/mod.rs              #[path = "../../tests/common/mock_fetch.rs"] mod mock_fetch;   lib.rs 末尾 #[cfg(test)] mod test_support;
scripts/check_provider_boundaries.sh   规则 7c、8b 的 find 跳过 */tests/*;规则 6 照常扫 tests/,只滤掉 src/anthropic_aws/tests/provider.rs 里上游的 workspace id 值 "wrkspc_from_env"
                                       (上游期望的 header 值 "wrkspc_from_env" 是测试数据)

2. aimux-providers/layout.toml(只有 [anthropic_aws])

src/anthropic-aws-fetch.ts       mapped → anthropic_aws/model.rs
  registered 2:bypass signing for non-POST / POST with no body       ref = AWS SigV4 fetch 也给 GET 请求签名(§9.1 已知差距)
  ts-only 1:handle non-string body by stringifying it
src/anthropic-aws-provider.ts    mapped → anthropic_aws/mod.rs
  registered 1:region is not resolvable at model creation time      ref = 不可失败的默认 provider 访问
  sub-feature 2:files()、skills()                                     issue = 275
  ts-only 1:provider function is called with new
src/index.ts                     mapped → anthropic_aws/mod.rs
src/version.ts                   not-applicable

harness 上的差别(期望值都没改):

  • 上游 mock 了 aws4fetch 和时钟,这里 SigV4 真签,authorization 只比格式和 credential scope,x-amz-date 只比格式和时间范围。
  • user-agent 比 ai-sdk-anthropic-aws/2.0.63(钉住的版本),不比上游 mock 的 0.0.0-test testenv。
  • JS 的 Request 和 init 两层合成一个 FetchRequest。Uint8Array、ArrayBuffer 都是 bytes。cache: 'no-cache' 没有对应字段,不比。
  • InvalidArgument 只带 message,argument: 'region' 改比变体、message 和 wire 上的 AI_InvalidArgumentError。
  • 凭证回调失败的用例只比 message。aimux 的错误仍是 ApiCall,前面带 failed to resolve AWS credentials: (见 fix: Anthropic AWS wraps credential provider failures #317 的说明)。

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change
  • New provider
  • New / updated binding
  • Docs / RFC
  • Tests(上游测试迁移)

Checklist

  • cargo +1.99 fmt -p aimux-providers -- --check passes
  • cargo +1.99 clippy -p aimux-providers --lib --tests -- -D warnings passes
  • cargo +1.99 test -p aimux-providers:lib 181(其中 anthropic_aws 46),129 个测试二进制共 2405 个,全过
  • 评审后(head 1a532dc1):fmt、clippy、边界检查通过,cargo test -p aimux-providers --lib 加 anthropic_aws* 测试程序共 2 个、198 个测试全过;anthropic_aws 的 lib 测试 47 个(46 个上游 + 1 个 aimux 自己的 SigV4 list_models 测试)
  • scripts/check_provider_boundaries.sh passes
  • Commit messages follow Conventional Commits

Notes for reviewers


Generated by Claude Code

cunninghamcard-bit and others added 2 commits October 20, 2026 09:13
Ports @ai-sdk/anthropic-aws 2.0.63 src/anthropic-aws-fetch.test.ts,
anthropic-aws-provider.test.ts and region-validation.test.ts into
src/anthropic_aws/tests/, one Rust file per upstream file: 46 of the 53
cases, each with its upstream marker. The other 7 are listed in
layout.toml. Expectations are upstream's; the harness differs where Rust
has no counterpart (real SigV4 signing instead of the mocked aws4fetch,
the pinned user-agent version).

The boundary check skips tests/ directories for the Anthropic family too,
since upstream's expected header value "wrkspc_from_env" is test data.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
…est/port-the-upstream-anthropic-aws-tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5
@cunninghamcard-bit
cunninghamcard-bit marked this pull request as ready for review October 24, 2026 11:02

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anthropic AWS test port:46 tests + 7 exclusions = 53,fixtures/断言精确,无生产变化。建议收窄 tests/ 的 boundary exclusion,并补一个 SigV4 list_models 断言。Approve。

… list_models

The boundary rule for builder-era Anthropic state scanned no tests/
directory at all; it now scans them and skips only the upstream
workspace-id value "wrkspc_from_env" in src/anthropic_aws/tests/provider.rs.

A new aimux-own test checks that list_models with SigV4 auth sends one
signed GET of {base_url}/models without an API key.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RoxcKU4zBD64Lri3e4Lrq5

cunninghamcard-bit commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor Author
eric 的意见(原话) 处理
建议收窄 tests/ 的 boundary exclusion 已收窄(1a532dc1)。规则 6 原来用 --exclude-dir=tests 跳过整个 tests 目录,现在 tests 目录照常扫描。被拦下的只有一处:src/anthropic_aws/tests/provider.rs 里上游的 workspace id 测试值 "wrkspc_from_env"。它带 from_env 字样,但只是数据,不是调用,所以只按文件名加这个字符串把它滤掉。规则的正则没有改。
并补一个 SigV4 list_models 断言。 已补,在 src/anthropic_aws/tests/fetch.rs 末尾新增 list_models_with_sigv4_auth_sends_a_signed_get。它是 aimux 自己的测试,没有上游标记。测试在 SigV4 鉴权下调用 list_models,断言只发了一次 GET {base_url}/models、authorization 是这个凭据的 SigV4 签名(service 为 aws-external-anthropic)、x-amz-date 是签名时间、带 x-amz-security-token、不带 x-api-key、带 user agent,并且响应被解析成模型列表。lib + anthropic_aws* 共 198 个测试全过,fmt、clippy 和边界检查也都通过。PR 描述已同步。

Generated by Claude Code

cunninghamcard-bit and others added 6 commits October 25, 2026 09:10
…efault-fetch-per-request' into test/port-the-upstream-anthropic-aws-tests
…t' into test/port-the-upstream-anthropic-aws-tests

# Conflicts:
#	aimux-providers/layout.toml
…t' into test/port-the-upstream-anthropic-aws-tests

# Conflicts:
#	aimux-providers/layout.toml
…t' into test/port-the-upstream-anthropic-aws-tests

# Conflicts:
#	aimux-providers/layout.toml
…t' into test/port-the-upstream-anthropic-aws-tests

# Conflicts:
#	aimux-providers/layout.toml
…t' into test/port-the-upstream-anthropic-aws-tests

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the post-approval commits on this head: only additive tests/docs or tightened boundary checks; no weakening or behavior regression. Re-approving the current head.

…t' into test/port-the-upstream-anthropic-aws-tests

# Conflicts:
#	aimux-providers/layout.toml
…t' into test/port-the-upstream-anthropic-aws-tests

# Conflicts:
#	aimux-providers/layout.toml
@eric8810
eric8810 changed the base branch from fix/anthropic-aws-resolves-the-default-fetch-per-request to master October 27, 2026 07:50
@eric8810
eric8810 merged commit 16be188 into master Oct 27, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants