Skip to content

feat(vertex): run Claude Code on Vertex AI through bili - #3323

Merged
ranxianglei merged 8 commits into
ranxianglei:masterfrom
aolshevskiy:master
Oct 17, 2026
Merged

ranxianglei merged 8 commits into
ranxianglei:masterfrom
aolshevskiy:master

Conversation

@aolshevskiy

@aolshevskiy aolshevskiy commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Adds GCP Vertex AI as a supported backend for the claude lane: a session launched with CLAUDE_CODE_USE_VERTEX=1 routes its Claude traffic through bili and gets the same compression as an Anthropic session — no MITM, no classifier change, no new wire.

Commits

commit scope
d6fb6af6 the wire, frozen: real client request captured + doc-derived response half, as a fixture with provenance, plus a lane-scoped fake upstream and ledger rules WC-019/020/021
93032cf3 launcher + client config: read the Vertex keys, emit ANTHROPIC_VERTEX_BASE_URL through bili, pin it in the managed install block
8b3d23b5 the proxy: accept the wire, derive the model from the path, host map, bootstrap routing check
0592ee4a contract bite: a Vertex lane row in the wire-contract suite, the path parser's unit table, mutation evidence
f943b629 fix: the managed block no longer re-wraps bili's own Vertex URL

Wire facts this rests on — measured, not inferred

  • Claude Code appends its own path to ANTHROPIC_VERTEX_BASE_URL verbatim: /v1/projects/<p>/locations/<loc>/publishers/anthropic/models/<id>:streamRawPredict. Two consequences the code depends on: the emitted value must end in /v1 (/v1/projects/…:rawPredict 429s where /projects/…:rawPredict 404s), and it must be the protocol-qualified form <bili>/bili/anthropic/https://aiplatform.googleapis.com/v1 — the bare /bili/<url> form classifies protocol = null (:rawPredict matches no suffix in the ladder) and forwards uncompressed.
  • The model is in the path, not the body. A Vertex Messages body carries anthropic_version: "vertex-2023-10-16" and no model field at all, so the id has to come off the path or the window/headroom decisions silently fall back to the floor.
  • anthropic_version is body-borne and survives the rebuild (the spread drops only prompt_cache_key); it was missing from the incremental-turn allowlist, which serves the native-extension lanes.
  • Vertex's Anthropic publisher serves no :countTokens, and a publisher body must not carry model.

Verification

  • Targeted suites green: tests/vertex.test.ts, tests/wire-contract.test.ts, tests/vertex-fake-upstream.test.ts, tests/vertex-wire-fixture.test.ts, tests/e2e-vertex-anthropic.test.ts, tests/claude-native.test.ts, tests/launcher.test.ts, tests/registry.test.ts, tests/incremental-lane.test.ts (the e2e drives the captured bytes through the real proxy against a lane-scoped fake upstream, and asserts the compression round trip).
  • The new tests are proven to bite by mutation, each reverted afterwards: dropping the vertex arm from the model resolution fails the new e2e test; making the path parser return undefined fails four; loosening the location regex fails the SSRF cases.
  • npm run typecheck, scripts/check-unused-exports.mjs, scripts/check-test-import-graph.mjs: clean.
  • Local full-suite runs on this branch fail only where current master fails too (golden storage-render ×4, #1086 T5, omp-native host, opencode-native, opencode-v2 ×3, #2443), plus #1292 runLaunch claude posix — which fails at HEAD as well when the shell exports CLAUDE_CODE_USE_VERTEX/CLOUD_ML_REGION, as this one does; CI remains the verdict per §7.2.
  • Upstream leg: the fixture's request half is a real capture; the live endpoint was used to establish the wire facts and the lane-scoped fake carries the acceptance run.

CI

The test lane's shard 3 is red on all four runners (storage-render golden ×4 and #1086 T5), and those five already fail on master at 6ee4a2ac — the run for that commit (38767006158) has 9 of 12 shards red, with storage-render golden 4 fail / 5 pass in four of them and #1086 T5 failing in three; both reproduce on a clean worktree at that commit. Every other check on this PR is green (39 passed).

Not touched

  • tests/golden/wire-contract/*.json — no regeneration, because the inventory (tests/wire-contract-inventory.ts) snapshots layer-level tool surfaces, so this lane adds no snapshot. The ledger still grew (WC-019/020/021 with provenance and their laneProbes enforcement entries).
  • kernel/, pi-subagents/, the MITM path, the suffix ladder, src/server/prepare-anthropic.ts's rebuild.

Config surface

CONFIGURATION.md and CONFIGURATION.zh-CN.md document the Vertex keys, the two variables bili writes (ANTHROPIC_VERTEX_BASE_URL, ANTHROPIC_BASE_URL) and the values they get; CHANGELOG.md carries an [Unreleased] entry.

…fake

- WC-019/020/021 join the ledger with provenance: the API version is a body field on this wire, the publisher path is the model's only source, stream errors arrive array-wrapped
- all three are lane-scoped (lane: "vertex-anthropic"), so the shared anthropic validator stays unwidened — WC-013 already forbids what WC-017 requires on another lane
- startFakeUpstream accepts { lane: "vertex-anthropic" }: its validators are written from the captured client bytes, which the first test replays verbatim, so the capture validates the fake rather than the reverse
- no rule for a body `model` field: Vertex's tolerance is unverified, and a constraint without provenance would reject legitimate traffic; a test pins the tolerance instead
- fixture pair in tests/fixtures/vertex-anthropic/: captured request byte-exact, response doc-derived and marked as such in the manifest, deliberately outside the golden wire-contract inventory
- lane probe table in tests/wire-contract.test.ts gives each new rule a live clause — the ledger gate fails without it
- tests: vertex-fake-upstream 8/8, vertex-wire-fixture 8/8, wire-contract 36/36 (97 across all fake importers)
Under CLAUDE_CODE_USE_VERTEX=1 the plain ANTHROPIC_BASE_URL is inert: Claude Code
dials <ANTHROPIC_VERTEX_BASE_URL>/projects/…:rawPredict|streamRawPredict and
appends that path to the base URL verbatim, so the launcher was rewriting a
variable nothing reads and Vertex traffic never reached bili.

- src/vertex.ts (new): vertexEnabled, vertexHostForLocation (explicit override →
  global → us/eu multi-region → <loc>-aiplatform), vertexBaseUrlForLocation
  (host + /v1, because the client appends /projects/… and Vertex 404s the
  version-less path), normalizeVertexModelId, and the region guard ported with
  the host derivation — the value is interpolated into a hostname, and a
  percent-encoded # reaches the metadata endpoint (headroom's recorded payload)
- wrapUpstream/unwrapUpstream take an optional protocol, so the emitted form is
  <origin>/bili/anthropic/<vertex host>/v1 and survives re-wrapping; the
  parameter is optional, leaving all 51 existing call sites byte-identical
- buildClientEnv resolves a rewrite per key: the claude preset gains
  ownBaseUrlKeys, because fanning baseUrlKeys[0]'s value out to a new key would
  write api.anthropic.com into the Vertex variable, while iFlow's alias pair
  keeps relying on that fan-out (regression pinned)
- discoverRoutes registers a second relayWrapRoute under ANTHROPIC_VERTEX_BASE_URL
  with protocol "anthropic" (":rawPredict" matches no suffix in the ladder, so the
  explicit marker is what classifies it); extractHttpsHosts reports the Vertex
  host on that lane instead of api.anthropic.com
- buildClaudePluginEnv, buildClaudeSettingsArg (inline payload and win32 temp
  file) and the native-install override emit both variables; the managed install
  block pins and recognises the Vertex key (isBiliClaudeBaseUrl gained the
  optional protocol segment), never clobbering a foreign value
- the plain key stays emitted and untouched (inert under Vertex, so flipping the
  provider off still works) and nothing is emitted at all with Vertex off
- docs: manual-mode Vertex subsection plus the redirect/discovery table rows in
  CONFIGURATION.md and CONFIGURATION.zh-CN.md, and a CHANGELOG [Unreleased] bullet

Tests: vertex.test.ts (host table incl. SSRF-shaped inputs, id normalisation),
launcher.test.ts (discovery, two-key resolution, iFlow regression, wrap/unwrap
round trip), launcher-plugin-mode.test.ts, claude-native.test.ts (managed block
writes and preserves the key), fix-1292-windows-claude-settings.test.ts (the
settings payload carries the second key). Full suite: 7214 tests, 7173 pass,
10 fail — all 10 reproduced at HEAD d6fb6af, none introduced here.
- read the model off the publisher path on the Vertex anthropic wire
  (`vertexAnthropicModelFromPath`), resolve it once
  (`bodyModel ?? googleModel ?? vertexPathModel ?? v1Model`) and normalise the
  `@version` tag out of the path-derived id, logging once per raw id
- thread the resolved model into `resolveKnownOutputCeiling`, `prepareAnthropic`
  and `Prepared`, which is what the relay's compress-loop rebuild reads
- map `aiplatform.googleapis.com` and both location-shaped host families to the
  `anthropic` provider
- keep `anthropic_version` in the incremental-lane parameter allowlist and its
  `src/agent/incremental-client.ts` mirror
- report `bypassed` for a Vertex session in `claudeNativeRoutingCheck`, where the
  plain `ANTHROPIC_BASE_URL` signal is inert
- tests: `tests/e2e-vertex-anthropic.test.ts` drives the captured fixture at the
  path the launcher emits (classified, compressed, Vertex fields intact, and
  verbatim passthrough without the protocol marker), plus the registry,
  native-routing and incremental-lane cases
…ake it bite

- the path parser's unit table: both method suffixes, the protocol-qualified path
  the launcher actually emits, a dated id returned raw (stripping stays
  normalizeVertexModelId's job, asserted separately), :countTokens and non-Anthropic
  publishers yielding nothing, a query or fragment tolerated, percent-encoding
  decoded, an undecodable escape falling back to the raw match instead of throwing
- Lane gains optional protocol/lane, so a contract row can carry the
  /bili/<protocol>/<url> marker and a lane-scoped fake; a Vertex row posts the
  captured 45 KB client body at the publisher path, which runs WC-019/020/021
  through the proxy for the first time
- no golden snapshot: the four committed ones are layer-level tool surfaces and this
  lane adds none, so the golden inventory is a deliberate no-op rather than a
  regenerated diff
- three mutations (the path-derived model arm, the parser, the location guard) were
  applied, observed failing named tests, and reverted
…x URL

- stripBiliWrappers (src/vertex.ts) collapses nesting of bili's own /bili/
  wrapper until the value is no longer one of ours; the single-shot
  unwrapBiliBaseUrl stays as it is (ANTHROPIC_BASE_URL's writer cannot nest).
- claudeNativeVertexBaseUrl (src/plugin-install.ts) and claudeVertexUpstream
  (src/launcher.ts) unwrap the configured value before deriving, so install and
  the SessionStart repin are idempotent: derived == current means the managed
  block writes nothing. Before this, a launch or a start added a
  /bili/anthropic/ layer each time, and the added hop targeted the proxy's own
  port — denied by the tunnel guard (403 detail:"self").
- stripClaudeManagedBlock takes ANTHROPIC_VERTEX_BASE_URL back when it is ours,
  so `bili plugin remove claude` no longer leaves the client dialing a dead
  /bili/ URL; a foreign endpoint is left alone, as at install.
- tests: stripBiliWrappers unit table (tests/vertex.test.ts); idempotent
  derivation, nested-value collapse and the installer round trip on the Vertex
  key (tests/claude-native.test.ts); discovery unwraps an already-managed base
  URL instead of nesting it (tests/launcher.test.ts).
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 open/closed issue+PR 全量检索 vertex:仅 #3167/#3159(已合并/关闭,Google native wire 的 part-order 问题,gemini 线)与本 PR 相关但机制不同——本 PR 是 claude 线新增 Anthropic-on-Vertex(AI Platform publisher) 后端,与 gemini 原生线无共享代码路径。无既有 issue 提出过 Vertex AI 支持。

开始按 PR Review 规范审查:方向 → 回归 → 代码 → 测试有效性 → diff 整洁度。外部提交按 ⑤b 硬标准从严。结论将用结构化八问给出。

…anxianglei#3323 review fixups)

Review fixups on the Vertex port, one defect class: unfilled issue references
and pointers at a provenance doc that was never committed.

- CHANGELOG: the ranxianglei#3266 bullet had been glued onto the end of the new Vertex
  bullet on one line — split it back into its own entry.
- #NNNN / (#vertex) placeholders -> ranxianglei#3323 (tests/vertex-wire-fixture.test.ts,
  tests/registry.test.ts, tests/claude-native.test.ts, src/registry.ts).
- reference/billion-context-vertex-grounding.md is not in this repo, so the
  four citations pointing at it (src/vertex.ts, tests/vertex-fake-upstream.test.ts,
  tests/wire-contract-fakes.ts WC-021 provenance, fixture README) now stop where
  the in-repo provenance (WIRE_RULES + fixture manifest) carries the record.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 完成。代码已逐文件读完(36 个文件全量 diff),定向测试本地跑绿,CI 状态核对完毕,发现的小问题已直接修掉推回分支。结论:质量达标,可合并(待 CI 的既有红清掉或 owner 判断后人工合并)。

结构化八问

  1. 是什么问题:Claude Code 开 CLAUDE_CODE_USE_VERTEX=1 时完全绕过 bili——它忽略 ANTHROPIC_BASE_URL,直连 <ANTHROPIC_VERTEX_BASE_URL>/projects/…:rawPredict|streamRawPredict,这条流量此前没有任何路由、分类和压缩。本 PR 把 Vertex 端点接入 claude 车道。
  2. 什么原因导致的:不是 bug,是能力缺口——Vertex 是 Claude Code 的第二条 base URL 变量,且路径形态(:rawPredict)不匹配现有后缀分类器,模型又不在 body 里而在路径里,所以旧机制一条都够不着。
  3. 影响范围与严重程度:新增能力,不改变任何既有用户行为(Vertex 关着时字节级不变,有测试钉死)。Vertex 用户此前是"完全无压缩",属特定条件下的功能缺失,非线上故障。
  4. 怎么修复的:复用 master 已有的 /bili/<protocol>/ 显式协议标记机制(server.ts:251 早已存在,本 PR 不加解析器,只加调用方):launcher/managed block 把 ANTHROPIC_VERTEX_BASE_URL 指到 <proxy>/bili/anthropic/<vertex origin>/v1;代理侧新增 vertexAnthropicModelFromPath(Gemini 解析器的姊妹函数,:countTokens 分支不会误匹配 Anthropic 路径),模型经单一决议点 bodyModel ?? googleModel ?? vertexPathModel ?? v1Model 进入所有按模型键控的决策;host→provider 映射补 aiplatform 三族主机;增量车道 allowlist 保留 anthropic_version。region 插值进主机名前有严格 SSRF 校验(百分号编码 # → metadata 端点这类载荷被拒)。
  5. 是否有回归问题:逐路径清点过——launcher claude 车道(Vertex 关→字节级不变,有测试)、iFlow 双别名镜像(唯一多 baseUrlKey 客户端,行为中性,有测试)、managed block 新变量沿用"外来值永不覆盖"纪律且 strip 对称、requestModel 链 body 优先所以非 Vertex anthropic 车道字节级不变。两处有意放宽,明确披露:① unwrapUpstream 现在会剥掉 /bili/ 后的已知协议段——此前这类值直接解析失败(路由静默丢失),现在能解析,属修复性放宽;② relay 压缩配置取模型时,body 无 model 字段的线(如 Google 路径模型)从"拿到 undefined"变为"拿到 resolved model"——只有当用户配了按模型覆盖压缩参数时结果才不同,且新值更正确。高危模式逐项过:无语义漂移、无 fail-fast 变吞错、无阈值/时序/输出格式变化。
  6. 是否破坏缓存:不涉及。会话身份、折叠账本、前缀缓存路径均未触碰;Vertex body 重建保真(anthropic_version/output_config/safeguards 存活,e2e 钉死)。
  7. 是否改了设计方向:不改。纯增量:新 leaf 模块(src/vertex.ts,零 import 断环)、additive env key、lane-scoped fake(WC-017 先例)、ledger 只增不减(WC-019/020/021 带完整 provenance)。kernel/、pi-subagents/、golden json、update.ts 均未触碰。
  8. 是否引入了新的配置:无新 bili 配置字段。读的是 Claude Code 自己的环境变量(CLAUDE_CODE_USE_VERTEX/CLOUD_ML_REGION/ANTHROPIC_VERTEX_PROJECT_ID/ANTHROPIC_VERTEX_BASE_URL,settings env block 优先、shell 兜底);bili 写的是两个客户端变量(ANTHROPIC_VERTEX_BASE_URL,及原有的 ANTHROPIC_BASE_URL 保持不动)。CONFIGURATION.md 中英同步 + CHANGELOG 条目齐全,config surface 段落符合规范。

审核细节

我直接修掉的小问题(已推回你的分支,commit 824f83e1a,PR head 已更新)

同一缺陷类——未填的引用/指向不存在文件的注释/一处排版:

修完后受影响套件复跑全绿。判定:可合并——等 #3314/#3317 把 master 既有红清掉后 CI 自然转绿,或由 owner 依据上述 CI 归因直接人工合并。

# Conflicts:
#	CHANGELOG.md
#	src/server/prepare-anthropic.ts
#	src/server/relay.ts
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Cross-note from the CI-flake triage in #3449: your windows-latest shard fails deterministically (not a flake) in tests/vertex-wire-fixture.test.ts → "every fixture artifact matches its recorded digest and size":

turn-streaming.meta.json: size
1941 !== 1899

Root cause: Git-for-Windows checks text files out as CRLF (autocrlf=true), adding 1 byte per line. turn-streaming.meta.json has 42 lines: 1899 bytes LF (as recorded in manifest.json) + 42 = 1941. The repo's .gitattributes already documents this exact failure class (#1305) and pins eol=lf per path, but your new tests/fixtures/vertex-anthropic/ directory is not covered — every Windows run will keep failing until it is.

One-line fix on your branch, same pattern as the existing entries:

tests/fixtures/vertex-anthropic/* text eol=lf

Reference: run 38982246862, job 117245612451 (test (windows-latest, 22, 1)).

@ranxianglei
ranxianglei merged commit 774da9b into ranxianglei:master Oct 17, 2026
46 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.499-master.774da9b (rolling master tag).

npm install -g billion-context@0.0.499-master.774da9b  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants