Skip to content

fix: stop leaking prompt_cache_key into anthropic upstreams (#1403) - #1404

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-26_pck-anthropic-leak
Sep 26, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-26_pck-anthropic-leak

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What changed

bili omp stamped a raw session UUID as top-level prompt_cache_key on every chat-shaped payload (stampPromptCacheKey, src/agent/pi.ts), regardless of where the request was headed. Strict-schema upstreams behind a blind TLS tunnel (opencode zen's https://…/zen/v1/messages, anthropic protocol) reject unknown fields:

400 {"type":"error","error":{"type":"invalid_request_error","message":"Upstream request failed: [invalid_request_error] prompt_cache_key: Extra inputs are not permitted"}}

→ 100% failure for every anthropic-protocol model through bili omp. Reproduced locally on v0.1.154 master before fixing (strict fake upstream + drive harness in the issue thread).

Two-layer fix, per the root-cause principle (fix the producer, plus defense-in-depth at the decision point):

Producer (src/agent) — stampPromptCacheKey now stamps only when the destination will actually reach the proxy's processing pipeline: proxyBaseFromUrl(baseUrl) defined (the /bili/ lane), OR origin === BILLION_CONTEXT_PROXY, OR hostname ∈ BILI_MITM_HOSTS (exact/subdomain). New helpers destinationRoutedThroughProxy / mitmHostsFromEnv in src/agent/shared.ts. Rationale: the pck stamp is load-bearing identity for the proxy (#268) but is useless for hosts the proxy cannot see — and harmful, because it corrupts the upstream request. This deliberately does NOT align with promptCache.routing: that setting governs the proxy's own openai-wire sticky-routing injection; pck is an identity signal on both wires, and routing it out would degrade every anthropic upstream (including api.anthropic.com) to anonymous affinity sessions.

Defense in depth (src/server) — new scrubAnthropicPck() strips top-level prompt_cache_key from anthropic bodies on every verbatim forward path: side request #388, chain verdict #1086, passthrough mark #1117, plugin-bypass header #920, non-conversation relay #1284, and the final fallback (which covers --passthrough global mode and route passthrough #661). On the anthropic wire a top-level prompt_cache_key is never legitimate API content (the processed path already strips it in prepareAnthropic), so stripping is always safe. Bodies without the field pass byte-identical (parse → delete → re-serialize only when the own-prop exists); openai/responses wires untouched (dsh native pck forwarding preserved — tests/openai-pck-identity.test.ts stays green).

Launcher — exports new additive env var BILI_MITM_HOSTS to pi/omp children (= DEFAULT_MITM_DOMAINS ∪ config-file ∪ BILI_MITM_DOMAINS ∪ extra domains ∪ auto-discovered hosts), mirroring exactly what the proxy's isMitmHost() sees.

Wire-contract ledger (#1304) — WC-007 added: top-level prompt_cache_key is not part of the Anthropic Messages API (provenance: #1403 production 400, opencode zen, 2026-09-26), with validator clause + self-check probe.

Behavior changes (old → new)

Area Old New Why
omp pck stamping unconditional for all chat-shaped payloads destination-gated (only proxy-visible hosts) blind-tunneled destinations lose pck identity (anonymous prefix-affinity sessions, #309) instead of leaking a foreign field that 400s strict upstreams
Verbatim anthropic forwards (side request / chain verdict / passthrough marks / non-conversation relay / global+route passthrough) client bytes forwarded verbatim, including any top-level prompt_cache_key field stripped; re-serialization only when present the field is bili's own injection or client noise that would 400 at real Anthropic anyway; never legitimate anthropic-wire content
Route passthrough #661 byte-for-byte guarantee ALL bodies pck-free bodies (the normal case — covered by a dedicated byte-exact test) consequence of the row above
pi/omp child env — + BILI_MITM_HOSTS additive; lets the extension gate stamping on the real MITM whitelist

Evidence (A/B, same harness, same strict upstream)

Request unpatched v0.1.154 this branch
main path (max_tokens 1024) 200, pck stripped (already handled by prepareAnthropic) 200, no pck upstream
side request (max_tokens ≤ 200) 400 exact zen error, pck leaked 200, no pck upstream
chain verdict (ACP tags, fresh session) 400 exact zen error, pck leaked 200, no pck upstream
global/route passthrough, header bypasses 400 where pck present 200, no pck upstream

pck occurrences in forwarded bodies: 0 across all cases after the fix.

Pre-flight

  • npm run typecheck clean
  • npm test: 2800 tests, 0 fail, 12 skipped
  • npm run build: success, dist-import-annotations OK
  • E2E zero-token preflight (E2E_CHECK=1) pass; full E2E runs in CI (diff touches request-pipeline hot files)

New regression suite: tests/anthropic-pck-verbatim.test.ts (strict-upstream A/B for all four leak paths); tests/route-passthrough.test.ts assertions updated to semantic strip checks + byte-exact no-pck test; plugin-agent matrix extended with destination-gate cases (external host → not stamped; BILI_MITM_HOSTS / BILLION_CONTEXT_PROXY match → stamped).

Closes #1403

The omp extension stamped a raw session UUID as top-level
prompt_cache_key on every chat-shaped payload regardless of where it
was headed. Strict-schema upstreams behind a blind TLS tunnel (e.g.
opencode zen's /v1/messages) reject unknown fields with 400 "Extra
inputs are not permitted" — 100% failure for every anthropic-protocol
model through `bili omp`.

Producer fix (src/agent): stampPromptCacheKey now stamps only when the
destination will actually reach the proxy's processing pipeline
(proxyBaseFromUrl / BILLION_CONTEXT_PROXY origin / BILI_MITM_HOSTS
membership). The pck stamp is load-bearing identity (#268) but is
useless — and harmful — for hosts the proxy cannot see.

Defense in depth (src/server): scrubAnthropicPck strips the field from
anthropic bodies on every verbatim forward path (side request #388,
chain verdict #1086, passthrough marks #1117/#920, non-conversation
relay #1284, final fallback covering --passthrough and route
passthrough #661). Bodies without the field stay byte-identical;
openai/responses wires are untouched (dsh native pck forwarding intact).

Launcher exports BILI_MITM_HOSTS to pi/omp children so the extension
sees exactly the proxy's MITM whitelist.

Wire-contract ledger: WC-007 records that top-level prompt_cache_key
is not part of the Anthropic Messages API (provenance: #1403).

Behavior changes (old -> new):
- omp pck stamping: unconditional -> destination-gated. Blind-tunneled
  destinations lose pck identity (anonymous prefix-affinity sessions,
  #309) instead of corrupting the upstream request with a foreign
  field.
- Verbatim anthropic forwards: client bytes verbatim including any
  top-level prompt_cache_key -> field stripped (re-serialization only
  when present). The #661 byte-for-byte guarantee now applies to
  pck-free bodies (the normal case).
- New additive env var BILI_MITM_HOSTS exported to pi/omp children.

Tests: new tests/anthropic-pck-verbatim.test.ts (strict-upstream A/B
for all four leak paths), route-passthrough assertions updated to
semantic strip checks plus a byte-exact no-pck test, plugin-agent
matrix extended with destination-gate cases, WC-007 self-check probe.
Full suite green (2800 tests / 0 fail), typecheck clean, build OK,
e2e zero-token preflight pass.

Closes #1403
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-26_pck-anthropic-leak (56ac6a7)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1404

Each push to this PR publishes a new version under the pr-1404 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1404.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,开始 review。分支已拉取(2026-09-26_pck-anthropic-leak @ 56ac6a7),base 与当前 master(36d654ff)一致,无 stale-base 问题。CI:e2e / build / version-guard / 各真实客户端 lane 均已绿,4 个 test job(ubuntu 22/24、windows 22/24)仍在跑。正在按 ①原 issue → ②回归 → ③代码 → ④测试 → ⑤diff 顺序审,完成后回帖结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #1404 review — 可合并(人工合并)

Reviewed head 56ac6a75 (branch 2026-09-26_pck-anthropic-leak, 1 commit, 10 files, +510/−29) against current master 36d654ff — base is fresh, no stale-base risk today. Evidence: local npm run typecheck clean; targeted run of the 4 changed test files = 80/80 pass; CI 13/13 green on head (incl. all 4 test matrices, e2e, build-artifact, version-guard, hermetic registry e2e, real pi/codex/dsh/opencode lanes, codex overlay check).


① Original issue direction — correct, and better than the options proposed in #1403.
The user's root-cause analysis in #1403 was accurate (unconditional stampPromptCacheKey). The PR deliberately rejects option 1 (align with promptCache.routing) — correctly, because pck on the anthropic wire is the proxy's identity signal (#268, consumed at src/server.ts:1524-1539 via preferPromptCacheKeyIdentity), not upstream routing config; routing it off would degrade identity binding for every MITM'd anthropic destination including api.anthropic.com. Option 3 (learn-from-400) would burn one failed request per host/session. The chosen approach fixes the actual decision point — stamp only where the field will be consumed — plus defense-in-depth scrubbing so any client-sent pck can't leak on any verbatim path. Root cause, not symptom; passes the litmus test (a different strict upstream surfacing tomorrow is still covered).

② Regression review — pre-existing behaviors on touched paths:

  1. omp pck stamping / identity binding (用bili omp 启动omp之后, /acp命令不能正确执行. #268) — MITM-whitelisted + /bili/-wrapped destinations: unchanged, still stamped. Blind-tunnel destinations: intentional change (old: stamped unconditionally → leaked verbatim to upstream; new: not stamped). No functional loss there — blind-tunneled traffic is opaque to the proxy, so those sessions never had proxy-side identity binding to lose; the disclosed "degrades to prefix-affinity sessions (# [Bug/Regression] 0.1.59 关闭内容指纹会话兜底 (#286):无会话头第三方 harness 请求被直接 400,不转发上游 #309)" applies to whitelist-mirror drift cases. Disclosed in the PR behavior table ✓
  2. pi agent — untouched: gate early-returns unless agent === "omp" ✓
  3. Processed anthropic path (prepareAnthropic) — unchanged; already stripped pck at src/server.ts:2856 (pre-existing) ✓
  4. Six verbatim forward paths (side-request 标题生成等侧请求污染主会话状态:prepareAnthropic 缺 isTitleGen 守卫、快照无条件覆盖、survivedCount 每请求空转 #388, chain verdict [Bug] 链检测把 bili 自己注入的 ACP 产物判成 bili→bili 链 —— 压缩内核永久跳过,长会话撞 1,048,576 上限 400(v0.1.133 回归) #1086, bypass opencode v1: keep opencode-acp legacy sessions working after switching to bili #920, passthrough-mark 「BUG」native-intercept 进程级 globalThis.fetch 补丁按 URL 形状接管所有模型流量,无关调用者(如 MemOS)被卷入宿主压缩流程 #1117, non-conversation relay [Bug] DSH native fetch 拦截误伤非 LLM 端点:任何以 /messages 结尾的 URL 都被当成模型 API #1284, final fallback incl. route/global passthrough billion-context ACP kernel 改写 body 导致 ZCode (zcode.z.ai) 405/3012 #661): old = client bytes byte-for-byte incl. any top-level pck; new = pck removed iff present, byte-exact iff absent (parse→delete→reserialize only on own-prop presence; key order preserved). Intentional, disclosed. Top-level prompt_cache_key is not part of the Anthropic Messages API, so no legitimate payload loses data. Verified all 7 forward() call sites in src/server.ts are covered (6 scrubbed + processed path) ✓
  5. OpenAI/Responses wire — untouched; dsh native pck forwarding unaffected ✓
  6. Launcher env — additive BILI_MITM_HOSTS export to pi/omp children only; no existing var altered ✓
  7. Wire-contract ledger — grew by WC-007 with provenance (production 400 + API reference), enforced in the validation-parity fake, ledger-test violation case added; nothing shrunk — complies with [CI/testing] wire-contract anti-regression system: validation-parity fake upstreams + golden schema snapshots + constraint ledger + staged 4-protocol x 3-mode client smoke matrix #1304 ✓
  8. Honest test rewrite: the old route-passthrough assertion "prompt_cache_key must survive" pinned the leaking behavior; the PR rewrites it (semantic strip + field-intact deepEqual) and adds a dedicated byte-exact no-pck contract test rather than silently dropping coverage ✓

③ Code review.

  • destinationRoutedThroughProxy (src/agent/shared.ts:78) mirrors the proxy's real MITM decision exactly: same exact-or-subdomain, lowercased matching as isMitmHost (src/mitm.ts:96-100); origin comparison uses the same normalized scheme://host form as proxyBaseFromEnv; malformed/relative/non-http(s) URLs fail closed (no stamp) — safe direction.
  • Whitelist mirror verified tier-by-tier: launcher's extMitmHosts = DEFAULT ∪ resolveMitmDomains(child-env tier) ∪ domains ∪ discoverMitmDomains(discoveryEnv); the child's effective set = DEFAULT ∪ opts.mitm.domains (config-file ∪ child BILI_MITM_DOMAINS, set at src/launcher.ts:2883) ∪ in-child discovery over the same machine config files. Tiers align; childMitmEnv correctly mirrors the empty-domains inheritance case too.
  • Residual boundary (documented, not blocking): the mirror is a launch-time snapshot. Two corners can drift: (a) attaching to a proxy already started by another lane whose CLI --mitm-domain list differs; (b) stale PI_CODING_AGENT_DIR affecting in-child discovery differently than the launcher's 注入零文件化:废除 overlay/临时目录机制(#531 设计返工) #535 strip. Both mostly fail safe (under-stamp → identity degrades, no leak); the one over-stamp corner (host known only to this launch's CLI flag, served by an attached foreign-lane proxy) coincides with compression-not-working anyway — a pre-existing boundary, not introduced here. Worth a README footnote someday; no action needed now.
  • scrubAnthropicPck: parse-fail/non-object/no-pck all return the original buffer (zero-copy, byte-exact); hasOwnProperty guard; debug log carries no secrets. No as any, no @ts-ignore, hex escapes used for tags in tests, loggerLog respected. Clean.

④ Test validity — real teeth.

  • New tests/anthropic-pck-verbatim.test.ts: fake upstream enforces zen's exact schema (top-level pck ⇒ the literal production 400 body). 5 paths covered (side-request, chain verdict, global passthrough, passthrough-mark + bypass header, non-conversation relay). On the old code each fails with 400 vs asserted 200 — genuine regression tests, not tautologies. The chain test additionally pins verbatimness (parsed-text equality, exactly one [chain] warn, no local session created).
  • plugin-agent.test.ts: three destination-gate cases (external host w/o env → not stamped; BILI_MITM_HOSTS match → stamped; BILLION_CONTEXT_PROXY origin → stamped), env save/restore hygiene correct; dependent Pi plugin cancels Pi's auto-compaction for conversations the proxy never sees (breaks pi-claude-bridge) #1382/fix: omp chat-completions sessions get pck identity + /acp panel #266 tests updated to routed ctx — shows the author traced dependent call sites, not just the repro.
  • route-passthrough.test.ts: semantic-strip + byte-exact-no-pck contract.
  • WC-007 ledger entry + enforcement clause + violation case.

⑤ Diff cleanliness — every line serves the fix; no unrelated churn, no whitespace noise, version untouched, single linear commit. Clean.

Behavior-change disclosure (old→new + why):

Hot-file contention note (§7.2): six open PRs also touch src/server.ts (#1397, #1364, #1215, #1358, #1350, #1247) and src/agent/pi.ts / src/config.ts / tests/wire-contract.test.ts (#1402, #1358, #1216). Regions are distinct, but whichever lands later must rebase onto the moved master and re-run CI — expect sequencing friction, especially #1402 sharing the wire-contract test file.

Auto-merge gate: NOT eligible — outbound wire/message-shape change on anthropic forwards + product trade-off (identity vs leak). Must stay human-merged, as expected.


结论:修复了 omp 插件无条件向 anthropic 请求体注入 prompt_cache_key 导致严格 schema 上游(opencode zen)100% 400 的问题——生产端按「目的地是否被代理实际解密」门控注入、服务端对全部 verbatim 转发路径兜底剥离,原有身份绑定(#268)与字节保真契约(#661)均有测试钉住,CI 13/13 全绿,可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review: ✅ merge-ready. Verified on the branch head locally — typecheck clean, full suite 2800 tests / 0 fail, the new suite's five branch tests (80/80 relevant) all green.

Producer gate (destinationRoutedThroughProxy) — correct and fail-closed. /bili/-wrap ∨ BILLION_CONTEXT_PROXY origin ∨ MITM-whitelist host (exact/subdomain); unparseable URL or no match → no stamp. The deliberate non-alignment with promptCache.routing is the right call and worth stating in the merge note: routing gates the proxy's own OpenAI-wire sticky injection, while this pck is the omp identity chain (#268) — gating it by the OpenAI whitelist would demote every anthropic upstream to anonymous prefix-affinity (#309) and break /acp binding. Two different consumers, two different gates; the PR documents this.

Proxy scrub (scrubAnthropicPck) — tight. anthropic-only, top-level-only, JSON-parse-fail → verbatim passthrough, absent-field → early return with no re-serialization (byte-identical, so #661 fingerprinting is untouched in the normal case; when the field IS present the fingerprint sees the scrubbed buffer consistently at forward time). All five verbatim branches covered (side #388, chain #1086, passthrough-mark/plugin-bypass, #1284 relay, final fallback) — T1–T5 pin each one end-to-end.

Launcher mirror — exactly the proxy's whitelist. resolveMitmDomains reuses loadOptions' own tier computation (config file ∪ BILI_MITM_DOMAINS) plus discovery, so extension and proxy can't disagree. Exported only when non-empty.

Ledger: WC-007 registered with provenance — follows the ledger-only-grows rule.

Non-blocking observations:

  1. Residual tradeoff (documented in code): unrouted/blind destinations now get no identity stamp → those sessions degrade to anonymous prefix-affinity (# [Bug/Regression] 0.1.59 关闭内容指纹会话兜底 (#286):无会话头第三方 harness 请求被直接 400,不转发上游 #309). Acceptable: compression still works, /acp lookup by session id does not — strictly better than a guaranteed 400.
  2. The issue's suggestion Use kernel renderTags:"text-only" + forbid tag echoing in prompt #3 (auto-remember per-host rejection) is moot under the producer gate — no objection to leaving it unimplemented.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,与你的结论完全一致——两条非阻塞观察我都同意,且已在上一轮 review 里各自留了痕(残差边界记为「launch-time snapshot 的固有边界、fail-safe 方向」;建议 #3 在 producer gate 下确实失去存在前提,不实现是对的)。

补一点你这条没覆盖的:合并顺序。当前还有 9 个 open PR 与 #1404 共享热文件——src/server.ts(#1397/#1364/#1215/#1358/#1350/#1247)、src/agent/pi.ts + src/config.ts + tests/wire-contract.test.ts(#1402/#1358/#1216)。区域互不重叠,但谁后合谁需要 rebase 到移动过的 master 并重跑 CI(尤其 #1402 共用 wire-contract 测试文件那几行)。CI 对 head 56ac6a75 已 13/13 全绿,若合并前 master 再动,rebase 后以新 head 的 CI 为准即可。

按仓库规则我不执行合并——PR 链接:#1404 ,请人工合并。

中文摘要:双方独立复核一致(生产端门控 fail-closed、服务端六条 verbatim 路径兜底剥离、whitelist 镜像逐层对齐、WC-007 入册),无阻塞项,仅剩合并顺序注意,可以合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant