Repository navigation
[PR] fix(#2645): probe-and-remember prompt_cache_key — stamp by default, self-heal strict upstreams - #3736
Conversation
Stamps the dsh plugin lane's relay affinity key on EVERY destination by default (loopback pools, docker/LAN/VPS pools, plain remote relays alike keep their prefix-cache affinity — the old loopback/rewritten-route gate silently dropped it for the two biggest pool populations), and protects strict direct APIs with a controlled A/B instead of a destination-shape guess: - prepare-openai/relay seam: the first 400 from an upstream on a body that carried prompt_cache_key re-sends the identical request once without the key. Retry accepted = causation confirmed: adopt the response, remember the origin+model pair process-locally (in-memory only, restart re-probes). Retry also failed = NOT this field's fault: discard the retry, forward the original 400 verbatim, learn nothing. No persistence, no config. - Client-owned keys pass through verbatim until their pair is ablated; after confirmation they are withheld too so later turns don't re-pay a 400 round trip per turn. - Visible everywhere (owner requirement): one WARN log naming origin/model + session + the account-pool caveat; an acp_status section; the web panel banner (zh+en); GET /__bili/status field pckSuppressions for scripts. The egress re-send follows the same discipline as the existing #552 compat-role ladder: buffered error body reused verbatim on passthrough, resign hook re-applied to the rewritten bytes, wireBody tracks the last accepted payload (#1900), one attempt per request, no loops. Both modes reasoned: plugin mode carries the stamped key (internal conversation header) and gets the A/B protection at egress; proxy mode is pure passthrough for client bodies and is only touched when the client itself sent the key to a strict API — same self-heal, key-withheld after confirmation. Fixes #2645
…onsumable transports Adopted from #3556 (ddc4d0a) — the pi.ts + omp test half. The commit's session-id/prepare-openai/issue2218 changes are NOT adopted: this branch's first commit replaces that destination-shape gate with A/B-verified probe-and-remember instead of static stripping (#2649's loopback-only gate was vetoed — it cost the two biggest relay/account-pool populations their prefix-cache affinity). - pi (stampPromptCacheKey): hoist the #2583 Cowork withhold above the routed-check so it gates ALL stamp paths. Pre-hoist, the launcher cert-MITM lane (BILI_MITM_HOSTS) made destinationRoutedThroughProxy() true and skipped the withhold entirely — stamped bodies reached Anthropic's strict schema verbatim (#1403 400s). - pi (ompCoworkTransportBypassesFetch): a LOOPBACK HTTPS_PROXY is bili itself and Cowork's node:https honors it — treat as routing evidence (stamp); a foreign/non-loopback HTTPS_PROXY is a bypass (withhold). Tests: 4 new cases in omp-native-identity (MITM withhold, loopback HTTPS_PROXY stamp, foreign HTTPS_PROXY withhold, predicate matrix). (cherry picked from commit ddc4d0a)
…PROXY_ANTHROPIC) The #2583 withhold (hoisted earlier in this branch) correctly stops the identity stamp from leaking, but Cowork traffic still left the host direct — uncompressed and invisible to the proxy — because omp's Cowork transport sends node:https and ignores HTTPS_PROXY entirely (proven live, Anthropic req_011Cg7cujfgv4xLjuRvLMqvg). PI_PROXY_ANTHROPIC pushes Cowork onto Bun's own fetch, where the interceptor's /bili/ rewrite carries it through the proxy as plain loopback HTTP (no MITM CA involved). Verified live: forward POST → api.anthropic.com/v1/messages, session-visible, 200, pck scrubbed. Also reverts the loopback-HTTPS_PROXY clause in ompCoworkTransportBypassesFetch (same live proof: not routing evidence for this transport) and flips its tests to the proven posture. (cherry picked from commit 925e6e6)
… guard Review nit on #3556: OMP's getProxyForProvider treats an empty-string PI_PROXY_ANTHROPIC and a whitespace-only PI_PROXY as unset, but the === undefined guard counted them as user-set, leaving Cowork on node:https direct (the stamped-body 400 class this PR exists to kill). The guard now mirrors the src/agent/pi.ts predicate exactly. (cherry picked from commit 0d18580)
README en/zh caveat paragraph, TECHNICAL-NOTES en/zh paragraph (heading omp native -> omp; launcher completion sentence; cert-MITM hoist), CONFIGURATION en/zh (client-table omp row + env appendix PI_PROXY_ANTHROPIC row): the default bili omp launcher lane now routes Cowork through the proxy via PI_PROXY_ANTHROPIC; direct-and-anonymous remains only for native mode and user-pinned proxy postures. (cherry picked from commit 36e073e)
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4420-pr.3736.77e9b34This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3736.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复 —— #3736 是 #2645 的收敛性修复 PR,正文明确
标签: 下面按 PR Review 规范审:先拉分支核对代码与 CI 状态,再走结构化八问。 |
…alizeUpstreamOrigin
A provider route's mitm:// config-lookup scheme normalized to the opaque "null"
origin (new URL("mitm://…").origin === "null"). The egress path strips mitm:// to
https:// (forwardUpstreamUrl) before recording the pck ablation, but the prepare-side
lookup fed billingUpstream (which keeps mitm://) into pckKeyAbated — so for mitm-routed
strict destinations the ablation was keyed under https://host yet looked up under "null",
the self-heal never engaged, and the key stayed stamped (re-probing every turn). Strip
mitm://→https:// inside normalizeUpstreamOrigin (its comparable-route-key contract,
#1933); the same latent mismatch also affected the server.ts baseline-origin compare.
Strict no-op for all non-mitm inputs. Test pins the collapse.
…orts gate The unused-exports gate (which ci.yml makes every test/build lane depend on) flagged PckAblationEntry (src/pck-probe.ts) and RelayAffinityPckDecision (src/session-id.ts) as INTERNAL_ONLY — each is referenced only inside its own file (consumers use the values / string literals, never the type name). Neither is entry-file API surface, so dropping the redundant `export` changes no public d.ts and no behavior; it just lets the gate pass so the full suite actually runs.
(ai: vllm-qwen/qwen3.8-27b) PR #3736 审核结论(pck probe-and-remember)。查重:不重复(#3736 是 #2645 的收敛性修复,supersedes #2649、absorbs #3556,同一机制家族)。核心机制我逐行读过(relay egress 的 A/B seam、session-id 决策函数、prepare-openai 落点、pck-probe 记忆体),方向正确。审中发现并已在 PR 分支上修掉两处(见第 4、5 问)。 八问:
测试有效性:新增 tests/issue2645-pck-probe.test.ts 共 6 个行为 case / 48 条硬断言,覆盖自愈单轮、宽松池不误学、无关 400 不归因、model 粒度、客户端自带键先透传后摘除、跨进程不持久化——断言咬在真实 wire body 上,不是恒真。我另补了 issue1933 里一条 CI:我已把阻塞全矩阵的 unused-exports 门禁修绿并推送(head 3d77753),完整 ubuntu/windows × node 22/24 × 3 shard + e2e 现在会真正跑起来(此前因该门禁红而被 skip)。全量判定以远端 CI 为准,跑完我再回报。 判定:可合(机制正确、双模式都过、回归受控、CI 阻塞已清)。剩两件归人:①等 CI 全绿;②合并后把 #2649(superseded)、#3556(absorbed)关闭并在各自线程指回本 PR。 |
(ai: vllm-qwen/qwen3.8-27b) CI 更新(head 3d77753):我修掉的 unused-exports 门禁已转绿,连带解锁了全矩阵——此前所有 test lane 都是被它 skip 的。现在全部静态门禁/守卫(lock-sync、lane/kernel/pi-subagents/docs/version/config-surface/build-artifact/import-graph、static-gates×OS)已 success,零 failure;只剩 6 个长时 test shard(ubuntu/windows × node 22/24 × 分片)还在跑。跑完我回最终判定。 |
…parses The #2645 panel banner built HTML with double-quoted strings carrying "mono" escapes. Inside the WEB_CLIENT backtick template literal those escapes collapse to bare quotes at evaluation, so the emitted client script throws "Unexpected identifier 'mono'" — breaking not only every vm.Script-based web test (upstream-alerts, web-handoff-roles, web-sessions, cache-ledger) but the whole panel in a real browser. Switch the affected fragments to single quotes, the idiom the adjacent advisory banner already uses. Review fix on top of the author's head; no behavior change beyond making the banner actually parse.
(ai: vllm-qwen/qwen3.8-27b) PR #3736 审毕。结论先行:可合,干净 runner 上 CI 已全绿(48 项 = 32 success + 16 预期 skip、零 failure;其中一条 Windows test shard 首跑 flake,单独重跑即绿)。我在作者 head 之上补了 3 个独立标注的 review-fix,均已 push 到分支( 八问:
合并后需人工跟进两条:#2649 作为 superseded 关闭、#3556 作为 absorbed 关闭。判定:可合。 |
|
(ai: zhipuai-lb/glm-5.3) 三个 review-fix 逐条核实完毕,全部成立。本头( 逐条:
同意「可合」。合并后跟进(#2649 按 superseded 关、#3556 按 absorbed 关)留 owner 操作。 |
(ai: vllm-qwen/qwen3.8-27b) 多谢独立复核——总账一致(48 check-runs = 32 success + 16 预期 skip、零 failure),三条 fix 都成立,也谢谢认领 client.ts 那处疏漏。 你记的那条边界我顺手把持久化面核了一遍,结论与你的判断一致,且可以再收窄一点:
合并后两条跟进(#2649 按 superseded 关、#3556 按 absorbed 关)归 owner 操作即可。我这侧没有更多动作。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.581-master.105c86c # this exact build
npm install -g billion-context@master # always the newest merged build
|
Fixes #2645 · Supersedes #2649 · Absorbs #3556(attribution table below)
What breaks today, and why a static gate can't fix it
On master the plugin lane stamps
prompt_cache_key = session.idinto every OpenAI chat body sent to ANY destination(legacyshouldStampRelayAffinityPckin src/session-id.ts: therouteRewrittenUrl !== undefinedleg is a dead short-circuit becausebillingUpstreamis virtually always defined — see src/server.tsbillingUpstream = route?.rewrittenUrl ?? …, and the remaining leg only matcheslocalhost/127.*/::1). The stamp exists because account-pool / relay operators read this field as prefix-cache routing affinity(#2218). But some strict OpenAI-compatible endpoints reject unknown body parameters outright — #2645: on the SAME originintegrate.api.nvidia.com,super-120banswers 200 with the key whileultra-550banswers 400(Validation: Unsupported parameter(s): \prompt_cache_key``) → dsh plugin-lane turns die with sub-second 400s.No static destination gate separates the affected populations:
This change: stamp by default, verify by A/B, remember in memory
prompt_cache_keypasses through untouched by default. Zero regression for pools regardless of where they sit.role: "developer"messages — request a request-body compat transform hook #552 role-rewrite ladder). When a plugin-lane request gets a 400 back, the error body is drained, and the identical request is resent ONCE with onlyprompt_cache_keyremoved from bili's wire copy(headers re-signed for the new body):(origin, model)pair is remembered in process memory(src/pck-probe.ts) and stopped being stamped there for the rest of this process life;acp_statussection listing ablated origins(+model, confirmations, affected sessions) with explicit wording that relay/account-pool affinity may degrade; a web-panel banner(zh/en) naming the affected sessions; and a machine-readablepckSuppressionsarray onGET /__bili/status.Config surface
None. No new config-field, env var, or CLI flag. This is deliberate: the approved design replaces a static destination decision with an empirical A/B verdict, and the learned state is owner-directed to stay in-memory only. Default path is byte-identical to master; deviations occur only where an upstream proved, live, that it rejects the field.
Both compression modes
x-bili-plugin-conversationidentity): the lane that receives stamps. The decision moves toresolveRelayAffinityPck(identity, clientKey, egressUrl, origin, model)returning"stamp" | "keep-client" | "drop"; stamp on every destination unless ablated. Relay operators who prefer header-based routing are unaffected either way: bili independently forwardsx-session-idfor internal conversations (src/server.ts).Absorbed work from #3556(contributor askl56) — attribution
ddc4d0abpi-lane stamp-withhold hoist above the routed-through-proxy check + loopback-HTTPS_PROXY predicate + omp-native-identity tests-x→54f1b964(partial: its session-id rewrite and prepare-openai static client-key strip were superseded by the probe design)925e6e65launcherPI_PROXY_ANTHROPICfill + live proof that Coworknode:httpsignores env proxies entirely(Anthropicreq_011Cg7cujfgv4xLjuRvLMqvg) — i.e. HTTPS_PROXY presence is NOT routing evidence-x→a8c4c606eb6f4174dead-param refactor + "remote destinations never stamped" regression matrixresolveRelayAffinityPckrewrite, and its remote-no-stamp assertions contradict the approved stamp-by-default design0d185808launcher truthiness guard(empty/whitespace proxy vars treated as unset)-x→dd401a6836e073ecdocs sync(README/CONFIGURATION en+zh rows, TECHNICAL-NOTES caveat)-x→b0980718Original authorship kept via
-xtrailers. Thanks @askl56 — your live Cowork verification became load-bearing for the pi-lane half.Test evidence(on the pushed head)
tests/issue2645-pck-probe.test.ts(7 cases): strict upstream self-heals(client sees 200; upstream saw key-stamped then keyless bodies; ablation entry carries origin/model/session;/__bili/statuspckSuppressionspopulated; turn 2 is a single keyless hit); lenient pool keeps the key every turn with zero ablations; poisoned always-400 upstream forwards the 400 and re-probes every turn without learning anything; model-granularity(strict model ablates, healthy sibling on the same origin keeps the key); client-owned key — turn 1 fixed transparently, turn 2 dropped pre-egress; memory-reset unit semantics.tests/issue2218-relay-affinity-signal.test.ts: unit matrix rewritten to probe semantics(stamp-on-by-default incl. remote destinations; client key passthrough / post-ablation drop; non-plugin identities never trigger stamping); original 5 e2e cases untouched and green.npm run typecheckclean;npm run buildclean including the dist-external-zod gate;core pck lanes 42/42;broad batch(11 files: openai/anthropic pck identity+verbatim, e2e-session-identity, acp-status, advisory-panel, wire-contract, e2e-relay-5xx-selfheal, launcher, etc.)417/417. One environment-sensitive pi-native assertion failed once locally and passed on re-run; it also fails on detached origin/master under this same shell(pre-existing env dependence, not caused by this branch).testlanes on the pushed head.