Skip to content

[PR] Register pi/omp bili tools before round 1 so request 1 caches the full prefix - #5465

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-01_pi-native-early-tool-registration
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-01_pi-native-early-tool-registration

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #5459. Related: #4998 / PR #5009 (same mechanism, different symptom).

What this fixes

In native mode (pi -e dist/agent/pi-native.js), bili's tools only joined the request from the 2nd request of each session. pi snapshots each round's payload (tools + system prompt) before its provider hooks run, and in native mode tool registration could only start inside round 1's awaited hooks (no load-time manifest prime exists there — #1217 recorded that residual for wire mode). Result per session:

  • Request 1 goes out with only pi's own tools but is already stamped x-bili-plugin → proxy runs plugin mode → no wire-side tool injection → zero bili tools on the wire.
  • Registration completes mid round 1; pi auto-activates the new tools and rebuilds the system prompt → request 2 carries them (appended after pi's builtins) → the Anthropic prefix breaks at exactly that point → the provider re-writes the whole cached prefix (~19k tokens with Haiku 4.5, measured by the reporter). Once per session; every subagent run is a new session, so it adds up.

Fix

Register as soon as the manifest source exists, not when a request arrives:

Both go through the existing registerTools path with an undefined ctx: plain pi/omp without a parent do no identity POST (identity stays gated on a real session id), and the first real request re-enters cleanly (pending dedupe + toolsFor guard → each tool registered exactly once, pinned by test). The factory is shared by all four entries (pi.ts, pi-native.ts, omp.ts, omp-native.ts), so the omp lane gets the same fix; dsh has its own separate registration path and is untouched.

pi's synchronous registry refresh auto-activates newly registered tools and rebuilds the system prompt, so round 1's natural snapshot already carries the full tool table + final system prompt → stable prefix from request 1.

Behavior change (old → new)

old new
Round 1 tools (pi + omp, launcher & native) host's own only; bili tools appear from round 2 full set from round 1
System prompt round 1→2 rebuilt after late registration (prefix break) stable from round 1
Cache ~19k-token prefix re-write once per session start none for this cause
Identity register timing round 1 (real sid) unchanged — eager entries carry no sid, so nothing moves

Residual (documented, strictly improved): a -p one-shot whose first dispatch beats bootstrap completion still loses the race for that one round (#1217-class). The payload-append alternative was rejected: pi normalizes input_schema on the wire (verified against @earendil-works/pi-ai@1.4.0 convertTools), so raw manifest bytes would not be byte-identical to pi's later serialization and would break the prefix anyway.

Tests

New tests/issue5459-pi-native-early-tools.test.ts (4 tests): registration happens with no request event fired (positive native + launcher); fast-first-request race never double-registers or clobbers the real sid (guard); give-up origin registers nothing and leaks no unhandled rejection (negative direction, #3617 discipline). Negative control verified: on pre-fix code tests 1–2 fail (registration only ever happened on a request event), 3–4 pass.

Local targeted green: new file 4/4, tests/plugin-agent.test.ts + tests/issue1333-derived-inheritance.test.ts 84/84, typecheck clean. Full-suite verdict = CI on this head.

No config surface change. No kernel/ change. No version bump.

查重: related-not-duplicate — same mechanism family as #4998 (capability gap on OpenAI wire, fixed server-side by PR #5009, still open); this fix removes the client-side seam itself, so both symptoms disappear with it; no file overlap between the two PRs.

…e full prefix

pi snapshots each round's payload (tools + system prompt) BEFORE its
provider hooks run, so tool registration that only started inside
round 1's awaited hooks landed after the snapshot: request 1 went out
without bili's tools while already stamped x-bili-plugin (plugin mode
suppresses wire-side injection), and request 2's gained tools broke
the provider prefix cache for the whole session (~19k token re-write
per session start, #5459; capability-gap facet #4998 / PR #5009).

Register as soon as the manifest source exists instead of waiting for
a request: launcher mode now consumes its load-time manifest prime
immediately; native mode registers when the bootstrap publishes the
proxy origin. pi's synchronous registry refresh auto-activates the
new tools and rebuilds the system prompt, so round 1's natural
snapshot already carries the full tool table. Identity registration
is untouched (still gated on a real session id; eager entries carry
none). Residual: a -p cold start whose first dispatch beats the
bootstrap still loses the race for that one round (#1217-class
residual, strictly improved, not widened).
@ranxianglei ranxianglei changed the title Register pi/omp bili tools before round 1 so request 1 caches the full prefix [PR] Register pi/omp bili tools before round 1 so request 1 caches the full prefix Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-01_pi-native-early-tool-registration (74956a4)

Latest build — install this exact version

npm install -g billion-context@0.0.7728-pr.5465.1e331fe

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
74956a4 billion-context@0.0.7728-pr.5465.1e331fe build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5465.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复(related-not-duplicate)。 与在途 PR #5009 同根因家族(pi 的 ACP 工具只在第 2 个请求才出现),但层次不同:#5465 修客户端接缝(pi 提前注册工具,与 wire 无关);#5009 是服务端广告门控(src/host-profiles.ts + src/server/lane-decisions.ts)。两者无文件重叠。目标 issue #5459 已独立核实(pi native / Anthropic wire,~19k token 前缀每会话重写一次)。

Reviewed the diff against master — single commit, based exactly on current head, mergeable=true. Things I verified independently rather than trusting the description:

测试有效性(④): 有牙。正例(native/launcher)在修复前代码会失败(注册只发生在请求事件时);竞态用例数的是总调用次数而非去重后的存在性,能抓住双循环;负例钉住「放弃 origin 不注册、不崩、不泄漏未处理 rejection」。确定性 OK(polling 而非固定 sleep、隔离 HOME/tmpdir、env 还原、finally 关 proxy)。小缺口(非阻塞):① 无专门的 omp 车道用例——但工厂共享,early-reg 关注点上结构等价;② 无端到端断言「round 1 线上真的带全工具」(需真 pi snapshotting,单测只能钉住时序契约这一必要条件),交给 CI 的 real-pi 车道兜底。

CI(镜像缓存读): 当前 0 failed / 13 pending;主 test 矩阵(ubuntu/windows × node 22/24)与 static-gates 仍在跑。按 §7.2,最终绿以推上去的 head 上 CI 为准——结论是「待 CI 全绿」,非本地全量预跑。

行为变更(旧→新,PR 表已披露): round 1 工具面 host-only → 全套;round 1→2 system prompt 重建 → 从 round 1 起稳定;缓存 每会话一次 ~19k 前缀重写 → 该原因下归零。残留:-p 一次性首派发若抢先于 bootstrap 完成,那一轮仍丢(#1217 类,严格不劣于现状)。

顺带一提(非阻塞,issue 卫生): PR 称「both symptoms disappear」覆盖 #4998(OpenAI-wire 版同症状)。客户端早注册确实 wire-无关,应能一并消掉 #4998 的表现;但 #5009 是服务端门控,合入后值得回看它现在是防御纵深还是已被取代——各自留在线程里处理,不扩本 PR 范围。


结构化八问

  1. 是什么问题:pi native/launcher 模式下 bili 的 ACP 工具注册太晚——只能在 round 1 被 await 的 provider hook 里开始,而 pi 在这些 hook 运行之前就快照了本轮 payload(tools+system prompt)。于是 request 1 只带 pi 自带工具却已打 x-bili-plugin(插件模式不做服务端注入),request 2 才补上工具+重建 system prompt,把 provider 前缀缓存打断一次(每会话 ~19k token,pi native: bili's tools join the request only from the 2nd request, so each pi session re-writes ~20k cache tokens once #5459 实测)。
  2. 什么原因导致的:注册时机 vs pi 快照顺序的错位。native 模式 load 时无 env base、async bootstrap 才发 origin,旧代码无法在快照前注册。修法把注册点前移到「manifest 源一存在就注册」:launcher 直接消费 load-time prime;native 等 bootstrap 发布 origin(awaitNativeProxyOrigin 在任何模型请求前解析)。疏(去掉产生缺陷的机制),不是堵。
  3. 影响范围与严重程度:所有 pi/omp 的 native + launcher 会话(共享工厂 pi.ts/pi-native.ts/omp.ts/omp-native.ts)。严重度=成本/缓存瑕疵:每会话启动一次性 ~19k token 前缀重写,p2 级,有界且后续请求自愈,非失控增长;dsh 不受影响;纯代理模式不受影响;可用性从未受损(最坏退化为旧的「首个请求时注册」)。
  4. 怎么修复的:走既有 registerTools 路径、ctx 传 undefined,尽早注册。createBiliPlugin 里加两个武装点:(a) launcher 挂 load-time manifestPrime;(b) native 挂 awaitNativeProxyOrigin() 解析。用 state.toolsFor===undefined 守卫,更快的真实请求若已带真 sid 认领则绝不覆写;并发经 state.pending 单飞去重。ctx=undefined⇒sid=""⇒既有 sid!=="" 门抑制身份 POST,身份时机不变。
  5. 是否有回归问题:逐个核过所有 registerTools 调用点(:1308/:1516/:1640)+ 两个新早注册路径,全走同一套 pending/toolsFor/sid 守卫;registerTool 幂等(pi-native: tools register once per session id, so a pi tool-registry rebuild silently removes compress/decompress/... for the rest of the session #2708)。身份 POST 仍由真 sid 门控(测试断言 registers===0)。放弃/bootstrap 失败路径=no-op+warn,无未处理 rejection(test 4,Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617 双向纪律)。无 fail-fast 变吞错、无阈值/时序漂移(除有意的前移)、无输出格式变化(除有意的 round-1 添工具)。未见无意回归。
  6. 是否破坏缓存:恰恰相反——这正是它修的:消掉会话启动那一次 ~19k 前缀断。不动任何缓存 key/ref/去重逻辑,无不利缓存效应。残留 -p 竞态仍丢一轮(与现状一致,严格不劣)。
  7. 是否改了设计方向:无架构转向。复用既有注册机制 + 既有 native-origin waiter 通道,只加一个更早的武装触发点。被否的备选(原始 manifest payload-append)判断正确:pi 在线上对 input_schema 做归一化(对照 pi-ai convertTools 验证过),原始字节不可能逐字一致,反而会再断前缀。
  8. 是否引入了新的配置:无。config-surface-guard CI 已过;无新 env/config/默认值。

自动合并判定: 基本满足 §7.4(单模块 fix: + 回归测试 + 干净 diff + 引用 issue + 无 kernel/config/version),唯一保留人工的理由是它改变了出站 round-1 工具面/注册时序(插件模式下出站消息形态变更)——低风险,但属产品可见行为变更,建议人工确认后再合。

判定:可合(待 CI 主 test 矩阵全绿;当前 0 failed / 13 pending)。方向正确、根因修复、测试有牙、diff 干净、无内核/配置/版本改动。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pi native: bili's tools join the request only from the 2nd request, so each pi session re-writes ~20k cache tokens once

1 participant