Skip to content

[PR] fix(cache): prevent mid-session generation-boundary false positives and register pi tools in before_agent_start - #5497

Merged
ranxianglei merged 3 commits into
ranxianglei:masterfrom
xulongwu4:fix/gen-boundary-mid-session-cache-miss
Nov 4, 2026
Merged

ranxianglei merged 3 commits into
ranxianglei:masterfrom
xulongwu4:fix/gen-boundary-mid-session-cache-miss

Conversation

@xulongwu4

@xulongwu4 xulongwu4 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes a mid-session prompt cache miss (and Anthropic thinking-block prefix_binding_mismatch drops) triggered on Turn 2 of cold-started pi sessions or when mid-session tool outputs contain m\d{4,} strings:

  1. Guard detectGenerationBoundary (Switching model mid-conversation restarts the ref ledger, but the rendered transcript still shows the old refs — the model keeps aiming compress at dead ranges until the circuit breaker fires | 切模型后消息编号整体换账本,但渲染历史仍显示旧编号,模型对着死区间反复压缩直至熔断 #4305) against mid-session and role: "system" false positives (src/generation-boundary.ts)

    • detectGenerationBoundary previously ran its initial stale-ref scan across all messages (including role: "system") on every request (session.stats.requests > 1 included).
    • When pi emitted an in-band role: "system" patch (Updated system prompt section "codemode_catalog") on Turn 2, bili's own compress tool description inside <codemode_catalog> ("each block starts with its 'm00150–m00220 optional topic' header line") matched m00150 and m00220 above the session frontier (m00017). Similarly, mid-session tool outputs containing two m\d{4,} tokens above the frontier triggered the detector mid-conversation.
    • Latching session.metadata.generationBoundary mid-session appended GENERATION_BOUNDARY_NOTE (562 chars) to system[1], invalidating the upstream prefix cache for all messages after system[0] and invalidating the cryptographic prefix binding of all prior signed Anthropic thinking blocks (thinking_dropped: prefix_binding_mismatch).
    • Fix: only run initial detection on the first request of a session ((session.stats?.requests ?? 0) <= 1, when a post-switch session generation actually starts at m00001 with carried-over history), and skip m.role === "system" messages.
  2. Await registerTools in before_agent_start (src/agent/pi.ts)

    • session_start calls void registerTools(pi, ctx, state, agent, false) unawaited, while the awaited registerTools calls only lived in before_provider_headers and before_provider_request.
    • In pi, _prepareRun emits before_agent_start and then freezes Turn 1's active tool loadout and <codemode_catalog> system prompt before before_provider_headers / before_provider_request fire. On a cold native-proxy boot, registerTools completed inside before_provider_request after Turn 1's <codemode_catalog> was already built without the ACP tools, causing pi to emit an in-band Updated system prompt section "codemode_catalog" delta on Turn 2.
    • Fix: await registerTools(pi, ctx, state, agent) in before_agent_start (idempotent and sid-cached) so the ACP tools are registered before Turn 1's tool loadout and system prompt are built.

Verification

  • Added unit tests in tests/generation-boundary.test.ts covering in-band role: "system" messages with m00150–m00220 and mid-session turns (stats.requests > 1).
  • Added unit test in tests/plugin-agent.test.ts verifying before_agent_start awaits registerTools and populates pi.tools before Turn 1's prompt preparation.
  • npx tsc --noEmit, npm run build, and test suites pass.

Fixes #5498

…nd register pi tools in before_agent_start

Two issues combined to cause a mid-session prompt cache miss (and Anthropic thinking block prefix_binding_mismatch drops) on Turn 2 of cold-started pi sessions:

1. `detectGenerationBoundary` (ranxianglei#4305) scanned all messages (including `role: "system"`) on every request, even after request ranxianglei#1 had already established a clean session generation (`session.stats.requests > 1`). When an in-band `role: "system"` patch (such as pi's `Updated system prompt section "codemode_catalog"` carrying `compress`'s own `'m00150–m00220 optional topic'` example) or a mid-session tool result quoted >=2 `m\d{4,}` strings above the current frontier, `detectGenerationBoundary` falsely latched `session.metadata.generationBoundary` mid-session and appended `GENERATION_BOUNDARY_NOTE` to the standing system prompt (`system[1]`), invalidating the upstream prefix cache for all messages and breaking Anthropic thinking-block prefix bindings.
   - Only run initial generation-boundary detection on the first request of a session (`(session.stats?.requests ?? 0) <= 1`).
   - Skip `m.role === "system"` messages when scanning for previous-generation ref citations.

2. In `src/agent/pi.ts`, `session_start` calls `registerTools(..., false)` unawaited and `registerTools` was only awaited in `before_provider_headers` and `before_provider_request` — which run after pi's `_prepareRun` has already emitted `before_agent_start` and frozen Turn 1's active tool loadout and `<codemode_catalog>` system prompt. On a cold native-proxy boot, Turn 1's `<codemode_catalog>` omitted the ACP tools, and Turn 2 emitted an in-band `Updated system prompt section "codemode_catalog"` system message.
   - Await `registerTools(pi, ctx, state, agent)` in `before_agent_start` so ACP tools are registered before Turn 1's tool loadout and system prompt are built.
@ranxianglei ranxianglei changed the title fix(cache): prevent mid-session generation-boundary false positives and register pi tools in before_agent_start [PR] fix(cache): prevent mid-session generation-boundary false positives and register pi tools in before_agent_start Nov 1, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 按失败签名(机制 = #4305 generation-boundary 检测器的 mid-session 误锁 + pi 车道 registerTools 竞态)检索 open/closed issue 与 PR:唯一同签名条目是本 PR 自身(#5497)。相关的 #4305/#4308 是该机制的原始立项与修复(已 closed),本 PR 修的是该机制引入后的新缺陷(mid-session false positive),不是重复。#5466/#5476(cache-seam tools-array 归因)同属前缀缓存域但症状与机制均不同,不合并。

标签:bug pi scope/wiring p1(p1 依据 owner 2026-10-19 修订②缓存泄露类:mid-session 锁存后整个会话前缀缓存永久失效 + 已签名的 Anthropic thinking 块被 prefix_binding_mismatch 丢弃,属缓存命中崩坏/数据丢失级)。

正在做完整审核(方向关/回归审查/代码审查/测试有效性 + 八问),结论随后在本帖给出。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论:可合并(独立审核,非复读 issue/PR 描述)。CI 在 head sha 442c69b 上全绿(57 success + 29 skipped + 0 failed;37 个 cancelled 属被取代的旧 run)。diff 干净:4 文件 +80/-1,每行都与本 PR 目的相关。以下按八问作答。

1. 是什么问题
pi 冷启动会话在 Turn 2、或任意会话中途有工具输出/系统补丁出现 ≥2 个 m\d{4,} 形态字符串时,bili 的 generation-boundary 检测器(#4305/#4308 引入)会误锁存「账本已重置」状态:562 字符的 GENERATION_BOUNDARY_NOTE 被追加进 system[1],导致整个会话剩余生命周期的上游前缀缓存全部失效 + Anthropic 已签名 thinking 块被持续丢弃(thinking_dropped: prefix_binding_mismatch)。

2. 什么原因导致的
两层因果,PR 两个改动各打一个 producer(不是症状遮蔽):

  • 检测决策过激(src/generation-boundary.ts):初始 stale-ref 扫描在每个请求都跑且包含 role:"system" 消息。compress 工具自身描述里的示例文本(m00150–m00220)、pi 的 in-band 系统补丁、工具输出引用的旧编号,都构成「frontier 之上 ≥2 distinct stale refs」的假证据。
  • pi 车道触发源(src/agent/pi.ts):session_start 里 void registerTools(..., false) 未 await,而 pi 的 _prepareRun 在 before_provider_headers/before_provider_request 之前就已冻结 Turn 1 的工具装载与 <codemode_catalog>。冷启动竞态下注册晚于 Turn 1 提示构建完成,pi 遂在 Turn 2 发出 in-band 系统补丁——恰好制造上面那类假证据。

3. 影响范围与严重程度
p1(owner 2026-10-19 修订②缓存泄露类):锁存粘滞且随会话元数据持久化,一旦误判即整个会话前缀缓存永久失效(成本爆炸级)+ 签名 thinking 块持续丢失。触发面:pi 原生代理冷启动为确定性路径之一;其他车道在中途文本含双 m-ref 引用时同样可触发。

4. 怎么修复的

  • 初始检测只在会话代首请求执行((session.stats?.requests ?? 0) <= 1)——模型/上游切换产生的新会话代从 m00001 重启账本,上一代残留恰好在首请求的历史里可见;并跳过 role === "system" 消息(环境性指令,不是上一代会话历史)。
  • before_agent_start 中 await registerTools,使 ACP 工具在 Turn 1 工具装载与系统提示冻结前就位。我核实了安全性:该函数幂等(sid 缓存 + pending 去重 + retryAt 节流),等待有界(与 provider 路径共用同一个 15s 上限的 readyOrigin waiter,src/agent/native-intercept.ts:676),不会引入无界挂起;before_agent_start 是真实 pi 事件且 src/agent/pi-subagents.ts:216 早已用它改 systemPrompt,证明它在 Turn 1 提示构建之前触发。

5. 是否有回归问题(逐项审过既有行为)

  • 已锁存会话:readGenerationBoundary 短路在新门禁之前 → 不受影响 ✅
  • side requests(title-gen 等):在 prepare* 之前就被 divert 到 passthrough(src/server/handle.ts:2116-2118),不递增 stats.requests,不消耗一次性检测窗口 ✅
  • 重启后恢复的会话:stats.requests 与锁存记录都持久化(src/persist.ts:3015)→ 语义不变 ✅
  • relay lane 与 prepare-anthropic/openai/google/responses 共用同一函数,语义一致 ✅
  • pi 车道逐回合开销:首次注册后走 sid fast-path(仅 reassertDroppedTools,无网络调用)✅
  • 测试基建改动(fakePi 事件链由并行改顺序 await):理论上可能改变其他用例的 handler 时序,CI 全量套件绿 ✅
  • 未触碰:config 面、持久化格式、wire 协议、kernel/、update.ts、message-ref/id 逻辑、version ✅
  • 已知残留(非阻塞):若新会话代的第一个请求恰好走 prepare* 内的 early-return 路径(auto-mode classifier passthrough src/server/prepare-anthropic.ts:95-98、threads lane :110+)或在 detect 调用点(:486)之前抛错,++session.stats.requests(:78)已消耗一次性窗口但未扫描 → boundary note 永不锁存,退化为 pre-Switching model mid-conversation restarts the ref ledger, but the rendered transcript still shows the old refs — the model keeps aiming compress at dead ranges until the circuit breaker fires | 切模型后消息编号整体换账本,但渲染历史仍显示旧编号,模型对着死区间反复压缩直至熔断 #4305 行为(reactive receipts + 熔断器兜底),无数据丢失、无缓存破坏。若要加固,建议把门禁从 requests 计数改为「扫描已执行」metadata 标志——属设计取舍,留 owner 定夺,不挡合并。

6. 是否破坏缓存
本 PR 正是恢复缓存完整性:消除误判破缓存的路径;合法检测路径(首请求)note 只追加一次、此后 byte-stable(#4308 原设计不变)。对缓存/重试/增量游标/去重键无其他影响。

7. 是否改了设计方向
没有。只收窄检测窗口 + 提前注册时机;架构/边界/数据流不变。行为变更披露(旧→新):generation-boundary 检测窗口由「每个请求直到锁存(含 system 消息)」收窄为「会话代首请求 + 跳过 role:system」——有意变更,原因见第 2/5 问(mid-session 翻转本身就是 p1 缺陷)。

8. 是否引入新配置
没有。

测试有效性:两条新负向用例有牙(去掉对应 guard 必红:system 假引用用例依赖 role-skip,mid-session 用例依赖 requests 门禁);正向控制沿用 #4308 存量(two-distinct-citations latch / fresh session / tag-shaped residue / below-frontier 不触发);plugin-agent 新用例驱动真实注册 handler 打真实 fake HTTP proxy,断言 Turn 1 准备前 pi.tools 恰为 {acp_status, compress}——非恒真、抓得住回归。#3617 双向门禁纪律满足。

补做的交付动作(原 PR 缺 issue 引用,违反 §4 追溯规则与 auto-merge gate 第 7 条):

  • 已立 issue #5498 记录该缺陷(带来源标记 + 查重声明);
  • 已在 PR 正文末尾追加一行 Fixes #5498(纯追加,未改动作者原文,未动标题)。

判定:可合并,待人工 merge(merge 权限在人)。

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

APPROVE — 独立复核通过。四处改动(src/generation-boundary.ts + src/agent/pi.ts + 两个测试文件)逻辑正确、有回归钉、无阻塞项、不触碰 kernel/。以下为独立分析结论(非转述 issue)。

1. 是什么问题:pi 冷启动 Turn 2,或任意会话中途工具输出 / in-band role:"system" 补丁里出现 ≥2 个 m\d{4,} 串时,generation-boundary 检测器误锁存,把 562 字符的 GENERATION_BOUNDARY_NOTE 追加进 standing system prompt → 前缀缓存自 system[0] 后整体失效 + Anthropic 已签名 thinking 块被丢(prefix_binding_mismatch)。缓存泄露类,p1。

2. 什么原因导致的(独立验证):两个面同症——①检测器在每个请求都扫全量消息且含 system;②pi session_start 里 registerTools(...,false) 未 await,注册只发生在 _prepareRun 冻结 Turn 1 loadout 之后的 provider hooks,冷启动竞态下 Turn 1 <codemode_catalog> 缺 ACP 工具 → Turn 2 发 in-band system 补丁(catalog 内 compress 描述的 m00150–m00220 示例落在 frontier 之上)构成假证据。

3. 影响范围与严重程度:所有 pi 原生代理冷启动会话(确定性复现)+ 任意车道中途双 m-ref 引用;一旦误锁存粘滞且持久化,整个会话生命周期前缀缓存永久失效(成本爆炸级)+ 签名 thinking 持续丢弃。p1。

4. 怎么修复的:①检测窗口收窄到「会话代首请求」。我逐一核对了四个 prepare* 路径都是先 ++session.stats.requests 再调 detect(openai:64→349 / anthropic:78→486 / google:85→306 / responses:70→437),且 runPrepare 内部重建那次自增在 server.ts:3847 减回,故 (requests ?? 0) > 1 恰好等价「首个客户端请求之后」;已锁存会话由 line 90 的 readGenerationBoundary 短路兜住,note 仍 byte-stable 全程渲染。②跳过 m.role === "system"(环境性指令非上一代历史)。③before_agent_start 里 await registerTools(幂等 + sid 快路径 + state.pending 去重)。

5. 是否有回归问题:逐项过既有行为——sticky 锁存保持(line 90 在前);frontier 单调不降(Kernel Contract:host 不得 shrink byRef 游标),故真实边界只可能在首请求可见,收窄不丢真阳;system 跳过不影响 user/tool/assistant 残留识别;pi Turn 1 现在含 ACP 工具 = 把冷启动对齐到热启动(修正而非漂移);before_agent_start 等待经 awaitNativeProxyOrigin → readyOrigin → withTimeout(…, readyTimeoutMs ?? 15000) 有界 15s,稳态(sid 已匹配走快路径)零额外延迟。均无无意回归。唯一收窄面已在 issue 披露:若某真实边界只在既有对象 request #2+ 才可见则不再新锁存——但后果仅为模型撞死 ref 直到熔断(#4305 原症状,自限),远轻于误锁存的 p1 代价,取舍合理。

6. 是否破坏缓存:方向相反——本 PR 修复的就是缓存失效;note 文本未变、只是更少误附加,前缀更稳。无破坏。

7. 是否改了设计方向:无架构转向,两处均为局部守卫 / 时序修正。

8. 是否引入了新的配置:无新 env/config/默认值(config-surface-guard、version-guard 均绿佐证)。

测试有效性:三条新用例都有牙——system-skip 用例(fixture requests=0,去掉 skip 会锁存 true)、mid-session 用例(requests=6,去掉 >1 守卫会锁存 true)、before_agent_start 用例(PR 前该 handler 不存在,assert.ok(beforeAgentStart) 直接红);makeFakePi 改为 await 前置 handler 返回的 promise,严格更正确。CI 30 checks 全绿(kernel-guard/config-surface-guard/version-guard/lane-guard/gate 均 success)。

判定:可合。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 复核完成 — APPROVE(GitHub PR #5497,review id 5730319239)。

独立复核结论(非转述 issue):

  • 改动:①src/generation-boundary.ts 检测窗口收窄到「会话代首请求」+ 跳过 role:"system";②src/agent/pi.ts 在 before_agent_start await registerTools。修 p1 缓存泄露类误锁存(前缀缓存自 system[0] 后整体失效 + Anthropic 已签名 thinking 块被丢 prefix_binding_mismatch)。
  • 正确性:四个 prepare* 路径均为先 ++session.stats.requests 再调 detect(openai:64→349 / anthropic:78→486 / google:85→306 / responses:70→437),runPrepare 内部重建那次自增在 server.ts:3847 减回,故 (requests ?? 0) > 1 恰等价「首个客户端请求之后」;sticky 短路(line 90 readGenerationBoundary)在前,已锁存会话 note 仍 byte-stable 全程渲染。frontier 单调不降(Kernel Contract),收窄不丢真阳。pi 侧 registerTools 幂等(sid 快路径 + state.pending 去重)、等待经 readyOrigin 有界 15s、稳态零额外开销。
  • 回归:逐项过既有行为,无无意回归;唯一收窄面(真实边界若只在既有对象 request fix: address all code review findings (condense, routing, session-id, dead code) #2+ 才可见则不再新锁存)已在 issue 披露且代价自限——后果仅为模型撞死 ref 至熔断(Switching model mid-conversation restarts the ref ledger, but the rendered transcript still shows the old refs — the model keeps aiming compress at dead ranges until the circuit breaker fires | 切模型后消息编号整体换账本,但渲染历史仍显示旧编号,模型对着死区间反复压缩直至熔断 #4305 原症状),远轻于误锁存的 p1 代价。缓存方向相反:本 PR 修复的正是失效,note 文本未变、只是更少误附加。
  • 测试有效性:三条新用例都有牙——system-skip(fixture requests=0)、mid-session(requests=6)、before_agent_start(PR 前该 handler 不存在,assert.ok 直接红);makeFakePi 改为 await 前置 handler 的 promise,严格更正确。CI 30 checks 全绿(kernel-guard / config-surface-guard / version-guard / lane-guard / pi-subagents-guard / gate 均 success)。
  • 边界:不触碰 kernel/、无新配置/默认值、diff 干净、无小修需顺手改 → 无需 needs-decision 门禁。

判定:可合。

@xulongwu4

Copy link
Copy Markdown
Contributor Author

CI on head 4e42646f (merge of master into this branch) fails 8 tests: storage-render golden (chat/anthropic/responses/google) and storage engine: full-mode restart is invisible on the … wire.

This failure comes from upstream master, not from this PR:

I will merge master into this branch again after #5579 lands.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 独立核对确认:这次红是继承自 master 的 golden 漂移,与本 PR 无关,APPROVE 结论不受影响。

我的独立验证(非转述你的结论):

处置建议(分支归你管,仅供参考):既然 golden 重生成已落在 ci-fixes,直接把 origin/ci-fixes 并进本分支即可立即转绿,不必等 #5579 先落 master 再重并——这是继承性失败的标准路径。我这边无需动作;CI 转绿后即可合并。

@ranxianglei
ranxianglei merged commit 0fa5058 into ranxianglei:master Nov 4, 2026
86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pi cold start or mid-session m-ref echo falsely latches the generation-boundary note — breaks prefix cache and drops signed thinking blocks

2 participants