Repository navigation
[PR] fix(cache): prevent mid-session generation-boundary false positives and register pi tools in before_agent_start - #5497
Conversation
…nd register pi tools in before_agent_start Two issues combined to cause a mid-session prompt cache miss (and Anthropic thinking block prefix_binding_mismatch drops) on Turn 2 of cold-started pi sessions: 1. `detectGenerationBoundary` (ranxianglei#4305) scanned all messages (including `role: "system"`) on every request, even after request ranxianglei#1 had already established a clean session generation (`session.stats.requests > 1`). When an in-band `role: "system"` patch (such as pi's `Updated system prompt section "codemode_catalog"` carrying `compress`'s own `'m00150–m00220 optional topic'` example) or a mid-session tool result quoted >=2 `m\d{4,}` strings above the current frontier, `detectGenerationBoundary` falsely latched `session.metadata.generationBoundary` mid-session and appended `GENERATION_BOUNDARY_NOTE` to the standing system prompt (`system[1]`), invalidating the upstream prefix cache for all messages and breaking Anthropic thinking-block prefix bindings. - Only run initial generation-boundary detection on the first request of a session (`(session.stats?.requests ?? 0) <= 1`). - Skip `m.role === "system"` messages when scanning for previous-generation ref citations. 2. In `src/agent/pi.ts`, `session_start` calls `registerTools(..., false)` unawaited and `registerTools` was only awaited in `before_provider_headers` and `before_provider_request` — which run after pi's `_prepareRun` has already emitted `before_agent_start` and frozen Turn 1's active tool loadout and `<codemode_catalog>` system prompt. On a cold native-proxy boot, Turn 1's `<codemode_catalog>` omitted the ACP tools, and Turn 2 emitted an in-band `Updated system prompt section "codemode_catalog"` system message. - Await `registerTools(pi, ctx, state, agent)` in `before_agent_start` so ACP tools are registered before Turn 1's tool loadout and system prompt are built.
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 按失败签名(机制 = #4305 generation-boundary 检测器的 mid-session 误锁 + pi 车道 registerTools 竞态)检索 open/closed issue 与 PR:唯一同签名条目是本 PR 自身(#5497)。相关的 #4305/#4308 是该机制的原始立项与修复(已 closed),本 PR 修的是该机制引入后的新缺陷(mid-session false positive),不是重复。#5466/#5476(cache-seam tools-array 归因)同属前缀缓存域但症状与机制均不同,不合并。 标签: 正在做完整审核(方向关/回归审查/代码审查/测试有效性 + 八问),结论随后在本帖给出。 |
(ai: vllm-qwen/qwen3.8-27b) 审核结论:可合并(独立审核,非复读 issue/PR 描述)。CI 在 head sha 1. 是什么问题 2. 什么原因导致的
3. 影响范围与严重程度 4. 怎么修复的
5. 是否有回归问题(逐项审过既有行为)
6. 是否破坏缓存 7. 是否改了设计方向 8. 是否引入新配置 测试有效性:两条新负向用例有牙(去掉对应 guard 必红:system 假引用用例依赖 role-skip,mid-session 用例依赖 requests 门禁);正向控制沿用 #4308 存量(two-distinct-citations latch / fresh session / tag-shaped residue / below-frontier 不触发);plugin-agent 新用例驱动真实注册 handler 打真实 fake HTTP proxy,断言 Turn 1 准备前 补做的交付动作(原 PR 缺 issue 引用,违反 §4 追溯规则与 auto-merge gate 第 7 条):
判定:可合并,待人工 merge(merge 权限在人)。 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE — 独立复核通过。四处改动(src/generation-boundary.ts + src/agent/pi.ts + 两个测试文件)逻辑正确、有回归钉、无阻塞项、不触碰 kernel/。以下为独立分析结论(非转述 issue)。
1. 是什么问题:pi 冷启动 Turn 2,或任意会话中途工具输出 / in-band role:"system" 补丁里出现 ≥2 个 m\d{4,} 串时,generation-boundary 检测器误锁存,把 562 字符的 GENERATION_BOUNDARY_NOTE 追加进 standing system prompt → 前缀缓存自 system[0] 后整体失效 + Anthropic 已签名 thinking 块被丢(prefix_binding_mismatch)。缓存泄露类,p1。
2. 什么原因导致的(独立验证):两个面同症——①检测器在每个请求都扫全量消息且含 system;②pi session_start 里 registerTools(...,false) 未 await,注册只发生在 _prepareRun 冻结 Turn 1 loadout 之后的 provider hooks,冷启动竞态下 Turn 1 <codemode_catalog> 缺 ACP 工具 → Turn 2 发 in-band system 补丁(catalog 内 compress 描述的 m00150–m00220 示例落在 frontier 之上)构成假证据。
3. 影响范围与严重程度:所有 pi 原生代理冷启动会话(确定性复现)+ 任意车道中途双 m-ref 引用;一旦误锁存粘滞且持久化,整个会话生命周期前缀缓存永久失效(成本爆炸级)+ 签名 thinking 持续丢弃。p1。
4. 怎么修复的:①检测窗口收窄到「会话代首请求」。我逐一核对了四个 prepare* 路径都是先 ++session.stats.requests 再调 detect(openai:64→349 / anthropic:78→486 / google:85→306 / responses:70→437),且 runPrepare 内部重建那次自增在 server.ts:3847 减回,故 (requests ?? 0) > 1 恰好等价「首个客户端请求之后」;已锁存会话由 line 90 的 readGenerationBoundary 短路兜住,note 仍 byte-stable 全程渲染。②跳过 m.role === "system"(环境性指令非上一代历史)。③before_agent_start 里 await registerTools(幂等 + sid 快路径 + state.pending 去重)。
5. 是否有回归问题:逐项过既有行为——sticky 锁存保持(line 90 在前);frontier 单调不降(Kernel Contract:host 不得 shrink byRef 游标),故真实边界只可能在首请求可见,收窄不丢真阳;system 跳过不影响 user/tool/assistant 残留识别;pi Turn 1 现在含 ACP 工具 = 把冷启动对齐到热启动(修正而非漂移);before_agent_start 等待经 awaitNativeProxyOrigin → readyOrigin → withTimeout(…, readyTimeoutMs ?? 15000) 有界 15s,稳态(sid 已匹配走快路径)零额外延迟。均无无意回归。唯一收窄面已在 issue 披露:若某真实边界只在既有对象 request #2+ 才可见则不再新锁存——但后果仅为模型撞死 ref 直到熔断(#4305 原症状,自限),远轻于误锁存的 p1 代价,取舍合理。
6. 是否破坏缓存:方向相反——本 PR 修复的就是缓存失效;note 文本未变、只是更少误附加,前缀更稳。无破坏。
7. 是否改了设计方向:无架构转向,两处均为局部守卫 / 时序修正。
8. 是否引入了新的配置:无新 env/config/默认值(config-surface-guard、version-guard 均绿佐证)。
测试有效性:三条新用例都有牙——system-skip 用例(fixture requests=0,去掉 skip 会锁存 true)、mid-session 用例(requests=6,去掉 >1 守卫会锁存 true)、before_agent_start 用例(PR 前该 handler 不存在,assert.ok(beforeAgentStart) 直接红);makeFakePi 改为 await 前置 handler 返回的 promise,严格更正确。CI 30 checks 全绿(kernel-guard/config-surface-guard/version-guard/lane-guard/gate 均 success)。
判定:可合。
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 复核完成 — APPROVE(GitHub PR #5497,review id 5730319239)。 独立复核结论(非转述 issue):
判定:可合。 |
|
CI on head This failure comes from upstream
I will merge |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 独立核对确认:这次红是继承自 master 的 golden 漂移,与本 PR 无关,APPROVE 结论不受影响。 我的独立验证(非转述你的结论):
处置建议(分支归你管,仅供参考):既然 golden 重生成已落在 |
Summary
Fixes a mid-session prompt cache miss (and Anthropic thinking-block
prefix_binding_mismatchdrops) triggered on Turn 2 of cold-startedpisessions or when mid-session tool outputs containm\d{4,}strings:Guard
detectGenerationBoundary(Switching model mid-conversation restarts the ref ledger, but the rendered transcript still shows the old refs — the model keeps aiming compress at dead ranges until the circuit breaker fires | 切模型后消息编号整体换账本,但渲染历史仍显示旧编号,模型对着死区间反复压缩直至熔断 #4305) against mid-session androle: "system"false positives (src/generation-boundary.ts)detectGenerationBoundarypreviously ran its initial stale-ref scan across all messages (includingrole: "system") on every request (session.stats.requests > 1included).piemitted an in-bandrole: "system"patch (Updated system prompt section "codemode_catalog") on Turn 2,bili's owncompresstool description inside<codemode_catalog>("each block starts with its 'm00150–m00220 optional topic' header line") matchedm00150andm00220above the session frontier (m00017). Similarly, mid-session tool outputs containing twom\d{4,}tokens above the frontier triggered the detector mid-conversation.session.metadata.generationBoundarymid-session appendedGENERATION_BOUNDARY_NOTE(562 chars) tosystem[1], invalidating the upstream prefix cache for all messages aftersystem[0]and invalidating the cryptographic prefix binding of all prior signed Anthropicthinkingblocks (thinking_dropped: prefix_binding_mismatch).(session.stats?.requests ?? 0) <= 1, when a post-switch session generation actually starts atm00001with carried-over history), and skipm.role === "system"messages.Await
registerToolsinbefore_agent_start(src/agent/pi.ts)session_startcallsvoid registerTools(pi, ctx, state, agent, false)unawaited, while the awaitedregisterToolscalls only lived inbefore_provider_headersandbefore_provider_request.pi,_prepareRunemitsbefore_agent_startand then freezes Turn 1's active tool loadout and<codemode_catalog>system prompt beforebefore_provider_headers/before_provider_requestfire. On a cold native-proxy boot,registerToolscompleted insidebefore_provider_requestafter Turn 1's<codemode_catalog>was already built without the ACP tools, causingpito emit an in-bandUpdated system prompt section "codemode_catalog"delta on Turn 2.registerTools(pi, ctx, state, agent)inbefore_agent_start(idempotent andsid-cached) so the ACP tools are registered before Turn 1's tool loadout and system prompt are built.Verification
tests/generation-boundary.test.tscovering in-bandrole: "system"messages withm00150–m00220and mid-session turns (stats.requests > 1).tests/plugin-agent.test.tsverifyingbefore_agent_startawaitsregisterToolsand populatespi.toolsbefore Turn 1's prompt preparation.npx tsc --noEmit,npm run build, and test suites pass.Fixes #5498