diff --git a/docs/technical-notes.md b/docs/technical-notes.md index c3ce68259d..7e343bc781 100644 --- a/docs/technical-notes.md +++ b/docs/technical-notes.md @@ -592,7 +592,7 @@ The lane now carries a watermark. Every successful plugin-lane chat-completions Relationship to the incremental lane (#3248/#4491) — SAFE TO RUN TOGETHER, they never double-process: both mechanisms attack the same pain point (plugin hosts re-sending full history) but partition by REQUEST SHAPE per turn. The dsh shim's divert only engages for ANTHROPIC-WIRE bodies (`wireMessagesLookAnthropic`; OpenAI tool shapes `role:"tool"`/`tool_calls` always ride full resend, #3433) — and #5403's shape is exactly the opposite wire (chat-completions / openai profile), where the incremental lane never diverts at all. So the two lanes are complements, not competitors: anthropic-wire plugin sessions are served by the incremental lane; OpenAI chat-completions plugin sessions by the watermark fastpath. A single request walks EXACTLY ONE path (the shim chooses per turn); server-side, any request carrying the `x-bili-plugin-turn` marker skips watermark evaluation entirely (no mixed stamping — the `incrementalTurn` row of the gate matrix), and diverted responses never carry `x-bili-pf-seq` (they carry `x-bili-applied-seq` instead). The synthetic re-entry request the lane builds carries an explicit header allowlist (`PASSTHROUGH_HEADERS`) with no pf-* names, so handshake headers cannot leak via that path either. Residual harmless case: in a mixed session a stale `x-bili-pf-count` may accompany one non-diverted turn — it costs at most ONE verification round, then the response re-stamps the truth. -Identity channel v2/v3 (the owner ladder msgid > order > hash, #11): a bare count pins the boundary only by POSITION — a client that rewrites elements while keeping the count constant stays invisible. When OUR shim is in the loop, the request additionally carries `x-bili-pf-tail-ids`: the host-side message id of element K−1 (the previous turn's anchor) plus every NEW element — “几个,不是几十个”. Hosts do not put ids in chat-completions bodies, but the injected extension reads them from host context (per-element rules shared with the incremental envelope, #4491) — so no host release is needed. Wire form is per-segment percent-encoding (`ids.map(encodeURIComponent).join(",")`): separator commas stay literal, commas inside an id travel as %2C, so the server's split-then-decode is unambiguous for any id alphabet. **No alignment proofs (owner ruling 2026-11-02):** hosts compress and REWRITE their histories, so proof-gating fights reality and killed the channel — msgids are unique forever, ORDER binds them inside the verified delta, and the byte-level head check is the boundary truth. Capture is therefore PER-SLOT: an element with no usable host id (native-compaction summary `acp_summary_*`, persona sidecar, foreign shape, duplicate id) rides the unknown-slot placeholder `?` (`%3F` on the wire) instead of poisoning its neighbours; only a wholly unresolvable list clears storage. Server verdicts: bad length / duplicate real id / undecodable segment ⇒ UNcredited silent degrade of THIS turn — never toward the breaker (shim↔proxy drift self-heals through the legacy path + restamp); prevId ≠ stored anchor ⇒ UNcredited, checked ONLY when BOTH sides are known ids — a `?` on either side means no identity claim (no penalty). Absent header ⇒ pure count mode; every pre-v2 behavior is byte-for-byte unchanged. Lane coverage: the channels are protocol-unified — lanes differ ONLY in the thin capture adapter that resolves THIS request's host ids (owner rule: between clients, "where the ids are read" may differ, nothing else): dsh reads them off the host's dispatch context per slot (`pfSlotIdsOf` — `?` placeholders, above), pi walks the session branch entries against the outgoing body family-by-family (anthropic `laneMsgIdsForWire`, openai-chat `laneMsgIdsForOpenAiWire`), opencode reuses the validated list its messages-transform hook already aligns for the incremental envelope (#4491), omp the fetch-layer alignment of the branch entries the host stashes in `before_provider_request` against the ACTUALLY-sent body (`ompTurnMsgIds` — the same copy the envelope rides) on **anthropic-wire turns only**: its strict count+role proof cannot bridge the config-assembled leading system run, and a failure-clear there would ERASE the shared hook's list for the same conversation id (the hook runs first, this planner last) — openai-wire turns are covered by the shared `laneMsgIdsForOpenAiWire` resolver alone (#2583 per-model egress). The pi/opencode/omp walkers are all-or-nothing and a failed resolution CLEARS the stored list (for them a failure means the POSITIONAL knowledge itself is missing, so clearing is the honest move) — an unresolvable request degrades to count mode rather than riding a stale anchor; dsh's per-slot capture is exempt (above). One protocol note: chat-completions bodies start with system|developer elements assembled from config/state that own NO branch entry; the openai resolver covers that leading run with position-pinned synthetic tokens (`bili-ps-` — stable per slot, charset-safe, cannot collide with an 8-hex uuid entry id), so the chain stays alive instead of dying on the first system element, and a moved slot breaks the anchor exactly like a real edit (uncredited, self-healing). +Identity channel v2/v3 (the owner ladder msgid > order > hash, #11): a bare count pins the boundary only by POSITION — a client that rewrites elements while keeping the count constant stays invisible. When OUR shim is in the loop, the request additionally carries `x-bili-pf-tail-ids`: the host-side message id of element K−1 (the previous turn's anchor) plus every NEW element — “几个,不是几十个”. Hosts do not put ids in chat-completions bodies, but the injected extension reads them from host context (per-element rules shared with the incremental envelope, #4491) — so no host release is needed. Wire form is per-segment percent-encoding (`ids.map(encodeURIComponent).join(",")`): separator commas stay literal, commas inside an id travel as %2C, so the server's split-then-decode is unambiguous for any id alphabet. The emitted value is byte-capped (2 KiB, #7212): when the window would exceed it — the learned stamp lagging far behind the current element count, e.g. after a small side request overwrote the stamp — the header is dropped for that turn (count only); the proxy degrades to the legacy full-parse and re-stamps the true count, so the window self-heals next turn, instead of tripping Node's 16 KiB `maxHeaderSize` and dropping the connection (which killed the whole session, unrecoverable by retry). **No alignment proofs (owner ruling 2026-11-02):** hosts compress and REWRITE their histories, so proof-gating fights reality and killed the channel — msgids are unique forever, ORDER binds them inside the verified delta, and the byte-level head check is the boundary truth. Capture is therefore PER-SLOT: an element with no usable host id (native-compaction summary `acp_summary_*`, persona sidecar, foreign shape, duplicate id) rides the unknown-slot placeholder `?` (`%3F` on the wire) instead of poisoning its neighbours; only a wholly unresolvable list clears storage. Server verdicts: bad length / duplicate real id / undecodable segment ⇒ UNcredited silent degrade of THIS turn — never toward the breaker (shim↔proxy drift self-heals through the legacy path + restamp); prevId ≠ stored anchor ⇒ UNcredited, checked ONLY when BOTH sides are known ids — a `?` on either side means no identity claim (no penalty). Absent header ⇒ pure count mode; every pre-v2 behavior is byte-for-byte unchanged. Lane coverage: the channels are protocol-unified — lanes differ ONLY in the thin capture adapter that resolves THIS request's host ids (owner rule: between clients, "where the ids are read" may differ, nothing else): dsh reads them off the host's dispatch context per slot (`pfSlotIdsOf` — `?` placeholders, above), pi walks the session branch entries against the outgoing body family-by-family (anthropic `laneMsgIdsForWire`, openai-chat `laneMsgIdsForOpenAiWire`), opencode reuses the validated list its messages-transform hook already aligns for the incremental envelope (#4491), omp the fetch-layer alignment of the branch entries the host stashes in `before_provider_request` against the ACTUALLY-sent body (`ompTurnMsgIds` — the same copy the envelope rides) on **anthropic-wire turns only**: its strict count+role proof cannot bridge the config-assembled leading system run, and a failure-clear there would ERASE the shared hook's list for the same conversation id (the hook runs first, this planner last) — openai-wire turns are covered by the shared `laneMsgIdsForOpenAiWire` resolver alone (#2583 per-model egress). The pi/opencode/omp walkers are all-or-nothing and a failed resolution CLEARS the stored list (for them a failure means the POSITIONAL knowledge itself is missing, so clearing is the honest move) — an unresolvable request degrades to count mode rather than riding a stale anchor; dsh's per-slot capture is exempt (above). One protocol note: chat-completions bodies start with system|developer elements assembled from config/state that own NO branch entry; the openai resolver covers that leading run with position-pinned synthetic tokens (`bili-ps-` — stable per slot, charset-safe, cannot collide with an 8-hex uuid entry id), so the chain stays alive instead of dying on the first system element, and a moved slot breaks the anchor exactly like a real edit (uncredited, self-healing). A turn is a HIT when: the window origin K is settled and the in-process seed (previous turn's canonical cores, module-level `WeakMap`, deliberately outside the Session object so the panel breakdown stays honest about what is session truth) was stamped at exactly that count under the same model. **v4 count relaxation (owner ruling 2026-11-02):** once a turn arrives with a shape-valid `x-bili-pf-tail-ids` list whose width matches the server's own bookkeeping (`len = T − K + 1`, re-verified SERVER-side), K comes from the LEDGER (`seed.stampT`) and the echoed `x-bili-pf-count` demotes to diagnostic — a missing or stale/desynced echo no longer produces a miss (the decision string leaves a trace as `hit(no-count)` / `hit(count-off=N)`, never breaker fuel); without such a list the pre-v4 ladder runs byte-for-byte unchanged (echo must equal `stampT`, strictly less than T). The first `HEAD_CHECK_ELEMENTS` (8) elements re-convert to a byte-identical system run, and the window head is not a system/developer element (a hoisted system would desync `reconcileSystemAnchor`). On a hit the pipeline receives `[seed cores] + [freshly parsed window cores [K..T)]` instead of a full parse, and `snapshotMessages` pins ONLY the tail window. Window cores are re-keyed against the prefix because `openaiToCore` mints cluster ids per intra-call occurrence order (`h_` bases with `_N` suffixes) and stamps `wireIndex` from the array position it was given — a repeated content base must continue its ordinal from the stashed prefix, or the ledger's identity contract breaks; with K=1 an un-rekeyed window would even MERGE two distinct wire messages (prune extends first-user-pin runs on equal wireIndex). diff --git a/docs/technical-notes.zh-CN.md b/docs/technical-notes.zh-CN.md index f051d78c66..fb97c1b91b 100644 --- a/docs/technical-notes.zh-CN.md +++ b/docs/technical-notes.zh-CN.md @@ -169,7 +169,7 @@ chat wire 上 bili 自己 vs 客户端自带:插件车道上,请求体里等于 **与增量车道(#3248/#4491)的关系——可同开、互不双处理**:两机制打的是同一个痛点(插件宿主全量重发),但按**请求形状逐回合分轨**。dsh shim 的 divert 只接 **anthropic wire** body(`wireMessagesLookAnthropic`;OpenAI tool 形态 `role:"tool"`/`tool_calls` 恒走全量重发,#3433)——而 #5403 恰是相反的线(chat-completions / openai profile),增量车道在那种会话里从不发散。所以两条是互补不是竞争:anthropic-wire 插件会话归增量车道,OpenAI chat-completions 插件会话归水位快路径。单个请求恒走其中一条路(shim 逐回合二选一);服务端对带 `x-bili-plugin-turn` 标记的请求整体跳过水位判定(不打混合标),divert 响应不盖 pf-seq(盖的是 `x-bili-applied-seq`);lane 重入的合成请求头上游用显式白名单(`PASSTHROUGH_HEADERS`),不含 pf-* → 握手头经这条路也漏不出去。残留无害情形:混用会话里陈旧 `x-bili-pf-count` 可能多陪跑一次验证轮,响应当轮重盖真值即收敛。 -身份通道 v2/v3(owner 层级 msgid>顺序>hash,#11):裸 count 只按**位置**钉边界——保元素数改写的客户端不可见。我们的 shim 在环时,请求额外携带 `x-bili-pf-tail-ids`:元素 K−1(上一轮锚点)的宿主 msgid + 全部新增元素——「几个,不是几十个」。宿主不在 chat-completions body 里发 id,但我们注入的扩展能从宿主上下文读到(与增量信封同一套逐元素规则,#4491),因此不需要宿主发版。线上形态为**逐段**百分号编码(`ids.map(encodeURIComponent).join(",")`):分隔逗号保持字面量、id 内部的逗号走 %2C,服务端「先拆后解」对任意 id 字母表都无歧义。**不做对齐证明**(owner 裁定 2026-11-02:宿主本就压缩/改写历史,证明门槛与现实对抗、反而杀死通道——msgid 永不复用,顺序在已验证增量窗内绑定身份,字节级头部比对才是边界真值)。采集因此是**逐槽**的:拿不到可用宿主 id 的元素(native-compaction 摘要 `acp_summary_*`、persona 副车、外来形状、重复 id)随未知槽占位符 `?`(线上 `%3F`)旅行,不毒害邻居;只有整份都解析不出来才清空存储。判决:长度不符/真实 id 重复/坏段 ⇒ **静默降级本轮、不计熔断**(shim↔代理失步靠 legacy 路径+重盖戳自愈);prevId ≠ 存锚 ⇒ UNcredited,且仅当**两侧都是已知 id**时才校验——任一侧是 `?` = 无身份主张(不惩罚)。缺头 ⇒ 纯计数模式,v2 之前的一切行为逐字不变。车道覆盖是**协议统一**的(owner 裁定:客户端之间只允许「在哪拿到 id」不同,其余一律归一):dsh 从宿主 dispatch 上下文逐槽直读(`pfSlotIdsOf`,`?` 占位,见上);pi 用共享 entry walker 按出体族对齐 session branch 条目(anthropic `laneMsgIdsForWire`,openai-chat `laneMsgIdsForOpenAiWire`);opencode 复用其 messages-transform 钩子为增量信封已对齐的同一份验证列表(#4491),omp 用 fetch 层「宿主 `before_provider_request` 暂存 branch 条目 vs 实际出体」的对齐结果(`ompTurnMsgIds`,与信封共用同一份;**仅跑 anthropic wire 回合**——其严格 count+role 证明桥不过 config-assembled 前导 system 段,失败清空会抹掉共享钩子对同一会话刚落下的列表;openai 形回合由共享 `laneMsgIdsForOpenAiWire` 解析器单独覆盖,#2583)。pi/opencode/omp 的 walker 解析器仍是全有全无、整份失败即清空存储列表(它们的失败意味着**位置知识本身缺失**,清空是诚实的)——无法证明的请求降级回纯计数,绝不骑陈旧锚点;dsh 逐槽采集不受此限(见上)。一条协议注记:chat-completions 的 body 以前导 system|developer 元素开头,它们由 config/state 组装、**没有 branch entry**;openai 解析器用位置钉死的合成令牌(`bili-ps-`——槽位稳定、字符安全、不可能与 8-hex uuid entry id 混淆)覆盖该前导段,链不会死在第一个 system 元素上,槽位挪动则像真实改写一样诚实断锚(UNcredited,自愈)。 +身份通道 v2/v3(owner 层级 msgid>顺序>hash,#11):裸 count 只按**位置**钉边界——保元素数改写的客户端不可见。我们的 shim 在环时,请求额外携带 `x-bili-pf-tail-ids`:元素 K−1(上一轮锚点)的宿主 msgid + 全部新增元素——「几个,不是几十个」。宿主不在 chat-completions body 里发 id,但我们注入的扩展能从宿主上下文读到(与增量信封同一套逐元素规则,#4491),因此不需要宿主发版。线上形态为**逐段**百分号编码(`ids.map(encodeURIComponent).join(",")`):分隔逗号保持字面量、id 内部的逗号走 %2C,服务端「先拆后解」对任意 id 字母表都无歧义。发出的值有**字节上限**(2 KiB,#7212):当窗口会超上限时——学到的戳 k 远落后于当前元素数 T(例如一次小请求的小 seq 覆盖了戳)——该轮**丢弃** tail-ids 头(只发 count);代理降级到 legacy 全量解析并重盖真实计数,下一轮窗口自愈——而不是撞穿 Node 的 16 KiB `maxHeaderSize` 把连接丢掉(那会让整段会话永久不可用,重试也救不回)。**不做对齐证明**(owner 裁定 2026-11-02:宿主本就压缩/改写历史,证明门槛与现实对抗、反而杀死通道——msgid 永不复用,顺序在已验证增量窗内绑定身份,字节级头部比对才是边界真值)。采集因此是**逐槽**的:拿不到可用宿主 id 的元素(native-compaction 摘要 `acp_summary_*`、persona 副车、外来形状、重复 id)随未知槽占位符 `?`(线上 `%3F`)旅行,不毒害邻居;只有整份都解析不出来才清空存储。判决:长度不符/真实 id 重复/坏段 ⇒ **静默降级本轮、不计熔断**(shim↔代理失步靠 legacy 路径+重盖戳自愈);prevId ≠ 存锚 ⇒ UNcredited,且仅当**两侧都是已知 id**时才校验——任一侧是 `?` = 无身份主张(不惩罚)。缺头 ⇒ 纯计数模式,v2 之前的一切行为逐字不变。车道覆盖是**协议统一**的(owner 裁定:客户端之间只允许「在哪拿到 id」不同,其余一律归一):dsh 从宿主 dispatch 上下文逐槽直读(`pfSlotIdsOf`,`?` 占位,见上);pi 用共享 entry walker 按出体族对齐 session branch 条目(anthropic `laneMsgIdsForWire`,openai-chat `laneMsgIdsForOpenAiWire`);opencode 复用其 messages-transform 钩子为增量信封已对齐的同一份验证列表(#4491),omp 用 fetch 层「宿主 `before_provider_request` 暂存 branch 条目 vs 实际出体」的对齐结果(`ompTurnMsgIds`,与信封共用同一份;**仅跑 anthropic wire 回合**——其严格 count+role 证明桥不过 config-assembled 前导 system 段,失败清空会抹掉共享钩子对同一会话刚落下的列表;openai 形回合由共享 `laneMsgIdsForOpenAiWire` 解析器单独覆盖,#2583)。pi/opencode/omp 的 walker 解析器仍是全有全无、整份失败即清空存储列表(它们的失败意味着**位置知识本身缺失**,清空是诚实的)——无法证明的请求降级回纯计数,绝不骑陈旧锚点;dsh 逐槽采集不受此限(见上)。一条协议注记:chat-completions 的 body 以前导 system|developer 元素开头,它们由 config/state 组装、**没有 branch entry**;openai 解析器用位置钉死的合成令牌(`bili-ps-`——槽位稳定、字符安全、不可能与 8-hex uuid entry id 混淆)覆盖该前导段,链不会死在第一个 system 元素上,槽位挪动则像真实改写一样诚实断锚(UNcredited,自愈)。 命中条件:窗口起点 K 已定局、进程内种子(上一轮的 canonical cores,模块级 `WeakMap`,刻意放在 Session 对象之外,让面板字段分解如实反映会话真相)恰在该 count、同模型下盖过戳。**v4 计数降级(owner 裁定 2026-11-02)**:某轮带形状合格且宽度与服务端账本吻合(`len = T − K + 1`,服务端重验)的 `x-bili-pf-tail-ids` 列表时,K 取自**服务端账本**(`seed.stampT`),回带的 `x-bili-pf-count` 该轮降级为纯诊断——缺失或错乱的回显不再产生 miss(决策串留痕 `hit(no-count)` / `hit(count-off=N)`,永不喂熔断);无合格列表时 pre-v4 阶梯逐字节不变(回带值须等于 `stampT` 且严格小于 T)。前 8 个元素重转后 system 逐字一致、窗口首非 system/developer(hoist 会失步 `reconcileSystemAnchor`)。命中时管道拿到的是「[种子 cores] + [尾窗 [K..T) 现解析的 cores]」而非全量解析,且 `snapshotMessages` 只钉尾窗。尾窗 cores 必须按前缀重新编号:`openaiToCore` 铸造 cluster id 按调用内出现次序(`h_` 基名加 `_N` 后缀)、`wireIndex` 取自所给数组的位置——前缀里出现过同样内容基名的,窗口这边必须接上序号,否则账本身份契约就断;K=1 时不重编号甚至会把两条不同的 wire 消息**合并**成一条(prune 的 first-user-pin 在同一 wireIndex 上扩展 run)。 diff --git a/src/agent/pf-channel.ts b/src/agent/pf-channel.ts index 104c33635f..5199eb9b4b 100644 --- a/src/agent/pf-channel.ts +++ b/src/agent/pf-channel.ts @@ -114,6 +114,18 @@ export function pfCaptureIds(sid: string, ids: unknown): void { slot.ids = known > 0 ? out : undefined; } +// #7212: the tail window is (T - k + 1) ids — UNBOUNDED when the learned stamp +// k lags far behind the current element count T (a side request with a smaller +// seq overwrites k). Node's default maxHeaderSize is 16 KiB for the WHOLE +// header block; an oversized x-bili-pf-tail-ids trips it and the parser drops +// the connection, killing the session (each retry re-sends the same oversized +// header, so it never recovers). Cap the encoded VALUE: over the budget, drop +// the tail-ids header (count only) — the proxy degrades to the legacy full-parse +// for this turn and restamps the true T, so the window self-heals next turn. +// The cap sits far under 16 KiB, leaving room for the other headers in the +// block. encodeURIComponent output is pure ASCII, so .length == byte length. +const MAX_TAIL_IDS_BYTES = 2048; + /** Chokepoint emit (native-intercept routed-send site): the handshake headers * this request should carry, given its outgoing header record (source of the * conversation id). {} whenever the session never saw a stamp yet. */ @@ -130,7 +142,8 @@ export function pfRequestHeaders(incoming: Record): Record = { [COUNT_HEADER]: String(k) }; const ids = slot.ids; if (ids !== undefined && ids.length >= k) { - out[TAIL_IDS_HEADER] = ids.slice(k - 1).map(encodeURIComponent).join(","); + const tail = ids.slice(k - 1).map(encodeURIComponent).join(","); + if (tail.length <= MAX_TAIL_IDS_BYTES) out[TAIL_IDS_HEADER] = tail; } return out; } diff --git a/tests/pf-channel.test.ts b/tests/pf-channel.test.ts index 95a3ea3679..edca1e289a 100644 --- a/tests/pf-channel.test.ts +++ b/tests/pf-channel.test.ts @@ -43,6 +43,33 @@ test("pf-channel: count mode; tail-ids ride only a validated list; encoding", () assert.deepEqual(pfChannel.pfRequestHeaders({ [CONVO]: sidShort }), { "x-bili-pf-count": "2" }); }); +test("pf-channel: oversized tail window drops the tail-ids header, never the count (#7212)", () => { + const sid = "sid-overflow-1"; + const inH = { [CONVO]: sid }; + // 13-char ids; a lagging stamp k=2 against T=450 ⇒ a 449-id window of + // ≈6.5 KB — over the 2 KiB cap this test pins. (The production overflow was + // the same shape at a much wider lag, 16+ KiB, tripping Node's 16 KiB + // maxHeaderSize and dropping the connection.) + learn(inH, "2"); + const ids = Array.from({ length: 450 }, (_, i) => `id-${i.toString(36).padStart(8, "0")}-${(i % 97).toString(16)}`); + pfChannel.pfCaptureIds(sid, ids); + const out = pfChannel.pfRequestHeaders(inH); + assert.equal(out["x-bili-pf-count"], "2", "count still rides (pure count mode)"); + assert.equal(out["x-bili-pf-tail-ids"], undefined, "oversized window ⇒ no tail-ids header"); + // the encoded window is genuinely over the budget (guards the test itself) + const full = ids.slice(1).map(encodeURIComponent).join(","); + assert.ok(full.length > 2048, `fixture must exceed the cap (got ${full.length}B)`); + // a tight window (k ≈ T) still rides the full tail-ids list + const sidTight = "sid-overflow-tight"; + learn({ [CONVO]: sidTight }, "448"); + pfChannel.pfCaptureIds(sidTight, ids); + assert.equal( + pfChannel.pfRequestHeaders({ [CONVO]: sidTight })["x-bili-pf-tail-ids"], + ids.slice(447).map(encodeURIComponent).join(","), + "small window ⇒ tail-ids intact", + ); +}); + test("pf-channel: failed whole-resolution CLEARS the stored list (no stale slice)", () => { const sid = "sid-clear-1"; const inH = { [CONVO]: sid };