Skip to content

[PR] msgid: dsh full-resend turns onto the biliMsgIds envelope (#6525 step 1) - #6532

Closed
ranxianglei wants to merge 2 commits into
masterfrom
2026-11-13_dsh-envelope-msgids
Closed

ranxianglei wants to merge 2 commits into
masterfrom
2026-11-13_dsh-envelope-msgids

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

#6525 第一步:dsh 全量重发轮接上 biliMsgIds 信封通道

一句话

dsh 的主力流量(openai wire 全量重发)每轮都不带宿主消息号,导致存储层永远用内容摘要当身份、fork 的 msgid 身份轨(#6293)在真实 dsh 上永远匹配不上;这个 PR 把 dsh 的全量重发请求接上现成的信封通道(biliMsgIds),让宿主号从第一轮就能落进存储。

背景(为什么这么惨)

msgid 基建(docs/msgid-infrastructure.md)设计了四条递送通道,服务端消费侧全部就绪(两 wire 的 prepare 都会读 biliMsgIds 并在 gate 打开时 pin 到 messages.msg_id),但信封通道①从来没有客户端写过(issue #6525 的覆盖矩阵)。dsh 的送号只长在 anthropic wire 的增量车道上(#4491),而 dsh 走 openai-completions 时走的是全量重发——每轮都裸奔。结果:

怎么做的

src/agent/dsh-native.ts 新增 maybeStampEnvelopeMsgIds(beforeSend 链,增量车道之后:车道换过 body 就不碰)与纯函数 resolveEnvelopeMsgIds:

  1. 配对必须被证明,绝不按位置假设——复用 pi/omp 车道 pin 用的同一对 pi-ai lockstep walker(laneMsgIdsForWire / laneMsgIdsForOpenAiWire,src/agent/pi-msgids.ts):按顺序保持的指纹把 wire 元素和宿主消息逐一对上,任何对不上 → 整轮 bail(全有或全无)。
  2. core 对齐由内核自己的转换器证明——openaiToCore/anthropicToCore 的 The intermediate message (core) is not protocol-agnostic: switching protocols/models causes identity churn; cross-protoc | 中间消息(core)没有做到协议无关:切协议/切模型制造身份 churn,补齐跨协议映射 #3642 wireIndex 回指:每个非 leading wire 元素必须恰好拥有 1 个 core(并行 tool_calls、reasoning+text、图文混排会把一条宿主消息拆成 N 个 core,N 个 core 不能共用一个宿主号,validateMsgIds 也拒重复);拆了就整轮 bail。
  3. system/developer 条目排除:它们走 hoisted/folded 的 leading system(openai wire 上是 bili-ps-N 槽位、anthropic 上是独立 system 字段),永不与 1:1 的非 leading wire 元素对齐;openai wire 的 leading 槽位从信封里剥掉,保证信封长度 == 服务端校验的 post-hoist core 数。
  4. 任何 bail = body 字节不动 → 内容摘要,与今天行为完全一致。不新增校验副本(msgid-rule-parity 的六副本矩阵不变)。

服务端零改动:字段本来就被 prepare-openai/anthropic 读取、转发前剥除、pin 受 pluginMode + pluginLaneServerHistory rollout 门控(默认 0%,owner 裁定 2026-11-07)。默认安装的存量行为一字不变(rollout 关时字段被 strip,不 pin)。

验证

  • npm run typecheck 干净。
  • 新增 tests/issue6525-dsh-envelope.test.ts 13 例(真实内核 + 真实 walker):openai lead 去头 / 无 lead / 图文混排 bail(内核 wireIndex 证明独占)/ 并行 tool_calls bail / reasoning_content bail / 非 lead system bail / 宿主缺号 bail / 字节不符 bail / 重复 id bail / anthropic 1:1(system 字段分离 + tool_result 块)/ 非 openai-anthropic payload / 退化输入不抛。
  • 回归:issue6525 / msgid-rule-parity / incremental-client / dsh-fork-adopt / pi-omp-fork-adopt / plugin-fork 六套件 117/117。
  • 真机 dsh e2e(dsh 0.2.0-rc.2 headless,假上游,400 行 filler + acp_status×3 + compress 的完整父会话,脚本 dsh-envelope-e2e.mjs):
    • A0 父会话完成;A1 R1 带 x-bili-plugin: dsh 戳;
    • A2 开 BILI_ROLLOUT_PLUGIN_LANE_SERVER_HISTORY=1 后,存储出现宿主 UUID pin 行(第一个非 watermark 请求必 pin, stamped 的宿主号确实落进 messages.msg_id)——链路端到端打通;
    • A3 bili.log 一次性 stamp 行(msgid envelope stamped on full-resend for <session> (N ids, openai wire))。

边界(不在本 PR 内)

相关

Carrier ① (the envelope body field) had zero client wiring: the server
consumes biliMsgIds on both wires (prepare-openai/anthropic, gated on
pluginMode + the pluginLaneServerHistory rollout), but no client ever
stamped it. dsh's stamping lived only on the anthropic-wire incremental
lane (#4491), so every openai-wire full-resend turn — dsh's main path —
arrived id-less and the store fell back to content-derived digests,
which in turn leaves the fork msgid identity track (#6293) with nothing
to match.

maybeStampEnvelopeMsgIds now stamps full-resend model bodies (both
wires) after the incremental lane had its chance. Resolution never
re-implements the host's wire transforms: the pi-ai lockstep walkers
(laneMsgIdsForWire / laneMsgIdsForOpenAiWire — the SAME functions the
pi/omp lanes pin with) prove wire-element ⇔ host-message pairing by
order-preserving fingerprints, bail all-or-nothing; the kernel's own
converters then prove core-alignment via the #3642 wireIndex
back-pointer (every non-leading wire element must own exactly one core
— parallel tool calls / reasoning+text / text+image parts split one
host message into N cores, and N cores cannot share one host id).
System-role entries are excluded (they ride the hoisted/folded leading
system, never 1:1 to a non-leading wire element). Any bail leaves the
body byte-identical → content-derived ids, exactly today's behavior;
with the rollout off the server strips the field, so default installs
are untouched.
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-13_dsh-envelope-msgids (ba30a95)

Latest build — install this exact version

npm install -g billion-context@0.0.9574-pr.6532.a2035dd

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
ba30a95 billion-context@0.0.9574-pr.6532.a2035dd build
57ae7ab billion-context@0.0.9535-pr.6532.041e1b1 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6532.tgz
npm install -g package

Install instructions are refreshed on each push.

#6525 B7)

dsh's flat host messages differ from the pi-ai shape the shared walker's
host-side fingerprint assumes in two places (role tool vs toolResult,
block tool-call vs toolCall), so every tool-bearing turn bailed the
all-or-nothing envelope and the turn-1 host-pinned rows churned back to
content digests on the next full-resend. Normalize dsh's shape for
fingerprinting only, in the dsh resolver: the envelope ids still come from
mm.id and the wire-side pairing is untouched, so no mis-pin is possible.

Found via a real-dsh e2e dump (after turn 1: live rows 11/11 content
digests). Now: live rows 11/11 host-pinned UUIDs. Two regression tests pin
the accept (a dsh tool turn core-aligns and stamps) and the retained bail
(parallel tool_calls still refuse, one host id cannot cover N cores).
@ranxianglei

Copy link
Copy Markdown
Owner Author

已收编进统一 PR #6564(提案 #6539 一个 PR 落地,用户要求不拆分)。本 PR 关闭;合并留 owner。

ranxianglei added a commit that referenced this pull request Nov 14, 2026
…6539)

Complements the client-side envelope wiring (#6532 shipped dsh openai+
anthropic). This is the 'record' half of the msgid lifecycle made whole
across every wire:

- prepare-google / prepare-responses: read + strip the biliMsgIds envelope
  before it can reach the upstream, then pin behind the same rollout gate
  as the openai/anthropic twins. A length/shape mismatch is rejected
  (bail) at the pin — never leaked, never mis-pinned.
- handle.ts: a non-body id source (x-bili-msg-ids header / loopback
  harvest) now stamps the parsed body on EVERY wire, not just
  openai/anthropic, so the four pin-consumers all see it uniformly.
- rollout: pluginLaneServerHistory 0% -> 100% (GA, owner ruling
  2026-11-13). No client calls the co-gated /__bili/plugin/render
  endpoint yet, so flipping only activates pinning. Roll back with
  BILI_ROLLOUT_PLUGIN_LANE_SERVER_HISTORY=0.

Three tests that pinned the old default-off contract now force the off
state explicitly (env=0) instead of deleting the var.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant