Skip to content

[PR] msgid Delivery: Proposal #6539 Full Implementation (Publish·Record·Use, dsh/pi/omp/opencode × Four Protocols) Consolidat | msgid 递送:提案 #6539 完整落地(发·记·用,dsh/pi/omp/opencode × 四协议)合成一个 PR - #6564

Closed
ranxianglei wants to merge 11 commits into
masterfrom
2026-11-14_msgid-proposal-6539

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

这是啥

msgid 递送提案 #6539(discussion)的完整落地,合成一个 PR。之前拆成了 5 个零散 PR,按用户要求收成一个。

一句话:让 dsh / pi / omp / opencode 四个宿主的会话,在四条协议上,把稳定宿主消息号(host id)真正发上去 → 服务端记下并 pin 住 → 快照暴露 → fork/继承能走 msgid 身份轨。 之前只有 dsh 的 anthropic 增量车道做得到,其余全是「发不上去」或「记不住」或「对不齐」。

覆盖矩阵(收 · 用 · 发)

阶段 做了什么 涉及文件
收(服务端记) 四条 wire(openai/anthropic/google/responses)全 pin + rollout pluginLaneServerHistory GA 100% src/server/handle.ts, prepare-google.ts, prepare-responses.ts, src/rollout.ts
用(消费) fork 协议加 msgid 身份轨:快照暴露父 live store 的 msg_id 序列,fork 带 prefixMsgIds+driftFrom,门=rawId 相等(宿主 pin 住后 rawId 就是宿主号);漂移→rekey;字节轨不变 src/plugin.ts, src/agent/fork-adopt.ts
发·dsh 全量重发轮接 biliMsgIds 信封(openai+anthropic),内核 wireIndex 证 core 对齐;修 B7 churn(dsh 扁平 host 形状 role:"tool"/块 type:"tool-call" 与共享 walker 的 pi-ai 形不符→每个工具轮都 bail→宿主号 churn 成摘要) src/agent/dsh-native.ts
发·pi/omp 复用现成已校验 walker,全量重发路由到信封,destinationRoutedThroughProxy 门(盲隧道永不戳,#1403) src/agent/pi.ts, pi-msgids.ts
发·opencode V2 core-align:跳过被内核 hoist 的开头 system/developer 段,头数= post-hoist core 数(V2 之前在 openai 线从没真正 pin 住过) src/agent/opencode-v2-msgids.ts

关键设计纪律

验证(整合分支上跑)

  • typecheck 干净。
  • 单元/集成:fork-msgid-track 8、dsh-envelope 15、pi-omp-envelope 23、opencode-v2-msgids 10、msgid-pin 9、msgid-rule-parity 5、msgid-lane-flow 6、openai-msgid-pin 1、plugin-render-endpoint 8 —— 全过。
  • fork-adopt:dsh 20/20、pi-omp 22/22;e2e google 4/4、responses 7/7 + 5/5(证 [PR] msgid: server-side record on all four wires + GA pin rollout (#6525/#6539) #6540 服务端改动不扰 google/responses 正常流量)。
  • 真机 dsh e2e:父会话 11/11 live 行 = 宿主 UUID(0 摘要,不 churn)→ 快照 rawId 全宿主 UUID → fork 对照(不带 prefixMsgIds)=201 exact(字节轨向后兼容)/ fork msgid 轨(带 prefixMsgIds,driftFrom=1)=201 drifted(轨真点亮)/ fork 错 id=201 exact 无 409(门严格校验)。
  • 唯一失败 = baseline 既有的环境性 CCR 冷盘恢复用例(plugin-fork 5、opencode-v2 3),非本 PR 引入。

本 PR 不含(带依据)

关系

取代并收编 #6293 / #6532 / #6540 / #6549 / #6553(那 5 个将关闭)。追踪 issue #6525,提案 discussion #6539。

@ranxianglei ranxianglei changed the title [PR] msgid 递送:提案 #6539 完整落地(发·记·用,dsh/pi/omp/opencode × 四协议)合成一个 PR [PR] msgid Delivery: Proposal #6539 Full Implementation (Publish·Record·Use, dsh/pi/omp/opencode × Four Protocols) Consolidat | msgid 递送:提案 #6539 完整落地(发·记·用,dsh/pi/omp/opencode × 四协议)合成一个 PR Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-14_msgid-proposal-6539 (5ab83dd)

Latest build — install this exact version

npm install -g billion-context@0.0.9967-pr.6564.287620f

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
5ab83dd billion-context@0.0.9967-pr.6564.287620f build
62f155a billion-context@0.0.9631-pr.6564.43f38cd build
a6b8d99 billion-context@0.0.9605-pr.6564.3953e97 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6564.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

双 Agent Review + 修复(合并前)

按 m03247 要求做了双 agent 独立 review(正确性/整合 一名 + 安全/降级 一名),然后在本分支直接把发现的问题修掉。

总体结论:降级不变式(never-misalign)在所有 pin 路径成立 —— 每个 pin 要么是服务端拿自己的内核换算做足长度证明、要么是整轮降级;fork 边界又独立按目标会话自己的行 re-gate。唯一真 gap 是 egress 卫生: stamped 的 full-resend body 在若干 verbatim-forward 车道会把 biliMsgIds 带到真模型端点。已修。

修掉的(commit 7e5656777)

级别 发现 修法
F1 MAJOR biliMsgIds/biliLedgerSeqs 只在四个 fully-processed prepare 里剥;所有 verbatim-forward 车道(unattributed/bypass/v1internal/commandcode/no-messages/tail、auto-mode forge、side-request body)只剥 prompt_cache_key → stamped body 会带着 bili 字段打到真端点,违反 #2821「永不进模型端点」+ 破 #661 body 字节指纹契约,strict-schema 上游 400「Extra inputs are not permitted」(#1403 同款) server.ts 加协议中立的 scrubMsgIdFields(无字段→同一 Buffer 字节不变,#661 正常路径不动);handle.ts 每个 verbatim 分支套上。#1884 signed-passthrough 故意不剥(body-covering sdk-hmac-sha256 签名,mutate 会破签,byte-untouched 契约赢)
F2 MINOR prepareCountTokens 重建 {...parsed} 只删 prompt_cache_key,会转发客户端带的 biliMsgIds 同路径删两字段
M1 MAJOR dsh resolveEnvelopeMsgIds 内联了 coreAlignEnvelopeIds 整段(cherry-pick 残留,两处实现同一不变式,未来修一次要修两次) dsh 改为 import 共享函数 coreAlignEnvelopeIds
m1 dsh fork msgid 轨是死路径:hostMsgIdsOf(body.messages) 按每条 wire message 一个 id(含 lead system),永远过不了 adopter 的 msgIds.length===core.length 锁(openai hoist 差 1 / anthropic 多块轮)→ 轨从不点亮 改用同一个 core-aligned resolver;undefined→字节轨,和之前一样安全
m2 maybeStampEnvelopeMsgIds 无「增量车道已 divert」显式 guard,只靠 walker bail 兜底 加 if (headers.has(INCREMENTAL_TURN_MARKER)) return(marker 与 swap 原子设置)
m3 opencode V2 注释写错了 wire 的 hoist 规则(写 openai/anthropic,实际消费端是 responsesToCore 的 sysDevHead) 注释改为真实规则;skip 逻辑本身恰好匹配,不动
n1 新注释含非 ASCII「发」 去掉
n2 ENVELOPE_MSG_IDS_FIELD 与 msgid-pin.ts MSG_IDS_BODY_FIELD 仅靠注释同步 提升模块级 + 一行 parity 单测(msgid-rule-parity.test.ts)

新增 tests/issue6539-verbatim-scrub.test.ts(7 条)。回归:msgid/envelope/fork 单测 70/70 + 28/28 + 22/22;wire e2e google+responses 17/17;plugin-fork 52/57(5 个为既有 cold-restore 环境失败,非本分支)。

记录为设计边界(修前确认,不改动)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

状态核查 — 整合分支已收敛,就绪待合并

分支 2026-11-14_msgid-proposal-6539 现 9 个 commit。在你 floor 4 的 review 修复(7e5656777)之后又加了两笔:opencode V1 全量重发信封(4072eecd0,原「本 PR 不含」的 V1 已落地)+ hermetic e2e(62f155a,真 HTTP 栈验信封 + verbatim scrub)。我对当前 head 做了独立定向验证:

  • typecheck 干净(kernel + pi-subagents build + tsc --noEmit)
  • 定向单测 102/102(fork-msgid-track / dsh-envelope / pi-omp-envelope / opencode-v1 / opencode-v2 / verbatim-scrub / msgid-pin / msgid-rule-parity / openai-msgid-pin / plugin-render-endpoint / msgid-lane-flow)
  • hermetic e2e 2/2(正常车道 pin 宿主号 + 剥字段;verbatim bypass 车道 scrub F1)
  • fork-adopt dsh+pi-omp 42/42
  • CI(mirror 缓存,sha=62f155a):30 项 0 失败 / 8 pending,guards(lifecycle/lock/plan/import-graph/docs-coverage/pi-subagents/gate)全绿,e2e 车道 hermetic zero-token skip

一处需你留意:PR 描述里「本 PR 不含 opencode V1」一段现已过时——V1 已进分支(4072eecd0)。描述是原始输入我不改写,建议你顺手更新那一段。

八问

  1. 是什么问题:把 msgid 递送提案 提案:msgid 全生命周期「发 + 记 + 用」补齐(车道无关 × 全协议) #6539 的完整落地(此前拆成 5 个零散 PR)合成一个 PR——让 dsh/pi/omp/opencode 四宿主在四条协议上把稳定宿主消息号真正发上去 → 服务端记下并 pin → 快照暴露 → fork/继承能走 msgid 身份轨。
  2. 什么原因导致的:此前四宿主 × 四协议里只有 dsh 的 anthropic 增量车道能发宿主号,其余要么发不上去(无信封/无对齐)、要么记不住(服务端只 pin 部分 wire)、要么对不齐(宿主号与 core 数不匹配)。
  3. 影响范围与严重程度:四宿主 × 四协议(openai/anthropic/google/responses)的 msgid 递送;属新增能力,非既有缺陷,无外部可见回归面。
  4. 怎么修复的:收(服务端四 wire 全 pin + rollout GA 100%)+ 用(fork 协议加 msgid 身份轨,门=rawId 相等、漂移→rekey)+ 发(四宿主全量重发轮接 biliMsgIds 信封,内核 wireIndex 证 core 对齐);降级不变式(never-misalign)贯穿所有 pin 路径。
  5. 是否有回归问题:无。降级不变式在所有 pin 路径成立(每个 pin 要么服务端做足长度证明、要么整轮降级);fork 边界独立按目标会话自己的行 re-gate;字节轨不变。定向验证 146 测试全过;CI 0 失败(8 pending 中,全量回归以 CI 为准)。
  6. 是否破坏缓存:不涉及。msgid 是身份轨,不动缓存/重试/增量游标/去重键;信封字段转发前全剥,模型 wire 仍不带消息级 id(公告:release v0.1.193 修复dsh 切模型导致上下文暴涨问题,修复fork问题,修复claude codex dsh等自动压缩导致会话破坏问题 . 另外修复几十个bug. 我本地场景有限,未必都测全,麻烦大家小场景先测试下是否有问题.任何问题及时反馈. #2821)。
  7. 是否改了设计方向:新增 msgid 身份轨(fork 可走),既有字节轨不变;宿主号 ≠ bili 自增号(pin 的是宿主自发稳定 UUID,bili 的 seq/mNNNNN 是代理内部、宿主不可见)。
  8. 是否引入了新的配置:无新配置键。pluginLaneServerHistory rollout 提到 GA 100%(既有 rollout 键,非新增)。

判定:整合 PR 就绪,等 CI 8 项 pending 转绿后由你合并(agent 不合并)。已补 PR 标签:enhancement / p2 / dsh / pi / omp / opencode / scope/wiring(按本 PR 自身 diff 判定:跨四宿主协议接线,无 kernel 改动)。

…6539)

Complements the client-side envelope wiring (#6532 shipped dsh openai+
anthropic). This is the 'record' half of the msgid lifecycle made whole
across every wire:

- prepare-google / prepare-responses: read + strip the biliMsgIds envelope
  before it can reach the upstream, then pin behind the same rollout gate
  as the openai/anthropic twins. A length/shape mismatch is rejected
  (bail) at the pin — never leaked, never mis-pinned.
- handle.ts: a non-body id source (x-bili-msg-ids header / loopback
  harvest) now stamps the parsed body on EVERY wire, not just
  openai/anthropic, so the four pin-consumers all see it uniformly.
- rollout: pluginLaneServerHistory 0% -> 100% (GA, owner ruling
  2026-11-13). No client calls the co-gated /__bili/plugin/render
  endpoint yet, so flipping only activates pinning. Roll back with
  BILI_ROLLOUT_PLUGIN_LANE_SERVER_HISTORY=0.

Three tests that pinned the old default-off contract now force the off
state explicitly (env=0) instead of deleting the var.
Principle 1 of the four-principles proposal (discussion #6262): identity,
not bytes, decides which prefix a fork inherits. The plugin-protocol
channel (dsh/omp/pi declared forks) gains an optional msgid track:

- a host-pinned parent exposes the host's stable message ids AS its
  snapshot rawIds, so the child may present the host id of each matched
  prefix message (prefixMsgIds) plus the hash-run length (driftFrom);
- the server accepts a byte-drifted prefix only when every presented id
  equals the parent's rawId at its slot; the receipt is marked
  status "drifted" (or "expanded+drifted") with the verified
  driftFrom echoed;
- a gate failure never fails the fork: the byte-verified strict fork
  proceeds exactly as before (no 409); malformed shapes are 400; parents
  without host-pinned ids make the track a no-op (byte path unchanged).

Client: lanes inject their existing parity-pinned collectors (dsh
hostMsgIdsOf; pi/omp laneMsgIdsForWire on the anthropic wire with the
same content-pinnable gate the incremental pin uses); fork-adopt
performs only the core-alignment length lock — no new validator copies,
msgid-rule-parity matrix untouched. opencode-v2 is not wired in this PR
(byte track, follow-up).

Tests: tests/issue6262-fork-msgid-track.test.ts (8) — client prefix
extension across a drift cell, the byte-fallback arms, a full-chain
proxy round trip, and the server gate (drifted receipt / strict
fallback / 400 arms). Docs: msgid-infrastructure.md consumer 6,
plugin-protocol.md section 8, architecture.md.
Carrier ① (the envelope body field) had zero client wiring: the server
consumes biliMsgIds on both wires (prepare-openai/anthropic, gated on
pluginMode + the pluginLaneServerHistory rollout), but no client ever
stamped it. dsh's stamping lived only on the anthropic-wire incremental
lane (#4491), so every openai-wire full-resend turn — dsh's main path —
arrived id-less and the store fell back to content-derived digests,
which in turn leaves the fork msgid identity track (#6293) with nothing
to match.

maybeStampEnvelopeMsgIds now stamps full-resend model bodies (both
wires) after the incremental lane had its chance. Resolution never
re-implements the host's wire transforms: the pi-ai lockstep walkers
(laneMsgIdsForWire / laneMsgIdsForOpenAiWire — the SAME functions the
pi/omp lanes pin with) prove wire-element ⇔ host-message pairing by
order-preserving fingerprints, bail all-or-nothing; the kernel's own
converters then prove core-alignment via the #3642 wireIndex
back-pointer (every non-leading wire element must own exactly one core
— parallel tool calls / reasoning+text / text+image parts split one
host message into N cores, and N cores cannot share one host id).
System-role entries are excluded (they ride the hoisted/folded leading
system, never 1:1 to a non-leading wire element). Any bail leaves the
body byte-identical → content-derived ids, exactly today's behavior;
with the rollout off the server strips the field, so default installs
are untouched.
#6525 B7)

dsh's flat host messages differ from the pi-ai shape the shared walker's
host-side fingerprint assumes in two places (role tool vs toolResult,
block tool-call vs toolCall), so every tool-bearing turn bailed the
all-or-nothing envelope and the turn-1 host-pinned rows churned back to
content digests on the next full-resend. Normalize dsh's shape for
fingerprinting only, in the dsh resolver: the envelope ids still come from
mm.id and the wire-side pairing is untouched, so no mis-pin is possible.

Found via a real-dsh e2e dump (after turn 1: live rows 11/11 content
digests). Now: live rows 11/11 host-pinned UUIDs. Two regression tests pin
the accept (a dsh tool turn core-aligns and stamps) and the retained bail
(parallel tool_calls still refuse, one host id cannot cover N cores).
…6525/#6539)

The server pins host ids against the POST-HOIST core count (prepare-openai
validates ids.length against openaiToCore(parsed).length, which hoists the
contiguous leading system/developer run out of the fold space). opencode V2's
x-bili-msg-ids header was stamped per WIRE message, INCLUDING the leading
system, so its count always exceeded the core count on the openai wire and the
pin was rejected (a safe content-derived degrade — but opencode V2 never
actually pinned on the openai wire).

Core-align the V2 extractor: skip the contiguous leading system/developer run
before collecting ids so the header count equals the post-hoist core count.
No-op on the anthropic wire (system is a separate field, never in messages);
mid-conversation system messages keep their ids; a leading-system-only turn
degrades (no empty header). Whole-turn poison contract unchanged.
…#6539 review)

Dual-agent review of the #6539 integration found the egress gap: a stamped
full-resend body rode biliMsgIds/biliLedgerSeqs out through the verbatim
forward branches that bypass the four fully-processed prepares — strict-schema
upstreams 400 'Extra inputs are not permitted', and #661 body fingerprinting
breaks.

- server.ts: protocol-neutral scrubMsgIdFields (byte-identical pass-through
  when the fields are absent, so #661's contract is untouched in the normal
  case) + prepareCountTokens egress hygiene.
- handle.ts: scrub applied on every verbatim branch (unattributed passthrough,
  bypass, v1internal background, commandcode unconvertible, no-messages
  fallback, side-request body, auto-mode forge, final fallback). The #1884
  signed-passthrough escape hatch is deliberately NOT scrubbed — the
  body-covering sdk-hmac-sha256 signature would break; byte-untouched wins.
- dsh-native.ts: resolveEnvelopeMsgIds now calls the shared
  coreAlignEnvelopeIds instead of the cherry-pick'd inline copy (review M1);
  the fork-adopt msgids use the SAME core-aligned resolver (m1: the old
  per-wire-message hostMsgIdsOf could never pass the adopter's
  msgIds.length===core.length lock); explicit incremental-lane guard (m2);
  ENVELOPE_MSG_IDS_FIELD hoisted to module level with a parity unit test (n2).
- opencode-v2-msgids.ts: comment corrected to the actual responsesToCore
  sysDevHead / anthropic separate-field rules (m3).
- non-ASCII note text removed (n1).

New: tests/issue6539-verbatim-scrub.test.ts (7).
Regression: msgid/envelope/fork unit 70/70 + 28/28 + 22/22; wire e2e
google+responses 17/17; plugin-fork 52/57 (5 pre-existing cold-restore fails).
)

opencode 1.x's ONLY host-id seam is experimental.chat.messages.transform
(harvested per-session by harvestMsgIdsTransform); the wire body only exists
at fetch time. So the alignment runs at the shared fetch chokepoint via a
new state.beforeSend — proving the harvested turn POSITIONALLY against the
ACTUAL wire (alignedMsgIdsForWire), never a verbatim copy of the stored
list: a same-session A-transform/B-transform/A-fetch interleave degrades to
content-derived instead of mis-pinning.

- anthropic-wire only; a turn pins only when every message lowers 1:1 to one
  core (pure-text user/assistant) — tool/reasoning parts or any count/role/
  text drift drop the whole list. The server re-proves count parity +
  canonical shape and strips the field before any upstream forward (#2821).
- Also feeds the shared watermark slot (pfCaptureIds) with the SAME proven
  list — this replaces the DEAD call that sat in harvestMsgIdsTransform
  (it fed host objects {info,parts} to a wire-shape proof, so it could only
  ever clear the slot; V1 watermark was count-only with no tail-ids).
- Guards: never restamp an incremental-lane (diverted) turn, never clobber an
  already-present x-bili-msg-ids (a V2 request stamps its own), budgets match
  the server parser (4096 ids / 16KiB).

New: tests/opencode-v1-envelope.test.ts (9).
…6539)

Boots a real startServer + fake upstream (no agent) and drives the exact
envelope the dsh/pi/omp/opencode clients stamp:
  A. normal plugin lane — biliMsgIds pins every host id into the store (the
     plugin snapshot's rawIds come back as the host ids, not content digests)
     and the field never reaches the upstream.
  B. verbatim (bypass) lane — the same stamped body is scrubbed before the
     forward (the F1 review fix), so biliMsgIds never reaches the upstream.

2/2 pass; typecheck clean.
The dsh/pi/omp full-resend envelope (biliMsgIds) only covered the openai and
anthropic wires; a /responses request (body.input) safely degraded to
content-derived ids. This adds the responses dialect so all three host agents
carry host msgids on every wire the proxy can pin.

- pi-msgids.ts: laneMsgIdsForResponsesWire + respWireFingerprint/respUserParts
  (lockstep pairing via the shared makeEntryTake/partsLowerThrough; conservative
  bail on mid-history system, opaque items, reasoning, and any item the kernel
  would split into N cores). coreAlignEnvelopeIds now accepts "responses" and
  proves 1:1 against the kernel's responsesToCore wireIndex — the perEl check is
  the final no-mis-pin safety net.
- dsh-native.ts / pi.ts: resolvers accept the responses wire (wire = parsed.input).
- tests: 11 new cases (1:1 tool round, lead-system hoist, multi-part/mid-system/
  opaque/reasoning bails, dup-id bail, resolver pairing with and without lead).

Regressions green: pi-omp 34/34, dsh 15/15, opencode-v2 10/10, fork-track 8/8,
msgid full suite, wire e2e google 4/4 + responses 13/13.
… the msgid GA (#6539)

The pluginLaneServerHistory GA (this PR, 0% -> 100%) turned on the
canonical-id/seq stamps on the incremental lane's re-entry body for every
plugin session. When that body carries the client's rolling cache_control
marks (opencode's cacheControlRetouch channel), the stamps make the kernel's
first-sight pass re-render the marked content into <acp>mNNNNN</acp> markers,
which drops the wire-level cache_control breakpoints and drifts the ACP refs.

Gate the stamps on the mark channel: coreToAnthropic splices entry.marks into
the assembled items, so a non-empty marks map is the precise signal that this
turn's body carries marks. Mark turns now re-enter exactly as they do with the
rollout off (no stamps) — the pre-GA behavior that always passed — while
non-mark turns keep pinning. The lane still owns the canonical ids.

Tests: opencode-rolling-cache-marks 12/12 (was 9/12); msgid/envelope/fork
148/148; incremental-lane suites green; dsh real-machine envelope e2e ALL PASS.
@ranxianglei

Copy link
Copy Markdown
Owner Author

已收编进统一 PR #6785(msgid 递送 × 快照平面,含本 PR 全部内容 + 前向合并 master f21b601 解冲突 + gate --snapbase-full 全开 lane + 全量本地验证)。请合并 #6785,本 PR 关闭。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant