Repository navigation
[PR] msgid Delivery: Proposal #6539 Full Implementation (Publish·Record·Use, dsh/pi/omp/opencode × Four Protocols) Consolidat | msgid 递送:提案 #6539 完整落地(发·记·用,dsh/pi/omp/opencode × 四协议)合成一个 PR - #6564
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9967-pr.6564.287620fThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6564.tgz
npm install -g packageInstall instructions are refreshed on each push. |
双 Agent Review + 修复(合并前)按 m03247 要求做了双 agent 独立 review(正确性/整合 一名 + 安全/降级 一名),然后在本分支直接把发现的问题修掉。 总体结论:降级不变式(never-misalign)在所有 pin 路径成立 —— 每个 pin 要么是服务端拿自己的内核换算做足长度证明、要么是整轮降级;fork 边界又独立按目标会话自己的行 re-gate。唯一真 gap 是 egress 卫生: stamped 的 full-resend body 在若干 verbatim-forward 车道会把 修掉的(commit
|
| 级别 | 发现 | 修法 |
|---|---|---|
| F1 MAJOR | biliMsgIds/biliLedgerSeqs 只在四个 fully-processed prepare 里剥;所有 verbatim-forward 车道(unattributed/bypass/v1internal/commandcode/no-messages/tail、auto-mode forge、side-request body)只剥 prompt_cache_key → stamped body 会带着 bili 字段打到真端点,违反 #2821「永不进模型端点」+ 破 #661 body 字节指纹契约,strict-schema 上游 400「Extra inputs are not permitted」(#1403 同款) |
server.ts 加协议中立的 scrubMsgIdFields(无字段→同一 Buffer 字节不变,#661 正常路径不动);handle.ts 每个 verbatim 分支套上。#1884 signed-passthrough 故意不剥(body-covering sdk-hmac-sha256 签名,mutate 会破签,byte-untouched 契约赢) |
| F2 MINOR | prepareCountTokens 重建 {...parsed} 只删 prompt_cache_key,会转发客户端带的 biliMsgIds |
同路径删两字段 |
| M1 MAJOR | dsh resolveEnvelopeMsgIds 内联了 coreAlignEnvelopeIds 整段(cherry-pick 残留,两处实现同一不变式,未来修一次要修两次) |
dsh 改为 import 共享函数 coreAlignEnvelopeIds |
| m1 | dsh fork msgid 轨是死路径:hostMsgIdsOf(body.messages) 按每条 wire message 一个 id(含 lead system),永远过不了 adopter 的 msgIds.length===core.length 锁(openai hoist 差 1 / anthropic 多块轮)→ 轨从不点亮 |
改用同一个 core-aligned resolver;undefined→字节轨,和之前一样安全 |
| m2 | maybeStampEnvelopeMsgIds 无「增量车道已 divert」显式 guard,只靠 walker bail 兜底 |
加 if (headers.has(INCREMENTAL_TURN_MARKER)) return(marker 与 swap 原子设置) |
| m3 | opencode V2 注释写错了 wire 的 hoist 规则(写 openai/anthropic,实际消费端是 responsesToCore 的 sysDevHead) |
注释改为真实规则;skip 逻辑本身恰好匹配,不动 |
| n1 | 新注释含非 ASCII「发」 | 去掉 |
| n2 | ENVELOPE_MSG_IDS_FIELD 与 msgid-pin.ts MSG_IDS_BODY_FIELD 仅靠注释同步 |
提升模块级 + 一行 parity 单测(msgid-rule-parity.test.ts) |
新增 tests/issue6539-verbatim-scrub.test.ts(7 条)。回归:msgid/envelope/fork 单测 70/70 + 28/28 + 22/22;wire e2e google+responses 17/17;plugin-fork 52/57(5 个为既有 cold-restore 环境失败,非本分支)。
记录为设计边界(修前确认,不改动)
- F4 opencode V2 头无客户端逐元素 core 证明 —— 依赖内核不变式「converter 从不把两条 wire message 合并成一个 core」(The intermediate message (core) is not protocol-agnostic: switching protocols/models causes identity churn; cross-protoc | 中间消息(core)没有做到协议无关:切协议/切模型制造身份 churn,补齐跨协议映射 #3642 起只 split)。该不变式 load-bearing 但当前无专门内核级测试,建议后续加;本分支靠服务端长度校验兜底(不匹配即降级)。
- F5 服务端验 count+shape,不验 id 语义 —— 客户端铸 id 的固有信任边界;错序 pin 会让 fork mismatch 降级,不会静默采纳错误血缘。
- F6 头通道字符集比客户端宽 —— 服务端是 canonical 校验,阻 log-injection/保留命名空间 squatting,安全。
- m4 fork msgid 轨只能延长字节验证前缀,不能建立(
nHash===0先 bail)—— 有意保守(字节证明播种信任,仅凭 id 会让有 bug 的宿主声称任意父状态),价值集中在「后段漂移」,正是 提案:fork 继承四原则(msgid 身份轨 / lineage 编号 / 无状态账本 / 纯块拷贝) #6262 的场景。 - F3
incomingMsgIdsHeader对多值头取[0]—— Node HTTP 下实际不可达(重复头逗号拼接后过不了空白检查→干净降级),MITM 本就能发任意单列表,cosmetic。
(ai: vllm-qwen/qwen3.8-27b) 状态核查 — 整合分支已收敛,就绪待合并 分支
一处需你留意:PR 描述里「本 PR 不含 opencode V1」一段现已过时——V1 已进分支( 八问
判定:整合 PR 就绪,等 CI 8 项 pending 转绿后由你合并(agent 不合并)。已补 PR 标签:enhancement / p2 / dsh / pi / omp / opencode / scope/wiring(按本 PR 自身 diff 判定:跨四宿主协议接线,无 kernel 改动)。 |
…6539) Complements the client-side envelope wiring (#6532 shipped dsh openai+ anthropic). This is the 'record' half of the msgid lifecycle made whole across every wire: - prepare-google / prepare-responses: read + strip the biliMsgIds envelope before it can reach the upstream, then pin behind the same rollout gate as the openai/anthropic twins. A length/shape mismatch is rejected (bail) at the pin — never leaked, never mis-pinned. - handle.ts: a non-body id source (x-bili-msg-ids header / loopback harvest) now stamps the parsed body on EVERY wire, not just openai/anthropic, so the four pin-consumers all see it uniformly. - rollout: pluginLaneServerHistory 0% -> 100% (GA, owner ruling 2026-11-13). No client calls the co-gated /__bili/plugin/render endpoint yet, so flipping only activates pinning. Roll back with BILI_ROLLOUT_PLUGIN_LANE_SERVER_HISTORY=0. Three tests that pinned the old default-off contract now force the off state explicitly (env=0) instead of deleting the var.
Principle 1 of the four-principles proposal (discussion #6262): identity, not bytes, decides which prefix a fork inherits. The plugin-protocol channel (dsh/omp/pi declared forks) gains an optional msgid track: - a host-pinned parent exposes the host's stable message ids AS its snapshot rawIds, so the child may present the host id of each matched prefix message (prefixMsgIds) plus the hash-run length (driftFrom); - the server accepts a byte-drifted prefix only when every presented id equals the parent's rawId at its slot; the receipt is marked status "drifted" (or "expanded+drifted") with the verified driftFrom echoed; - a gate failure never fails the fork: the byte-verified strict fork proceeds exactly as before (no 409); malformed shapes are 400; parents without host-pinned ids make the track a no-op (byte path unchanged). Client: lanes inject their existing parity-pinned collectors (dsh hostMsgIdsOf; pi/omp laneMsgIdsForWire on the anthropic wire with the same content-pinnable gate the incremental pin uses); fork-adopt performs only the core-alignment length lock — no new validator copies, msgid-rule-parity matrix untouched. opencode-v2 is not wired in this PR (byte track, follow-up). Tests: tests/issue6262-fork-msgid-track.test.ts (8) — client prefix extension across a drift cell, the byte-fallback arms, a full-chain proxy round trip, and the server gate (drifted receipt / strict fallback / 400 arms). Docs: msgid-infrastructure.md consumer 6, plugin-protocol.md section 8, architecture.md.
Carrier ① (the envelope body field) had zero client wiring: the server consumes biliMsgIds on both wires (prepare-openai/anthropic, gated on pluginMode + the pluginLaneServerHistory rollout), but no client ever stamped it. dsh's stamping lived only on the anthropic-wire incremental lane (#4491), so every openai-wire full-resend turn — dsh's main path — arrived id-less and the store fell back to content-derived digests, which in turn leaves the fork msgid identity track (#6293) with nothing to match. maybeStampEnvelopeMsgIds now stamps full-resend model bodies (both wires) after the incremental lane had its chance. Resolution never re-implements the host's wire transforms: the pi-ai lockstep walkers (laneMsgIdsForWire / laneMsgIdsForOpenAiWire — the SAME functions the pi/omp lanes pin with) prove wire-element ⇔ host-message pairing by order-preserving fingerprints, bail all-or-nothing; the kernel's own converters then prove core-alignment via the #3642 wireIndex back-pointer (every non-leading wire element must own exactly one core — parallel tool calls / reasoning+text / text+image parts split one host message into N cores, and N cores cannot share one host id). System-role entries are excluded (they ride the hoisted/folded leading system, never 1:1 to a non-leading wire element). Any bail leaves the body byte-identical → content-derived ids, exactly today's behavior; with the rollout off the server strips the field, so default installs are untouched.
#6525 B7) dsh's flat host messages differ from the pi-ai shape the shared walker's host-side fingerprint assumes in two places (role tool vs toolResult, block tool-call vs toolCall), so every tool-bearing turn bailed the all-or-nothing envelope and the turn-1 host-pinned rows churned back to content digests on the next full-resend. Normalize dsh's shape for fingerprinting only, in the dsh resolver: the envelope ids still come from mm.id and the wire-side pairing is untouched, so no mis-pin is possible. Found via a real-dsh e2e dump (after turn 1: live rows 11/11 content digests). Now: live rows 11/11 host-pinned UUIDs. Two regression tests pin the accept (a dsh tool turn core-aligns and stamps) and the retained bail (parallel tool_calls still refuse, one host id cannot cover N cores).
…6525/#6539) The server pins host ids against the POST-HOIST core count (prepare-openai validates ids.length against openaiToCore(parsed).length, which hoists the contiguous leading system/developer run out of the fold space). opencode V2's x-bili-msg-ids header was stamped per WIRE message, INCLUDING the leading system, so its count always exceeded the core count on the openai wire and the pin was rejected (a safe content-derived degrade — but opencode V2 never actually pinned on the openai wire). Core-align the V2 extractor: skip the contiguous leading system/developer run before collecting ids so the header count equals the post-hoist core count. No-op on the anthropic wire (system is a separate field, never in messages); mid-conversation system messages keep their ids; a leading-system-only turn degrades (no empty header). Whole-turn poison contract unchanged.
…#6539 review) Dual-agent review of the #6539 integration found the egress gap: a stamped full-resend body rode biliMsgIds/biliLedgerSeqs out through the verbatim forward branches that bypass the four fully-processed prepares — strict-schema upstreams 400 'Extra inputs are not permitted', and #661 body fingerprinting breaks. - server.ts: protocol-neutral scrubMsgIdFields (byte-identical pass-through when the fields are absent, so #661's contract is untouched in the normal case) + prepareCountTokens egress hygiene. - handle.ts: scrub applied on every verbatim branch (unattributed passthrough, bypass, v1internal background, commandcode unconvertible, no-messages fallback, side-request body, auto-mode forge, final fallback). The #1884 signed-passthrough escape hatch is deliberately NOT scrubbed — the body-covering sdk-hmac-sha256 signature would break; byte-untouched wins. - dsh-native.ts: resolveEnvelopeMsgIds now calls the shared coreAlignEnvelopeIds instead of the cherry-pick'd inline copy (review M1); the fork-adopt msgids use the SAME core-aligned resolver (m1: the old per-wire-message hostMsgIdsOf could never pass the adopter's msgIds.length===core.length lock); explicit incremental-lane guard (m2); ENVELOPE_MSG_IDS_FIELD hoisted to module level with a parity unit test (n2). - opencode-v2-msgids.ts: comment corrected to the actual responsesToCore sysDevHead / anthropic separate-field rules (m3). - non-ASCII note text removed (n1). New: tests/issue6539-verbatim-scrub.test.ts (7). Regression: msgid/envelope/fork unit 70/70 + 28/28 + 22/22; wire e2e google+responses 17/17; plugin-fork 52/57 (5 pre-existing cold-restore fails).
) opencode 1.x's ONLY host-id seam is experimental.chat.messages.transform (harvested per-session by harvestMsgIdsTransform); the wire body only exists at fetch time. So the alignment runs at the shared fetch chokepoint via a new state.beforeSend — proving the harvested turn POSITIONALLY against the ACTUAL wire (alignedMsgIdsForWire), never a verbatim copy of the stored list: a same-session A-transform/B-transform/A-fetch interleave degrades to content-derived instead of mis-pinning. - anthropic-wire only; a turn pins only when every message lowers 1:1 to one core (pure-text user/assistant) — tool/reasoning parts or any count/role/ text drift drop the whole list. The server re-proves count parity + canonical shape and strips the field before any upstream forward (#2821). - Also feeds the shared watermark slot (pfCaptureIds) with the SAME proven list — this replaces the DEAD call that sat in harvestMsgIdsTransform (it fed host objects {info,parts} to a wire-shape proof, so it could only ever clear the slot; V1 watermark was count-only with no tail-ids). - Guards: never restamp an incremental-lane (diverted) turn, never clobber an already-present x-bili-msg-ids (a V2 request stamps its own), budgets match the server parser (4096 ids / 16KiB). New: tests/opencode-v1-envelope.test.ts (9).
…6539) Boots a real startServer + fake upstream (no agent) and drives the exact envelope the dsh/pi/omp/opencode clients stamp: A. normal plugin lane — biliMsgIds pins every host id into the store (the plugin snapshot's rawIds come back as the host ids, not content digests) and the field never reaches the upstream. B. verbatim (bypass) lane — the same stamped body is scrubbed before the forward (the F1 review fix), so biliMsgIds never reaches the upstream. 2/2 pass; typecheck clean.
The dsh/pi/omp full-resend envelope (biliMsgIds) only covered the openai and anthropic wires; a /responses request (body.input) safely degraded to content-derived ids. This adds the responses dialect so all three host agents carry host msgids on every wire the proxy can pin. - pi-msgids.ts: laneMsgIdsForResponsesWire + respWireFingerprint/respUserParts (lockstep pairing via the shared makeEntryTake/partsLowerThrough; conservative bail on mid-history system, opaque items, reasoning, and any item the kernel would split into N cores). coreAlignEnvelopeIds now accepts "responses" and proves 1:1 against the kernel's responsesToCore wireIndex — the perEl check is the final no-mis-pin safety net. - dsh-native.ts / pi.ts: resolvers accept the responses wire (wire = parsed.input). - tests: 11 new cases (1:1 tool round, lead-system hoist, multi-part/mid-system/ opaque/reasoning bails, dup-id bail, resolver pairing with and without lead). Regressions green: pi-omp 34/34, dsh 15/15, opencode-v2 10/10, fork-track 8/8, msgid full suite, wire e2e google 4/4 + responses 13/13.
c236dd5 to
52e54c7
Compare
… the msgid GA (#6539) The pluginLaneServerHistory GA (this PR, 0% -> 100%) turned on the canonical-id/seq stamps on the incremental lane's re-entry body for every plugin session. When that body carries the client's rolling cache_control marks (opencode's cacheControlRetouch channel), the stamps make the kernel's first-sight pass re-render the marked content into <acp>mNNNNN</acp> markers, which drops the wire-level cache_control breakpoints and drifts the ACP refs. Gate the stamps on the mark channel: coreToAnthropic splices entry.marks into the assembled items, so a non-empty marks map is the precise signal that this turn's body carries marks. Mark turns now re-enter exactly as they do with the rollout off (no stamps) — the pre-GA behavior that always passed — while non-mark turns keep pinning. The lane still owns the canonical ids. Tests: opencode-rolling-cache-marks 12/12 (was 9/12); msgid/envelope/fork 148/148; incremental-lane suites green; dsh real-machine envelope e2e ALL PASS.
这是啥
msgid 递送提案 #6539(discussion)的完整落地,合成一个 PR。之前拆成了 5 个零散 PR,按用户要求收成一个。
一句话:让 dsh / pi / omp / opencode 四个宿主的会话,在四条协议上,把稳定宿主消息号(host id)真正发上去 → 服务端记下并 pin 住 → 快照暴露 → fork/继承能走 msgid 身份轨。 之前只有 dsh 的 anthropic 增量车道做得到,其余全是「发不上去」或「记不住」或「对不齐」。
覆盖矩阵(收 · 用 · 发)
pluginLaneServerHistoryGA 100%prefixMsgIds+driftFrom,门=rawId 相等(宿主 pin 住后 rawId 就是宿主号);漂移→rekey;字节轨不变biliMsgIds信封(openai+anthropic),内核wireIndex证 core 对齐;修 B7 churn(dsh 扁平 host 形状role:"tool"/块type:"tool-call"与共享 walker 的 pi-ai 形不符→每个工具轮都 bail→宿主号 churn 成摘要)destinationRoutedThroughProxy门(盲隧道永不戳,#1403)关键设计纪律
wireIndex,校验走 canonical validator(非镜像)。options.messages[].id/ opencodemsg_*),bili 的 seq/mNNNNN 是代理内部、宿主不可见,不能当 fork 锚点。biliMsgIds/ 增量msgIds/ watermark 头 / ledger 四通道转发前全剥,模型 wire 仍不带消息级 id(公告:release v0.1.193 修复dsh 切模型导致上下文暴涨问题,修复fork问题,修复claude codex dsh等自动压缩导致会话破坏问题 . 另外修复几十个bug. 我本地场景有限,未必都测全,麻烦大家小场景先测试下是否有问题.任何问题及时反馈. #2821 原则)。验证(整合分支上跑)
本 PR 不含(带依据)
关系
取代并收编 #6293 / #6532 / #6540 / #6549 / #6553(那 5 个将关闭)。追踪 issue #6525,提案 discussion #6539。