From c8d7b14e2bb323e32383678555e557f1b77814be Mon Sep 17 00:00:00 2001 From: ranxianglei Date: Mon, 26 Oct 2026 23:32:05 +0800 Subject: [PATCH] docs: message-identity hierarchy is now msgid > order > hash (#4491/#4834) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The doctrine said stable client ids may only ever be join hints. That was written before the plugin lanes started shipping host msgids in a bili-private envelope (msgIds on the incremental turn / biliMsgIds stripped at ingress) and before #4834 made resolution id-first. Agents reading the docs concluded 'we have no msgids' — true on the model wire, false on the plugin lanes. - AGENTS.md KDD #11 retitled to the full hierarchy: msgid first where the envelope carries it (plugin lanes only, never forwarded upstream), order still primary on id-less resend reconciliation, hash stays the per-cell gate. Three-part PK (session_id, msg_id, content_hash) noted as safety belt. - AGENTS.md KDD #13 evolution updated to four stages (hash -> order -> msgid pinning -> snapshot render). - MESSAGE-IDENTITY.md: new section 'The msgid envelope' documenting the channel, all-or-nothing semantics, resync re-claim, #4834 id-first, #4863 claim-after-guard, and the boundary (model wire unchanged, tool-call ids are pairing keys, proxy-mode resends keep the ladder); Future-direction paragraph refreshed. --- AGENTS.md | 4 ++-- MESSAGE-IDENTITY.md | 48 +++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 48 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a7d63704e0..6fd4c3310e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,10 +49,10 @@ The full file-by-file map lives in [reference/architecture.md](reference/archite 10. **Thinking-family blocks are signature-verified — NEVER rewrite them (#1960)** — Anthropic `thinking`/`redacted_thinking` blocks and Gemini `thoughtSignature` parts are verified byte-for-byte when the latest assistant message is replayed ("thinking blocks ... cannot be modified"). The bytes the proxy forwards become the client's persisted history, and the bytes the loop accumulates become the re-request rebuild — BOTH must stay verbatim. Any transformation (tag/marker stripping, dedup, trimming, re-laying-out block order, dropping empty segments) desynchronizes text from signature and bricks the session: first on the in-turn re-request, then on every subsequent request via the client's echo. #1882 put a prose filter on the thinking channel and it was exactly this bug. Treat ANY future change touching thinking-family payloads as high-risk: it needs loop-level byte-equality tests (re-request body AND client-visible stream) on both signed wires, not just adapter-level ones. The prose filters (#206/#1881) are for the TEXT channel only. -11. **顺序为中心,hash 为辅助 (order-centric reconciliation — owner rule, 2026-10-12, #2821/#2915)** — fold-state reconciliation must treat ORDER as the primary carrier of message identity across resent histories: position claims verified spans (with per-cell fingerprint gates), content hashes and tool-call ids are auxiliary evidence, and NO single dimension's miss may cascade into total unfold. The identity dimensions survive different churn classes — order survives re-encoding AND id re-minting, content survives id churn, ids survive re-encoding — so the ladder is the union of their survival domains, and each rung fails soft: an id/fingerprint mismatch voids that CELL's claim only; the positional scans resync at the next run of same-index agreements (MIN_RESYNC_RUN consecutive matches, bounded gaps) and keep claiming verified spans behind the break. Genuinely changed cells (real edits, regenerations) stay honestly unmatched and re-enter the wire — "never fuzzy similarity" (mis-attribution is the most expensive failure, MESSAGE-IDENTITY.md) is unchanged: the hash stays a HARD per-cell gate, it just no longer holds a global veto over its neighbors. Escalation to "substrate destroyed" may only count gaps that survived resync AND the content passes (byTool/byTwin/byNorm/byTurn). #2821 was the canonical violation: a mid-session model switch re-minted every tool-call id on re-projection (content byte-identical), both positional flank scans died at their FIRST cell (changed preamble at the head, appended turn at the tail), 3289 covered ids went missing, three passes → destroyed verdict → total unfold → doomed-probe 502. #2396's detection-only twins are promoted to a claiming pass (Pass 1.5, k-th discipline + length guard) so id re-mints recover even without a stored positional copy — and after one churn pass the self-saved {ids, canon} copy upgrades recovery to the positional path. A Pass-0 gap cell (order break) is resolved by tiers, owner rule 2026-10-12: any exact channel that matches claims it; failing that, simhash corroboration (≤8 bits on the 64-bit word-shingle sketch — measured margins: re-wording ≈6-8 bits inside, negation flips ≈11 / word drops ≈13 / key reorders ≈21 outside) claims it; failing that, the ORDER BRACKET — same index strictly between VERIFIED claims on both flanks — asserts the SLOT (the owner-sanctioned delete+insert tradeoff: coverage rides the slot, the stale bytes do not); with zero corroboration and no fence the cell stays honest unmatched drift. Deletion is honest loss: the flanks absorb the shift, the deleted id never gets a bracket claim (its slot cell is consumed by the verified tail). Tests: `tests/issue2915-order-centric-reconcile.test.ts`. +11. **身份层级:msgid > 顺序 > hash (owner rule 2026-10-12 顺序中心; 2026-10-26 增补 #4491/#4834: 有 msgid 时 msgid 最优先)** — 消息身份按三层裁决:**(1) msgid 第一**:四条插件车道(dsh/omp/opencode/pi)把宿主 msgid 作为 bili 私有信封字段随增量回合上行(`msgIds` 平行数组 on `POST /__bili/plugin/turn`,src/agent/incremental-client.ts;服务端重打 `biliMsgIds` src/incremental-lane.ts,入站即剥 src/server/prepare-anthropic.ts)——**只存在于宿主↔bili 之间,从不出现在 bili↔模型线上**(模型线上至今没有消息级 id;tool-call id 是模型铸造、客户端回显的配对键,不是消息身份,#2821)。msgid 有效时账本身份以它为准:跨模型改写重编码不再否决 id(#4834),被拦截回合不再烧毁认领(#4863);全有全无——长度不符/筛查失败/已认领 id 使整个字段静默降级(增量车道上该回合退内容 hash)。账本主键仍是 (session_id, msg_id, content_hash) 三元组安全带:msgid 定身份,hash 兜底防错绑。**(2) 顺序为中心**:没有 msgid 的场合(代理模式全量重发、模型线证据)顺序是身份主载体,下面的 ladder 原文继续适用。**(3) hash 为辅助**:逐格硬门,但无全局否决权。fold-state reconciliation must treat ORDER as the primary carrier of message identity across resent histories: position claims verified spans (with per-cell fingerprint gates), content hashes and tool-call ids are auxiliary evidence, and NO single dimension's miss may cascade into total unfold. The identity dimensions survive different churn classes — order survives re-encoding AND id re-minting, content survives id churn, ids survive re-encoding — so the ladder is the union of their survival domains, and each rung fails soft: an id/fingerprint mismatch voids that CELL's claim only; the positional scans resync at the next run of same-index agreements (MIN_RESYNC_RUN consecutive matches, bounded gaps) and keep claiming verified spans behind the break. Genuinely changed cells (real edits, regenerations) stay honestly unmatched and re-enter the wire — "never fuzzy similarity" (mis-attribution is the most expensive failure, MESSAGE-IDENTITY.md) is unchanged: the hash stays a HARD per-cell gate, it just no longer holds a global veto over its neighbors. Escalation to "substrate destroyed" may only count gaps that survived resync AND the content passes (byTool/byTwin/byNorm/byTurn). #2821 was the canonical violation: a mid-session model switch re-minted every tool-call id on re-projection (content byte-identical), both positional flank scans died at their FIRST cell (changed preamble at the head, appended turn at the tail), 3289 covered ids went missing, three passes → destroyed verdict → total unfold → doomed-probe 502. #2396's detection-only twins are promoted to a claiming pass (Pass 1.5, k-th discipline + length guard) so id re-mints recover even without a stored positional copy — and after one churn pass the self-saved {ids, canon} copy upgrades recovery to the positional path. A Pass-0 gap cell (order break) is resolved by tiers, owner rule 2026-10-12: any exact channel that matches claims it; failing that, simhash corroboration (≤8 bits on the 64-bit word-shingle sketch — measured margins: re-wording ≈6-8 bits inside, negation flips ≈11 / word drops ≈13 / key reorders ≈21 outside) claims it; failing that, the ORDER BRACKET — same index strictly between VERIFIED claims on both flanks — asserts the SLOT (the owner-sanctioned delete+insert tradeoff: coverage rides the slot, the stale bytes do not); with zero corroboration and no fence the cell stays honest unmatched drift. Deletion is honest loss: the flanks absorb the shift, the deleted id never gets a bracket claim (its slot cell is consumed by the verified tail). Tests: `tests/issue2915-order-centric-reconcile.test.ts`. 12. **资格判定与执行共用单一投影 (shared unit-projection judgment, #3236)** — whether a candidate range is foldable has exactly ONE definition: `projectUnit` in `kernel/src/unit-projection.ts` (applySingleRange's pre-block sequence extracted verbatim: boundary widening → carrier/coverage/media/protected-tool carve → protected-zone carve → turn-integrity withdrawal → emptiness), returning either a structured projection or the ready-to-throw fatal error (`zone`/`split`). The apply side throws it byte-identically; the recommend side re-screens every FINAL merged unit through it and demotes fatal units to gaps (no re-merge after filtering — anti-oscillation); the acp_status uncompressed view runs it per displayed row and appends an additive ⚠ flag (diagnostic surface: rows are flagged, never dropped). The judgment must run on the EXACT array class `applyCompression` executes on: hosts whose render view differs from their apply class pass it via `ProcessTurnInput.applyView` / `StatusReportOptions.originalMessages` (plugin lane = raw history per #2627; relay loop = `prepared.originalMessages`); absent it, judgment falls back to the pre-prune original (#2663) or the given view. The bug fixed: recommend judged turn-integrity over the GLOBAL compressible set while apply judged the PER-SUBMITTED-RANGE set — a turn whose partners sat in different advertised units passed global screening yet split per range, producing a permanent advertise→reject loop ("Range would split N turn(s)"). Tests: `kernel/tests/issue3236-recommend-apply-consistency.test.ts`. -13. **出站历史渲染:快照即会话(snapshot render — #3756,rollout `snapshotRender` 默认 0%)** — 消息身份的演进分三段:content-hash 认身份(账本 join,至今有效)→ order-centric reconcile(#2821/#2915/#3641:顺序为主、hash 为辅助证据、判决分类)→ **快照渲染**:对绑定了 session id 且统一存储可读的会话,prepare 层的出站**前缀**由服务端正史渲染(`src/server/snapshot-render.ts`),消息身份 = 服务端行 `seq`,hash/签名只作链回溯兜底(定位客户端逐字尾巴的 join 点);覆盖由块的 seq/签名集**引擎自拥**(不信 reconcile 的 re-key 产物),锚从块状态原位重渲;尾巴保持客户端逐字(缓存稳定)。任何失步(开关关/无活跃块/存储不可读/链不上)整轮回退 legacy 路径——**降级,不猜**;客户端头部 checkpoint 一轮采用滞后,永不复活已覆盖行。观测与渲染刻意分离:`reconcileFoldCoverage` 继续观察客户端视图。完整教义与诚实边界:MESSAGE-IDENTITY.md「快照渲染引擎」。规则推论:凡讨论「覆盖 id 失配/destroyed 误诊/断缓存」类 bug,先问该会话是否走 D 车道——D 上这类病理结构性不存在,修不到点子上等于在修已死的路径。 +13. **出站历史渲染:快照即会话(snapshot render — #3756,rollout `snapshotRender` 默认 0%)** — 消息身份的演进分四段:content-hash 认身份(账本 join,至今有效)→ order-centric reconcile(#2821/#2915/#3641:顺序为主、hash 为辅助证据、判决分类)→ **msgid 钉身份**(#4491/#4834:插件信封 msgIds 有则最优先,抗跨模型改写;见 KDD #11)→ **快照渲染**:对绑定了 session id 且统一存储可读的会话,prepare 层的出站**前缀**由服务端正史渲染(`src/server/snapshot-render.ts`),消息身份 = 服务端行 `seq`,hash/签名只作链回溯兜底(定位客户端逐字尾巴的 join 点);覆盖由块的 seq/签名集**引擎自拥**(不信 reconcile 的 re-key 产物),锚从块状态原位重渲;尾巴保持客户端逐字(缓存稳定)。任何失步(开关关/无活跃块/存储不可读/链不上)整轮回退 legacy 路径——**降级,不猜**;客户端头部 checkpoint 一轮采用滞后,永不复活已覆盖行。观测与渲染刻意分离:`reconcileFoldCoverage` 继续观察客户端视图。完整教义与诚实边界:MESSAGE-IDENTITY.md「快照渲染引擎」。规则推论:凡讨论「覆盖 id 失配/destroyed 误诊/断缓存」类 bug,先问该会话是否走 D 车道——D 上这类病理结构性不存在,修不到点子上等于在修已死的路径。 14. **verif 模块:kernel 决策核的求解器验证(#3848,PR #3853)** —— `verif/` 目录 + `tests/verif-*.test.ts` 把守护清单内文件(`verif/guarded-files.ts`,Tier A:compress / hide-consumed / refs / render-refs / truncate / truncate-tools / tokenize / rebuild / wire/openai / wire/responses)的真实 TypeScript 翻译成 SMT 公式,用 cvc5/z3 找违反缓存稳定不变量的反例,反例必须回放真代码复现。约束:**改守护清单内的文件,除了常规测试还必须过 `tests/verif-subset-guard.test.ts`(子语言守卫,秒级)和涉及的 `tests/verif-bug*.test.ts` / `tests/verif-generic-*.test.ts` / `tests/verif-induction-*.test.ts`(求解器测试需 `VERIF_PYTHON` 指向装了 cvc5 的 python;无求解器环境自动 skip 不会红)**。新语法想进守护范围,翻译器先支持(守卫报啥补啥),例外在 `verif/exceptions.ts` 登记且过期即红。写新 harness 遵守 `verif/README.md` 的无作弊规则(只调真入口/不变量通用/反例回放/负控制必备)。求解器发现的 bug 一律开 issue + 回归钉(已产出:#3691/#3769/#3676/#3805 形状钉 + 新发现 #3935/#3945)。模块教义与边界:先读 `verif/README.md`。 diff --git a/MESSAGE-IDENTITY.md b/MESSAGE-IDENTITY.md index 2cef137514..8c389ec760 100644 --- a/MESSAGE-IDENTITY.md +++ b/MESSAGE-IDENTITY.md @@ -67,6 +67,44 @@ locally-minted ids twice in production. Ingress deletes replayed `msg-proxy-*` ids (`src/loop/adapter-responses.ts:62-68`) precisely because they are per-round artifacts, not identities. +## The msgid envelope: host-vouched ids ride the bili lane (#4491/#4834) + +The lane survey above answers a narrower question than it seems: it surveys +the **model wire**. The plugin lanes added a second channel where message ids +DO exist — a bili-private envelope, never forwarded to any model provider: + +- `msgIds` — a parallel string array beside `messages` on the incremental + turn envelope (`POST /__bili/plugin/turn`; built in + `src/agent/incremental-client.ts` `buildBody`). The server re-stamps it as + `biliMsgIds` on the assembled body it re-enters the pipeline with + (`src/incremental-lane.ts`), and `src/server/prepare-anthropic.ts` strips + the field and applies the pin (`src/msgid-pin.ts` `applyMsgIdPin`) before + the request leaves for upstream. +- Semantics: the CLIENT vouches that these ids are stable for that host's + view of the conversation (dsh/omp/opencode/pi storage ids). Valid ids are + pinned onto the ledger rows as `msg_id` — the client's identity becomes + the ledger identity, replacing content-derived ids for those messages. +- All-or-nothing: a length mismatch, a screen failure, or an + already-claimed id drops the WHOLE `msgIds` field (never a partial or + misaligned array); on the incremental lane that turn degrades to + content-derived identity. Reusing a live canonical id is a 400, not a + silent rebind. `resync: true` re-claims the full set wholesale. +- #4834: resolution is id-FIRST — a cross-model rewrite that re-encodes + content (pi `transformMessages` family) no longer vetoes the id; the id + wins over content churn. The ledger primary key stays the three-part + `(session_id, msg_id, content_hash)` — the hash remains the safety belt + against wrong-binding, it just no longer outranks a valid client id. +- #4863: id claims are taken only when a planned turn is actually + dispatched (claim-after-guard), so a turn rejected by the single-flight + guard no longer burns its ids. + +What this does NOT change: the model wire still carries no message-level +ids. Tool-call ids stay what they always were — model-minted pairing keys +the client echoes back, not identities (#2821 re-mints them on projection +rebuild). Proxy-mode full resends still reconcile through the order-centric +ladder below. The envelope only exists where a bili plugin lane is present +and the host can vouch the full set. + ## What "original bytes never change" actually means The claim is directionally right with one mandatory refinement: **bytes @@ -369,8 +407,14 @@ byte-identical. Identity bookkeeping is never involved. ## Future direction -If a future host or lane offers a genuinely stable client-side message id, -it may be adopted **only as an additional join hint** — never as a replacement +That question is now answered on the plugin lanes: hosts that can vouch a +stable id DO ship it (#4491 msgid envelope, above), and where it is present +it outranks order and hash as the ledger identity (#4834 id-first resolution) +— with the content-hash remaining the three-part-key safety belt. Hosts that +cannot vouch ids (and the model wire itself, which has none) keep the +content-hash base. If a future host or lane offers a genuinely stable +client-side message id, it may be adopted **only as an additional join hint** +on the hash-based lanes — never as a replacement for the content-hash base of the LEDGER join. The OUTBOUND join is a different question and already has its answer: with `snapshotRender` on, the outbound prefix is keyed by server snapshot `seq` (#3756, above) and the