Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/msgid-infrastructure.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Message-ID lane infrastructure: the sessionid+msgid production/consumption map

> Status: companion doc for the gap fix mandated by the #5826 release review (owner ruling 2026-11-07). Identity-ladder doctrine lives in message-identity.md (kernel) and session-identity.md (session side); this file does ONE job — explain where sessionid+msgid, the cross-lane public substrate, is produced, how it travels, who validates it (six copies), where it is pinned, its five consumers, and its declared trust boundaries. Read this before touching any of it.
> Status: companion doc for the gap fix mandated by the #5826 release review (owner ruling 2026-11-07). Identity-ladder doctrine lives in message-identity.md (kernel) and session-identity.md (session side); this file does ONE job — explain where sessionid+msgid, the cross-lane public substrate, is produced, how it travels, who validates it (six copies), where it is pinned, its six consumers, and its declared trust boundaries. Read this before touching any of it.

## 1. What it is

Expand Down Expand Up @@ -48,15 +48,16 @@ One-off closure from the review: `parseTailIds` alone lacked the reserved-prefix

The unified-store `messages.msg_id` column is the ONLY id truth: ingestion pins validated ids into rows (T5 ruling: no parallel ledger; skeleton rows carry no ids). Ledger primary key `(session_id, msg_id, content_hash)`; canonical rows key on **MSGID, not content hash** (#4491); fork clones preserve numbers (#4316 L3 PR3/3 `cloneSessionRows`).

## 6. Consumers (five; what question each one asks of the ids)
## 6. Consumers (six; what question each one asks of the ids)

| Consumer | Location | Question answered | Id-usage boundary |
|---|---|---|---|
| ① storage-column identity | unified store `messages.msg_id` | join key for every settle/reconcile path | the single identity-space entry point; the other four's id evidence ultimately folds in here |
| ① storage-column identity | unified store `messages.msg_id` | join key for every settle/reconcile path | the single identity-space entry point; the other five's id evidence ultimately folds in here |
| ② session-resolve last-resort rung #5719 | tryMsgIdRescue (src/session-resolve.ts ~:324; fork-exit call sites ~:432/:481/:501) | after bytes/tag evidence was destroyed (compaction deleted cells, system rewrite deleted fragments), is this request the SAME session? | ANY id overlap under the same raw key ⇒ same session; NOTE: no LANE attribution (§7 boundary B) |
| ③ settlement membership probe #4805 | probeRequestMembership (src/request-membership.ts; call sites src/server/prepare-openai.ts:597, prepare-anthropic.ts:640; consumed cache-ledger.ts:1866) | was this id admitted by this session's ledger? needle = latest admitted id | **pure evidence — never enters the identity space**, never re-keys/forks; worst case one unsettled cycle, self-heals |
| ④ incremental-lane canonical rows #4491/#4834 | src/incremental-lane.ts (validateMsgIds :623, claim-reuse 400 :628, cache-control rekey :631-635; dsh-side burned-claim pre-screen ~:2584) | which canonical rows did this delta turn append? | claim semantics: re-sent claimed id = retry-storm signal, turn 400s |
| ⑤ watermark tail continuity v2-v4 #5403/#5418 | PF_TAIL_IDS_HEADER (src/server/fastpath.ts:48) | does the client's delta window still originate where the server seed says? | under v4 width self-consistency (len=T−K+1, re-verified server-side) the count echo degrades to diagnostic-only; ids prove WINDOW ORIGIN ONLY, never prefix integrity (owner ruling 2026-11-02; §7.4 blind spot stands, §7 boundary A) |
| ⑥ fork-adopt msgid track #6262 | tryForkAdoption(src/agent/fork-adopt.ts — client slot-matches collected host ids against the parent snapshot's rawIds) + handlePluginFork(src/plugin.ts — request.prefixMsgIds[k] === prefix[k].rawId over the child's resent prefix) | after a byte drift (host rewrite/re-render of an earlier message), is this prefix still the parent session's? | compared ONLY against rawId — never enters the identity space, never re-keys; ANY mismatch ⇒ silent fallback to the strict byte fork (fields ignored); malformed field shape ⇒ 400 INVALID_REQUEST; reuses identifier(), no new validation copy (§4) |

## 7. Trust boundaries (declared as part of the contract, not bugs)

Expand Down
7 changes: 4 additions & 3 deletions docs/msgid-infrastructure.zh-CN.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# 消息身份公用设施:sessionid+msgid 的生产与消费地图

> 状态:#5826 release review(2026-11-07)owner 裁定落地的缺口修复配套文档。
> 身份层级教义见根目录 message-identity.md 与会话侧 session-identity.md;本文只负责一件事——把 **sessionid+msgid 这套跨车道的公用设施**的生产端、传输形态、校验副本、持久化单一事实源、五个消费方、信任边界,一次说明白。改动任何一处之前先读这里。
> 身份层级教义见根目录 message-identity.md 与会话侧 session-identity.md;本文只负责一件事——把 **sessionid+msgid 这套跨车道的公用设施**的生产端、传输形态、校验副本、持久化单一事实源、六个消费方、信任边界,一次说明白。改动任何一处之前先读这里。

## 1. 这是什么

Expand Down Expand Up @@ -49,15 +49,16 @@

统一存储 `messages.msg_id` 列是唯一 id 事实源:ingest 时把校验过的 id 钉进行(T5 裁定:无双平行 ledger、skeleton 行不携 id)。账本主键 `(session_id, msg_id, content_hash)`;canonical 行按 **MSGID 而非内容 hash** 键控(#4491);fork 克隆保号(#4316 L3 PR3/3 `cloneSessionRows`)。

## 6. 消费方(五个,id 各自回答什么问题)
## 6. 消费方(六个,id 各自回答什么问题)

| 消费方 | 位置 | 回答的问题 | id 使用边界 |
|---|---|---|---|
| ① 存储列身份 | unified store `messages.msg_id` | settle/reconcile 一切 join 的主键 | 唯一身份空间入口,其余四方的 id 证据最终都折到这里 |
| ① 存储列身份 | unified store `messages.msg_id` | settle/reconcile 一切 join 的主键 | 唯一身份空间入口,其余五方的 id 证据最终都折到这里 |
| ② session-resolve 最后一档 #5719 | tryMsgIdRescue(src/session-resolve.ts ~:324;fork-exit 调用点 ~:432/:481/:501) | 字节/标签证据被压缩或 system 重写摧毁后,这条请求还是不是既有会话? | 同一 raw key 下**任意 id 重叠 ⇒ 认定同会话**;注意它不做 lane 归属判定(§7 边界 B) |
| ③ 结算 membership 探针 #4805 | probeRequestMembership(src/request-membership.ts;调用点 src/server/prepare-openai.ts:597、prepare-anthropic.ts:640;消费 cache-ledger.ts:1866) | 这个 id 是否已被本会话账本准入?needle = 最近准入 id | **纯证据,永不进身份空间**、永不 re-key/fork;误伤仅致一轮结算未决,自癒 |
| ④ 增量车道 canonical 行 #4491/#4834 | src/incremental-lane.ts(validateMsgIds :623、认领复用 400 :628、cache-control rekey :631-635;dsh 侧 burned-claim 预筛 ~:2584) | 本轮增量追加了哪些正史行? | 认领语义:已认领 id 再上行=重试风暴信号,整回合 400 |
| ⑤ 水位尾连续性 v2-v4 #5403/#5418 | PF_TAIL_IDS_HEADER(src/server/fastpath.ts:48) | 客户端 delta 窗口的起点还和服务端 seed 对齐吗? | v4 宽度自洽(len=T−K+1,服务端重验)时计数回声降为纯诊断;id 只证**窗口来源**,永不证前缀完整性(owner 裁定 2026-11-02;§7.4 盲点照旧,§7 边界 A) |
| ⑥ fork 收养 msgid 轨 #6262 | tryForkAdoption(src/agent/fork-adopt.ts——客户端逐格比对采集到的宿主 id 与父快照 rawId)+ handlePluginFork(src/plugin.ts——request.prefixMsgIds[k] === prefix[k].rawId,对子会话重发前缀逐格验) | 字节漂移(宿主改写/重渲染了较早消息)之后,这段前缀还是不是父会话的? | 只与 rawId 比对——永不进身份空间、永不 re-key;任一失配 ⇒ 静默回退严格字节 fork(附加字段忽略);字段形状非法 ⇒ 400 INVALID_REQUEST;复用 identifier(),无新增校验副本(§4) |

## 7. 信任边界(声明为契约的一部分,不是 bug)

Expand Down
10 changes: 7 additions & 3 deletions docs/plugin-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,15 +191,19 @@ Discover `capabilities.fork` from the manifest; do not infer support from the pa
"parentRevision": "<snapshot.parentRevision>",
"branchPoint": { "messageCount": 1, "orderHash": "<prefix order hash>" },
"orderedMessages": [{ "rawId": "h_example", "ref": "m00001", "identityHash": "<snapshot identity>" }],
"idempotencyKey": "stable-operation-id"
"idempotencyKey": "stable-operation-id",
"prefixMsgIds": ["host-uuid-1", "host-uuid-2"],
"driftFrom": 1
}
```

`prefixMsgIds` and `driftFrom` are optional (#6262). On a parent whose kernel ids are the host's own stable message ids (a pinned session, where snapshot `rawId` IS the host id), a client may additionally present the host id of every matched prefix message, plus the hash-run length as `driftFrom` (`0 ≤ driftFrom < prefixMsgIds.length`). The server accepts a byte-drifted prefix only when every presented id equals the parent's `rawId` — ids prove prefix identity where bytes no longer match — and marks the receipt `status: "drifted"` (or `"expanded+drifted"` when a block also crosses the branch) with the verified `driftFrom` echoed. A gate failure never fails the fork: the request falls back to the strict byte-verified fork (`"exact"`/`"expanded"`) and the extra fields are ignored. Malformed shapes (wrong length, bad identifier, `driftFrom` out of range or without `prefixMsgIds`) return `400 INVALID_REQUEST`. Parents without host-pinned ids make the track a no-op, byte path unchanged.

`protocolVersion` is `2` (current) or `1` (legacy) in both directions; the proxy accepts either (`acceptsProtocolVersions` in the manifest advertises this) and rejects anything newer with `400 INVALID_REQUEST`. Under protocol 2 the `parentRevision` token covers exactly the session id, the ordered-prefix length (`messageCount`), and the active block set — so routine stats churn (context estimates, usage bookkeeping) no longer changes the revision, while any structural change (messages appended or rewritten, a block folded or unfolded) still does. Protocol-1 clients that present the legacy whole-store revision token are still matched during the transition window; treat `parentRevision` as opaque in both cases: on `409 PARENT_REVISION_CONFLICT` take a fresh snapshot and re-match, regardless of which protocol version you speak.

The branch is an ordered prefix, including an empty prefix. Compute its existing order hash as lowercase SHA-256 of UTF-8 `JSON.stringify(orderedMessages)` with each identity object's keys in order `rawId`, `ref`, `identityHash`; for a full prefix reuse snapshot `orderHash`. Do not hash descriptors or folded text. Extra child messages, including a final model reply that has not left the proxy yet, may be appended on the child's first model request after copying the matched prefix.

New success is `201`; exact payload replay is `200` with `replayed: true`, even after restart or later parent changes. Both return `status`, `parentRevision`, `childRevision`, `sessionId`, `branchPoint`, `inheritedBlocks`, and `expandedBlocks`. Reusing the child id with another parent, key or payload returns `409 CHILD_CONFLICT`. Invalid shapes/version/identifiers return `400 INVALID_REQUEST`; absent parent returns `404 PARENT_NOT_FOUND`; stale parent returns `409 PARENT_REVISION_CONFLICT`; altered order/raw/ref/identity returns `409 BRANCH_POINT_CONFLICT`. Missing nested originals return `409 PARENT_STATE_INCOMPLETE` with `status: unavailable`. Publication/persistence failure returns `503 FORK_FAILED` with `status: unavailable` and exposes no child; the same request may be retried.
New success is `201`; exact payload replay is `200` with `replayed: true`, even after restart or later parent changes. Both return `status`, `parentRevision`, `childRevision`, `sessionId`, `branchPoint`, `inheritedBlocks`, and `expandedBlocks` (plus `driftFrom` when the msgid track accepted a drift, above). Reusing the child id with another parent, key or payload returns `409 CHILD_CONFLICT`. Invalid shapes/version/identifiers return `400 INVALID_REQUEST`; absent parent returns `404 PARENT_NOT_FOUND`; stale parent returns `409 PARENT_REVISION_CONFLICT`; altered order/raw/ref/identity returns `409 BRANCH_POINT_CONFLICT`. Missing nested originals return `409 PARENT_STATE_INCOMPLETE` with `status: unavailable`. Publication/persistence failure returns `503 FORK_FAILED` with `status: unavailable` and exposes no child; the same request may be retried.

Boundary handling is `exact` when complete blocks fit inside the prefix, `expanded` when a block crosses it (crossing summaries are omitted, affected nested descendants are expanded, matched originals are retained), and `unavailable` when safe reconstruction is impossible. Blocks, nested caches, raw messages, issued ref namespace, token snapshots, rules and prefix CCR payloads are independent copies. Note that the child inherits the parent's ENTIRE issued-ref namespace (dead refs included): refs are never reused, so forks keep allocating fresh refs without colliding with ids a sibling may still hold. The child never reads its parent's later state as a fallback. Registration and first-request resume heuristics do not replace its copied state. With existing persistence enabled, raw snapshots, receipts, nested originals and the child's CCR payloads survive restart; with persistence disabled, durability is not promised. Atomic child publication/idempotency covers concurrent requests to one proxy process; concurrent writers sharing one state directory across processes are not supported by this protocol.

Expand All @@ -211,7 +215,7 @@ Marker divert (#3248 phase 2 — native extensions): instead of switching the pr

中文增量要点(#3248):`POST /__bili/plugin/turn`,首次/恢复用 `sinceSeq:0` 全量(种子隐含 `resync:true`,永不 409);此后每轮只发新增(含上一轮 assistant 回复,客户端自持)。响应即上游原文,`x-bili-applied-seq` 为应用后的长度,下一轮作 `sinceSeq` 回传。409 必须按 serverSeq 对齐或全量重同步;503 表示未开 storage.unified。会话身份沿用客户端自己的 conversation id——增量与全量重发同一会话可任意切换,折叠/工具/状态栏不变。阶段 2(标记分流):普通模型请求上盖 `x-bili-plugin-turn: 1` 头 + 增量形状 body(顶层 conversationId/sinceSeq/messages、无顶层 model)即可,增量形状 turn 属 lane 流量,与直连 turn 端点同信任级(#3343):非回环/隧道/不可信源一律 403(其探针本就 403,正常宿主不会远程发增量);管理前缀(`/__bili/`、`/__acp/`)永不走分流。lane 按原始 URL 转发;探针 `GET /__bili/plugin/turn`;开关 rollout `pluginIncrementalTurns`(默认 0%=零变化,`BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1` 或配置强制开)。客户端状态机在 src/agent/incremental-client.ts,新宿主复用勿重写。已接宿主:pi/pi-native(同一扩展工厂,后者零代码继承)、dsh-native(fetch 补丁 beforeSend/headersFor/onResponse)、opencode(http.request 钩子 incremental 适配回调换 body+盖章)、omp(无头事件通道,读 body 里 prompt_cache_key 当会话 id,直接在 init 上盖章);OpenAI 工具形状(tool_calls/role:tool)自动回退全量重发;每 URL 单飞行+30s 过期。

中文集成要点:fork 协议现为 2(服务端同时接受 1);先读取 manifest 能力及 snapshot,按原文、role、工具身份确认匹配前缀,仅提交该前缀的 orderedMessages,不猜测多模态分支点。被 `compress.protectedTools` 硬排除的消息在两个数组中均为 `"ref": null`(参与顺序与身份、无地址 ref),与其他前缀条目一样逐字提交;ref 合法值仅 `mNNNNN` 字符串或 `null`。父 revision 变化必须重新取快照并重新匹配。子会话使用独立稳定 id,首次请求可追加尚未出站的模型回复,不能重置继承的 refs/blocks。手动摘要使用 snapshot 的 refs 和 `expectedRevision: snapshot.parentRevision` 调用公开 tool;摘要期间历史发生变化时返回 409,不执行旧摘要。状态栏读取 `contextTokens`、`contextTokensSource`、`contextTokensAt`、`model`、`contextLimit`、`contextGeneration`、`sessionId` 与 `sessionRevision`,不得用累计账单 input/cache 冒充有效上下文。手动 compress/decompress 后为独立时间及 generation 的 estimate,不改累计账单或 last-usage 基线;下一次正数 usage 才恢复 usage,零值或缺失 usage 不得复活旧观测。未知来源返回 unavailable,tokens/time/generation 为 null。
中文集成要点:fork 协议现为 2(服务端同时接受 1);先读取 manifest 能力及 snapshot,按原文、role、工具身份确认匹配前缀,仅提交该前缀的 orderedMessages,不猜测多模态分支点。父会话若已 pin 宿主 msgid(snapshot 的 rawId 即宿主 id),可另附 `prefixMsgIds`(每个前缀消息的宿主 id)与 `driftFrom`(哈希前缀长度):宿主 id 全等时服务端接受字节已漂移的前缀,回执 `status: "drifted"`(跨块时 `"expanded+drifted"`)并回显已验证的 `driftFrom`;gate 失败不影响 fork 成功(回退严格字节 fork,附加字段被忽略);字段形状非法返回 400,未 pin 的父会话该轨自动无效。被 `compress.protectedTools` 硬排除的消息在两个数组中均为 `"ref": null`(参与顺序与身份、无地址 ref),与其他前缀条目一样逐字提交;ref 合法值仅 `mNNNNN` 字符串或 `null`。父 revision 变化必须重新取快照并重新匹配。子会话使用独立稳定 id,首次请求可追加尚未出站的模型回复,不能重置继承的 refs/blocks。手动摘要使用 snapshot 的 refs 和 `expectedRevision: snapshot.parentRevision` 调用公开 tool;摘要期间历史发生变化时返回 409,不执行旧摘要。状态栏读取 `contextTokens`、`contextTokensSource`、`contextTokensAt`、`model`、`contextLimit`、`contextGeneration`、`sessionId` 与 `sessionRevision`,不得用累计账单 input/cache 冒充有效上下文。手动 compress/decompress 后为独立时间及 generation 的 estimate,不改累计账单或 last-usage 基线;下一次正数 usage 才恢复 usage,零值或缺失 usage 不得复活旧观测。未知来源返回 unavailable,tokens/time/generation 为 null。

### Incremental cache-mark updates

Expand Down
Loading
Loading