Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ The full file-by-file map lives in [reference/architecture.md](reference/archite
11. **顺序为中心,hash 为辅助 (order-centric reconciliation — owner rule, 2026-10-12, #2821/#2915)** — fold-state reconciliation must treat ORDER as the primary carrier of message identity across resent histories: position claims verified spans (with per-cell fingerprint gates), content hashes and tool-call ids are auxiliary evidence, and NO single dimension's miss may cascade into total unfold. The identity dimensions survive different churn classes — order survives re-encoding AND id re-minting, content survives id churn, ids survive re-encoding — so the ladder is the union of their survival domains, and each rung fails soft: an id/fingerprint mismatch voids that CELL's claim only; the positional scans resync at the next run of same-index agreements (MIN_RESYNC_RUN consecutive matches, bounded gaps) and keep claiming verified spans behind the break. Genuinely changed cells (real edits, regenerations) stay honestly unmatched and re-enter the wire — "never fuzzy similarity" (mis-attribution is the most expensive failure, MESSAGE-IDENTITY.md) is unchanged: the hash stays a HARD per-cell gate, it just no longer holds a global veto over its neighbors. Escalation to "substrate destroyed" may only count gaps that survived resync AND the content passes (byTool/byTwin/byNorm/byTurn). #2821 was the canonical violation: a mid-session model switch re-minted every tool-call id on re-projection (content byte-identical), both positional flank scans died at their FIRST cell (changed preamble at the head, appended turn at the tail), 3289 covered ids went missing, three passes → destroyed verdict → total unfold → doomed-probe 502. #2396's detection-only twins are promoted to a claiming pass (Pass 1.5, k-th discipline + length guard) so id re-mints recover even without a stored positional copy — and after one churn pass the self-saved {ids, canon} copy upgrades recovery to the positional path. A Pass-0 gap cell (order break) is resolved by tiers, owner rule 2026-10-12: any exact channel that matches claims it; failing that, simhash corroboration (≤8 bits on the 64-bit word-shingle sketch — measured margins: re-wording ≈6-8 bits inside, negation flips ≈11 / word drops ≈13 / key reorders ≈21 outside) claims it; failing that, the ORDER BRACKET — same index strictly between VERIFIED claims on both flanks — asserts the SLOT (the owner-sanctioned delete+insert tradeoff: coverage rides the slot, the stale bytes do not); with zero corroboration and no fence the cell stays honest unmatched drift. Deletion is honest loss: the flanks absorb the shift, the deleted id never gets a bracket claim (its slot cell is consumed by the verified tail). Tests: `tests/issue2915-order-centric-reconcile.test.ts`.
12. **资格判定与执行共用单一投影 (shared unit-projection judgment, #3236)** — whether a candidate range is foldable has exactly ONE definition: `projectUnit` in `kernel/src/unit-projection.ts` (applySingleRange's pre-block sequence extracted verbatim: boundary widening → carrier/coverage/media/protected-tool carve → protected-zone carve → turn-integrity withdrawal → emptiness), returning either a structured projection or the ready-to-throw fatal error (`zone`/`split`). The apply side throws it byte-identically; the recommend side re-screens every FINAL merged unit through it and demotes fatal units to gaps (no re-merge after filtering — anti-oscillation); the acp_status uncompressed view runs it per displayed row and appends an additive ⚠ flag (diagnostic surface: rows are flagged, never dropped). The judgment must run on the EXACT array class `applyCompression` executes on: hosts whose render view differs from their apply class pass it via `ProcessTurnInput.applyView` / `StatusReportOptions.originalMessages` (plugin lane = raw history per #2627; relay loop = `prepared.originalMessages`); absent it, judgment falls back to the pre-prune original (#2663) or the given view. The bug fixed: recommend judged turn-integrity over the GLOBAL compressible set while apply judged the PER-SUBMITTED-RANGE set — a turn whose partners sat in different advertised units passed global screening yet split per range, producing a permanent advertise→reject loop ("Range would split N turn(s)"). Tests: `kernel/tests/issue3236-recommend-apply-consistency.test.ts`.

13. **出站历史渲染:快照即会话(snapshot render — #3756,rollout `snapshotRender` 默认 0%)** — 消息身份的演进分三段:content-hash 认身份(账本 join,至今有效)→ order-centric reconcile(#2821/#2915/#3641:顺序为主、hash 为辅助证据、判决分类)→ **快照渲染**:对绑定了 session id 且统一存储可读的会话,prepare 层的出站**前缀**由服务端正史渲染(`src/server/snapshot-render.ts`),消息身份 = 服务端行 `seq`,hash/签名只作链回溯兜底(定位客户端逐字尾巴的 join 点);覆盖由块的 seq/签名集**引擎自拥**(不信 reconcile 的 re-key 产物),锚从块状态原位重渲;尾巴保持客户端逐字(缓存稳定)。任何失步(开关关/无活跃块/存储不可读/链不上)整轮回退 legacy 路径——**降级,不猜**;客户端头部 checkpoint 一轮采用滞后,永不复活已覆盖行。观测与渲染刻意分离:`reconcileFoldCoverage` 继续观察客户端视图。完整教义与诚实边界:MESSAGE-IDENTITY.md「快照渲染引擎」。规则推论:凡讨论「覆盖 id 失配/destroyed 误诊/断缓存」类 bug,先问该会话是否走 D 车道——D 上这类病理结构性不存在,修不到点子上等于在修已死的路径。

### Install-Lane & Update-Ownership Contract (#1196)

Every bili presence on a machine follows ONE contract — decide changes against it, not ad hoc:
Expand Down Expand Up @@ -231,7 +233,7 @@ Commit convention: `feat:` / `fix:` / `refactor:` / `test:` / `docs:` / `release
- **Root cause, not symptom (疏, not 堵 — first principle for every fix).** A fix removes the mechanism that PRODUCES the defect; it never merely hides the defect's visible trace at whatever layer is cheapest to intercept. Before writing any fix, name the producer of the offending bytes/behavior and fix it at the source or decision point — never by rewriting or filtering payload in transit to make the symptom invisible. Litmus test: "if the same root cause surfaced in a different shape tomorrow, would this fix still hold?" If not, it is a mask, and a mask is a latent bug. Canonical failure: #933 (model-echoed render tags leaking into a TUI) was "fixed" by stripping tool-call arguments mid-stream; the mask itself became #1039's silent data corruption once users legitimately wrote tag-shaped payloads through write/edit/bash. When the true root cause is out of scope (another repo, a product decision), fix the in-scope part, leave a documented boundary note pointing at the real home of the fix, and do NOT compensate in the wrong layer (#1039's echo-noise belongs on the injection side — host renderTags policy — not on the wire).
- **Never silently clobber or drop user config.** Any read-modify-write on user config needs a parse-state guard; reject malformed input loudly (HTTP 400/409) instead of merging into defaults or dropping fields. Whitelists must be complete (#155: a missing key silently erased custom compress prompts).
- **Sane defaults & fallbacks.** Fallback values must be reasonable (never a too-small value that thrashes); a static/fallback source must always lose to a fresher authoritative source when both are cheaply available (#282: window fallback 200K/min 100K, not 64K; the bundled registry snapshot must not outrank the live registry).
- **Prefer native stable identifiers.** Use a client's native stable session id when available (survives credential/model/provider switches); report clients that expose none; never build identity from derived hashes that drift on switch (#280).
- **Prefer native stable identifiers.** Use a client's native stable session id when available (survives credential/model/provider switches); report clients that expose none; never build identity from derived hashes that drift on switch (#280). With the snapshot-render lane on (#3756), the session id + server transcript IS the outbound message identity — see Key Design Decisions 13 and MESSAGE-IDENTITY.md.
- **Wire fidelity (host-side duty).** Never alter upstream protocol shape beyond intended injection: preserve tool_call ids/ordering, SSE structure, and upstream invariants (e.g. `compaction_trigger` must remain the last input item, #283/#209). Reason in BOTH compression modes (§6).
- *Tool-call arguments are user intent (#1039).* Any payload a host will EXECUTE or PERSIST — tool-call `arguments` in every wire shape (OpenAI `tool_calls[].function.arguments`, Anthropic `input_json_delta.partial_json` / `tool_use.input`, Responses `function_call_arguments.*`), fragment or whole — is forwarded **byte-exact**. NEVER filter, strip, or "clean" it, not even to remove model-echoed render tags: a shape-based filter cannot distinguish an echo from a literal the user genuinely wants written (bash command strings, write/edit file contents), so any such "fix" silently corrupts executed/persisted data. Echoed tags surfacing in a host TUI is cosmetic noise — the fix belongs on the injection side (host renderTags policy, #933), never in the argument path. Tag-echo stripping applies to model PROSE only (content/reasoning/thinking/summary text fields); see the invariant block atop `src/loop/tag-echo-filter.ts`.
- *Kernel-owned split:* the FORMAT CONTRACT of the kernel-emitted ACP artifacts (the compression tags, block refs, `acp_summary` structure — KDD #2) and the **id-never-reused guarantee** belong to **acp-kernel**, not this repo (§2 Kernel Contract). This repo only consumes them faithfully. Codifying the kernel-side spec is a change under `kernel/` — human-gated per the Kernel Boundary.
Expand Down
55 changes: 52 additions & 3 deletions MESSAGE-IDENTITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,49 @@ migration, applies to old and new sessions alike, storage-independent
(legacy JSON, shadow and full modes behave identically; bookkeeping rides
session metadata). The kernel ledger's content-hash contract is untouched.

## The snapshot render engine: outbound identity moves server-side (#3756)

#3641 classified verdicts when no channel verifies. #3756 removes the
dependence itself for the outbound path: with the `snapshotRender` rollout
feature on (default 0%, `BILI_ROLLOUT_SNAPSHOT_RENDER`), the prepare layer
no longer rebuilds history FROM the client's resent bytes — it renders the
outbound prefix FROM the server's own unified-storage transcript
(`src/server/snapshot-render.ts`):

- **Identity ladder, final form.** Conversation identity = the session
binding (host-native session id whenever one exists — §"Prefer native
stable identifiers"). Message identity inside a live conversation = the
server snapshot row (`seq`), never re-minted. Content hashes/signatures
demote to **chain-fallback evidence only** — they locate the divergence
point where the client's verbatim tail hands over to the server prefix
(id fast path, `role|contentType|toolName|text` + image-length signature
fallback — the #3642 canonical form, so a full id re-mint still links).
- **Coverage is self-owned by the engine.** Covered rows are suppressed by
the block's seq/signature coverage set; `acp_summary_*` anchors re-render
in place from block state. The engine does NOT trust the reconcile
ladder's re-keyed block ids for this — reconcile may legitimately re-key
blocks to freshly minted client ids (#3752's exact failure shape when a
16384-anchor cap then dropped them).
- **The tail stays client-verbatim.** Everything above the join point is
forwarded byte-for-byte as the client sent it — the unfolded working set
remains the client's own words, and prefix-cache stability is preserved
turn-over-turn. Rewound-away rows sit below the join and never render.
- **Degrade, never guess.** Feature off, no active blocks, storage
unreadable, or the chain cannot link — the whole turn falls back to the
legacy path unchanged. A client-side checkpoint of the head is adopted
with a one-turn lag (the view is the previous turn's observation) and can
never resurrect covered rows.
- What this kills structurally: the #3752 anchor-cap cliff (MAX_ANCHORS ×
id re-mints × stateless resend) and the #3751 image `_N` family churn —
neither can drop or duplicate folded history anymore, because the
rendered prefix follows block seqs and content signatures, not client
bytes.

The #3641 reconcile/verdict layers stay: they remain the safety net for
the lanes where D is off, unavailable, or degraded — and the observation
channel (`reconcileFoldCoverage`) still watches the CLIENT view (render and
observation are deliberately separate inputs).

## Known cost of the hash axiom — and how it was fixed

Same bytes ⇒ same identity has a cost face: #1476 — a user re-sends a short
Expand Down Expand Up @@ -307,12 +350,18 @@ byte-identical. Identity bookkeeping is never involved.

If a future host or lane offers a genuinely stable client-side message id,
it may be adopted **only as an additional join hint** — never as a replacement
for the content-hash base. Until then: bytes are the anchor, hashes are the
join, mNNNNN is the ledger, tags are the view.
for the content-hash base of the LEDGER join. The OUTBOUND join is a
different question and already has its answer: with `snapshotRender` on,
the outbound prefix is keyed by server snapshot `seq` (#3756, above) and the
content-hash ladder serves only as the chain-fallback that finds the join
point. Summary: bytes are the anchor, hashes are the ledger join, `seq` is
the render join (rollout-gated), mNNNNN is the ledger, tags are the view.

Related: #1496 (this document's prompt), `SESSION-IDENTITY.md` (session
granularity), #1476 + kernel #459/#463 (echo discrimination), #242/#1475
(Responses id constraints), #206/#673 (tag echo), #1479/#1482 (strict-echo
repair), #1039 (tool-call byte-exactness invariant), #2396 (host-rewritten tool-call ids across a provider switch), #2480 (positional fold coverage;
`tests/identity-proof.test.ts` is its proof matrix), #2695/#3588/#3640/#3641
(snapshot-as-session doctrine; `tests/fold-anchor-doctrine.test.ts`).
(snapshot-as-session doctrine; `tests/fold-anchor-doctrine.test.ts`),
#3642 (canonical cross-protocol identity — the chain-fallback's signature
form), #3756 (the snapshot render engine; `tests/snapshot-render.test.ts`).
12 changes: 12 additions & 0 deletions MESSAGE-IDENTITY.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,18 @@ bili 的应对遵循上文的失败模式公理:**不猜** old→new 配对(猜

兼容性:教义是匹配/判决层变更——零迁移,新老会话一体适用,与存储无关(legacy JSON、shadow、full 三模式行为完全一致;记账走 session metadata)。内核账本的 content-hash 契约分毫米未动。

## 快照渲染引擎:出站身份移到服务端(#3756)

#3641 定论了「没有通道验证成功时怎么判」;#3756 则把依赖本身从出站路径上拆掉:灰度开关 `snapshotRender`(默认 0%,`BILI_ROLLOUT_SNAPSHOT_RENDER`)开启时,prepare 层不再**从客户端重发的字节**重建历史,而是**从服务端统一存储正史**渲染出站前缀(`src/server/snapshot-render.ts`):

- **身份阶梯最终形态**:会话身份 = 会话绑定(宿主原生 session id 优先,凡有即用);会话内消息身份 = 服务端快照行(`seq`),永不重铸;内容 hash/签名降级为**仅作链回溯兜底证据**——定位「客户端逐字尾巴让位给服务端前缀」的分歧点(id 快路径 + `role|contentType|toolName|text`/图片长度签名兜底,即 #3642 的 canonical 形态——id 全部重铸仍可链接)。
- **覆盖由引擎自拥**:被覆盖行按块的 seq/签名覆盖集抑制;`acp_summary_*` 锚从块状态原位重渲。引擎**不信任** reconcile 阶梯 re-key 后的块 id——reconcile 合法地把块 re-key 到客户端新铸 id(恰是 #3752 在 16384 锚封顶下把它们丢掉的失败形状)。
- **尾巴保持客户端逐字**:分歧点之上按客户端原字节转发——未折叠工作区仍是客户端自己的话,轮轮前缀缓存稳定。rewind 掉的行位于 join 之下,永不渲染。
- **降级,不猜**:开关关/无活跃块/存储不可读/链接不上——整轮回退 legacy 路径,零变化。客户端头部 checkpoint 有一轮采用滞后(视图=上一轮的观察),且永不复活已覆盖行。
- 结构性消灭:#3752 的锚封顶悬崖(MAX_ANCHORS × id 重铸 × 无状态重发)与 #3751 的图片 `_N` 族位移 churn——渲染前缀跟随块 seq 与内容签名,不再跟随客户端字节,这两类都再也无法丢/重折叠历史。

#3641 的 reconcile/判决层全部保留:它们仍是 D 关闭/不可用/降级时的安全网;观测通道(`reconcileFoldCoverage`)继续观察**客户端**视图(渲染与观测刻意分离)。

## hash 公理的已知代价——以及修复方向

同字节 ⇒ 同身份有代价面:#1476——用户原样重发与已折叠内容相同的短文本;裸 `h_` id 重新派生;`isCovered()` 吞掉新消息。注意**修复方向**:kernel #459 加实例重编号(`_1/_2…`),#463 加 `lastPassIds` 快照区分折叠后**回声**(id ∈ covered ∧ ∈ 上一轮 → 保持 id,prune 吞掉)与真**新实例**(id ∈ covered ∧ ∉ 上一轮 → 重编号到空闲 `_k`)。content hash 始终是基座,只在其上叠加实例判别——连 hash 公理自己的 bug 都靠**保留 hash** 修,而不是退回位置 id。
Expand Down
19 changes: 19 additions & 0 deletions SESSION-IDENTITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,25 @@ request matched the openai-lane session at prefix depth 9/11, no new
anonymous session, no re-fold, answers served from the fold summary, cache
hit 97 % within two rounds. Captures and log in the #2791 thread.

## Snapshot render: the session id becomes the whole message identity (#3756)

The anonymous-adoption ladder above exists because the wire bytes were the
only record of an unauthenticated conversation. For sessions with a bound
id (host-native session header or affinity key) and unified storage
available, #3756 closes the loop: the OUTBOUND history prefix is rendered
from the server's own transcript keyed by that session id
(`src/server/snapshot-render.ts`, rollout feature `snapshotRender`, default
0%). Message identity for the rendered prefix is the server row `seq` —
content hashes demote to chain-fallback evidence that only locates where
the client's verbatim tail joins the server prefix. Client edits, host
checkpoints and protocol switches can no longer move, drop or duplicate
folded coverage, because coverage is computed from the block's seq/
signature sets, not from what the client happens to resend. The
adoption ladder above remains the fallback for the lanes where the feature
is off, the storage is unavailable, or the chain cannot link — degrade,
never guess. Full mechanism and honesty boundaries:
`MESSAGE-IDENTITY.md` §"The snapshot render engine".

## Rejected alternatives

| Direction | Why rejected |
Expand Down
Loading
Loading