Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag

## [Unreleased]

### Added

- **seq ledger foundations — block interval bounds + ledger-numbered refs (#4316 layer 3, phase 1).** Block coverage now carries its ledger interval: `planIngest` fills `blocks.start_seq/end_seq` with the min/max unified-store seq of the covered messages (`-1/-1` when no rows resolve) — coverage bounds are id-independent, so client-side byte churn can never invalidate them. `CoreMessage.ledgerSeq` sidecar: store-served views (snapshot-render prefix rows) stamp the row's `(session, seq)`; `assignRefs` issues the ref as the direct display of that number (seq+1 in the 1-based display space), advancing the allocation cursor past the stamp so counter-issued refs never collide with future stamps. A store-served message's number is assigned once at ingest and is structurally immune to the #4316 churn family (model-switch re-stamps, image re-encodes re-minting every content-hash id). Stakes first in `tests/seq-ledger.test.ts`: interval bounds (S1), ref-set stability across a full id re-mint under snapshot render (S2), and the kernel seam (S3). Follow-ups: incremental-lane stamping, fork `cloneRows` number-preserving copy, and the one-time legacy re-mint window handling — tracked in #4316.

### Storage — 行身份与史官化(#4390,storage review 四个 p2 + 三个顺路 p3)

- **行身份去污染(D4-1)**:统一存储 messages 行新增 `identity` 列——跨保存 JOIN 键改为**位置无关**序列化(剥离 `id`/`wireIndex` 后哈希)。此前身份掺入 wire 位置元数据,客户端头部截断一条消息就会让后续全部行换血(旧行被清扫、seq 重铸、#4335 的区间账记随之抖动)。`content_hash` 保持字节忠实 CAS 键不变。存量库 v8 迁移:回填 identity 并合并 wireIndex 分裂世代(保最低 seq,重复行退还 CAS 引用)。MySQL 同款(information_schema 探测幂等 ALTER + 回填)。
Expand Down
23 changes: 23 additions & 0 deletions kernel/src/refs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,29 @@ export function assignRefs(
continue;
}

// #4316 layer 3: store-served views stamp ledgerSeq — the unified
// transcript position, assigned once and never re-minted. The ref IS the
// ledger number (direct display), so byte churn cannot renumber
// citations. Collisions (should be impossible — (session, seq) is a PK)
// fall through to the counter path; the cursor advances past the stamp so
// later counter-allocated refs never collide with future stamps either.
const ledgerSeq: number | undefined = message.ledgerSeq;
if (
typeof ledgerSeq === "number" &&
Number.isInteger(ledgerSeq) &&
ledgerSeq >= MIN_INDEX &&
ledgerSeq <= MAX_INDEX
) {
const text = indexToRef(ledgerSeq);
if (!map.byRef[text]) {
map.byRaw[message.id] = text;
map.byRef[text] = message.id;
newlyAssigned++;
if (ledgerSeq >= cursor) cursor = ledgerSeq + 1;
continue;
}
}

const ref = allocateFreeRef(map, cursor);
cursor = ref.index + 1;
map.byRaw[message.id] = ref.text;
Expand Down
10 changes: 10 additions & 0 deletions kernel/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,16 @@ export interface CoreMessage {
* inactive/unknown block folds normally. Reserved-prefix ids
* (`acp_summary_*`) stay kernel-internal and never need this marker. */
summaryOfBlockId?: string;
/** #4316 layer 3 — server-ledger numbering. Set by store-served views
* (snapshot-render prefix rows, incremental canonical rows): the unified
* transcript's (session, seq) for this message, assigned once at ingest
* and never re-minted. When present, assignRefs issues the ref as the
* direct display of this number, so client-side byte churn (model-switch
* re-stamps, image re-encodes) cannot renumber the model's citations.
* Callers must guarantee per-session uniqueness; a collision falls back
* to the counter path for the loser. Not serialized into wire payloads —
* it rides CoreMessage sidecars and the persisted store rows verbatim. */
ledgerSeq?: number;
}

export type CompressionTier = 1 | 2 | 3;
Expand Down
11 changes: 10 additions & 1 deletion src/server/snapshot-render.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,16 @@ function parseRow(content: string, seq: number): BiliMessage | null {
try {
const parsed = JSON.parse(content) as { id?: unknown; role?: unknown };
if (typeof parsed.id !== "string" || typeof parsed.role !== "string") return null;
return parsed as BiliMessage;
const msg = parsed as BiliMessage;
// #4316 layer 3: the row's (session, seq) is the message's permanent
// ledger position — stamp it so assignRefs issues the ref as the
// direct display of the seq (never re-minted across byte churn).
// Store seqs are 0-based; the ref display space is 1-based
// (MIN_INDEX=1, "m00001" is the first number) — the stamp carries the
// DISPLAY index. Rows written before this change simply lack the
// field; the kernel counter path keeps serving them unchanged.
msg.ledgerSeq = seq + 1;
return msg;
} catch {
return null;
}
Expand Down
21 changes: 19 additions & 2 deletions src/storage/ingest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -314,16 +314,33 @@ function finishPlan(
...(e.media && e.media.length > 0 ? { media: e.media } : {}),
}));

// #4316 layer 3: the ledger positions of a block's covered messages —
// min/max seq over the rows its coverage resolves to. Ghost ids (no row
// in this plan's log) are skipped; a block whose coverage resolves to no
// row keeps the -1/-1 "unknown bounds" shape (pre-layer-3
// rows stored 0/0 placeholders; both are simply "no interval evidence"). These bounds are the durable interval form of coverage:
// independent of message ids, so byte churn can never invalidate them.
const seqById = new Map<string, number>();
for (const e of log) if (!seqById.has(e.msgId)) seqById.set(e.msgId, e.seq);

const blocks: BlockRow[] = session.state.blocks.map((b, ord) => {
const views = completeBlockContents(session.blockContents, session.state.blocks).get(b.blockId);
const full = views?.full ?? { text: b.summary, count: 0 };
const one = views?.one ?? null;
const covered = { effectiveMessageIds: b.effectiveMessageIds, directMessageIds: b.directMessageIds };
let startSeq = -1;
let endSeq = -1;
for (const id of b.effectiveMessageIds) {
const s = seqById.get(id);
if (s === undefined) continue;
if (startSeq === -1 || s < startSeq) startSeq = s;
if (s > endSeq) endSeq = s;
}
return {
blockId: b.blockId,
ord,
startSeq: 0,
endSeq: 0,
startSeq,
endSeq,
startRef: b.startRef !== undefined ? refToIndex(b.startRef) : null,
endRef: b.endRef !== undefined ? refToIndex(b.endRef) : null,
tier: b.tier,
Expand Down
Loading
Loading