Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag

### Added

### #4316 层 3 PR3/3 — 存量编号一次性对齐正史 + fork 克隆保号

- **kernel `restampRefsToLedger`**:把既有引用空间一次性重排到正史账本(ref = store 行 seq + 1)。计数器路径按到达顺序发号但受保护消息不占号,而行 seq 照发 → 两者天然漂移;编辑与 churn 拉大缺口。重排后引用空间与存储真值一致,churn 不再改写编号。诚实边界:无映射 id 留给 assignRefs 的章路径;BLOCKED 仍不占号;死 ref 保留;被夺号的无行引用重安居新号(id 永不复用契约不破);多代(同 id 多行编辑)id 由 `seqByRaw` 排除不误绑。`CompressionState.refLedgerAligned` 标记一次性(`state-hydration` 字段单一事实源自动持久化)。
- **挂点=服务触达(getSession)非 loadSync**:loadSync 保持纯对等视图(corpus manifest 摘要钉死不动);`getSession` 双路径(内存命中/盘重载)首次触达触发对齐——同进程重启边界语义不变(新进程无 in-flight compress,半重排映射不可见)。
- **superseded 守卫**:重排换手的旧 ref → rawId 映射记在 `Session.restampSuperseded`(进程生命周期);重发 in-flight compress 调用引用旧号 → 诚实拒绝(绝不静默折错段),提示重读 acp_status 后重发。拒绝安全不会错折,一次性代价后消失。
- **fork 克隆保号**:`cloneSessionRows`(store+mysql 双后端)把父会话行整段复制到子(session,seq) 保号 + lineage 列(parentSessionId/forkPointSeq 真值)+content/media CAS refcount 提升 + 幂等重放;子 append 自然从父 maxSeq+1(remapPlanToAbsolute)。fork 端点接线,存储不可用降级=kernel-only 克隆(现行为)。
- **验收桩**:kernel 4 测(BLOCKED 偏移重排/交换原子性/无行引用保留+被夺号重安居+死 ref 不动/已对齐幂等)+host 4 测(整段克隆保号+lineage+幂等+子续号/部分分支/superseded 拒绝+当前号通过/冷加载纯视图+服务触达对齐一次+二次幂等)。

### 修复 master 回归(搭乘)

- **#4525 合并把 `sidScope` 改名 `dispatchScope` 漏改一处**:dsh fork-blind-spot 日志(C10b,#4215)仍读旧名 → master tsc 红(`TS2304: Cannot find name 'sidScope'`,dsh-native.ts:1287,CI run 40093629255 failure 同因)。修复=读 `dispatchScope.getStore()` 并按 `{sid,msgIds}` 形状取 `.sid`,语义不变(作用域归因活跃判定)。
Expand Down
4 changes: 2 additions & 2 deletions kernel/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion kernel/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "acp-kernel",
"version": "0.0.162",
"version": "0.0.163",
"description": "Framework-agnostic context-compression engine (model-driven, 3-tier LSM). Pure core: no host dependency.",
"license": "MIT",
"author": "ranxianglei",
Expand Down
2 changes: 2 additions & 0 deletions kernel/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ export {
refForRaw,
rawForRef,
BLOCKED_REF,
restampRefsToLedger,
type RefRestampOutcome,
} from "./refs.js";
export {
prune,
Expand Down
104 changes: 103 additions & 1 deletion kernel/src/refs.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { CoreMessage, MessageRefMap } from "./types.js";
import type { CompressionState, CoreMessage, MessageRefMap } from "./types.js";

// Ref tags render zero-padded to REF_WIDTH digits up to 99,999 ("m00001");
// above that they widen naturally ("m100000" … "m9999999") so every tag ever
Expand Down Expand Up @@ -184,6 +184,108 @@ export function rebuildRefIndex(map: MessageRefMap): MessageRefMap {
return { byRaw: { ...map.byRaw }, byRef };
}

export interface RefRestampOutcome {
/** Row-backed refs that already equalled seq+1 — untouched. */
kept: number;
/** Row-backed refs re-pointed onto seq+1. */
restamped: number;
/** Unbacked refs displaced by a stamp and re-homed onto fresh numbers. */
rehomed: number;
/** BLOCKED entries — protected messages keep taking no number. */
blocked: number;
/** Old ref → rawId for EVERY mapping that changed hands. Hosts keep this
* for the process lifetime so a compress call replaying pre-restart refs
* (a resent in-flight tool call) is honestly rejected instead of silently
* folding whatever message now owns the number. */
superseded: Map<string, string>;
}

/** #4316 layer 3 PR3/3 — align the existing citation space with the unified
* ledger, exactly once per state. The counter path numbers messages in
* arrival order but skips protected messages (BLOCKED takes no number) while
* the store still gives them a row seq, so refs minted after a protected
* message drift ahead of seq+1; edits and churn widen the gap. After this
* call every row-backed, currently-mapped raw id holds ref = seq+1 — the
* same numbers the stamped lanes (assignRefs' ledgerSeq path) mint — so the
* model citation space and the storage truth agree and churn cannot renumber
* citations anymore.
*
* Scope honesty: raw ids with NO current mapping are left to assignRefs (the
* stamp path aligns them on next sight); BLOCKED entries stay BLOCKED; dead
* refs stay reserved; an unbacked ref displaced by a stamp is re-homed onto a
* fresh number (the id-never-reused contract holds — it keeps A ref). */
export function restampRefsToLedger(
state: CompressionState,
seqByRawId: Map<string, number>,
): RefRestampOutcome {
const out: RefRestampOutcome = {
kept: 0,
restamped: 0,
rehomed: 0,
blocked: 0,
superseded: new Map(),
};
if (state.refLedgerAligned === true) return out;
const refs = state.messageRefs;
if (!refs || !refs.byRaw || !refs.byRef) return out;

// Scratch copies — the state is swapped only after the full pass computes
// without throwing (a mid-way failure must not leave a half-renumbered map).
const byRaw: Record<string, string> = { ...refs.byRaw };
const byRef: Record<string, string> = { ...refs.byRef };
const scratch: MessageRefMap = { byRaw, byRef };
const displaced: string[] = [];
// MessageRefMap carries no cursor field — the caller recomputes it from the
// highest used index (compress.ts:1047 pattern). We still track a local
// cursor for free-number allocation during this pass.
let nextIndex = highestUsedIndex(refs) + 1;

for (const [rawId, seq] of seqByRawId) {
const current = byRaw[rawId];
// Unmapped ids (never rendered, or already dead) are assignRefs'
// business, not ours.
if (current === undefined) continue;
if (current === BLOCKED_REF) {
out.blocked++;
continue;
}
const targetIndex = seq + 1;
if (targetIndex < MIN_INDEX || targetIndex > MAX_INDEX) continue;
const target = indexToRef(targetIndex);
if (current === target) {
out.kept++;
if (targetIndex + 1 > nextIndex) nextIndex = targetIndex + 1;
continue;
}
const occupant = byRef[target];
if (occupant !== undefined && occupant !== rawId) displaced.push(occupant);
if (byRef[current] === rawId) delete byRef[current];
out.superseded.set(current, rawId);
byRaw[rawId] = target;
byRef[target] = rawId;
out.restamped++;
if (targetIndex + 1 > nextIndex) nextIndex = targetIndex + 1;
}

// Displaced occupants lost their number to a stamp — re-home them onto
// fresh numbers at/after the cursor (never a dead or low reserved number).
for (const rawId of displaced) {
const held = byRaw[rawId];
if (held === undefined) continue;
if (byRef[held] === rawId) continue;
const free = allocateFreeRef(scratch, nextIndex);
out.superseded.set(held, rawId);
byRaw[rawId] = free.text;
byRef[free.text] = rawId;
nextIndex = free.index + 1;
out.rehomed++;
}

state.messageRefs = { byRaw, byRef };
state.refLedgerAligned = true;
return out;
}

export function highestUsedIndex(map: MessageRefMap): number {
let highest = 0;
for (const ref of Object.values(map.byRaw)) {
Expand Down
6 changes: 6 additions & 0 deletions kernel/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,13 @@ export interface CompressionState {
* the next successful fold (applyCompression) — after a fold the watermark
* re-evaluates from scratch. Optional: pre-feature states have none (the
* pre-latch node clipped from scratch each turn). */
/** #3939 ③: emergency truncation latch entries (see above). Optional:
pre-existing persisted states lack it. */
emergencyTruncateLatch?: EmergencyTruncateLatchEntry[];
/** #4316 layer 3 PR3/3: refs have been aligned onto the unified ledger
(ref = store row seq + 1) exactly once. Set by restampRefsToLedger;
assignRefs' stamp path keeps new arrivals aligned from then on. */
refLedgerAligned?: boolean;
/** Ref-migration render deferral (#3939 ②/#3805): padded refs minted by the
* BLOCKED→numeric migration (assignRefs, #2607). The number exists (citable
* via decompress({ imageRef }) bookkeeping) but the carrier message keeps
Expand Down
130 changes: 130 additions & 0 deletions kernel/tests/seq-restamp.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
import assert from "node:assert/strict";
import test from "node:test";
import { createInitialState } from "../src/state.js";
import { restampRefsToLedger, indexToRef, highestUsedIndex } from "../src/refs.js";
import type { CompressionState } from "../src/types.js";

/** #4316 layer 3 PR3/3 — legacy ref re-stamp to the ledger seq.
*
* The counter path numbers messages in arrival order but SKIPS protected
* messages (BLOCKED takes no number) while the unified store still gives them
* a row seq — so every ref minted after a protected message runs one ahead of
* seq+1. The re-stamp aligns refs with (row seq + 1) exactly once per state:
* refs become the ledger numbers the stamped lanes already use, and the model
* citation space matches the storage truth.
*/

function stateWith(
byRaw: Record<string, string>,
extra?: { deadRefs?: string[]; nextIndex?: number },
): CompressionState {
const state = createInitialState();
state.messageRefs = { byRaw: { ...byRaw }, byRef: {} };
for (const [rawId, ref] of Object.entries(byRaw)) {
if (ref !== "BLOCKED") state.messageRefs.byRef[ref] = rawId;
}
if (extra?.deadRefs) state.deadRefs = [...extra.deadRefs];
if (extra?.nextIndex !== undefined) state.messageRefs.nextIndex = extra.nextIndex;
return state;
}

test("S13a: re-stamp aligns refs to seq+1, BLOCKED stays, once, superseded recorded", () => {
// Rows: seq0=a, seq1=b, seq2=c(protected), seq3=d, seq4=e.
// Legacy counter refs: c is BLOCKED (no number), so d/e run one ahead of seq+1.
const state = stateWith(
{ a: "m00001", b: "m00002", c: "BLOCKED", d: "m00003", e: "m00004" },
{ nextIndex: 5 },
);
const seqByRaw = new Map<string, number>([
["a", 0],
["b", 1],
["c", 2],
["d", 3],
["e", 4],
]);
const out = restampRefsToLedger(state, seqByRaw);
// a/b were already aligned (seq+1); d/e shift by one; c stays BLOCKED.
assert.equal(state.messageRefs.byRaw.a, "m00001");
assert.equal(state.messageRefs.byRaw.b, "m00002");
assert.equal(state.messageRefs.byRaw.c, "BLOCKED");
assert.equal(state.messageRefs.byRaw.d, "m00004");
assert.equal(state.messageRefs.byRaw.e, "m00005");
// byRef mirrors the new ownership.
assert.equal(state.messageRefs.byRef.m00004, "d");
assert.equal(state.messageRefs.byRef.m00005, "e");
assert.equal(state.messageRefs.byRef.m00003, undefined);
// counters: 2 aligned before, 2 re-stamped, 1 blocked.
assert.equal(out.kept, 2);
assert.equal(out.restamped, 2);
assert.equal(out.blocked, 1);
// superseded: d's old number now means something else (nothing) — recorded
// so a replayed compress call holding the pre-restart mapping is rejected
// instead of silently folding the wrong range.
// superseded: every old ref→rawId mapping that changed hands — a replayed
// compress call holding pre-restart refs gets rejected instead of silently
// folding the wrong range.
assert.equal(out.superseded.get("m00003"), "d");
assert.equal(out.superseded.get("m00004"), "e");
// once-flag set; a second call is a no-op.
assert.equal(state.refLedgerAligned, true);
const again = restampRefsToLedger(state, seqByRaw);
assert.equal(again.restamped, 0);
assert.equal(again.kept, 0);
// The derived cursor (highestUsedIndex+1, the compress.ts:1047 pattern)
// advances past the highest stamp: e now holds m00005 → next mint is m00006.
assert.ok(highestUsedIndex(state.messageRefs) >= 5);
});

test("S13b: swapped refs re-stamp atomically — both sides move, superseded carries both old mappings", () => {
// a lives at seq0 but holds m00002; b at seq1 holds m00001 (swapped by history).
const state = stateWith({ a: "m00002", b: "m00001" }, { nextIndex: 3 });
const seqByRaw = new Map<string, number>([
["a", 0],
["b", 1],
]);
const out = restampRefsToLedger(state, seqByRaw);
assert.equal(state.messageRefs.byRaw.a, "m00001");
assert.equal(state.messageRefs.byRaw.b, "m00002");
assert.equal(state.messageRefs.byRef.m00001, "a");
assert.equal(state.messageRefs.byRef.m00002, "b");
assert.equal(out.restamped, 2);
// Both old mappings changed hands — both are superseded.
assert.equal(out.superseded.get("m00001"), "b");
assert.equal(out.superseded.get("m00002"), "a");
});

test("S13c: unbacked raw ids keep their refs; dead refs stay reserved; unbacked displaced by a stamp is re-homed", () => {
// f has no row (never stored) and keeps m00007. g holds m00005 with no row,
// but the stamped e (seq 4) needs m00005 — g is displaced to a free number.
const state = stateWith(
{ e: "m00004", f: "m00007", g: "m00005" },
{ deadRefs: ["m00009"], nextIndex: 8 },
);
const seqByRaw = new Map<string, number>([["e", 4]]);
const out = restampRefsToLedger(state, seqByRaw);
assert.equal(state.messageRefs.byRaw.e, "m00005");
assert.equal(state.messageRefs.byRef.m00005, "e");
// f untouched (unbacked, not displaced).
assert.equal(state.messageRefs.byRaw.f, "m00007");
// g displaced off m00005 onto a fresh number (never reuse the id contract
// still holds — it keeps A ref).
assert.notEqual(state.messageRefs.byRaw.g, "m00005");
assert.match(state.messageRefs.byRaw.g!, /^m\d{5,}$/);
assert.equal(out.rehomed, 1);
// dead refs untouched.
assert.deepEqual(state.deadRefs, ["m00009"]);
});

test("S13d: no-op for an already-aligned state (kept only, superseded empty)", () => {
const state = stateWith({ a: "m00001", b: "m00002" }, { nextIndex: 3 });
const seqByRaw = new Map<string, number>([
["a", 0],
["b", 1],
]);
const out = restampRefsToLedger(state, seqByRaw);
assert.equal(out.kept, 2);
assert.equal(out.restamped, 0);
assert.equal(out.rehomed, 0);
assert.equal(out.superseded.size, 0);
assert.equal(state.refLedgerAligned, true);
});
6 changes: 5 additions & 1 deletion scripts/update-storage-corpus.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,12 +138,16 @@ async function writeSessionAwaited(dir: string, session: Session): Promise<strin

/** Digest over an in-memory session (pre-write reference — NOT the pin). */
function sessionDigest(loaded: Session): string {
// #4316 PR3/3: strip the cold-load alignment marker (a flag, not view
// content) — mirrors the parity digest in tests/storage-unified.test.ts.
const state = { ...(loaded.state as unknown as Record<string, unknown>) };
delete state.refLedgerAligned;
const view = {
id: loaded.id,
meta: loaded.meta,
stats: loaded.stats,
metadata: loaded.metadata,
state: loaded.state,
state,
blockContents: [...loaded.blockContents.entries()],
lastMessages: loaded.lastMessages,
lastMessagesFolded: loaded.lastMessagesFolded,
Expand Down
Loading
Loading