Skip to content
Open
4 changes: 2 additions & 2 deletions kernel/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion kernel/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "acp-kernel",
"version": "0.0.200",
"version": "0.0.201",
"description": "Framework-agnostic context-compression engine (model-driven, 3-tier LSM). Pure core: no host dependency.",
"license": "MIT",
"author": "ranxianglei",
Expand Down
125 changes: 115 additions & 10 deletions kernel/src/sync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,13 @@ import type { CompressionState, CoreMessage } from "./types.js";
* inside the block's exact interval claim. Unbounded blocks and unstamped
* views answer false — the id path decides those. The -1/-1 "no claim"
* sentinel (PR #4725 N1) also answers false: it is absence of evidence, not
* a positional claim (a [0,0] row IS a real claim over seq 0). */
* a positional claim (a [0,0] row IS a real claim over seq 0).
*
* Kept exactly as upstream states it: when NO covered id survives, a position
* inside the interval is far more likely id churn (re-mint, merged #2636 pair)
* than deletion, and a merge legitimately covers fewer ids than positions. The
* coverage-shape rule below only governs the shapes where ids are present.
*/
function blockSeqPresent(
block: { startSeq?: number; endSeq?: number },
messages: CoreMessage[],
Expand All @@ -32,6 +38,71 @@ function blockSeqPresent(
return false;
}

type CoverageShape = "full" | "tail-gap" | "head-gap" | "holes" | "none";

/** Where the missing covered ids sit, in the block's own coverage order.
*
* - `tail-gap` survivors are a prefix, everything missing is the newest end:
* EITHER a rewind (the tail was withdrawn -> the summary is
* stale) OR id churn (a re-stamp re-minted the tail in place ->
* still live). The shape alone cannot tell them apart; the
* liveness check (tailGapIsChurn) does.
* - `head-gap` survivors are a suffix, everything missing is the oldest end:
* the host dropped old history; the summary is the ONLY record
* of it, so dropping it too would be the actual loss.
* - `holes` missing cells are interior/ragged: the churn shape (client
* re-serialized history), never the rewind shape. Do not guess.
* - `none` nothing covered is visible; ordering evidence decides.
*/
function coverageShape(
block: { effectiveMessageIds: string[] },
presentBases: Set<string>,
): CoverageShape {
const present = block.effectiveMessageIds.map((id) => presentBases.has(baseIdOf(id)));
if (present.length === 0 || present.every(Boolean)) return "full";
if (!present.some(Boolean)) return "none";
const firstPresent = present.indexOf(true);
const lastPresent = present.lastIndexOf(true);
if (present.slice(0, lastPresent + 1).every(Boolean)) return "tail-gap";
if (present.slice(firstPresent).every(Boolean)) return "head-gap";
return "holes";
}

/** #5166: a tail gap is id churn (a model re-stamp / re-serialization re-minted
* the newest covered ids in place), not a rewind, when the current history
* carries at least as many messages BEYOND the surviving covered prefix as
* there are missing covered ids — the re-minted tail occupies the same slots,
* so the span is still live and the summary still describes content on the
* wire. A true rewind leaves fewer messages beyond the prefix than the
* withdrawn tail (those slots are empty), so the summary is stale. Works
* without ledgerSeq stamps: the re-stamp e2e (#2265) is unstamped, so the
* positional witness cannot carry this case.
*/
function tailGapIsChurn(
block: { effectiveMessageIds: string[] },
presentBases: Set<string>,
messages: CoreMessage[],
): boolean {
const effIds = block.effectiveMessageIds;
let lastPresent = -1;
for (let i = 0; i < effIds.length; i++) {
if (presentBases.has(baseIdOf(effIds[i]!))) lastPresent = i;
}
const missing = effIds.length - lastPresent - 1;
if (missing <= 0) return true;
const anchorBase = baseIdOf(effIds[lastPresent]!);
let anchorIdx = -1;
for (let i = messages.length - 1; i >= 0; i--) {
const id = messages[i]!.id;
if (typeof id === "string" && baseIdOf(id) === anchorBase) {
anchorIdx = i;
break;
}
}
if (anchorIdx < 0) return true; // defensive: anchor absent -> keep
return messages.length - anchorIdx - 1 >= missing;
}

export interface SyncResult {
state: CompressionState;
deactivated: string[];
Expand Down Expand Up @@ -178,15 +249,49 @@ export function syncBlocks(
// (pruned view) is still present — the summary IS the block's visible
// representation. Without this, hosts passing pruned views would lose
// block activity every turn.
// #4316 layer 3: a block with an exact ledger interval is also still
// present when ANY visible message carries a ledgerSeq stamp inside the
// interval — the host restored/re-stamped the originals (fresh ids, same
// positions), and the positional claim survives the id churn that the
// id-membership check above just missed.
const stillPresent =
block.effectiveMessageIds.some((id) => presentBases.has(baseIdOf(id))) ||
presentIds.has(summaryMessageId(block.blockId)) ||
blockSeqPresent(block, messages);
// #4592: the question is COVERAGE, not existence. `some` answered "is any of
// it still there" and kept injecting a summary that described a half-rewound
// span; `every` answered "is all of it there" but treated id churn like a
// deletion. The shape of the gap decides:
//
// tail gap -> either a rewind (the host withdrew the newest end ->
// stale, drop) OR id churn (a re-stamp re-minted the newest
// covered ids in place -> still live, keep). The two share
// one shape, so the gap alone cannot decide: tailGapIsChurn
// asks whether the history still carries the missing tail
// (enough messages beyond the surviving prefix) before
// treating the gap as a withdrawal.
// head gap -> the host dropped OLD history; the summary is the only
// record of it, so keep it (dropping it would lose content)
// holes -> id churn (interior gaps are never the rewind shape):
// keep it; `fold-reconcile` re-keys churned ids upstream
// none -> nothing id-shaped survived: defer to the positional
// witness (upstream #4316, unchanged), which is the churn /
// merged-pair shape rather than a deletion
//
// Everything except the tail gap keeps today's keep-alive behaviour, so the
// only behaviour that changes is the one shape that can be named — and even
// there only the confirmed-withdrawal half of it.
const shape = coverageShape(block, presentBases);
let stillPresent: boolean;
if (presentIds.has(summaryMessageId(block.blockId))) {
// Pruned view: the summary IS the block's visible representation.
stillPresent = true;
} else {
switch (shape) {
case "full":
case "head-gap":
case "holes":
stillPresent = true;
break;
case "tail-gap":
stillPresent = tailGapIsChurn(block, presentBases, messages);
break;
case "none":
stillPresent = blockSeqPresent(block, messages);
break;
}
}
if (!stillPresent) {
block.active = false;
deactivated.push(block.blockId);
Expand Down
131 changes: 127 additions & 4 deletions kernel/tests/sync-config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,144 @@ test("syncBlocks deactivates blocks whose messages are all gone", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["gone1", "gone2"] }),
makeBlock({ blockId: "b2", effectiveMessageIds: ["kept", "also-gone"] }),
);

const result = syncBlocks([msg("kept")], state);
assert.deepEqual(result.deactivated, ["b1"]);
assert.equal(result.state.blocks[0]!.active, false);
assert.equal(result.state.blocks[1]!.active, true);
});

test("syncBlocks leaves blocks intact when at least one message remains", () => {
// #4592: upstream kept a block alive as long as SOME covered message remained.
// That is exactly the shape a host rewind leaves behind — the survivors are a
// prefix and the newest end is gone, so the summary describes history that no
// longer exists. Coverage SHAPE decides, not existence: see the head-gap and
// holes tests below for the shapes that must KEEP their summary.
test("syncBlocks deactivates a block whose span was partially truncated (rewind)", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["kept", "also-gone"] }),
);
const result = syncBlocks([msg("kept")], state);
assert.deepEqual(result.deactivated, ["b1"]);
assert.equal(result.state.blocks[0]!.active, false);
});

// #5166: the mirror of the rewind above. A model re-stamp re-mints the newest
// covered ids in place (same slots, fresh content-hash ids), leaving the SAME
// tail-gap shape as a rewind — but the span is still live, so the summary must
// survive. The discriminator works without ledgerSeq stamps: the history
// carries as many messages beyond the surviving prefix as there are missing
// covered ids, so the gap is churn, not a withdrawal. (The #2265 re-stamp e2e
// is unstamped, so the positional witness cannot carry this case.)
test("syncBlocks keeps a tail-gap block whose missing tail was re-minted in place (re-stamp churn)", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["u1", "a1", "u2"] }),
);
// u1 survives; a1 and u2 were re-stamped (fresh ids a1-re, u2-re); the
// history continues with newer messages.
const visible: CoreMessage[] = [
msg("u1"),
{ id: "a1-re", role: "assistant", contentType: "text", text: "[via model:x] a1" },
{ id: "u2-re", role: "user", contentType: "text", text: "<system-reminder>u2" },
msg("a2"),
msg("u3"),
msg("a3"),
msg("u4"),
];
const result = syncBlocks(visible, state);
assert.deepEqual(result.deactivated, []);
assert.equal(result.state.blocks[0]!.active, true);
});

// #5166: the boundary the churn rule must NOT cross. A rewind that withdraws
// the tail and appends FEWER new messages than it withdrew leaves a tail gap
// with too few messages beyond the surviving prefix to account for the missing
// ids -> the tail was genuinely cut -> stale, drop.
test("syncBlocks deactivates a tail-gap block whose withdrawn tail outnumbers the appended messages (rewind + short append)", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["m1", "m2", "m3"] }),
);
// m1 survives; m2, m3 were withdrawn; only ONE new message (n1) appended.
const result = syncBlocks([msg("m1"), msg("n1")], state);
assert.deepEqual(result.deactivated, ["b1"]);
assert.equal(result.state.blocks[0]!.active, false);
});

// #4592: the mirror shape of a rewind. The host dropped OLD history from the
// head; the summary is the only record of it, so dropping the summary too is the
// actual loss. This is what makes sliding-window / head-truncating hosts safe.
test("syncBlocks keeps a block whose oldest coverage the host dropped (head gap)", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["old-1", "old-2", "kept"] }),
);
const result = syncBlocks([msg("kept"), msg("fresh")], state);
assert.deepEqual(result.deactivated, []);
assert.equal(result.state.blocks[0]!.active, true);
});

// #4592: interior gaps are the id-churn shape (client re-serialized its
// history), never the rewind shape. Do not guess: keep the summary and let
// fold-reconcile re-key the churned ids.
test("syncBlocks keeps a block with interior id gaps (churn shape)", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["a", "churned", "c"] }),
);
const result = syncBlocks([msg("a"), msg("c")], state);
assert.deepEqual(result.deactivated, []);
assert.equal(result.state.blocks[0]!.active, true);
});

// #4592: nothing id-shaped survives, but a visible message carries a ledger
// position INSIDE the block's interval — the churn / merged-pair shape (a merge
// legitimately covers fewer ids than positions). The positional witness is
// upstream's #4316 rule, unchanged by the shape decision.
test("syncBlocks keeps a fully-gone block with a position inside its interval (churn witness)", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({
blockId: "b1",
effectiveMessageIds: ["gone-1", "gone-2"],
startSeq: 1,
endSeq: 2,
}),
);
const visible: CoreMessage[] = [
{ id: "re-minted", role: "user", contentType: "text", text: "merged", ledgerSeq: 2 },
];
const result = syncBlocks(visible, state);
assert.deepEqual(result.deactivated, []);
assert.equal(result.state.blocks[0]!.active, true);
});

// #4592: no id and no position survives -> dead (the plain deletion shape).
test("syncBlocks deactivates a fully-gone block with no positional witness", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({
blockId: "b1",
effectiveMessageIds: ["gone-1"],
startSeq: 1,
endSeq: 1,
}),
);
const visible: CoreMessage[] = [
{ id: "newer", role: "user", contentType: "text", text: "newer", ledgerSeq: 9 },
];
const result = syncBlocks(visible, state);
assert.deepEqual(result.deactivated, ["b1"]);
assert.equal(result.state.blocks[0]!.active, false);
});

test("syncBlocks leaves blocks intact when every covered message remains", () => {
const state = createInitialState();
state.blocks.push(
makeBlock({ blockId: "b1", effectiveMessageIds: ["a", "b"] }),
);
const result = syncBlocks([msg("a")], state);
const result = syncBlocks([msg("a"), msg("b")], state);
assert.deepEqual(result.deactivated, []);
assert.equal(result.state.blocks[0]!.active, true);
});
Expand Down
2 changes: 1 addition & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading