Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ All notable changes to **billion-context** are documented here.
Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag.

## [Unreleased]
- **Stale-guard ownership fast path (#3049):** in shadow/full modes the per-save rollback guard no longer re-reads and re-parses the entire session envelope (the ~320 ms/turn, full-file read on >40 MiB sessions — a large slice of #3049's read amplification). The builder now carries an ownership token (the two monotonic counters it just ingested, plus its build time); when the unified row still matches those counters and the envelope file's mtime sits inside the build window, no other writer can have committed and the guard skips the read. Cross-instance commits are still caught through the shared `index.db` row, foreign file writes (legacy sibling, manual edit) through the mtime check, and legacy-only installs keep the exact per-save full read. Deliberate residual (documented on `ownershipFastPath`): a sibling whose sidecar is disabled writing within the ~1.5 s grace window is missed for one cycle — strictly narrower than an unconditional size/mtime skip.
- **Image pre-compression (`compress.imageCompression`) and the `image_full` tool removed (#3266).** The opt-in downscale pass and its restore channel are gone from host and kernel: no more arrival-time shrinking, no `image_full` injection, no `[Downscaled screenshots]` notes, no image stats — `sharp` is no longer an optional dependency. Rationale: the restore loop was structurally unreachable on hosts that reject bili-injected tools (codex), and where reachable the model had no reliable way to learn the right ref, producing the 38-round guess-reject-resend loop in #3266 (cache hit 75%→28%). Every image now passes through byte-identical at original resolution. Sessions persisted by older versions load unchanged; their legacy `imageShrinks`/`imageFullRestored` state keys and `imageShrunk*` stats are ignored on load (never materialize). A future redesign is tracked on the issue.
- **Incremental marker divert + native client helpers (#3248 phase 2).** A native extension can now send incremental turns on its ORDINARY model requests: stamp `x-bili-plugin-turn: 1` (requires the usual `x-bili-plugin`) and post the incremental payload — the body shape (top-level `conversationId`/`sinceSeq`/`messages`, no top-level `model`) is the final word, so a marker on a non-incremental body rides the normal pipeline unchanged and no provider URL changes. The lane forwards to the ORIGINAL request path; `GET /__bili/plugin/turn` probes capability (`{ok, protocol: 1, feature, enabled, storage}`). Gated by the rollout feature `pluginIncrementalTurns` — default 0% keeps every install byte-identical; force with `rollout.features.pluginIncrementalTurns: true` or `BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1` (still requires a live unified sidecar). Seeds imply `resync: true` (a first incremental turn never 409s, and interop full-resend traffic is replaced client-authoritatively); a system-only change rides an empty delta. Reference wirings: pi (`before_provider_headers`/`before_provider_request`/`after_provider_response`) and dsh-native (fetch patch `beforeSend`/`headersFor`/`onResponse`), both on the shared state machine in `src/agent/incremental-client.ts` (probe TTL cache, prefix-hash diffing, sticky system, per-conversation dead flag) — new hosts should reuse it.
- **Incremental turns (#3248): `POST /__bili/plugin/turn`.** Native hosts that keep their own full local history can stop resending it: the first turn seeds the canonical (server-side, unified messages table), later turns send only the delta (client-added messages including the previous assistant reply), and the lane assembles the full body and re-enters the proxy pipeline in-process — same session (the client's own conversation id), same fold ledger, same nudges/tools, freely interleaving with full-resend traffic. Strict ack contract (`x-bili-applied-seq`; 409 seq-mismatch/turn-in-flight forces resync), sticky system prompt, anthropic request params via allowlist, loopback-gated like every `/__bili/` surface. Requires `storage.unified` (503 otherwise); v1 keeps the O(full) pipeline walk (zero-hit, cheap) — the server-authored prefix fast path is tracked on the issue. A divergent `resync: true` records a base-seq epoch (`session_ext` `inc:base-seq`), so cold rebuilds slice past the rejected branch's append-only rows — one resync heals, restarts included; legacy-only storage degrades to the previous full-union read.
Expand Down
139 changes: 138 additions & 1 deletion src/persist.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,13 @@ const MIGRATION_MARKER = ".bili-migration-286.done";

const STALE_WARN_THROTTLE_MS = 60_000;

/** #3049: slack for the ownership fast path's mtime check — covers this
* process's own write latency (builder → atomic write lands a few ms after
* builtAtMs). Foreign writes landing INSIDE this window are missed for one
* cycle only if their writer also bypasses the shared index (disabled
* sidecar); see ownershipFastPath. */
const OWN_COMMIT_GRACE_MS = 1_500;

interface PersistedSession {
version: number;
savedAt: number;
Expand Down Expand Up @@ -314,6 +321,25 @@ export class SessionStore {
private readonly store: StateStore<PersistedSession>;
private readonly log: Logger;
private readonly staleWarnAt = new Map<string, number>();

/** #3049: ownership token for the stale-guard fast path. Set INSIDE the
* guardedBuild closure after the sidecar ingest of the record we are
* about to write (or the disk record we adopted): {counters of that
* commit, wall-clock build time}. The next guard compares this token
* against the unified row + the envelope file's mtime to prove nobody
* else committed since — skipping the full-file read+parse that costs
* ~320ms/turn on >40MiB sessions. Cleared on the direct-write paths
* (restore/rekey) that bypass guardedBuild. */
private readonly ownCommit = new Map<string, { requests: number; nextBlockId: number; builtAtMs: number }>();

/** #3049 test seam: how many times the stale guard took the FULL
* envelope read (the expensive path). The fast path collapses this to
* one read per external writer, not one read per save. */
private staleGuardFullReads = 0;

__staleGuardFullReadsForTests(): number {
return this.staleGuardFullReads;
}
private readonly codec?: StateStoreCodec;
private readonly readCodec: StateStoreCodec;
private boundedReads = false;
Expand Down Expand Up @@ -564,6 +590,11 @@ export class SessionStore {
// negative value is gone from disk.
await this.store.writeNow(id, () => buildRecord(session));
clamped++;
// #3049: the clamp rewrite writes a DIFFERENT payload than any
// ingested record (clamped stats) without a sidecar ingest —
// drop any stale token; boot-time loads have none, but keep
// this correct for future callers.
this.ownCommit.delete(id);
}
out.set(id, session);
}
Expand Down Expand Up @@ -998,6 +1029,10 @@ export class SessionStore {
}
session.id = label;
await this.store.writeNow(label, () => buildRecord(session));
// #3049: direct write bypassing guardedBuild — the boot rekey
// rewrites envelopes without the sidecar ingest, so any prior
// ownership token for this id is stale.
this.ownCommit.delete(label);
await this.removeLegacyFile(id, session);
const envelope = loaded.get(id);
if (envelope) {
Expand Down Expand Up @@ -1406,6 +1441,13 @@ export class SessionStore {
record.meta = { ...record.meta, restoredFrom: cp.session_id, restoredCheckpoint: cp.cp_id } as PersistedSession["meta"];
await this.store.writeNow(newId, () => record);
shadow.record(record);
// #3049: the replica record was ingested from the checkpoint payload
// — the ownership token must mirror exactly those counters.
this.ownCommit.set(newId, {
requests: record.stats?.requests ?? record.requests ?? 0,
nextBlockId: record.state?.nextBlockId ?? 0,
builtAtMs: Date.now(),
});
// Materialize the content-store sidecar for the restored id too:
// buildRecord drops forkContentStore on the next live save (no fork
// receipt), so the record alone is not durable — the sidecar is what
Expand Down Expand Up @@ -1482,8 +1524,18 @@ export class SessionStore {
* holding a STALE in-memory copy would roll counters back (requests:3 →
* 2). Both signals below are monotonic per session id, so either being
* strictly smaller proves staleness. Returns the fresher-on-disk payload
* when the incoming record is stale, else null. */
* when the incoming record is stale, else null.
*
* #3049: the guard used to re-read and re-parse the ENTIRE envelope on
* every save just to compare these two numbers (~320ms and a full file
* read per turn on >40MiB sessions). ownershipFastPath proves — without
* reading the file — that no other writer has committed since OUR last
* build, in which case the disk record is exactly the one this builder
* chain last wrote (or converged to), and the in-process comparison
* below (monotonic counters, same session object) is sufficient. */
private staleDiskPayload(incoming: PersistedSession): PersistedSession | null {
if (this.ownershipFastPath(incoming)) return null;
this.staleGuardFullReads++;
const envelope = this.loadEnvelope(incoming.id, incoming.meta);
if (!envelope) return null;
const disk = envelope.payload;
Expand All @@ -1498,6 +1550,76 @@ export class SessionStore {
return null;
}

/** #3049 ownership fast path — proves "the disk envelope is still the
* record this process last committed" with three cheap signals, no
* payload read:
*
* 1. Unified row counters equal OUR last commit's counters. Every
* shadow/full instance sharing the directory shares index.db (#394
* multi-process topology), and each commit ingests before it writes
* the file — so a sibling's newer commit bumps the row first and this
* comparison fails, falling back to the exact full read.
* 2. The envelope file's mtime is not newer than our last build (plus
* grace for our own write latency). A writer that does NOT share the
* index (legacy-mode sibling, manual edit) still touches the file, and
* an mtime past our build window forces the full read.
* 3. The incoming counters are not behind our own commit — they cannot
* be, in-process (same session object, monotonic counters), but the
* check keeps this path self-contained; any surprise defers to the
* full read.
*
* Residual (deliberate, strictly narrower than an unconditional
* size/mtime ownership skip): a sibling whose sidecar is DISABLED (3-
* strike drift) writing a newer envelope within the same grace window
* (~1.5s) as our build is missed for one cycle; its next write wins the
* exact guard again. Crash safety is clean: a process that dies between
* its DB ingest and its file write leaves no token here (fresh map), so
* the next boot always takes the full read. */
private ownershipFastPath(incoming: PersistedSession): boolean {
const own = this.ownCommit.get(incoming.id);
if (!own) return false;
const shadow = this.shadow;
if (!shadow || shadow.stats.disabled) return false; // shadow/full only — legacy stays exact
const incRequests = incoming.stats?.requests ?? incoming.requests ?? 0;
const incBlocks = incoming.state?.nextBlockId ?? 0;
if (incRequests < own.requests || incBlocks < own.nextBlockId) return false;
let row: { requests: number; nextBlockId: number } | null = null;
try {
row = shadow.sessionCounters(incoming.id);
} catch {
return false; // sidecar hiccup — full read decides
}
if (!row || row.requests !== own.requests || row.nextBlockId !== own.nextBlockId) return false;
const stat = this.statEnvelope(incoming.id, incoming.meta);
if (!stat) return false;
if (stat.mtimeMs > own.builtAtMs + OWN_COMMIT_GRACE_MS) return false;
return true;
}

/** #3049: stat the envelope the way loadEnvelope would resolve it — the
* kernel-discovered source file first, then the canonical live paths.
* Null when no file exists yet (first save: the full read correctly
* returns null-fast via loadEnvelope's miss). */
private statEnvelope(id: string, meta?: { protocol?: string; upstreamOrigin?: string }): { mtimeMs: number } | null {
const candidates: string[] = [];
const known = this.sourceFiles.get(id);
if (known) candidates.push(known);
candidates.push(
path.join(this.sessionsDir, relPathFor(id, meta?.protocol, meta?.upstreamOrigin)),
path.join(this.sessionsDir, relPathFor(id)),
path.join(this.sessionsDir, flatFileNameFor(id)),
);
for (const file of candidates) {
try {
const st = statSync(file);
return { mtimeMs: st.mtimeMs };
} catch {
// try next
}
}
return null;
}

private guardedBuild(session: Session): () => PersistedSession {
// #2671: capture the storage-mode stamp EAGERLY, at enqueue time —
// the kernel write chain may invoke the builder on a later tick than
Expand Down Expand Up @@ -1525,6 +1647,7 @@ export class SessionStore {
this.shadow?.record(record, tail && tail.length > 0
? { lastMessages: tail, lastMessagesFolded: session.lastMessagesFolded }
: undefined, session.contentStore ?? undefined);
this.noteOwnCommit(session.id, record);
return record;
}
const now = Date.now();
Expand All @@ -1546,10 +1669,24 @@ export class SessionStore {
this.shadow?.record(disk);
// Rewrite the disk's own payload: content-identical no-op that
// preserves the newer state while satisfying the write chain.
this.noteOwnCommit(session.id, disk);
return disk;
};
}

/** #3049: record what THIS builder chain just committed (the record it
* returned — our fresh build, or the disk payload the guard made us
* adopt). The unified row was ingested from the same payload in the same
* closure, so token counters === row counters until any other writer
* commits. Called from inside the guardedBuild closure only. */
private noteOwnCommit(id: string, committed: PersistedSession): void {
this.ownCommit.set(id, {
requests: committed.stats?.requests ?? committed.requests ?? 0,
nextBlockId: committed.state?.nextBlockId ?? 0,
builtAtMs: Date.now(),
});
}

/** #2401: read-side convergence. Replace the live session's persisted
* surfaces with the newer disk record, in place (the object identity is
* load-bearing — request handlers and the pool map hold references to
Expand Down
9 changes: 9 additions & 0 deletions src/storage/shadow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,15 @@ export class UnifiedShadow {
return this.store.lookupConversation(namespace, conversationId);
}

/** #3049: O(1) counter probe backing the persist stale-guard fast path.
* Safe on a disabled sidecar? NO — call sites must check stats first
* (this throws like every other pass-through; the guard treats any
* failure as "no fast path" and falls back to the full envelope read). */
sessionCounters(sessionId: string): { requests: number; nextBlockId: number } | null {
this.assertLive();
return this.store.sessionCounters(sessionId);
}

listConversationBindings(namespace?: string): { namespace: string; conversationId: string; sessionId: string; claimedAt: number }[] {
this.assertLive();
return this.store.listConversationBindings(namespace);
Expand Down
23 changes: 23 additions & 0 deletions src/storage/store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,29 @@ export class UnifiedStore {
this.db.close();
}

/** #3049: O(1) ownership probe for the persist stale-guard. Reads only
* the two monotonic counters the rollback guard compares (stats.requests,
* state.nextBlockId) — never the payload columns — so the guard can skip
* its full envelope read+parse when nothing else has committed since this
* process's last save. next_block_id is a dedicated column; requests is
* the one small stats_json parse. Null = no row yet (organic import
* pending / foreign id). */
sessionCounters(sessionId: string): { requests: number; nextBlockId: number } | null {
const row = this.db
.prepare("SELECT next_block_id, stats_json FROM sessions WHERE session_id = ?")
.get(sessionId) as { next_block_id: number; stats_json: string } | undefined;
if (!row) return null;
try {
const stats = JSON.parse(row.stats_json) as { requests?: unknown };
return {
requests: typeof stats.requests === "number" ? stats.requests : 0,
nextBlockId: row.next_block_id,
};
} catch {
return null;
}
}

/** Content CAS: insert-if-absent + refcount bump; returns hash. */
private putContent(content: string | Buffer, precomputedHash?: string): string {
const bytes = typeof content === "string" ? Buffer.from(content, "utf8") : content;
Expand Down
Loading
Loading