Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,7 @@ Environment variables take precedence over the config file. They are useful for
| `BILI_UPSTREAM_PROXY` | Upstream proxy for the proxy's own outbound connections — highest priority, above per-URL/per-provider config. See the README *Upstream proxy* section. |
| `BILI_PERSIST` | Set `0` to disable session persistence (in-memory only, lost on restart). |
| `BILI_PERSIST_DEBOUNCE_MS` | Debounce window for persistence writes to disk, in ms (default `500`). |
| `BILI_PERSIST_TAIL_TOKENS` | Token budget for the persisted conversation snapshot (#401). The disk record stores the **folded view** (block summaries in place of compressed ranges) truncated to the newest messages within this budget — never the raw full history. Default `16384`; `0` disables message persistence entirely (block summaries and compressed originals still persist; `bili export` falls back to block-only rendering). Live in-memory sessions are unaffected — `bili export` of a live session is always complete. |
| `BILI_PERSIST_EPERM_ALERT_THRESHOLD` | N consecutive persist write failures (`EPERM`/`EBUSY`/`EACCES`) on one session before the one-time "add this dir to antivirus exclusions" alert fires (default `5`). Windows only. See [Windows: exclude the sessions dir](#windows-exclude-the-sessions-dir-from-antivirus-362). |
| `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | Re-alert window for the persist EPERM alert, in ms. `0` (default) = alert once then stay silent; `>0` = re-alert at most every that many ms while the failures continue. |
| `BILI_TUNNEL_ALLOWED_HOSTS` | `/bili/<absolute-url>` tunnel admission for **remote clients** (#409): comma-separated `host` or `host:port` entries that unlock loopback/private destinations (e.g. a LAN relay or the machine's own sglang) for non-loopback clients. The proxy itself and link-local/metadata addresses are always denied; local (loopback) clients always pass. |
Expand Down
1 change: 1 addition & 0 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,7 @@
| `BILI_UPSTREAM_PROXY` | 代理自身出站连接的上游代理 —— 优先级最高,高于 per-URL/per-provider 配置。见 README「上游代理」一节。 |
| `BILI_PERSIST` | 设 `0` 关闭会话持久化(仅内存,重启即丢)。 |
| `BILI_PERSIST_DEBOUNCE_MS` | 持久化写盘的防抖窗口(毫秒,默认 `500`)。 |
| `BILI_PERSIST_TAIL_TOKENS` | 持久化会话快照的 token 预算(#401)。盘上记录的是**折叠视图**(压缩范围以块摘要替代)并截断到该预算内的最新消息 —— 不再存全量原始历史。默认 `16384`;设 `0` 彻底不持久化消息(块摘要与压缩原件仍会持久化,`bili export` 退回块级渲染)。活会话内存不受影响 —— 活会话的 `bili export` 始终完整。 |
| `BILI_PERSIST_EPERM_ALERT_THRESHOLD` | 同一会话连续 N 次持久化写失败(`EPERM`/`EBUSY`/`EACCES`)后触发一次性「把该目录加入杀软排除项」告警的阈值(默认 `5`)。仅 Windows。见下文「Windows:把会话目录加入杀软排除项」章节。 |
| `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | persist EPERM 告警的重复窗口(毫秒)。`0`(默认)= 只告警一次后静默;`>0` = 失败持续期间最多每这么久重复告警一次。 |
| `BILI_MAX_SESSIONS` | 内存中最多保留的会话数(默认 `256`;LRU 淘汰 —— 磁盘是事实源)。 |
Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -418,7 +418,12 @@ pass an explicit `x-acp-session` header per conversation to avoid collisions.

The proxy persists each session's compression state to the sessions dir
(`%USERPROFILE%\.local\share\billion-context\` by default) and rewrites the
file every turn of a long session. On Windows, real-time antivirus (Windows
file every turn of a long session. Persisted per session: the compression
state (block summaries), the compressed originals cache (`blockContents`,
what `bili export --full` recovers), and a bounded folded-view snapshot of
the recent conversation (newest `BILI_PERSIST_TAIL_TOKENS` tokens, default
16k) — the raw full history is never duplicated on disk (#401). On
Windows, real-time antivirus (Windows
Defender), the search indexer, or a sync tool (OneDrive) can lock that
directory mid-write, so the rename fails with `EPERM` and every persist for
that session fails until the lock clears.
Expand Down
34 changes: 30 additions & 4 deletions src/export.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,24 @@ export function renderHandoff(s: Session, full: boolean): string {

const messages = s.lastMessages;
if (messages && messages.length > 0) {
// Full-conversation snapshot (v3 files): render exactly what the model
// Conversation snapshot (v3 files): render exactly what the model
// saw. Default = folded view via the kernel's own renderer (summaries
// in place of compressed ranges); --full = every original message.
lines.push(full ? `## Full conversation (${messages.length} messages)` : `## Conversation (folded view as the model saw it, ${messages.length} client messages)`);
// A restored #401 snapshot is ALREADY pruned (and truncated) — its
// message ids no longer align with the state ranges, so re-running
// prune() would resurrect dropped summaries at index 0. Render it
// as-is; --full recovers folded originals from blockContents below.
const folded = s.lastMessagesFolded === true;
const view = full || folded ? messages : prune(messages, s.state);
lines.push(
full && !folded
? `## Full conversation (${messages.length} messages)`
: folded
? `## Conversation (persisted folded snapshot, ${messages.length} messages)`
: `## Conversation (folded view as the model saw it, ${messages.length} client messages)`,
);
lines.push("");
let lastRole = "";
const view = full ? messages : prune(messages, s.state);
for (const m of view) {
if (m.role !== lastRole) {
lines.push(`### ${m.role}`);
Expand All @@ -79,6 +90,21 @@ export function renderHandoff(s: Session, full: boolean): string {
lines.push(renderMessage(m));
}
lines.push("");
// A folded snapshot has no originals of the folded ranges — --full
// recovers them from the blockContents cache (same source the v2
// fallback uses).
if (full && folded) {
for (const b of s.state.blocks.filter((x) => x.active)) {
const content = s.blockContents.get(b.blockId);
if (!content) continue;
lines.push(`## Block ${b.blockId}${b.topic ? ` — ${b.topic}` : ""}`);
lines.push("");
lines.push(`### Original messages (${content.full.count})`);
lines.push("");
lines.push(content.full.text.trim());
lines.push("");
}
}
return lines.join("\n");
}

Expand Down Expand Up @@ -148,7 +174,7 @@ export async function exportSession(selector: string | undefined, opts: ExportOp
const store = new SessionStore({ dir: opts.dir, enabled: true });
const all = [...(await store.loadAll()).values()];
if (all.length === 0) {
return "No persisted sessions found. Sessions are written under the sessions directory once the proxy has served a request (compression state and compressed originals only — uncompressed conversation text is not persisted).";
return "No persisted sessions found. Sessions are written under the sessions directory once the proxy has served a request (compression state, compressed originals, and a bounded folded-view snapshot of the recent conversation).";
}
if (!selector) {
const list = await listSessions(opts);
Expand Down
62 changes: 57 additions & 5 deletions src/persist.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { StateStore, flatFileNameFor, type PersistedEnvelope } from "acp-kernel/
import { sessionsDir } from "./paths.js";
import { log as loggerLog } from "./logger.js";
import { PersistEpermAlert } from "./persist-eperm.js";
import { createInitialState, type CompressionState, type CoreMessage } from "acp-kernel";
import { createInitialState, defaultCountTokens, prune, type CompressionState, type CoreMessage } from "acp-kernel";
import type { Session, BlockContent, BlockView } from "./session.js";

/**
Expand Down Expand Up @@ -108,10 +108,18 @@ interface PersistedSession {
state: CompressionState;
/** blockContents serialized as a plain record (Maps do not survive JSON). */
blockContents: Record<string, BlockContent>;
/** Latest full-conversation snapshot (v3+): the client's raw messages
* from its most recent request, overwritten every turn. Absent on v2
* files — export falls back to block-only rendering. */
/** Latest folded-view conversation snapshot (v3+): prune() rendered
* summaries in place of folded ranges, then truncated to the newest
* BILI_PERSIST_TAIL_TOKENS tokens (#401) — the raw full history is NOT
* persisted (it duplicated 63% of the corpus; originals of folded
* ranges remain available offline via blockContents). Absent on v2
* files and when the tail budget is 0 — export falls back to
* block-only rendering. */
messages?: CoreMessage[];
/** True when `messages` is an already-pruned folded snapshot (see above);
* absent on records written before #401, whose `messages` held the raw
* history and must still be pruned at export time. */
messagesFolded?: boolean;
}

type Logger = (level: "info" | "warn" | "error", msg: string) => void;
Expand Down Expand Up @@ -398,13 +406,15 @@ export class SessionStore {
}

function buildRecord(session: Session): PersistedSession {
const snapshot = boundedFoldedSnapshot(session);
return {
version: PERSIST_VERSION,
savedAt: Date.now(),
id: session.id,
meta: { ...session.meta },
stats: { ...session.stats },
messages: session.lastMessages,
messages: snapshot,
messagesFolded: snapshot ? true : undefined,
metadata: { ...session.metadata },
state: session.state,
blockContents: Object.fromEntries(session.blockContents),
Expand Down Expand Up @@ -466,6 +476,7 @@ function buildSession(parsed: PersistedSession): Session {
restored: true,
blockContents,
lastMessages: Array.isArray(parsed.messages) ? parsed.messages : undefined,
lastMessagesFolded: parsed.messagesFolded === true,
inFlight: 0,
persisted: true,
};
Expand Down Expand Up @@ -496,6 +507,47 @@ function persistEnabled(): boolean {
return true;
}

/** Token budget for the persisted folded-view snapshot (#401). The raw full
* history is never persisted — prune() first replaces folded ranges with
* their summaries (exactly what `bili export` renders by default), then the
* OLDEST messages are dropped until the view fits. 0 disables message
* persistence entirely (block summaries + blockContents survive). */
function persistTailTokens(): number {
const env = process.env.BILI_PERSIST_TAIL_TOKENS;
if (env) {
const n = Number.parseInt(env, 10);
if (Number.isFinite(n) && n >= 0) return n;
}
return 16384;
}

/** Bounded folded-view snapshot for the on-disk record (#401). See
* PersistedSession.messages. Truncation keeps whole messages from the NEWEST
* end; at least one message always survives (even if it alone exceeds the
* budget — a handoff doc with an empty tail is useless). */
function boundedFoldedSnapshot(session: Session): CoreMessage[] | undefined {
const msgs = session.lastMessages;
if (!msgs || msgs.length === 0) return undefined;
const budget = persistTailTokens();
if (budget === 0) return undefined;
let view = prune(msgs, session.state);
let total = 0;
for (const m of view) total += defaultCountTokens(m.text ?? "");
if (total > budget) {
let acc = 0;
let start = 0;
for (let i = view.length - 1; i >= 0; i--) {
acc += defaultCountTokens(view[i]!.text ?? "");
if (acc > budget) {
start = Math.min(i + 1, view.length - 1);
break;
}
}
if (start > 0) view = view.slice(start);
}
return view;
}

function epermAlertThreshold(): number {
const env = process.env.BILI_PERSIST_EPERM_ALERT_THRESHOLD;
if (env) {
Expand Down
12 changes: 11 additions & 1 deletion src/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,13 @@ export type Session = {
* request keeps a bounded, up-to-date copy — that is what makes offline
* export complete. Bounded by MAX_SESSIONS, same as blockContents. */
lastMessages?: CoreMessage[];
/** True when lastMessages holds an already-pruned folded-view snapshot
* (restored from disk — #401: the persisted record stores the bounded
* folded view, not the raw history). Export must render it as-is instead
* of re-running prune() (the snapshot's message ids no longer align with
* the state ranges). Cleared by snapshotMessages on the next live
* request — the client re-sends full raw history, restoring the invariant. */
lastMessagesFolded?: boolean;
/** Number of in-flight requests using this session. A session with
* inFlight > 0 must NOT be LRU-evicted: evicting it mid-stream flushes a
* half-mutated snapshot and then a miss reloads a SECOND Session object,
Expand Down Expand Up @@ -268,7 +275,10 @@ export function peekSession(id: string): Session | undefined {
* session, replaced every request — bounded, always the newest state. Empty
* arrays (parse failures) never clobber a good snapshot. */
export function snapshotMessages(session: Session, messages: CoreMessage[]): void {
if (messages.length > 0) session.lastMessages = messages;
if (messages.length > 0) {
session.lastMessages = messages;
session.lastMessagesFolded = false;
}
}

/** Mark a session's state as changed so it is persisted on the next debounce.
Expand Down
Loading
Loading