From ec0352cf0156401b95d6d40003cae5269e242af1 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Fri, 11 Sep 2026 22:23:33 +0800 Subject: [PATCH 1/6] feat: encrypt session files at rest behind BILI_ENCRYPTION_KEY (#708) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AES-256-GCM over zstd-compressed JSON (BILIENC1 format), wired through the kernel StateStore codec hook (acp-kernel#249). Key comes only from the environment variable; invalid values fail fast at startup. Boot takes over legacy plaintext files once: each is re-encoded and atomically replaced in place (rename IS the old-file deletion; crash mid-run self-heals). Off by default — unset key = today's plaintext behavior. Requires acp-kernel >= 0.0.64 (codec hook); pin bump + lockfile follow in a separate commit once 0.0.64 is published. --- CONFIGURATION.md | 1 + CONFIGURATION.zh-CN.md | 1 + src/encrypt.ts | 103 ++++++++++++++++ src/persist.ts | 100 ++++++++++++++- tests/persist-encryption.test.ts | 201 +++++++++++++++++++++++++++++++ 5 files changed, 403 insertions(+), 3 deletions(-) create mode 100644 src/encrypt.ts create mode 100644 tests/persist-encryption.test.ts diff --git a/CONFIGURATION.md b/CONFIGURATION.md index a09f5f8db2..2a8e405359 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -439,6 +439,7 @@ Environment variables take precedence over the config file. They are useful for | `BILI_TUNNEL_ALLOWED_HOSTS` | `/bili/` tunnel admission for **remote clients** (#409): comma-separated `host` or `host:port` entries that unlock loopback/private destinations (e.g. a LAN relay or the machine's own sglang) for non-loopback clients. The proxy itself and link-local/metadata addresses are always denied; local (loopback) clients always pass. | | `BILI_MAX_SESSIONS` | Max sessions held in memory (default `256`; LRU eviction — disk is the source of truth). | | `BILI_SESSIONS_DIR` | Directory for persisted session state (default XDG data dir). | +| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. | | `BILLION_CONTEXT_PROXY` | Exported by the launcher; client-side bili plugins/extensions detect it and self-disable their own compression (no double compression). | | `BILLION_CONTEXT_PLUGIN` | Set `0` to disable plugin mode entirely (wire-level tool injection resumes). | | `BILI_LAUNCHER_MODEL_WINDOWS` | Internal: the launcher hands the client's own per-model context windows (pi `models.json`, omp `models.yml`, opencode `models..limit`, codex `model_context_window`) to the spawned proxy as JSON, so the nudge denominator matches the real window for self-hosted models. Only the launcher sets it — no user configuration. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index b31421c38e..031c0e3cd0 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -435,6 +435,7 @@ | `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | persist EPERM 告警的重复窗口(毫秒)。`0`(默认)= 只告警一次后静默;`>0` = 失败持续期间最多每这么久重复告警一次。 | | `BILI_MAX_SESSIONS` | 内存中最多保留的会话数(默认 `256`;LRU 淘汰 —— 磁盘是事实源)。 | | `BILI_SESSIONS_DIR` | 会话持久化目录(默认 XDG data 目录)。 | +| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。 | | `BILLION_CONTEXT_PROXY` | launcher 会导出它;客户端侧 bili 插件/扩展检测到后自禁用自身压缩(避免双重压缩)。 | | `BILLION_CONTEXT_PLUGIN` | 设 `0` 彻底关闭插件模式(恢复 wire 层工具注入)。 | | `BILI_LAUNCHER_MODEL_WINDOWS` | 内部使用:launcher 把客户端自身配置里的逐模型上下文窗口(pi `models.json`、omp `models.yml`、opencode `models..limit`、codex `model_context_window`)以 JSON 传给自己拉起的代理,让 nudge 分母对自托管模型也用真实窗口。只有 launcher 会设置,无需用户配置。 | diff --git a/src/encrypt.ts b/src/encrypt.ts new file mode 100644 index 0000000000..b7ffe707b8 --- /dev/null +++ b/src/encrypt.ts @@ -0,0 +1,103 @@ +import { createCipheriv, createDecipheriv, randomBytes } from "node:crypto"; +import * as zlib from "node:zlib"; +import type { StateStoreCodec } from "acp-kernel/persist"; + +/** + * Session-file encryption at rest (#708): AES-256-GCM over an optionally + * zstd-compressed JSON envelope, applied around every StateStore write/read. + * + * THREAT MODEL: the proxy may run on untrusted nodes; session files hold + * block summaries plus up to ~16k tokens of folded conversation per session + * — effectively full conversation content (code, pasted credentials). The + * key comes ONLY from the BILI_ENCRYPTION_KEY environment variable: a key + * file next to the data sits on the same untrusted filesystem and defeats + * the purpose. + * + * FORMAT (v1): + * offset 0..7 magic "BILIENC1" + * offset 8 format version (0x01) + * offset 9 body mode (0x00 raw, 0x01 zstd) + * offset 10..21 GCM nonce (random per write) + * offset 22.. AES-256-GCM ciphertext, final 16 bytes = auth tag + * + * Compression runs BEFORE encryption (GCM ciphertext is incompressible). + * node:zlib gained zstd in Node 22.15; on older runtimes the mode byte + * records a raw body so every supported version can still decrypt it. + */ + +export const ENCRYPT_MAGIC = Buffer.from("BILIENC1", "utf8"); +const FORMAT_VERSION = 0x01; +const MODE_RAW = 0x00; +const MODE_ZSTD = 0x01; +const NONCE_LEN = 12; +const TAG_LEN = 16; +const HEADER_LEN = ENCRYPT_MAGIC.length + 2 + NONCE_LEN; +const MIN_ENCRYPTED_LEN = HEADER_LEN + TAG_LEN; + +/** Parse the BILI_ENCRYPTION_KEY value: hex or base64, must decode to + * exactly 32 bytes. Hex wins when both parse (a base64 string made only of + * hex digits is ambiguous — hex-first is the documented rule). Throws with + * an actionable message; the caller surfaces it as a startup crash (fail + * fast — never run silently unencrypted when the operator asked for it). */ +export function parseEncryptionKey(value: string): Buffer { + const v = value.trim(); + let buf: Buffer | null = null; + if (/^[0-9a-fA-F]+$/.test(v) && v.length % 2 === 0) { + buf = Buffer.from(v, "hex"); + } else if (/^[A-Za-z0-9+/]+={0,2}$/.test(v)) { + buf = Buffer.from(v, "base64"); + } + if (!buf || buf.length !== 32) { + const got = buf ? `${buf.length} bytes` : "an undecodable value"; + throw new Error( + `[encrypt] BILI_ENCRYPTION_KEY must be exactly 32 bytes encoded as hex (64 chars) or base64 — got ${got}`, + ); + } + return buf; +} + +function zstdAvailable(): boolean { + return typeof zlib.zstdCompressSync === "function" && typeof zlib.zstdDecompressSync === "function"; +} + +/** Build the StateStore codec for encrypted session files. decode() passes + * buffers without the magic through untouched (legacy plaintext), so mixed + * plaintext+encrypted trees load fine and the boot-time migration can peek + * the magic before rewriting. A decode failure (wrong key, corruption) + * throws and the kernel store treats the file as corrupt (warn + skip). */ +export function createSessionCodec(key: Buffer): StateStoreCodec { + return { + encode(data: string): Buffer { + const plain = Buffer.from(data, "utf8"); + const useZstd = zstdAvailable(); + const body = useZstd ? zlib.zstdCompressSync(plain) : plain; + const nonce = randomBytes(NONCE_LEN); + const cipher = createCipheriv("aes-256-gcm", key, nonce); + const ct = Buffer.concat([cipher.update(body), cipher.final()]); + return Buffer.concat([ + ENCRYPT_MAGIC, + Buffer.from([FORMAT_VERSION, useZstd ? MODE_ZSTD : MODE_RAW]), + nonce, + ct, + cipher.getAuthTag(), + ]); + }, + decode(buf: Buffer): string { + if (buf.length < ENCRYPT_MAGIC.length || !buf.subarray(0, ENCRYPT_MAGIC.length).equals(ENCRYPT_MAGIC)) { + return buf.toString("utf8"); + } + if (buf.length < MIN_ENCRYPTED_LEN || buf[ENCRYPT_MAGIC.length] !== FORMAT_VERSION) { + throw new Error(`[encrypt] unsupported session file format version ${buf[ENCRYPT_MAGIC.length]}`); + } + const mode = buf[ENCRYPT_MAGIC.length + 1]; + const nonce = buf.subarray(HEADER_LEN - NONCE_LEN, HEADER_LEN); + const tag = buf.subarray(buf.length - TAG_LEN); + const ct = buf.subarray(HEADER_LEN, buf.length - TAG_LEN); + const decipher = createDecipheriv("aes-256-gcm", key, nonce); + decipher.setAuthTag(tag); + const body = Buffer.concat([decipher.update(ct), decipher.final()]); + const plain = mode === MODE_ZSTD ? zlib.zstdDecompressSync(body) : body; + return plain.toString("utf8"); + }, + }; +} diff --git a/src/persist.ts b/src/persist.ts index 254b6fc92d..03efeb71cf 100644 --- a/src/persist.ts +++ b/src/persist.ts @@ -1,10 +1,11 @@ import { createHash } from "node:crypto"; -import { existsSync, mkdirSync, writeFileSync } from "node:fs"; -import { rm } from "node:fs/promises"; +import { existsSync, mkdirSync, renameSync, writeFileSync } from "node:fs"; +import { open, readdir, readFile, rm } from "node:fs/promises"; import * as path from "node:path"; -import { StateStore, flatFileNameFor, type PersistedEnvelope } from "acp-kernel/persist"; +import { StateStore, flatFileNameFor, type PersistedEnvelope, type StateStoreCodec } from "acp-kernel/persist"; import { sessionsDir } from "./paths.js"; import { log as loggerLog } from "./logger.js"; +import { createSessionCodec, ENCRYPT_MAGIC, parseEncryptionKey } from "./encrypt.js"; import { PersistEpermAlert } from "./persist-eperm.js"; import { createInitialState, defaultCountTokens, prune, type CompressionState, type CoreMessage } from "acp-kernel"; import type { Session, BlockContent, BlockView } from "./session.js"; @@ -33,6 +34,11 @@ import type { Session, BlockContent, BlockView } from "./session.js"; * - Forward-compat: `mergeState` fills any fields missing on a file written * by an older version, so a schema change never breaks old files. * - Disable with BILI_PERSIST=0 for ephemeral/test runs. + * - Encryption at rest (#708): BILI_ENCRYPTION_KEY (hex/base64, exactly 32 + * bytes) wraps every file as BILIENC1 AES-256-GCM(zstd(JSON)) via the + * kernel store's codec hook. Legacy plaintext files are re-encoded in + * place once at boot (migrateLegacyFiles); key material never touches + * disk or logs. * * MECHANISM lives in `acp-kernel/persist` (StateStore: atomic write, rename * retries, debounce, per-id serialization, corrupt-tolerant load, recursive @@ -186,6 +192,7 @@ export class SessionStore { private readonly store: StateStore; private readonly log: Logger; private readonly staleWarnAt = new Map(); + private readonly codec?: StateStoreCodec; constructor(opts?: { dir?: string; debounceMs?: number; enabled?: boolean; log?: Logger }) { const debounceMs = opts?.debounceMs ?? defaultDebounce(); @@ -193,6 +200,14 @@ export class SessionStore { this.dir = opts?.dir ?? defaultDir(); const baseLog = opts?.log ?? defaultLogger; this.log = baseLog; + // #708: env-only key (a key file next to the data sits on the same + // untrusted filesystem). Invalid values throw here — fail fast at + // startup instead of running silently unencrypted. + const keyEnv = process.env.BILI_ENCRYPTION_KEY; + if (keyEnv) { + this.codec = createSessionCodec(parseEncryptionKey(keyEnv)); + baseLog("info", "[persist] session-file encryption enabled (AES-256-GCM)"); + } const epermAlert = new PersistEpermAlert({ dir: this.dir, threshold: epermAlertThreshold(), @@ -203,6 +218,7 @@ export class SessionStore { version: PERSIST_VERSION, debounceMs: Math.max(0, debounceMs), enabled: this.enabled, + codec: this.codec, log: (level, msg) => { epermAlert.observe(level, msg); baseLog(level, msg); @@ -248,6 +264,7 @@ export class SessionStore { * tree twice per start. */ async boot(): Promise> { if (!this.enabled) return new Map(); + await this.migrateLegacyFiles(); const loaded = await this.store.loadAll(); await this.applyLegacyMigration(loaded); const out = new Map(); @@ -257,6 +274,58 @@ export class SessionStore { return out; } + /** #708: when encryption is enabled, take over legacy plaintext files: + * every .json under the sessions dir lacking the BILIENC1 magic is + * re-encoded in place — temp write + rename onto the SAME path, so the + * atomic replace IS the old-file deletion (no window where both, or + * neither, copy exists). A crash mid-run leaves each file either old or + * new; the next boot finishes the job. Self-terminating: the 8-byte + * magic peek decides per file, so later boots cost O(files × 8 bytes). */ + private async migrateLegacyFiles(): Promise { + if (!this.codec) return; + let files: string[]; + try { + files = await walkJsonFiles(this.dir); + } catch { + return; + } + let migrated = 0; + let failed = 0; + for (const file of files) { + let head: Buffer; + try { + head = await readFileHead(file); + } catch { + continue; + } + if (head.equals(ENCRYPT_MAGIC)) continue; + let parsed: unknown; + try { + parsed = JSON.parse(await readFile(file, "utf8")); + } catch { + failed++; + this.log("warn", `[persist] encryption migration (#708): leaving unreadable file in place: ${file}`); + continue; + } + const tmp = `${file}.tmp-enc-${process.pid}-${Date.now()}`; + try { + writeFileSync(tmp, this.codec.encode(JSON.stringify(parsed))); + renameSync(tmp, file); + migrated++; + } catch { + failed++; + this.log("warn", `[persist] encryption migration (#708): failed to re-encode: ${file}`); + await rm(tmp, { force: true }).catch(() => {}); + } + } + if (migrated > 0 || failed > 0) { + this.log( + failed > 0 ? "warn" : "info", + `[persist] encryption migration (#708): re-encoded ${migrated} legacy session file(s)${failed > 0 ? `, ${failed} failed` : ""}`, + ); + } + } + /** One-time migration for the #286 identity change: sessions persisted * under the old derived hash id are re-keyed to the client-provided * conversation value stored in meta.label (which is now the session id @@ -596,6 +665,31 @@ function persistEnabled(): boolean { return true; } +async function walkJsonFiles(dir: string): Promise { + const entries = await readdir(dir, { withFileTypes: true }); + const out: string[] = []; + for (const e of entries) { + const full = path.join(dir, e.name); + if (e.isDirectory()) { + out.push(...(await walkJsonFiles(full))); + } else if (e.isFile() && e.name.endsWith(".json") && !e.name.startsWith(".tmp-")) { + out.push(full); + } + } + return out; +} + +async function readFileHead(file: string, len: number = ENCRYPT_MAGIC.length): Promise { + const fh = await open(file, "r"); + try { + const buf = Buffer.alloc(len); + const { bytesRead } = await fh.read(buf, 0, len, 0); + return buf.subarray(0, bytesRead); + } finally { + await fh.close(); + } +} + /** Token budget for the persisted folded-view snapshot (#401). The raw full * history is never persisted — prune() first replaces folded ranges with * their summaries (exactly what `bili export` renders by default), then the diff --git a/tests/persist-encryption.test.ts b/tests/persist-encryption.test.ts new file mode 100644 index 0000000000..663a260e8d --- /dev/null +++ b/tests/persist-encryption.test.ts @@ -0,0 +1,201 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { tmpdir } from "node:os"; +import { createHash, randomBytes } from "node:crypto"; +import { mkdtempSync, rmSync, mkdirSync, writeFileSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { SessionStore } from "../src/persist.ts"; +import { createSessionCodec, ENCRYPT_MAGIC, parseEncryptionKey } from "../src/encrypt.ts"; +import type { Session } from "../src/session.ts"; +import { createInitialState } from "acp-kernel"; + +const KEY = randomBytes(32).toString("hex"); +const KEY_OTHER = randomBytes(32).toString("hex"); + +type LogLine = { level: string; msg: string }; + +function makeSession(id: string): Session { + return { + id, + meta: { protocol: "openai", upstreamOrigin: "http://upstream" }, + stats: { requests: 0, tokensSaved: 0, inputTokens: 0, cachedTokens: 0, outputTokens: 0, cacheSamples: 0, lastInputTokens: 0, contextTokens: 0 }, + metadata: {}, + state: createInitialState(), + createdAt: Date.now(), + lastSeen: Date.now(), + blockContents: new Map(), + inFlight: 0, + persisted: false, + }; +} + +interface Harness { + dir: string; + logs: LogLine[]; +} + +async function withTempDir(name: string, fn: (h: Harness) => Promise): Promise { + await test(name, async () => { + const dir = mkdtempSync(join(tmpdir(), "bili-encrypt-")); + const logs: LogLine[] = []; + const h = { dir, logs }; + try { + await fn(h); + } finally { + delete process.env.BILI_ENCRYPTION_KEY; + rmSync(dir, { recursive: true, force: true }); + } + }); +} + +function newStore(h: Harness, debounceMs = 0): SessionStore { + return new SessionStore({ + dir: h.dir, + debounceMs, + enabled: true, + log: (level, msg) => h.logs.push({ level, msg }), + }); +} + +test("parseEncryptionKey accepts hex and base64, rejects everything else", () => { + const raw = randomBytes(32); + assert.deepEqual(parseEncryptionKey(raw.toString("hex")), raw); + assert.deepEqual(parseEncryptionKey(raw.toString("base64")), raw); + assert.throws(() => parseEncryptionKey("beef"), /exactly 32 bytes/); + assert.throws(() => parseEncryptionKey(""), /exactly 32 bytes/); + assert.throws(() => parseEncryptionKey("!!!not-a-key!!!"), /exactly 32 bytes/); + assert.throws(() => parseEncryptionKey(randomBytes(31).toString("hex")), /exactly 32 bytes/); +}); + +test("codec: roundtrip, magic prefix, per-write nonce, tamper and wrong-key rejection", () => { + const codec = createSessionCodec(Buffer.from(KEY, "hex")); + const other = createSessionCodec(Buffer.from(KEY_OTHER, "hex")); + const json = JSON.stringify({ hello: "world", n: [1, 2, 3] }); + + const enc = Buffer.isBuffer(codec.encode(json)) ? codec.encode(json) : Buffer.from(codec.encode(json)); + assert.ok(enc.subarray(0, ENCRYPT_MAGIC.length).equals(ENCRYPT_MAGIC), "file starts with BILIENC1 magic"); + assert.equal(codec.decode(enc), json); + + const enc2 = Buffer.from(codec.encode(json)); + assert.notDeepEqual(enc, enc2, "random nonce per write"); + assert.equal(codec.decode(enc2), json); + + const tampered = Buffer.from(enc); + tampered[tampered.length - 1] ^= 0xff; + assert.throws(() => codec.decode(tampered), undefined, "tampered tag must throw"); + assert.throws(() => other.decode(enc), undefined, "wrong key must throw"); +}); + +test("codec passes legacy plaintext through untouched", () => { + const codec = createSessionCodec(Buffer.from(KEY, "hex")); + const json = '{"version":3,"id":"x"}'; + assert.equal(codec.decode(Buffer.from(json, "utf8")), json); +}); + +await withTempDir("writes are encrypted on disk when the key is set", async (h) => { + process.env.BILI_ENCRYPTION_KEY = KEY; + const store = newStore(h); + try { + await store.writeNow(makeSession("s-enc")); + const file = join(h.dir, "openai", "upstream_" + createHash("sha256").update("s-enc", "utf8").digest("hex").slice(0, 24) + ".json"); + const head = readFileSync(file).subarray(0, ENCRYPT_MAGIC.length); + assert.ok(head.equals(ENCRYPT_MAGIC), "on-disk file is BILIENC1-encrypted"); + const reloaded = store.loadSync("s-enc", { protocol: "openai", upstreamOrigin: "http://upstream" }); + assert.ok(reloaded, "session still loads through the codec"); + assert.ok(h.logs.some((l) => l.msg.includes("encryption enabled"))); + } finally { + store.cancelAll(); + } +}); + +await withTempDir("boot migrates legacy plaintext files to encrypted in place", async (h) => { + // Phase 1: legacy plaintext tree written by an unencrypted store. + const legacy = newStore(h); + await legacy.writeNow(makeSession("s-1")); + await legacy.writeNow(makeSession("s-2")); + legacy.cancelAll(); + + // Phase 2: boot with the key — every unencoded file must be taken over. + process.env.BILI_ENCRYPTION_KEY = KEY; + const store = newStore(h); + try { + const loaded = await store.boot(); + assert.equal(loaded.size, 2, "both legacy sessions load"); + assert.ok(loaded.has("s-1") && loaded.has("s-2")); + for (const id of ["s-1", "s-2"]) { + const file = join(h.dir, "openai", "upstream_" + createHash("sha256").update(id, "utf8").digest("hex").slice(0, 24) + ".json"); + const head = readFileSync(file).subarray(0, ENCRYPT_MAGIC.length); + assert.ok(head.equals(ENCRYPT_MAGIC), `${id} file re-encoded in place`); + } + assert.ok(h.logs.some((l) => l.msg.includes("re-encoded 2 legacy session file(s)")), "migration logged"); + + // Phase 3: second boot is a no-op (no double-wrap, no re-migration log). + store.cancelAll(); + const logs2: LogLine[] = []; + const again = new SessionStore({ dir: h.dir, debounceMs: 0, enabled: true, log: (level, msg) => logs2.push({ level, msg }) }); + try { + const loaded2 = await again.boot(); + assert.equal(loaded2.size, 2, "still exactly 2 sessions — no double encoding"); + assert.ok(!logs2.some((l) => l.msg.includes("re-encoded")), "no migration on an already-encoded tree"); + } finally { + again.cancelAll(); + } + } finally { + store.cancelAll(); + } +}); + +await withTempDir("migration covers spill-style .fb.json files and leaves corrupt files alone", async (h) => { + mkdirSync(join(h.dir, "openai"), { recursive: true }); + const spill = join(h.dir, "openai", "s-spill.fb.json"); + writeFileSync(spill, JSON.stringify({ version: 3, savedAt: Date.now(), id: "s-spill", payload: {} }), "utf8"); + const corrupt = join(h.dir, "openai", "garbage_deadbeef.json"); + writeFileSync(corrupt, "%%% not json %%%", "utf8"); + + process.env.BILI_ENCRYPTION_KEY = KEY; + const store = newStore(h); + try { + const loaded = await store.boot(); + assert.equal(loaded.size, 0, "neither fake file is a valid session record"); + assert.ok(readFileSync(spill).subarray(0, ENCRYPT_MAGIC.length).equals(ENCRYPT_MAGIC), "spill file re-encoded"); + assert.equal(readFileSync(corrupt, "utf8"), "%%% not json %%%", "unreadable file left in place"); + assert.ok(h.logs.some((l) => l.level === "warn" && l.msg.includes("leaving unreadable file in place"))); + } finally { + store.cancelAll(); + } +}); + +await withTempDir("an encrypted tree booted with the WRONG key loses those sessions as corrupt (no crash)", async (h) => { + process.env.BILI_ENCRYPTION_KEY = KEY; + const writer = newStore(h); + await writer.writeNow(makeSession("s-secret")); + writer.cancelAll(); + + process.env.BILI_ENCRYPTION_KEY = KEY_OTHER; + const reader = newStore(h); + try { + const loaded = await reader.boot(); + assert.equal(loaded.has("s-secret"), false, "wrong key -> auth failure -> corrupt-file skip"); + } finally { + reader.cancelAll(); + } +}); + +await withTempDir("invalid key fails fast at construction", async (h) => { + process.env.BILI_ENCRYPTION_KEY = "beef"; + assert.throws(() => newStore(h), /BILI_ENCRYPTION_KEY.*exactly 32 bytes/); +}); + +await withTempDir("without a key, files stay plaintext (zero behavior change)", async (h) => { + const store = newStore(h); + try { + await store.writeNow(makeSession("s-plain")); + const file = join(h.dir, "openai", "upstream_" + createHash("sha256").update("s-plain", "utf8").digest("hex").slice(0, 24) + ".json"); + assert.equal(readFileSync(file, "utf8")[0], "{", "plain JSON on disk"); + const loaded = await store.boot(); + assert.equal(loaded.size, 1); + assert.ok(!h.logs.some((l) => l.msg.includes("encryption enabled"))); + } finally { + store.cancelAll(); + } +}); From 29d0403a8f1097b4739b01a8b6ac6791ac5fd275 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Fri, 11 Sep 2026 22:26:05 +0800 Subject: [PATCH 2/6] docs: changelog entry for session-file encryption (#708, PR #711) --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19bf0905e6..9287995608 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ### Features +- **Session-file encryption at rest behind `BILI_ENCRYPTION_KEY` (#708, PR #711)**: for deployments on untrusted nodes, persisted session files (block summaries plus up to ~16k tokens of folded conversation per session — effectively full conversation content) can now be stored as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd via built-in node:zlib on Node ≥ 22.15, raw mode byte on older runtimes; compression also shrinks files ~5–10×). The key is env-only — deliberately NO key-file support, since a key file would sit on the same untrusted filesystem as the data — exactly 32 bytes hex or base64, never written to disk or logs; an invalid value aborts startup (fail fast instead of silently running unencrypted). When enabled, boot takes over legacy plaintext files once: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on the next boot), covering `.fb.json` spill files too; unreadable files are left in place with a warning. Wrong key at boot → GCM auth failure → affected sessions skip as corrupt (logged, no crash). Mechanism is the kernel's optional `StateStore` codec hook (acp-kernel#249 → 0.0.64); unset key = zero behavior change. - **Per-route `passthrough` option for providers (#661, PR #662)**: `"providers": { "mitm://zcode.z.ai": { "passthrough": true } }` forwards matching requests byte-for-byte — no kernel round-trip, no render tags, no `prompt_cache_key` removal, response piped verbatim, no session state — the scoped sibling of the global `passthrough` switch. `mitm://` keys target only MITM (login-client) traffic; plain `https://` host keys cover both MITM and `/bili/` traffic. - **Round-evidence closure for compress-reasoning drop (#651, #348 twin)**: the closure gate shipped with the reasoning drop — "compress call followed by a genuine user message" — is unreachable in long agentic sessions (no user messages after the opening prompt, observed: 0 drops while 30 compress rounds retained 20.6K/8.4K/10.6K-char thinking floors). A round now closes on tool-result evidence: the compress call's `tool-result` (matching `toolCallId`) exists at a later index and at least one message follows it. In-flight rounds (result missing or still the last message) stay untouched; per-provider `compress.providers..reasoning.drop=false` escape hatch preserved for reasoning-replay models (GLM). Mirrors billion-context-pi #348 (PR #349). From 0f9c3f1a97aa7769cedf30f088ebd4c19657b275 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Fri, 11 Sep 2026 23:01:03 +0800 Subject: [PATCH 3/6] fix: sweep orphaned .tmp-enc-* temps left by a crashed encryption migration (#708) --- src/persist.ts | 19 +++++++++++++++---- tests/persist-encryption.test.ts | 25 ++++++++++++++++++++++++- 2 files changed, 39 insertions(+), 5 deletions(-) diff --git a/src/persist.ts b/src/persist.ts index 03efeb71cf..ee1033c1bb 100644 --- a/src/persist.ts +++ b/src/persist.ts @@ -278,9 +278,10 @@ export class SessionStore { * every .json under the sessions dir lacking the BILIENC1 magic is * re-encoded in place — temp write + rename onto the SAME path, so the * atomic replace IS the old-file deletion (no window where both, or - * neither, copy exists). A crash mid-run leaves each file either old or - * new; the next boot finishes the job. Self-terminating: the 8-byte - * magic peek decides per file, so later boots cost O(files × 8 bytes). */ + * neither, copy exists). A crash mid-run leaves each file either old or + * new; the next boot finishes the job and sweeps the crashed run's + * orphaned temps. Self-terminating: the 8-byte magic peek decides per + * file, so later boots cost O(files × 8 bytes). */ private async migrateLegacyFiles(): Promise { if (!this.codec) return; let files: string[]; @@ -292,6 +293,10 @@ export class SessionStore { let migrated = 0; let failed = 0; for (const file of files) { + if (STALE_ENC_TEMP_RE.test(path.basename(file))) { + await rm(file, { force: true }).catch(() => {}); + continue; + } let head: Buffer; try { head = await readFileHead(file); @@ -665,6 +670,12 @@ function persistEnabled(): boolean { return true; } +/** Temp name used by migrateLegacyFiles: `.tmp-enc--`. A + * process death between write and rename orphans it; any such name present + * at boot is stale by definition (the walk runs before this boot writes + * anything) and gets swept. */ +const STALE_ENC_TEMP_RE = /\.tmp-enc-\d+-\d+$/; + async function walkJsonFiles(dir: string): Promise { const entries = await readdir(dir, { withFileTypes: true }); const out: string[] = []; @@ -672,7 +683,7 @@ async function walkJsonFiles(dir: string): Promise { const full = path.join(dir, e.name); if (e.isDirectory()) { out.push(...(await walkJsonFiles(full))); - } else if (e.isFile() && e.name.endsWith(".json") && !e.name.startsWith(".tmp-")) { + } else if (e.isFile() && (STALE_ENC_TEMP_RE.test(e.name) || (e.name.endsWith(".json") && !e.name.startsWith(".tmp-")))) { out.push(full); } } diff --git a/tests/persist-encryption.test.ts b/tests/persist-encryption.test.ts index 663a260e8d..6d0e7649dd 100644 --- a/tests/persist-encryption.test.ts +++ b/tests/persist-encryption.test.ts @@ -2,7 +2,7 @@ import { test } from "node:test"; import assert from "node:assert/strict"; import { tmpdir } from "node:os"; import { createHash, randomBytes } from "node:crypto"; -import { mkdtempSync, rmSync, mkdirSync, writeFileSync, readFileSync } from "node:fs"; +import { mkdtempSync, rmSync, mkdirSync, writeFileSync, readFileSync, existsSync } from "node:fs"; import { join } from "node:path"; import { SessionStore } from "../src/persist.ts"; import { createSessionCodec, ENCRYPT_MAGIC, parseEncryptionKey } from "../src/encrypt.ts"; @@ -165,6 +165,29 @@ await withTempDir("migration covers spill-style .fb.json files and leaves corrup } }); +await withTempDir("boot sweeps orphaned .tmp-enc-* temps left by a crashed migration", async (h) => { + const legacy = newStore(h); + await legacy.writeNow(makeSession("s-crash")); + legacy.cancelAll(); + + // Simulate a process death between the temp write and the rename: a + // stale temp sits next to an unencoded legacy file. + const file = join(h.dir, "openai", "upstream_" + createHash("sha256").update("s-crash", "utf8").digest("hex").slice(0, 24) + ".json"); + const orphan = `${file}.tmp-enc-99999-1700000000000`; + writeFileSync(orphan, "stale temp from a crashed migration"); + + process.env.BILI_ENCRYPTION_KEY = KEY; + const store = newStore(h); + try { + const loaded = await store.boot(); + assert.equal(loaded.size, 1, "legacy session loads after migration"); + assert.ok(readFileSync(file).subarray(0, ENCRYPT_MAGIC.length).equals(ENCRYPT_MAGIC), "file re-encoded in place"); + assert.equal(existsSync(orphan), false, "orphaned temp swept on next boot"); + } finally { + store.cancelAll(); + } +}); + await withTempDir("an encrypted tree booted with the WRONG key loses those sessions as corrupt (no crash)", async (h) => { process.env.BILI_ENCRYPTION_KEY = KEY; const writer = newStore(h); From 85c31214e611deee439a7440d6694c2cc7b9c809 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Fri, 11 Sep 2026 23:29:17 +0800 Subject: [PATCH 4/6] docs: clarify encryption threat model (offline file access, not live root) in #708 discussion --- CONFIGURATION.md | 2 +- CONFIGURATION.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 2a8e405359..77e84d96de 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -439,7 +439,7 @@ Environment variables take precedence over the config file. They are useful for | `BILI_TUNNEL_ALLOWED_HOSTS` | `/bili/` tunnel admission for **remote clients** (#409): comma-separated `host` or `host:port` entries that unlock loopback/private destinations (e.g. a LAN relay or the machine's own sglang) for non-loopback clients. The proxy itself and link-local/metadata addresses are always denied; local (loopback) clients always pass. | | `BILI_MAX_SESSIONS` | Max sessions held in memory (default `256`; LRU eviction — disk is the source of truth). | | `BILI_SESSIONS_DIR` | Directory for persisted session state (default XDG data dir). | -| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. | +| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. Symmetric by design (the same process encrypts and decrypts): it defends against **offline** file access (stolen volumes, leaked backups, cloud-synced state dirs) — not against a live full-root attacker on the running node, who reads the key from the process environment and the conversation from process memory alike; for that threat use a TEE or `BILI_PERSIST=0`. | | `BILLION_CONTEXT_PROXY` | Exported by the launcher; client-side bili plugins/extensions detect it and self-disable their own compression (no double compression). | | `BILLION_CONTEXT_PLUGIN` | Set `0` to disable plugin mode entirely (wire-level tool injection resumes). | | `BILI_LAUNCHER_MODEL_WINDOWS` | Internal: the launcher hands the client's own per-model context windows (pi `models.json`, omp `models.yml`, opencode `models..limit`, codex `model_context_window`) to the spawned proxy as JSON, so the nudge denominator matches the real window for self-hosted models. Only the launcher sets it — no user configuration. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 031c0e3cd0..ea7a704016 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -435,7 +435,7 @@ | `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | persist EPERM 告警的重复窗口(毫秒)。`0`(默认)= 只告警一次后静默;`>0` = 失败持续期间最多每这么久重复告警一次。 | | `BILI_MAX_SESSIONS` | 内存中最多保留的会话数(默认 `256`;LRU 淘汰 —— 磁盘是事实源)。 | | `BILI_SESSIONS_DIR` | 会话持久化目录(默认 XDG data 目录)。 | -| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。 | +| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。对称加密为刻意设计(同一进程既加密又解密):防御的是文件**离线**流出(磁盘镜像、备份泄露、被云同步的状态目录),不防御运行节点上的实时全权 root 攻击者——那种攻击者同样能从进程环境取到密钥、从进程内存直接读到会话内容;针对该威胁请用 TEE 或 `BILI_PERSIST=0`。 | | `BILLION_CONTEXT_PROXY` | launcher 会导出它;客户端侧 bili 插件/扩展检测到后自禁用自身压缩(避免双重压缩)。 | | `BILLION_CONTEXT_PLUGIN` | 设 `0` 彻底关闭插件模式(恢复 wire 层工具注入)。 | | `BILI_LAUNCHER_MODEL_WINDOWS` | 内部使用:launcher 把客户端自身配置里的逐模型上下文窗口(pi `models.json`、omp `models.yml`、opencode `models..limit`、codex `model_context_window`)以 JSON 传给自己拉起的代理,让 nudge 分母对自托管模型也用真实窗口。只有 launcher 会设置,无需用户配置。 | From e90e589385809b83a8f55d7cb074fb2fe01f7b80 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 12 Sep 2026 09:27:14 +0800 Subject: [PATCH 5/6] docs: encode owner-confirmed two-scenario threat model for BILI_ENCRYPTION_KEY (#708) --- CONFIGURATION.md | 2 +- CONFIGURATION.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 77e84d96de..026976ee2c 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -439,7 +439,7 @@ Environment variables take precedence over the config file. They are useful for | `BILI_TUNNEL_ALLOWED_HOSTS` | `/bili/` tunnel admission for **remote clients** (#409): comma-separated `host` or `host:port` entries that unlock loopback/private destinations (e.g. a LAN relay or the machine's own sglang) for non-loopback clients. The proxy itself and link-local/metadata addresses are always denied; local (loopback) clients always pass. | | `BILI_MAX_SESSIONS` | Max sessions held in memory (default `256`; LRU eviction — disk is the source of truth). | | `BILI_SESSIONS_DIR` | Directory for persisted session state (default XDG data dir). | -| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. Symmetric by design (the same process encrypts and decrypts): it defends against **offline** file access (stolen volumes, leaked backups, cloud-synced state dirs) — not against a live full-root attacker on the running node, who reads the key from the process environment and the conversation from process memory alike; for that threat use a TEE or `BILI_PERSIST=0`. | +| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. Symmetric by design (the same process encrypts and decrypts). Threat model (#708, owner-confirmed): it defends against **offline/mechanical** file acquisition — provider disk swaps, offline disk snapshots after node-image drift, stolen volumes, leaked backups, cloud-synced state dirs — where an offline third party cannot read the content without the key. It does NOT defend against targeted adversaries with live access to the node; for that tier move the trust root out of the proxy (KMS / TEE / confidential VMs, hardened permissions) instead of solving it inside the proxy — at that level far more than the key is exposed, so proxy-level key handling is not the boundary to hold (`BILI_PERSIST=0` disables persistence entirely). | | `BILLION_CONTEXT_PROXY` | Exported by the launcher; client-side bili plugins/extensions detect it and self-disable their own compression (no double compression). | | `BILLION_CONTEXT_PLUGIN` | Set `0` to disable plugin mode entirely (wire-level tool injection resumes). | | `BILI_LAUNCHER_MODEL_WINDOWS` | Internal: the launcher hands the client's own per-model context windows (pi `models.json`, omp `models.yml`, opencode `models..limit`, codex `model_context_window`) to the spawned proxy as JSON, so the nudge denominator matches the real window for self-hosted models. Only the launcher sets it — no user configuration. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index ea7a704016..20aeaafeee 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -435,7 +435,7 @@ | `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | persist EPERM 告警的重复窗口(毫秒)。`0`(默认)= 只告警一次后静默;`>0` = 失败持续期间最多每这么久重复告警一次。 | | `BILI_MAX_SESSIONS` | 内存中最多保留的会话数(默认 `256`;LRU 淘汰 —— 磁盘是事实源)。 | | `BILI_SESSIONS_DIR` | 会话持久化目录(默认 XDG data 目录)。 | -| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。对称加密为刻意设计(同一进程既加密又解密):防御的是文件**离线**流出(磁盘镜像、备份泄露、被云同步的状态目录),不防御运行节点上的实时全权 root 攻击者——那种攻击者同样能从进程环境取到密钥、从进程内存直接读到会话内容;针对该威胁请用 TEE 或 `BILI_PERSIST=0`。 | +| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。对称加密为刻意设计(同一进程既加密又解密)。威胁模型(#708,owner 确认):防的是**离线/机械性**的文件获取——云厂商换盘、节点镜像漂移后的离线磁盘快照、磁盘镜像失窃、备份泄露、被云同步的状态目录——离线第三方拿不到密钥即无法读取内容。不防御对活节点有访问权的定向攻击者;那一档应把信任根移出 proxy(KMS / TEE / 机密虚拟机 + 强化权限体系),而不是在 proxy 本身想办法——到了那个程度暴露的远不止密钥,proxy 层不是该守的边界(`BILI_PERSIST=0` 可彻底关闭持久化)。 | | `BILLION_CONTEXT_PROXY` | launcher 会导出它;客户端侧 bili 插件/扩展检测到后自禁用自身压缩(避免双重压缩)。 | | `BILLION_CONTEXT_PLUGIN` | 设 `0` 彻底关闭插件模式(恢复 wire 层工具注入)。 | | `BILI_LAUNCHER_MODEL_WINDOWS` | 内部使用:launcher 把客户端自身配置里的逐模型上下文窗口(pi `models.json`、omp `models.yml`、opencode `models..limit`、codex `model_context_window`)以 JSON 传给自己拉起的代理,让 nudge 分母对自托管模型也用真实窗口。只有 launcher 会设置,无需用户配置。 | From 1653250a190180630280d78bf198f82aec214a8a Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 12 Sep 2026 09:34:04 +0800 Subject: [PATCH 6/6] docs: note why same-trust-domain double-encryption adds nothing (#708) --- CONFIGURATION.md | 2 +- CONFIGURATION.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 026976ee2c..a70fa5a2bd 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -439,7 +439,7 @@ Environment variables take precedence over the config file. They are useful for | `BILI_TUNNEL_ALLOWED_HOSTS` | `/bili/` tunnel admission for **remote clients** (#409): comma-separated `host` or `host:port` entries that unlock loopback/private destinations (e.g. a LAN relay or the machine's own sglang) for non-loopback clients. The proxy itself and link-local/metadata addresses are always denied; local (loopback) clients always pass. | | `BILI_MAX_SESSIONS` | Max sessions held in memory (default `256`; LRU eviction — disk is the source of truth). | | `BILI_SESSIONS_DIR` | Directory for persisted session state (default XDG data dir). | -| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. Symmetric by design (the same process encrypts and decrypts). Threat model (#708, owner-confirmed): it defends against **offline/mechanical** file acquisition — provider disk swaps, offline disk snapshots after node-image drift, stolen volumes, leaked backups, cloud-synced state dirs — where an offline third party cannot read the content without the key. It does NOT defend against targeted adversaries with live access to the node; for that tier move the trust root out of the proxy (KMS / TEE / confidential VMs, hardened permissions) instead of solving it inside the proxy — at that level far more than the key is exposed, so proxy-level key handling is not the boundary to hold (`BILI_PERSIST=0` disables persistence entirely). | +| `BILI_ENCRYPTION_KEY` | Encrypt session files at rest (#708) for deployments on untrusted nodes. Exactly 32 bytes, hex (64 chars) or base64; unset = plain JSON files (default behavior, unchanged). When set: every session file is written as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd on Node ≥ 22.15, raw body on older runtimes) — compression also shrinks the file ~5–10×. Existing unencoded files are taken over once at boot: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on next boot). The key comes ONLY from this environment variable — never written to disk or logs — so keep it out of the same filesystem's reach. Invalid values abort startup (fail fast, never silently unencrypted). Booting with the wrong key skips the affected sessions as corrupt (logged, no crash). Losing the key makes encrypted sessions permanently unreadable. Symmetric by design (the same process encrypts and decrypts). Threat model (#708, owner-confirmed): it defends against **offline/mechanical** file acquisition — provider disk swaps, offline disk snapshots after node-image drift, stolen volumes, leaked backups, cloud-synced state dirs — where an offline third party cannot read the content without the key. It does NOT defend against targeted adversaries with live access to the node; for that tier move the trust root out of the proxy (KMS / TEE / confidential VMs, hardened permissions) instead of solving it inside the proxy — at that level far more than the key is exposed, so proxy-level key handling is not the boundary to hold (`BILI_PERSIST=0` disables persistence entirely). Double-encrypting the key with a second key from the same process/environment adds nothing: in every offline-theft scenario the attacker lacks exactly one artifact — your off-disk secret — whether it is called the data key or the wrapping key; only a wrapping key in a different trust domain (KMS/TPM/TEE) raises the bar, and that is scenario 2 above. | | `BILLION_CONTEXT_PROXY` | Exported by the launcher; client-side bili plugins/extensions detect it and self-disable their own compression (no double compression). | | `BILLION_CONTEXT_PLUGIN` | Set `0` to disable plugin mode entirely (wire-level tool injection resumes). | | `BILI_LAUNCHER_MODEL_WINDOWS` | Internal: the launcher hands the client's own per-model context windows (pi `models.json`, omp `models.yml`, opencode `models..limit`, codex `model_context_window`) to the spawned proxy as JSON, so the nudge denominator matches the real window for self-hosted models. Only the launcher sets it — no user configuration. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 20aeaafeee..294c904137 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -435,7 +435,7 @@ | `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | persist EPERM 告警的重复窗口(毫秒)。`0`(默认)= 只告警一次后静默;`>0` = 失败持续期间最多每这么久重复告警一次。 | | `BILI_MAX_SESSIONS` | 内存中最多保留的会话数(默认 `256`;LRU 淘汰 —— 磁盘是事实源)。 | | `BILI_SESSIONS_DIR` | 会话持久化目录(默认 XDG data 目录)。 | -| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。对称加密为刻意设计(同一进程既加密又解密)。威胁模型(#708,owner 确认):防的是**离线/机械性**的文件获取——云厂商换盘、节点镜像漂移后的离线磁盘快照、磁盘镜像失窃、备份泄露、被云同步的状态目录——离线第三方拿不到密钥即无法读取内容。不防御对活节点有访问权的定向攻击者;那一档应把信任根移出 proxy(KMS / TEE / 机密虚拟机 + 强化权限体系),而不是在 proxy 本身想办法——到了那个程度暴露的远不止密钥,proxy 层不是该守的边界(`BILI_PERSIST=0` 可彻底关闭持久化)。 | +| `BILI_ENCRYPTION_KEY` | 会话文件静态加密(#708),适用于部署在不可信节点的场景。密钥必须恰好 32 字节,hex(64 字符)或 base64;不设置 = 明文 JSON 文件(默认,行为不变)。设置后:每个会话文件以 `BILIENC1` AES-256-GCM 加密 zstd 压缩后的 JSON 写入(Node ≥ 22.15 启用 zstd,旧版本写原始数据)——压缩同时可使文件缩小约 5–10 倍。已有的未编码文件在启动时一次性接管:逐个重新编码并原子替换到原路径(rename 即旧文件的删除;迁移中途崩溃会在下次启动自愈)。密钥只从该环境变量读取——永不落盘、永不进日志——请确保它不受同一文件系统上的其他进程触及。非法值会导致启动中止(快速失败,绝不静默明文运行)。用错误的密钥启动时,受影响的会话按损坏文件跳过(有日志,不崩溃)。丢失密钥将使已加密的会话永久不可读。对称加密为刻意设计(同一进程既加密又解密)。威胁模型(#708,owner 确认):防的是**离线/机械性**的文件获取——云厂商换盘、节点镜像漂移后的离线磁盘快照、磁盘镜像失窃、备份泄露、被云同步的状态目录——离线第三方拿不到密钥即无法读取内容。不防御对活节点有访问权的定向攻击者;那一档应把信任根移出 proxy(KMS / TEE / 机密虚拟机 + 强化权限体系),而不是在 proxy 本身想办法——到了那个程度暴露的远不止密钥,proxy 层不是该守的边界(`BILI_PERSIST=0` 可彻底关闭持久化)。用同一进程/环境中的第二把密钥对密钥做二次加密不增加任何安全性:所有离线失窃场景里攻击者缺的始终只有一个工件——你的非落盘秘密——无论它叫数据密钥还是包裹密钥;只有把包裹密钥放进不同信任域(KMS/TPM/TEE)才能提高门槛,而那属于上面的场景 2。 | | `BILLION_CONTEXT_PROXY` | launcher 会导出它;客户端侧 bili 插件/扩展检测到后自禁用自身压缩(避免双重压缩)。 | | `BILLION_CONTEXT_PLUGIN` | 设 `0` 彻底关闭插件模式(恢复 wire 层工具注入)。 | | `BILI_LAUNCHER_MODEL_WINDOWS` | 内部使用:launcher 把客户端自身配置里的逐模型上下文窗口(pi `models.json`、omp `models.yml`、opencode `models..limit`、codex `model_context_window`)以 JSON 传给自己拉起的代理,让 nudge 分母对自托管模型也用真实窗口。只有 launcher 会设置,无需用户配置。 |