diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 8909c081ad..b9771e18ef 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -214,6 +214,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `compress.tiers` | boolean | true | — | Tiered T1→T3 distillation spreads folding cost across generations. | | `compress.protectedTools` | string[] | none | — | Hard exclusion across all history: results of listed tools never fold. Path patterns (skill/) select skills by name (#1947). | | `compress.protectedLatestTools` | string[] | none | — | Protects only the LATEST instance of cumulative-snapshot tools whose newest result supersedes older ones (e.g. todo lists). Path patterns keep the latest instance per skill (skill/*, #1947). | +| `compress.protectedContentMarkers` | string[] | none | — | User-declared content-carrier substrings: any text message containing one survives every fold (#2446); proxy-mode complement to plugin-mode durable guards. | | `compress.neverPreserveRecentTools` | string[] ([] valid) | ["decompress", "search_context", "read", "bash"] (kernel) | — | Excluded from the recent protection zone (immediately compressible); an empty array excludes nothing (maximum protection). | | `compress.preserveRecentTools` | string[] | n/a (subtraction form) | — | Subtraction form: recent-zone tools minus this list get full protection; an empty array here is rejected as a typo. | | `compress.stripImages` | boolean | false | — | Strip image payloads from foldable history. | @@ -1235,6 +1236,13 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t - **Cumulative snapshots** — each newer result supersedes the older ones (a client's todo/task list): use `protectedLatestTools`. Protecting **all** instances of such a tool makes its history grow unboundedly — the exact failure #639 worked around by protecting only the latest. - Rule of thumb: low-frequency, high-value tools → `protectedTools`; chatty tools → never full-history protect (context grows without bound); cumulative-snapshot tools → `protectedLatestTools`. +#### `protectedContentMarkers` + +- **Type:** `string[]` (substrings) +- **Default:** unset → no content-marker protection +- **Status:** ACTIVE +- **Description:** Substrings that mark a TEXT message as a durable content carrier. Any text message **containing** any listed substring is protected from every fold — the same HARD exclusion as `protectedTools` (renders untagged with a `BLOCKED` ref, covered by neither suggested nor explicit compress ranges, both compression modes, every wire) — but keyed on message TEXT instead of tool name. This is the proxy-mode escape hatch for third-party hosts that inject standing instructions into user-role turns bili knows nothing about (#2446): instead of bili hard-coding a guard per host, the user declares the carrier signature once — e.g. dsh's system-reminder carriers `"Instructions from:"` or `` `` ``. Prefer the host-native mechanisms when available (plugin `protect` endpoint, #2556; `bili-durable` first-line markers, #2555): those are per-message and self-identifying, while substring-anywhere matching is a heuristic. **⚠ Over-pinning risk:** a user paste that happens to contain a marker is pinned forever (unbounded growth) — keep markers long and specific, and note that markers can NEVER be retroactively "unprotected" for messages already sent (changing the list does not resurrect folded context either; it only affects protection of messages seen afterwards). Empty array is rejected as a typo (as with `protectedTools`); non-string or blank entries reject loudly at config parse. Whole-array replace at the deepest defined level. Example: `{ "compress": { "protectedContentMarkers": ["Instructions from:", ""] } }`. + #### `neverPreserveRecentTools` - **Type:** `string[]` (tool-name patterns) diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 1e0a6752ca..251b5ff834 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -214,6 +214,7 @@ | `compress.tiers` | boolean | true | — | T1→T3 分级蒸馏把折叠成本摊到多代。 | | `compress.protectedTools` | string[] | none | — | 全历史硬排除:列出工具的结果永不折叠。路径模式(skill/)可按名指定 skill(#1947)。 | | `compress.protectedLatestTools` | string[] | none | — | 只保护累积快照类工具「最新一次」实例(新结果覆盖旧结果的工具,如 todo 列表)。路径模式按 skill 分组各保最新(skill/*,#1947)。 | +| `compress.protectedContentMarkers` | string[] | none | — | 用户自报的内容载体子串:含任一子串的文本消息在每次折叠中存活(#2446);插件模式硬编码守卫的 proxy 模式补位。 | | `compress.neverPreserveRecentTools` | string[] ([] valid) | ["decompress", "search_context", "read", "bash"] (kernel) | — | 从近期保护区排除(立即可压);空数组合法=不排除任何工具(最大保护)。 | | `compress.preserveRecentTools` | string[] | n/a (subtraction form) | — | 减法形式:近期区工具减去本列表得到完全保护;此处空数组按笔误拒绝。 | | `compress.stripImages` | boolean | false | — | 从可折叠历史中剥离图片载荷。 | @@ -1238,6 +1239,13 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 - **累积快照** —— 每条新结果取代旧结果(客户端的 todo/任务清单):用 `protectedLatestTools`。对这类工具保护**全部**实例会让其历史无限膨胀 —— 正是 #639 通过只保护最新一条来规避的故障。 - 经验法则:低频高价值工具 → `protectedTools`;高频刷屏工具 → 绝不做全历史保护(上下文无界增长);累积快照型工具 → `protectedLatestTools`。 +#### `protectedContentMarkers` + +- **类型:** `string[]`(子串) +- **默认值:** 未设置 → 无内容标记保护 +- **状态:** ACTIVE +- **说明:** 标记「持久内容载体」的子串列表:任何**包含**任一子串的 text 消息在每次折叠中受保护 —— 与 `protectedTools` 同为硬排除(渲染为不带 tag 的 `BLOCKED` ref,推荐范围与显式范围都无法覆盖;两种压缩模式、所有 wire 一致生效),但匹配键是消息**文本**而非工具名。这是为「第三方 host 往 user 轮次里注入常驻指令、bili 无从知晓」的 proxy 模式场景准备的逃生门(#2446):不再由 bili 为每个 host 硬编码守卫,用户一次性声明载体签名 —— 例如 dsh 的 system-reminder 载体 `"Instructions from:"` 或 `` `` ``。能用 host 原生机制时优先用原生(插件 `protect` 端点 #2556;`bili-durable` 首行标记 #2555):它们按消息自识别,而子串任意位置匹配只是启发式。**⚠ 过度钉住风险:** 用户粘贴的内容若碰巧含标记子串也会被永久钉住(上下文无界增长)—— 标记要长且具体;且标记不能对已发消息追溯"解除保护"(改列表也不会复活已折叠的上下文,只影响之后的消息)。空数组按笔误拒绝(同 `protectedTools`);非字符串/空白项在配置解析时大声拒绝。跨层级整体替换(最深层胜出)。示例:`{ "compress": { "protectedContentMarkers": ["Instructions from:", ""] } }`。 + #### `neverPreserveRecentTools` - **类型:** `string[]`(工具名模式) diff --git a/src/compress-settings.ts b/src/compress-settings.ts index 70c4031262..68fc29db1a 100644 --- a/src/compress-settings.ts +++ b/src/compress-settings.ts @@ -87,6 +87,7 @@ export function mergeCompress( tiers: pick("tiers"), protectedLatestTools: pick("protectedLatestTools"), protectedTools: pick("protectedTools"), + protectedContentMarkers: pick("protectedContentMarkers"), neverPreserveRecentTools: pick("neverPreserveRecentTools"), preserveRecentTools: pick("preserveRecentTools"), prompts: promptLevels.length > 0 ? Object.assign({}, ...promptLevels) : undefined, diff --git a/src/config.ts b/src/config.ts index 204afb14ba..ecc979975e 100644 --- a/src/config.ts +++ b/src/config.ts @@ -270,6 +270,21 @@ export type CompressSettings = { * Default: none — opt in per client/agent, since tool names are * client-specific. */ protectedTools?: string[]; + /** Text markers (plain substrings) that pin ANY text message containing + * one from folding — the user-config lane of the durable-message + * protocol (#2446 方案A). Works in proxy mode and plugin mode alike: + * matched messages join the same `isMessageProtected` guard chain as + * the built-in lane table, the `\x3cbili-durable\x3e` first-line marker + * (#2555) and `POST /__bili/plugin/protect` registrations (#2556). + * Built for hosts bili has no lane entry for (proxy mode carries no + * client identity at all): the user declares the carrier text himself + * (e.g. `["Instructions from:", "\x3cmcp_catalog\x3e"]` for dsh's + * core carriers in proxy mode). ⚠ Markers match as raw substrings + * anywhere in the message text — a pasted message containing the marker + * pins too, so use distinctive strings, not prose fragments. Deepest + * level wins (global → provider → model), whole-array replace. Default: + * none. */ + protectedContentMarkers?: string[]; /** Tool-name patterns EXCLUDED from the soft-protected recent zone — * matching tool results inside the recent zone become compressible * immediately instead of aging out first (kernel @@ -1795,7 +1810,7 @@ const COMPRESS_SETTING_FIELDS = new Set([ "nudgeGrowthTokens", "tierNudgeTokens", "nudgeModelDecided", "nudgeDecisionMaxTokens", "preserveRecentMessages", "preserveRecentTokens", "minCompressRange", "minCompressRangeChars", "stripImagesKeepRecent", - "outputHeadroomMaxPct", "tiers", "protectedLatestTools", "protectedTools", + "outputHeadroomMaxPct", "tiers", "protectedLatestTools", "protectedTools", "protectedContentMarkers", "neverPreserveRecentTools", "preserveRecentTools", "stripImages", "visibilityMarkers", "rules", "injectTool", "injectNudge", "acknowledgePromptsRisk", "absorb", "ccr", "search", "imageCompression", @@ -2252,7 +2267,7 @@ export function parseCompressSettings(v: unknown): (CompressSettings & { injectT if (typeof obj.nudgeModelDecided !== "boolean") ok = false; else out.nudgeModelDecided = obj.nudgeModelDecided; } - for (const key of ["protectedLatestTools", "protectedTools"] as const) { + for (const key of ["protectedLatestTools", "protectedTools", "protectedContentMarkers"] as const) { if (!(key in obj) || obj[key] === undefined) continue; const v = obj[key]; if (!Array.isArray(v) || v.length === 0 || v.some((x) => typeof x !== "string" || x.trim().length === 0)) ok = false; diff --git a/src/durable-message-guards.ts b/src/durable-message-guards.ts index c0c0ef4692..e0d779093b 100644 --- a/src/durable-message-guards.ts +++ b/src/durable-message-guards.ts @@ -156,3 +156,19 @@ export const durableMessageGuards: Record = { dsh: (msg) => dshSkillCatalogGuard(msg) || dshWorkspaceInstructionsGuard(msg) || dshRuntimeContextGuard(msg) || dshMcpCatalogGuard(msg), }; + +/** #2446 方案A: user-configured content markers — the config lane of the + * durable protocol. Unlike everything above this is per-OPERATOR policy, + * not per-lane evidence: the user declares carrier substrings for hosts bili + * has no lane entry for (proxy mode has no client identity at all). The + * factory returns undefined for an empty/absent list so the guard chain + * never grows a dead predicate. Matching is raw substring-anywhere over + * text messages — deliberately the same granularity the built-in lane + * guards use (e.g. "Instructions from:"), so a config marker can reproduce + * any lane entry one-to-one (the documented escape hatch while a host waits + * for lane evidence). */ +export function protectedContentMarkerGuard(markers: readonly string[]): ((msg: CoreMessage) => boolean) | undefined { + const list = markers.filter((m) => m.length > 0); + if (list.length === 0) return undefined; + return (msg) => msg.contentType === "text" && typeof msg.text === "string" && list.some((m) => msg.text!.includes(m)); +} diff --git a/src/server/handle.ts b/src/server/handle.ts index 4749e4a105..231b4cae6c 100644 --- a/src/server/handle.ts +++ b/src/server/handle.ts @@ -14,7 +14,7 @@ import { codexAlignedWindow } from "../codex-models.js"; import { MAX_REQUEST_BYTES } from "../fetch-util.js"; import { hostIdForLog, maskHeadersForLog, maskUrlForLog, maskUrlsInText } from "../log-mask.js"; import { buildIncomingImageIndex, foldAnchoredCutoff, pruneRetrieveImgExports } from "../image-restore.js"; -import { biliDurableMarkerGuard, durableMessageGuards } from "../durable-message-guards.js"; +import { biliDurableMarkerGuard, durableMessageGuards, protectedContentMarkerGuard } from "../durable-message-guards.js"; import { biliToolsDeclaredOnWire, countBiliToolUses, evaluateSelfHealRound, nudgeSuppressed, pluginLaneDegraded, pluginLaneRestore } from "../session-self-heal.js"; import { compressBreakerArmed } from "../stream.js"; import { acquireInFlight, getSession, hasProcessedState, markDirty, peekSession, registeredProtectedRawIds, releaseInFlight, storeEffectiveConfig, tickPostRebuildAnchor, withSessionLock, type Session } from "../session.js"; @@ -1746,8 +1746,20 @@ export async function handle( // resolution as session.ts effectiveConfig, so a registration via POST // /__bili/plugin/protect protects the very next fold on the wire path. const registered = registeredProtectedRawIds(session); + // #2446 方案A: fold in operator-declared content markers, resolved + // through the standard three-level compress cascade for THIS request + // and stamped onto the session (clone-safe strings) so the read path + // (effectiveConfig) sees the same list without request context. + const cfgMarkers = resolveCompress(opts.routes, route?.rewrittenUrl, requestModel, opts.compress).protectedContentMarkers; + if (cfgMarkers && cfgMarkers.length > 0) session.metadata["protectedContentMarkers"] = cfgMarkers; + else delete session.metadata["protectedContentMarkers"]; + const cfgGuard = protectedContentMarkerGuard(cfgMarkers ?? []); const durableGuard: (msg: CoreMessage) => boolean = - laneGuard ? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id) : (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id); + cfgGuard + ? (msg) => (laneGuard ? laneGuard(msg) : false) || biliDurableMarkerGuard(msg) || registered.has(msg.id) || cfgGuard(msg) + : laneGuard + ? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id) + : (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id); reqConfig = { ...reqConfig, isMessageProtected: durableGuard }; // acquireInFlight must precede the lock so evictOldest() cannot flush // this session between getSession and lock acquisition (inFlight===0 diff --git a/src/session.ts b/src/session.ts index 8d890e2edf..e30842f4b0 100644 --- a/src/session.ts +++ b/src/session.ts @@ -3,7 +3,7 @@ import { createHash } from "node:crypto"; import { log as loggerLog } from "./logger.js"; import { getStore } from "./persist.js"; import { maxSessions as knobMaxSessions } from "./knobs.js"; -import { biliDurableMarkerGuard, durableMessageGuards } from "./durable-message-guards.js"; +import { biliDurableMarkerGuard, durableMessageGuards, protectedContentMarkerGuard } from "./durable-message-guards.js"; import type { WireProtocol } from "./util.js"; export type BlockView = { text: string; count: number }; @@ -442,11 +442,31 @@ export function effectiveConfig(session: Session | undefined, fallback: Config): // session-keyed raw ids, persisted via metadata; composes with the marker // (#2555) and lane guards. Same read-time re-resolution as the guards. const registered = registeredProtectedRawIds(session); + // #2446 方案A: operator-declared content markers — stamped onto the + // session by the wire path from the request-resolved compress settings + // (clone-safe string array, same metadata discipline as the registration + // set), so this request-context-free reader sees the newest config. + const cfgGuard = protectedContentMarkerGuard(protectedContentMarkersOf(session)); const guard: (msg: CoreMessage) => boolean = - laneGuard ? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id) : (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id); + cfgGuard + ? (msg) => (laneGuard ? laneGuard(msg) : false) || biliDurableMarkerGuard(msg) || registered.has(msg.id) || cfgGuard(msg) + : laneGuard + ? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id) + : (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id); return !base.isMessageProtected ? { ...base, isMessageProtected: guard } : base; } +/** #2446 方案A: the operator-declared marker list as read from session + * metadata. The wire path (server/handle.ts) stamps it per request from the + * three-level-resolved compress settings, so plugin-tool / status readers — + * which have no request context — resolve the same protection the wire used. + * Empty array removes the key (config dropped the markers → stop pinning; + * the session reflects current config, not historical config). */ +export function protectedContentMarkersOf(session: Session | undefined): string[] { + const raw = session?.metadata["protectedContentMarkers"]; + return Array.isArray(raw) ? raw.filter((v): v is string => typeof v === "string" && v.length > 0) : []; +} + /** #2556: the protect-tool registration as a membership set. Reading it from * session.metadata at guard-resolution time (never cached across calls) keeps * POST /__bili/plugin/protect effective for the next fold without any diff --git a/tests/issue2446-config-markers.test.ts b/tests/issue2446-config-markers.test.ts new file mode 100644 index 0000000000..fb455c73f2 --- /dev/null +++ b/tests/issue2446-config-markers.test.ts @@ -0,0 +1,279 @@ +// #2446 方案A: compress.protectedContentMarkers — the user-config lane of +// the durable-message protocol. The built-in lane table (dsh) is evidence- +// gated and proxy mode carries no client identity at all; this knob lets the +// OPERATOR declare carrier substrings himself (e.g. dsh's "Instructions +// from:" through proxy mode) pending lane evidence or host-side marker +// adoption. This suite pins: config validation (loud rejects, trims), the +// fold-time effect in a lane-less session (a marked message survives a +// positional fold that swallows its neighbors, an unmarked one does not), +// composition with the #2555 first-line marker, and that an absent/removed +// marker list stops pinning (config reflects current policy, not history). +import test from "node:test"; +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import { defaultConfig } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import { parseCompressSettings } from "../src/config.ts"; +import type { ProxyOptions } from "../src/config.ts"; +import { resetPersonaAnchorsForTest } from "../src/persona-anchor.ts"; +import { _resetPluginStateForTest } from "../src/plugin.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { _resetSessionsForTest } from "../src/session.ts"; + +process.env.NODE_ENV = "test"; +process.env.BILI_PERSIST = "0"; + +const MODEL = "gpt-test"; +const MAIN_SYSTEM = "You are the main coding agent."; +const CARRIER = "Instructions from: never force-push master, never merge PRs"; +const MARKERS = ["Instructions from:"]; +const DURABLE_MARKED = "\x3cbili-durable kind=\"memory\"\x3e\nmarked channel: never force-push master"; + +function filler(tag: string): string { + return `${tag}-` + "z".repeat(900); +} + +type Rig = { proxyPort: number; upstreamPort: number; proxy: http.Server; upstream: http.Server; bodies: string[] }; + +async function startRig(markers: string[] | undefined): Promise { + const bodies: string[] = []; + const upstream = http.createServer((req, res) => { + let b = ""; + req.on("data", (c: Buffer) => (b += c.toString("utf8"))); + req.on("end", () => { + bodies.push(b); + res.writeHead(200, { "content-type": "application/json" }); + res.end(JSON.stringify({ + id: "chatcmpl-1", + object: "chat.completion", + choices: [{ index: 0, message: { role: "assistant", content: "ok" }, finish_reason: "stop" }], + usage: { prompt_tokens: 10, completion_tokens: 3, total_tokens: 13 }, + })); + }); + }); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as { port: number }).port; + + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + _resetSessionsForTest(); + _resetPluginStateForTest(); + resetPersonaAnchorsForTest(); + const proxy = await startServer({ + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: { [`http://127.0.0.1:${upstreamPort}`]: {} }, + modelContextLimit: 200_000, + kernelConfig: defaultConfig(200_000, { + preserveRecentMessages: 3, + preserveRecentTokens: 0, + compress: { minCompressRange: 500, maxSummaryLength: 20_000, minSummaryLength: 50 }, + }), + compress: markers === undefined ? { injectTool: true, injectNudge: false } : { injectTool: true, injectNudge: false, protectedContentMarkers: markers }, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + autoUpdate: false, + compat: { roles: {} }, + streamErrorShape: "protocol", + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + advisoryCheck: false, + releaseNotesCheck: false, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions); + await once(proxy, "listening"); + return { proxyPort: (proxy.address() as { port: number }).port, upstreamPort, proxy, upstream, bodies }; +} + +async function closeRig(rig: Rig): Promise { + rig.proxy.close(); + await once(rig.proxy, "close"); + rig.upstream.close(); + await once(rig.upstream, "close"); +} + +/** Refs from NON-SYSTEM message content only (system prompt carries literal + * ACP-tag examples that a whole-body scan would mistake for live refs). */ +function parseRefIds(body: string): string[] { + let messages: Array<{ role?: string; content?: unknown }> = []; + try { + messages = (JSON.parse(body) as { messages?: Array<{ role: string; content?: unknown }> }).messages ?? []; + } catch { + return []; + } + const text = messages + .filter((m) => m.role !== "system") + .map((m) => (typeof m.content === "string" ? m.content : JSON.stringify(m.content ?? ""))) + .join("\n"); + const ids: string[] = []; + const re = /<(?:acp|dcp-message-id)[^>]*>\s*(m\d+)\s*<\/(?:acp|dcp-message-id)>/g; + let m: RegExpExecArray | null; + while ((m = re.exec(text)) !== null) ids.push(m[1]!); + return ids; +} + +async function compressCall(port: number, conv: string, startId: string, endId: string): Promise<{ status: number; json: Record }> { + const r = await fetch(`http://127.0.0.1:${port}/__bili/plugin/tool`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + conversationId: conv, + tool: "compress", + args: { + topic: "i2446c fold", + content: [{ startId, endId, topic: "i2446c fold", summary: "i2446c fold summary: filler turns were discussed and consumed; nothing else survived this range." }], + }, + }), + }); + return { status: r.status, json: (await r.json()) as Record }; +} + +async function grow(rig: Rig, conv: string, texts: string[]): Promise { + const url = `http://127.0.0.1:${rig.proxyPort}/bili/http://127.0.0.1:${rig.upstreamPort}/v1/chat/completions`; + const headers = { "content-type": "application/json", "x-acp-session": conv }; + const msgs: Array<{ role: string; content: string }> = []; + for (const [i, t] of texts.entries()) { + msgs.push({ role: i % 2 === 0 ? "user" : "assistant", content: t }); + const r = await fetch(url, { method: "POST", headers, body: JSON.stringify({ model: MODEL, max_tokens: 64_000, messages: [{ role: "system", content: MAIN_SYSTEM }, ...msgs] }) }); + assert.equal(r.status, 200); + await r.text(); + } +} + +async function post(rig: Rig, conv: string, history: Array<{ role: string; content: string }>): Promise { + const url = `http://127.0.0.1:${rig.proxyPort}/bili/http://127.0.0.1:${rig.upstreamPort}/v1/chat/completions`; + const r = await fetch(url, { method: "POST", headers: { "content-type": "application/json", "x-acp-session": conv }, body: JSON.stringify({ model: MODEL, max_tokens: 64_000, messages: [{ role: "system", content: MAIN_SYSTEM }, ...history] }) }); + assert.equal(r.status, 200); + await r.text(); + return rig.bodies.at(-1)!; +} + +test("config validation: accepted shape, trims, loud rejects", () => { + const ok = parseCompressSettings({ protectedContentMarkers: [" Instructions from: ", "\x3cmcp_catalog\x3e"] }); + assert.deepEqual(ok?.protectedContentMarkers, ["Instructions from:", "\x3cmcp_catalog\x3e"], "valid markers parse and trim"); + assert.equal(parseCompressSettings({ protectedContentMarkers: [] }), undefined, "empty array rejects (a bare [] is almost certainly a typo, and a no-op list pins nothing)"); + assert.equal(parseCompressSettings({ protectedContentMarkers: "Instructions from:" }), undefined, "non-array rejects"); + assert.equal(parseCompressSettings({ protectedContentMarkers: ["ok", 42] }), undefined, "non-string entries reject"); + assert.equal(parseCompressSettings({ protectedContentMarkers: ["ok", " "] }), undefined, "blank-string entries reject"); +}); + +test("lane-less session: a configured carrier survives a positional fold; unmarked content does not", async () => { + const rig = await startRig(MARKERS); + try { + const conv = "i2446c-fold"; + await grow(rig, conv, [ + filler("F1"), filler("A1"), + CARRIER, + filler("F3"), filler("A3"), filler("F4"), filler("A4"), filler("F5"), filler("A5"), + ]); + const refIds = parseRefIds(rig.bodies.at(-1)!); + assert.ok(refIds.length >= 8, `expected >=8 refs, got ${refIds.length}`); + // The protected carrier renders UNTAGGED (kernel assignRefs: protected + // messages get BLOCKED_REF — never advertised, never folded), so the + // tagged ref stream skips it: 8 refs for 9 messages is the pin itself. + const fold = await compressCall(rig.proxyPort, conv, refIds[0]!, refIds[refIds.length - 4]!); + assert.equal(fold.status, 200); + assert.ok(fold.status === 200 && fold.json.ok === true, `compress must succeed: ${JSON.stringify(fold.json)}`); + const hist = histOf(conv); + const body = await post(rig, conv, [...hist, { role: "user", content: filler("F6") }]); + assert.ok(body.includes("[Compressed conversation section]"), "the fold happened"); + assert.ok(body.includes("never force-push master"), "the configured carrier survives the fold"); + assert.ok(!body.includes("never force-push master, never merge PRs\u003c/acp"), "carrier is UNTAGGED prose (BLOCKED_REF: never advertised, never citable-by-ref)"); + assert.ok(!body.includes("F3-"), "post-carrier neighbors fold — the fold is real, not vetoed wholesale"); + } finally { + await closeRig(rig); + } +}); + +test("composition: config marker and #2555 first-line marker co-protect in one session", async () => { + const rig = await startRig(MARKERS); + try { + const conv = "i2446c-comp"; + const durableMarked = "\x3cbili-durable kind=\"memory\"\x3e\nmarked channel: never force-push master"; + await grow(rig, conv, [ + filler("F1"), filler("A1"), + CARRIER, + durableMarked, + filler("F3"), filler("A3"), filler("F4"), filler("A4"), + ]); + const refIds = parseRefIds(rig.bodies.at(-1)!); + const fold = await compressCall(rig.proxyPort, conv, refIds[0]!, refIds[refIds.length - 4]!); + assert.equal(fold.status, 200); + assert.ok(fold.status === 200 && fold.json.ok === true, `compress must succeed: ${JSON.stringify(fold.json)}`); + const body = await post(rig, conv, [...histOf(conv), { role: "user", content: filler("F5") }]); + assert.ok(body.includes("never force-push master"), "config-marker channel survives"); + assert.ok(body.includes("marked channel"), "#2555 marker channel survives"); + // "never merge PRs" is CARRIER-unique (the line above matches both carriers): + // pins the CONFIG side independently so a config-guard regression under a + // co-present #2555 marker is caught, not masked by the durable channel's survival. + assert.ok(body.includes("never merge PRs"), "config-marker carrier survives its own token, not just the shared substring"); + assert.ok(body.includes("[Compressed conversation section]"), "the fold happened"); + assert.ok(!body.includes("A1-"), "the pre-protected segment folds — the fold is real"); + } finally { + await closeRig(rig); + } +}); + +test("absent markers: identical content folds — the knob is the only trigger", async () => { + const rig = await startRig(undefined); + try { + const conv = "i2446c-absent"; + await grow(rig, conv, [filler("F1"), filler("A1"), CARRIER, filler("F3"), filler("A3"), filler("F4"), filler("A4")]); + const refIds = parseRefIds(rig.bodies.at(-1)!); + const fold = await compressCall(rig.proxyPort, conv, refIds[0]!, refIds[refIds.length - 4]!); + assert.equal(fold.status, 200); + assert.ok(fold.status === 200 && fold.json.ok === true, `compress must succeed: ${JSON.stringify(fold.json)}`); + const body = await post(rig, conv, [...histOf(conv), { role: "user", content: filler("F5") }]); + assert.ok(!body.includes("never force-push master"), "without the config the SAME carrier text folds — protection comes from the knob, not the text"); + assert.ok(body.includes("[Compressed conversation section]"), "the fold happened"); + } finally { + await closeRig(rig); + } +}); + +/** Raw client-side history per conversation — re-sent verbatim (NOT the + * upstream-bound body, whose injected tags would switch the proxy into + * ref-placeholder rendering and hide inline text from the assertions). */ +function histOf(conv: string): Array<{ role: string; content: string }> { + const carriers: Record> = { + "i2446c-fold": [ + { role: "user", content: filler("F1") }, + { role: "assistant", content: filler("A1") }, + { role: "user", content: CARRIER }, + { role: "assistant", content: filler("F3") }, + { role: "user", content: filler("A3") }, + { role: "assistant", content: filler("F4") }, + { role: "user", content: filler("A4") }, + { role: "assistant", content: filler("F5") }, + { role: "user", content: filler("A5") }, + ], + "i2446c-comp": [ + { role: "user", content: filler("F1") }, + { role: "assistant", content: filler("A1") }, + { role: "user", content: CARRIER }, + { role: "assistant", content: DURABLE_MARKED }, + { role: "user", content: filler("F3") }, + { role: "assistant", content: filler("A3") }, + { role: "user", content: filler("F4") }, + { role: "assistant", content: filler("A4") }, + ], + "i2446c-absent": [ + { role: "user", content: filler("F1") }, + { role: "assistant", content: filler("A1") }, + { role: "user", content: CARRIER }, + { role: "assistant", content: filler("F3") }, + { role: "user", content: filler("A3") }, + { role: "assistant", content: filler("F4") }, + { role: "user", content: filler("A4") }, + ], + }; + return carriers[conv]!; +} diff --git a/website/config-reference/compress.yaml b/website/config-reference/compress.yaml index d4ae1e9383..e8aea28e89 100644 --- a/website/config-reference/compress.yaml +++ b/website/config-reference/compress.yaml @@ -183,6 +183,16 @@ entries: desc_zh: "只保护累积快照类工具「最新一次」实例(新结果覆盖旧结果的工具,如 todo 列表)。路径模式按 skill 分组各保最新(skill/*,#1947)。" site_en: "Protects only the LATEST instance of cumulative-snapshot tools whose newest result supersedes older ones (e.g. todo lists)." site_zh: "只保护累积快照类工具的「最新一次」实例(新结果覆盖旧结果的工具,如 todo 列表)。" + - key: compress.protectedContentMarkers + type: "string[]" + default: "none" + group: compress + id: ct.pcm + render: compress-row + desc_en: "User-declared content-carrier substrings: any text message containing one survives every fold (#2446); proxy-mode complement to plugin-mode durable guards." + desc_zh: "用户自报的内容载体子串:含任一子串的文本消息在每次折叠中存活(#2446);插件模式硬编码守卫的 proxy 模式补位。" + site_en: "User-declared content-carrier substrings: any text message containing one is hard-excluded from every fold (untagged BLOCKED ref, both compression modes). Proxy-mode complement to the host-native durable guards (#2555/#2556); prefer those when available. Empty array rejects as a typo." + site_zh: "用户自报的内容载体子串:含任一子串的文本消息被硬排除于每次折叠(不带 tag 的 BLOCKED ref,两种压缩模式一致)。是 host 原生持久守卫(#2555/#2556)的 proxy 模式补位,能用原生时优先用原生。空数组按笔误拒绝。" - key: compress.neverPreserveRecentTools type: "string[] ([] valid)" default: "[\"decompress\", \"search_context\", \"read\", \"bash\"] (kernel)" diff --git a/website/docs/index.html b/website/docs/index.html index c3f1408631..a764874d4b 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -196,6 +196,7 @@

compress — every knob

tiersbooleanOn by default: tiered T1→T3 distillation spreads folding cost across generations. protectedToolsstring[]Hard exclusion across all history: results of listed tools never fold (use with care). protectedLatestToolsstring[]Protects only the LATEST instance of cumulative-snapshot tools whose newest result supersedes older ones (e.g. todo lists). + protectedContentMarkersstring[]User-declared content-carrier substrings: any text message containing one is hard-excluded from every fold (untagged BLOCKED ref, both compression modes). Proxy-mode complement to the host-native durable guards (#2555/#2556); prefer those when available. Empty array rejects as a typo. neverPreserveRecentToolsstring[] ([] valid)Excluded from the recent protection zone (immediately compressible); default ["decompress", "search_context", "read", "bash"]. An empty array is valid — it excludes nothing (maximum protection). preserveRecentToolsstring[]Subtraction form: recent-zone tools minus this list get full protection. An empty array here is rejected as a typo. stripImagesbooleanStrip image payloads from foldable history (off by default). diff --git a/website/main.js b/website/main.js index 1b1d77ed13..8630eca2be 100644 --- a/website/main.js +++ b/website/main.js @@ -79,6 +79,7 @@ "cw.tiers": "On by default: tiered T1→T3 distillation spreads folding cost across generations.", "ct.pt": "Hard exclusion across all history: results of listed tools never fold (use with care).", "ct.plt": "Protects only the LATEST instance of cumulative-snapshot tools whose newest result supersedes older ones (e.g. todo lists).", + "ct.pcm": "User-declared content-carrier substrings: any text message containing one is hard-excluded from every fold (untagged BLOCKED ref, both compression modes). Proxy-mode complement to the host-native durable guards (#2555/#2556); prefer those when available. Empty array rejects as a typo.", "ct.nprt": "Excluded from the recent protection zone (immediately compressible); default [\"decompress\", \"search_context\", \"read\", \"bash\"]. An empty array is valid — it excludes nothing (maximum protection).", "ct.prt2": "Subtraction form: recent-zone tools minus this list get full protection. An empty array here is rejected as a typo.", "ct.si": "Strip image payloads from foldable history (off by default).", @@ -315,6 +316,7 @@ "cw.tiers": "默认开:T1→T3 分级蒸馏,把折叠成本摊到多代。", "ct.pt": "全历史硬排除:列出工具的结果永不折叠(慎用)。", "ct.plt": "只保护累积快照类工具的「最新一次」实例(新结果覆盖旧结果的工具,如 todo 列表)。", + "ct.pcm": "用户自报的内容载体子串:含任一子串的文本消息被硬排除于每次折叠(不带 tag 的 BLOCKED ref,两种压缩模式一致)。是 host 原生持久守卫(#2555/#2556)的 proxy 模式补位,能用原生时优先用原生。空数组按笔误拒绝。", "ct.nprt": "从近期保护区排除(立即可压);默认 [\"decompress\", \"search_context\", \"read\", \"bash\"]。空数组合法=不排除任何工具(最大保护)。", "ct.prt2": "减法形式:近期区工具减去本列表得到完全保护;此处空数组按笔误拒绝。", "ct.si": "从可折叠历史中剥离图片载荷(默认关)。",