Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
| `compress.tiers` | boolean | true | — | Tiered T1→T3 distillation spreads folding cost across generations. |
| `compress.protectedTools` | string[] | none | — | Hard exclusion across all history: results of listed tools never fold. Path patterns (skill/<name>) select skills by name (#1947). |
| `compress.protectedLatestTools` | string[] | none | — | Protects only the LATEST instance of cumulative-snapshot tools whose newest result supersedes older ones (e.g. todo lists). Path patterns keep the latest instance per skill (skill/*, #1947). |
| `compress.protectedContentMarkers` | string[] | none | — | User-declared content-carrier substrings: any text message containing one survives every fold (#2446); proxy-mode complement to plugin-mode durable guards. |
| `compress.neverPreserveRecentTools` | string[] ([] valid) | ["decompress", "search_context", "read", "bash"] (kernel) | — | Excluded from the recent protection zone (immediately compressible); an empty array excludes nothing (maximum protection). |
| `compress.preserveRecentTools` | string[] | n/a (subtraction form) | — | Subtraction form: recent-zone tools minus this list get full protection; an empty array here is rejected as a typo. |
| `compress.stripImages` | boolean | false | — | Strip image payloads from foldable history. |
Expand Down Expand Up @@ -1235,6 +1236,13 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t
- **Cumulative snapshots** — each newer result supersedes the older ones (a client's todo/task list): use `protectedLatestTools`. Protecting **all** instances of such a tool makes its history grow unboundedly — the exact failure #639 worked around by protecting only the latest.
- Rule of thumb: low-frequency, high-value tools → `protectedTools`; chatty tools → never full-history protect (context grows without bound); cumulative-snapshot tools → `protectedLatestTools`.

#### `protectedContentMarkers`

- **Type:** `string[]` (substrings)
- **Default:** unset → no content-marker protection
- **Status:** ACTIVE
- **Description:** Substrings that mark a TEXT message as a durable content carrier. Any text message **containing** any listed substring is protected from every fold — the same HARD exclusion as `protectedTools` (renders untagged with a `BLOCKED` ref, covered by neither suggested nor explicit compress ranges, both compression modes, every wire) — but keyed on message TEXT instead of tool name. This is the proxy-mode escape hatch for third-party hosts that inject standing instructions into user-role turns bili knows nothing about (#2446): instead of bili hard-coding a guard per host, the user declares the carrier signature once — e.g. dsh's system-reminder carriers `"Instructions from:"` or `` `<mcp_catalog>` ``. Prefer the host-native mechanisms when available (plugin `protect` endpoint, #2556; `bili-durable` first-line markers, #2555): those are per-message and self-identifying, while substring-anywhere matching is a heuristic. **⚠ Over-pinning risk:** a user paste that happens to contain a marker is pinned forever (unbounded growth) — keep markers long and specific, and note that markers can NEVER be retroactively "unprotected" for messages already sent (changing the list does not resurrect folded context either; it only affects protection of messages seen afterwards). Empty array is rejected as a typo (as with `protectedTools`); non-string or blank entries reject loudly at config parse. Whole-array replace at the deepest defined level. Example: `{ "compress": { "protectedContentMarkers": ["Instructions from:", "<mcp_catalog>"] } }`.

#### `neverPreserveRecentTools`

- **Type:** `string[]` (tool-name patterns)
Expand Down
8 changes: 8 additions & 0 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@
| `compress.tiers` | boolean | true | — | T1→T3 分级蒸馏把折叠成本摊到多代。 |
| `compress.protectedTools` | string[] | none | — | 全历史硬排除:列出工具的结果永不折叠。路径模式(skill/<name>)可按名指定 skill(#1947)。 |
| `compress.protectedLatestTools` | string[] | none | — | 只保护累积快照类工具「最新一次」实例(新结果覆盖旧结果的工具,如 todo 列表)。路径模式按 skill 分组各保最新(skill/*,#1947)。 |
| `compress.protectedContentMarkers` | string[] | none | — | 用户自报的内容载体子串:含任一子串的文本消息在每次折叠中存活(#2446);插件模式硬编码守卫的 proxy 模式补位。 |
| `compress.neverPreserveRecentTools` | string[] ([] valid) | ["decompress", "search_context", "read", "bash"] (kernel) | — | 从近期保护区排除(立即可压);空数组合法=不排除任何工具(最大保护)。 |
| `compress.preserveRecentTools` | string[] | n/a (subtraction form) | — | 减法形式:近期区工具减去本列表得到完全保护;此处空数组按笔误拒绝。 |
| `compress.stripImages` | boolean | false | — | 从可折叠历史中剥离图片载荷。 |
Expand Down Expand Up @@ -1238,6 +1239,13 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报
- **累积快照** —— 每条新结果取代旧结果(客户端的 todo/任务清单):用 `protectedLatestTools`。对这类工具保护**全部**实例会让其历史无限膨胀 —— 正是 #639 通过只保护最新一条来规避的故障。
- 经验法则:低频高价值工具 → `protectedTools`;高频刷屏工具 → 绝不做全历史保护(上下文无界增长);累积快照型工具 → `protectedLatestTools`。

#### `protectedContentMarkers`

- **类型:** `string[]`(子串)
- **默认值:** 未设置 → 无内容标记保护
- **状态:** ACTIVE
- **说明:** 标记「持久内容载体」的子串列表:任何**包含**任一子串的 text 消息在每次折叠中受保护 —— 与 `protectedTools` 同为硬排除(渲染为不带 tag 的 `BLOCKED` ref,推荐范围与显式范围都无法覆盖;两种压缩模式、所有 wire 一致生效),但匹配键是消息**文本**而非工具名。这是为「第三方 host 往 user 轮次里注入常驻指令、bili 无从知晓」的 proxy 模式场景准备的逃生门(#2446):不再由 bili 为每个 host 硬编码守卫,用户一次性声明载体签名 —— 例如 dsh 的 system-reminder 载体 `"Instructions from:"` 或 `` `<mcp_catalog>` ``。能用 host 原生机制时优先用原生(插件 `protect` 端点 #2556;`bili-durable` 首行标记 #2555):它们按消息自识别,而子串任意位置匹配只是启发式。**⚠ 过度钉住风险:** 用户粘贴的内容若碰巧含标记子串也会被永久钉住(上下文无界增长)—— 标记要长且具体;且标记不能对已发消息追溯"解除保护"(改列表也不会复活已折叠的上下文,只影响之后的消息)。空数组按笔误拒绝(同 `protectedTools`);非字符串/空白项在配置解析时大声拒绝。跨层级整体替换(最深层胜出)。示例:`{ "compress": { "protectedContentMarkers": ["Instructions from:", "<mcp_catalog>"] } }`。

#### `neverPreserveRecentTools`

- **类型:** `string[]`(工具名模式)
Expand Down
1 change: 1 addition & 0 deletions src/compress-settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ export function mergeCompress(
tiers: pick("tiers"),
protectedLatestTools: pick("protectedLatestTools"),
protectedTools: pick("protectedTools"),
protectedContentMarkers: pick("protectedContentMarkers"),
neverPreserveRecentTools: pick("neverPreserveRecentTools"),
preserveRecentTools: pick("preserveRecentTools"),
prompts: promptLevels.length > 0 ? Object.assign({}, ...promptLevels) : undefined,
Expand Down
19 changes: 17 additions & 2 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,21 @@ export type CompressSettings = {
* Default: none — opt in per client/agent, since tool names are
* client-specific. */
protectedTools?: string[];
/** Text markers (plain substrings) that pin ANY text message containing
* one from folding — the user-config lane of the durable-message
* protocol (#2446 方案A). Works in proxy mode and plugin mode alike:
* matched messages join the same `isMessageProtected` guard chain as
* the built-in lane table, the `\x3cbili-durable\x3e` first-line marker
* (#2555) and `POST /__bili/plugin/protect` registrations (#2556).
* Built for hosts bili has no lane entry for (proxy mode carries no
* client identity at all): the user declares the carrier text himself
* (e.g. `["Instructions from:", "\x3cmcp_catalog\x3e"]` for dsh's
* core carriers in proxy mode). ⚠ Markers match as raw substrings
* anywhere in the message text — a pasted message containing the marker
* pins too, so use distinctive strings, not prose fragments. Deepest
* level wins (global → provider → model), whole-array replace. Default:
* none. */
protectedContentMarkers?: string[];
/** Tool-name patterns EXCLUDED from the soft-protected recent zone —
* matching tool results inside the recent zone become compressible
* immediately instead of aging out first (kernel
Expand Down Expand Up @@ -1795,7 +1810,7 @@ const COMPRESS_SETTING_FIELDS = new Set([
"nudgeGrowthTokens", "tierNudgeTokens", "nudgeModelDecided", "nudgeDecisionMaxTokens",
"preserveRecentMessages", "preserveRecentTokens",
"minCompressRange", "minCompressRangeChars", "stripImagesKeepRecent",
"outputHeadroomMaxPct", "tiers", "protectedLatestTools", "protectedTools",
"outputHeadroomMaxPct", "tiers", "protectedLatestTools", "protectedTools", "protectedContentMarkers",
"neverPreserveRecentTools", "preserveRecentTools", "stripImages",
"visibilityMarkers", "rules", "injectTool", "injectNudge",
"acknowledgePromptsRisk", "absorb", "ccr", "search", "imageCompression",
Expand Down Expand Up @@ -2252,7 +2267,7 @@ export function parseCompressSettings(v: unknown): (CompressSettings & { injectT
if (typeof obj.nudgeModelDecided !== "boolean") ok = false;
else out.nudgeModelDecided = obj.nudgeModelDecided;
}
for (const key of ["protectedLatestTools", "protectedTools"] as const) {
for (const key of ["protectedLatestTools", "protectedTools", "protectedContentMarkers"] as const) {
if (!(key in obj) || obj[key] === undefined) continue;
const v = obj[key];
if (!Array.isArray(v) || v.length === 0 || v.some((x) => typeof x !== "string" || x.trim().length === 0)) ok = false;
Expand Down
16 changes: 16 additions & 0 deletions src/durable-message-guards.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,3 +156,19 @@ export const durableMessageGuards: Record<string, DurableMessageGuard> = {
dsh: (msg) =>
dshSkillCatalogGuard(msg) || dshWorkspaceInstructionsGuard(msg) || dshRuntimeContextGuard(msg) || dshMcpCatalogGuard(msg),
};

/** #2446 方案A: user-configured content markers — the config lane of the
* durable protocol. Unlike everything above this is per-OPERATOR policy,
* not per-lane evidence: the user declares carrier substrings for hosts bili
* has no lane entry for (proxy mode has no client identity at all). The
* factory returns undefined for an empty/absent list so the guard chain
* never grows a dead predicate. Matching is raw substring-anywhere over
* text messages — deliberately the same granularity the built-in lane
* guards use (e.g. "Instructions from:"), so a config marker can reproduce
* any lane entry one-to-one (the documented escape hatch while a host waits
* for lane evidence). */
export function protectedContentMarkerGuard(markers: readonly string[]): ((msg: CoreMessage) => boolean) | undefined {
const list = markers.filter((m) => m.length > 0);
if (list.length === 0) return undefined;
return (msg) => msg.contentType === "text" && typeof msg.text === "string" && list.some((m) => msg.text!.includes(m));
}
16 changes: 14 additions & 2 deletions src/server/handle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import { codexAlignedWindow } from "../codex-models.js";
import { MAX_REQUEST_BYTES } from "../fetch-util.js";
import { hostIdForLog, maskHeadersForLog, maskUrlForLog, maskUrlsInText } from "../log-mask.js";
import { buildIncomingImageIndex, foldAnchoredCutoff, pruneRetrieveImgExports } from "../image-restore.js";
import { biliDurableMarkerGuard, durableMessageGuards } from "../durable-message-guards.js";
import { biliDurableMarkerGuard, durableMessageGuards, protectedContentMarkerGuard } from "../durable-message-guards.js";
import { biliToolsDeclaredOnWire, countBiliToolUses, evaluateSelfHealRound, nudgeSuppressed, pluginLaneDegraded, pluginLaneRestore } from "../session-self-heal.js";
import { compressBreakerArmed } from "../stream.js";
import { acquireInFlight, getSession, hasProcessedState, markDirty, peekSession, registeredProtectedRawIds, releaseInFlight, storeEffectiveConfig, tickPostRebuildAnchor, withSessionLock, type Session } from "../session.js";
Expand Down Expand Up @@ -1746,8 +1746,20 @@ export async function handle(
// resolution as session.ts effectiveConfig, so a registration via POST
// /__bili/plugin/protect protects the very next fold on the wire path.
const registered = registeredProtectedRawIds(session);
// #2446 方案A: fold in operator-declared content markers, resolved
// through the standard three-level compress cascade for THIS request
// and stamped onto the session (clone-safe strings) so the read path
// (effectiveConfig) sees the same list without request context.
const cfgMarkers = resolveCompress(opts.routes, route?.rewrittenUrl, requestModel, opts.compress).protectedContentMarkers;
if (cfgMarkers && cfgMarkers.length > 0) session.metadata["protectedContentMarkers"] = cfgMarkers;
else delete session.metadata["protectedContentMarkers"];
const cfgGuard = protectedContentMarkerGuard(cfgMarkers ?? []);
const durableGuard: (msg: CoreMessage) => boolean =
laneGuard ? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id) : (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id);
cfgGuard
? (msg) => (laneGuard ? laneGuard(msg) : false) || biliDurableMarkerGuard(msg) || registered.has(msg.id) || cfgGuard(msg)
: laneGuard
? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id)
: (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id);
reqConfig = { ...reqConfig, isMessageProtected: durableGuard };
// acquireInFlight must precede the lock so evictOldest() cannot flush
// this session between getSession and lock acquisition (inFlight===0
Expand Down
24 changes: 22 additions & 2 deletions src/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { createHash } from "node:crypto";
import { log as loggerLog } from "./logger.js";
import { getStore } from "./persist.js";
import { maxSessions as knobMaxSessions } from "./knobs.js";
import { biliDurableMarkerGuard, durableMessageGuards } from "./durable-message-guards.js";
import { biliDurableMarkerGuard, durableMessageGuards, protectedContentMarkerGuard } from "./durable-message-guards.js";
import type { WireProtocol } from "./util.js";

export type BlockView = { text: string; count: number };
Expand Down Expand Up @@ -442,11 +442,31 @@ export function effectiveConfig(session: Session | undefined, fallback: Config):
// session-keyed raw ids, persisted via metadata; composes with the marker
// (#2555) and lane guards. Same read-time re-resolution as the guards.
const registered = registeredProtectedRawIds(session);
// #2446 方案A: operator-declared content markers — stamped onto the
// session by the wire path from the request-resolved compress settings
// (clone-safe string array, same metadata discipline as the registration
// set), so this request-context-free reader sees the newest config.
const cfgGuard = protectedContentMarkerGuard(protectedContentMarkersOf(session));
const guard: (msg: CoreMessage) => boolean =
laneGuard ? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id) : (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id);
cfgGuard
? (msg) => (laneGuard ? laneGuard(msg) : false) || biliDurableMarkerGuard(msg) || registered.has(msg.id) || cfgGuard(msg)
: laneGuard
? (msg) => laneGuard(msg) || biliDurableMarkerGuard(msg) || registered.has(msg.id)
: (msg) => biliDurableMarkerGuard(msg) || registered.has(msg.id);
return !base.isMessageProtected ? { ...base, isMessageProtected: guard } : base;
}

/** #2446 方案A: the operator-declared marker list as read from session
* metadata. The wire path (server/handle.ts) stamps it per request from the
* three-level-resolved compress settings, so plugin-tool / status readers —
* which have no request context — resolve the same protection the wire used.
* Empty array removes the key (config dropped the markers → stop pinning;
* the session reflects current config, not historical config). */
export function protectedContentMarkersOf(session: Session | undefined): string[] {
const raw = session?.metadata["protectedContentMarkers"];
return Array.isArray(raw) ? raw.filter((v): v is string => typeof v === "string" && v.length > 0) : [];
}

/** #2556: the protect-tool registration as a membership set. Reading it from
* session.metadata at guard-resolution time (never cached across calls) keeps
* POST /__bili/plugin/protect effective for the next fold without any
Expand Down
Loading
Loading