Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1359,6 +1359,27 @@ A URL-keyed lane (or a named entry with `bind`) can send **its own** credential

Compression behaviour is controlled by the `compress` block, which can appear at three levels. They merge **per-field, deepest wins**: a field set at a deeper level overrides the same field higher up, but an *unset* field at a deeper level never clears a value set higher up. In other words, the child covers the parent field-by-field — it never replaces the whole object.

### Per-session opt-out (#5300)

The merge above is the only *static* scope. For one session whose history must never be folded (e.g. a long-lived companion/roleplay conversation where the original wording itself is the asset), exclude it at runtime — no config key, **exact session id** (deliberately no prefix/label matching: a mis-hit would fail silently, defeating the purpose), and the flag persists in the session record across restarts:

```bash
# turn compression OFF for one session
curl -s -X POST -H 'content-type: application/json' -d '{"enabled":false}' \
http://127.0.0.1:<bili-port>/__bili/session/<session-id>/compress

# read it back — the guard against silent re-enablement (resident sessions
# answer from memory, cold ones from the store; unknown ids 404)
curl -s http://127.0.0.1:<bili-port>/__bili/session/<session-id>/compress
# → {"ok":true,"sessionId":"…","compression":{"enabled":false},"resident":true}

# re-enable when you are done
curl -s -X POST -H 'content-type: application/json' -d '{"enabled":true}' \
http://127.0.0.1:<bili-port>/__bili/session/<session-id>/compress
```

Semantics while excluded (the same verbatim tail as [`passthrough`](#passthrough), scoped to one session): no fold, no nudge, no tool injection, no preflight, no marker — the client's bytes go out untouched, so the conversation grows unbounded and **window overflow surfaces as the upstream's own error** instead of a quiet fold (that is the point: an explicit failure you can see beats irreversible silent loss). Already-folded blocks stay folded (`decompress` still restores them), and plugin-lane hosts calling the compress tool get a loud refusal (`409 COMPRESSION_DISABLED`) instead of a silent no-op. Every other session on the same model keeps compressing normally. Loopback-only like every `/__bili/` route.

### Shared external summary service

The optional `compress.externalSummary` block sends compression summaries to one or more separately configured models. It lives at all three `compress` levels like every other field, with whole-chain semantics: a chain set at a deeper level (provider or model) **replaces** the entire chain above it — there is no per-target or per-budget sub-merge, exactly like `tiers`. It is disabled unless `enabled` is `true`; when enabled, targets are tried in order and a failed or unusable target falls through to the next one. The main request provider, model, and authorization are never reused for these calls. This feature changes the compression tool contract so `summary` is an optional non-authoritative hint; the proxy keeps the original messages recoverable and commits only a validated returned summary. Because the conversation model no longer writes the committed summary, its injected compress prompt drops the summary-writing rules (`HOW TO COMPRESS`, `KEEP VERBATIM`, `DROP`, `PRIORITY`) and the per-summary length budget while the chain is enabled; the summary requests themselves still carry the full rules (#4460).
Expand Down
20 changes: 20 additions & 0 deletions docs/configuration.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1356,6 +1356,26 @@ URL 键通道(或带 `bind` 的命名条目)可以向上游发送**自己的

压缩行为由 `compress` 块控制,它可以出现在三个层级。它们按**逐字段、最深层胜出**的方式合并:在更深层设置的字段会覆盖上层同名字段,但更深层*未设置*的字段**永远不会**清除上层已设置的值。换言之,子级按字段覆盖父级 —— 它绝不是整体替换对象。

### 按会话排除压缩(#5300)

上面的三级合并是唯一的**静态**作用域。如果某个会话的历史绝不能被折叠(例如长期陪伴 / 角色扮演会话,原始措辞本身就是资产),可以在运行时把它排除——没有配置键,**精确匹配会话 id**(刻意不做前缀 / 标签匹配:误命中会静默失效,正好毁掉这个功能的目的),开关存进会话记录、跨重启生效:

```bash
# 对单个会话关闭压缩
curl -s -X POST -H 'content-type: application/json' -d '{"enabled":false}' \
http://127.0.0.1:<bili端口>/__bili/session/<session-id>/compress

# 读回——防「静默恢复压缩」的保险(驻留会话从内存回答,冷会话从存储回答,未知 id 返回 404)
curl -s http://127.0.0.1:<bili端口>/__bili/session/<session-id>/compress
# → {"ok":true,"sessionId":"…","compression":{"enabled":false},"resident":true}

# 用完后重新开启
curl -s -X POST -H 'content-type: application/json' -d '{"enabled":true}' \
http://127.0.0.1:<bili端口>/__bili/session/<session-id>/compress
```

排除期间的语义(与 [`passthrough`](#passthrough) 同一条 verbatim 转发尾,但只作用于一个会话):不折叠、不 nudge、不注入工具、不 preflight、不打 marker——客户端的字节原样发出,会话因此无限增长,**窗口溢出会以上游自己的错误形式显式冒出来**,而不是被悄悄折掉(这正是设计目的:看得见的显式失败好过不可逆的静默丢失)。已折叠的块保持折叠(`decompress` 仍能恢复),插件车道宿主调用 compress 工具会得到响亮的拒绝(`409 COMPRESSION_DISABLED`),而不是静默空转。同一模型上的其余会话照常压缩。与所有 `/__bili/` 路由一样仅限 loopback。

### 共享外部摘要服务

可选的 `compress.externalSummary` 会把压缩摘要交给一个或多个独立配置的模型。它与其余 `compress` 字段一样存在于全部三个层级,采用整链替换语义:在更深层级(provider 或 model)设置的链会**整体替换**上层链,不做按目标或按预算的子字段合并,与 `tiers` 完全一致。只有 `enabled` 为 `true` 时才启用;启用后按顺序调用目标,目标失败或返回不可用摘要时继续使用下一个目标。摘要请求不会复用主请求的 provider、模型或认证信息。启用该功能后,压缩工具中的 `summary` 变为可选的、非权威提示;代理仍保留原文可恢复,只提交通过校验的外部摘要。由于会话模型不再撰写最终摘要,链启用期间注入的压缩系统提示会去掉摘要撰写规则(`HOW TO COMPRESS`、`KEEP VERBATIM`、`DROP`、`PRIORITY`)和单条摘要长度预算;摘要请求本身仍携带完整规则(#4460)。
Expand Down
12 changes: 11 additions & 1 deletion src/incremental-lane.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ import { createHash } from "node:crypto";
import { Readable } from "node:stream";
import { anthropicToCore, coreToAnthropic, type BiliMessage } from "acp-kernel/wire";
import { getStore } from "./persist.js";
import { peekSession } from "./session.js";
import { isCompressionDisabled, peekSession, snapshotMessages } from "./session.js";
import { pluginAgentHeader, pluginReportedModel } from "./plugin.js";
import { headerValue, readBody } from "./server.js";
import { featureEnabled } from "./rollout.js";
Expand Down Expand Up @@ -1034,6 +1034,16 @@ export async function handleIncrementalTurn(
(res as { writeHead?: unknown }).writeHead = originalWriteHead as unknown as never;
};
log("info", `[incremental] turn ${conversationId} since=${String(sinceSeq)} applied=${String(appliedSeq)} agent=${agent} model=${String(model)} → forward ${path}`);
// #5300: the pipeline gate forwards excluded sessions verbatim, which
// skips prepare-anthropic — the one place rawTail durability gets
// armed. Arm it here from the lane's authoritative core view so the
// post-forward scheduleSave ingests THIS turn; without it the canonical
// table would miss excluded turns and a cold rebuild would 409-resync
// against history the client already pruned past its acked seq.
const liveSession = peekSession(sessionId);
if (liveSession && isCompressionDisabled(liveSession)) {
snapshotMessages(liveSession, entry.msgs);
}
// Best-effort durability close-out: flush the session once the
// response has fully left so the canonical table catches up with the
// ack before the client can act on the next turn.
Expand Down
10 changes: 9 additions & 1 deletion src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import path from "node:path";
import { createHash } from "node:crypto";
import { CORRUPT_SIBLING_CAP, getStore, pruneCorruptSiblings, storageStatusLabel, kvStore } from "./persist.js";
import { cloneStoreForRefs } from "./store.js";
import { acquireInFlight, createSession, getSessionAsync, publishForkSession, diagnoseSuccessWithoutUsage, effectiveConfig, findSessionByCanonicalId, listSessions, markCompactionBoundary, markDirty, noteAbortedTurn, noteTruncatedUpstream, peekSession, releaseInFlight, releaseNudgeHold, statusInputBaseline, withSessionLock, type Session } from "./session.js";
import { acquireInFlight, createSession, getSessionAsync, publishForkSession, diagnoseSuccessWithoutUsage, effectiveConfig, findSessionByCanonicalId, isCompressionDisabled, listSessions, markCompactionBoundary, markDirty, noteAbortedTurn, noteTruncatedUpstream, peekSession, releaseInFlight, releaseNudgeHold, statusInputBaseline, withSessionLock, type Session } from "./session.js";
import { clientConversationHeader } from "./session-id.js";
import { buildContextView } from "./context-view.js";
import { ABSORB_TOOL_NAME, BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, PROXY_TOOL_NAMES, READ_ORIGINAL_IMAGE_TOOL, READ_ORIGINAL_IMAGE_TOOL_NAME, READ_ORIGINAL_IMAGE_TOOL_OPENAI, READ_ORIGINAL_IMAGE_TOOL_RESPONSES, RETRIEVE_TOOL_NAME, RULE_TOOL, RULE_TOOL_NAME, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, SEARCH_CONTEXT_TOOL_NAME, absorbToolsFor, blockOpsTools, retrieveToolsFor, withAcpNotes, withAcpNotesOpenAI } from "./compress-tool.js";
Expand Down Expand Up @@ -2422,6 +2422,14 @@ export async function handlePluginTool(
}));
return;
}
// #5300: an excluded session's pipeline never folds, so a compress call can
// only fail — refuse it loudly instead of letting the model believe a fold
// happened. decompress stays available (restoring a pre-exclusion block is
// still meaningful while the session record retains it).
if (tool === "compress" && isCompressionDisabled(session)) {
deps.log("info", `[${session.id}] [plugin] compress refused: compression is disabled for this session (#5300)`);
return forkReply(res, 409, { ok: false, code: "COMPRESSION_DISABLED", error: `compression is disabled for session "${session.id}" (#5300 per-session opt-out) — nothing was folded. Re-enable it (POST /__bili/session/${encodeURIComponent(session.id)}/compress {"enabled":true}) or continue without folding.` });
}
// Absorb/rules enablement is per-session (last resolved config), so the
// gate needs the session — it runs after the lookup above.
if (!isProxyToolFor(tool, session, deps.config)) {
Expand Down
80 changes: 79 additions & 1 deletion src/server/admin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import { dismissHostPlugin, visibleHostPluginNotices } from "../host-plugin-noti
import { dismissUpgradeNotice } from "../pck-notice.js";
import { mergedNotices, localDismissKey, localDismissMode, markInstanceDismissal } from "../announcements.js";
import { remoteAnnouncementPayloads, remoteDismissKey, remoteDismissMode } from "../remote-notices.js";
import { listSessions, memoryReport, peekSession, splitSessionWarnings, totalInFlight } from "../session.js";
import { isCompressionDisabled, listSessions, markDirty, memoryReport, peekSession, splitSessionWarnings, totalInFlight } from "../session.js";
import { BILI_TUNNEL_HEADER } from "../tunnel-guard.js";
import { isLoopbackAddress } from "../util.js";
import { clearUpstreamAlertsForHost, getUpstreamAlerts, getUpstreamHealthReport } from "../upstream-alerts.js";
Expand Down Expand Up @@ -199,6 +199,28 @@ function auxProxyShot(opts: ProxyOptions): { proxy: string | null; source: strin
}
}

// #5300: parse /__bili/session/<id>/compress → decoded id, or undefined when
// the shape doesn't match (method is judged by the caller; query strings are
// tolerated and stripped). Exact-match discipline: ids never contain "/",
// overlong or malformed encodings are rejected rather than guessed.
const SESSION_COMPRESS_PREFIX = "/__bili/session/";
const SESSION_COMPRESS_SUFFIX = "/compress";
function sessionCompressIdFromUrl(url: string | undefined): string | undefined {
if (!url || !url.startsWith(SESSION_COMPRESS_PREFIX)) return undefined;
const rest = url.slice(SESSION_COMPRESS_PREFIX.length);
const q = rest.indexOf("?");
const pathPart = q === -1 ? rest : rest.slice(0, q);
if (!pathPart.endsWith(SESSION_COMPRESS_SUFFIX) || pathPart.length <= SESSION_COMPRESS_SUFFIX.length) return undefined;
let id: string;
try {
id = decodeURIComponent(pathPart.slice(0, pathPart.length - SESSION_COMPRESS_SUFFIX.length));
} catch {
return undefined;
}
if (id.length === 0 || id.length > 512 || id.includes("/")) return undefined;
return id;
}

export async function handleAdminRoute(req: http.IncomingMessage, res: http.ServerResponse, ctx: AdminCtx): Promise<void> {
const { opts, core, config, log, instanceId, instanceStartedAt, proxyWatchers, initialWatcherPid, watcherLastSeen } = ctx;
// SECURITY: the /__bili/ management endpoints (config read/write, reload,
Expand Down Expand Up @@ -695,6 +717,62 @@ export async function handleAdminRoute(req: http.IncomingMessage, res: http.Serv
return;
}
}
// #5300: per-session compression opt-out — the scope between the
// global/provider/model compress merge and all-or-nothing route
// passthrough. Flag lives in session.metadata (persisted with the record,
// survives restarts); EXACT id match only — a silent prefix/label mis-hit
// would defeat the feature's purpose (no silent folding). Excluded
// sessions take the pipeline's verbatim-forward tail (handle.ts): no fold,
// no nudge, no injection, no preflight, no marker; window overflow surfaces
// as an explicit upstream error instead of a quiet fold. Loopback-gated
// like every /__bili/ route.
const sessionCompressId = req.method === "GET" || req.method === "POST" ? sessionCompressIdFromUrl(req.url) : undefined;
if (req.method === "GET" && sessionCompressId !== undefined) {
const id = sessionCompressId;
const resident = peekSession(id);
if (resident) {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, sessionId: id, compression: { enabled: !isCompressionDisabled(resident) }, resident: true }));
return;
}
const loaded = await getStore().loadAsync(id);
if (!loaded) {
res.writeHead(404, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: `unknown session "${id}" — no such record in this proxy instance` }));
return;
}
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, sessionId: id, compression: { enabled: !isCompressionDisabled(loaded) }, resident: false }));
return;
}
if (req.method === "POST" && sessionCompressId !== undefined) {
const id = sessionCompressId;
try {
const parsed = JSON.parse((await readBody(req)).toString("utf8")) as { enabled?: unknown };
if (typeof parsed.enabled !== "boolean") {
res.writeHead(400, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: 'expected { "enabled": true|false }' }));
return;
}
const session = peekSession(id);
if (!session) {
res.writeHead(404, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: `unknown session "${id}" — not resident in this proxy instance; a model request under its id re-binds it` }));
return;
}
if (parsed.enabled) delete session.metadata.compressionDisabled;
else session.metadata.compressionDisabled = true;
markDirty(session);
log("info", `[admin] POST /__bili/session/${id}/compress — compression ${parsed.enabled ? "re-enabled" : "disabled"} (#5300)`);
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: true, sessionId: id, compression: { enabled: parsed.enabled } }));
return;
} catch (err) {
res.writeHead(err instanceof BodyTooLargeError ? 413 : 400, { "content-type": "application/json" });
res.end(JSON.stringify({ ok: false, error: String(err) }));
return;
}
}
// #2671 Phase 2: replica/checkpoint management over the unified store.
// Loopback-gated like every /__bili/ route; 501 in legacy mode so the
// operator knows the knob, not the URL, is what is missing.
Expand Down
Loading
Loading