Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -748,8 +748,8 @@ Environment variables take precedence over the config file. They are useful for
| `BILI_STREAM_ERROR_SHAPE` | Set `"completion"` to restore the legacy failure shape on the anthropic/openai wire (failure text inside a synthesized successful completion); default `"protocol"` = protocol-native error frames (#1455). Same knob as `compat.streamErrorShape`; this env var wins. |
| `BILI_LOG_MASK_HOSTS` | Set `0` to turn OFF host masking in proxy logs (#897): non-public target hosts (private relays, internal domains) are logged verbatim instead of `<private-host>`. Default is ON (#255 — logs get pasted into public issues); credential-header masking is independent and always on. Real target hosts are always available without touching this flag: `GET /__bili/stats` → `blindTunnels`, `GET /__bili/health` (both loopback-only), and the `acp_status` output. |
| `BILI_SUBAGENT_SPLIT` | Set `0` to turn OFF Claude Code subagent session splitting (#970): by default, anthropic-wire requests carrying the `x-claude-code-agent-id` + `x-claude-code-parent-agent-id` pair (background subagents) get their own `<session>\|sub:<agent-id>` session — own lock chain and compression state — instead of queueing behind the main session's lock. Default is ON. `"subagentSplit": false` in the config file does the same; the env var wins. |
| `BILI_FORK_ADOPTION` | Set `1` to turn ON fork block-adoption (#629): when an anonymous (prefix-affinity) client forks its history mid-conversation (edit-and-resend / regenerate an earlier turn), the new session inherits the parent's compression blocks whose source content is fully present in the forked request — instead of restarting with zero compression state and re-folding the shared prefix from scratch. Default is OFF. `"forkAdoption": true` in the config file does the same; the env var wins. The adoptable inventory is logged on every anonymous fork even while disabled, so you can size the win before flipping it on. |
| `BILI_RESUME_INHERITANCE` | Set `0` to turn OFF resume inheritance (ON by default) (#1486): when an identified client (sends its own session id, e.g. Claude Code's `x-claude-code-session-id`) resumes a conversation under a NEW session id while replaying the full transcript (`cc --resume` forks a fresh UUID), bili matches the incoming history byte-exactly against that client's tracked chains (≥8 messages, append-only tracking) and, on the resumed session's first request, inherits the parent's ref assignments — stale model citations then resolve to their ORIGINAL messages instead of mis-hitting renumbered ones — adopts the fully-present compression blocks (gated by the same `forkAdoption` switch as #629), and records the `derivedFrom` lineage. The parent is never modified and fresh messages number above the parent's ref space. A resume must strictly EXTEND the parent's history — an equal-depth byte-exact replay under another id is a duplicate, not a resume. Anonymous sessions are unaffected (they keep their own pfa-* world, #309). `"resumeInheritance": false` in the config file does the same; the env var wins. |
| `BILI_FORK_ADOPTION` | Set `1` to turn ON fork block-adoption (#629): when an anonymous (prefix-affinity) client forks its history mid-conversation (edit-and-resend / regenerate an earlier turn), the new session inherits the parent's compression blocks whose source content is fully present in the forked request — instead of restarting with zero compression state and re-folding the shared prefix from scratch. Default is OFF. Identified resume-forks are NOT governed by this switch — they adopt blocks together with `BILI_RESUME_INHERITANCE` (#1834). `"forkAdoption": true` in the config file does the same; the env var wins. The adoptable inventory is logged on every anonymous fork even while disabled, so you can size the win before flipping it on. |
| `BILI_RESUME_INHERITANCE` | Set `0` to turn OFF resume inheritance (ON by default) (#1486): when an identified client (sends its own session id, e.g. Claude Code's `x-claude-code-session-id`) resumes a conversation under a NEW session id while replaying the full transcript (`cc --resume` forks a fresh UUID), bili matches the incoming history byte-exactly against that client's tracked chains (≥8 messages, append-only tracking) and, on the resumed session's first request, inherits the parent's ref assignments — stale model citations then resolve to their ORIGINAL messages instead of mis-hitting renumbered ones — adopts the fully-present compression blocks (together with this inheritance, #1834: losing them on resume meant the folded originals came back on the wire and the upstream request ballooned; the old `forkAdoption` co-gate now applies only to anonymous forks, #629), and records the `derivedFrom` lineage. The parent is never modified and fresh messages number above the parent's ref space. A resume must strictly EXTEND the parent's history — an equal-depth byte-exact replay under another id is a duplicate, not a resume. Anonymous sessions are unaffected (they keep their own pfa-* world, #309). `"resumeInheritance": false` in the config file does the same; the env var wins. |
| `BILI_STABLE_SYSTEM_ANCHOR` | Set `1` to enable stable-system anchoring (#1085) — a **best-effort wire-layer fallback** for prefix caching: the root fix belongs client-side (the client owns its history and decides how to present instruction changes), this only stops the proxy from letting a changed head invalidate the whole cached prefix. **Plain-proxy mode only**: plugin-mode agents (`x-bili-plugin`) own their context management and are never anchored, so clients that already inject cache-friendly updates (e.g. claude-code-style system reminders) are not double-processed. When enabled, bili remembers each session's first-seen head system/instructions block and keeps re-sending those exact bytes even when the client's system prompt later changes. A **localized** change (a file-style edit sharing ≥70% of lines with the version in effect so far) appends a trailing `[System context update] …` user note carrying a compact line diff (`-` removed / `+` added; each note composes sequentially onto the previous one). A **non-localized** change (structural reshuffle, tool-def churn, timestamped banners, heads over 400 lines) is adopted outright — one deliberate cache miss beats appending noise that would mislead the model about its instructions. Churn guard: more than 8 accumulated notes likewise replace the anchor with the newest text and clear the log. The anchor and note log persist with the session and survive compression/compaction (session metadata, not kernel state). Known residual limitation: client-placed `cache_control` breakpoints may still misalign after a head swap. Excluded from anchoring: title-gen micro-requests (OpenAI/Google), Responses compaction-trigger requests, auto-mode classifier requests. Clients implementing their own variant (stable prompt + in-history updates) get zero extra injection — such updates pass through as ordinary history. Default is OFF. `"stableSystemAnchor": true` in the config file does the same; the env var wins. |
| `BILI_NO_CACHE_CONTROL` | Set `1` to disable bili's Anthropic-lane `cache_control` breakpoint stamping (#1637, shipped in #1639). On by default: Anthropic-style upstreams cache only what is explicitly breakpointed (max 4 per request, counted across system + tools + message blocks combined), so bili marks the system block plus up to 3 cumulative message marks — a marked message stays marked (byte-stable prefix), marks die only by folding, and the last 3 stable messages carry the advancing frontier. Any client-set `cache_control` (message blocks, tools entries) suppresses bili's stamps entirely — client-managed caching wins. Marks persist with the session. This switch is the escape hatch for upstreams that reject the field or relays with their own breakpoint policy. Plain-proxy Anthropic lane only; OpenAI/Responses lanes cache implicitly and are never stamped. Env-only; no config-file key. |
| `BILI_CHAIN_CONTENT` | Set `1` to turn ON the ACP-artifact / `<bili-chain …/>` checkpoint **body-content** detection of bili→bili chain awareness (#1086/#1421): when an inbound request carries compression artifacts (render tags / historical `acp_status`+`search_context` tool calls) or a digest-bearing checkpoint in its BODY but neither the `x-bili-hop` header nor local compression state for its session, bili records an advisory observation and/or applies first-processor-wins passthrough. **Default is OFF** (#1683 follow-up): by default ONLY the `x-bili-hop` header drives chain recognition, because scanning the request BODY can false-positive on CCR/file-introduced text and model-echoed tags that look like real markers. Enable it only for the narrow multi-bili relay case where an intermediate box strips `x-bili-hop` and you accept that false-positive risk. `"chainContentDetection": true` in the config file does the same; the env var wins. The `x-bili-hop` signal itself is unaffected by this switch. |
Expand Down
4 changes: 2 additions & 2 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -745,8 +745,8 @@
| `BILI_STREAM_ERROR_SHAPE` | 设为 `"completion"` 恢复 anthropic/openai 线上旧的失败形状(失败文本包在合成的成功完成里);默认 `"protocol"` = 协议原生错误帧(#1455)。与 `compat.streamErrorShape` 同一开关,此环境变量优先。 |
| `BILI_LOG_MASK_HOSTS` | 设为 `0` 关闭代理日志的 host 脱敏(#897):非公开目标主机(私有 relay、内网域名)原样记录,而不是 `<private-host>`。默认开启(#255 —— 日志常被整段贴进公开 issue);凭据头脱敏与之独立、始终开启。真实目标域名不依赖此开关也可查:`GET /__bili/stats` → `blindTunnels`、`GET /__bili/health`(均仅 loopback),以及 `acp_status` 输出。 |
| `BILI_SUBAGENT_SPLIT` | 设为 `0` 关闭 Claude Code subagent 会话分流(#970):默认情况下,anthropic 线路上同时携带 `x-claude-code-agent-id` + `x-claude-code-parent-agent-id` 头的请求(后台 subagent)会获得独立的 `<session>\|sub:<agent-id>` 会话 —— 独立的锁链与压缩状态 —— 不再排在主会话的锁后面。默认开启。配置文件中设 `"subagentSplit": false` 效果相同;环境变量优先。 |
| `BILI_FORK_ADOPTION` | 设为 `1` 开启 fork 块继承(#629):匿名(prefix-affinity)客户端在会话中途分叉历史(编辑重发 / 从更早轮次重新生成)时,新会话直接继承父会话中"源内容在分叉请求里完整存在"的压缩块 —— 而不是从零开始、把共享前缀重新折叠一遍。默认关闭。配置文件中设 `"forkAdoption": true` 效果相同;环境变量优先。无论开关如何,匿名 fork 发生时日志都会记录可继承的块清单,便于先评估收益再开启。 |
| `BILI_RESUME_INHERITANCE` | 设为 `0` 关闭 resume 继承(默认开启)(#1486):当带自有会话 id 的客户端(如 Claude Code 的 `x-claude-code-session-id`)以**新**会话 id 重放完整历史来续接会话时(`cc --resume` 会 fork 出新 UUID),bili 通过字节级前缀匹配(≥8 条消息、append-only 跟踪)识别出它与该客户端已跟踪历史的父子关系,并在续接会话的首个请求上继承父会话的 ref 分配 —— 模型引用的旧代际 refs 因此命中**原始**消息、而不是错配到重新编号的新消息 —— 同时继承源内容完整存在的压缩块(与 #629 共用 `forkAdoption` 开关),并记录 `derivedFrom` 血缘。父会话不受影响;新消息在父会话 ref 空间之上继续编号。resume 必须**严格扩展**父历史 —— 同深度的字节级重放(不同 id)视为重复会话而非 resume。匿名会话不受影响(保留自己的 pfa-* 世界,#309)。配置文件中设 `"resumeInheritance": false` 效果相同;环境变量优先。 |
| `BILI_FORK_ADOPTION` | 设为 `1` 开启 fork 块继承(#629):匿名(prefix-affinity)客户端在会话中途分叉历史(编辑重发 / 从更早轮次重新生成)时,新会话直接继承父会话中"源内容在分叉请求里完整存在"的压缩块 —— 而不是从零开始、把共享前缀重新折叠一遍。默认关闭。带自有 id 的 resume-fork 不受此开关管 —— 它们随 `BILI_RESUME_INHERITANCE` 一并继承压缩块(#1834)。配置文件中设 `"forkAdoption": true` 效果相同;环境变量优先。无论开关如何,匿名 fork 发生时日志都会记录可继承的块清单,便于先评估收益再开启。 |
| `BILI_RESUME_INHERITANCE` | 设为 `0` 关闭 resume 继承(默认开启)(#1486):当带自有会话 id 的客户端(如 Claude Code 的 `x-claude-code-session-id`)以**新**会话 id 重放完整历史来续接会话时(`cc --resume` 会 fork 出新 UUID),bili 通过字节级前缀匹配(≥8 条消息、append-only 跟踪)识别出它与该客户端已跟踪历史的父子关系,并在续接会话的首个请求上继承父会话的 ref 分配 —— 模型引用的旧代际 refs 因此命中**原始**消息、而不是错配到重新编号的新消息 —— 同时继承源内容完整存在的压缩块(随本继承一并生效,#1834:resume 丢块会导致被折叠原文重新回到线上、上游请求膨胀;旧的 `forkAdoption` 联动门现仅作用于匿名 fork,#629),并记录 `derivedFrom` 血缘。父会话不受影响;新消息在父会话 ref 空间之上继续编号。resume 必须**严格扩展**父历史 —— 同深度的字节级重放(不同 id)视为重复会话而非 resume。匿名会话不受影响(保留自己的 pfa-* 世界,#309)。配置文件中设 `"resumeInheritance": false` 效果相同;环境变量优先。 |
| `BILI_STABLE_SYSTEM_ANCHOR` | 设为 `1` 开启稳定 system 锚定(#1085)—— **wire 层兜底(best-effort)**:根治在客户端(会话历史与指令变更的呈现方式由客户端决定),本开关只是阻止代理因头部变化而使整个已缓存前缀失效。**仅限 plain-proxy 模式**:plugin-mode agent(`x-bili-plugin`)自管上下文、永不参与锚定,避免对已自带 cache-friendly 更新注入的客户端(如 claude-code 的 system-reminder)做双重处理。开启后,bili 按会话记住客户端首次发送的头部 system/instructions 块并持续原样重发。**局部变更**(文件式编辑,与当前生效版本共享 ≥70% 行)追加末尾 `[System context update] …` user 注记,内含紧凑行级 diff(`-` 删除 / `+` 新增;每条注记顺序叠加在前一条之上)。**非局部变更**(结构性重排、tool 定义增删、带时间戳的 banner、超 400 行的头部)直接采用新文本 —— 一次有意的缓存失效好过追加会误导模型的噪声 diff。防抖保护:累积超过 8 条注记同样直接替换锚点为最新文本并清空日志。锚点与注记日志随会话持久化,不受压缩/compaction 影响(session metadata 而非 kernel state)。已知残留限制:客户端自放的 `cache_control` 断点在换头后仍可能错位。不参与锚定的请求:标题生成微请求(OpenAI/Google)、Responses compaction-trigger 请求、auto-mode classifier 请求。客户端自身已实现同类机制(稳定 prompt + 历史内更新)时零额外注入 —— 这类更新作为普通历史透传。默认关闭。配置文件中设 `"stableSystemAnchor": true` 效果相同;环境变量优先。 |
| `BILI_NO_CACHE_CONTROL` | 设为 `1` 关闭 bili 在 Anthropic 通道上的 `cache_control` 断点标注(#1637,随 #1639 落地)。默认开启:Anthropic 系上游只缓存被显式打断点的内容(每请求最多 4 个,按 system + tools + 消息块合并计数),因此 bili 会标注 system 块加上至多 3 个累积消息断点——被标注的消息保持标注(前缀字节稳定),断点只随折叠消亡,最近 3 条稳定消息承载推进前沿。客户端自设的任何 `cache_control`(消息块、tools 条目)都会完全抑制 bili 的标注——客户端自管缓存优先。断点随会话持久化。本开关是逃生阀:用于拒绝该字段的上游或有自己断点策略的中继。仅限 plain-proxy Anthropic 通道;OpenAI/Responses 通道隐式缓存,从不标注。仅环境变量;无配置文件键。 |
| `BILI_CHAIN_CONTENT` | 设为 `1` 开启 bili→bili 链感知的 ACP 产物 / `<bili-chain …/>` 检查点**正文内容**检测(#1086/#1421):当入站请求的正文携带压缩产物(渲染标签 / 历史 `acp_status`+`search_context` 工具调用)或带摘要的检查点,但既无 `x-bili-hop` 头、本实例也无该会话的压缩状态时,bili 记录一次告警性观察和/或应用「首个处理器优先」透传。**默认关闭**(#1683 后续):默认下只有 `x-bili-hop` 头驱动链识别,因为扫描请求正文可能把 CCR/文件引入的文本和模型回声标签误判为真实标记。仅在中间盒子剥掉 `x-bili-hop`、且你接受该误判风险的狭窄多 bili 中继场景下才启用。配置文件中设 `"chainContentDetection": true` 效果相同;环境变量优先。`x-bili-hop` 信号本身不受此开关影响。 |
Expand Down
29 changes: 23 additions & 6 deletions src/affinity-persist.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,22 @@ import { log } from "./logger.js";
* debounced atomic write and hydrate on boot.
*/

const PERSIST_DEBOUNCE_MS = 5_000;
/** #1834: quiesce window. Every chain mutation restarts this debounce (a
* busy proxy would otherwise push the write out indefinitely and a SIGKILL
* lost EVERYTHING since the last flush, not just 5s worth), and the
* max-delay budget caps how long a pending write can be starved: the file
* is on disk within ~500ms of the last mutation, or within 5s under
* sustained traffic — previously a flat 5s trailing window. */
const PERSIST_DEBOUNCE_MS = 500;
const PERSIST_MAX_DELAY_MS = 5_000;

function affinityFile(): string {
return path.join(stateDir(), "prefix-affinity.json");
}

let timer: NodeJS.Timeout | null = null;
let writing = false;
let pendingSince: number | null = null;

/** Entries another process left on disk since we hydrated; null on read failure
* (the caller then writes its own snapshot unchanged). */
Expand Down Expand Up @@ -104,16 +112,25 @@ function writeSnapshot(): void {
log("warn", `[prefix-affinity] persist failed (${e instanceof Error ? e.message : String(e)}); affinity survives in memory`);
} finally {
writing = false;
pendingSince = null;
}
}

/** Debounced snapshot write — call after every affinity mutation. */
/** Debounced snapshot write — call after every affinity mutation. The
* trailing debounce quiesces bursts; the #1834 max-delay budget guarantees
* the write still lands under sustained traffic (see constants above). */
export function scheduleAffinityPersist(): void {
const now = Date.now();
if (pendingSince === null) pendingSince = now;
if (timer) clearTimeout(timer);
timer = setTimeout(() => {
timer = null;
writeSnapshot();
}, PERSIST_DEBOUNCE_MS);
const budget = PERSIST_MAX_DELAY_MS - (now - pendingSince);
timer = setTimeout(
() => {
timer = null;
writeSnapshot();
},
Math.max(0, Math.min(PERSIST_DEBOUNCE_MS, budget)),
);
timer.unref?.();
}

Expand Down
Loading
Loading