diff --git a/CONFIGURATION.md b/CONFIGURATION.md index f3b9f48ec1..ab12a3029b 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -1769,6 +1769,7 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes | `BILI_AFFINITY_SIMHASH` | Set `0` to turn OFF simhash chain-alignment adoption (#2265): anonymous requests whose exact hash chain was broken by a client-side pervasive decorative rewrite (e.g. Trae re-stamping model tags on every assistant message) re-attach the existing session with its compression state instead of minting fresh and re-folding from zero. Guards: ≥90% coverage within Hamming 10, ≥20% mutated-but-similar positions (point edits still fork, #629), ≥1 byte-identical user message, unique candidate only. Default ON. `"affinitySimhash": false` in the config file does the same; the env var wins. | | `BILI_RESUME_INHERITANCE` | Set `0` to turn OFF resume inheritance (ON by default) (#1486): when an identified client (sends its own session id, e.g. Claude Code's `x-claude-code-session-id`) resumes a conversation under a NEW session id while replaying the full transcript (`cc --resume` forks a fresh UUID), bili matches the incoming history byte-exactly against that client's tracked chains (≥8 messages, append-only tracking) and, on the resumed session's first request, inherits the parent's ref assignments — stale model citations then resolve to their ORIGINAL messages instead of mis-hitting renumbered ones — adopts the fully-present compression blocks (together with this inheritance, #1834: losing them on resume meant the folded originals came back on the wire and the upstream request ballooned; the old `forkAdoption` co-gate now applies only to anonymous forks, #629), and records the `derivedFrom` lineage. The parent is never modified and fresh messages number above the parent's ref space. A resume must strictly EXTEND the parent's history — an equal-depth byte-exact replay under another id is a duplicate, not a resume. Anonymous sessions are unaffected (they keep their own pfa-* world, #309). `"resumeInheritance": false` in the config file does the same; the env var wins. | | `BILI_STABLE_SYSTEM_ANCHOR` | Set `1` to enable stable-system anchoring (#1085) — a **best-effort wire-layer fallback** for prefix caching: the root fix belongs client-side (the client owns its history and decides how to present instruction changes), this only stops the proxy from letting a changed head invalidate the whole cached prefix. **Plain-proxy mode only**: plugin-mode agents (`x-bili-plugin`) own their context management and are never anchored, so clients that already inject cache-friendly updates (e.g. claude-code-style system reminders) are not double-processed. When enabled, bili remembers each session's first-seen head system/instructions block and keeps re-sending those exact bytes even when the client's system prompt later changes. A **localized** change (a file-style edit sharing ≥70% of lines with the version in effect so far) appends a trailing `[System context update] …` user note carrying a compact line diff (`-` removed / `+` added; each note composes sequentially onto the previous one). A **non-localized** change (structural reshuffle, tool-def churn, timestamped banners, heads over 400 lines) is adopted outright — one deliberate cache miss beats appending noise that would mislead the model about its instructions. Churn guard: more than 8 accumulated notes likewise replace the anchor with the newest text and clear the log. The anchor and note log persist with the session and survive compression/compaction (session metadata, not kernel state). Known residual limitation: client-placed `cache_control` breakpoints may still misalign after a head swap. Excluded from anchoring: title-gen micro-requests (OpenAI/Google), Responses compaction-trigger requests, auto-mode classifier requests. Clients implementing their own variant (stable prompt + in-history updates) get zero extra injection — such updates pass through as ordinary history. Default is OFF. `"stableSystemAnchor": true` in the config file does the same; the env var wins. | +| `BILI_ALLOW_DSH_COMPACTION` | Set `1` to let dsh's built-in auto-compaction run through bili (#2028). By default the wire-level guard (#1729) **refuses dsh native compaction calls locally** (422) on every lane bili serves — openai, anthropic, and responses (responses added by #2360; it was the lane the pre-#2360 whitelist short-circuited before the marker was ever examined, so dsh desktop's compaction calls passed silently): dsh's `compaction-basic` answers context pressure by replaying the conversation prefix plus a fixed summarization directive as the final user message, and when such a call LANDS its checkpoint durably shadows the raw history — irreversible, and it destroys the proxy's compression substrate. This switch lifts the refusal so dsh-native compaction can actually execute. While the guard refuses, dsh's own context-pressure counter (its raw ledger) only grows — refused checkpoints never land — while bili's folds keep the wire payload small, so the session keeps working past the point where the client's gauge says "full", and dsh keeps re-requesting compaction (refused every time). Since #3384 the outgoing output budget is also no longer planned against that raw ledger, EXCEPT when bili's own folded view has already reached the enforced headroom target or the requested budget is >= 25% of the native window (the #2490 protection regimes) — before that fix the growing ledger silently shrank the model's output budget every turn, ending in error-free empty replies on onlyReasoning models. Scope note: on non-web profiles the shipped bundle patch (`auto: false`) still suppresses AUTO-triggering, so only manual `/compact` benefits there; on web profiles no patch layer can address the preset-nested row by id (#1772), so auto-triggering works as-is once allowed unless you apply the #2474 full-snapshot override of preset-standard from your profile's cordis.patch.yml. The web config page exposes the same switch; the env var wins over the file. Set `"dsh": { "allowDshCompaction": true }` in the config file to the same effect (a bare top-level `"allowDshCompaction"` in an older file is auto-relocated there on load). | | `BILI_ALLOW_DSH_COMPACTION` | Set `1` to let dsh's built-in auto-compaction run through bili (#2028). By default the wire-level guard (#1729) **refuses dsh native compaction calls locally** (422) on every lane bili serves — openai, anthropic, and responses (responses added by #2360; it was the lane the pre-#2360 whitelist short-circuited before the marker was ever examined, so dsh desktop's compaction calls passed silently): dsh's `compaction-basic` answers context pressure by replaying the conversation prefix plus a fixed summarization directive as the final user message, and when such a call LANDS its checkpoint durably shadows the raw history — irreversible, and it destroys the proxy's compression substrate. This switch lifts the refusal so dsh-native compaction can actually execute. Scope note: with bili ON, dsh native auto-compaction is now off on every plane — the host-plane row via the shipped bundle patch (`auto: false`), and web-profile preset instances at load time via the #3354 overlay — so this switch mainly governs manual `/compact` on non-web profiles (auto-triggering stays suppressed there) and any residual auto-triggering on dsh builds where the #3354 overlay does not apply. The web config page exposes the same switch; the env var wins over the file. Set `"dsh": { "allowDshCompaction": true }` in the config file to the same effect (a bare top-level `"allowDshCompaction"` in an older file is auto-relocated there on load). | | `BILI_NO_CACHE_CONTROL` | Set `1` to disable bili's Anthropic-lane `cache_control` breakpoint stamping (#1637, shipped in #1639). On by default: Anthropic-style upstreams cache only what is explicitly breakpointed (max 4 per request, counted across system + tools + message blocks combined), so bili marks the system block plus up to 3 cumulative message marks — a marked message stays marked (byte-stable prefix), marks die only by folding, and the last 3 stable messages carry the advancing frontier. Any client-set `cache_control` (message blocks, tools entries) suppresses bili's stamps entirely — client-managed caching wins. Marks persist with the session. This switch is the escape hatch for upstreams that reject the field or relays with their own breakpoint policy. Plain-proxy Anthropic lane only; OpenAI/Responses lanes cache implicitly and are never stamped. Config-file twin: [`compat.noCacheControl`](#compat) — the env var wins. | | `BILI_CHAIN_CONTENT` | Set `1` to turn ON the ACP-artifact / `` checkpoint **body-content** detection of bili→bili chain awareness (#1086/#1421): when an inbound request carries compression artifacts (render tags / historical `acp_status`+`search_context` tool calls) or a digest-bearing checkpoint in its BODY but neither the `x-bili-hop` header nor local compression state for its session, bili records an advisory observation and/or applies first-processor-wins passthrough. **Default is OFF** (#1683 follow-up): by default ONLY the `x-bili-hop` header drives chain recognition, because scanning the request BODY can false-positive on CCR/file-introduced text and model-echoed tags that look like real markers. Enable it only for the narrow multi-bili relay case where an intermediate box strips `x-bili-hop` and you accept that false-positive risk. `"chainContentDetection": true` in the config file does the same; the env var wins. The `x-bili-hop` signal itself is unaffected by this switch. | diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 4e3da80fba..b7d26def9f 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -1765,6 +1765,7 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 | `BILI_AFFINITY_SIMHASH` | 设为 `0` 关闭 simhash 链对齐收养(#2265):匿名请求的精确哈希链被客户端侧大面积装饰性改写(如 Trae 切模型后给每条 assistant 消息重打模型标签)打断时,重新挂回既有会话并保留压缩状态,而不是每次新铸会话、从零重折。护栏:覆盖率 ≥90%(Hamming ≤10)、变异位置 ≥20%(单点编辑仍走 fork,#629)、至少一条字节相同的用户消息、双候选歧义拒猜。默认开启。配置文件中设 `"affinitySimhash": false` 效果相同;环境变量优先。 | | `BILI_RESUME_INHERITANCE` | 设为 `0` 关闭 resume 继承(默认开启)(#1486):当带自有会话 id 的客户端(如 Claude Code 的 `x-claude-code-session-id`)以**新**会话 id 重放完整历史来续接会话时(`cc --resume` 会 fork 出新 UUID),bili 通过字节级前缀匹配(≥8 条消息、append-only 跟踪)识别出它与该客户端已跟踪历史的父子关系,并在续接会话的首个请求上继承父会话的 ref 分配 —— 模型引用的旧代际 refs 因此命中**原始**消息、而不是错配到重新编号的新消息 —— 同时继承源内容完整存在的压缩块(随本继承一并生效,#1834:resume 丢块会导致被折叠原文重新回到线上、上游请求膨胀;旧的 `forkAdoption` 联动门现仅作用于匿名 fork,#629),并记录 `derivedFrom` 血缘。父会话不受影响;新消息在父会话 ref 空间之上继续编号。resume 必须**严格扩展**父历史 —— 同深度的字节级重放(不同 id)视为重复会话而非 resume。匿名会话不受影响(保留自己的 pfa-* 世界,#309)。配置文件中设 `"resumeInheritance": false` 效果相同;环境变量优先。 | | `BILI_STABLE_SYSTEM_ANCHOR` | 设为 `1` 开启稳定 system 锚定(#1085)—— **wire 层兜底(best-effort)**:根治在客户端(会话历史与指令变更的呈现方式由客户端决定),本开关只是阻止代理因头部变化而使整个已缓存前缀失效。**仅限 plain-proxy 模式**:plugin-mode agent(`x-bili-plugin`)自管上下文、永不参与锚定,避免对已自带 cache-friendly 更新注入的客户端(如 claude-code 的 system-reminder)做双重处理。开启后,bili 按会话记住客户端首次发送的头部 system/instructions 块并持续原样重发。**局部变更**(文件式编辑,与当前生效版本共享 ≥70% 行)追加末尾 `[System context update] …` user 注记,内含紧凑行级 diff(`-` 删除 / `+` 新增;每条注记顺序叠加在前一条之上)。**非局部变更**(结构性重排、tool 定义增删、带时间戳的 banner、超 400 行的头部)直接采用新文本 —— 一次有意的缓存失效好过追加会误导模型的噪声 diff。防抖保护:累积超过 8 条注记同样直接替换锚点为最新文本并清空日志。锚点与注记日志随会话持久化,不受压缩/compaction 影响(session metadata 而非 kernel state)。已知残留限制:客户端自放的 `cache_control` 断点在换头后仍可能错位。不参与锚定的请求:标题生成微请求(OpenAI/Google)、Responses compaction-trigger 请求、auto-mode classifier 请求。客户端自身已实现同类机制(稳定 prompt + 历史内更新)时零额外注入 —— 这类更新作为普通历史透传。默认关闭。配置文件中设 `"stableSystemAnchor": true` 效果相同;环境变量优先。 | +| `BILI_ALLOW_DSH_COMPACTION` | 设为 `1` 放行 dsh 内置自动压缩(#2028)。默认由 wire 级守卫(#1729)**在本地拒绝 dsh 原生压缩调用**(422),覆盖 bili 服务的全部线路——openai、anthropic、responses(responses 由 #2360 补上:此前的协议白名单在检查标记之前就把 responses 短路了,而 dsh 桌面端的压缩恰好走这条线路,调用因此静默穿过):dsh 的 `compaction-basic` 应对上下文压力时会重放会话前缀、并把固定摘要指令作为最后一条 user 消息发出,此类调用一旦落地,其 checkpoint 会永久覆盖原始历史——不可逆,且摧毁代理的压缩基底。本开关解除该拒绝,让 dsh 原生压缩真正执行。在守卫拒绝期间,dsh 自己的上下文压力计数(原始账本)只增不减——被拒的 checkpoint 永不落地——而 bili 的折叠保持 wire 体积很小,因此会话在客户端仪表显示「已满」之后仍能继续工作,dsh 会不断重新请求压缩(持续被拒)。自 #3384 起,出站输出额度也不再按该原始账本规划——除非 bili 自己的折叠视图已到达强制的 headroom 目标线,或请求的额度达到原生窗口的 25% 及以上(#2490 保护区间);修复前,增长的账本会逐轮悄悄压掉模型的输出额度,在 onlyReasoning 模型上最终表现为无报错的空回复。作用范围:非 web profile 下随附 bundle patch(`auto: false`)仍抑制**自动**触发,因此那里只有手动 `/compact` 受益;web profile 下任何 patch 层都无法按 id 直寻 preset 嵌套行(#1772),因此放行后自动触发照常工作——除非在你的 profile `cordis.patch.yml` 里按 #2474 施加对 preset-standard 的完整快照覆盖。网页配置页提供同一开关;环境变量优先于文件。在配置文件的 `"dsh"` 段下设 `"allowDshCompaction": true` 效果相同(旧文件里裸写在顶层的 `"allowDshCompaction"` 会在加载时自动迁移到该位置)。 | | `BILI_ALLOW_DSH_COMPACTION` | 设为 `1` 放行 dsh 内置自动压缩(#2028)。默认由 wire 级守卫(#1729)**在本地拒绝 dsh 原生压缩调用**(422),覆盖 bili 服务的全部线路——openai、anthropic、responses(responses 由 #2360 补上:此前的协议白名单在检查标记之前就把 responses 短路了,而 dsh 桌面端的压缩恰好走这条线路,调用因此静默穿过):dsh 的 `compaction-basic` 应对上下文压力时会重放会话前缀、并把固定摘要指令作为最后一条 user 消息发出,此类调用一旦落地,其 checkpoint 会永久覆盖原始历史——不可逆,且摧毁代理的压缩基底。本开关解除该拒绝,让 dsh 原生压缩真正执行。作用范围:bili 开启时,dsh 原生自动压缩现已在所有平面关闭 —— 宿主平面行经随附 bundle patch(`auto: false`)、web profile 的 preset 实例经 #3354 加载期 overlay —— 因此该开关主要治理非 web profile 的手动 `/compact`(那里的自动触发仍被 `auto: false` 抑制),以及 #3354 overlay 不适用的 dsh 构建上残留的自动触发。网页配置页提供同一开关;环境变量优先于文件。在配置文件的 `"dsh"` 段下设 `"allowDshCompaction": true` 效果相同(旧文件里裸写在顶层的 `"allowDshCompaction"` 会在加载时自动迁移到该位置)。 | | `BILI_NO_CACHE_CONTROL` | 设为 `1` 关闭 bili 在 Anthropic 通道上的 `cache_control` 断点标注(#1637,随 #1639 落地)。默认开启:Anthropic 系上游只缓存被显式打断点的内容(每请求最多 4 个,按 system + tools + 消息块合并计数),因此 bili 会标注 system 块加上至多 3 个累积消息断点——被标注的消息保持标注(前缀字节稳定),断点只随折叠消亡,最近 3 条稳定消息承载推进前沿。客户端自设的任何 `cache_control`(消息块、tools 条目)都会完全抑制 bili 的标注——客户端自管缓存优先。断点随会话持久化。本开关是逃生阀:用于拒绝该字段的上游或有自己断点策略的中继。仅限 plain-proxy Anthropic 通道;OpenAI/Responses 通道隐式缓存,从不标注。配置文件对应项:[`compat.noCacheControl`](#compat) —— 环境变量优先。 | | `BILI_CHAIN_CONTENT` | 设为 `1` 开启 bili→bili 链感知的 ACP 产物 / `` 检查点**正文内容**检测(#1086/#1421):当入站请求的正文携带压缩产物(渲染标签 / 历史 `acp_status`+`search_context` 工具调用)或带摘要的检查点,但既无 `x-bili-hop` 头、本实例也无该会话的压缩状态时,bili 记录一次告警性观察和/或应用「首个处理器优先」透传。**默认关闭**(#1683 后续):默认下只有 `x-bili-hop` 头驱动链识别,因为扫描请求正文可能把 CCR/文件引入的文本和模型回声标签误判为真实标记。仅在中间盒子剥掉 `x-bili-hop`、且你接受该误判风险的狭窄多 bili 中继场景下才启用。配置文件中设 `"chainContentDetection": true` 效果相同;环境变量优先。`x-bili-hop` 信号本身不受此开关影响。 | diff --git a/src/server/budget.ts b/src/server/budget.ts index 6bb7b6a8eb..265ac8e372 100644 --- a/src/server/budget.ts +++ b/src/server/budget.ts @@ -15,6 +15,20 @@ const OUTPUT_CLAMP_MIN_MARGIN = 2048; const OUTPUT_CLAMP_FLOOR = 1024; // #3071/#2490: capability floor for the clamped budget — see clampOutputBudget. const OUTPUT_CLAMP_MIN_OUTPUT = 4096; +// #3384: giant-request line for applying the refused-dsh-ledger planning floor +// (#2490). A requested output budget at or above this fraction of the native +// window can push a near-wall raw ledger past the window in ONE turn — the +// #2490 shape (384K requested against a 1M window / ~900K ledger produced the +// 7.5MB un-foldable turn). Below the line no single-turn output can overflow +// the ledger, so applying the floor there only reserves output budget against +// bytes bili never sends (its folds keep the wire small while the client's raw +// ledger grows because compaction is refused) — the #3384 starvation: folded +// wire at 8–11% of a 1M window, budget clamped 50670 -> 5909, every byte eaten +// by thinking on an onlyReasoning model -> stopReason=length -> blank turns. +// Aligned with the #896 default outputHeadroomMaxPct (0.25) and the #2096 note +// below ("when max_tokens > 25% of the window the overflow boundary lies BELOW +// the headroom target"). +const OUTPUT_LEDGER_FLOOR_GIANT_REQUEST_PCT = 0.25; // #453 mitigation: host-side escalation line. The kernel already force-injects // every turn once usage >= nudge.maxContextLimitPct (its pressure branch has no // cadence gate), so the only cadence-silent zone is BELOW that line. Force the @@ -282,7 +296,10 @@ export function clampOutputBudget(requested: number, inputEstimate: number, nati * chain. Lifetime ends with the session (metadata eviction). Known * conservative residue: flipping allowDshCompaction on mid-session lets * checkpoints land and shrink the ledger, but the floor stays at its old - * high until eviction — fail-closed over-tightening, accepted. */ + * high until eviction — fail-closed over-tightening, accepted. + * Note (#3384): recording stays sticky; what changed is APPLICATION — the + * clamp gates where this floor may drive planning (see clampOutgoingOutput), + * so a stale high-water mark can no longer starve a healthy session. */ export function dshLedgerFloorTokens(metadata: Record | undefined | null): number { const bytes = metadata?.["dshCompactionRefusedBytes"]; if (typeof bytes !== "number" || !Number.isFinite(bytes) || bytes <= 0) return 0; @@ -341,13 +358,18 @@ export function clampOutgoingOutput( * window the overflow boundary lies BELOW the headroom target, so capping * against it would no-op exactly when post-compression turns need the * guarantee most (#453). */ headroomWindow?: number; - /** #2490: floor for the PLANNING input when the host (dsh native compaction) - * keeps a raw ledger bili's folded view cannot see. The guard's refusals - * record the replayed envelope size on the session; that ledger will be - * re-sent whole after this turn (and after every turn — the compaction - * never lands), so a turn planned only against the folded input invites - * output-sized overhang the next raw replay cannot survive. Raising the - * planning basis only ever tightens the cap (fail-closed direction). */ + /** #2490: floor for the PLANNING input when the host (dsh native compaction) + * keeps a raw ledger bili's folded view cannot see. The guard's refusals + * record the replayed envelope size on the session; that ledger will be + * re-sent whole after this turn (and after every turn — the compaction + * never lands), so a turn planned only against the folded input invites + * output-sized overhang the next raw replay cannot survive. Raising the + * planning basis only ever tightens the cap (fail-closed direction). + * #3384: APPLIED ONLY WHEN GATED — the floor engages iff the folded view + * already sits at the enforced headroom target OR the requested budget is + * giant (>= OUTPUT_LEDGER_FLOOR_GIANT_REQUEST_PCT of the window); below + * both lines it would reserve output budget against bytes never sent and + * starve healthy sessions (see clampOutgoingOutput). */ ledgerFloorTokens?: number; /** #2122/#1933 F1: per-route estimator calibration — scales the local-est * side of estimateInputTokens so the clamp judges the payload on the same @@ -373,7 +395,24 @@ export function clampOutgoingOutput( // internal max() (usage baseline never scales DOWN), so all three calibers stay // conservative in the same direction. const ledgerFloor = typeof ctx.ledgerFloorTokens === "number" && Number.isFinite(ctx.ledgerFloorTokens) ? Math.max(0, Math.floor(ctx.ledgerFloorTokens)) : 0; - const planningInput = Math.max(inputEstimate, ledgerFloor); + // #3384: the floor only applies where it can still defend something — two + // independent grounds, either suffices: + // - viewAtEnforcedTarget: bili's OWN folded view has reached the + // headroom-adjusted target nudge/preflight enforce — the session really + // is at the wall, so planning against the worst-case ledger is right; + // - giantRequestedBudget: the requested output alone can push a near-wall + // ledger past the window in one turn (the #2490 shape). + // Otherwise the floor is pure over-reservation: the client's raw ledger only + // grows because bili refuses compaction, while the wire bili actually sends + // stays small via its folds — reserving output budget against bytes never + // sent starves the session's own replies (#3384: folded wire 8–11% of a 1M + // window, budget clamped to 5909, all eaten by thinking -> blank turns). + // Both pinned protection regimes survive: the #2490 incident request (384K + // >= 25% of 1M) takes giantRequestedBudget; near-wall sessions take + // viewAtEnforcedTarget. + const viewAtEnforcedTarget = typeof ctx.headroomWindow === "number" && Number.isFinite(ctx.headroomWindow) && ctx.headroomWindow > 0 && inputEstimate >= ctx.headroomWindow; + const giantRequestedBudget = ctx.nativeWindow > 0 && raw >= OUTPUT_LEDGER_FLOOR_GIANT_REQUEST_PCT * ctx.nativeWindow; + const planningInput = viewAtEnforcedTarget || giantRequestedBudget ? Math.max(inputEstimate, ledgerFloor) : inputEstimate; const capped = clampOutputBudget(raw, planningInput, ctx.nativeWindow); if (capped !== undefined) { writeOutputBudget(rebuilt, field, capped); diff --git a/tests/issue3148-upstream-circuit.test.ts b/tests/issue3148-upstream-circuit.test.ts index 9fd79d8869..edc145beef 100644 --- a/tests/issue3148-upstream-circuit.test.ts +++ b/tests/issue3148-upstream-circuit.test.ts @@ -124,15 +124,14 @@ test("#3148 review: an UNCOUNTED probe failure (timeout/client abort) must not p }); test("#3148: clampOutgoingOutput records the exact clamp event without changing the wire decision", () => { - const rebuilt: Record = { max_tokens: 65536 }; + const rebuilt: Record = { max_tokens: 300_000 }; let seen: { raw: number; capped: number; window: number; planningInput: number; ledgerFloor: number } | undefined; - // Reproduces the reporter's exact numbers: floor 997348, window 1048576 → - // margin ceil(997348*0.05)=49868, cap floor(1048576-997348-49868)=1360 — - // then #3071's capability floor lifts the cap to max(1360, 4096)=4096 - // (bounded by raw headroom floor(1048576-997348)=51228, so input+output - // <= window still holds). Pinned post-#3071; this expectation went stale - // when the two PRs landed in the same window — master push CI skips the - // test matrix, so the mismatch surfaced only on the next PR. + // The reporter's numbers (floor 997348, window 1048576 -> margin + // ceil(997348*0.05)=49868, cap floor(1048576-997348-49868)=1360, lifted to + // 4096 by #3071's capability floor bounded by raw headroom 51228) are kept; + // #3384 re-gated the floor, so the request is raised to 300K >= + // 0.25*1048576=262144 to take the giant-request ground and keep the hook + // path covered. clampOutgoingOutput( rebuilt, "max_tokens", @@ -150,9 +149,35 @@ test("#3148: clampOutgoingOutput records the exact clamp event without changing () => {}, (info) => { seen = info; }, ); - assert.deepEqual(seen, { raw: 65536, capped: 4096, window: 1048576, planningInput: 997348, ledgerFloor: 997348 }, "exact clamp event captured"); + assert.deepEqual(seen, { raw: 300_000, capped: 4096, window: 1048576, planningInput: 997348, ledgerFloor: 997348 }, "exact clamp event captured"); assert.equal(rebuilt.max_tokens, 4096, "wire decision unchanged (same cap as before the hook)"); + // #3384: the OLD expectation here (modest 65536 request against this same + // floor clamps to 4096) IS the starvation regression — below both gate + // lines the floor must not drive planning, or a healthy session (folded + // wire ~9% of the window) loses its output budget to bytes never sent. + const modest: Record = { max_tokens: 65_536 }; + let modestCalls = 0; + clampOutgoingOutput( + modest, + "max_tokens", + { + systemText: "", + tools: [], + processedMessages: [], + lastInputTokens: 97730, + lastInputTokensSource: "usage", + nativeWindow: 1048576, + imageTokens: 0, + ledgerFloorTokens: 997348, + }, + "t", + () => {}, + () => { modestCalls++; }, + ); + assert.equal(modestCalls, 0, "#3384: no clamp event — the floor is gated off for a modest request"); + assert.equal(modest.max_tokens, 65_536, "#3384: budget untouched while the folded view is healthy"); + const healthy: Record = { max_tokens: 65536 }; let healthyCalls = 0; clampOutgoingOutput( diff --git a/tests/issue3384-ledger-floor-starvation.test.ts b/tests/issue3384-ledger-floor-starvation.test.ts new file mode 100644 index 0000000000..e8bd7ecf4c --- /dev/null +++ b/tests/issue3384-ledger-floor-starvation.test.ts @@ -0,0 +1,275 @@ +// #3384: the #2490 refused-ledger planning floor was applied UNCONDITIONALLY — +// bili planned every outgoing output budget against the client's raw ledger +// even though its own folds kept the wire small. In the reported session +// (window 1M, DSH profile, default config) the ledger climbed 832K -> 947K +// while the folded wire sat at 8–11%, so the clamp shrank the requested +// budget 128000 -> 5909 over four turns; the model was onlyReasoning / +// canDisableThinking:false, so each clamped budget was eaten entirely by +// thinking -> upstream stopReason=length -> no text, no tool-call, blank GUI +// turns, and DSH (which does not auto-continue on max-tokens) just replayed +// the same dead loop on every "continue". Fix: the floor now engages only on +// one of two independent grounds — the folded view already sits at the +// enforced headroom target, or the requested budget is giant (>= 25% of the +// native window, the #2490 monster-request line). Both protection regimes +// stay pinned here AND in tests/issue2490-ledger-floor-doomed-probe.test.ts +// (giant request) — this file pins the starvation side they left open. +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import test from "node:test"; + +process.env.NODE_ENV = "test"; + +import { defaultConfig } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import type { ProxyOptions } from "../src/config.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { getSession } from "../src/session.ts"; +import { clampOutgoingOutput } from "../src/server/budget.ts"; +import { DSH_COMPACTION_INSTRUCTION_PREFIX } from "../src/server/dsh-compaction-guard.ts"; +import { rmrf } from "./tmp-rm.ts"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; + +const WINDOW = 1_000_000; // the reported deployment's declared window +const FLOOR = 946_753; // the incident's final planning floor (refused replay bytes / 4) + +type CtxArgs = Parameters[2]; + +// Mirrors the incident log line "output budget clamped 50670 -> 5909 +// (input~89699.41..., planning floor 946753 ..., window=1000000)": folded +// wire ~89.7K (9% of the window), ledger floor 946.7K, modest request. +const incidentCtx = (over: Partial): CtxArgs => ({ + systemText: "", + tools: undefined, + processedMessages: [{ role: "user", text: "hello" } as never], + lastInputTokens: 89_699, + lastInputTokensSource: "usage", + nativeWindow: WINDOW, + headroomWindow: WINDOW, // real dsh deployments run headroom ~= declared window + imageTokens: 0, + ...over, +}) as CtxArgs; + +const clampLog = (lines: string[]): ((level: string, msg: string) => void) => + (level, msg) => { if (level === "info" || level === "warn") lines.push(msg); }; + +test("#3384: the incident shape keeps its full budget — modest request, healthy folded view, huge refused ledger", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 50_670 }; + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-a", clampLog(lines)); + // Pre-fix this clamped to 5909 (margin ceil(946753*0.05)=47338, cap + // floor(1000000-946753-47338)=5909) — the starvation the session died in. + assert.equal(rebuilt.max_tokens, 50_670, "budget untouched below both gate lines"); + assert.equal(lines.length, 0, "no clamp, no log"); +}); + +test("#3384: the giant-request ground still plans against the ledger (#2490 regime preserved)", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 384_000 }; + // 384K >= 0.25 * 1M — the #2490 incident's own request size. + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-b", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 5_909, "planned against the refused ledger: cap floor(1M-946753-47338)=5909, above the 4096 capability floor"); + assert.match(lines[0] ?? "", /planning floor 946753 from the refused dsh compaction replay \(#2490\)/); +}); + +test("#3384: the boundary is inclusive — exactly 25% of the window takes the giant ground", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 250_000 }; + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-c", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 5_909, "raw == 0.25*window engages the floor (fail-closed side of the line)"); + assert.ok(lines.length >= 1, "the clamp is logged"); +}); + +test("#3384: one token under the giant line stays unclamped while the view is healthy", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 249_999 }; + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-d", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 249_999, "below the line the ledger must not drive planning"); + assert.equal(lines.length, 0); +}); + +test("#3384: the view-at-enforced-target ground engages when bili's OWN view reaches the wall", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 50_670 }; + // Folded view 110K already past an enforced 100K headroom target — the + // session really is at the wall, so worst-case-ledger planning applies + // even for a modest request. + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ headroomWindow: 100_000, lastInputTokens: 110_000, ledgerFloorTokens: FLOOR }), "s-3384-e", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 5_909, "view at the enforced target re-engages the floor"); + assert.match(lines[0] ?? "", /planning floor 946753/); +}); + +test("#3384: legacy paths stay byte-identical — no refusal recorded, or floor below the folded view", () => { + const legacy: Record = { max_tokens: 50_670 }; + clampOutgoingOutput(legacy, "max_tokens", incidentCtx({}), "s-3384-f", clampLog([])); + assert.equal(legacy.max_tokens, 50_670, "no floor recorded -> unchanged legacy behavior"); + + const small: Record = { max_tokens: 50_670 }; + clampOutgoingOutput(small, "max_tokens", incidentCtx({ ledgerFloorTokens: 10_000 }), "s-3384-g", clampLog([])); + assert.equal(small.max_tokens, 50_670, "a floor smaller than the folded view changes nothing"); +}); + +// --------------------------------------------------------------------------- +// e2e: refuse a compaction envelope (records the ledger high-water), then a +// normal turn with a modest budget must go out UNCLAMPED. Pre-fix the second +// turn's max_tokens came back ~8056 instead of 20000 (floor ~87.5K against a +// 100K window), which is exactly how the reported session starved. +// --------------------------------------------------------------------------- + +const E2E_WINDOW = 100_000; + +function chatSse(text: string, usage?: { prompt_tokens: number; completion_tokens: number }): string { + const chunk = (delta: Record, finish: string | null, extra?: Record): string => + `data: ${JSON.stringify({ id: "chatcmpl-1", object: "chat.completion.chunk", created: 1, model: "gpt-test", choices: [{ index: 0, delta, finish_reason: finish }], ...extra })}\n\n`; + const u = usage ? { usage } : {}; + return chunk({ role: "assistant" }, null) + chunk({ content: text }, null) + chunk({}, "stop", u) + "data: [DONE]\n\n"; +} + +type Call = { maxTokens: number | null; contentChars: number }; + +function makeUpstream(): { server: http.Server; calls: Call[] } { + const calls: Call[] = []; + const server = http.createServer((req, res) => { + const chunks: Buffer[] = []; + req.on("data", (c: Buffer) => chunks.push(c)); + req.on("end", () => { + const raw = Buffer.concat(chunks).toString("utf8"); + try { + const parsed = JSON.parse(raw) as { messages?: Array<{ content?: unknown }>; max_tokens?: number }; + const chars = (parsed.messages ?? []).reduce((n, m) => n + (typeof m.content === "string" ? m.content.length : 0), 0); + calls.push({ maxTokens: typeof parsed.max_tokens === "number" ? parsed.max_tokens : null, contentChars: chars }); + res.writeHead(200, { "content-type": "text/event-stream" }); + res.end(chatSse("forwarded answer", { prompt_tokens: Math.max(1, Math.round(chars / 40)), completion_tokens: 3 })); + } catch { + res.writeHead(400, { "content-type": "application/json" }); + res.end('{"error":{"message":"unparseable"}}'); + } + }); + }); + return { server, calls }; +} + +function proxyOptions(routes: Record): ProxyOptions { + return { + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: routes as ProxyOptions["routes"], + modelContextLimit: E2E_WINDOW, + kernelConfig: defaultConfig(E2E_WINDOW, { preserveRecentMessages: 2, preserveRecentTokens: 2000 }), + compress: { injectTool: true, injectNudge: true }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + autoUpdate: false, + compat: { roles: {} }, + streamErrorShape: "protocol", + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + advisoryCheck: false, + releaseNotesCheck: false, + promptCache: { routing: "auto" }, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions; +} + +type Ctx = { url: string; headers: Record }; + +async function setup(): Promise<{ ctx: Ctx; calls: Call[]; close: () => Promise }> { + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const { server: upstream, calls } = makeUpstream(); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as { port: number }).port; + const proxy = await startServer(proxyOptions({ [`http://127.0.0.1:${upstreamPort}`]: { models: { "gpt-test": { context: E2E_WINDOW } } } })); + await once(proxy, "listening"); + const proxyPort = (proxy.address() as { port: number }).port; + return { + ctx: { + url: `http://127.0.0.1:${proxyPort}/bili/http://127.0.0.1:${upstreamPort}/chat/completions`, + headers: { "content-type": "application/json", "x-acp-session": "" }, + }, + calls, + close: async () => { + proxy.close(); + proxy.closeIdleConnections?.(); + upstream.close(); + upstream.closeIdleConnections?.(); + await Promise.allSettled([once(proxy, "close"), once(upstream, "close")]); + }, + }; +} + +test("e2e #3384: after a refused compaction envelope, a modest-budget turn goes out unclamped", async () => { + const root = mkdtempSync(path.join(tmpdir(), "bili-3384-")); + const cfgFile = path.join(root, "billion-context.json"); + writeFileSync(cfgFile, '{"providers":{}}\n', "utf8"); + const prevConfig = process.env.BILI_CONFIG_FILE; + process.env.BILI_CONFIG_FILE = cfgFile; + const { ctx, calls, close } = await setup(); + try { + ctx.headers["x-acp-session"] = "s-3384-e2e"; + const postRaw = (body: string) => fetch(ctx.url, { method: "POST", headers: { ...ctx.headers }, body }); + + // Turn 1: seed the session with a small exchange. + let r = await postRaw(JSON.stringify({ model: "gpt-test", stream: true, messages: [{ role: "user", content: "hello" }] })); + assert.equal(r.status, 200, "seed turn accepted"); + await r.text(); + + // The dsh compaction envelope (~350KB of core text -> ledger floor + // ~87.5K tokens): the guard refuses it (default posture) and records + // the high-water byte count. The size is chosen so the pre-fix cap + // floor(100000 - 87.5K - ceil(87.5K*0.05)) = ~8056 sits BELOW the 20K + // request — this assertion only discriminates in that band. + const envelope = JSON.stringify({ + model: "gpt-test", + stream: true, + messages: [ + { role: "system", content: "sys" }, + { role: "user", content: `${DSH_COMPACTION_INSTRUCTION_PREFIX} ${"x".repeat(350_000)}` }, + ], + }); + r = await postRaw(envelope); + assert.equal(r.status, 422, "guard refuses the compaction call (default)"); + await r.text(); + const s = getSession("s-3384-e2e"); + assert.ok(s, "session exists"); + assert.equal(s.metadata.dshCompactionRefused, true, "refusal flag set"); + const floor = Math.ceil(Buffer.byteLength(envelope) / 4); + assert.equal(s.metadata.dshCompactionRefusedBytes, Buffer.byteLength(envelope), "high-water bytes recorded"); + assert.ok(floor > E2E_WINDOW * 0.25 && floor < E2E_WINDOW, `floor ${floor} sits in the discriminating band (25%, window) of a ${E2E_WINDOW} window`); + + // Turn 2: a normal short turn with a MODERATE budget (20K < 25K = the + // giant line). Pre-fix the planning basis became the ~87.5K floor and + // this went out clamped to ~8056 — the reported session's death spiral + // in miniature. Post-fix the wire is what it is (~a few hundred + // tokens), so the full 20K rides out. + r = await postRaw(JSON.stringify({ + model: "gpt-test", + stream: true, + max_tokens: 20_000, + messages: [ + { role: "user", content: "hello" }, + { role: "assistant", content: "forwarded answer" }, + { role: "user", content: "and now?" }, + ], + })); + assert.equal(r.status, 200, "normal turn accepted"); + await r.text(); + + const forwarded = calls[calls.length - 1]; + assert.ok(forwarded, "the normal turn reached the upstream"); + assert.equal(forwarded.maxTokens, 20_000, "#3384: the refused-ledger floor did not starve a modest budget on a healthy wire"); + } finally { + if (prevConfig === undefined) delete process.env.BILI_CONFIG_FILE; + else process.env.BILI_CONFIG_FILE = prevConfig; + await close(); + rmrf(root); + } +});