You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453
14:19:34–14:20:49 nudge INJECT T1 reason="OVER-LIMIT T1" usage 77% → 85% — still no compress
14:27:39 upstream rejects: 400 Requested token count exceeds the model's maximum context length of 262144 tokens. You requested a total of 262817 tokens: 131745 tokens from the input messages and 131072 tokens for the completion.
The client (pi) then falls back to its own native compaction → the model loses all working state and re-derives its analysis from scratch (compounding the perceived "loop" in #14). A second session (01a062a7, qwen3.8-flash) sat at 80–85% usage with nudge INJECT T1 firing every turn — the model never called compress at all.
Impact
In plugin mode the nudge is advisory: bili trusts the ACP-native agent to execute compress. When the model simply ignores the injected nudge (weaker instruction-following, e.g. qwen3.8-27b), nothing stops the payload from overflowing the window. Proxy mode has preflight/preflightCompressIfNeeded as the backstop; plugin mode has none.
Suggested fix (discussion)
Options, roughly in ascending invasiveness:
Clamp the outgoing output budget: max_tokens/max_completion_tokens = min(requested, window − estimated input − margin) so input+output can never exceed the window on request-rebuilding upstreams (vLLM rejects instead of clamping). Cheap, no wire changes; doesn't fix input-overflow itself.
Proxy-side emergency truncation: when estimated input tokens exceed the window and the agent hasn't compressed after N over-limit nudges, drop/truncate the oldest inessential history server-side for that request (like proxy-mode preflight) even though the session is plugin-mode. Needs care to keep the agent's view consistent.
Escalating nudge: make the over-limit nudge increasingly forceful / repeat every turn once usage > 85% instead of cadence-blocking (the cadence block kept nudges silent for 5 minutes in the repro while usage grew 56%→70%).
(3) is probably the right first step; (2) needs a design pass.
Environment
bili 0.1.63 (dev build of master@7135709), plugin mode via bili pi, upstream vLLM private host, MITM tunnel
Traced this through the code (master@7135709) rather than taking the log at face value. The diagnosis holds, but the root cause sits one layer deeper than the framing — and it changes which fix is the real backstop.
What I verified
Plugin mode delegates all compression to the agent. Nudge injection is gated !pluginMode (src/server.ts:1452/1556/1721) and the response passes through verbatim (src/server.ts:2538). The proxy computes the nudge decision, exposes it via /__bili/plugin/status, then waits for the agent to call compress. If the model ignores it, nothing the proxy does folds history server-side.
The server-side backstop exists in BOTH modes.preflightCompressIfNeeded is called unconditionally (src/server.ts:1268), not gated on pluginMode. So "plugin mode has none" isn't quite right — it has preflight. The problem is preflight's trigger, below.
There is no clamp on the outgoing max_tokens. The client's value passes through untouched; the only max_tokens writes in src/ are preflight's internal 8192-budgeted summarization sub-calls (src/preflight.ts:143-148).
Root cause — this is an input+OUTPUT overflow, not input-alone
The 400 is 131745 (input) + 131072 (completion) = 262817 > 262144. Input alone (131745) is well under the window. Two things conspire:
Preflight's trigger is tokenCount = max(lastInputTokens, payloadEstimate) >= limit (src/server.ts:2179), where payloadEstimate = estimateCoreMessages(...) counts message text only — it excludes the system prompt + tool definitions (src/preflight.ts:90-94). So the trigger undershoots the true input by the fixed system+tools overhead (~10–20k here).
On non-Anthropic wires the effective limit is already reduced by the output-headroom reservation to window − maxOutput (src/server.ts:1231-1246) — exactly the overflow boundary. Preflight is meant to fire the moment input crosses window − maxOutput, but because the estimate undershoots, the true input crosses the line while the trigger is still just below it → no preflight → the request goes out → vLLM rejects input+output.
In plugin mode this is maximally exposed: the reactive nudge/truncate path is handed to the agent, and qwen3.8-27b never calls compress, so preflight is the only thing between the session and the 400 — and it misses by the undershoot margin. The cadence block (56→70%, silent) is a secondary contributor: it kept the advisory nudge quiet during the fastest growth phase.
Assessment of the three options
(1) Clamp the outgoing output budget — max_tokens = min(requested, window − estimated_input − margin). This is the actual hard backstop for the input+output class and the only option that makes the overflow impossible on request-rebuilding upstreams, because the proxy controls both sides of the sum. Fixes the exact repro. Applies to both modes. Residual risk only if the input estimate undershoots more than the margin — close that by counting system+tools into the estimate (or a conservative margin). Anthropic stays exempt (its input limit is enforced independently of max_tokens).
(3) Escalating nudge — closes the silent cadence-gap and raises compliance odds, but does not guarantee: the repro shows per-turn OVER-LIMIT T1 nudges at 77–85% were still ignored. A mitigation, not a backstop.
(2) Proxy-side emergency truncation in plugin mode — the definitive backstop, but the consistency hazard is real and larger than noted: the plugin-mode summary carrier is the agent's own compress tool call, whereas preflight builds tool-call-less blocks whose acp_summary carrier is re-voiced as a user message via systemToUser (a proxy-mode semantic). Folding behind the agent's back risks it re-sending unfolded ranges next turn, and preflight's carrier handling was never validated for plugin mode. Genuinely needs the design pass you flagged.
Recommendation (staged, complementary — not competing)
Ship (1) first — small, pure-safety (can only ever lower max_tokens), both modes, kills the reported 400 outright.
Ship (3) alongside — stop cadence-blocking above the emergency threshold so weak models get per-turn pressure instead of a 5-min silence.
File (2) as a separate follow-up issue for the design pass (agent-view consistency + carrier semantics).
Happy to implement (1)+(3) now and open a PR — confirm the staging (or tell me to reorder/split) and I'll go.
Fixed by #467 (merged c2f0229): tier-1 max_tokens clamp (clampOutputBudget/estimateInputTokens, closes the input+output>window 400) + tier-2 host-side emergency nudge at 0.7. Tier-3 (proxy-side emergency truncation) remains tracked in #470 as intentionally deferred. Closing this as fixed — remaining work lives in #470.
Activity
ranxianglei commentedon Sep 2, 2026
Traced this through the code (master@7135709) rather than taking the log at face value. The diagnosis holds, but the root cause sits one layer deeper than the framing — and it changes which fix is the real backstop.
What I verified
!pluginMode(src/server.ts:1452/1556/1721) and the response passes through verbatim (src/server.ts:2538). The proxy computes the nudge decision, exposes it via/__bili/plugin/status, then waits for the agent to callcompress. If the model ignores it, nothing the proxy does folds history server-side.preflightCompressIfNeededis called unconditionally (src/server.ts:1268), not gated onpluginMode. So "plugin mode has none" isn't quite right — it has preflight. The problem is preflight's trigger, below.max_tokens. The client's value passes through untouched; the onlymax_tokenswrites insrc/are preflight's internal 8192-budgeted summarization sub-calls (src/preflight.ts:143-148).Root cause — this is an input+OUTPUT overflow, not input-alone
The 400 is
131745 (input) + 131072 (completion) = 262817 > 262144. Input alone (131745) is well under the window. Two things conspire:tokenCount = max(lastInputTokens, payloadEstimate) >= limit(src/server.ts:2179), wherepayloadEstimate = estimateCoreMessages(...)counts message text only — it excludes the system prompt + tool definitions (src/preflight.ts:90-94). So the trigger undershoots the true input by the fixed system+tools overhead (~10–20k here).window − maxOutput(src/server.ts:1231-1246) — exactly the overflow boundary. Preflight is meant to fire the moment input crosseswindow − maxOutput, but because the estimate undershoots, the true input crosses the line while the trigger is still just below it → no preflight → the request goes out → vLLM rejectsinput+output.In plugin mode this is maximally exposed: the reactive nudge/truncate path is handed to the agent, and qwen3.8-27b never calls
compress, so preflight is the only thing between the session and the 400 — and it misses by the undershoot margin. The cadence block (56→70%, silent) is a secondary contributor: it kept the advisory nudge quiet during the fastest growth phase.Assessment of the three options
max_tokens = min(requested, window − estimated_input − margin). This is the actual hard backstop for the input+output class and the only option that makes the overflow impossible on request-rebuilding upstreams, because the proxy controls both sides of the sum. Fixes the exact repro. Applies to both modes. Residual risk only if the input estimate undershoots more than the margin — close that by counting system+tools into the estimate (or a conservative margin). Anthropic stays exempt (its input limit is enforced independently ofmax_tokens).OVER-LIMIT T1nudges at 77–85% were still ignored. A mitigation, not a backstop.compresstool call, whereas preflight builds tool-call-less blocks whoseacp_summarycarrier is re-voiced as a user message viasystemToUser(a proxy-mode semantic). Folding behind the agent's back risks it re-sending unfolded ranges next turn, and preflight's carrier handling was never validated for plugin mode. Genuinely needs the design pass you flagged.Recommendation (staged, complementary — not competing)
max_tokens), both modes, kills the reported 400 outright.Happy to implement (1)+(3) now and open a PR — confirm the staging (or tell me to reorder/split) and I'll go.
ranxianglei commentedon Sep 2, 2026
在当前 master(
ac4fbcb,已含 #455/#456/#452)上逐条复核了楼上的分析,全部属实,补充精确代码位置:(a) preflight 双模式无条件运行 ✅
server.ts:1273(在!countTokens && !responsesCompact内,不受 pluginMode 门控)。(b) 转发请求无 max_tokens 钳制 ✅ 全代码库唯一设置 max_tokens 的地方是 preflight 内部子调用(
src/preflight.tssummaryPayload,固定MAX_SUMMARY_OUTPUT_TOKENS);主请求 body 原样透传。server.ts:1236-1250的reserveOutputHeadroom只降低内部波段窗口(reqConfig.modelContextLimit= window − maxOutput),它影响 nudge/truncate 决策,不影响实际转发出去的请求。(c) 触发下冲 → input+output 400 ✅
server.ts:2199-2200:tokenCount = max(session.stats.lastInputTokens, estimateCoreMessages(prepared.processedMessages))而
src/preflight.ts:90-95的estimateCoreMessages只累加消息文本(defaultCountTokens(m.text)),不含 system+tools(约 10-20k)。复现数字完全对上:真实 input 131745 > 保留限 131072,但 payloadEstimate ≈ 117k、lastInputTokens 又滞后一轮 → 双双低于限 → preflight 不触发 →131745 + 131072 = 262817 > 262144→ vLLM 直接 400(拒绝而非截断)。次因 cadence 消音可完全 host 侧修复:
NudgeDecision携带contextUsage/reason/tier,且renderNudgeText(decision, prompts)不依赖shouldInject(server.ts:1447/1547/1707三处注入点都是 host 侧判断)→ 紧急升级(usage ≥ 阈值时绕过 cadence)不需要动 acp-kernel。过时前提:本 issue 开头描述的 "nudge 被
!pluginMode门控" 是 0.1.69 的状态,#456(v0.1.70)已恢复 plugin 模式 nudge 注入。当前剩余问题 = 触发下冲 + 无 max_tokens 钳制 + cadence 消音三件事。建议按楼上分级落地:
util.ts:222shouldReserveOutputHeadroom),在 prepare* 重建 body 时max_tokens = min(请求值, window − 有效输入估算 − margin)。这是唯一让 input+output 溢出不可能发生的选项,双模式受益。server.ts三处判断,零 kernel 改动。维护者确认分级后我来实现 1+2。
ranxianglei commentedon Sep 2, 2026
[bot] 🏷 已按确认的分级实现 1 + 2,PR: #467 (基于
ac4fbcb,未动version)。1. max_tokens 钳制(硬兜底) —
prepareOpenai/prepareResponses重建 body 时把max_tokens/max_completion_tokens/max_output_tokens压到window − 输入估算 − margin。estimateInputTokens()取「上轮真实 input」与「消息 + system + tools 现算」的较大值,补上了 preflight 估算漏掉 system/tools 的下冲。Anthropic 豁免(输入预算独立)。cap 低于地板(输入本身快撑满窗口)时不动,交给 preflight/self-heal——不饿死输出。这是唯一让 input+output 溢出不可能发生的一层,双模式受益。2. 紧急升级 nudge(缓解) — host 侧在 usage ≥ 阈值且被 kernel cadence/growth 消音时强制注入,三处判断各 OR 一个
emergencyNudge(),零 kernel 改动;diagNudge现在标INJECT-ESC,不再谎报 idle。decideNudge,发现 pressure 分支(usage ≥nudge.maxContextLimitPct,默认 0.75)没有 cadence 门——kernel 在那之上本来就每轮注入。所以唯一被 cadence 消音的区间在 0.75 之下;issue 里说的「≥85%」落在 kernel 已在注入的区间里,是空操作。因此我把升级线设在 0.70(压在 0.75 正下方,补 pre-limit 静默爬升那段),是单个命名常量,改起来一行。想要别的值、或做成三级 compress 可配,说一声即调。测试:新增
tests/issue453-backstop.test.ts——三个 helper 单测(clamp 触发/适配/floor;estimateInputTokens 真实值主导 + turn1 system+tools;emergencyNudge armed/线/自定义/空-range guard)+ 两个真实 proxy+mock upstream 集成测试(clamp 在大 system 把真实输入顶过保留窗口而 preflight 跳过时确实压低转发 max_tokens,复现本 issue 下冲;escalation 在 72% 恰好多一条尾随 nudge、60% 无)。typecheck ✅ · 全量 924/925(唯一失败是launcher.test.ts在本机解析到已安装的 codex 二进制,属环境性——stash 掉我的改动后同样失败)· build ✅ · e2e preflight ✅(完整 e2e 需活的 Responses upstream → CI/手动)。option 3(plugin 模式代理侧截断)按你的计划未包含——载体语义风险(preflight 建无 tool-call 块 + acp_summary 变 user = proxy 语义,agent 可能重发未折叠区间),留给你另开 issue 单独跟踪。
等你 review / merge。
Merge pull request #467 from ranxianglei/2026-09-03_fix-453-plugin-ov…
ranxianglei commentedon Sep 3, 2026
Fixed by #467 (merged c2f0229): tier-1
max_tokensclamp (clampOutputBudget/estimateInputTokens, closes the input+output>window 400) + tier-2 host-side emergency nudge at 0.7. Tier-3 (proxy-side emergency truncation) remains tracked in #470 as intentionally deferred. Closing this as fixed — remaining work lives in #470.fix(ranxianglei#453): hard backstop for plugin-mode context overflow
fix: count system+tools wire overhead in the preflight trigger (ranxi…
fix: gate the clamp baseline on usage-grade source; pin the gate in #453
rework: arm at the raw view on transform-failure fallback; floor the …
test(wire-contract): add WC-014 — Anthropic rejects input+max_tokens …
fix(ranxianglei#2096): make window-reservation logs honest — [window]…
fix: scale output clamp's local input estimate by per-route calibrati…
fix: gate emergency nudge override on submit-gate feasibility (ranxia…
test: pin judge denominator via declared output ceilings where tests …
test: pin judge denominator via declared output ceilings where tests …
test: pin judge denominator via declared output ceilings where tests …
fix: bound the ranxianglei#546 restore and ranxianglei#453 clamp by t…
fix: identified sessions with zero upstream evidence or a natively-fi…
fix: identified sessions with zero upstream evidence or a natively-fi…
fix(ranxianglei#4438): route dsh auto-review requests off the main se…
fix: size the ranxianglei#453 output clamp on a safety-inflated input…