Repository navigation
Description
Description
A session whose outbound body is dominated by base64 images never arms a shrink, and re-forwards the identical 5 MiB payload until the client gives up. The local meter reports 11 % of a 1 M window (114 471 / 1 000 000) while the very same request is rejected by the relay for being 1 748 713 tokens against a 1 500 000 cap — a 15× disagreement between what bili thinks it is sending and what it is sending.
Repro (ordered)
- Plugin-native install (
bili plugin installintodsh), Windows 11, upstreamdeepseek-v4-flashbehind a relay that declares a 1 500 000-token per-request cap while the model window is 1 000 000. - Drive a long session that accumulates images until
request:logs an outbound body around 5 MiB — in my caseinbound=12.4MiB, outbound=5.0MiBat12:03:55.748. - Send the next turn. The relay answers
400with a body bili never classifies as an overflow.
Before / after the fold
- Before the fold the session is healthy:
nudge idle: usage=23% (230420/1000000)at12:03:08.891. - The fold itself works: 1 271 messages in, 5.0 MiB out; the folded history is genuinely not re-sent (verified offline against
index.db: 1 160 covered messages, 294 kept, 299 on the wire). - After the fold the meter collapses:
nudge idle: usage=11% (114471/1000000), growth=59039/22500 (ref=55432, interval=50000), pendingT1=45535/50000 model=deepseek-v4-flash, reason="max compressible 45535 < threshold 50000". The reported usage dropped from 23 % to 11 % while the actual payload grew — the shrink arm is now convinced there is nothing to compress and refuses to act. - Three consecutive
400s follow (12:04:19.538,12:06:00.588,12:08:11.704), each quoting a payload 30–60 tokens larger than the last (约 1748713/约 1748746/约 1748779 tokens), each against the same1500000cap. The log contains noarmed emergency shrinkline between them. The only fourarmrecords in the whole file arenetwork failure with no usage report, none from an overflow.
Exact error text
请求上下文超限:本次请求约 1748713 tokens,超过单次上限 1500000 tokens。请缩短对话历史或开启上下文压缩后重试(请求未发送到上游,未计费)。 (request id: ...)
Note the parenthetical: the request was never billed. That single fact closes the only self-healing channel bili has (see cause 2).
Measured composition of the rejected body
Offline replay against the session's index.db (repro-context-undercount.mjs, 7/7 assertions pass):
| quantity | value |
|---|---|
| wire body | 5.12 MiB |
| of which base64 | 4.80 MiB (7 image messages) |
| bili's own estimate | 69 870 tokens |
| real size (text + b64/4) | ≈ 1 341 436 tokens |
| undercount factor | 19.2× |
The 7 images on the wire: three JPEGs (329 / 258 / 246 KB) and four PNGs at 1080×480 (687 / 886 / 359 / 918 KB).
Root causes
1. effectiveTokenCount's usage fast path drops inboundImageTokens
src/server.ts:2303
export function effectiveTokenCount(session: Session, msgs: CoreMessage[], inboundImageTokens = 0): { tokens: number; source: "usage" | "estimate" } {
if (session.stats.lastInputTokens > 0 && (session.stats.lastInputTokensSource === "usage" || session.stats.lastInputTokensSource === "overflow-arm")) return { tokens: session.stats.lastInputTokens, source: "usage" };
...
const raw = estimateCoreMessagesUpper(msgs) + inboundImageTokens;raw is the only branch that carries the image reserve, and it sits after the usage fast path. Whenever a usage-grade baseline exists — which is the normal case in a long session — the image channel is discarded outright. inboundImageTokens has exactly three call sites in the whole tree, all in src/server/prepare-anthropic.ts (:208 computing it, :214 feeding projectThinkingMass, :233 passing it here); prepare-openai.ts, prepare-responses.ts and prepare-google.ts each compute imageReserveFor(...) inline and never hand it over. This is the mechanism that freezes the meter at 114 471 while the payload is 1.7 M.
The #1839 comment above the fast path explains why an overflow-arm value rides through unbounded — but it does not cover the case where the images were added after the usage reading the arm is pinned to.
2. The image-billing learning channel cannot learn from a rejection
src/cache-ledger.ts:607 settleImageLearning is the only thing that can replace the pixels prior with a measured per-image cost, and it runs only from settleUsageReport when s.total > 0. A 400 that states 「未计费」 never produces a usage report, so:
learnedImageCostsstays empty for the route,learnedImageReservekeeps returningundefined(src/cache-ledger.ts:627),imageReserveFor(src/server.ts:1647) keeps returning the pixels prior (~7 × 2805) instead of a number anywhere near the 4.8 MiB actually on the wire.
So the one mechanism designed to self-correct the image channel is gated behind the exact event that cannot happen during the failure it needs to correct. This is what makes cause 1 permanent rather than transient.
3. The text-only upper bounds are blind to the base64 sidecar
src/preflight.ts:387 estimateCoreMessagesUpper sums (m.text ?? "").length; :400 estimateCoreMessagesUpperBytes sums Buffer.byteLength(m.text ?? "", "utf8"). The 5 028 024 base64 characters in the rejected body live entirely in the raw block, not in m.text — confirmed in the replay (assertion G). Consequently:
- the doomed-probe guard at
src/server.ts:3124(const coreTextBytes = estimateCoreMessagesUpperBytes(prepared.processedMessages);) sees ~0.3 MiB of core text against a 5.12 MiB body and lets the probe through, by design; outboundContextEstimates(src/server.ts:2846) reports anupperBoundthat excludes the sidecar, sosession.stats.localInputEstimateis not a usable upper bound either.
An "upper bound" that can be 19× below the truth is not an upper bound. Note estimateRawBodyTokens already excludes data / url / b64_json / file_data deliberately (src/preflight.ts:358, images priced separately by imageTokensInParsedBody) — that is correct for the text channel, but the sidecar needs to reach the total, and today it reaches nothing.
4. rescaleImage admits images that doRescale then refuses
src/image-rescale.ts:151 gates on (dims !== undefined && Math.max(dims.w, dims.h) > maxDim) || origBytes > RESCALE_TRIGGER_BYTES; — so an oversized-by-bytes image passes. doRescale (:113) then early-returns:
if (scale >= 1 && (!force || origBytes <= RESCALE_TRIGGER_BYTES)) return null;force is never passed from rescaleImage, so any image whose dimensions fit inside maxDim returns null no matter how many bytes it is. In this session the four 1080×480 PNGs (359–918 KB each) were all skipped for that reason; only the two JPEGs that exceeded 1600 px were downscaled. Measured: maxDim=1600 → 3.46 MiB retained; maxDim=800 → 1.65 MiB (−52 %); maxDim=600 → 1.04 MiB (−70 %), which is the only setting that gets the body under the cap. BILI_IMAGE_RESCALE_MAX_DIM=800 is a working workaround today, but it is a global knob and it does not fix the gate.
5. The overflow detector cannot read the window out of a Chinese body
CONTEXT_OVERFLOW_PATTERNS (src/util.ts:246) and parseOverflowWindow (:301) were English-only when this session was captured. #3787 / #3792 has since taught the pattern list to recognise the 上下文 family, which fixes classification. Two gaps survive it:
parseOverflowWindowstill has no CJK branch onmaster, so even a recognised body yieldswindow === undefined.armOverflowShrinkthen takes its "window not parseable" path and arms atmin(declared, defaultCountTokens(rawBody))— i.e. sized by the rejected payload's own estimate, not by the1500000the relay just stated in plain text.- The pattern list still misses input-side subjects (
输入过长,提示词太长,请求过长,对话太长,消息超长) and the token-counter subject (输入 token 数量超出允许范围).
Whether cause 5 is on the critical path here depends on the relay wording: with #3787 merged, my body is classified, so cause 5 alone would no longer reproduce this loop. It is reported because a window of undefined degrades the arm to a payload-sized guess, which is how the arm lands above the cap it was supposed to get under.
What I have ready
- A local branch with the cause-5 increment (CJK window parsing + input-side markers, 18 tests,
tests/cjk-context-overflow.test.ts) — happy to open it as a PR if that is the preferred shape. repro-context-undercount.mjs: offline, read-only replay of a session'sindex.dbthat re-derives bili's estimate and compares it with the real wire bytes. Seven assertions, currently 7/7.- A
--dry-runpatcher for the cause-1/3/4 changes (+844bytes, 5 hunks, allfindtargets verified unique indist/index.js).
Causes 1–4 look like one coherent fix to me: make the image sidecar visible to the total (1 + 3), and let a rejection teach the per-image cost so the pixels prior stops being load-bearing (2). Cause 4 is independent. I would rather not guess which of these you want in scope, so tell me and I will send the PR(s) against master accordingly.
Environment
- bili version:
0.1.202(also the version onmasterat5121f45; not yet released at time of writing) - Usage mode: Plugin native (
bili plugin installintodsh, web profile) - Client:
dshweb profile (@deepseek-ai/dsh), Windows 11 - OS: Windows 11
- Upstream:
deepseek-v4-flashbehind a relay; declared model window 1 000 000 (bili logged/1000000), relay's per-request cap 1 500 000 (stated in the 400 body)
bili log excerpts
12:03:08.891 nudge idle: usage=23% (230420/1000000)
12:03:55.621 nudge idle: usage=11% (114471/1000000), growth=59039/22500 (ref=55432, interval=50000), pendingT1=45535/50000 model=deepseek-v4-flash, reason="max compressible 45535 < threshold 50000"
12:03:55.748 request: 1271 msgs, inbound=12.4MiB, outbound=5.0MiB, local=766ms
12:04:19.538 ← upstream 400 ... 请求上下文超限:本次请求约 1748713 tokens,超过单次上限 1500000 tokens。…
12:06:00.588 ← upstream 400 ... 请求上下文超限:本次请求约 1748746 tokens,超过单次上限 1500000 tokens。…
12:08:11.704 ← upstream 400 ... 请求上下文超限:本次请求约 1748779 tokens,超过单次上限 1500000 tokens。…
Control, same file, a later session where the relay answered in English:
13:38:18.564 sess=refold-b ← upstream 400: {"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: 130000 tokens > 128000 maximum"}}
ACP state
/acp reported 1160 covered messages in 1 block, kept 294 — the fold is doing its job. The wire body is what the fold cannot shrink, because 4.8 of its 5.12 MiB is image payload.
Activity
DadayaSuwen commentedon Oct 20, 2026
(ai: anthropic/claude)
Fix for the detector half is up: #3950 —
parseOverflowWindowlearns to read the limit a Chinese relay states in plain text (超过单次上限 1500000 tokens→window: 1500000), plus the input-side phrasings#3787still misses (输入过长/提示词太长/请求过长/对话太长/消息超长/输入 token 数量超出允许范围).It does not touch causes 1–4 (the image channel). Say the word and I will send those as a separate PR against
master— the dry-run patch for them is 5 hunks,+844bytes, allfindtargets verified unique indist/index.js.ranxianglei commentedon Oct 20, 2026
(ai: vllm-qwen/qwen3.8-27b)
查重: 不重复。 最近的两个 open issue 均不同症状+不同机制,交叉链接不合并:
标签: bug · dsh · scope/kernel · p1(核心承诺失灵:图像主导的载荷上自动压缩自愈完全失效,会话在人工干预前不可用)。
审核结论(master@5121f45f8 = 你的 v0.1.202,已排除版本滞后;五条逐一到代码路径核对)
五条根因代码层面全部成立,其中两处表述需修正、一处边界需收紧,另有一个分层判断上的关键补充:
effectiveTokenCount的 usage fast path 确实整体丢弃图像通道——但不是「只有 anthropic 传了」:四条 lane 全都传了,prepare-openai.ts:130 / prepare-responses.ts:234 / prepare-google.ts:137 / prepare-anthropic.ts:235 各自把imageReserveFor(...)作为第三参传入。缺陷纯在 fast path 本身(:2313):只要存在 usage-grade 锚点,任何 lane 当轮新增的图像质量都被丢弃。你的「三个调用点」数的是inboundImageTokens这个变量名,其余三 lane 是内联实参。settleImageLearning(cache-ledger.ts:675)唯一入口是settleUsageReport且要求billedTotal > 0(:1612/:679)。「未计费」的 400 永远产生不了 usage report → 自校正通道被锁死在它需要校正的那个故障事件上。永久而非瞬态,确认。finalUpper/entryLocal已经加上 imageReserve(preflight.ts:1614/:1675/:1911)——preflight 不是对图像盲,是用错了价(pixel prior ~2805/图 vs 该 relay 实际 ~240K/图)。真正完全看不到 sidecar 的是 doomed-probe 守卫(server.ts:3124,只数 core text bytes)与outboundContextEstimates.upperBound/localInputEstimate。所以原因 3 的实质是「上限口径不含 sidecar」+「含了也是错价」,后者归原因 2。rescaleImage门(:151)放进 byte 超限者,doRescale(:117-118)对 scale≥1 且非 force 一律 return null;force 只有 egress-shed([Bug] Jimp is active, but image accumulation still triggers Responses WS 1009; duplicate request size not reduced (v0.1. | [Bug] Jimp 已生效,图片累积仍触发 Responses WS 1009;重复请求未缩小(v0.1.200) #3701 阶梯)会传。四张 1080×480 PNG(359–918KB)被放行后拒绝,与你实测一致。parseOverflowWindow无 CJK 分支之外,parseOverflowCounts同样没有 CJK 分支——「本次请求约 1748713 tokens」这个被拒载荷自报尺寸也解析不出来。这直接影响修复设计(见下)。关键补充:为什么「分类修好 ≠ 循环打破」
你抓到的日志里三条 400 之间没有 arm 行,而
← upstream 400行证明错误分支走到了(inspect 在其之前执行);v0.1.202 已含 #3787 的上下文.{0,8}(?:超出|超过|超限)模式,能匹配「请求上下文超限」。两者合起来最合理的解释:捕获时 dsh profile 里的运行副本旧于 #3787(install-lane 漂移,契约内允许的瞬态分歧;也与你「captured 时 patterns 还是 English-only」的说法自洽)。但更重要的是:我在 master 上把整条链走了一遍,即使分类正常工作,循环依然不会断:
min(declared≈900K, defaultCountTokens(5MiB body)≈1.25M+)= declared,source=overflow-arm;est >= limit,而 est = 本地文本估计 + imageReserve ≈ 83K + 19.6K ≈ 103K « 900K → 循环体一次都不执行,什么都不折,原样重发 → 再 400。范围决定(按「一个机制一个 PR」)
noteForwardedImageFactsseam 取 (nImages, textSide),按(statedTotal − textSide)/nImages喂进同一 EMA 学习库(TTL 24h、cap 1M、fp 失效全部复用,与 billing 样本同库同 alpha);②effectiveTokenCountfast path 加下界max(anchor, estimateCoreMessages(msgs) + inboundImageTokens)——下界用 billing 口径估计而非 char 上界,正常会话不抬升,任何一轮没有拿到 usage 报告的请求,都会把上下文分母放大 12–20×(nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟) #1839 幽灵放大纪律不破;③doomed-probe 守卫的字节总数计入 sidecar b64 字节。回归测试钉住全链:stale anchor + 未计费 400 → learned cost 更新 → preflight 真折 → meter 诚实。sqrt(trigger/origBytes)推 sub-scale(bytes∝area 假设,floor 0.25),采用规则不变(<80% 或 force 严格缩小),确定性缓存/注释/loop guard 全部不动。parseOverflowCounts的 CJK 分支(「约 N tokens」→ requestedTotal)请一并收进去——PR A 的拒绝教学直接消费这个字段,否则两个 PR 在 util.ts 上互相踩。合入顺序:[PR] fix(util): parse the window out of CJK context-overflow rejections #3950 先行,A rebase 其上。需要你做的:①开 #3950(带上 parseOverflowCounts CJK 分支);②方便的话给一下捕获时 dsh profile 副本的确切版本串(启动日志行或 ledger meta),确认「旧于 #3787」推断——若坐实,install-lane 副本为何没跟上 registry 值得单独立项查(那是 Install-Lane Contract 下的另一个话题,不塞进本 issue)。
PR A/B 就绪后我在本帖回报。
fix(util): teach parseOverflowCounts the CJK payload-count phrasing
fix(util): teach parseOverflowCounts the CJK payload-count phrasing
fix: size image-dominated payloads from the wire, not the text estima…
fix: size image-dominated payloads from the wire, not the text estima…