Repository navigation
fix(#2490): plan output budgets against the refused dsh raw ledger; fail fast doomed evidence probes - #2577
Merged
Conversation
…ail fast doomed evidence probes Root cause of the #2490 turn-37→40 death: the host's raw ledger (replayed whole by every refused dsh native-compaction envelope) is invisible to bili's folded view, so the output clamp planned against the small folded input while the host re-sent the full raw history every turn — a 1M window with ~200K folded input let max_tokens=384000 through and one 7.5MB assistant turn landed un-folding. Two guards: - ledger floor: the #1835 guard now records the refused replay size on the session (high-water bytes + count) and clampOutgoingOutput plans against max(folded estimate, ledger floor) on the openai/google/responses lanes. Only ever tightens; cap < floor surrenders to preflight as before. - doomed probe: the #2313 forward-once-for-evidence bet assumes some tokenizer might fit the payload. Beyond window×7.5 BYTES of core text (densest live-route billing observed ~6.8 B/token, #2122) none can, so the probe is a guaranteed 400 with an 8MB body — fail fast 502 with the byte math instead of paying it. tests/issue2490-ledger-floor-doomed-probe.test.ts: 6 pins (floor unit trio, incident clamp 384K→55K, guard accounting e2e, doomed-probe e2e, #2313 control).
Closed
Contributor
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2775-pr.2577.ef1d55fThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2577.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Owner
Author
CI caught what my neighbor runs missed: the #1001 rewrite test rides a legal 86KB payload at 8.6 bytes/token against a 10K window — repetitive text is exactly what real tokenizers compress past 7.5 B/token, so a pure ratio bar 502'd a probe that by design must forward. Dense-but- small payloads now keep the bet (it costs one small request); only past max(2MB, window*7.5) is forwarding 'for evidence' a guaranteed multi-MB 400 — the incident's 7.5MB turn still trips it.
Contributor
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.205-master.8e7bb12 # this exact build
npm install -g billion-context@master # always the newest merged build
|
This was referenced Oct 10, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2490 (root-cause path: the kill chain, not the noise).
Root cause (independent of the 198 refused compactions)
The turn-37→40 death chain, verified against the reported telemetry:
clampOutgoingOutputplanned output budgets against the folded input (~200K) against a 1M window →max_tokens=384000passed through (the dsh↔proxy tug-of-war: dsh sends 1, proxy raises to 384000).probeForEvidenceforwards the 8MB body once "for evidence" (billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313) → upstream 400 → session dead.The 198 hammering compactions are chronic noise, not the killer: bili's folding was working the whole time (the 100-300K sawtooth).
Fix A — ledger floor (the key guard)
handle.tsnow records the refused replay size on the session:dshCompactionRefusedBytes(high-water mark — the replay is the whole ledger every time, so max bounds every future re-send) +dshCompactionRefusalscount.dshLedgerFloorTokens()(budget.ts) translates it at the estimator's own bytes/4 caliber.clampOutgoingOutputplans againstmax(folded input estimate, ledger floor)on the openai / google / responses lanes. Direction of failure is tighten-only: a smaller floor changes nothing, cap <OUTPUT_CLAMP_FLOORstill surrenders to preflight, legacy sessions (no refusals) are byte-identical.Fix B — doomed-probe fail-fast
The #2313 probe forward is a bet that some tokenizer might fit the payload. That bet has positive EV while the payload is experiment-sized — CI just proved the converse too: the #1001 rewrite test rides a legal 86KB payload at 8.6 bytes/token against a 10K window (repetitive text is exactly what real tokenizers compress past the densest live-route billing of ~6.8 B/token, #2122). So the doom bar is two-condition, both required:
estimateCoreMessagesUpperBytes()(preflight.ts) — UTF-8 byte twin of the char upper bound (bytes, not chars: CJK monsters are char-cheap but byte-doomed).max(2MB, window × 7.5)— dense-but-small keeps probing (one cheap request), while past the bar no tokenizer in the wild accepts it and the "probe" is a guaranteed 400 with a multi-MB body: fail fast 502 with the byte math in the operator-facing message. The incident's 7.5MB single-turn payload still trips it.What this deliberately does NOT do
allowDshCompaction(在bili网页配置界面支持关闭和开启dsh的自动压缩功能 #2028) stays opt-in-only — the landed-checkpoint irreversibility argument (nudge 死锁无检测:同一 session 交替收到两种结构互斥的请求流(巨型单消息 ↔ 多轮)时不告警 #1729) is unchanged.Tests —
tests/issue2490-ledger-floor-doomed-probe.test.ts(6 pins)dshLedgerFloorTokenscaliber/high-water/legacy-safety (NaN, strings, 0 all neutral).Verification: typecheck clean; new suite 6/6 (3×); neighbors 79/79 across dsh-compaction-guard / issue2028 toggle / issue2313 probe / issue2122 clamp / issue2096 / preflight-fail-fast / issue1993 / issue1987 / preflight-compress / emergency-fold / hold / nonstream-400-probe / issue1841 / max-output-tokens / window-clamp / issue2451.