Skip to content

fix(#2490): plan output budgets against the refused dsh raw ledger; fail fast doomed evidence probes - #2577

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-10_dsh-ledger-floor-doomed-probe
Oct 10, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-10_dsh-ledger-floor-doomed-probe

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Fixes #2490 (root-cause path: the kill chain, not the noise).

Root cause (independent of the 198 refused compactions)

The turn-37→40 death chain, verified against the reported telemetry:

  1. dsh's raw ledger (~900K tokens) is replayed whole by every native-compaction envelope; the guard (fix(#1729): refuse dsh native compaction calls at the proxy gate #1835) refuses all 198 at the wire — correct, but that means the raw ledger never shrinks and stays invisible to bili's folded view.
  2. clampOutgoingOutput planned output budgets against the folded input (~200K) against a 1M window → max_tokens=384000 passed through (the dsh↔proxy tug-of-war: dsh sends 1, proxy raises to 384000).
  3. One assistant turn produced a 7.5MB (7,498,192-byte) single message, truncated by max-tokens, landed in the recent tail — un-folding by construction.
  4. Next turns cannot get it under the window; the last-known-good baseline is gone → probeForEvidence forwards the 8MB body once "for evidence" (billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313) → upstream 400 → session dead.

The 198 hammering compactions are chronic noise, not the killer: bili's folding was working the whole time (the 100-300K sawtooth).

Fix A — ledger floor (the key guard)

  • The guard in handle.ts now records the refused replay size on the session: dshCompactionRefusedBytes (high-water mark — the replay is the whole ledger every time, so max bounds every future re-send) + dshCompactionRefusals count.
  • dshLedgerFloorTokens() (budget.ts) translates it at the estimator's own bytes/4 caliber.
  • clampOutgoingOutput plans against max(folded input estimate, ledger floor) on the openai / google / responses lanes. Direction of failure is tighten-only: a smaller floor changes nothing, cap < OUTPUT_CLAMP_FLOOR still surrenders to preflight, legacy sessions (no refusals) are byte-identical.
  • Incident math now: 1M window − 900K ledger floor − 45K margin → cap 55K, not 384K. The monster budget cannot exist.

Fix B — doomed-probe fail-fast

The #2313 probe forward is a bet that some tokenizer might fit the payload. That bet has positive EV while the payload is experiment-sized — CI just proved the converse too: the #1001 rewrite test rides a legal 86KB payload at 8.6 bytes/token against a 10K window (repetitive text is exactly what real tokenizers compress past the densest live-route billing of ~6.8 B/token, #2122). So the doom bar is two-condition, both required:

  • estimateCoreMessagesUpperBytes() (preflight.ts) — UTF-8 byte twin of the char upper bound (bytes, not chars: CJK monsters are char-cheap but byte-doomed).
  • doom only when core text bytes exceed max(2MB, window × 7.5) — dense-but-small keeps probing (one cheap request), while past the bar no tokenizer in the wild accepts it and the "probe" is a guaranteed 400 with a multi-MB body: fail fast 502 with the byte math in the operator-facing message. The incident's 7.5MB single-turn payload still trips it.

What this deliberately does NOT do

Tests — tests/issue2490-ledger-floor-doomed-probe.test.ts (6 pins)

Verification: typecheck clean; new suite 6/6 (3×); neighbors 79/79 across dsh-compaction-guard / issue2028 toggle / issue2313 probe / issue2122 clamp / issue2096 / preflight-fail-fast / issue1993 / issue1987 / preflight-compress / emergency-fold / hold / nonstream-400-probe / issue1841 / max-output-tokens / window-clamp / issue2451.

…ail fast doomed evidence probes

Root cause of the #2490 turn-37→40 death: the host's raw ledger (replayed
whole by every refused dsh native-compaction envelope) is invisible to bili's
folded view, so the output clamp planned against the small folded input while
the host re-sent the full raw history every turn — a 1M window with ~200K
folded input let max_tokens=384000 through and one 7.5MB assistant turn
landed un-folding. Two guards:

- ledger floor: the #1835 guard now records the refused replay size on the
  session (high-water bytes + count) and clampOutgoingOutput plans against
  max(folded estimate, ledger floor) on the openai/google/responses lanes.
  Only ever tightens; cap < floor surrenders to preflight as before.

- doomed probe: the #2313 forward-once-for-evidence bet assumes some
  tokenizer might fit the payload. Beyond window×7.5 BYTES of core text
  (densest live-route billing observed ~6.8 B/token, #2122) none can, so the
  probe is a guaranteed 400 with an 8MB body — fail fast 502 with the byte
  math instead of paying it.

tests/issue2490-ledger-floor-doomed-probe.test.ts: 6 pins (floor unit trio,
incident clamp 384K→55K, guard accounting e2e, doomed-probe e2e, #2313
control).
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-10_dsh-ledger-floor-doomed-probe (a648d7d)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2775-pr.2577.ef1d55f

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2577.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到。这是修 #2490 的 PR(分支 2026-10-10_dsh-ledger-floor-doomed-probe)。我按 PR review 流程处理:拉取分支核对代码与描述一致、跑定向验证、确认 CI,然后给出结构化结论。先拉代码和 diff。

CI caught what my neighbor runs missed: the #1001 rewrite test rides a
legal 86KB payload at 8.6 bytes/token against a 10K window — repetitive
text is exactly what real tokenizers compress past 7.5 B/token, so a
pure ratio bar 502'd a probe that by design must forward. Dense-but-
small payloads now keep the bet (it costs one small request); only past
max(2MB, window*7.5) is forwarding 'for evidence' a guaranteed multi-MB
400 — the incident's 7.5MB turn still trips it.
@ranxianglei
ranxianglei merged commit 8e7bb12 into master Oct 10, 2026
28 of 29 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.205-master.8e7bb12 (rolling master tag).

npm install -g billion-context@0.0.205-master.8e7bb12  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dsh压缩与bili压缩冲突

1 participant