Skip to content

[PR] fix: dsh ledger floor follows the authored snapshot instead of the envelope peak - #5364

Open
ranxianglei wants to merge 2 commits into
masterfrom
2026-10-31_ledger-floor-authored-decay
Open

ranxianglei wants to merge 2 commits into
masterfrom
2026-10-31_ledger-floor-authored-decay

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Problem (#5289)

A small-window DSH session (reporter: local llama.cpp, 128K window) got its output truncated permanently after its first dsh native-compaction attempt: bili's output clamp planned against a raw-ledger floor that had ratcheted to the compaction ENVELOPE's byte count (~93K tokens) and never decayed — even when bili ANSWERED that compaction locally with its own folded snapshot (#3733), which genuinely shrinks dsh's store to ~50KB. Result: max_tokens clamped from 384000 to ~33K on every subsequent turn while the folded wire sat at ~49K, truncating long replies at stop_reason=length until the user inflated DSH's declared window to 256K just to dodge the clamp.

Root cause: the guard stamped dshCompactionRefusedBytes from inboundBytes BEFORE branching into authored/refused (src/server/handle.ts), so both paths inherited the pre-answer raw peak, and the value stayed sticky for the session's life.

Fix (option A from the in-thread decision request, owner-approved)

  • Authored branch: floor := exact bytes of the folded snapshot sent (Buffer.byteLength(authored.body)), replacing any earlier high-water — the snapshot IS dsh's ride-out scale from now on.
  • Refusal branch: unchanged sticky high-water (Math.max(prev, inbound)), byte-for-byte — the dsh压缩与bili压缩冲突 #2490 monster-output protection stays intact.
  • Docs aligned at both sites (stamp-site comment split per branch; dshLedgerFloorTokens() documents both regimes).

Behavior change (old → new)

Path Old New
authored checkpoint lands floor := max(old, envelope bytes), sticky for life floor := snapshot bytes we sent (decays to reality)
refusal max(old, envelope bytes) unchanged

Consequence: small-window sessions whose compaction gets an authored answer no longer starve (the incident); sessions whose compaction keeps being refused behave exactly as before. The #2490 kill chain stays closed: post-checkpoint store growth is bounded by each turn's clamped output, and the fail-open edge (floor ≥ window) can only trigger on genuine overflow now.

Config surface

None added or changed. BILI_DSH_AUTHORED_COMPACTION / dsh.authoredCompaction semantics untouched.

Tests

Labeled drive-by fix in the same PR: docs: commit aligning the stale "Default OFF" docstring on BILI_DSH_AUTHORED_COMPACTION (knobs.ts) with the ON default shipped since 0.1.203 (#3901 flip) — doc/code mismatch found while working this mechanism. No behavior change.

Closes #5289

ci-check added 2 commits October 31, 2026 22:34
The compaction guard stamped dshCompactionRefusedBytes from the envelope
bytes BEFORE branching into authored/refused, so even the #3733 local
answer — which genuinely shrinks dsh's raw store to our folded snapshot —
ratcheted the planning floor to the pre-answer raw peak and kept it sticky
for the session's life. On a small window the output clamp then planned
against that stale peak forever: 128K window - 93K floor - margin ~= 33K
max_tokens while the folded wire sat at ~49K, truncating long replies until
the user inflated DSH's declared window to dodge the clamp (#5289).

Authored branch now stamps the exact bytes of the snapshot sent (the
ride-out scale from now on); the refusal branch keeps the #2490 sticky
high-water byte-for-byte. Docs aligned at both sites.
Drive-by found while working #5289: the docstring still said 'Default OFF'
while the function has returned true by default since the #3723/#3901 flip
(0.1.203). Doc/code mismatch corrected; no behavior change.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-31_ledger-floor-authored-decay (b7f40c2)

Latest build — install this exact version

npm install -g billion-context@0.0.7556-pr.5364.3148869

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
b7f40c2 billion-context@0.0.7556-pr.5364.3148869 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5364.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: dsh ledger floor follows the authored snapshot instead of the envelope peak [PR] fix: dsh ledger floor follows the authored snapshot instead of the envelope peak Oct 31, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 按失败签名(dshCompactionRefusedBytes / ledger floor / authored snapshot decay)检索 open+closed issue 与近期 PR,无其他条目覆盖同一机制;本 PR 即 #5289 的修复本体。

标签:pr / bug / dsh / scope/wiring / p1 —— dsh 车道 compaction 后输出预算被永久钳死、长回复持续截断,核心输出能力对受影响会话不可用,按 owner 修订判 p1。

开始 PR 审核(方向关 → 回归 → 代码 → 测试 → diff),稍后给结构化结论。

@ranxianglei ranxianglei added bug Something isn't working dsh p2 scope/wiring labels Oct 31, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR 审核结论(独立重审,未照抄 issue 侧结论):diff 逐行重读、机制从代码重新推导、回归面逐项清点、CI 红分片逐一定位到日志行。代码与测试通过;CI 3 个红分片为 master 继承性失败(已报 #4599),master 转绿后 rebase/重触发即可,分支侧无需任何改动。

结构化八问

  1. 是什么问题:小窗口(如 128K)DSH 会话在第一次 dsh 原生压缩之后,发往后端的 max_tokens 被永久钳到 ~33K,长回复全部中途截断——不是上下文真满,是规划输入下限错。
  2. 什么原因导致的:guard 在 authored/refused 分支之前就用原始信封字节数盖了 dshCompactionRefusedBytes(Math.max 棘轮、终身粘滞)。于是即使 bili 用本地折叠快照应答了压缩(dsh 存储真缩到几十 KB),下限仍钉在应答前的信封峰值;此后每轮 clampOutgoingOutput 按 max(折叠 wire, 下限) 规划输入,小窗口下可用输出 = 窗口 − 峰值 − 安全垫 ≈ 30K 量级。病灶=该值不随账本实际缩小而回落。
  3. 影响范围与严重程度:所有「触发过 dsh 原生压缩 + bili 本地应答(0.1.203 起默认 ON)+ 窗口小到信封接近窗口」的会话。能用但长回复必截断,有绕行(256K / 关 dsh 自动压缩)→ p2(接单时初判 p1,读完全程后改标:不属 owner 修订的两类 p1——无会话不可恢复/数据丢失,无缓存泄露)。
  4. 怎么修复的:owner 拍板的方案 A——authored 分支下限 := 本次交回的快照精确字节(Buffer.byteLength(authored.body),直接赋值覆盖旧高水位);refused 分支粘滞高水位逐字节不变。修的是生产者(分支前统一盖章),不是在消费侧打补丁。
  5. 是否有回归问题:逐项审过既有行为——
  6. 是否破坏缓存:不涉及(只改一个会话元数据字段的写入时机与取值,出站前缀/去重键不动)。
  7. 是否改了设计方向:无架构转向;仅把一个字段的语义从「永远取最大」修正为「应答路径跟随现实」。
  8. 是否引入新配置:无。

行为变更披露(旧→新):authored checkpoint 落地时,规划下限从「信封字节高水位、终身粘滞」变为「本次快照字节数(可低于历史峰值)」;refused 路径不变。PR 正文已披露,与本审核判定一致。

测试质量:新 e2e 走真实 startServer + mock SSE 上游,双向门都钉了(#3617):authored 衰减到精确快照字节 + 巨额预算回合解饿(>100K vs 修复前 ~33K)、refused 高水位逐字节、authored→refused 组合再棘轮;RED→GREEN 声称的数字自洽(372242 = 372_000 字符 filler 的信封字节,704 = 快照体 < 10KB sanity 界内)。既有 dsh-authored-compaction 断言从 typeof 加强为精确字节相等,有牙。

CI 状态(head b7f40c2,merge-commit 树):static-gates/build/mysql-contract/gate 全绿;14 个 test 分片 11 绿;红的 3 个(ubuntu-24-2 / ubuntu-22-2 / windows-22-3)各只挂 2 条 web-i18n #1024(q_pack 占位符/死键)——master tip 自身同样 3 片同签红,根因 b844b7512(#5345 pack editor),已在 #4599 报根因+两行修法。非本 PR 引入,分支侧不改。

判定:代码审核通过,可合——待 #4599 所报 master i18n 红清掉后 rebase/重触发转全绿,由人合并。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Token truncation in small context windows | 小窗口上下文 token 截断

1 participant