Repository navigation
fix: gate dsh compaction refusal on live compression substrate (#3016) - #3030
ranxianglei wants to merge 3 commits into
Conversation
A dsh native compaction call was refused unconditionally (422 dsh_compaction_refused) even when bili held no compression substrate (state.blocks=0) — there was nothing to protect. On the unattributed passthrough lane (x-bili-passthrough) this meant that when bili's own nudge/decision channel was dead (#3004), the last remaining shrink backstop was also disabled, pinning context at the wall until the hard limit. Gate the refusal on whether a substrate actually exists to protect: - pipeline path (target resolved): relay verbatim when session.state.blocks.length === 0. - passthrough lane (target unknowable — only x-bili-passthrough is stamped): refuse iff anyLiveSessionHasCompressionBlocks(); otherwise stand down and relay. No new config/env surface; the existing escape hatch allowDshCompaction / BILI_ALLOW_DSH_COMPACTION (#2028) still short-circuits first. Updates the pre-existing refusal fixtures (which used fresh sessions) to seed a substrate so they keep pinning the protective path. Closes #3016
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3431-pr.3030.675f082This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3030.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…#3016 review) The passthrough gate scans only the in-memory live-session map. Since #2803 (LRU session eviction) landed on master after this PR's base, a substrate-holding session can be flushed out of memory and become invisible to the scan. The previous wording ('disk-only sessions are not actively being compacted' / 'Never allows when the target could have substrate') documented a guarantee the merged code does not provide. Comments only; no behavior change.
(ai: opencode/vllm-qwen-qwen3.8-27b) Review 结论:代码正确,已在当前 master 上验证全绿;原 CI 红灯是继承自 master 的 golden 断供(已由 #3035 修复),与本 PR diff 无关。分支已更新到当前 master,等新 CI 跑完即可合。 八问
行为变更披露(旧→新):passthrough dsh compaction:原一律 422 → 现仅当内存中任一活 session 持有基底时 422,否则逐字 relay;pipeline 路径:原可解析出 session 即 422 → 现 Review 中新发现的残留风险(已写入代码注释):master 在 PR base 之后落了 #2803(LRU session eviction)——持有基底的 session 可能被逐出内存,passthrough 门的内存扫描就看不见它 → 一个窄窗口内、恰好瞄准被逐出 session 的 checkpoint 可能被 relay(#1729 类遮蔽)。窗口很窄(目标须持有基底 ∧ 已被逐出 ∧ compaction 在归因轮重新解析前经不可归因 lane 到达),pipeline 路径不受影响(getSession 会从盘重载)。已把两处夸大保证的注释改准(docs-only commit CI 红灯根因说明:原 head 的 4 个 test leg 失败 = storage-render golden 的 metadata 段漂移,根因是 #3023 落 master 时未 regen goldens(其 commit message 自述:master-push CI 跳过 test leg,所以 master 静默红了一段时间),#3035(已并入 master)regen 后的新 pin 分支处置:已把 origin/master(含 #3035)merge 进 PR 分支并 push(merge commit 收工判定:审查通过、定向验证绿,可合。此 PR 触碰 session-binding/wire-shape 判定,按 §7.4 属人工 review 层——请 @ranxianglei 手动合并:#3030 |
(ai: opencode/vllm-qwen-qwen3.8-27b) CI 全绿,收工确认:更新后的 head 补充两件已处置的事:
最终判定:可合,等 @ranxianglei 手动合并:#3030 |
…ger client-insert attribution ranxianglei#3023 (20c4258) added client-insert attribution to session.metadata.cacheLedger (inbound-view fingerprints, bucket fields) but shipped without regenerating the storage-render goldens: the metadata section digest drifted on all four wires while state/blockContents/stats stayed identical. Its PR CI was red on the test legs and was merged anyway, and master-push CI skips the test lane entirely (38389227456: test=skipped, static-gates=skipped), so master has silently failed tests/golden-storage-render.test.ts since 2026-10-13 18:55 (+0800). Drift verified metadata-only by bisect (a1dba97 green -> 20c4258 red) and by raw metadata diff (timestamps/uuids aside, only cacheLedger changed; canonicalize already normalizes volatile values). Regen diff: 8 digest lines, 4 files. storage-corpus pins untouched (they pin legacy-format parsing, not live render). Also un-blocks PR ranxianglei#3030, whose CI failure is entirely this inherited master breakage, not its own diff (verified: ranxianglei#3030 head alone is green locally).
|
按 owner 裁定关闭:放行方向违反顺位契约(见 #3016 评论)。取证保留在 issue 侧;422 文案升级另行处理。 |
Closes #3016
Problem
When a dsh native compaction call lands on the unattributed passthrough lane (
x-bili-passthrough, #1117/#2709), bili refused it unconditionally with422 dsh_compaction_refused— even when bili itself held no compression substrate (state.blocks = 0), i.e. nothing existed to protect. The same unconditional refusal applied in the pipeline guard. Consequence: exactly when bili's own nudge/decision channel was dead (#3004 deadlock), the last remaining shrink backstop (dsh's own compaction) was also disabled, pinning context at the wall until the hard limit.Root cause
The guard's job (#1729/#2028) is to stop an unattributed dsh checkpoint from durably shadowing an existing compression substrate. It refused before ever asking whether such a substrate exists. A refusal that protects nothing is pure loss.
Fix (mechanism)
Gate the refusal on whether a substrate actually exists to protect:
session.state.blocks.length === 0; refuse only when the target holds substrate.x-bili-passthrough, never a session id, because the call is agentless): refuse iffanyLiveSessionHasCompressionBlocks(); otherwise stand down and relay verbatim.anyLiveSessionHasCompressionBlocks()(new,src/session.ts) scans the in-memory live-session map — fast, never walks disk.Behavior change (old -> new)
422422only while any live session holds substrate; otherwise relayed verbatim (dsh's checkpoint lands)422422only while the resolved session holds substrate; otherwise relayed verbatimThis is intentional and the point of the fix. When nothing is compressed, a landed dsh checkpoint destroys nothing, so relaying restores the last shrink backstop. The protective path (substrate present) is byte-for-byte unchanged.
Residual (documented): on the passthrough lane the exact target is unknowable, so the check is conservative — it refuses whenever any live session holds substrate. In a rare multi-profile case where only other sessions hold substrate (and the true target does not), it still refuses. That is the safe direction (never allows when the target could be protected); tightening it to the exact target would need a client-stamped session id (see Design gap below).
Not changed
allowDshCompaction/BILI_ALLOW_DSH_COMPACTION(在bili网页配置界面支持关闭和开启dsh的自动压缩功能 #2028) still short-circuits first -> always relay. No behavior change there.kernel/**.422, same body, sameretryable:false.Design gap left in place (not fixed here)
On the unattributed lane the server cannot identify the target session (the client can't stamp one — agentless call, #1117/#2709). A precise per-target check would require either routing compactions through session-attributed routing or a single-active-session fallback. This PR ships the conservative-sound version (any-substrate gate) which fixes the reported deadlock scenario without that mechanism; the precise-target refinement remains a follow-up.
Tests
tests/issue3016-compaction-substrate-gate.test.ts: passthrough relay-no-substrate (core fix, byte-exact), passthrough refuse-with-substrate, pipeline relay-blocks=0, pipeline refuse-with-substrate (+ dsh压缩与bili压缩冲突 #2490 bytes recorded).issue2709-passthrough-compaction-guard.test.ts(fix: keep system prompt byte-stable for prefix cache #1, Use kernel renderTags:"text-only" + forbid tag echoing in prompt #3),issue2028-dsh-compaction-toggle.test.ts,issue2490-ledger-floor-doomed-probe.test.ts(e2e-a) — each seeds a substrate so they keep pinning the protective path.Review tier
Touches identity/session-binding logic and wire shape (both modes reasoned) -> human-review tier, no auto-merge (AGENTS.md section 7.4).