Repository navigation
[PR] kernel: all-or-nothing summary-length pre-check for multi-range compress (#3003) - #3670
ranxianglei wants to merge 5 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4372-pr.3670.2f2ac9eThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3670.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复 —— 按失败签名(多 range compress 部分落地双缝 / all-or-nothing 长度预检)检索 open+closed issue/PR,无同机制在途条目。最近的邻居都不同域:#3634/#3635 是 external-summary 模式下超长摘要被静默丢弃(不同路径、不同症状),PR #3002 是已关闭的缝稳定性探针,#2997 是已关闭的 prefix-excess 稳定性 bug。本 PR 即 #3003 拍板后((floor 9「接单」))的唯一实现。 标签: pr + bug + dsh + scope/kernel + p2(缺陷类成本问题;#2997 现场为 dsh-lane;修复主体在内核压缩引擎;特定条件下多 range 折叠有实际成本损失、非致命)。 开工审核。这是 kernel 改动(human-only review tier),我按 PR Review 规范走:先核对 #3003 方向与批准链 → 全量读 diff → 回归清点(重点:external-summary 路径与新门的交互、consumed/refold 作用域、零失败时字节不变性、单 range 错误面、plugin/proxy 两模式)→ 测试有效性 → CI。结论用八问给出,稍后回帖。 |
(ai: vllm-qwen/qwen3.8-27b) PR #3670 review complete. Bottom line up front: 方向正确、实现与 #3003 批准的设计一致、未发现无意回归 —— 可合入(kernel 变更 → 人工审核层,等你合并)。审查中我按「小问题直接修」向 PR 分支补了 3 个标注 commit(见下)。 结构化八问
高危域逐项点名核查
审查时补进 PR 分支的 3 个小修(additive commits,作者署名不动)
验证证据(定向本地)
判定:可合入——方向、机制、候选集边界、测试双向咬合都经独立复核成立,3 个审查小修已就位。kernel 变更为人工审核层,请自行合并:#3670 |
…idation (#3003) A multi-range compress batch committed per-range, so one length-class failure (summary over/under cap) among otherwise-valid folds let the valid siblings land FIRST (opening a seam), and the model's retry of the bad range paid a second seam. Summary length is a pure function of (spec.summary, cfg) — no state, no cross-range coverage — so screen it all-or-nothing BEFORE any block commits: any length-class failure now rejects the WHOLE batch (0 commits, 0 seams). The scope is exactly the fold-candidate set that mutates state; content-class failures (protected-zone / turn-integrity / no-new-messages) stay per-range. Behavior change: a length-class mixed batch goes from PARTIAL (valid ranges land, bad one errors) to FULL-reject (nothing lands, actionable error names each offending range + overflow and offers two exits: shorten/split-and-resend, or drop-the-offender-and-resend-rest). Batches with zero length failures are byte-identical. The existing nudge guidance ("a single oversized summary fails the WHOLE compress call") now matches actual behavior. Bumps acp-kernel 0.0.131 -> 0.0.132.
Each landed fold re-bills the remaining suffix to the prefix cache (one seam). Count fold-rounds per turn and log when a turn carries >=2 — the residual multi-seam cost that survives after #3003's all-or-nothing length gate. Purely additive logging; feeds the deferred decision on delayed materialization.
…view) The new all-or-nothing length pre-check adds a failure class the #2362 classifier did not know: 'Batch rejected before folding any range' fell into the 'gate:other' catch-all, losing operator visibility into WHY the batch died. Add a dedicated gate:summary-length branch.
…review) The assertion 'the valid sibling must NOT land either' was vacuous: with minSummaryLength=50 the 2-char sibling summary failed its OWN length check, so no valid sibling existed in the batch. Make it 60 chars (max check disabled at 0) so the test actually proves a valid range is held back by a foreign range's length failure.
… review) The pre-check screens consumed+refold candidates alongside fresh folds, but the new suite only covered fresh folds. Add the missing direction: fold m00001..m00002, restore the block inline (refold candidate), then request an oversized refold summary plus a fresh valid sibling in one batch — nothing may commit and the restored block must keep its old summary (no partial in-place refold may precede the batch rejection). Fails on the pre-gate per-range commit (the fresh sibling would land).
6458980 to
eed105e
Compare
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决 —— PR #3670 分支已 rebase 到当前 master( 结构化八问
行为变更披露:acp-kernel 版本记录 0.0.132(原 PR)→ 0.0.134——纯因等待期间 master 内核版本前进的机械同步,无行为差异。 判定:冲突已清、分支就绪,仍待人工合并(kernel 变更 = 人工审核层):#3670 |
…one seam per burst (#3003) Consecutive compress rounds currently pay one cache seam each: every committed fold rewrites the outbound prefix immediately, so the next re-request inside the burst re-pays the whole cache. #3670 already made the commit itself all-or-nothing for length-class failures; this closes the success-path seam. Design (fold-hold): - The fold still commits to session state immediately (ledger, refs, consumed classification, block contents persisted — restart-safe). - The outbound VIEW defers while the burst continues: any request that carries a NEW successful compress result renders held blocks raw (prefix byte-identical → full cache hit on the raw tail), the first request WITHOUT a new fold result materializes all held blocks — exactly one seam for the whole burst. - Window pressure (tokenCount >= 90% of the effective window, below the 95% emergency band) materializes immediately; preflight/emergency folds pass holdFold:false and render folded on the rescue view. - Loop lane: refresh condition now skips the settle re-request refresh while folds are held (raw re-request view is prefix-stable with the request that triggered it — better than today, where the settle round itself pays a seam); relay refreshFolded callback materializes. - Hold view is a clone with held blocks {active:false, expanded:true} (the kernel's sanctioned render-raw channel, sync.ts deliberately leaves expanded blocks inactive; fork-adoption precedent). Active flags are restored before state adoption; a genuine user decompress mid-burst is never resurrected (keyed on pre-clone activity). - Nudge ranges fully covered by held blocks are filtered from the injected nudge during hold legs (the view cannot fold them; kernel already-covered refusals would spin #156). - Knob: BILI_DEFER_MATERIALIZATION / compress.deferMaterialization, default on; =0 restores the eager contract everywhere. Worst case (gap-separated folds) is byte-identical to today's behavior. Eager-vs-hold full-state dump: persisted state identical, only the settle-round and next-request bodies stay raw; goldens regenerated with the semantic assert updated (carrier by materialization body, foldAt+2). Tests: unit fold-hold (10), loop raw-hold + pressure materialize (#3003a/b), proxy seam e2e (K consecutive folds → exactly ONE raw→folded transition; escape hatch pins eager), storage goldens regen, 29-file regression batch green.
What & why
Multi-range
compresscommitted per-range: when one range's summary failed length validation (over/under cap) while sibling ranges were valid, the valid siblings landed first — opening a seam (the next request re-pays everything after the fold point) — and the model's retry of the bad range paid a second seam. This is the PARTIAL-fold double-seam cost tracked in #3003 (production instance #2997 fold #12: S=34.1K saved vs T=237.9K re-paid, ledger verdict NOT PAID BACK). It is a cost issue, not a stability bug — probe PR #3002 confirmed seams are head-stable.Summary length is a pure function of
(spec.summary, spec.summaryMaxChars, cfg)— no session state, no cross-range coverage — so it can be screened before any block commits. This makesapplyCompressionreject the WHOLE batch all-or-nothing when any fold candidate fails length validation: zero commits, zero seams, so the model's single retry pays once instead of twice.Content-class failures stay per-range. Protected-zone-all-empty, turn-integrity split, and no-new-messages errors depend on state + cross-range coverage and cannot be pre-screened against the original state; they keep their existing per-range behavior.
Scope of the gate (deliberate)
The pre-check runs over exactly the fold-candidate set that mutates state (
status === "ok"non-skipped, plusstatus === "consumed"refolds whose decision kind is"refold"). It does NOT run over every requested range: a consumed non-refold range or an unknown/invalid ref whose summary merely happens to be out of range must not kill a batch whose real candidates all validate.Behavior change (old -> new)
Intentional and approved (#3003 floor 13): trade one extra model round-trip (re-writing the good summaries once more) to save one prefix-cache seam — a seam (tens-hundreds of K tokens re-paid) dwarfs the rewrite (output tokens). Escape hatch: the rejection message offers two exits — shorten/split the offending summary and resend the whole batch, OR drop the offending range(s) and resend only the rest.
The existing nudge guidance (
src/compress-tool.tsSUMMARY_BUDGET_NOTE: "a single oversized summary fails the WHOLE compress call — nothing gets folded") now matches actual behavior — previously it described the intended contract while the kernel did partial commits.Rejection message
Names each offending range + overflow amount and gives both exits:
Host-side companion (separate commit)
src/loop/core.ts: count fold-rounds per turn and log when a turn carries >=2 folds — the residual multi-seam cost that survives after the length gate (multiple independent compress calls across rounds still pay multiple seams). Purely additive logging; feeds the deferred decision on delayed materialization (not implemented here — needs instrumentation data first).kernel-change
kernel-change:the per-range commit lives inside the kernel'sapplyCompression; only the kernel can reorder length-validation ahead of the commit loop. No host-side change can make a partial commit atomic, because the host receives an already-partially-applied result.Config surface
None. No new fields/env vars/flags. The existing
compress.maxSummaryLength/minSummaryLength/ per-specsummaryMaxCharsare reused unchanged.Tests
New
kernel/tests/issue3003-all-or-nothing-length.test.tspins both directions of the gate (#3617): too-long rejects the batch, too-short rejects the batch, all-valid commits every range (control), and a non-candidate range with an out-of-range summary does NOT reject a valid fresh fold (scoping). Three existing tests that asserted the old partial behavior were updated to the new contract (compress.test.tsbatch test + empty-summary assertion;validation.test.tsempty-summary batch test). Kernel suite green locally (1304/1304); full bili suite/e2e gated on CI.Closes #3003.