Repository navigation
fix: drain first-sight mass in one batched compress call (#2326) - #2437
Conversation
Hosts that hand bili a pre-inflated history (dsh plugin mode, CLI hand-off, resumed rollout) make the first requests see a large ready mass with zero growth baseline; the kernel marks those nudges ' [first-sight mass]' and re-arms after every successful fold, and the model folded the backlog one small call per request. Each front-positioned fold rewrites the request prefix, so every piecemeal landing re-pays the ENTIRE remaining tail to the prefix cache (2.29x/tok re-pay for a 5-fold cascade vs 0.032x/tok for one big fold, ~70x — issue log eb985b31 vs 5c5c26d1). A one-call batch was also physically unexecutable without external summaries: the model's per-response output budget cannot hold N 20K-char summaries. withFirstSightDrain(text, reason, external) keys on the kernel marker at all 12 nudge injection points (4 adapters x advisory + decide directive): during digestion it appends a byte-stable batched-drain note and suppresses the smooth-transition staged note (whose smaller/tail-biased steering is exactly wrong for a backlog that must be drained); the decide-directive variant carries an explicit precedence sentence over the stock single-span wording. No marker -> byte-identical to the legacy staged path; no new config surface. The external variant tells the model the summary service writes the summaries so its own response budget is not a limit — combined with compress.externalSummary the batched instruction becomes executable for arbitrarily large backlogs (targets may even point at the same endpoint+key: the win comes from the channel, not the author). Tests: tests/first-sight-drain.test.ts (marker keying, byte-identity outside digestion, staged suppression inside it, external variant, byte-stability). Verified: typecheck, full unit suite (4841 pass / 42 fail, failure set byte-identical to clean master in this sandbox — pre-existing env flakes), build, and ACP_TEST_E2E=1 codex lane (real client, live compression) against local sglang.
…ght-batch # Conflicts: # src/server/prepare-responses.ts
[bot] 🏷 冲突已解决,push 完成,GitHub 重新判定 处理过程
验证(定向,全量交 CI)
结构化八问
结论:冲突解除,分支已就绪,CI 绿灯后可直接合并。 |
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2663-pr.2437.65b828aThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2437.tgz
npm install -g packageInstall instructions are refreshed on each push. |
PR #2437 正式 review(结构化八问)先纠一个我上条的误报:我一度以为 diff 有 110 文件/8962 行,那是拿过期的本地 master 当 base 算的——用新鲜 ① 方向关命中根因(「每请求一折×N」由 kernel 重武装 + 模型输出预算共同造成),且选最外科的宿主侧路径避开 kernel 门禁 + 用 externalSummary 解除输出预算瓶颈。方向无大问题。 ② 回归关(审过的既有行为 + 各自结论)
③ 代码关
④ 测试有效性关
⑤ diff 整洁度干净。单提交,8 文件,全部与本 PR 目的相关,无跑题文件 / whitespace 噪音 / lockfile 连带。✅ 八问结论
判定:方向与实现合格,但暂不可直接合并,两个前置(a) 必须 rebase 当前 master(§7.2):落后 20 提交,#2404 与本 PR 挤在同一批 nudge 注入行,rebase 后本地 typecheck + 目标测试 + build 复绿再交 CI。 满足 (a)(b) 我认为可合。(a) 是你正在实现的分支,我不擅自动;如果你 rebase 后想让我把那条「非 external 措辞软化」的小修推上去,说一声我就办。 |
#2326) Fake-upstream harness (no model): when the kernel nudge reason carries ' [first-sight mass]', the forwarded body must carry the batched-drain note and must NOT carry the staged smooth-transition note (suppression scoped to digestion); steady-state nudges stay byte-identical to the legacy guidance; small/thin sessions never see either artifact. Also documents two sizing pitfalls the harness has to steer around: - a fresh session's first request sizes on 0 (#1492/#1820), so the first-sight nudge rides r2, sized by r1's usage report; - the model-table window (claude-sonnet-4-5 → 200K) outranks the startServer modelContextLimit fallback — pin via compress.modelContextLimit.
…ght-batch # Conflicts: # src/server.ts # src/server/prepare-openai.ts
[bot] 🏷 第二轮冲突已解决并推送( 过程:master 又前进了 ~20 个提交(#2429 relay 抽取把 src/server.ts 削掉 1579 行、#2432 dsh local compaction rebase + breaker-aware 进入 prepare-openai.ts、kernel bump 0.0.109 等),与本 PR 在同一批 import 块上再次相撞。另外分支上先 fast-forward 收进了另一会话推入的 wire-level 回归测试提交 冲突面与处置(逐符号 grep 合并后正文按实际使用裁决,不做盲目并集):
八问:
GitHub 现报 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.191-master.4b56664 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Fixes #2326 (stage 1, host-side — the ework-recommended main route; kernel
decideNudgestructural change stays as fallback behind the human gate).What
First-sight mass digestion (pre-inflated history handed to bili by a host: dsh plugin mode after its own digestion, CLI hand-off, resumed rollout) cascaded small front-positioned folds — one compress call per re-armed nudge. Every front fold rewrites the request prefix, so each piecemeal landing re-pays the ENTIRE remaining tail to the prefix cache: measured 2.29×/tok re-pay for a 5-fold cascade (eb985b31, 1,383,030 tok) vs 0.032×/tok for one big fold (5c5c26d1, 20,305 tok) ≈ 70×; multi-fold penalty ≈ 1.36M tok ≈ 26% of that session's total miss.
Root constraint beyond guidance: a one-call batch was physically unexecutable for big masses — the model's per-response output budget cannot hold N × 20K-char summaries, so the existing batching hints (SUMMARY_BUDGET_NOTE, ONE_CALL_HINT) degrade to piecemeal folds no matter how they are phrased.
How (no kernel change, no new config surface)
withFirstSightDrain(text, reason, external)(src/compress-tool.ts) keys on the kernel's existing[first-sight mass]reason marker (compress.ts:1880;applyCompressionclears the baseline so the marker re-rides every re-armed nudge — the whole digestion window is covered, not just the first nudge).withStagedCompressGuidance, smaller/tail-biased/"keep the prefix intact") is suppressed during digestion: it steers exactly opposite to a backlog drain.compress.externalSummaryenabled the independent summary service writes the summaries, so the model's own response-length budget is not a limit. Recommended pairing: enablecompress.externalSummary(operator config, no code) — targets may even point back at the same endpoint+key, because the win comes from the channel (disjoint side requests, ONE landing on the live wire, main response writes no summaries), not the author. Caveats: recipebaseUrlmust point at the upstream directly (never at bili itself — recursion), andoutputTokensmust be raised manually (default 8192 truncates long CJK summaries).Verification
npm run typecheck,npm run build— green (build-infod189be53-dirty).tests/first-sight-drain.test.ts; the 42-failure set is byte-identical on clean master in this sandbox (process-spawn-heavy lanes: dsh-native, claude-native, installer, 给deepseek定制一个跳转到状态的链接 #1590/dsh中工具会突然消失 #2082 — pre-existing environment flakes, not regressions).ACP_TEST_E2E=1codex lane (real codex CLI → dist build → local sglang qwen3.8-27b): warmup / load growth / live ACP compress / purity all pass — exercises the touched prepare-responses nudge path.Not in this PR (per ework review)
decideNudgestructural change (fallback if prose guidance proves insufficient in logs) — human-only gate.