Skip to content

fix: drain first-sight mass in one batched compress call (#2326) - #2437

Merged
ranxianglei merged 4 commits into
masterfrom
2026-10-08_first-sight-batch
Oct 9, 2026
Merged

ranxianglei merged 4 commits into
masterfrom
2026-10-08_first-sight-batch

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #2326 (stage 1, host-side — the ework-recommended main route; kernel decideNudge structural change stays as fallback behind the human gate).

What

First-sight mass digestion (pre-inflated history handed to bili by a host: dsh plugin mode after its own digestion, CLI hand-off, resumed rollout) cascaded small front-positioned folds — one compress call per re-armed nudge. Every front fold rewrites the request prefix, so each piecemeal landing re-pays the ENTIRE remaining tail to the prefix cache: measured 2.29×/tok re-pay for a 5-fold cascade (eb985b31, 1,383,030 tok) vs 0.032×/tok for one big fold (5c5c26d1, 20,305 tok) ≈ 70×; multi-fold penalty ≈ 1.36M tok ≈ 26% of that session's total miss.

Root constraint beyond guidance: a one-call batch was physically unexecutable for big masses — the model's per-response output budget cannot hold N × 20K-char summaries, so the existing batching hints (SUMMARY_BUDGET_NOTE, ONE_CALL_HINT) degrade to piecemeal folds no matter how they are phrased.

How (no kernel change, no new config surface)

  • withFirstSightDrain(text, reason, external) (src/compress-tool.ts) keys on the kernel's existing [first-sight mass] reason marker (compress.ts:1880; applyCompression clears the baseline so the marker re-rides every re-armed nudge — the whole digestion window is covered, not just the first nudge).
  • All 12 nudge injection points (4 adapters × advisory/legacy + 新功能:增加模型完全决策的压缩时机nudge触发 #2228 decide-directive) now route through it: during digestion a byte-stable "First-sight digestion" note is appended — drain the WHOLE backlog in ONE batched compress call, split oversized spans into multiple ranges at logical boundaries but submit them all together, keep draining while the nudge repeats.
  • The smooth-transition staged note (withStagedCompressGuidance, smaller/tail-biased/"keep the prefix intact") is suppressed during digestion: it steers exactly opposite to a backlog drain.
  • The decide-directive path keeps the stock span text but the note carries an explicit precedence sentence ("the cited span is the starting point of the batch, not its limit").
  • No marker → byte-identical to the legacy staged path (steady-state guidance unchanged; prefix-cache anchors intact).
  • External variant of the note: with compress.externalSummary enabled the independent summary service writes the summaries, so the model's own response-length budget is not a limit. Recommended pairing: enable compress.externalSummary (operator config, no code) — targets may even point back at the same endpoint+key, because the win comes from the channel (disjoint side requests, ONE landing on the live wire, main response writes no summaries), not the author. Caveats: recipe baseUrl must point at the upstream directly (never at bili itself — recursion), and outputTokens must be raised manually (default 8192 truncates long CJK summaries).

Verification

  • npm run typecheck, npm run build — green (build-info d189be53-dirty).
  • Full unit suite on the branch: 4841 pass / 42 fail, 5 new tests in tests/first-sight-drain.test.ts; the 42-failure set is byte-identical on clean master in this sandbox (process-spawn-heavy lanes: dsh-native, claude-native, installer, 给deepseek定制一个跳转到状态的链接 #1590/dsh中工具会突然消失 #2082 — pre-existing environment flakes, not regressions).
  • ACP_TEST_E2E=1 codex lane (real codex CLI → dist build → local sglang qwen3.8-27b): warmup / load growth / live ACP compress / purity all pass — exercises the touched prepare-responses nudge path.

Not in this PR (per ework review)

  • Kernel decideNudge structural change (fallback if prose guidance proves insufficient in logs) — human-only gate.
  • Direction D root-cause of eb98 (5 folds) vs 5c5c (1 fold).
  • No new config keys (aligns with the review's "倾向不加").

Hosts that hand bili a pre-inflated history (dsh plugin mode, CLI
hand-off, resumed rollout) make the first requests see a large ready
mass with zero growth baseline; the kernel marks those nudges
' [first-sight mass]' and re-arms after every successful fold, and the
model folded the backlog one small call per request. Each
front-positioned fold rewrites the request prefix, so every piecemeal
landing re-pays the ENTIRE remaining tail to the prefix cache
(2.29x/tok re-pay for a 5-fold cascade vs 0.032x/tok for one big fold,
~70x — issue log eb985b31 vs 5c5c26d1). A one-call batch was also
physically unexecutable without external summaries: the model's
per-response output budget cannot hold N 20K-char summaries.

withFirstSightDrain(text, reason, external) keys on the kernel marker
at all 12 nudge injection points (4 adapters x advisory + decide
directive): during digestion it appends a byte-stable batched-drain
note and suppresses the smooth-transition staged note (whose
smaller/tail-biased steering is exactly wrong for a backlog that must
be drained); the decide-directive variant carries an explicit
precedence sentence over the stock single-span wording. No marker ->
byte-identical to the legacy staged path; no new config surface. The
external variant tells the model the summary service writes the
summaries so its own response budget is not a limit — combined with
compress.externalSummary the batched instruction becomes executable
for arbitrarily large backlogs (targets may even point at the same
endpoint+key: the win comes from the channel, not the author).

Tests: tests/first-sight-drain.test.ts (marker keying, byte-identity
outside digestion, staged suppression inside it, external variant,
byte-stability). Verified: typecheck, full unit suite (4841 pass /
42 fail, failure set byte-identical to clean master in this sandbox —
pre-existing env flakes), build, and ACP_TEST_E2E=1 codex lane (real
client, live compression) against local sglang.
…ght-batch

# Conflicts:
#	src/server/prepare-responses.ts
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 冲突已解决,push 完成,GitHub 重新判定 mergeable: true(当前 blocked 仅为常规 review/CI 门禁,非冲突)。

处理过程

验证(定向,全量交 CI)

  • npm run typecheck 绿、npm run build 绿(build-info 6dbaaffff,dist-import-annotations OK);
  • 触达冲突文件的两个 lane 定向测试:tests/first-sight-drain.test.ts + tests/codex-local-compaction.test.ts 共 10 pass / 0 fail;
  • PR API 复核:head=6dbaaffff,mergeable: true。

结构化八问

  1. 是什么问题:PR 分支与 master 产生合并冲突,无法进入合并判定。
  2. 什么原因导致的:热文件竞争——本 PR 与 master 的 fix: rebase fold substrate on Codex local auto-compaction (#2372) #2373 同时改写了 prepare-responses.ts 顶部同一段 import 行(AGENTS.md §7.2 典型模式),文本级碰撞,无语义分歧。
  3. 影响范围与严重程度:仅 PR 自身可合并性;对任何运行行为零影响。
  4. 怎么修复的:merge master 进分支,import 冲突按两侧实际引用取并集,逐符号 grep 验证后删掉本文件中已无引用的旧导入。
  5. 是否回归:无。逐项核过既有行为——master 的 [bug] Codex local auto-compaction can destroy existing bili fold substrate instead of rebasing #2372 codex local-compaction rebase 检测块原样自动合入(代码未动)、本 PR 的 12 处 nudge 注入改动未动、import 解析后每个保留符号都有对应使用点、每个被删符号在合并体内均无引用;typecheck+build+双 lane 测试全绿,无 fail-fast 变吞错、无默认值/时序/输出格式漂移。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及。

结论:冲突解除,分支已就绪,CI 绿灯后可直接合并。

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-08_first-sight-batch (9dcc381)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2663-pr.2437.65b828a

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2437.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

PR #2437 正式 review(结构化八问)

先纠一个我上条的误报:我一度以为 diff 有 110 文件/8962 行,那是拿过期的本地 master 当 base 算的——用新鲜 origin/master 重算,本 PR 就是干净的单提交 b554d4d,8 文件 +127/−17,纯 host 侧、不碰 kernel、不动 version/lockfile。方向与实现都对得上你上一条的设计。下面按八问。

① 方向关

命中根因(「每请求一折×N」由 kernel 重武装 + 模型输出预算共同造成),且选最外科的宿主侧路径避开 kernel 门禁 + 用 externalSummary 解除输出预算瓶颈。方向无大问题。

② 回归关(审过的既有行为 + 各自结论)

  1. 稳态 nudge 文案(无 first-sight 标记):byte-identical(设计委托 withStagedCompressGuidance,测试fix: address all code review findings (condense, routing, session-id, dead code) #2 用 exact-equality 钉住)。✅保持
  2. 消化期 nudge 文案:有意变更(追加排水注记 + 抑制 staged),CHANGELOG 已披露 old→new。✅有意变更已披露
  3. BILI_MAX_SHRINK 用户稳态 staged 引导:不变(抑制仅 scope 到标记窗口内;测试fix: address all code review findings (condense, routing, session-id, dead code) #2「switch on, no marker → staged still applies」)。✅保持
  4. 新功能:增加模型完全决策的压缩时机nudge触发 #2228 decide-directive 路径:消化期内被排水注记 + 优先级句覆盖(有意);稳态 byte-identical。✅
  5. google adapter 缺 summary-budget note 的既有不对称:保留未动(非本 PR 引入;排水注记自带 cap 说明,消化期自洽)。中性
  6. 两压缩模式:标记由 kernel 打(不分模式),12 处注入点在共用 prepare*(proxy/plugin 都走)→ 两模式一致。✅
    高危模式逐项:无 fail-fast→吞错;无阈值/时序漂移(唯一 gate 是精确子串匹配);输出格式仅消化期变(=修复本身);缓存:排水注记 byte-stable(测试feat: persistent + namespaced sessions, multi-tenant session isolation, stream robustness #5),窗口内每请求字节相同→不引入额外 churn。✅无无意回归。

③ 代码关

  • helper 逻辑正确:no-marker→委托旧路径(undefined-safe);marker→追加对应变体且不调 staged(=抑制)。✅
  • 12 处注入点全覆盖且一致(4 adapters × {base nudge / decide-directive yes-branch / fallback}),全部传 turn.nudge.reason + externalSummaryEnabled(config)。✅ 我上条 checkpoint#1 满足。
  • 排水注记 vs summary-budget-note 并存:两者其实兼容(都要求「拆小 range 且同一调用提交」),排水注记额外强调「覆盖全部 backlog + 别跨 turn」,无实质矛盾。✅
  • 非阻塞小瑕疵:非 external 变体断言「ONE batched call / cover EVERY ready range」,大 backlog 下受模型单次输出预算限制可能装不下全部摘要→措辞偏乐观,建议软化为「一次调用能容纳多少就排多少」。低危。

④ 测试有效性关

  • 5 例是真测试、有牙:marker 键控、稳态 byte-identity(exact equality)、消化期抑制 staged、external 变体、byte-stable。✅ 文本管道该钉的不变量都钉了。
  • 缺口:没有测试验证行为结果(级联是否真减少折叠次数/重付)。这是散文引导法固有局限(无法单测模型服从度),但意味着上线时没有实证证明主症状(70× 重付)确实下降——尤其对 non-external 默认部署(issue 里的 dsh 场景)。e2e codex lane 全绿只证明压缩仍工作(无回归),不证明级联缩小。

⑤ diff 整洁度

干净。单提交,8 文件,全部与本 PR 目的相关,无跑题文件 / whitespace 噪音 / lockfile 连带。✅


八问结论

  1. 是什么问题:宿主交付预膨胀历史后,bili 首见消化把同一份积压拆成多次靠前小折,每次落线把剩余尾部整段重付前缀缓存(单发大折 vs 五连折实测 ~70×)。
  2. 什么原因:kernel 首见 mass 每次成功折叠后重武装 nudge(每请求一折)+ 模型单次响应输出预算装不下 N 份摘要→批处理指令退化为逐请求小折。
  3. 影响范围/严重度:dsh plugin 等交付预膨胀会话的场景;随宿主交付频率线性放大(日志一段五连折≈占 miss 26%)。属成本/缓存效率问题,非功能损坏(会话仍能正常压缩,只是多花钱)。
  4. 怎么修复:host 以 kernel 既有 [first-sight mass] 标记为键,在消化窗口 12 个 nudge 注入点追加 byte-stable 排水注记 + 抑制反向的平滑过渡注记;externalSummary 联动把摘要移出主响应,使「一次批量」在大 backlog 下可执行。不改 kernel、不加配置面。
  5. 是否回归:无无意回归(见②)。但分支落后 master 20 提交,其中 post-compress tail immediately re-advertises newly-released ranges -> chained folds at zero growth -> upstream prefix-cache misses #2404「pin post-compress tail to the pre-fold nudge offer」同样改这 12 处的 nudge-offer 组装,merge-tree 显示 3 文件 changed-in-both→必须先 rebase 当前 master 并人工核对 post-compress tail immediately re-advertises newly-released ranges -> chained folds at zero growth -> upstream prefix-cache misses #2404 与 fix: drain first-sight mass in one batched compress call (#2326) #2437 在这 12 处是否正确合成,否则语义冲突风险。
  6. 是否破坏缓存:不破坏反而省。排水注记 byte-stable、窗口内每请求字节相同;折叠本就重写(与旧行为同)。
  7. 是否改设计方向:否。仍在「kernel 决策 + host 注入文案」边界内,只是让 host 文案首见感知。kernel 结构化改法留作后续(人工门)。
  8. 是否引入新配置:否。只读既有 compress.externalSummary,零新增 config 面。

判定:方向与实现合格,但暂不可直接合并,两个前置

(a) 必须 rebase 当前 master(§7.2):落后 20 提交,#2404 与本 PR 挤在同一批 nudge 注入行,rebase 后本地 typecheck + 目标测试 + build 复绿再交 CI。
(b) efficacy 证据缺口:non-external(默认,含 dsh)场景效果受模型输出预算+服从度限制,现有测试只钉文本管道、不钉行为结果。二选一:①补一个 dsh-like non-external first-sight 定向 e2e,断言消化期折叠次数/重付 token 相对 baseline 下降;②至少在 PR/issue 明确「完整收益需 externalSummary,non-external 为尽力而为」并附实测数据。

满足 (a)(b) 我认为可合。(a) 是你正在实现的分支,我不擅自动;如果你 rebase 后想让我把那条「非 external 措辞软化」的小修推上去,说一声我就办。

ranxianglei and others added 2 commits October 9, 2026 09:53
#2326)

Fake-upstream harness (no model): when the kernel nudge reason carries
' [first-sight mass]', the forwarded body must carry the batched-drain
note and must NOT carry the staged smooth-transition note (suppression
scoped to digestion); steady-state nudges stay byte-identical to the
legacy guidance; small/thin sessions never see either artifact.

Also documents two sizing pitfalls the harness has to steer around:
- a fresh session's first request sizes on 0 (#1492/#1820), so the
  first-sight nudge rides r2, sized by r1's usage report;
- the model-table window (claude-sonnet-4-5 → 200K) outranks the
  startServer modelContextLimit fallback — pin via compress.modelContextLimit.
…ght-batch

# Conflicts:
#	src/server.ts
#	src/server/prepare-openai.ts
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 第二轮冲突已解决并推送(c2567d578..9dcc3816c,merge 方式,无 force-push)。

过程:master 又前进了 ~20 个提交(#2429 relay 抽取把 src/server.ts 削掉 1579 行、#2432 dsh local compaction rebase + breaker-aware 进入 prepare-openai.ts、kernel bump 0.0.109 等),与本 PR 在同一批 import 块上再次相撞。另外分支上先 fast-forward 收进了另一会话推入的 wire-level 回归测试提交 c2567d578(tests/first-sight-drain-wire.test.ts,+247 行,test-only,与本次修复同机制,保留)。

冲突面与处置(逐符号 grep 合并后正文按实际使用裁决,不做盲目并集):

  • src/server/prepare-openai.ts:取 master 新增的 compressBreakerArmed(stream.js)、carriesDshLocalCompactionSummary/DSH_LOCAL_COMPACTION_MIN_MISSING(dsh-compaction-guard.js)、session.js 的 markNativeCompactionBoundary/reconcileNativeCompactionBoundary;compress-tool 行保留本 PR 的 withFirstSightDrain 替换、去掉 withStagedCompressGuidance(合并后正文零引用)。
  • src/server.ts:取 master 抽取后精简的 absorb.js/store.js/compress-tool 三行;其中 compress-tool 行同样去掉 withStagedCompressGuidance——master 侧它也只是抽取遗留的死 import(正文零引用,本 PR 本来就已移除)。
  • 验证:全仓无残留冲突标记;12 个 drain 调用点(4 适配器 × advisory/legacy/directive)全部在位;withStagedCompressGuidance 在 src/compress-tool.ts 之外零引用。typecheck 绿、build 绿(build-info 9dcc3816c)、定向 lane 39/39 过(first-sight-drain + wire + codex-local-compaction + issue2432-dsh-local-compaction + dsh-compaction-guard——覆盖 merge 两侧代码)。全量套件交 CI。

八问:

  1. 是什么问题:PR 分支与 master 第二次产生文本冲突(两个文件的 import 块),merge state 转 dirty。
  2. 什么原因导致的热文件竞争:本 PR 改的是「staged→drain」的 import 替换,master 侧 refactor(server): extract forward() relay zone into src/server/relay module (#1440 P2 cut 4) #2429 大规模抽取精简了同一批 import、[bug] DSH 桌面端 0.1.188:宿主原生压缩落地摧毁折叠基底 → 同一区间永久不可锚定、compress 连片失败并熔断(compaction-basic auto:false 仍无效) #2432 又往 prepare-openai.ts 加新 import——三方在同几行上各自为政。属 §7.2 点名的 src/server*.ts 热点文件模式,非任何一方错误。
  3. 影响范围与严重程度:仅本 PR 分支不可合并状态;对 master、线上用户零影响。
  4. 怎么修复的:merge master 进分支,冲突按「合并后正文实际引用」逐符号裁决——master 的新增 import 与其新代码一体保留,本 PR 的 drain 路由完整保留,死 import 不复活。
  5. 是否有回归问题:审过的既有行为——12 个 nudge 注入点的 drain 路由(本 PR 核心)、master 新增的 dsh/codex local compaction rebase 与 breaker-aware 路径,均被定向 lane 钉住且全绿;未触碰任何语义,纯 import 层收敛。
  6. 是否破坏缓存:不涉及(import 解析,无运行时行为变化)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及。

GitHub 现报 mergeable: true(state blocked 是常规 review/CI 门禁,非冲突)。CI 绿灯后可直接合并。

@ranxianglei
ranxianglei merged commit 4b56664 into master Oct 9, 2026
28 checks passed
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.191-master.4b56664 (rolling master tag).

npm install -g billion-context@0.0.191-master.4b56664  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@ranxianglei
ranxianglei deleted the 2026-10-08_first-sight-batch branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[kernel] first-sight-mass 消化期连发小折重复重付尾部(单发大折可省约 70x)

1 participant