Skip to content

[PR] fix: stop the auto-fold fold storm on unreachable targets; document config reload semantics - #3460

Closed
ranxianglei wants to merge 4 commits into
masterfrom
2026-10-17_autofold-sharp-edges
Closed

ranxianglei wants to merge 4 commits into
masterfrom
2026-10-17_autofold-sharp-edges

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Closes #3444

What this fixes

Observation 1 (bug) — sub-floor autoFoldTargetTokens causes a fold storm. When the configured growth target sits below the session's irreducible baseline (system prompt + tool schemas + live tail), no fold can ever reach it: the trigger re-fired on every request and burned a full external-summary round trip per request for ever-smaller slivers (the report shows saves down to ~685 tokens, each costing up to ~60s of chain latency). Two complementary facets, one mechanism family (growth-fold economics):

  1. Measured floor — an exhausted honest growth pass (folded everything foldable, no chain failure) whose rebuilt payload still sits at/above the target now records the post-fold size as the session's irreducible floor (session.metadata.autoFoldFloorTokens, latest measurement wins, tracks baseline drift both ways). The effective target becomes min(max(configured-clamped, floor), window); a floor at/above the overflow bar disarms growth folding entirely and releases the nudge suppression (growthFoldingArmed gains an optional measuredFloor argument, threaded through all four prepare-* lanes so suppression and arming stay consistent).
  2. Benefit gate — a growth fold only fires when the text channel exceeds the budget by at least AUTO_FOLD_MIN_BENEFIT_TOKENS (new internal constant, 4096) of expected saving, and a pass that actually saves less than that arms the standard 10-minute backoff (mirroring the zero-progress path, with auto-fold backoff arms on pure queue-drops (attempts=[]) as if the summary service had failed #2662's queue-drop congestion exemption unchanged). Image-dominant overage keeps its original unconditional arm (images are not foldable); classic overflow preflight (bar = window) is untouched.
    The trigger-side gate stops boundary-crossing churn before it costs a round trip; the post-hoc backoff catches estimation error where the pass runs anyway. Both are needed: floor-only leaves the trigger re-firing at payload≈floor every request (equality fires); gate-only still pays one wasteful pass per real boundary crossing.

Observation 2 (gap) — file edits do not take effect until PUT /__bili/config. Verified mechanism: the engine reads the startup snapshot plus what PUT /__bili/config / POST /__bili/config/reload inject into memory (opts.compress); GET /__bili/config reads the file fresh, hence the asymmetry that trapped the reporter. Nothing watches the raw file — no field does, providers/routes included. This PR ships the docs half (CONFIGURATION.md en+zh now state the reload semantics explicitly). The optional fs-watch auto-reload is a feature decision requested in the issue thread — not built here.

Observation 3 — acknowledged, no action: the 1M window resolution is correct and the per-provider warning is cosmetic by design.

Behavior changes (old → new)

# Old New Why
1 Any payload ≥ configured growth target fired the external chain, every request Growth fold requires ≥4096 tokens expected saving; image-dominant overage unchanged A round trip saving less than its fixed cost was pure waste (the storm)
2 A cheap-but-nonzero fold repeated every request forever A pass actually saving <4096 arms the existing 10-min backoff; classic nudges resume after Same economics as the zero-progress path
3 Unreachable target → permanent per-request re-fire; nudge suppression stuck on First exhausted pass measures the floor, effective target raised to it for the session + actionable warn log naming compress.externalSummary.autoFoldTargetTokens; floor ≥ window disarms growth folding and releases nudge suppression The target is physically unreachable; classic mode is the honest state
4 Silent New warn logs (floor measurement, sub-benefit backoff) Operator needs to see the misconfiguration and its effect

No change to wire shape, tool calls, message format, persistence format (additive session-metadata key, same pattern as autoFoldBackoffUntil), or classic overflow preflight. Healthy configs (target reachable with headroom) behave exactly as before — the pre-existing e2e lanes (fold fires / off-control / fail-open / backoff-on-failure / queue-drop / degenerate-window) all pass unchanged.

Config surface

No new config fields, env vars, or CLI flags. AUTO_FOLD_MIN_BENEFIT_TOKENS is an internal behavior constant (same class as AUTO_FOLD_BACKOFF_MS), deliberately not owner-tunable — the escape hatch for big-window users wanting a lazier cadence remains compress.nudgeGrowthTokens/raising the target itself.

Evidence

  • New e2e lane A: ~15K-token system-prompt baseline vs 8192 target — turn 1 folds once (floor recorded), turn 2 adds ~1K real growth and must NOT re-run the chain (pre-fix it unconditionally did).
  • New e2e lane B: ~15K-token system-prompt baseline + two small pairs — turn 1 fires on a large expected benefit yet saves only ~1K real tokens (< 4096), arming the backoff; turn 2 adds one fresh ~8K-token foldable pair that pre-fix code re-folds every request (the reported storm shape) while the fixed build skips it during cooldown.
  • Review follow-up (commit c271011): the original lane-B fixture assumed its giant final user message was protected live-tail content, but nothing guards it — turn 1 folded it too (~17K saved ≥ 4096), so the sub-benefit arm never triggered and the lane passed even with the fix reverted. Fixture rebuilt on the lane-A pattern; teeth re-verified: all four new auto-fold sharp edges: a sub-floor autoFoldTargetTokens causes a fold storm, and externalSummary.* file edits do not take effect until a PUT /__bili/config #3444 tests fail with src reverted to origin/master.
  • Truth-table rows for the measuredFloor argument + autoFoldFloor invalid-value unit test.
  • Local targeted run (rebased head): tests/external-summary-autofold.test.ts 11/11 green; npm run typecheck clean. Rebased onto current master (was stale/conflicted vs the Vertex merge feat(vertex): run Claude Code on Vertex AI through bili #3323); full-suite verdict = CI on this head.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到。这是已完成的 PR 工作项(Closes #3444),不重做,进入核对流程:代码审查 + 既有行为回归清点 + CI 状态确认。

查重:不重复。 相关但在途的其他条目机制不同,不构成重复:#2638/#2660(preflight 每轮重新提名结构性死区间——死 span 重提名,非目标不可达)、#3353(提交前拒绝覆盖回退的 fold——另一失败域)、#3442(compress 绑到无 ref 的 |sub: lane)、#3443(Web UI Save Key 路由 404)。#3460 是 #3444 的唯一修复项。

一个需要处理的状态:PR 分支基于 ce9bc90a3(#3430 合入点),当前 master 已推进到 774da9b3c(#3323、#3421 已合入),GitHub 标记为 dirty(冲突)。我会把分支 rebase 到当前 master、解冲突、本地定向验证后 push,再按八问给最终结论。

@ranxianglei
ranxianglei force-pushed the 2026-10-17_autofold-sharp-edges branch from 3dc9af6 to 4892680 Compare October 17, 2026 00:46
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-17_autofold-sharp-edges (33d5a18)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4093-pr.3460.28c3d0d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3460.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei
ranxianglei force-pushed the 2026-10-17_autofold-sharp-edges branch from 4892680 to ec50a32 Compare October 17, 2026 00:47
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到上游冲突转发,已把本 PR 分支 rebase 到最新 master、解决冲突并强推回 PR head。下面是结构化结论(八问)。

1. 是什么问题
PR #3460(auto-fold 折叠风暴修复 + 配置重载语义文档)在开放期间,master 前进了 12 个提交,导致 PR 分支与目标分支产生文本冲突,GitHub 标为 dirty / 不可合并。这是一次例行的「rebase 到当前 master」,不是代码缺陷。

2. 什么原因导致的
真正的冲突只有一处:src/server/prepare-anthropic.ts 里同一行被两个独立改动同时碰到——master 把 nudge 诊断日志的参数从 parsed.model 改成了 resolvedModel ?? parsed.model;本 PR 给 growthFoldingArmed(...) 新增了「实测下限」参数 autoFoldFloor(session)。两处改动落在相邻文本上 → 文本重叠 → 冲突。其余文件均自动合并成功。

3. 影响范围与严重程度
仅限 PR 分支与 master 的同步,不引入任何超出 PR 本身已声明范围的用户可见行为变化。严重度:无——此前它只是把 PR 卡死在 dirty 状态,现已清除。

4. 怎么修复的
把 2 个提交的 PR rebase 到当前 master。唯一冲突的 hunk 取「两者意图的并集」:既保留 master 的 resolvedModel ?? parsed.model 日志参数,也保留 PR 的 autoFoldFloor(session) 下限参数;并已确认两个符号都在作用域内(一个是函数形参、一个是已 import 的符号)。随后本地验证:typecheck 通过、bundle build 通过、PR 自带的回归套件 tests/external-summary-autofold.test.ts 11/11 全绿。最后把 rebase 后的分支强推回 PR head。

5. 是否有回归问题
无。rebase 完整保留了 PR 原有 diff(已核对:相对 master 的 diff == 原始 PR diff,10 个文件 +247/-19)。手工解决处是两个已各自审过的改动的严格并集,没有引入任何新逻辑。CI 静态门禁(kernel-guard / version-guard / config-surface-guard / docs-* / lock-sync / build-artifact / unused-exports 等)在新 head 上全部通过;全量 test 分片仍在 CI 上跑,其结果为准。

6. 是否破坏缓存
不涉及。未改 wire / 消息形状 / 持久化格式;冲突点在一条 nudge 注入布尔表达式和一行日志上,前缀缓存与会话持久化均未触碰。

7. 是否改了设计方向
不涉及。设计与 PR 原方案一致,本次仅做 rebase。

8. 是否引入了新的配置
不涉及。无新增配置字段 / 环境变量 / CLI flag(与 PR 自身「Config surface: none」一致)。


收工判定:冲突已解决,PR 现为可合并状态(mergeable=True),当前仅因全量 CI test 分片仍在运行而显示 blocked。CI 转绿后即可人工 review / 合并。

ranxianglei pushed a commit that referenced this pull request Oct 17, 2026
…he under-cost arm

Review finding (PR #3460): the original lane-B fixture assumed its ~15K-token
final user message was protected live-tail content, but with
preserveRecentMessages: 0 nothing guards it — turn 1 actually folds it too
(measured: 2 range(s), ~17242 tokens saved vs 4096 cost bar), so the
sub-benefit backoff at server.ts never armed and the lane passed even with
the fix reverted. It pinned the legacy zero-progress path, not the new one.

Rebuilt the shape on the lane-A pattern: ~15K-token BIG_SYSTEM (unfolderable)
+ two small pairs (~2K foldable). Turn 1 then fires on a large expected
benefit yet saves ~1K real tokens (< 4096) — arming the sub-benefit backoff;
turn 2 adds one fresh ~8K-token pair that pre-fix code re-folds every request
(the reported storm shape) while the fixed build skips it during cooldown.
Verified both timelines: 11/11 post-fix; tests 6-9 all fail pre-fix.
@ranxianglei
ranxianglei force-pushed the 2026-10-17_autofold-sharp-edges branch 4 times, most recently from c215af5 to 4b20600 Compare October 17, 2026 01:22
awork added 4 commits October 17, 2026 09:34
…floor + benefit gate (#3444)

A configured autoFoldTargetTokens below the session's irreducible baseline (system prompt + tool schemas + live tail) could never be reached: the trigger re-fired on every request and burned a full external-summary round trip per request for ever-smaller slivers (reported saves down to ~685 tokens, each costing up to ~60s of chain latency).

Two complementary facets, same mechanism family (growth-fold economics):

1. Measured floor: an exhausted honest growth pass (folded everything foldable, no chain failure) whose rebuilt payload still sits at/above the target records the post-fold size as the session's irreducible floor (session metadata, latest measurement wins). The effective target is raised to max(configured, floor, MIN) capped at the window; a floor at/above the overflow bar disarms growth folding entirely and releases the nudge suppression (growthFoldingArmed gains a measuredFloor argument, threaded through all four prepare-* lanes).

2. Benefit gate: a growth fold only fires when the text channel exceeds the budget by at least AUTO_FOLD_MIN_BENEFIT_TOKENS (4096) of expected saving, and a pass that actually saves less than that arms the standard 10-minute backoff (#2662 queue-drop congestion exemption mirrored). Image-dominant overage keeps its original unconditional arm (images are not foldable). Classic overflow preflight (bar = window) is untouched.

Regression tests: two e2e lanes (sub-floor system-prompt baseline stops re-folding marginal growth; giant-live-tail sub-benefit pass arms the backoff) plus truth-table rows for the floor argument and an autoFoldFloor invalid-value unit test.
…oad semantics (#3444)

CONFIGURATION.md en+zh: the ~4096-token benefit gate and the measured-floor clamp for unreachable targets (with the actionable warning naming compress.externalSummary.autoFoldTargetTokens), and the reload asymmetry behind the second observation — file edits take effect only via Web UI/API save, POST /__bili/config/reload, or restart (every section, providers/routes included), while GET /__bili/config reflects the file immediately.
…he under-cost arm

Review finding (PR #3460): the original lane-B fixture assumed its ~15K-token
final user message was protected live-tail content, but with
preserveRecentMessages: 0 nothing guards it — turn 1 actually folds it too
(measured: 2 range(s), ~17242 tokens saved vs 4096 cost bar), so the
sub-benefit backoff at server.ts never armed and the lane passed even with
the fix reverted. It pinned the legacy zero-progress path, not the new one.

Rebuilt the shape on the lane-A pattern: ~15K-token BIG_SYSTEM (unfolderable)
+ two small pairs (~2K foldable). Turn 1 then fires on a large expected
benefit yet saves ~1K real tokens (< 4096) — arming the sub-benefit backoff;
turn 2 adds one fresh ~8K-token pair that pre-fix code re-folds every request
(the reported storm shape) while the fixed build skips it during cooldown.
Verified both timelines: 11/11 post-fix; tests 6-9 all fail pre-fix.
…benefit gate

CI (ubuntu-latest/22 shard 3 on c271011): 'cancelled growth auto-fold
stops before fail-open forwarding and backoff' failed at its trigger-log
assertion. The lane predates this PR and pinned the OLD trigger semantics:
any payload >= target fired the external chain. Its fixture (usage baseline
10K vs 8192 = AUTO_FOLD_TARGET_MIN) carries only ~1.8K expected saving, so
the new benefit gate (>= 4096, behavior-change row 1 of this PR — intended)
no longer arms the fold, and the lane lost its subject.

Raise the usage baseline to 20K (target cannot go lower — 8192 is the
documented minimum): expected saving 20_000 - 8192 = 11_808 >= 4_096, the
fold arms, gets cancelled mid-pass as before, and every original assertion
(fail-fast override, no main forward, no backoff on cancellation, floor
trigger log) is unchanged. Test-only; no src change.
@ranxianglei
ranxianglei force-pushed the 2026-10-17_autofold-sharp-edges branch from 4b20600 to 33d5a18 Compare October 17, 2026 01:36
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #3460 审核完成。结论先行:修复逻辑成立,全量 CI 绿,可提交人工审核合并。审核过程中我在 PR 分支上直接修了两个测试缺陷(一个本地发现、一个被 CI 抓住),详见第 5 问。

结构化八问

1. 是什么问题:auto-fold 目标值(autoFoldTargetTokens)低于会话不可压缩基线时(如 40K 目标 vs ~43K 基线),growth fold 每个请求都重新触发、每次都跑外部摘要链(最高 ~60s),却只省几百 token——即「fold storm」;同时配置文件编辑要经 UI/API 保存或重启才生效,文档没写清。

2. 什么原因导致的:同一机制族(growth-fold 经济性)的两个面:① 触发条件只看「payload > 目标」不看目标是否可达——基线高于目标时折叠毫无收益仍会开火;② 开火没有收益下限——省 685 token 也算进度,既有 no-progress 保护接不住。配置重载是快照模型(引擎读启动快照 + PUT/reload 注入的内存态),文件监听从未实现,属设计现状而非缺陷。

3. 影响范围与严重程度:仅影响开启 external-summary auto-fold 且配置目标低于实际不可压缩基线的会话(报告者场景);其余用户行为不变(见回归清单)。严重度为性能/成本退化(每轮风暴多几十秒延迟 + 外部调用费用),无数据丢失、无功能不可用。

4. 怎么修复的:两个机制 + 文档,一个闭环:

  • 测得地板(measured floor):一次无链失败走完的诚实 growth pass 后,若重建后 payload 仍 ≥ 目标,把该尺寸记入 session.metadata.autoFoldFloorTokens;有效目标 = clamp(配置值, 地板, overflow)。地板 ≥ 门槛后 growth fold 解除武装并释放 nudge 抑制——不再反复徒劳尝试。
  • 收益门(benefit gate):预期节省(decisionTrigger − textBudget)≥ 4096 token 才允许 growth fold 开火(新内部常量 AUTO_FOLD_MIN_BENEFIT_TOKENS,非配置项);实际节省 < 4096 的 pass 武装标准 10 分钟退避。图像主导超量与经典 overflow preflight 不受影响。
  • 文档:CONFIGURATION en/zh 各加一段,说明收益门、地板钳制(warn 日志会点名出问题的配置键)、重载语义(文件编辑经 UI/API 保存、reload 端点或重启生效)。

5. 是否有回归问题:逐项审过触碰路径的既有行为,无意回归为零:

  • 经典 overflow preflight 路径不变(!growthArmed ⇒ growthWorthIt=true,字节等价旧行为);
  • zero-progress 退避与 auto-fold backoff arms on pure queue-drops (attempts=[]) as if the summary service had failed #2662 queue-drop 豁免路径未动;
  • nudge 抑制只收窄(地板 ≥ 门槛时释放),不会扩大;
  • wire 形状、tool-call 参数、thinking 家族块未触碰;plugin/proxy 两种模式都推演过;
  • 新增 session-metadata 键纯增量,持久化格式无变化。
  • 有意行为变更(PR 表格已披露 4 行):健康配置下首次折叠可能比目标晚 ≤4K token 触发;sub-benefit pass 武装 10 分钟退避;不可达目标在地板测量后解除 growth fold;新增两条 warn 日志。
  • 审核中发现并已在分支上直接修掉的两处测试缺陷(均仅测试代码,src 零改动):① 原 lane-B e2e 夹具的巨型尾部自己先被折叠(~17242 节省 ≥ 4096),sub-benefit 分支实际从未被执行——按 lane-A 的不可压缩基线模式重建,现在 pre-fix 失败、post-fix 通过,回归覆盖真正钉住该机制;② CI ubuntu-22-3 抓到一个既有 abort-lane 测试钉死了旧触发语义(usage 基线 10K → 预期收益 1808 < 4096,新门下折叠不再开火,断言落空)——usage 提到 20K,取消语义的测试主体保真。

6. 是否破坏缓存:不涉及。前缀缓存、重试、增量游标、去重键均未触碰;唯一状态变化是一个增量 metadata 字段。

7. 是否改了设计方向:不涉及。仍是「目标驱动的 growth fold + 退避」,只是补上可达性与性价比约束;fs-watch 自动重载刻意不做(feature 决策,留给 owner)。

8. 是否引入了新的配置:不涉及。无新 env/config/CLI flag;AUTO_FOLD_MIN_BENEFIT_TOKENS=4096 是内部常量;既有逃生阀保留(compress.externalSummary.autoFoldTargetTokens、compress.nudgeGrowthTokens)。

过程记录

  • 查重:不重复(首条回复已给结论)。
  • 分支全程 stale/冲突(base ce9bc90 → master 先后推进到 774da9b / 896e734 / 269accd,含 Vertex feat(vertex): run Claude Code on Vertex AI through bili #3323 等合并);每一跳都做 union 重排并与 PR 内容做字节级比对。最终 head 33d5a18 位于当前 master 269accd 之上,含全部 4 个 commit(fix + docs + 两个测试修复),PR 内容与本地验证过的树字节一致。
  • 本地验证(在远端精确 head 上执行):typecheck 干净;定向跑 tests/external-summary-autofold.test.ts + tests/preflight-client-abort.test.ts = 20/20 通过;pre-fix teeth 检查确认四个新测试在 src 回退时全部失败(有牙)。
  • 最终 head 33d5a18 的 CI:全量绿——48 个 check-run:32 success + 16 skipped,零失败(ubuntu/windows × 22/24 × 3 shard 全过)。
  • PR 描述已同步更新(更正 lane-B 描述、review follow-up、CI follow-up、最终 head 信息)。

判定

代码质量达标、测试有牙、CI 全绿、与 master 无冲突。可以人工审核合并——按 §7.4,阈值/默认值类变更不属自动合并范围,请自行合并:#3460

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

已并入 #4590。关单收尾;误判请 reopen。

@ranxianglei ranxianglei changed the title fix: stop the auto-fold fold storm on unreachable targets; document config reload semantics [PR] fix: stop the auto-fold fold storm on unreachable targets; document config reload semantics Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant