Repository navigation
[PR] fix: stop the auto-fold fold storm on unreachable targets; document config reload semantics - #3460
[PR] fix: stop the auto-fold fold storm on unreachable targets; document config reload semantics#3460ranxianglei wants to merge 4 commits into
Conversation
(ai: vllm-qwen/qwen3.8-27b) 收到。这是已完成的 PR 工作项(Closes #3444),不重做,进入核对流程:代码审查 + 既有行为回归清点 + CI 状态确认。 查重:不重复。 相关但在途的其他条目机制不同,不构成重复:#2638/#2660(preflight 每轮重新提名结构性死区间——死 span 重提名,非目标不可达)、#3353(提交前拒绝覆盖回退的 fold——另一失败域)、#3442(compress 绑到无 ref 的 一个需要处理的状态:PR 分支基于 |
3dc9af6 to
4892680
Compare
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4093-pr.3460.28c3d0dThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3460.tgz
npm install -g packageInstall instructions are refreshed on each push. |
4892680 to
ec50a32
Compare
(ai: vllm-qwen/qwen3.8-27b) 收到上游冲突转发,已把本 PR 分支 rebase 到最新 master、解决冲突并强推回 PR head。下面是结构化结论(八问)。 1. 是什么问题 2. 什么原因导致的 3. 影响范围与严重程度 4. 怎么修复的 5. 是否有回归问题 6. 是否破坏缓存 7. 是否改了设计方向 8. 是否引入了新的配置 收工判定:冲突已解决,PR 现为可合并状态(mergeable=True),当前仅因全量 CI test 分片仍在运行而显示 blocked。CI 转绿后即可人工 review / 合并。 |
…he under-cost arm Review finding (PR #3460): the original lane-B fixture assumed its ~15K-token final user message was protected live-tail content, but with preserveRecentMessages: 0 nothing guards it — turn 1 actually folds it too (measured: 2 range(s), ~17242 tokens saved vs 4096 cost bar), so the sub-benefit backoff at server.ts never armed and the lane passed even with the fix reverted. It pinned the legacy zero-progress path, not the new one. Rebuilt the shape on the lane-A pattern: ~15K-token BIG_SYSTEM (unfolderable) + two small pairs (~2K foldable). Turn 1 then fires on a large expected benefit yet saves ~1K real tokens (< 4096) — arming the sub-benefit backoff; turn 2 adds one fresh ~8K-token pair that pre-fix code re-folds every request (the reported storm shape) while the fixed build skips it during cooldown. Verified both timelines: 11/11 post-fix; tests 6-9 all fail pre-fix.
c215af5 to
4b20600
Compare
…floor + benefit gate (#3444) A configured autoFoldTargetTokens below the session's irreducible baseline (system prompt + tool schemas + live tail) could never be reached: the trigger re-fired on every request and burned a full external-summary round trip per request for ever-smaller slivers (reported saves down to ~685 tokens, each costing up to ~60s of chain latency). Two complementary facets, same mechanism family (growth-fold economics): 1. Measured floor: an exhausted honest growth pass (folded everything foldable, no chain failure) whose rebuilt payload still sits at/above the target records the post-fold size as the session's irreducible floor (session metadata, latest measurement wins). The effective target is raised to max(configured, floor, MIN) capped at the window; a floor at/above the overflow bar disarms growth folding entirely and releases the nudge suppression (growthFoldingArmed gains a measuredFloor argument, threaded through all four prepare-* lanes). 2. Benefit gate: a growth fold only fires when the text channel exceeds the budget by at least AUTO_FOLD_MIN_BENEFIT_TOKENS (4096) of expected saving, and a pass that actually saves less than that arms the standard 10-minute backoff (#2662 queue-drop congestion exemption mirrored). Image-dominant overage keeps its original unconditional arm (images are not foldable). Classic overflow preflight (bar = window) is untouched. Regression tests: two e2e lanes (sub-floor system-prompt baseline stops re-folding marginal growth; giant-live-tail sub-benefit pass arms the backoff) plus truth-table rows for the floor argument and an autoFoldFloor invalid-value unit test.
…oad semantics (#3444) CONFIGURATION.md en+zh: the ~4096-token benefit gate and the measured-floor clamp for unreachable targets (with the actionable warning naming compress.externalSummary.autoFoldTargetTokens), and the reload asymmetry behind the second observation — file edits take effect only via Web UI/API save, POST /__bili/config/reload, or restart (every section, providers/routes included), while GET /__bili/config reflects the file immediately.
…he under-cost arm Review finding (PR #3460): the original lane-B fixture assumed its ~15K-token final user message was protected live-tail content, but with preserveRecentMessages: 0 nothing guards it — turn 1 actually folds it too (measured: 2 range(s), ~17242 tokens saved vs 4096 cost bar), so the sub-benefit backoff at server.ts never armed and the lane passed even with the fix reverted. It pinned the legacy zero-progress path, not the new one. Rebuilt the shape on the lane-A pattern: ~15K-token BIG_SYSTEM (unfolderable) + two small pairs (~2K foldable). Turn 1 then fires on a large expected benefit yet saves ~1K real tokens (< 4096) — arming the sub-benefit backoff; turn 2 adds one fresh ~8K-token pair that pre-fix code re-folds every request (the reported storm shape) while the fixed build skips it during cooldown. Verified both timelines: 11/11 post-fix; tests 6-9 all fail pre-fix.
…benefit gate CI (ubuntu-latest/22 shard 3 on c271011): 'cancelled growth auto-fold stops before fail-open forwarding and backoff' failed at its trigger-log assertion. The lane predates this PR and pinned the OLD trigger semantics: any payload >= target fired the external chain. Its fixture (usage baseline 10K vs 8192 = AUTO_FOLD_TARGET_MIN) carries only ~1.8K expected saving, so the new benefit gate (>= 4096, behavior-change row 1 of this PR — intended) no longer arms the fold, and the lane lost its subject. Raise the usage baseline to 20K (target cannot go lower — 8192 is the documented minimum): expected saving 20_000 - 8192 = 11_808 >= 4_096, the fold arms, gets cancelled mid-pass as before, and every original assertion (fail-fast override, no main forward, no backoff on cancellation, floor trigger log) is unchanged. Test-only; no src change.
4b20600 to
33d5a18
Compare
(ai: vllm-qwen/qwen3.8-27b) PR #3460 审核完成。结论先行:修复逻辑成立,全量 CI 绿,可提交人工审核合并。审核过程中我在 PR 分支上直接修了两个测试缺陷(一个本地发现、一个被 CI 抓住),详见第 5 问。 结构化八问1. 是什么问题:auto-fold 目标值( 2. 什么原因导致的:同一机制族(growth-fold 经济性)的两个面:① 触发条件只看「payload > 目标」不看目标是否可达——基线高于目标时折叠毫无收益仍会开火;② 开火没有收益下限——省 685 token 也算进度,既有 no-progress 保护接不住。配置重载是快照模型(引擎读启动快照 + PUT/reload 注入的内存态),文件监听从未实现,属设计现状而非缺陷。 3. 影响范围与严重程度:仅影响开启 external-summary auto-fold 且配置目标低于实际不可压缩基线的会话(报告者场景);其余用户行为不变(见回归清单)。严重度为性能/成本退化(每轮风暴多几十秒延迟 + 外部调用费用),无数据丢失、无功能不可用。 4. 怎么修复的:两个机制 + 文档,一个闭环:
5. 是否有回归问题:逐项审过触碰路径的既有行为,无意回归为零:
6. 是否破坏缓存:不涉及。前缀缓存、重试、增量游标、去重键均未触碰;唯一状态变化是一个增量 metadata 字段。 7. 是否改了设计方向:不涉及。仍是「目标驱动的 growth fold + 退避」,只是补上可达性与性价比约束;fs-watch 自动重载刻意不做(feature 决策,留给 owner)。 8. 是否引入了新的配置:不涉及。无新 env/config/CLI flag; 过程记录
判定代码质量达标、测试有牙、CI 全绿、与 master 无冲突。可以人工审核合并——按 §7.4,阈值/默认值类变更不属自动合并范围,请自行合并:#3460 |
|
(ai: zhipuai-lb/glm-5.3) 已并入 #4590。关单收尾;误判请 reopen。 |
Closes #3444
What this fixes
Observation 1 (bug) — sub-floor
autoFoldTargetTokenscauses a fold storm. When the configured growth target sits below the session's irreducible baseline (system prompt + tool schemas + live tail), no fold can ever reach it: the trigger re-fired on every request and burned a full external-summary round trip per request for ever-smaller slivers (the report shows saves down to ~685 tokens, each costing up to ~60s of chain latency). Two complementary facets, one mechanism family (growth-fold economics):session.metadata.autoFoldFloorTokens, latest measurement wins, tracks baseline drift both ways). The effective target becomesmin(max(configured-clamped, floor), window); a floor at/above the overflow bar disarms growth folding entirely and releases the nudge suppression (growthFoldingArmedgains an optionalmeasuredFloorargument, threaded through all four prepare-* lanes so suppression and arming stay consistent).AUTO_FOLD_MIN_BENEFIT_TOKENS(new internal constant, 4096) of expected saving, and a pass that actually saves less than that arms the standard 10-minute backoff (mirroring the zero-progress path, with auto-fold backoff arms on pure queue-drops (attempts=[]) as if the summary service had failed #2662's queue-drop congestion exemption unchanged). Image-dominant overage keeps its original unconditional arm (images are not foldable); classic overflow preflight (bar = window) is untouched.The trigger-side gate stops boundary-crossing churn before it costs a round trip; the post-hoc backoff catches estimation error where the pass runs anyway. Both are needed: floor-only leaves the trigger re-firing at payload≈floor every request (equality fires); gate-only still pays one wasteful pass per real boundary crossing.
Observation 2 (gap) — file edits do not take effect until
PUT /__bili/config. Verified mechanism: the engine reads the startup snapshot plus whatPUT /__bili/config/POST /__bili/config/reloadinject into memory (opts.compress);GET /__bili/configreads the file fresh, hence the asymmetry that trapped the reporter. Nothing watches the raw file — no field does, providers/routes included. This PR ships the docs half (CONFIGURATION.md en+zh now state the reload semantics explicitly). The optional fs-watch auto-reload is a feature decision requested in the issue thread — not built here.Observation 3 — acknowledged, no action: the 1M window resolution is correct and the per-provider warning is cosmetic by design.
Behavior changes (old → new)
compress.externalSummary.autoFoldTargetTokens; floor ≥ window disarms growth folding and releases nudge suppressionNo change to wire shape, tool calls, message format, persistence format (additive session-metadata key, same pattern as
autoFoldBackoffUntil), or classic overflow preflight. Healthy configs (target reachable with headroom) behave exactly as before — the pre-existing e2e lanes (fold fires / off-control / fail-open / backoff-on-failure / queue-drop / degenerate-window) all pass unchanged.Config surface
No new config fields, env vars, or CLI flags.
AUTO_FOLD_MIN_BENEFIT_TOKENSis an internal behavior constant (same class asAUTO_FOLD_BACKOFF_MS), deliberately not owner-tunable — the escape hatch for big-window users wanting a lazier cadence remainscompress.nudgeGrowthTokens/raising the target itself.Evidence
autoFoldTargetTokenscauses a fold storm, andexternalSummary.*file edits do not take effect until aPUT /__bili/config#3444 tests fail with src reverted to origin/master.measuredFloorargument +autoFoldFloorinvalid-value unit test.tests/external-summary-autofold.test.ts11/11 green;npm run typecheckclean. Rebased onto current master (was stale/conflicted vs the Vertex merge feat(vertex): run Claude Code on Vertex AI through bili #3323); full-suite verdict = CI on this head.