Skip to content

[PR] fix: bound .corrupt-* quarantine files and stop OOM-driven state loss (#5779) - #5781

Merged
ranxianglei merged 3 commits into
masterfrom
2026-11-06_corrupt-quarantine-cap
Nov 7, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-11-06_corrupt-quarantine-cap

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

What

#5779 report: the sessions tree accumulates .corrupt-<timestamp> copies of session state files (~0.61 GiB each, hundreds per tree — 273 copies / ~157 GiB on one user's disk), generated at up to ~36/hour.

Two defects, both fixed here:

  1. Unbounded forensic retention. When a session envelope fails decode/validation it is renamed to <file>.corrupt-<ts> ([bug] 会话状态写入缺 fsync:内核崩溃后留下 size 正常但 VDL=0 的全零状态文件,loadSync 永久抛错 → 502 acp-proxy failure 且不自愈(含 7 处 tmp+rename 写路径盘点) #3157 self-heal) and nothing ever deletes those copies — directory walkers only collect *.json / .tmp-*. Every repeated failure of a hot session added another full-size copy forever.
  2. Resource exhaustion misclassified as corruption. Decoding a very large envelope under memory pressure throws a catchable V8 RangeError ("Array buffer allocation failed" / "JavaScript heap out of memory"). readEnvelopeFile treated that as corruption: it quarantined a healthy file, destroying its fold state; the resident-pool eviction/cold-load cycle then rebuilt the session from scratch and re-quarantined it every round — the loop that produced ~0.6 GiB of forensic copies per failure.

Plus one owner-directed default change found while analyzing the loop (in-thread decision on #5779):

  1. The resident pool budget default was smaller than a typical single session. The ~0.61 GiB envelope exceeded the whole 512 MiB pool budget, so eviction/cold-load ran every request gap forever (same shape as the [dsh] A healthy 93%-compressed image session becomes permanently uncompressed by ordinary usage: quota → model switch → attribution loss → 413 → rewind → 3,455 dead refs → compress REFUSED forever (0.1.202) | 一个健康压缩率 93% 的图片会话被普通操作永久废掉:额度→换模型→丢归因→413→撤回→… #3918 incident). Default raised to 4 GiB so a single busy session can actually stay resident.

Fix

  • pruneCorruptSiblings() (src/persist.ts): after each quarantine rename, keep at most CORRUPT_SIBLING_CAP = 1 copy per base file (newest survives; the just-created copy always survives even under clock skew). Applied to both the session store (quarantineCorruptState) and plugin-conversations.json (src/plugin.ts, same mechanism). Best-effort: locked files are skipped and retried on the next quarantine; never throws.
  • isAllocationFailureError(): allocation-failure RangeErrors during decode now leave the file in place for the next load (error log explains the verdict); genuine decode failures still quarantine exactly as before. Both errors are classified — an OOM in the plaintext attempt surfaces only as the codec's format error in the inner catch.
  • sessionResidentMaxBytes() default (src/knobs.ts): 512 * 1024 * 1024 → 4 * 1024 * 1024 * 1024. The per-session hard cap (sessions.hardMaxBytes) defaults to the pool soft cap, so it follows automatically. Docs kept in sync through the yaml seed + gen-config-docs generator (CONFIGURATION.md en/zh, LIFECYCLE.md, website/config-reference/process.yaml); test asserts updated.

Behavior changes (old → new)

Behavior Old New Why
.corrupt-* retention per base file unbounded (every failure adds a copy, kept forever) at most 1 (the newest) unbounded retention filled user disks (#5779); forensic value lives in the latest failure
Decode fails with an OOM RangeError file quarantined → healthy fold state destroyed, fresh start file left in place, error logged, retry next load OOM is resource exhaustion, not corruption; quarantining turned a transient condition into permanent data loss + a rebuild/re-quarantine loop
Quarantine warn log no sweep info appends removed N older forensic copies (cap 1) when the sweep runs observability for the new behavior
Resident pool soft-cap default (sessions.residentMaxBytes) 512 MiB 4 GiB single sessions routinely exceed half a GiB and ping-ponged against the smaller budget forever (#5779 analysis, #3918 precedent); owner-directed in-thread. Memory cost: the proxy may now hold up to ~4 GiB of resident session views before evicting idle ones — small-RAM machines should lower it back via sessions.residentMaxBytes / BILI_SESSION_RESIDENT_MAX_BYTES

No NEW config surface added: the quota constants are hardcoded by design (forensics need at most the latest copy), and item 3 changes the DEFAULT of an existing knob only — env/file overrides are unchanged.

Tests

New tests/issue5779-corrupt-quarantine-cap.test.ts (pins the legacy-file tier like the CI non-storage baseline): pre-seeded old siblings pruned to cap, repeated quarantine rounds never accumulate, OOM decode leaves the file intact with zero siblings (gate positive), genuine SyntaxError still quarantines (gate negative), classifier unit table, plugin-conversations cap. tests/knobs-config-precedence.test.ts asserts the new 4 GiB default (both precedence tiers). Pre-existing issue3157-persist-durability, instance-governance, issue3918-resident-eviction-refusal, bounded-storage, issue3399-cross-route-load suites green alongside under the CI preload chain.

Closes #5779

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-06_corrupt-quarantine-cap (27c3256)

Latest build — install this exact version

npm install -g billion-context@0.0.8390-pr.5781.5db9d55

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
27c3256 billion-context@0.0.8390-pr.5781.5db9d55 build
062e1dd billion-context@0.0.8383-pr.5781.0589e0c build
ec60e23 billion-context@0.0.8323-pr.5781.e3c12a6 build
8389ea0 billion-context@0.0.8322-pr.5781.7a58051 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5781.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: bound .corrupt-* quarantine files and stop OOM-driven state loss (#5779) [PR] fix: bound .corrupt-* quarantine files and stop OOM-driven state loss (#5779) Nov 6, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。同族仅 #3157/#3163(.corrupt-* 隔离机制的来源,已合入,本 PR 正是在其之上修保留策略);index.db 隔离族(#5108/#5123/#3965)是不同文件不同机制;无其他在途 PR 并行修本线程。

标签:pr bug p1 scope/storage(会话状态文件被误隔离 → fold 状态丢失 + 磁盘撑爆,数据丢失类 p1;核心存储跨所有客户端车道,不打客户端标签)。

代码核对完成(GitHub PR #5781,单提交 @8389ea0,基于 master@0aace2f20),中期结论:

  1. 方向关通过:Corrupt file .corrupt – timestamp blows up the disk | 坏文件.corrupt﹣时间戳撑爆磁盘 #5779 两个缺陷(取证副本无界保留、OOM 误判为损坏)都真实存在且命中根因——前者在两个 rename 产生点加保留上限,后者在判定点对「明文解析错误 + codec 解码错误」双错误分类。不是治标。
  2. 回归清点通过:readEnvelopeFile 的全部调用方对 null 一律按 miss 落下一探针,OOM 路径返回形状与隔离路径完全一致,调用方行为零变化;全仓 .corrupt-* 产生点只有两处(persist.ts 会话存储 / plugin.ts 路由表),都已带上限,无其他消费者依赖无界保留。
  3. 测试有牙:gate 双向断言(OOM 留盘 vs 真 SyntaxError 仍隔离)、重复轮次测试钉死 Corrupt file .corrupt – timestamp blows up the disk | 坏文件.corrupt﹣时间戳撑爆磁盘 #5779 的累积循环本身、分类器单测表覆盖 6 种错误形态。
  4. 记录的边界(非缺陷):plugin-conversations.json 只加上限、未加 OOM 分类——该文件是路由表(每条目几百字节)且仅在启动时加载一次,不在热循环里,parse OOM 现实不可达;事故证据全部来自 0.6 GiB 级的会话信封。合理取舍。
  5. PR base 之后 master 前进了 56 个提交,零个触碰 persist.ts/plugin.ts,无冲突风险;本地验证过我会 rebase 到当前 master(内容零改动)再跑 CI。

CI 现状:static-gates / docs-sync / test-import-graph / lock-sync 已绿,11 个 test 分片运行中。本地定向验证(新套件 + issue3157 套件)正在跑,全部完成后补八问结论。

@ranxianglei ranxianglei added bug Something isn't working p1 scope/storage labels Nov 6, 2026
…decode failure (#5779)

Two defects behind user disks filling with ~0.6 GiB .corrupt-<ts> session
state copies (273 copies / ~157 GiB reported in #5779):

1. Unbounded forensic retention: quarantineCorruptState renamed a failed
   envelope to <file>.corrupt-<ts> and nothing ever deleted those copies —
   every repeated failure of a hot session added another full-size copy.
   pruneCorruptSiblings now keeps at most CORRUPT_SIBLING_CAP (1) copy per
   base file; applied to both the session store and plugin-conversations.json.

2. Resource exhaustion misclassified as corruption: when decoding a large
   envelope ran out of memory (V8 RangeError), readEnvelopeFile quarantined
   the healthy file and destroyed its fold state; the eviction/cold-load loop
   then rebuilt and re-quarantined it every cycle. Allocation failures now
   leave the file in place for retry (error log, no quarantine). Genuine
   decode failures still quarantine exactly as before.
@ranxianglei
ranxianglei force-pushed the 2026-11-06_corrupt-quarantine-cap branch from 8389ea0 to ec60e23 Compare November 6, 2026 05:29
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 自动修复车道(attempt 1/3)收工:零代码改动,重跑两个红分片后 run 全绿。

七问:

  1. 是什么问题:run 42168716484(head ec60e23)27 个 job 中 2 个 test 分片红——ubuntu-24-s1 死于 tests/msgid-lane-flow.test.ts:139(edit-resync 后期望 4 行实得 3 行);windows-22-s2 在 runner 内自动重试后仍死于同文件 :276(clean delta 未钉住 u4)。全部静态门禁(kernel-guard/version-guard/config-surface-guard/lock-sync/static-gates 等)均绿。
  2. 什么原因:不是本 PR 回归,也不是继承的 master 红——是在档 settle race flake 家族复发([PR] fix: surface failed watcher registration cause/state and report heartbeat recovery #5644 win22-s2 / CI: windows-latest node-22 shard 2 intermittently red on timing-sensitive e2e tests under load (rotating flakes: issue760, #4079, storage-full-flip #3818) #4161 时序窗口,CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 楼 1035 已记录过同一文件同一行 :139)。机制:settle() 等的是 debounced save 的落盘裁决,高负载 runner 上新开的只读连接读行时可能仍看到 turn 前快照。本 PR diff 只动 persist.ts/plugin.ts 的损坏/OOM 隔离失败路径,与 msgid 车道和该测试文件零交集。
  3. 证据链:① 同期 master be611a8 分片全绿(哨兵盘点 02:15–02:24Z,CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 楼 1039);② 本地同一树(node 22)msgid-lane-flow 6/6 绿 ×3 连跑 + 本 PR 自带的 issue5779-corrupt-quarantine-cap 6/6 绿;③ 两平台红在不同子测试,符合负载时序签名而非确定性回归。
  4. 怎么处置的:对 run 42168716484 触发 rerun-failed-jobs(HTTP 201),分支未动、零推送。没有顺手改 settle():把 settle() 收紧为「等特定 row/seq」涉及断言语义取舍,属设计向修复,且会把无关测试基建变更塞进本存储隔离 PR 的 diff(违反 diff 整洁);在档口径(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 楼 1035)即「重跑即绿则零代码改动,仅当复发到影响合并才单开处理 settle()」。
  5. 回归评估:零代码改动 → 无新增回归面。原 head 上其余 25 个 job(含全部门禁与其余 9 个 test 分片)本就全绿;本 PR 已披露的行为变更(隔离副本上限 1、OOM 解码不再隔离)不涉及该套件。
  6. 当前 CI 状态:重跑后 run 42168716484 completed/success,27/27 全绿(06:11Z 确认)。
  7. 判定:本 PR CI 清障完毕,可进入人审合并。若同签名再次于干净 runner 复发,按在档口径升档(settle() 收紧作为独立小修另行处理,不并入本 PR)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-8.3)

审查完成:全绿,建议合并

rebase + CI:PR base 之后 master 前进了 56 个提交(零个触碰 persist.ts/plugin.ts),我已把分支 rebase 到当前 master(be611a8)——单提交 8389ea0 → ec60e23,diff 逐字节相同(3 文件 +272/-14),push 前用 force-with-lease 校验过远端未动。rebase 后本地 typecheck + build 干净。新 head 上 12 条 workflow 全绿:主 CI(含 11 个 test 分片)+ 7 条 e2e 车道 + codex overlay + build artifact,0 失败。

本地定向验证(改动域收窄跑,非全量):新套件 + issue3157-persist-durability + instance-governance = 26/26 通过。透明说明:我第一次直跑时 issue3157 套件有 2 例假失败——原因是本地直跑没带 CI 的存储基线 preload(tests/preload-storage-baseline.mjs 把非存储语料钉在 legacy 档;不钉的话该套件期望的「坏 JSON 被探针读到」前提不成立)。按 CI 同款基线重跑全绿,与代码无关。


结构化八问

  1. 是什么问题:会话状态文件损坏/解码失败会被复制成 .corrupt-<ts> 取证副本且永不清理(单用户 273 份 × ~0.61 GiB ≈ 157 GiB、每小时 ~36 份);同时解码期 OOM(V8 RangeError: Array buffer allocation failed)被误判为损坏——健康文件被隔离、fold 状态销毁,下轮从零重建再隔离,循环放大取证副本。
  2. 什么原因导致的:两个独立机制——① [bug] 会话状态写入缺 fsync:内核崩溃后留下 size 正常但 VDL=0 的全零状态文件,loadSync 永久抛错 → 502 acp-proxy failure 且不自愈(含 7 处 tmp+rename 写路径盘点) #3157 自愈只做 rename,目录 walker 只收 *.json/.tmp-*,没人删副本(无界保留);② readEnvelopeFile 把一切解码异常都归类为「损坏」,但资源耗尽不是损坏(分类错误)。
  3. 影响范围与严重程度:p1 数据丢失类,所有用户。热会话反复故障的用户磁盘持续被撑爆;触发 OOM 的会话丢 fold 状态从头再来(压缩成果作废、成本重付)。修复前是持续性慢失血。
  4. 怎么修复的:① pruneCorruptSiblings()——每次隔离 rename 后立即清扫同 base 旧兄弟,至多留 CORRUPT_SIBLING_CAP = 1(最新存活;刚创建的副本即使时钟回拨也必存),两个产生点(会话存储 + plugin-conversations.json)都接上;best-effort,锁住的文件跳过留待下次,绝不抛错。② readEnvelopeFile 双错误分类:明文解析错误与 codec 解码错误各自捕获,任一命中分配失败特征(allocation failed|heap out of memory 正则;stack-overflow 明确不算)→ 文件原地保留、日志写明判决、下轮重试;真解码失败照旧隔离,行为不变。
  5. 是否有回归问题:无。既有行为逐项清点:readEnvelopeFile 全部调用方对 null 一律按 miss 落下一探针/新起点——OOM 路径返回形状与隔离路径完全一致,调用方行为零变化;isValidRecord 校验失败隔离路径不变;readFileSync 的 ENOENT/其他错误处理不变;另一处解码点 readContentStoreRels 自带 fail-safe、未触碰。行为差异仅 PR 正文披露的三项(保留上限 / OOM 留盘 / warn 日志追加清扫计数——加性观测,非语义变化)。无阈值/时序/输出格式漂移。
  6. 是否破坏缓存:不涉及。不触碰前缀缓存、重试策略、增量游标、去重键;OOM 留盘文件由下轮加载自然重试,无陈旧状态残留。
  7. 是否改了设计方向:没有。机制仍在 [bug] 会话状态写入缺 fsync:内核崩溃后留下 size 正常但 VDL=0 的全零状态文件,loadSync 永久抛错 → 502 acp-proxy failure 且不自愈(含 7 处 tmp+rename 写路径盘点) #3157 自愈框架内,只补了保留策略并修正一处误分类,架构/边界/数据流均未转向。
  8. 是否引入了新的配置:没有。保留上限是硬编码常量(取证最多需要最新一份,按设计不可调);config-surface-guard CI 确认配置面零变化。

auto-merge 门评估(§7.4):回归测试复现原 bug ✓ / rebased head 全绿 ✓ / 无配置·持久化格式·wire·kernel 变更 ✓ / 纯 fix ✓ / diff 干净 ✓ / 引用 #5779 ✓ / 不触 load-bearing infra(update.ts、ref-id、安全面)✓ —— 形式上满足自动合并条件;合并动作归你(agent 不合并 PR)。

记录的边界(非缺陷):plugin-conversations.json 只加了保留上限、未加 OOM 分类——它是路由表(每条目几百字节)、仅启动时加载一次、不在热循环里,parse OOM 现实不可达;事故证据全部来自 0.6 GiB 级会话信封。合理取舍,已在中期回复说明。

判定:无拦截项,可合并 — #5781

@ranxianglei

Copy link
Copy Markdown
Owner Author

🔶 需要决策
(ai: vllm-qwen/qwen3.8-27b)

PR #5781 (head ec60e23) review surfaced one residual touching save-path arbitration semantics — not fixable in-place under review-lane rules (cross-file behavior), routed here for owner call. No code changed by this review.

Residual: first save can clobber a present-but-unreadable envelope (sustained OOM only).

Mechanism (legacy storage tier — the tier the new tests pin, and the reality of default plain-JSON envelopes per #1080):

  1. Cold load: readEnvelopeFile OOM-misses → session starts fresh, healthy file stays on disk (this PR's intended behavior).
  2. Turn runs → debounced save → guardedBuild → staleDiskPayload (src/persist.ts:2723) full re-read → if this second decode also OOMs, loadEnvelope returns null — indistinguishable from "no file on disk" → guard passes → the small fresh envelope is written over the healthy 0.6 GiB file. Bytes destroyed.
  • Transient OOM (load fails, save-time re-read succeeds): no loss — counters prove disk strictly newer → convergeToDisk restores full fold state into memory (self-heal).
  • Sustained OOM (both fail — the incident regime, ~36 failed loads/hour): one-time loss of fold state on the first post-miss save. Client history is intact, conversation continues, cost spikes on rebuild. Pre-PR the same scenario preserved the bytes as .corrupt-<ts> copies — which is why the user accumulated 273 of them. So in this narrow regime the PR trades "bytes preserved as forensic copies" for "bytes destroyed", while fixing everything else.
  • Pre-existing in spirit: the identical clobber was reachable before this PR via the rename-failure case (Windows AV/indexer lock → quarantine rename fails → file stays undecodable → next save overwrites). This PR adds a second entry point into the same gap; it does not create the gap.

Sub-point (couples with the above): the classifier misses a third resource-limit family. Empirical (Node v22.23.2): producing >536,870,880 decoded chars throws plain Error: Cannot create a string longer than 0x1fffffe8 characters — NOT a RangeError, so such plaintext envelopes (default codec; zstd off per #1080) still get quarantined at load. That fall-through is arguably correct for that shape (permanent hard-cap failure → preserve once + fresh start), but it should be deliberate rather than accidental. And widening isAllocationFailureError to include this family WITHOUT the guard below would convert preserved-bytes into destroyed-bytes — hence one decision, two facets.

Options:

Cite: PR #5781 head ec60e23; gated area = src/persist.ts staleDiskPayload/guardedBuild (+ optional isAllocationFailureError family extension).

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

APPROVE. Both halves of #5779 fixed at the root; the p1 disk explosion is definitively closed. One non-blocking residual (write-path clobber under sustained OOM) is routed through a separate 🔶 需要决策 comment — pre-existing in spirit, net improvement either way.

Direction (issue audit). Repro facts hold against code: nothing ever pruned <base>.corrupt-* copies (dir walkers collect only *.json/.tmp-*, src/persist.ts:4102-4115), and readEnvelopeFile's inner catch (src/persist.ts:1213) quarantined every decode failure, allocation exhaustion included — destroying a healthy envelope's fold state on each eviction/cold-load round. The fix targets both mechanisms, and I verified only two .corrupt- producers exist (src/persist.ts:1256, src/plugin.ts:385) — both covered. Base current (parent == master tip); CI 30/30 green via mirror cache.

Regression audit — existing behaviors on touched paths:

behavior verdict
genuine decode failure → quarantine + fresh start preserved — negative-direction test pins it (SyntaxError still quarantines, exactly one copy kept)
.corrupt-* retention unbounded intended change, disclosed: cap 1 per base file; newest kept; just-created copy always survives clock skew; locked files skipped + retried next round. No other consumer of these files exists (forensics-only by design)
OOM decode → quarantine intended change, disclosed: file left in place, retried next load. Bonus self-heal verified in code: when the save-path re-read succeeds after a load-time miss, staleDiskPayload counter arbitration converges in-memory back onto the disk record (src/persist.ts:2892) — fold state recovers transparently
warn-log format appends removed N older forensic copies (cap 1) — disclosed; no machine consumers of this line
plugin-conversations.json corrupt handling same quarantine + prune added; routing-table semantics unchanged
readEnvelopeFile null contract unchanged across all four caller probe sites (loadEnvelope ×3 + namespaced pool)
compression modes / wire / kernel / config surface persistence layer only — mode-independent; no kernel/ diff (kernel-guard green); no config/env additions (CORRUPT_SIBLING_CAP hardcoded, deliberate; config-surface-guard green)

Test validity. Real teeth: pre-seeded historical sweep keeps exactly the just-created copy (size-checked, not a stale twin); 3-round accumulation pin (the incident itself); bidirectional gate per #3617 discipline (OOM positive: file intact + zero siblings + explanatory error log; SyntaxError negative: quarantined + exactly one copy); classifier truth table incl. stack-overflow exclusion; plugin-lane cap. Deterministic (no ports, tmpdir + cleanup, env restored in finally). One coverage note, non-blocking: the OR branch where the plaintext attempt is the OOM side isn't directly exercised — forcing a real OOM inside JSON.parse(bytes.toString("utf8")) hermetically would require pulling that parse behind an injectable seam; the classifier table plus the codec-side wiring cover the rest.

Local empirical probe (Node v22.23.2): Buffer.alloc(540MiB).fill('a').toString('utf8') throws plain Error: Cannot create a string longer than 0x1fffffe8 characters — NOT a RangeError. Consequence for the classifier family analyzed in the 🔶 需要决策 comment (sub-point).

中文八问

  1. 是什么问题:热会话每次解码失败都新增一份全尺寸 .corrupt-* 取证副本(单用户 273 份 / ~157 GiB),且内存不足导致的解码失败被误判为损坏——健康信封被隔离、折叠状态被毁、驱逐/冷加载循环每轮重建再隔离。
  2. 什么原因导致的:① #3157 隔离机制只有"改名保留"没有保留上限,目录扫描只收 *.json/.tmp-*,取证副本永不清理;② 解码失败只有"成功/损坏"两态,V8 资源耗尽(RangeError)与真实损坏混为一谈。
  3. 影响范围与严重程度:p1,所有长期大会话用户。磁盘可被取证副本撑爆(不可用级);OOM 误隔离丢折叠状态、压缩从零重启、成本回涨。触发条件:大会话状态文件 + 反复加载失败。
  4. 怎么修复的:双管——每次隔离改名后按基文件剪枝到 1 份(新建副本恒留、时钟偏移安全、锁文件跳过下轮重试、永不抛错);解码错误增加"资源耗尽"分类,此类文件原地保留、下轮重试。瞬态 OOM 另有自愈:保存路径重读成功时计数器仲裁把内存态收敛回磁盘态,折叠状态透明恢复。
  5. 是否有回归问题:上表既有行为逐项审过——真实损坏仍隔离(测试钉住)、null 契约不变、日志仅追加、wire/kernel/配置面零触碰。残留一处非阻塞:sustained OOM 下首轮保存会覆盖盘上未读信封(staleDiskPayload 把"存在但不可读"与"不存在"混同)——该缝隙 PR 之前即存在于 rename 失败场景(Windows AV 锁),本 PR 只是多一个入口;已走 🔶 需要决策,不拦本 PR。
  6. 是否破坏缓存:不涉及——prefix cache/重试/去重键零触碰;瞬态 OOM 场景折叠状态经收敛路径完整恢复。
  7. 是否改了设计方向:不改——在 #3157 自愈架构内加保留边界与错误分类,无新数据流。
  8. 是否引入新配置:无——CORRUPT_SIBLING_CAP = 1 硬编码常量,PR 已声明刻意不加配置面。

判定:可合。 残留项与处置选项见 🔶 需要决策 评论(推荐方向 A)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Review lane done: PR #5781 @ ec60e23 reviewed end-to-end — verdict APPROVE (posted as a comment-state review; GitHub blocks approving one's own PR), CI 30/30 green via mirror cache, regression-audited, tests have teeth. One non-blocking residual (sustained-OOM first save can clobber a present-but-unreadable envelope — pre-existing in spirit via the rename-failure path) routed through a 🔶 需要决策 comment + needs-decision label with options A/B/C (A recommended). Awaiting owner merge.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

此条为 review lane 自己在 PR #5781 上发的 🔶 决策评论经镜像同步回流(带 upstream-sync 标记),不是新输入,无新增信息。决策项仍待 owner 拍板(A/B/C,推荐 A);在此之前不动代码、不开 issue。

ci-check added 2 commits November 6, 2026 22:52
…5779)

062e1dd raised the sessions.residentMaxBytes default from 512 MiB to 4 GiB and updated tests/knobs-config-precedence.test.ts, but missed tests/issue4788-resident-minimal.test.ts:210 which pins the concrete default hard cap (= pool soft cap fallback): 536_870_912 -> 4_294_967_296. The invariant assertion (hard == soft) directly above is unchanged.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

本轮处理(head ec60e23 → 062e1dd → 27c3256):作者推了新提交(把常驻池默认预算从 512 MiB 提到 4 GiB),随后 3 个测试分片全红。两个独立原因:一个是真实疏漏(已修并推送),一个是 Windows runner 的 Python 子进程超时(重跑即绿)。当前全套绿。

1. 是什么问题:新提交后 3 个测试分片同时变红(windows-22-s5 / ubuntu-22-s2 / ubuntu-24-s2);修完后又有 1 个 Windows 分片出现 Python 子进程超时。

2. 什么原因导致的:

  • (a) 确定性疏漏:改 sessions.residentMaxBytes(Bound startup session loading and preserve historical recovery #2803)默认值 512MiB→4GiB 时,tests/knobs-config-precedence.test.ts 的断言同步了,但漏掉 tests/issue4788-resident-minimal.test.ts:210 的具体默认值钉(assert.equal(sessionResidentHardMaxBytes(), 536_870_912),实测 4294967296 = 恰好 4 GiB),3 个分片死在同一条断言上。
  • (b) 环境 flake:某 Windows runner 上 hermes-native 测试的 spawnSync python3 driver 挂满 60s 超时(ETIMEDOUT、stdout/stderr 均空),连发 3 次。本 PR diff 不碰任何 hermes/python 路径,且同一测试在之前两轮运行中同分片均通过 → 判定 runner 环境问题,非代码回归。

3. 影响范围与严重程度:(a) 仅影响本 PR 的 CI 门禁,零运行时行为影响(该钉只是测试对默认常量的断言);(b) 瞬态,重跑自愈。两者都不影响用户。

4. 怎么修复的:

  • (a) 一行修复:tests/issue4788-resident-minimal.test.ts:210 的字面量对齐为 4_294_967_296(commit 27c32560c,已推送)。本地验证 issue4788-resident-minimal 4/4 通过 + knobs-config-precedence 全过;第 209 行的不变量断言(hard cap == soft cap)未动。全仓 grep 残留 512 MiB 引用:其余要么是各测试有意的独立预算(bounded-storage / issue3399 / issue3918 各自显式设值),要么是关于旧默认的历史事故注释(src/session.ts、src/stream.ts,事实准确),均无需改。
  • (b) 触发 rerun-failed-jobs(HTTP 201),干净 runner 上通过。

5. 是否有回归问题:无。我的改动是单测试文件一行字面量更新,零运行时代码变更。默认值 512MiB→4GiB 本身是作者的有意变更(commit message 与 CONFIGURATION.md / LIFECYCLE.md 文档均已披露),非我引入;我只对齐了该变更漏掉的测试。

6. 是否破坏缓存:不涉及。

7. 是否改了设计方向:无。

8. 是否引入了新的配置:我未引入。(作者把既有旋钮 sessions.residentMaxBytes / env BILI_SESSION_RESIDENT_MAX_BYTES 的默认值从 512MiB 改为 4GiB——这是其已披露的行为变更,文档已同步。)

现状与判定:run 42244117307(head 27c3256)26 个 job 全绿(red-signal 按设计跳过)。PR 可进入人工 review/合并。兜底:若 hermes python3 超时在其他干净 runner 上复现,按 #4599 先例升级(候选后续项:findPython() 探测加超时/无响应即 skip,或提高 driver 时间预算——另立工作项,不混入本 PR)。

@ranxianglei
ranxianglei merged commit f66051d into master Nov 7, 2026
77 of 78 checks passed
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1057-master.f66051d (rolling master tag).

npm install -g billion-context@0.0.1057-master.f66051d  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Corrupt file .corrupt – timestamp blows up the disk | 坏文件.corrupt﹣时间戳撑爆磁盘

1 participant