Skip to content

Stale-guard ownership fast path: skip the per-save envelope read (#3049) - #3286

Merged
ranxianglei merged 5 commits into
masterfrom
2026-10-15_guard-ownership
Oct 16, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-10-15_guard-ownership

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes the read-amplification half of #3049 (the production report's ~320 ms/turn guard cost and its share of the 1.42 TB/day I/O).

What

The #405/#2401 rollback guard re-read + re-parse of the ENTIRE envelope on every save — just to compare two monotonic counters — is now skipped whenever the saving process can PROVE nothing else committed since its last build:

signal catches
unified row counters === our token (the payload we just ingested, in the same builder) any sidecar sibling sharing index.db (#394 multi-process) — their commit bumps the row before it writes the file
envelope mtime inside our build window (+1.5 s grace) writers that never touch the index: legacy-mode sibling, manual edit
incoming counters not behind token self-contained; any surprise defers to the exact full read

All three hold ⇒ the disk record is exactly the one our chain last wrote (or converged to) and the in-process comparison suffices. Any miss ⇒ the exact full read runs unchanged.

What it does NOT change

  • Legacy-only installs: zero change — no sidecar, no fast path, exact per-save full read (pinned).
  • Write amplification is untouched — that is the envelope rewrite itself (test: pin persistence write-amplification shape and mode cost ordering (#3049) #3275's regression budget / Phase 3 jsonWriter=off territory). This PR kills the READ side of the guard only.
  • Cross-instance semantics, the tie residual (persist.ts:81-84), and the storm contract are pinned byte-identical: sibling detection through the shared row, foreign-write detection through mtime, interleaved storm exact.

Deliberate residual

A sibling whose sidecar is DISABLED (3-strike drift) writing a newer envelope within the ~1.5 s grace window is missed for one cycle; its next write hits the exact guard again — strictly narrower than the unconditional size/mtime ownership skip evaluated on the issue (option 2). Crash-safe by construction: a fresh process starts with an empty token map, so the next boot always takes the full read; the rekey / #408-clamp / restoreReplica direct-write paths drop or re-stamp the token.

Tests

tests/persist-guard-ownership.test.ts — 5 new pins: single-instance collapse (12 saves → ≤2 full reads, disk counters exact), legacy exactness (one full read per save), cross-instance commit detection (shared row), foreign file write detection (mtime), interleaved storm exact with the sidecar shared. Full persist/storage regression batch green locally (multi-instance 16, tail/404/json-writer-off 16, unified/replicas/memory 28, encryption/bounded/corpus/ccr 25); tsc + unused-export gates clean.

…elope read (#3049)

The #405/#2401 rollback guard re-read and re-parsed the ENTIRE session
envelope on every save just to compare two monotonic counters — ~320 ms
and a full file read per turn on >40 MiB sessions (a large slice of the
read amplification in #3049's production reports).

In shadow/full mode the guardedBuild closure now records an ownership
token (counters of the payload it just ingested into the unified row +
its build time). The next guard proves 'no other writer committed since'
without reading the file:

  1. unified row counters === token counters — every sidecar instance
     sharing the dir shares index.db, and each commit ingests BEFORE it
     writes the file, so a sibling's newer commit bumps the row first;
  2. envelope mtime not newer than the token's build (+1.5 s grace for
     our own write latency) — catches writers that never touch the index
     (legacy-mode sibling, manual edit);
  3. incoming counters not behind the token — defers any surprise to the
     exact full read.

Legacy-only installs keep the exact per-save full read. The rekey,
#408-clamp and restoreReplica direct-write paths drop or re-stamp the
token so it never describes a payload it did not ingest. Deliberate
residual (narrower than an unconditional size/mtime ownership skip): a
sibling whose sidecar is disabled writing within the grace window is
missed for one cycle; its next write hits the exact guard again.

Tests: single-instance collapse (12 saves -> <=2 full reads, counters
exact), legacy exactness (one full read per save), cross-instance
commit detection through the shared row, foreign file write detection
through mtime, interleaved storm stays exact with the sidecar shared.
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-15_guard-ownership (9834911)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3849-pr.3286.7d0c836

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3286.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

review

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

收到 review 请求。查重: 不重复——本 PR 是 #3049 保存侧守卫读放大(read-amplification)的唯一在途修复;同域相邻项 #3275(写放大 pin,纯测试)与 #3020/#3250(Phase 3 / WAL reclaim)机制不同,无重叠。

开始按规范审:原 issue 方向 → 既有行为回归清点 → 代码 → 测试有效性 → diff 整洁度。PR head 已 rebase 到最新 master(0db928099),审的就是这个 head。结论稍后按八问体例给出。

…view)

#3286 review: statEnvelope returned { size, mtimeMs } but only mtimeMs is
consumed (ownershipFastPath). Dead field removed per the no-dead-code rule.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 结论:可合并(已顺手修掉一处死字段,见下)

独立审核说明(不照抄 PR 描述):checkout PR head 逐段读了 guardedBuild / staleDiskPayload / loadEnvelope / statEnvelope / token 生命周期与 store schema,本地跑了定向测试套件,并拉了 CI 失败日志逐条核对。以下结论全部基于自己的验证。

结构化八问

  1. 是什么问题:shadow/full 存储模式下,每次保存都要把整个 envelope 重新读一遍、parse 一遍,只为比较两个单调计数器——>40MiB 会话上约 320ms/turn,是 [perf] large active sessions (50-60MB payloads, 23k msgs) serialize the main thread: 1276ms avg local processing per turn + full 57MB snapshot rewrite for +270B deltas #3049 主线程卡顿两个成因之一(另一成因是写放大,本 PR 不动)。
  2. 什么原因导致的:ACP_PASSTHROUGH 一键静默关死全部压缩:无日志/无横幅/面板不可见且不可清除,同族 env 旋钮全部生效于无声 #405/[Bug][State] stale-write guard preserves newer disk record but stale resident keeps serving old in-memory fold state #2401 双实例回滚守卫缺少比"重读全文件"更便宜的证明手段——没有"自我们上次构建以来没有其他写入者提交过"这个所有权信号。
  3. 影响范围与严重程度:只影响 shadow/full 模式保存侧的 CPU/IO(生产报告主机恰好都是 default shadow);legacy-only 安装零变化(有测试钉死);写放大(envelope 重写本身)完全未动(test: pin persistence write-amplification shape and mode cost ordering (#3049) #3275 / Phase 3 领地)。纯性能收益,无用户可见退化。
  4. 怎么修复的:三信号所有权快路径——① unified row 计数器 === 我们的 token(每个 commit 先 ingest index.db 再写文件,兄弟实例的任何新 commit 必然先 bump row)② envelope mtime 不晚于我们的构建窗口(+1.5s grace,兜住不碰 index 的写入者:legacy 兄弟、手工编辑)③ incoming 计数器不落后于 token。三者全过才跳过全量读,任一 miss 走原样精确全读。代码层面核实过前提:guardedBuild 闭包内顺序是 guard → shadow.record(ingest)→ noteOwnCommit(token)→ 文件落盘,"先 ingest 后写文件"对任何实例成立,信号①的推理成立;rekey/代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408-clamp 路径 drop token、restoreReplica 重新 stamp、新进程空 map ⇒ 重启后首存必走全读,崩溃安全。
  5. 是否有回归问题:逐项清点过既有行为——legacy-only 路径(!shadow||disabled → false,测试断言每存一次恰一次全读);跨实例 catch-up 拒绝(T3:共享 row 检测 + warn 保留);外部文件写入(T4:mtime 检测 + warn 保留);交错 storm(T5:共享 sidecar 下 exact);tie/catch-up 残余(persist.ts:81-99 KNOWN LIMITATIONS 不在 diff 内,字节级未动);等计数器 clobber 语义(旧代码对相等计数器本就放行,快路径无新增面);两进程 ingest→write 交错竞态(row 已 bump 时仍走全读,仲裁与旧行为一致,未被放大)。未发现回归。一个理论角落(非阻塞,记录在案):若 legacy alias 文件与 canonical 文件并存(L1086 rm 失败被吞,如 Windows AV 锁)且 disabled-sidecar 兄弟在 grace 窗口内写入,statEnvelope 可能先 stat 到较旧的 alias——严格窄于文档化的 residual。
  6. 是否破坏缓存:不涉及(不触碰 prefix cache/fold 状态/重试/去重键,只改变守卫是否读文件)。
  7. 是否改了设计方向:无架构转向。行为变更披露(旧→新):shadow/full 模式保存期守卫从"无条件全量读 envelope"变为"可证明所有权时跳过"——这正是 PR 目的(性能优化),守卫的拒绝判定逻辑(单调计数器比较)字节级未动;legacy 模式保持原样。
  8. 是否引入了新的配置:无。OWN_COMMIT_GRACE_MS=1500 是具名常量,不是配置项。

测试有效性(质量关)

5 个 pin 有牙:T1 断言磁盘计数器精确 + 全读 ≤2 次(无快路径会是 12 次);T2 断言 legacy 恰好 5/5 次全读;T3/T4/T5 分别钉住三个逃生门且带内容 + warn 断言;sleepSync 用 Atomics.wait,确定性无 flake。缺口:文档化 residual(disabled-sidecar 兄弟在 grace 内写入)没有直接 pin——可接受,属刻意残留且模拟 3-strike 代价高。

CI 状态(已拉日志核对)

Run@0db928099:30 项检查 26 绿。全部 guard job(kernel-guard / pi-subagents-guard / config-surface-guard / version-guard / unused-exports / lock-sync / static-gates×4)与全部 E2E lane(real codex / fake-codex / persona / pi-codex-ws / native-dsh/pi/opencode / local-registry)绿。4 红全部是 shard-1 测试 lane(ubuntu/windows × 22/24),日志确认是同一个既有用例:tests/issue3148-upstream-circuit.test.ts:148 期望 capped: 1360 实际 4096——#3071/#3101 引入的 4096 capability floor 让旧 pin 失效,与本 PR diff 无关(本 PR 五个文件都不触碰 upstream-circuit/clamp 路径;已在本地复现同一断言差异)。该 re-pin 修复已在 PR #3275 在途(commit ec1be03,分支 2026-10-15_storage-write-amplification_pin),合入 master 后本 PR 的合并结果自动带上修正后的 pin(本 PR 未改过该测试文件)。查重:已有在途修复,不另立 issue。本地定向运行:persist-guard-ownership 5/5、multi-instance-persist 3/3、rollback-guard/json-writer-off/unified/replicas 31 pass / 0 fail / 1 skip;tsc clean。

顺手小修(已推 PR 分支)

e3d6d35:statEnvelope 返回值 { size, mtimeMs } 中 size 从未被消费(死字段),删除,2 行,suite + tsc 复验通过。

主要风险判断(residual)

刻意留下的残留:sidecar 被禁用的兄弟(3-strike drift)在我们构建后 ~1.5s 内写入更新的 envelope,会被漏一个周期;该兄弟存活时其下次写入命中精确守卫即自愈,若在下次写入前崩溃,其已保存的那轮 turns 会丢到我们内存推进为止。对比 option 2 无条件 size/mtime skip(尺寸不变就一直漏),这是严格更窄的取舍,且用罕见多进程场景换主流单实例路径的 320ms/turn。判定:可接受,PR 描述对它的刻画准确。

判定:可合并。 e3d6d35 的新 CI run 出来后预期同样只有那 4 个既有红(待 #3275 合入后转绿)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

这条是 floor 3「review」的重复投递——该请求已在上方按八问体例完整答复(结论:可合并),线程里没有新的用户输入;github-actions 的构建产物评论无需处理。补充两条最新状态:

  1. 修复提交后的新 CI run(head e3d6d35):全部 E2E lane(real/fake/persona codex、pi-codex-ws、native-dsh/pi/opencode)、PR Build Artifact、Windows codex overlay 及首批 guard job 已绿;shard 测试 lane 还在 runner 队列中。预期不变:唯一红项仍是那 4 个既有 shard-1 用例([bug] 上游在会话中途不可用时,bili 把一次故障放大成四重代价:固定节奏 20 连击全部冷发、输出预算被静默钳制 83 次(最狠 65536→1360)、紧急收缩与 nudge 停摆均不可见 #3148 旧 pin capped:1360 vs [bug] 0.1.195 输出预算三路饿死仍复现:无 output 上限模型 restore 无下限(1→252、fork 原样转发 1)、#2490 planning floor 钳到 1479–2623 → 3 会话 9 次 finish=length 空轮不重试(#546/#1665/#1840/#2904/#2756 残留) #3071 的 4096 floor)。刚核实过:PR test: pin persistence write-amplification shape and mode cost ordering (#3049) #3275(re-pin 修复)仍 open、master 上该测试文件尚未修正,所以这 4 个红要等 test: pin persistence write-amplification shape and mode cost ordering (#3049) #3275 合入后才转绿——本 PR 未改过该测试文件,test: pin persistence write-amplification shape and mode cost ordering (#3049) #3275 合入后合并结果自动带上修正 pin。
  2. master 前进了一个无关 PR(fix: #3272 disarm CCR arming on the codex responses lane (unroutable acp_retrieve) #3277 ccr-codex-lane-disarm,本 PR 分支显示 behind):与本 PR 无冲突(GitHub mergeable=True),领域不交叉(CCR/codex lane vs persist 守卫),不影响审核结论。

判定维持:可合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 预测核对完毕(拉了失败 shard 日志):head e3d6d35 上首个完成的测试 shard(ubuntu 24, shard-1)2887 用例仅 1 败——not ok 982 - #3148: clampOutgoingOutput …(tests/issue3148-upstream-circuit.test.ts),与上一轮 run 完全同一签名,即那个等 #3275 re-pin 的既有用例;其余 guard/static/E2E lane 全绿。判定不变:可合并,红项待 #3275 合入后自动转绿。

@ranxianglei
ranxianglei merged commit 7f8fbbb into master Oct 16, 2026
35 of 40 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.478-master.7f8fbbb (rolling master tag).

npm install -g billion-context@0.0.478-master.7f8fbbb  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Oct 16, 2026
…; output-restore floor respects proven budgets (#3406 items 2+4)

Item 2 — the #3286 ownership fast path judged envelope ownership by a
builtAtMs+1.5s grace window, so a sibling writing inside that window (legacy
mode, sidecar disabled — never touches index.db) was adopted as ours and its
envelope clobbered on the next save. The token now records the stat taken
right after OUR write landed (size+mtime); the fast path fires only when the
file still matches that stamp byte-for-byte. Unstamped tokens (debounced
kernel-internal saves, restore-replica seeds, crash before stamping) always
take the full read. Residual window shrinks from 1.5s to ~µs.

Item 4 — restoreOutputBudget's no-ceiling branch raised a session's recorded
high-water to OUTPUT_RESTORE_ABSOLUTE_FLOOR (32768); providers hard-capping
maxOutputTokens below that reject the raised value outright (400). A high-water
at/above the new trust line (4096, aligned with OUTPUT_CLAMP_MIN_OUTPUT) is
the provider-accepted budget and is restored as-is, never raised above;
missing/below-line high-waters keep the #3071 floor (born-starved, fork
rattle, death-spiral decay chains).

Item 3 (label resume vs background scan race, #3214) — NOT changed: no
low-risk mutex exists without a persistence-format/index change (details in
the PR body); deliberately stopped per the risk gate.
@github-actions github-actions Bot mentioned this pull request Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant