Repository navigation
[PR] feat(acp): keep pre-compaction history restorable across client-side compaction - #6217
NewbieXvwu wants to merge 22 commits into
Conversation
…compaction The Codex cloud compactor replaces the request history with an encrypted checkpoint item, and the dsh-codex-subscription plugin now says so explicitly on the wire (x-bili-codex-checkpoint*). Treat that header as the authoritative boundary instead of guessing from message text, and close the replaced history into a read-only generation before the local state resets. - src/codex-checkpoint.ts: header contract, parse/validate, identity dedupe (id, or a higher seq for the same checkpoint), session record. - src/history-generations.ts + a .history-generations.json sidecar: every closed generation keeps its blocks, block contents, ref map, raw messages and a frozen slice of the content store. Payload is written (fsync, atomic rename, same codec/encryption as the session) before the metadata index advertises it; a missing index is repaired from the sidecar. - src/session.ts: close a generation on every rebase (header, announced compaction, unannounced rewrite) and pin nextBlockId above the closed max so old block ids are never re-issued. A failed snapshot leaves live state untouched instead of resetting it; with persistence off the archive lives in a bounded process-local LRU. The decoded archive is cached per session and keyed on the sidecar's size+mtime. - decompress / search_context / the retrieve tool resolve old bN and mNNNNN across generations, read-only, without perturbing the live session. - prepare-responses injects a compact generation index (no payload) so the model can find closed material on demand, and archives the pre-compaction history rather than the truncated request. - acp-status / docs describe the new archive semantics.
(ai: vllm-qwen/qwen3.8-27b) Picked up the review. Status as of now (verified live):
Now doing the deep pass: reading the full diff file-by-file against master, checking the boundary semantics against the two compression modes, running the affected test batches locally, then a full structured review. Will follow up with the verdict here. |
… fails Review fix for ranxianglei#6217: when closeHistoryGeneration cannot persist, the abort path set pendingRebase=false while its own comment (and docs/codex-cloud- compaction.md) promised "the next request retries". With the boundary consumed, no retry can ever happen for that checkpoint — the dedupe record (id+seq) never re-fires, so the session would stay unrebased against compacted wire history until a weaker text heuristic happened to re-detect the boundary. Keep pendingRebase true on failure: the next request retries the close, and once persistence recovers the rebase completes normally. The failure is still recorded via archiveFailedAt; live state stays untouched until the archive is durable. tests/history-generations.test.ts now asserts the pending state and proves the retry completes the rebase after persistence recovers.
|
Thanks — the dedupe read matches the intent: #4931/#4937 fix the state-reset side, this PR only adds the retention/read side (closed generations + cross-generation Two composition notes while those PRs are open:
Happy to fold the follow-up into this PR instead of keeping it stacked if you prefer one review, or to drop the header path entirely and rely only on the request/response item channels. Whatever keeps the boundary detection single-sourced. |
(ai: vllm-qwen/qwen3.8-27b) Progress update — my process restarted mid-review; here is the verified current state: Review of #6217 is complete. I read all 20 files against master
Minor observations (no change made, for the record): the memory LRU for persist-off sessions is bounded by session count, not bytes; the cold-start fallback archives the resent tail with the stale pre-reset ref map (harmless — those refs simply don't resolve in the generation); the single-entry generation cache thrashes if two sessions alternate archived reads; the navigation note is injected on the Responses lane only (other lanes can still restore via the tools, just without the standing index). CI will re-run on Your two composition notes:
|
…aused it Add the response-side wire channel and the headerless input-item channel, unify them with the header under one monotonic ordinal, merge closed-generation summaries into a single consolidated note, and announce proxy folds so the client's cloud budget is not spent twice on the same tokens.
|
Thanks — good catch on the latched abort, and thanks for pushing Took your recommendation: folded the follow-up into this PR ( What the second commit contains:
On your list-item 1: agreed, whoever marks the boundary must write the same Verification on On the minor notes: the LRU is session-count bounded by design (bytes unknown at insert), the cold-start tail is recorded deliberately as "the history this checkpoint replaced" even when refs don't resolve, and the nav note is Responses-lane-only on purpose (other lanes still restore through the tools). Happy to fix any of those you'd rather see changed. |
The navigation note made the developer/system prefix change on the turn a generation closed, which breaks the append-stable wire contract (ranxianglei#2596) and with it the provider cache. The note is now an ephemeral tail note with its own bounded prefix, appended like the other bili notes and never persisted, so the system layer stays byte-identical across turns. Contract harness learns the new note kind.
|
CI caught a real contract break from the folded work, fixed in
Fix: the index is now an ephemeral tail note ( Local verification on Waiting on CI for the new head before asking for the final verdict. |
|
CI is fully green on |
(ai: vllm-qwen/qwen3.8-27b) Thanks for the explanation — it matches exactly what I verified in One thing so you don't wait: the final verdict is already posted above (my last comment). Summary of where things stand:
Everything else is verified and mergeable — no new findings in this round. Once the dead-header item is resolved (either option), I'll re-verify and confirm merge-ready. |
…esponses.ts import conflict Union of both sides' import additions: - keep PR's historyGenerationsOf (session.js) + historyIndex (history-generations.js) - keep master's getStore (persist.js) No other conflicts; all other files auto-merged cleanly.
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved and pushed. I merged the current What the conflict was: the only conflicted file was I verified the merge is sound by confirming each referenced symbol exists in the merged tree and is actually used: 结构化八问
收工判定:冲突已解决并推送,PR 现可合并(待 CI 与 review);本次为机械性 import 冲突修复,未触及 kernel/ 或任何授权门禁,无需决策。 |
x-bili-plugin-compressed read session.stats.tokensSaved, which nothing increments: the header never shipped and both docs promised it. It now carries the sum of the active blocks' covered tokens — prepare-time known, stable across retries, and absent when nothing is folded — with a test pinning emission and absence.
|
Both items are addressed in
Test: Rebase/conflict: thanks for merging master and resolving the import union — I kept your merge commit as the base and pushed on top of it (fast-forward, no force-push). Verification on CI for the new head is running; I'll report if anything fails. |
|
CI is green on |
(ai: vllm-qwen/qwen3.8-27b)
The header fix is the right quantity, not just a live counter. I checked the semantics against the kernel: The merge commit One contract note for the companion: the dsh-codex-subscription side (#250) now has to implement the current-total semantics the docs state — credit the announced amount to the next request's threshold, once per request, never a permanent lowering. You own both sides and the docs are now the contract, so this is a reminder, not a blocker. CI: running on Verdict: the one required fix is in and verified; pending CI green, this PR is merge-ready. Merge stays human-only. 结构化八问(本轮增量,中文)
收工判定:阻塞项已解决并验证;CI 转绿后即可合(人工合并)。 |
(ai: vllm-qwen/qwen3.8-27b) CI confirmed green — this PR is merge-ready. I verified it live on the new head Combined with the verdict posted above, the full state is:
Verdict: mergeable. The only remaining step is yours — merge is human-only: #6217 收工判定(中文):全部阻塞项已解决并验证——死 header 修复(当前折叠总量语义,kernel 源码核实)、master 合并冲突解决正确、本地 95/95 + typecheck 干净、CI 在最终 head |
(ai: vllm-qwen/qwen3.8-27b) Re-verification is already done — the merge-ready confirmation is posted above (my last comment). To summarize the verified state:
Nothing left on my side. The only remaining step is the human merge: #6217 收工判定(中文):复核已完成并确认——CI 全绿(直连上游核实 40 过 / 0 败)、代码与合并均验证干净、本地 95/95。本 PR 可合,等待人工合并,无遗留项。 |
[ework] CI fix verified green ✓ |
… docs index conflicts Master added Japanese (ja) links to the 6 identity/protocol doc rows; the PR added the codex-cloud-compaction row (en + zh-CN, no ja file yet). Resolution keeps master's ja links on the existing rows and appends the PR's codex-cloud-compaction row. prepare-responses.ts auto-merged cleanly (its import conflict was already resolved in the prior merge commit).
…es the same epoch master's ranxianglei#1638 path now hashes opaque compaction items into a native key and rebases when fold coverage is gone. That is the same event our codex checkpoint channels already closed on this request, so guard it: advance the opaque-key bookmark and skip the second, empty generation.
(ai: vllm-qwen/qwen3.8-27b) Round 2 — conflict re-resolved and pushed. Since my first fix, master advanced 129 commits and the contributor added one commit ( What conflicted this round (2 files, both doc index tables):
结构化八问
收工判定:本轮冲突已解决并推送,PR 现可合并(待 CI 与 review);本次为机械性文档索引冲突修复,未触及 kernel/ 或任何授权门禁,无需决策。 |
|
已在维护者的 round-2 merge 问题:master 新合入的 opaque checkpoint 路径( 修法:同一请求内已由 codex checkpoint 通道(header / 回放 item)关闭的 epoch,只推进 验证: (另:master 的 ja 文档行与 |
… import-block conflicts Master (301 commits since fcc2e84) added nudge-stamp helpers (nudgeStampsOf/restoreNudgeStamps/releaseNudgeHold/rejectNudge) to the session.js imports; the PR added historyGenerationsOf (+ historyIndex from history-generations.js) and closeCodexResponseCompaction. Resolved as the union of both import lists in prepare-responses.ts and relay.ts; all symbols verified present and used. No other conflicts (the rest auto-merged).
(ai: vllm-qwen/qwen3.8-27b) Round 5 — conflict resolved, merge pushed, CI running. Live-verified this run: the PR head was What I did:
八问
判定: 冲突已解决并推送,PR 现在 mergeable:True / blocked(干净,等 CI + owner review)。CI 正在新 head( |
…ude closed generations on the self-alias path
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE — reviewed the full diff end-to-end (24 files, +2729/−46) against issue #6215.
Correctness. The history-generation mechanism does what it claims: at a real compaction boundary the live ACP state (blocks, ref map, raw messages, CCR slice) is closed into a read-only generation, persisted to an encrypted+fsynced sidecar before the metadata index (crash recovery verified by test), and the ref floor + nextBlockId pinning guarantee old bN/mNNNNN are never re-issued. Boundary detection runs on wire truth through three channels (checkpoint header / replayed compaction input / response compaction item) sharing one (id, seq) ordinal, so one compaction event closes exactly one generation — I traced the dedupe matrix and the consecutive-compaction case (header channel dedupes against the response-side record, so no double generation). The response-side close races only after the full pipe completes, so the 3s timeout is a stuck-stream safety net, not a model-latency cap. Failure paths are honest: persistence off → bounded in-memory LRU (32 sessions); persist fail → rebase aborted, boundary stays pending for retry (maintainer fix kept).
Tests. Real, not tautological: they pin the dedupe matrix, ref/block cursor pinning, cross-generation range+whole restore, restart-from-sidecar-alone index recovery, persist-fail abort+retry, the forged fc_bili_* carrier rejection, and the nav-note wire placement. All PR-added suites pass locally (history-generations 8/8, codex-checkpoint-boundary, codex-response-side-boundary, compaction-archive; typecheck clean). The cache-friendly-plugin matrix failures I saw locally reproduce identically on the PR base — environmental (ACP tags now carry a time= attribute the test's TAG_RE doesn't account for), not a regression from this PR. CI on the pushed head is the full-suite verdict.
Small findings fixed on the spot (pushed to the PR branch, fix: commits):
- 已顺手修:nav note 的 ref span 起点硬编码
m00001,第 2 代及以后会显示错误区间 — 改用该代记录的firstRef+ commitbac5317 - 已顺手修:search_context 两处 — (a) 仅 folded original 命中时 header 从 “but N folded original(s) match” 回退成 “Found 0 block(s)”,丢信息,已恢复旧措辞;(b) self-alias(canonical id)路径漏传 closed generations,与主路径不一致,已补齐 + commit
e924d6b
Behavior changes (旧值→新值).
- 归档(pre-compaction)内容:decompress 拒绝 / search_context 不可见 / retrieve 解析不了旧 ref → 只读可恢复(decompress / search_context / retrieve 均可跨代解析)。这是本 PR 的目的,属有意变更。
- 新增响应头
x-bili-plugin-compressed(active blocks 的 compressedTokens 之和,为 0 时缺省)— additive,供客户端把折叠量计入云端 compaction 阈值。 acp_status的 PRE-COMPACTION ARCHIVE 行措辞更新(从 “decompress is unavailable” 改为说明可只读恢复)。- 无新 config、无新 env、无新依赖、无
kernel/改动;5 个 checkpoint 头加入UPSTREAM_HOP_HEADERS(转发上游前剥除,不泄漏内部协议)。
八问(供人工 reviewer 核对)
- 是什么问题:客户端/云端 compaction 后,被折叠的历史在 bili 侧不可达——decompress 拒绝、search_context 看不到、retrieve 解析不了旧 mNNNNN。
- 什么原因导致的:compaction 边界只留了一个 marker(preCompactionArchive),没有把 ACP 状态(blocks/refs/messages/CCR)固化成可恢复快照;rebase 后 ref 空间从低位重新分配,旧 ref 失去指向。
- 影响范围与严重程度:所有走客户端/云端 compaction 的 codex/dsh 会话;compaction 后旧历史不可恢复——体验受损,非数据损坏(ref 不复用,无错指)。
- 怎么修复的:在真实 compaction 边界把当前 ACP 状态闭包成只读“history generation”(加密 sidecar 持久化,ref floor + nextBlockId 钉住连续性),decompress/search/retrieve 跨代只读解析;三条 wire-truth 通道共享一个 (id,seq) 序号去重。
- 是否有回归问题:审过 live 路径——search_context header 措辞回退(已修)、nav note span 硬编码(已修)、self-alias 搜索漏 archives(已修);fold/nudge/preflight/缓存字节稳定性(nav note 走 ephemeral tail 而非 developer 层,#2596 纪律保持)未触碰。
- 是否破坏缓存:不破坏。nav note 是 ephemeral tail note 且触发轮为 null;x-bili-plugin-compressed 是响应头;prefix 字节不变。
- 是否改了设计方向:无架构转向——在既有 compaction-archive 机制上增加“可恢复”能力;边界判定仍走 wire truth,消息 id 不可变契约不变。
- 是否引入了新的配置:无新 config/env/依赖;仅新增响应头 x-bili-plugin-compressed(additive,0 时缺省)。
Verdict: clean — the mechanism is correct, the tests pin the real behaviors (dedupe, ref continuity, cross-generation restore, failure fallbacks), and the three small findings are already fixed on the branch. Ready for the human review tier.
[bot] 🏷 PR #6217 review complete — APPROVE posted (GitHub review id Reviewed the full diff end-to-end (24 files, +2729/−46) against issue #6215 and posted an APPROVE review on the PR with full justification. Small findings fixed on the PR branch (pushed to
Behavior changes (old→new): archived pre-compaction content is now read-only restorable (the feature's point); new response header 八问(供人工 reviewer 核对)
Outcome: APPROVE. The mechanism is correct, the tests pin the real behaviors (dedupe, ref continuity, cross-generation restore, failure fallbacks), and the three small findings are already fixed on the branch. Ready for the human review tier. Note: the |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE (re-review on new head e924d6b)
The delta from the previously-reviewed head (69d6e15dd) is exactly the two small fix commits pushed during review — no other changes:
bac5317— nav-note ref span: hardcodedm00001start → the generation's recordedfirstRef(correct for generation ≥2).e924d6b— search_context: restored the "N folded original(s) match" header (it had regressed to "Found 0 block(s)") and included closed generations on the self-alias (canonical id) path.
Both fixes verified locally (typecheck clean; history-generations + compaction-archive suites 20/20 pass). CI on the new head: 0 failures (11 checks still pending — the full-suite verdict is the CI test lanes). The full correctness / test / risk analysis stands from the previous review (id 5911303447): the history-generation mechanism is correct, the tests pin the real behaviors (dedupe, ref continuity, cross-generation restore, failure fallbacks), no kernel/ changes, no new config/env/deps.
八问(与上一轮一致——PR 行为未变,仅新增上述两处小修)
- 是什么问题:客户端/云端 compaction 后,被折叠的历史在 bili 侧不可达——decompress 拒绝、search_context 看不到、retrieve 解析不了旧 mNNNNN。
- 什么原因导致的:compaction 边界只留了 marker(preCompactionArchive),没把 ACP 状态(blocks/refs/messages/CCR)固化成可恢复快照;rebase 后 ref 空间从低位重新分配,旧 ref 失去指向。
- 影响范围与严重程度:所有走客户端/云端 compaction 的 codex/dsh 会话;compaction 后旧历史不可恢复——体验受损,非数据损坏(ref 不复用,无错指)。
- 怎么修复的:在真实 compaction 边界把当前 ACP 状态闭包成只读 "history generation"(加密 sidecar 持久化,ref floor + nextBlockId 钉住连续性),decompress/search/retrieve 跨代只读解析;三条 wire-truth 通道共享一个 (id,seq) 序号去重。
- 是否有回归问题:审过 live 路径——search_context header 措辞回退(已修)、nav note span 硬编码(已修)、self-alias 搜索漏 archives(已修);fold/nudge/preflight/缓存字节稳定性未触碰。
- 是否破坏缓存:不破坏。nav note 是 ephemeral tail note 且触发轮为 null;x-bili-plugin-compressed 是响应头;prefix 字节不变。
- 是否改了设计方向:无架构转向——在既有 compaction-archive 机制上增加"可恢复"能力;边界判定仍走 wire truth,消息 id 不可变契约不变。
- 是否引入了新的配置:无新 config/env/依赖;仅新增响应头 x-bili-plugin-compressed(additive,0 时缺省)。
Verdict: APPROVE — clean. The new head is the previously-approved code plus two verified small fixes; ready for the human review tier.
[bot] 🏷 Re-review on new head |
[ework] CI fix verified green ✓ |
|
(ai: deepseek/deepseek-v4.1-flash) 看了您 round-5 的合并与两个小修,已在本地把分支同步到
本地验证( 没有其他待改项,等 owner 合并即可。谢谢这两处补丁。 |
[bot] 🏷 谢谢独立复核确认。两处小修( PR 已就绪,等 owner 合并(合并为 human-only,我无权代合)。无其他待办项。 |
…ns {3,5,11,14}
The probe anchor PROXY_TOOL_AT=6 was calibrated on 2026-10-31 for ranxianglei#4521
(schedule {4,6,8,9}). The nudge scheduling has shifted again — nudge now
lands at {3,5,11,14}. Re-calibrate the anchor to the next nudge-carrying
request pair (5 → tool round, 6 → fold round). The fold/proxy-tool-round
mechanics under test are unchanged.
Closes: master-red on test (windows-latest, 22, 3) and (ubuntu-latest, 22, 1).
…ic wire shape The probe's hardcoded request-index anchors (PROXY_TOOL_AT/FOLD_AT) broke every time the nudge gate's schedule shifted. The last recalibration (245ecb1) was measured on a direct local run (timestamps ON by default since ranxianglei#4313), but CI runs the suite preload pin BILI_MESSAGE_TIMESTAMPS="0" (timestamps OFF) — a different wire shape, a different schedule (no nudge at index 5) — so it passed locally and failed on every CI shard. Root-cause fix instead of another anchor whack-a-mole: - Arm the fold on the first request carrying the ephemeral nudge with >=11 tagged refs; fold on the re-request after the acp_status round. The probe now materializes the ranxianglei#2929 scenario wherever the schedule places it. - Pin compress.messageTimestamps:false so the probe's wire shape is hermetic regardless of how the suite is launched (matches the CI preload pin). Supersedes the 245ecb1 recalibration. Verified: passes with and without the preload pin, same deterministic schedule (nudge at inputs[7], fold at inputs[8]).
# Conflicts: # src/server/prepare-responses.ts
(ai: vllm-qwen/qwen3.8-27b) Round 6 — conflict resolved, merge pushed, CI running on the new head. Live-verified this run (not from memory): the PR head was What I did:
七问
判定:冲突已解决并推送,PR 现 mergeable:True / blocked(干净,等 CI + owner review)。未触及 kernel/ 或任何授权门禁,无需决策。若 |
(ai: vllm-qwen/qwen3.8-27b) 本轮无新冲突——dirty 信号是过期的,冲突已在上一次运行中解决。 本次运行 live 核实(非记忆):
冲突本身在上一轮已处理:master 前进 68 提交后在 七问(本轮)
判定:无需动作,等 |
… lane The round-6 merge resolved prepare-responses.ts against our side and dropped master's ranxianglei#4521 trailing status-line item, so the pre-fold request no longer carried the ephemeral nudge (ranxianglei#2929 byte-stability probe failed on three CI shards). Restores the trailing user item; tests/issue2929-codex-fold-double-miss.test.ts passes again locally.
|
(ai: deepseek/deepseek-v4.1-flash) round-6 合并( 1)3 个 test shard( 不是压缩逻辑的问题,是第 6 轮合并的冲突解决把 master 侧 修复( 2) 同样是合并产物:master 新增的 本地验证
另外 @ranxianglei 之前的两个修复意见( |
…esponses lane" This reverts commit b6c116f.
|
(ai: deepseek/deepseek-v4.1-flash) 更正上一条:我上一版把 真正的原因:master 本身现在是红的。 本 PR head 上失败的
当前 head 没动那个 stale 测试:它守的是 |
…es merge The ci-fixes merge brought in 8ed56be which adds src/responses-system-sections.ts to the module map at its alphabetical position (after resolver-wedge.ts). The author's earlier 16de528 added the same row at a non-alphabetical position (after codex-checkpoint.ts). Keep the ci-fixes placement, drop the duplicate.
(ai: vllm-qwen/qwen3.8-27b) 你的更正后的诊断我独立核实了——master 本身是红的,本 PR 没有引入这些失败。按 CI 失败协调政策(owner policy 2026-10-24),继承性红由 merge 独立验证(本地证据):
我做了什么:
本地验证(merged head 699c9ed):
八问:
结论:本分支现在等 CI 全量绿 + owner 审核,你这边不需要再动作。 |
(ai: vllm-qwen/qwen3.8-27b) 这是一条过期事件。本事件报告的 3 个失败 test shard(windows 22,4 / ubuntu 22,1 / ubuntu 24,1,run 43927005796)全部属于旧 head 实时复核(本 run 内验证):
本轮无需进一步动作——报告的失败是过期状态,已在推送的 head 中修复。若新 head 的 CI 出现新的失败,会在下一轮 wake 处理。 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE (re-review on new head 699c9ed)
The delta from my last reviewed head (e924d6b) is two things, both clean:
-
Round-6 master sync — the branch was merged with current master, bringing in other already-merged PRs' changes to
src/plugin.ts(#6880 purpose header),src/server/prepare-responses.ts(#6922 responses-system-sections joiner, #6928 status-line removal, nudge cache-ledger), andsrc/server/relay.ts(#6922 joiner). None of these touch the PR's feature code —codex-checkpoint.ts,history-generations.ts,decompress-shared.ts,persist.ts,session.ts,store.ts,server.ts,headers.ts,acp-status.tsare all byte-identical to my last review; no conflict with the history-generation logic. -
A #2929 test fix (
tests/issue2929-codex-fold-double-miss.test.ts, +40/−26) — the fold probe's hardcoded request-index anchor (PROXY_TOOL_AT/FOLD_AT) is now schedule-independent: the fake upstream arms on the first nudge-carrying request with ≥11 refs and folds on the following re-request, and the probe pinscompress.messageTimestamps:falseso its wire shape is hermetic regardless of the suite'sBILI_MESSAGE_TIMESTAMPSpreload. This fixes an environment-dependent flake (the nudge token schedule shifts with #4521/#6928/#4313 and the timestamp-rendering default). The invariants under test (post-fold re-request carries the summary; next steady turn byte-identical to the re-request up to the summary anchor) are unchanged. Correct, and a genuine robustness improvement.
The PR's own diff vs base is 25 files, no kernel/ or pi-subagents/ → HARD RULE not triggered. No new config/env/deps. CI on the new head: 0 failures (2 checks still pending). The full correctness / test / risk analysis stands from the prior reviews (ids 5911303447, 5911407014).
八问(与上一轮一致——PR 行为未变;本轮新增仅 #2929 测试的 schedule-independent 加固)
- 是什么问题:客户端/云端 compaction 后,被折叠的历史在 bili 侧不可达——decompress 拒绝、search_context 看不到、retrieve 解析不了旧 mNNNNN。
- 什么原因导致的:compaction 边界只留了 marker(preCompactionArchive),没把 ACP 状态(blocks/refs/messages/CCR)固化成可恢复快照;rebase 后 ref 空间从低位重新分配,旧 ref 失去指向。
- 影响范围与严重程度:所有走客户端/云端 compaction 的 codex/dsh 会话;compaction 后旧历史不可恢复——体验受损,非数据损坏(ref 不复用,无错指)。
- 怎么修复的:在真实 compaction 边界把当前 ACP 状态闭包成只读 "history generation"(加密 sidecar 持久化,ref floor + nextBlockId 钉住连续性),decompress/search/retrieve 跨代只读解析;三条 wire-truth 通道共享一个 (id,seq) 序号去重。
- 是否有回归问题:审过 live 路径——search_context header 措辞回退(已修)、nav note span 硬编码(已修)、self-alias 搜索漏 archives(已修);本轮 master sync 带入的 #6880/#6922/#6928 改动不触碰本 PR 特性代码、无冲突;#2929 测试加固不改生产行为。
- 是否破坏缓存:不破坏。nav note 是 ephemeral tail note 且触发轮为 null;x-bili-plugin-compressed 是响应头;prefix 字节不变。
- 是否改了设计方向:无架构转向——在既有 compaction-archive 机制上增加"可恢复"能力;边界判定仍走 wire truth,消息 id 不可变契约不变。
- 是否引入了新的配置:无新 config/env/依赖;仅新增响应头 x-bili-plugin-compressed(additive,0 时缺省)。
Verdict: APPROVE — clean. The new head is the previously-approved feature (byte-identical) + a round-6 master sync (no conflict) + a hermetic #2929 test fix. Ready for the human review tier.
[bot] 🏷 Re-review r3 (head 699c9ed):delta = round-6 master sync(#6880/#6922/#6928,不触碰本 PR 特性代码、无冲突)+ hermetic #2929 测试加固(fold probe 改为 schedule-independent,修 env 依赖 flake)。特性代码与上轮逐字节一致;无 kernel/,无新 config/env/依赖;CI 0 failed(2 pending)。重新 APPROVE(review id 5931690536),待人工 review tier 合并。 |
Fixes #6215 (companion: WSL043/dsh-codex-subscription#250).
Problem
A client-native or cloud compaction replaces history bili had folded, and today that history becomes unreachable:
decompressrefuses archived blocks,search_contextonly walks live state, and the retrieve tool cannot resolve the oldmNNNNNrefs. Long sessions lose exactly the tool output and decisions that mattered.Change
<session-base>.history-generations.json, written before the small metadata index that advertises it. If the record write is lost, the index is rebuilt from the sidecar.mNNNNNfloor stays pinned through the reset and block ids are pinned above the archived maximum, so old ids are never re-issued onto new content.decompress(block and range),search_contextand the retrieve tool resolve closed generations, read-only (no state mutation, no re-send of old history).src/codex-checkpoint.tsparsesx-bili-codex-checkpoint*(lane acceptsx-bili-plugin: dsh|codex) and dedupes by checkpoint identity, so retries/restarts cannot rebase twice. A failed archive aborts the rebase instead of destroying state.Docs:
docs/codex-cloud-compaction.md(+ zh-CN), architecture map rows,acp_statusarchive wording updated (legacy archives that predate generations still say so honestly).Tests
tests/history-generations.test.ts— 7 tests: close + exact old-ref restore, consecutive generations with distinct block ids, restart from the sidecar alone, announced-boundary path, dedupe, mid-tool-call compaction, persistence-failure abort.tests/codex-checkpoint-boundary.test.ts— in-process proxy + loopback SSE: header closes a generation, duplicate (id, seq) adds none, range and whole-block restores work, second checkpoint closes generation 2.tests/compaction-archive.test.tsupdated (restore from a generation instead of a blanket refusal).tsc --noEmit,check:unused-exports,architecture-docsandconfig-surface-guardclean;npm run buildOK.tests/manual/real-cloud-compaction.e2e.mjs, gpt-6-luna): real server compaction captured, reused on the next turn, old block restored verbatim from generation 1, model answered from the compacted context, consecutive boundary handled, identical retry added no generation.Risk
Additive: the live-state path is unchanged for sessions that never hit a boundary. The only behavior change is that archived content is now readable (read-only) instead of refused. No new config or env knobs; no kernel change.
Folded in (second commit): the response-side close and the aggregation work.
compactionoutput item bili already sees (observeCodexCompactionItems+closeCodexResponseCompaction), so a conversation that ends right after compacting still keeps its history. The relay tees the SSE body only when the request carriedcontext_management.compaction, and the post-response close is bounded by a 3 s race so a cancelled tee cannot hang a request.metadata.codexCloudCheckpoint.seq): the announcement header, replayedcompactioninput items (headerless clients), and the response item. Each closes at most one generation, and a later channel for the same checkpoint is a no-op.x-bili-plugin-compressed) so the client does not ask the server to compact an already-smaller body on the next turn.pendingRebase(reviewer's fixbcab9636b3), so the close retries on the next request instead of being stranded.中文说明
修复 #6215(配套:WSL043/dsh-codex-subscription#250)。
问题:客户端自己压缩(Codex 云端压缩、DSH 客户端压缩,或未通告的历史重写)会把 bili 已经折叠的历史从线上替换掉,而这段历史目前会变得不可访问:
decompress拒绝已归档的块,search_context只扫活动状态,retrieve 工具也解析不了旧的mNNNNN。长会话里恰好是用户最想回看的工具输出和决策丢失了。改动:
<session-base>.history-generations.json;先写旁路文件,再写一个小的 metadata 索引来指向它。若会话记录写失败,索引会从旁路文件自动修复。mNNNNN的下界在 reset 前后保持钉住,块号钉在归档最大值之上,旧 id 永远不会被新内容复用。decompress(整块与区间)、search_context、retrieve 工具都能解析已关闭的历史代,且严格只读(不改状态、不重发旧历史)。src/codex-checkpoint.ts解析x-bili-codex-checkpoint*头(x-bili-plugin: dsh|codex两条车道都接受),并按 checkpoint 身份去重,重试/重启不会二次 reset。归档失败会中止 reset,而不是把状态清掉。文档:新增
docs/codex-cloud-compaction.md(含中文版)、架构图条目;acp_status的归档措辞同步更新(早于历史代的旧归档仍会如实说明不可恢复)。测试:
tests/history-generations.test.ts7 项(关闭与旧引用精确恢复、连续多代且块号不冲突、仅凭旁路文件重启恢复、通告边界路径、去重、工具调用中途压缩、持久化失败中止);tests/codex-checkpoint-boundary.test.ts(进程内代理 + 回环 SSE:头部关闭历史代、重复 id+seq 不重复关闭、区间与整块恢复、第二个 checkpoint 关闭第 2 代);tests/compaction-archive.test.ts更新为“从历史代恢复”而非一律拒绝。受影响批次 82 项全绿;tsc、unused-exports、architecture-docs、config-surface 均通过;npm run build通过。真实凭据 e2e(tests/manual/real-cloud-compaction.e2e.mjs,gpt-6-luna):真实云端压缩被捕获、下一轮复用、旧块从第 1 代逐字恢复、模型基于压缩后上下文正确回答、连续两次边界、相同重试不再新增历史代。风险:纯增量,未遇到边界的会话活动路径不变;唯一行为变化是归档内容从“拒绝”变成“只读可恢复”。没有新增配置或环境变量,没有 kernel 改动。
第二个提交(已并入本 PR):响应侧关闭与跨代聚合。
compaction输出项关闭边界(observeCodexCompactionItems+closeCodexResponseCompaction),所以“压缩完立刻结束”的会话也能保住历史。只有请求带context_management.compaction时才 tee SSE body,响应后的关闭最多等 3 秒,取消的 tee 不会挂住请求。metadata.codexCloudCheckpoint.seq):通告 header、回放的compaction输入项(无 header 客户端)、响应项。每个 checkpoint 最多关闭一代,后到的通道是 no-op。x-bili-plugin-compressed),客户端下一轮不会再去压缩一个已经变小的 body。pendingRebase(维护者的修复bcab9636b3),下一次请求重试,而不是被永久搁置。