Skip to content

[PR] fix(#4521): codex local auto-compaction rebase on the OpenAI wire lane - #4542

Merged
ranxianglei merged 6 commits into
masterfrom
2026-10-24_codex-openai-local-compact
Nov 5, 2026
Merged

ranxianglei merged 6 commits into
masterfrom
2026-10-24_codex-openai-local-compact

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4521

What

codex configured for chat completions (wire_api = "chat", or a declared providers[].protocol: openai outranking the URL heuristics per #1909) rides bili's OpenAI wire lane — it never transits /v1/responses. The codex local auto-compaction detector (#2372) exists only in the responses lane, so its landings fell into the generic unannounced-rewrite path:

Field evidence (#4521 diagnostics dump, bili v0.1.205, win32): a codex session with unannounced-rewrite + orphan-reap: reaped 8 zombie fold block(s), plus the native compaction boundary: archived 0 … pruned ref maps to 17 live raw id(s) log line that only the openai-lane generic path produces.

Fix

Port the #2372 detector to src/server/prepare-openai.ts, same gates as the responses lane: codex host identity (isCodexHost, #3686) + active fold blocks + decimated fold coverage (foldCoverage) + the codex summary template prefix in resent history (carriesCodexLocalCompactionSummary) + the threshold constants (CODEX_LOCAL_COMPACTION_MIN_MISSING, missing*2 >= expected). On hit: record native-compaction, then markNativeCompactionBoundary + reconcileNativeCompactionBoundary back-to-back (the same immediate reset the responses/dsh lanes perform), and skip the generic rewrite path so nothing is double-recorded (same discipline as #1001/#2432). The block sits AFTER the dsh block, so a profile matching both keeps the responses-lane precedence (dsh wins there).

Behavior change (flagged per owner policy)

For codex sessions riding the OpenAI wire whose local auto-compaction lands:

  • old: unannounced-rewrite ledger event (unconfirmed tier), no state reset; orphan blocks cleaned ~1 min later by the zombie reap;
  • new: native-compaction ledger event (handled tier) + immediate same-request state reset (ref re-seed + ref-floor pinning).

Both paths converge to the same healthy end state; the new path reaches it within ONE request instead of three drift passes, and the banner/ledger framing becomes accurate ("already rebased" instead of "cause not identified"). No change for any other client or lane.

Tests

New tests/issue4521-codex-openai-local-compact.test.ts mirrors tests/codex-local-compaction.test.ts on the chat-completions wire: positive e2e (rebase + ledger-kind pin + no double-record + post-compaction turn keeps working) plus three negatives (non-codex UA, decimated coverage without the template, template pasted with intact history). Verified the positive case FAILS on the unfixed source (repro) and passes with the fix. Existing suites green locally: codex-local-compaction.test.ts + codex-compact.test.ts (22/22); typecheck + build clean. Full-suite verdict: remote CI.

查重: 不重复 — nearest relative is closed #2372 (the responses-lane fix); this PR covers the same mechanism missing on the OpenAI wire lane.

@ranxianglei ranxianglei changed the title fix(#4521): codex local auto-compaction rebase on the OpenAI wire lane [PR] fix(#4521): codex local auto-compaction rebase on the OpenAI wire lane Oct 24, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-24_codex-openai-local-compact branch from d8384ff to 62df243 Compare October 24, 2026 13:58
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-24_codex-openai-local-compact (3797405)

Latest build — install this exact version

npm install -g billion-context@0.0.6178-pr.4542.d8f4825

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
3797405 billion-context@0.0.6178-pr.4542.d8f4825 build
479fa0a billion-context@0.0.6092-pr.4542.f1d598f build
8db7279 billion-context@0.0.5996-pr.4542.5b926ac build
5a48844 billion-context@0.0.5734-pr.4542.64ab380 build
62df243 billion-context@0.0.5510-pr.4542.fa4ceeb build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4542.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

冲突已解决,rebase 完成并推送。 d8384ff47 → 62df2433c(单 commit 变基到当前 master 0d9fdae96,diff 内容不变:2 files, +326/−4)。PR 已恢复可合并(mergeable=true),等 CI 全量套件在推送后的 head 上跑绿即可合并。

结构化八问

  1. 是什么问题:PR 分支基于旧 master(fe5ba5912)开出,master 此后前进了 447 个提交(含 [PR] kernel: fold coverage keyed on ledger seq intervals — churn-immune blocks (#4316 layer 3, PR 2/3) #4552/[PR] merge: tier-2 consolidation train — 15 PRs + repair of #4525's sidScope compile break #4551 等),其中多个提交改动了本 PR 同一片区域(src/server/prepare-openai.ts 的 import 区与 unannounced-rewrite 判定门),导致分支与 base 冲突、merge state 变 dirty。
  2. 什么原因导致的:纯文本重叠——master 侧在窗口内给 compress-tool import 加了 withMcpDialectNote、把 detectUnannouncedHistoryRewrite 升级为带 reconcileOutcome 第 4 参的调用并新增 [Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131 churnExplained 采纳分支;本 PR 恰好在同一 import 行插入 codex-compact import、并把 rewrite 门条件改成 !dshRebased && !codexRebased。两侧改动相邻,git 无法自动合并。
  3. 影响范围与严重程度:仅影响本 PR 自身的可合并状态;无用户面影响、无行为变化。严重度:流程阻塞级,非功能缺陷。
  4. 怎么修复的:rebase 单 commit 到当前 master,两处冲突按「双方意图取并集」解——① import 区:保留 master 的 withMcpDialectNote 版本 + 加入本 PR 的 codex-compact.js import;② rewrite 门:采用本 PR 的条件 !dshRebased && !codexRebased 与更新后的注释(同时点名 [bug] DSH 桌面端 0.1.188:宿主原生压缩落地摧毁折叠基底 → 同一区间永久不可锚定、compress 连片失败并熔断(compaction-basic auto:false 仍无效) #2432 与 [bug] Codex local auto-compaction can destroy existing bili fold substrate instead of rebasing #2372/Compression conflict banner (bili v0.1.205, node v24.19.0, win32-x64) #4521),函数体保留 master 的新签名(4 参调用含 reconcileOutcome)和 [Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131 churnExplained 采纳分支。语义核对:codex 命中 rebase 时整条通用 rewrite 路径(含 churn-adoption)被跳过,与 dsh 命中时的纪律完全一致;codex 检测块位置维持原设计(dsh 块之后、[Fold-state reconciliation: classify resent-history divergence (edit/fork/new/system-only) and make compression blocks re-anchorable chain units #1921] fold-reconcile 之前,dsh 优先权不变);检测仍量客户端原始重发历史(msgs),不受服务端渲染视图干扰。
  5. 是否有回归问题:审过的既有行为——dsh native-compaction 优先权(保持)、[Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131 跨构建 id churn 采纳(未受新条件影响,测试钉住)、模型切换后压缩块跨模型失配,新模型收到 61.3 万 token 全量上下文 #1001 通用 unannounced-rewrite 检测(行为不变,测试钉住)、Fold-state reconciliation: classify resent-history divergence (edit/fork/new/system-only) and make compression blocks re-anchorable chain units #1921 fold-reconcile 时序(保持)、Egress Judge false positive on Snapshot Render Lane I2: prefix comparison baseline needs to be aware of the render view | 出站判官对快照渲染车道误报 I2:前缀比对基准需要感知渲染视图 #4241 snapshot-render 相关改动(未触碰)。本地验证:typecheck 干净;定向三套测试 16/16 通过(Compression conflict banner (bili v0.1.205, node v24.19.0, win32-x64) #4521 正负例 ×4、fix-1001 ×7、issue4131 ×5)。全量回归交给远程 CI,不本地预跑。
  6. 是否破坏缓存:不涉及。纯合并操作,未触碰任何缓存/重试/游标/去重逻辑。
  7. 是否改了设计方向:不涉及。只解文本冲突,两侧的设计决策(本 PR 的检测+即时 rebase;master 的 [Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131/Egress Judge false positive on Snapshot Render Lane I2: prefix comparison baseline needs to be aware of the render view | 出站判官对快照渲染车道误报 I2:前缀比对基准需要感知渲染视图 #4241)均原样保留。
  8. 是否引入了新的配置:不涉及。无新 env/config/默认值。

收工判定:冲突闭环,分支已推,等 CI 绿灯后由人工合并即可。

@ranxianglei ranxianglei added bug Something isn't working codex p2 scope/wiring labels Oct 24, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #4542 审核完成。结论:代码与方案通过,等 rebase + 远程 CI 绿即可合。

当前状态:head 仍是原提交 d8384ff4;GitHub 对 master 标 dirty(开 PR 时落后 447 个提交),floor 2 通知显示 conflict-worker-N-r1 在处理 rebase,尚未推新 head,CI 也还没有结果。以下审核基于当前 head,本地已完整验证。

结构化八问:

  1. 是什么问题:codex 走 chat completions(wire_api="chat",或 providers[].protocol=openai 压过 URL 启发式,[feature] configurable wire-path table (per-provider custom paths) - Claude Desktop's claude.ai private protocol as motivating case #1909)时全程走 bili 的 OpenAI wire lane、从不经过 /v1/responses;而 codex 本地自动压缩检测器([bug] Codex local auto-compaction can destroy existing bili fold substrate instead of rebasing #2372)只存在于 responses lane,于是它的本地压缩落地后掉进通用 unannounced-rewrite 路径——ledger 记成 unannounced-rewrite(未确认档,banner 说「原因未明」)而不是 native-compaction,且不立即 rebase:半活 fold 块要等约 1 分钟由 [Bug][proxy] unframed host-side history deletion leaves zombie fold blocks rendering carriers forever — one carrier flaps in/out every turn (persistent prefix-cache churn) #2695 僵尸清扫才清掉,中间窗口里 compress 会失败「cannot be anchored」。
  2. 什么原因导致的:车道覆盖缺口——同一套检测机制没有移植到 OpenAI wire lane,不是检测逻辑本身有缺陷(Compression conflict banner (bili v0.1.205, node v24.19.0, win32-x64) #4521 现场证据:v0.1.205/win32 的 ledger + orphan-reap + openai-lane 专属日志行与此因果链吻合)。
  3. 影响范围与严重程度:仅影响骑 chat-completions wire、本地自动压缩落地时有活跃 fold 的 codex 会话;后果是约 1 分钟的死锚窗口(compress 失败)+ ledger/banner 误分类,僵尸清扫最终收敛到健康态,无数据丢失。p2(能用、有明显瑕疵、自愈)。
  4. 怎么修复的:把 [bug] Codex local auto-compaction can destroy existing bili fold substrate instead of rebasing #2372 的检测器+立即 rebase 原样移植进 src/server/prepare-openai.ts,门禁与 responses lane 完全一致(isCodexHost [Pi] Codex-compatible User-Agent activates Codex-only compaction policy despite explicit native host identity #3686 + 活跃块 + foldCoverage 减员 + 摘要模板前缀 carriesCodexLocalCompactionSummary + 阈值 missing>=8 && missing*2>=expected);命中即 markNativeCompactionBoundary+reconcileNativeCompactionBoundary 背靠背(同一请求内 rebase),并跳过通用 rewrite 路径防双记(模型切换后压缩块跨模型失配,新模型收到 61.3 万 token 全量上下文 #1001/[bug] DSH 桌面端 0.1.188:宿主原生压缩落地摧毁折叠基底 → 同一区间永久不可锚定、compress 连片失败并熔断(compaction-basic auto:false 仍无效) #2432 纪律)。块放在 dsh 块之后,双匹配 profile 保持 dsh 优先,与 responses lane 一致。
  5. 是否有回归问题:触碰路径的既有行为逐项清点过——①openai lane 的 dsh 压缩检测/rebase([bug] DSH 桌面端 0.1.188:宿主原生压缩落地摧毁折叠基底 → 同一区间永久不可锚定、compress 连片失败并熔断(compaction-basic auto:false 仍无效) #2432/bili-authored client compaction: let the host's own compaction flow run, but bili writes the checkpoint #3733):代码未动、仍先跑,新增 !dshRebased 条件只防重复触发,保持;②通用 unannounced-rewrite(模型切换后压缩块跨模型失配,新模型收到 61.3 万 token 全量上下文 #1001)+[Bug] #1001 归因缺口:客户端原生压缩落地后只剩「未确认档」——bili 已记录的 17 次拒绝(dshCompactionRefusals)与 10 次 DIRECT 旁路未被用于归因 #2709 见证归因+master 新进的 [Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131 id-churn-adoption:非 codex 客户端、以及 codex 无模板/无减员的场景行为不变(负测试钉住),仅 codex rebase 命中时整体跳过(有意变更,已在 PR 披露);③title-gen sidecar 由 !isTitleGen 排除(与同文件 dsh 块同口径);④responses/anthropic/google lane 文件未动;⑤ref 空间复用既有原语含 [Bug] #2436 的宿主本地压缩检测只接在 OpenAI 车道,Anthropic 车道缺失——未宣告宿主压缩后 nudge 广播的 compress ref 永久不可解析 #2592/F5 ref-floor pin(测试钉死 rebase 后恰好 4 条)。一个语义合并点需在 rebase 时处理:master 的 [Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131 churnExplained 分支正好在本 PR 改动的通用块内部,需保留 [Bug][opencode lane] Cross-version restart replays the whole session: old v0.1.197 and new v0.1.202 write the same sessions dir, 212 folded refs re-anchor onto churned bytes and the ref map is pruned to 392 raw ids #4131 逻辑在新 !dshRebased && !codexRebased 门内(intent 取并集);drift diff 逐段核过,其余组合干净。
  6. 是否破坏缓存:未命中会话零影响;命中时 rebase 本身就是状态重置(ref 重种 + ref-floor pin),语义与 responses lane 既有行为相同,不引入新的缓存失效面。
  7. 是否改了设计方向:无——既有机制的车道对等补齐,无架构转向。
  8. 是否引入新配置:无。

本地验证(定向,全量留给远程 CI):

  • 新套件 tests/issue4521-codex-openai-local-compact.test.ts:4/4 过(正 e2e + 3 负例:非 codex UA / 有减员无模板 / 模板贴入但历史完好);
  • 复现独立确认:把 src/server/prepare-openai.ts 单独还原到修复前,正 e2e 失败、3 个负例仍过——负例不是恒真断言;
  • 姊妹套件 codex-local-compaction.test.ts + codex-compact.test.ts:22/22;typecheck + build 干净;
  • 测试文件与 responses-lane twin 逐段对照:消息形状/session/UA/断言全对齐,仅 SSE 编码不同(responses events vs chat chunks),移植忠实;断言有牙(rebase 后 byRaw 恰好 4 条、追加一轮变 5 条、ledger kind 钉死、上游转发字节核对);
  • reconcileNativeCompactionBoundary 返回值语义核实:仅在 boundary 非 pending 时返回 false,本流程 mark 紧跟 reconcile 必为 true → 双记录在该路径下不可能发生。

判定:代码质量、门禁双向性、无双记、命名风格均达标,无需返工;conflict-worker 的 rebase 落地且远程 CI 绿后即可合并。我不碰 merge/close。

标签: bug / codex / scope/wiring / p2(OpenAI wire lane 检测缺口,~1 分钟自愈、无数据丢失 → p2;pr 标签沿用)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

接力追平:分支此前 behind master,已 merge origin/master(无冲突)并推送(5a4884446)。合并后本地定向跑过本 PR 的回归测试 tests/issue4521-codex-openai-local-compact.test.ts 4/4 绿;全量以 CI 为准。

背景:家族报告在持续新增——#4648(10-25 早)、#4688(10-25 午,v0.1.207 = 当前最新,纯 orphan-reap、clients 未解析)。本 PR 是「codex 走 OpenAI 车道时本地自动压缩不被识别」的根因修复,只要它不发版,这类事件就会一直落到 unannounced-rewrite/orphan-reap。与 #4650(报告证据覆盖)一起,这两个 PR 是收敛本家族的在途工作,都在等合并拍板。

ci-check added 2 commits October 26, 2026 15:22
master window commits 293ae65/de5e03659 bumped kernel/package.json to
0.0.165 without regenerating the lockfiles; lock-sync re-resolution then
rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and
fails. Pure recorded-version sync via npm install --package-lock-only;
no dependency or semantic change.
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1042-master.b947a03 (rolling master tag).

npm install -g billion-context@0.0.1042-master.b947a03  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit to mariopasquali/billion-context that referenced this pull request Nov 5, 2026
…anxianglei#4542/etc (conflict resolution for ranxianglei#5743)

# Conflicts:
#	tests/opencode-rolling-cache-marks.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codex p2 scope/wiring

Projects

None yet

1 participant