Skip to content

[PR] fix: judge codex compaction gate against the request's own view - #4040

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-21_codex-compact-gate-view-aware
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-21_codex-compact-gate-view-aware

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4036

What

The Codex compaction gate (codexCompactGatePre) decided whether to intercept a compaction_trigger from the session's stored block state — but that state was last written by the previous request's view. Field sequence from #4036: a short side request (2 msgs) ran syncBlocks on a view carrying no fold carriers → all blocks retained but set active=false → ~12s later a trigger carrying the full 3580-message history arrived; its gatePre was computed before any prepare/sync could re-activate the blocks → gate false → the 18.5 MiB trigger forwarded verbatim → upstream 400 Unsupported parameter: generate.

Fix (decision-point judgment, not deactivation isolation)

codexCompactGatePre now takes an optional third argument — the trigger request's own message view (projected to core messages). When present, fold activity is judged on a pure probe: syncBlocks(viewMessages, session.state) (kernel function, already public; it deep-clones its input state and mutates nothing) instead of the stored flags. When absent (unparseable view) it falls back to the legacy stored-flag check — byte-for-byte old behavior.

Deliberately NOT done (per the issue's warning): runPrepare() is still not run before the gate — the non-intercept path stays byte-identical passthrough with zero session mutation (#335). The probe is read-only.

Behavior changes (old → new)

  1. Gate-false triggers whose own view carries the fold (stored-inactive after a preceding short view) are now intercepted instead of forwarded verbatim — the intended fix.
  2. Stale-active triggers whose own view does NOT carry the fold are now forwarded verbatim with zero state mutation. Previously they fell through after running prepare (state mutated, then forwarded raw anyway) — passthrough outcome unchanged, side effect removed (aligns the non-intercept path with fix: detect codex compaction requests before preflight; verbatim passthrough when not intercepted (#332) #335's no-state-mutation constraint).
  3. The limit>0 wall and the 90%-baseline wall are untouched; the view only resolves block activity.
  4. Additive observability fields on the existing not-intercepted log line only.

No config surface change. No kernel change. No wire-shape change (intercepted requests were forged identically before; passthrough bytes unchanged).

Tests

  • tests/codex-compact.test.ts — new 4-row matrix on codexCompactGatePre: stored-inactive + carrier in view → forge; stored-active + carrier absent → honest passthrough; no view → legacy judgment; view does not loosen the 90% wall.
  • tests/codex-compact-e2e.test.ts — two pipeline tests implementing the issue's recommended real-pipeline regression exactly: (a) seed compressed main session → short side request on the same session id (asserts blocks retained but all inactive) → immediate full-history trigger → asserts forged SSE, zero upstream calls, blocks re-activated; (b) negative: trigger whose own view carries no fold is forwarded byte-identical AND leaves session state JSON-identical (no prepare ran). Both RED on pre-fix master (verified), green post-fix.
  • Targeted local runs green: codex-compact unit + e2e, native-compact-terminal, compaction-trigger-finality, preflight-codex-headroom, proxy-noinject-tag-strip, issue1542-chain-embedded-strip, codex-local-compaction, issue3398-opaque-media-reserve, chain-checkpoint-step2/3; typecheck clean. Full-suite verdict = CI on this head.

Out of scope (noted in #4036 thread)

Why the 2-message side request entered the MAIN session at all is the #398 routing-isolation domain — separate mechanism, separate fix.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 本 PR 是 #4036 的修复交付(Closes #4036),与 #3844/#3845(zombie reap)、#398(短请求路由隔离)、#335(gate-before-prepare 约束)、#3008/#3018(estimate-grade 幻影基线)机制均不同——不重复。PR 自身无重复在途条目。

标签: pr / bug / codex / scope/wiring / p2 —— codex 宿主车道 compaction gate 误判;能用,特定时序下超大原始请求被静默直转上游。

这是上一会话处理 #4036 时开的 PR,按规范我不重做工作,而是做独立审核:拉分支通读全部 diff → 触碰路径的既有行为逐项回归审计 → 测试有效性(断言是否有牙、回归时抓得住吗)→ CI 状态核查。结论按结构化八问回复。

@ranxianglei ranxianglei added bug Something isn't working codex p2 scope/wiring labels Oct 20, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-21_codex-compact-gate-view-aware branch from 0f425b6 to 8217164 Compare October 20, 2026 18:55
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-21_codex-compact-gate-view-aware (814a2cd)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4861-pr.4040.0347013

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4040.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #4040 审核结论(独立重审 + rebase 冲突解决 + CI 事件处置)

八问

  1. 是什么问题:codex 的 compaction 请求(trigger 形式与 /responses/compact 端点形式)过闸时,判断「有没有可折叠块」用的是上一次请求留下的存储态 active 标志;一旦上一个请求是短视图(如 side 请求),块被合法停用,紧跟其后的 compaction 请求就被误判为不满足前置条件、原样转发,上游直接 400([Suspected][Codex] After the short view deactivates blocks, the immediately following compaction trigger may be erroneou | [Suspected][Codex] 短视图失活 blocks 后,紧接着的 compaction trigger 可能被 gate 误放行 #4036 现场:Unsupported parameter: generate)。
  2. 什么原因导致的:判定层读的是陈旧证据——存储标志由前一个请求的 syncBlocks 写入,而 compaction 请求自带完整历史,才是权威视图。根因在判定点选错了证据源,不是门控阈值或时序问题。
  3. 影响范围与严重程度:仅 codex 宿主 + intercept 模式 + 「短视图请求 → 立即 compaction」这一精确序列触发;触发后果是该次 compaction 失败(上游 400),会话本身可用,后续正常请求会重新激活块。p2。
  4. 怎么修复的:判定点改为读取本请求自己的视图——用 kernel 导出的纯函数 syncBlocks(viewMessages, session.state) 做零副作用探测(深拷贝 state,不改会话),视图不可解析时回退旧存储标志语义;两种形态(handle.ts trigger / prepare-responses.ts endpoint)同修;runPrepare() 仍在门后执行,fix: detect codex compaction requests before preflight; verbatim passthrough when not intercepted (#332) #335 的「未拦截 = 字节一致透传 + 零状态变更」保持不动。
  5. 是否有回归问题:逐项审过既有行为——① limit>0 墙与 90% baseline 墙原样且先于视图逻辑求值(单测钉住「视图不能放宽墙」);② 未拦截路径字节一致 + 会话状态 JSON 级不变(e2e 负例双断言);③ 分发顺序不变(fix: detect codex compaction requests before preflight; verbatim passthrough when not intercepted (#332) #335);④ master 侧 [Pi] Codex-compatible User-Agent activates Codex-only compaction policy despite explicit native host identity #3686 的 isCodexHost 宿主策略语义在冲突解决中完整保留(endpoint 门与 trigger 谓词都用它,字段事故形态经 UA 匹配仍覆盖);⑤ 日志行仅追加观测字段。有意行为变更两处(均已在 PR 描述披露):拦截集合增加「存储态 inactive + 视图含载体」(即本修复),同时剔除「存储态 active + 视图无折叠物」——后者从「先改状态再原样转发」变为「零状态变更直接转发」(对外结果相同,副作用被移除)。
  6. 是否破坏缓存:不涉及。缓存/重试/去重/增量游标均未触碰;透传路径字节不变,前缀缓存行为不受影响。
  7. 是否改了设计方向:不涉及。仍是同一机制族,只是判定证据从存储态移到请求自身视图(疏而非堵:修的是判定点,不是去隔离停用逻辑)。
  8. 是否引入了新的配置:不涉及。无新 env/config/默认值。

过程事实

判定:无阻断发现,主 CI 车道转绿即可合并。标签:pr/bug/codex/scope/wiring/p2(首条回复已打)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 跟踪更新(head b436fd002):11 条 e2e 车道全部 success;主 CI 工作流(多 shard 全量测试)仍在跑,持续观察中。此前唯一红项 lane-guard 已确认为注释措辞误报(#2170 符号表命中 "side request" 散文短语),最终 head 本地验证不再命中;守卫机制本身的缺陷已立项 #4047。

结论不变:无阻断发现,主 CI 车道转绿即可合并。若任一 shard 变红我会定向排查并跟进。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

主 CI 红项定性完成:全部属于已知 master flake 家族,非本 PR 引入。

八问(本轮 CI triage)

  1. 是什么问题:head b436fd002 的主 CI 6 个 test shard 红了 5 个,阻塞转绿。
  2. 什么原因导致的:逐个 job 拉日志核对失败签名——挂的是 issue3769-upgrade-compat('record hydrates before the upgrade turn')、issue3769-pooled-bare-load([Bug] MCP failed compression record rotation prematurely rewrites the history prefix, triggering expensive cache recompu | [Bug] MCP 失败压缩记录轮换提前改写历史前缀,触发大额缓存重算 #3769 flake)、opencode-fork-adopt 两条(dsh 扩展接入 fork 协议(stage 1/#2394 L0):fork 子会话首请求前 snapshot→match→fork 收养 #2399 stage 3)+ 一个 ubuntu-24 shard 日志中途截断(疑似 OOM/超时被杀)。全是 storage hydrate/fork 继承域的时序敏感测试,与本 PR 的 codex compaction gate 零交集。
  3. 影响范围与严重程度:对本 PR = 门禁被卡;对仓库 = master 自身也在同签名上红(master run 同批 e2e-dsh-plugin-lane 也失败过),属全局 CI 风暴而非本分支问题。
  4. 怎么修复的:本 PR 无需修——根因修复已在途(PR [PR] fix: stop the three CI-shard flake families (#3769 pair + dsh attach-mode) | 修掉三个 CI 分片抖动家族 #4058「修掉三个 CI 分片抖动家族」、PR [PR] fix: hydrate pooled session envelopes from disk when the unified sidecar fails to open (#4041) #4067「unified sidecar 打不开时从盘上 hydrate」)。我的处置:把本次 5 个 shard 的完整失败签名表 + 两个新线索(同一测试第二个失败断言点、fork-adopt 疑似第三成员)补充到跟踪 issue Master CI has been red: two regression tests from #3769 keep failing across multiple shards, blocking all PRs from going | master CI 持续红:#3769 的两条回归测试在多 shard 反复失败,阻塞所有 PR 转绿 #4063。
  5. 是否有回归问题:无——反证链完整:① 失败测试与本 diff 触碰路径无交集;② 三个失败文件在 PR head 与干净 master bc4a3f87e 上本地单独跑均 4/4 通过;③ master 自身 CI 同签名红(Master CI has been red: two regression tests from #3769 keep failing across multiple shards, blocking all PRs from going | master CI 持续红:#3769 的两条回归测试在多 shard 反复失败,阻塞所有 PR 转绿 #4063 有 8-shard 记录)。
  6. 是否破坏缓存:不涉及(本 PR 不触碰缓存路径;红项是既有 flake 家族)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

查重声明:CI 红项未新开 issue——已存在 #4063/#4049/#4051/#4039/#4041 一族跟踪条目与 #4058/#4067 两个在途修复 PR,按 master 红风暴防线只在 #4063 下补证据,不重复立项、也不并行开修。

判定更新:代码面结论不变(可合并);绿 CI 取决于 #4058 或 #4067 落地。不建议现在逐轮 rerun(#4061 先例:rerun 后同签名再红,纯耗算力)——flake 修复合入后重跑一次即可。若 owner 想现在就盖绿章,手动 re-run failed jobs 一次也可,我这边不再自动触发。

@ranxianglei
ranxianglei force-pushed the 2026-10-21_codex-compact-gate-view-aware branch from b436fd0 to 814a2cd Compare October 21, 2026 02:12
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 处置结论(head b436fd002 → 已 rebase 至 814a2cdbd)

先说结论:这轮 CI 红不是本 PR 代码的问题。5 个失败 shard 挂的全是仓库里既有的、与本 PR 无关的抖动测试(#3769 存储模式 + #2399 v2 fork),而它们已经在 master 上被修好了(602996bc2,来自 #4077)。本 PR 分支落后 master 17 个提交、没带上那个修复,CI 一跑就踩中。处置=把分支 rebase 到当前 master,继承那个已合入的修复。

八问

  1. 是什么问题:PR 的 CI 挂了,但挂的不是它自己改的东西——是同一批 shard 里顺带跑的、别的子系统的老测试在抖([Bug] MCP failed compression record rotation prematurely rewrites the history prefix, triggering expensive cache recompu | [Bug] MCP 失败压缩记录轮换提前改写历史前缀,触发大额缓存重算 #3769 pooled-bare-load / upgrade-compat、dsh 扩展接入 fork 协议(stage 1/#2394 L0):fork 子会话首请求前 snapshot→match→fork 收养 #2399/[Bug] opencode system-context reconcile:每次 AGENTS.md 编辑使 Responses 会话 key 里的 instructions 指纹漂移,压缩状态被重置(master 现存) #1102 fork adopt)。
  2. 什么原因导致的:根因不在本 PR。这些抖动测试的确定性修复(给两个 [Bug] MCP failed compression record rotation prematurely rewrites the history prefix, triggering expensive cache recompu | [Bug] MCP 失败压缩记录轮换提前改写历史前缀,触发大额缓存重算 #3769 suite 钉死 BILI_STORAGE_UNIFIED=full、让 plugin-fork 冷恢复真正冷)已在 master 落地(602996bc2)。本 PR 基于更老的 commit(bc4a3f87e)没带这个修复;CI runner 的旧存储基线 preload 把这两个「按设计走 full 模式」的 suite 翻成 legacy 车道,loadSync 拿不到记录 → 断言约 3s 后失败。纯版本滞后,不是设计缺口——本 PR 的 codex-compact 改动与这些测试零交集。
  3. 影响范围与严重程度:只挡了本 PR 的合并门禁,无用户面影响;任何 base 早于该修复的分支都会同样中招。属 CI 门禁问题,非产品缺陷。
  4. 怎么修复的:rebase 到当前 master(b436fd002 → 814a2cdbd,父提交 = 56a0f5a46),继承已合入的抖动修复。没动本 PR 的任何代码——17 个提交的差距不碰本 PR 的 5 个文件,所以是干净 rebase、零冲突;用 lease 钉死旧 head 后 force 更新 PR 分支。
  5. 是否有回归问题:无。rebase 只在顶上叠了 master 的提交,本 PR 的 5 文件 diff 字节不变(仍 +144/-8)。本地验证:typecheck 0 错、build 通过;CI 里红的每一个测试在新 head 上都本地跑绿(codex-compact 的 [Suspected][Codex] After the short view deactivates blocks, the immediately following compaction trigger may be erroneou | [Suspected][Codex] 短视图失活 blocks 后,紧接着的 compaction trigger 可能被 gate 误放行 #4036 view-aware gate、[Bug] MCP failed compression record rotation prematurely rewrites the history prefix, triggering expensive cache recompu | [Bug] MCP 失败压缩记录轮换提前改写历史前缀,触发大额缓存重算 #3769 两个、plugin-fork、opencode-fork-adopt 的 dsh 扩展接入 fork 协议(stage 1/#2394 L0):fork 子会话首请求前 snapshot→match→fork 收养 #2399/[Bug] opencode system-context reconcile:每次 AGENTS.md 编辑使 Responses 会话 key 里的 instructions 指纹漂移,压缩状态被重置(master 现存) #1102)。master 自身在 56a0f5a46 上是绿的,而新树 = master + 孤立的 codex 改动。
  6. 是否破坏缓存:不涉及。未触碰缓存/重试/去重/增量游标逻辑;改动只是换 base + 继承测试侧修复。
  7. 是否改了设计方向:不涉及。架构/边界/数据流都没转向,仍是原机制族。
  8. 是否引入了新的配置:不涉及。无新增对外 env/config/默认值(继承的修复只是在两个测试文件内部钉了一个进程内 env,不进发布配置面)。

判定:无阻断发现。分支已对齐当前 master,之前红 shard 的根因(存储模式抖动)由继承的 master 修复解决;待新 head 814a2cdbd 的 CI 转绿即可合并。此为自动修复通道第 1 次处置(上限 2 次),未触及需要决策的设计取舍。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codex p2 scope/wiring

Projects

None yet

1 participant