Skip to content

[PR] fix(kernel): fold plain-text developer items carrying Codex metadata fields (#6949) - #6952

Open
ranxianglei wants to merge 8 commits into
masterfrom
2026-11-16_codex-developer-metadata-foldable
Open

ranxianglei wants to merge 8 commits into
masterfrom
2026-11-16_codex-developer-metadata-foldable

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #6949

Problem

Codex Desktop stamps mid-history developer items with top-level id + internal_chat_message_metadata_passthrough (per-message bookkeeping: turn_id, content_item_kinds, provenance). The kernel's plain-text check isPlainTextResponsesSystemItem (kernel/src/protected.ts) only admitted the top-level keys type/role/content, so those plain-text items were classified "unrecoverable" -> assignRefs minted BLOCKED_REF -> the messages were permanently un-foldable, and their BLOCKED rows also poisoned the acp_status uncompressed range view (mNNNNN–BLOCKED ⚠ not foldable: refs unresolvable).

Reproduced on master before the fix: with the two fields -> unrecoverable=true; without -> false (matches the issue's "0 of 30 raw pass, 30 of 30 pass after stripping the fields").

Fix

kernel/src/protected.ts: admit two known-harmless top-level keys in isPlainTextResponsesSystemItem, with strict shape checks:

  • id — must be a string (standard Responses item id; the plain-text rewrite path in kernel/src/wire/responses.ts already re-emits plain-text items without it);
  • internal_chat_message_metadata_passthrough — must be a plain object (Codex per-message bookkeeping; the model never reads it and it dies with the message on fold).

Any OTHER unknown top-level key still pins (the conservative #4166 fallback is unchanged). Content-parts check, the verbatim re-emit path, and the report layer are untouched. kernel/package.json 0.0.198 -> 0.0.199 (CI-enforced version discipline).

Behavior change (disclosed)

Plain-text developer/system items carrying these two fields: permanently un-foldable (BLOCKED) -> foldable (numeric ref). Why: the fields carry no model-visible payload and the kernel's own plain-text rewrite path already drops them. Previously-BLOCKED sessions migrate to numeric refs on the next ref-assignment pass via the existing #2607/#3945 migration path (fresh number minted, carrier re-render deferred to the next fold boundary — no mid-turn cached-prefix rewrite).

kernel-change: #6949

isPlainTextResponsesSystemItem over-rejected plain-text developer items carrying Codex's id + metadata passthrough (provably-false guard: the kernel's own plain-text rewrite path already treats these fields as disposable for plain-text items). Filed under the 2026-11-06 mechanical-bug exemption (no separate pre-approval issue); the human-only review tier still applies — please review.

Tests

  • New: kernel/tests/issue6949-codex-developer-metadata.test.ts — reproduces the bug (with fields: was BLOCKED, now foldable), negative gates (other unknown key / wrong value shapes / structured content still pin), byte-exact verbatim re-emit INCLUDING the fields, and numeric ref minting via assignRefs.
  • Kernel unit suite: 1648 pass, 0 fail (local).
  • Targeted bili side: wire-contract + responses ordering + round-trip: 66 pass.
  • typecheck + build green. Full-suite verdict: CI on the pushed head.

Related

Codex Desktop stamps mid-history developer items with top-level id +
internal_chat_message_metadata_passthrough (per-message bookkeeping:
turn_id, content_item_kinds, provenance). The #4166 plain-text check
only admitted type/role/content, so those plain-text items were
classified unrecoverable, got a BLOCKED ref and were permanently
un-foldable; their BLOCKED rows also poisoned the acp_status range
view (billion-context #6949).

Admit the two known-harmless keys with strict shape checks (id must be
a string, the passthrough must be a plain object); any other unknown
top-level key still pins. The plain-text rewrite path
(wire/responses.ts) already re-emits plain-text items without these
fields, so the kernel already treated them as disposable for
plain-text items.

kernel-change: #6949 — isPlainTextResponsesSystemItem over-rejected
plain-text developer items carrying Codex id + metadata passthrough;
admit the two known-harmless keys, regression test
kernel/tests/issue6949-codex-developer-metadata.test.ts.
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-16_codex-developer-metadata-foldable (ca47a5d)

Latest build — install this exact version

npm install -g billion-context@0.0.11013-pr.6952.07e2a87

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
ca47a5d billion-context@0.0.11013-pr.6952.07e2a87 build
8345b28 billion-context@0.0.10653-pr.6952.eba478d build
053b5a6 billion-context@0.0.10544-pr.6952.f8abc0d build
2c6e5a5 billion-context@0.0.10515-pr.6952.3ed4c50 build
5c69c8a billion-context@0.0.10448-pr.6952.2e9f56a build
fd0e5a6 billion-context@0.0.10445-pr.6952.b6efd4c build
7606a19 billion-context@0.0.10426-pr.6952.8bab709 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6952.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei added bug Something isn't working codex p2 scope/kernel labels Nov 16, 2026
@ranxianglei ranxianglei changed the title fix(kernel): fold plain-text developer items carrying Codex metadata fields (#6949) [PR] fix(kernel): fold plain-text developer items carrying Codex metadata fields (#6949) Nov 16, 2026
ci-check added 2 commits November 16, 2026 23:07
The 0.0.199 bump in kernel/package.json (commit 7606a19) left the
kernel lockfile at 0.0.198, so the lock-sync lane (npm install
--package-lock-only + git diff --exit-code) fails on every run of this
head and — because the test matrix job needs lock-sync — the full test
suite never runs on this PR.

Small in-pass fix found during PR review: two version fields, no other
content. Verified idempotent: kernel/ npm install --package-lock-only
makes no further change.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — no other open [PR] row targets this thread; this is the only PR fixing #6949.

Review verdict: positive — code correct, claims verified, tests real

Code review (independent, all verified in-tree):

Defect found & fixed in review (small in-pass fix, labeled commit):

  • lock-sync red (PR-caused, not inherited): the PR bumped kernel/package.json to 0.0.199 but left kernel/package-lock.json at 0.0.198 → the lock-sync lane failed on EVERY run of the original head, and — because the test-matrix job needs lock-sync — the full test suite never ran on this head. Fixed in commit 6356ee8dfb (two version fields; verified idempotent: npm install --package-lock-only in kernel/ makes no further change).

CI state (verified live on original head 7606a19):

结构化八问

  1. 是什么问题:Codex Desktop 给 mid-history 的 developer item 盖顶层 id + internal_chat_message_metadata_passthrough 两个字段;内核 plain-text 检查只放行 type/role/content,这些纯文本 item 被判 "unrecoverable" → BLOCKED_REF → 永久不可折叠,且 BLOCKED 行污染 acp_status 未压缩区间视图。
  2. 什么原因导致的:isPlainTextResponsesSystemItem 的 key 白名单过严——一个可证伪的 guard:内核自己的 plain-text 重写路径(kernel/src/wire/responses.ts:948-955)本来就丢弃这两个字段(视为 plain-text item 上可丢弃),按它们 pin 与内核自身的重建行为自相矛盾。
  3. 影响范围与严重程度:Codex(responses wire)会话中携带这两个字段的 mid-history developer item——受影响消息永久不可折叠(上下文无法压缩、只增不减),acp_status 视图出现 BLOCKED 行。能用但核心承诺失灵,p2 级(非数据丢失、非会话损坏)。
  4. 怎么修复的:白名单放行两个已证明无害的 key,各带严格形状检查(id 必须 string;metadata 必须 plain object);其他任何未知 key 仍 pin(保守回退不变);verbatim 重放逐字保留字段;原 BLOCKED 会话经既有 Messages with duplicate IDs overwrite the rendered bytes of already-sent messages: protected messages are re-tagged by l | 重复 id 的消息会改写已发送消息的渲染字节:受保护消息被后来的同名 id 消息重新打标 #3945 迁移路径在下次 ref 分配时换新数字 ref(不中途重写已缓存前缀)。
  5. 是否有回归问题:未发现。已审:唯一调用点、负向门禁(其他未知 key / 错误值形状 / 结构化内容仍 pin)、含字段的逐字节重放、anthropic/openai 镜像检查未动、内核身份/去重不受影响(两字段不进 canonical id)。lock-sync 红是 PR 自身缺陷(lockfile 未随版本号同步),已在 6356ee8dfb 修复。
  6. 是否破坏缓存:否。只改可折叠性分类;缓存键、重试、增量游标、去重键均未动。原 BLOCKED 会话在下次 ref 分配时获得新数字 ref,载体重渲染推迟到下一个 fold 边界。
  7. 是否改了设计方向:否。仍是保守的 pin-by-default 设计,只把两个已证明无害的 key 加入白名单。
  8. 是否引入了新的配置:否。

判定:代码可合并(kernel 变更,人工评审档仍适用)。CI:新 head fd0e5a643a 的运行应交付 architecture-docs 绿 + lock-sync 绿 + 真正的全量测试结论;唯一预期残留红是继承的 issue2929 born-red(待 #4599 裁决,哨兵在 ci-fixes 收敛)。

Second half of the lock-sync fix (6356ee8 covered kernel/package-lock.json):
the root lockfile records the file: dependency's version under the 'kernel'
folder key, which the 0.0.199 bump in kernel/package.json left stale. Same
lock-sync lane, same root cause (version bump without lockfile sync).

Verified: root npm install --package-lock-only produces exactly this one-line
change and nothing else; kernel/ side already idempotent.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Follow-up — first re-run exposed a second lockfile drift; fixed and pushed (new head 5c69c8ac24).

First re-run on fd0e5a643a:

  • architecture-docs: green ✅ (the ci-fixes merge fixed the inherited red)
  • lock-sync: still red ❌ — the root package-lock.json ALSO records the file: dependency's version under the kernel folder key, and the 0.0.199 bump left that stale too (my first fix 6356ee8dfb only covered kernel/package-lock.json).
  • Test matrix: skipped again (needs lock-sync).

Fix: 5c69c8ac24 — one line in the root lockfile (kernel entry 0.0.198 → 0.0.199). Verified both lock-sync checks idempotent locally (root + kernel: npm install --package-lock-only then git diff --exit-code clean).

The new run on 5c69c8ac24 should finally deliver lock-sync green + the test matrix actually running → the full-suite verdict. Expected residual red unchanged: the issue2929 born-red (inherited from master #6928, pending the #4599 decision).

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Final CI verdict (head 5c69c8ac24): the full suite ran; the only red is the known inherited issue2929 born-red.

Final verdict: merge-ready. The code fix is correct and now fully verified by a real full-suite run on the rebased head; the only residual red is inherited and outside this PR's scope. The human-only review tier applies (kernel change).

ework-agent and others added 2 commits November 17, 2026 06:35
…olving

closeLogger resolved when the FINAL stream ended, but rotated-out streams are
end()-ed asynchronously and their flush can still be in flight. On a loaded
Windows runner a slow replaced flush let the shutdown read a sealed generation
before its tail was written, losing a rotated record (logger-rotate 'successive
rotations retain distinct payloads' red on windows-22 shard 2, PR #6952).

Track the in-flight replaced streams with a drain-waiter list: settle() already
decrements pendingRetentionDrains and now notifies waiters at zero, and
closeLogger waits for BOTH the final stream and pendingRetentionDrains === 0,
honoring its documented contract ('resolves once every buffered line has
drained to disk'). 'close' always fires, so the counter always settles — no
new hang path.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI re-verification + fix (this run — head now 053b5a66b7).

Re-verified the live state first: the 3 failing shards the event reported (ubuntu-24-3, ubuntu-22-3, windows-22-4) were ALL the inherited #2929 fold-probe test — already converged in ci-fixes (7247410976), which a previous wake merged into this branch. Those are resolved.

The post-merge head (2c6e5a5b5f) had ONE remaining red: test (windows-latest, 22, 2) = logger-rotate ("successive rotations retain distinct payloads without overwriting"). Root-caused it: not caused by this PR (the diff does not touch the logger) — it is a real race in closeLogger (src/logger.ts): it resolved when the FINAL stream ended, but rotated-out streams are end()-ed asynchronously and their flush can still be in flight. On a loaded Windows runner a slow replaced flush let the shutdown read a sealed generation before its tail was written, losing one rotated record (rot-1). Local repro: 60/60 pass on Linux — a rare, load-dependent race.

Fix (on the PR branch, labeled commit 053b5a66b7): closeLogger now waits for BOTH the final stream AND pendingRetentionDrains === 0 (new drainWaiters list; settle() notifies waiters at zero). This honors its documented contract ("resolves once every buffered line has drained to disk"). 'close' always fires, so the counter always settles — no new hang path. Verified locally: typecheck + build green; logger suite green. (The one local log-retention failure is a pre-existing UTC+8 timezone artifact — not in the CI failure set, unrelated to this change.)

Note: the logger fix is technically outside this PR's primary scope (the kernel fold fix) but was required to clear the red CI; I flagged it on #4599. The new head 053b5a66b7 triggers the branch's one full CI pass (running now). The kernel-change: #6949 declaration is present in the PR body, so the kernel-guard is satisfied; the human-only review tier applies (kernel change).

结构化八问

  1. 是什么问题:PR 的 CI 红。事件报的 3 个失败分片全是继承的 codex 命中率问题分析 #2929 fold-probe;merge ci-fixes 后新 head 还剩 1 个红 = logger-rotate(windows 22/2)。
  2. 什么原因导致的:codex 命中率问题分析 #2929 是继承红(probe 硬编码请求下标锚点,[PR] feat: pi lane — show the injected nudge text verbatim in the terminal when debug is on #6151 落 master 后 nudge 调度漂移;ci-fixes 7247410976 已修)。logger-rotate 是 closeLogger 的真实竞态:只在最终流 end 时 resolve,不等已轮换流的异步 drain;Windows 负载下慢 flush 让测试在 rot-1 落盘前读了 sealed 文件,丢 1 条记录。
  3. 影响范围与严重程度:codex 命中率问题分析 #2929 继承红 —— 所有未 merge ci-fixes 的 PR 都红(已在 ci-fixes 收敛)。logger-rotate 竞态 —— 罕见(1/11 分片、单条记录、Windows 负载特定),不影响生产(仅 shutdown 时可能丢最后几条日志),但违反 closeLogger 文档契约。
  4. 怎么修复的:codex 命中率问题分析 #2929 走 ci-fixes(已 merge)。logger-rotate 在 PR 分支修:closeLogger 现在同时等最终流 + pendingRetentionDrains===0(新增 drainWaiters,settle 归零即通知),兑现「resolves once every buffered line has drained」契约;'close' 必触发,无新挂起路径。
  5. 是否有回归问题:审了 logger 既有行为 —— rotation/seal/retention 逻辑未动,只改了 closeLogger 的 resolve 时机(更保守:多等已轮换流 drain)。本地 logger 套件绿。无回归。
  6. 是否破坏缓存:不涉及(纯日志关闭时机,无缓存/重试/游标/去重键)。
  7. 是否改了设计方向:不涉及(未改架构/边界/数据流;只是让 closeLogger 兑现其既有文档契约)。
  8. 是否引入了新的配置:不涉及(无新 env/config/默认值)。

判定:红 CI 已修 —— 3 个继承 #2929 经 ci-fixes 收敛,1 个 logger-rotate 竞态在 PR 分支修(commit 053b5a66b7)。新 head 的全量 CI 运行中;kernel-change 声明已在 PR 正文,kernel-guard 满足,人工评审档适用(kernel 变更)。待新 CI 绿 + 人工评审后可合。

ci-check added 2 commits November 17, 2026 22:45
kernel-change: #6949 — acceptance follow-up requested in the issue
thread (field audit, Codex CLI 0.169.0): pin the three observed
plain-text classes (host_skills.instructions / tools.deferred_namespaces
/ generic.turn_aborted) with valid id + metadata object, and pin the
stale-BLOCKED migration so pre-existing mislabeled sessions recover on
re-assignment without a new session.
…sion

Master advanced kernel 0.0.198->0.0.199->0.0.200 (its own changes: #6709, compress-receipt, #7252); the PR independently bumped 0.0.198->0.0.199 for the #6949 protected.ts fix. The merged kernel tree carries BOTH, so the version must move above master's 0.0.200 -> 0.0.201 (required by kernel-guard rule 2: SRC_CHANGED via protected.ts implies BUMPED). Lockfiles regenerated idempotent (root + kernel).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved + pushed (new head ca47a5dca81).

The PR was in a dirty merge state because master advanced 135 commits past the branch's base (merge-base 7247410976, kernel 0.0.198) — including master's own kernel bumps 0.0.198→0.0.199→0.0.200, which collided with the PR's independent 0.0.198→0.0.199. The only textual conflicts were the three version/lockfile files; all code merged cleanly (master never touched the PR's kernel/src/protected.ts).

What I did this run:

  1. Integrated the remote head (fast-forwarded local to 8345b28863d), then merged origin/master into the PR branch.
  2. Resolved the version collision: the merged kernel tree carries BOTH master's changes (→0.0.200) and the PR's Codex message metadata causes BLOCKED endpoints to appear in the uncompressed range. | Codex 的消息元数据导致 BLOCKED 端点出现在未压缩范围中 #6949 protected.ts fix, so the version must move above master's 0.0.200 → 0.0.201. This is required by kernel-guard rule 2 (the PR touches kernel/src/protected.ts ⇒ SRC_CHANGED, so the version field must differ from master ⇒ BUMPED; resolving to 0.0.200 would fail the guard).
  3. Regenerated both lockfiles (package-lock.json + kernel/package-lock.json) to 0.0.201 — verified idempotent (npm install --package-lock-only produces no diff in either, so the lock-sync lane passes).
  4. Verified locally: typecheck clean, full build green, Codex message metadata causes BLOCKED endpoints to appear in the uncompressed range. | Codex 的消息元数据导致 BLOCKED 端点出现在未压缩范围中 #6949 kernel test 10/10 pass, version-guard + kernel-guard logic confirmed against the committed head.
  5. Pushed the merge commit as the final head (no [ework:ci-held] marker → this branch's one full CI pass).

State: master is now an ancestor of the head, so the PR is mergeable/clean. The mirror cache is stale (fetched 2026-11-17, still showing the old sha 8345b28863d), so I could not read the new run's live check state from it this run — the push is confirmed at the git level (remote head = ca47a5dca81) and the full-suite verdict will come from CI on that head. The kernel-change: #6949 declaration remains in the PR body, so kernel-guard is satisfied; the human-only review tier applies (kernel change).

结构化八问

  1. 是什么问题:PR 分支与 master 冲突(dirty)——master 在分支基线(merge-base 7247410976,kernel 0.0.198)之后前进了 135 个提交,含 master 自己的 kernel 版本推进 0.0.198→0.0.199→0.0.200,与 PR 的 0.0.198→0.0.199 撞车。
  2. 什么原因导致的:两条分支从同一 base(0.0.198)各自独立 bump kernel 版本——PR 为 Codex message metadata causes BLOCKED endpoints to appear in the uncompressed range. | Codex 的消息元数据导致 BLOCKED 端点出现在未压缩范围中 #6949 的 protected.ts 修复 bump 到 0.0.199,master 为 Version 0.214: compression issues still persist. The problem of old blocks failing to compress remains unresolved. | 0.214版本,压缩依旧存在问题,旧块无法压缩问题依旧存在 #6709/compress-receipt/[PR] compress.reasoning.tail: drop reasoning of any closed round in the protected window (#7144 follow-up) #7252 等变更 bump 到 0.0.200;版本/lockfile 三个文件因此文本冲突。代码无冲突(master 从未改过 PR 的 protected.ts)。
  3. 影响范围与严重程度:仅影响本 PR 的合并状态(无法合并);不影响任何运行时行为。冲突解决后 PR 恢复 mergeable。
  4. 怎么修复的:merge origin/master 进 PR 分支;版本取 0.0.201(高于 master 的 0.0.200,因为合并后的 kernel 同时含 master 变更与 PR 的 Codex message metadata causes BLOCKED endpoints to appear in the uncompressed range. | Codex 的消息元数据导致 BLOCKED 端点出现在未压缩范围中 #6949 变更,必须再 bump 一次——kernel-guard rule 2 强制 SRC_CHANGED 配 BUMPED);两个 lockfile 同步到 0.0.201 并验证幂等。
  5. 是否有回归问题:无。代码零改动(纯版本/lockfile 解决);typecheck+build 绿,Codex message metadata causes BLOCKED endpoints to appear in the uncompressed range. | Codex 的消息元数据导致 BLOCKED 端点出现在未压缩范围中 #6949 kernel 测试 10/10 通过;master 的 135 个提交是 master 自身已验证绿的变更,与 PR 变更无交互(master 未碰 protected.ts)。
  6. 是否破坏缓存:不涉及(纯版本/lockfile,无缓存/重试/游标/去重键)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:冲突已解决并推送(新 head ca47a5dca81),PR 恢复 mergeable/clean。新 head 的全量 CI 运行中;kernel 变更,人工评审档适用。待 CI 绿 + 人工评审后可合。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codex p2 scope/kernel

Projects

None yet

1 participant