Skip to content

[PR] fix: responses rebuild keeps images when a plain-text user item sits before a multi-part one (#4150) - #4155

Closed
ranxianglei wants to merge 5 commits into
masterfrom
2026-10-21_issue4150-responses-image-drop
Closed

ranxianglei wants to merge 5 commits into
masterfrom
2026-10-21_issue4150-responses-image-drop

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4150

kernel-change: bugfix in kernel/src/wire/responses.ts rebuild path — when a plain-text (string-content) user item was adjacent ahead of a multi-part (text+image) item, outbound reconstruction silently dropped the image parts (#4150). Owner-approved via issue triage.

What was broken

flushUserRun (multi-core path) took the FIRST sidecar core as THE carrier for the whole run of adjacent user cores. When the leading item had string content, imageParts=[], so image cores resolved to undefined and were skipped — the model never saw that image again. Cross-item runs also merged into ONE wire item, and full-cover required only a count match against one raw item (stale bytes could be resurrected after text rewrites).

Fix

Behavior changes (old → new)

area old new
[text, [text,image]] full rebuild image dropped, items merged into one both items verbatim, image kept
[text,text] / [multipart,text] full rebuild merged into one array item separate items, byte-exact per item
multi-core run full-cover count-only match vs one raw item (could resurrect stale bytes after a text rewrite) per-part byte equality — verbatim only when unchanged (matches single-core and patch-lane discipline)
partially-surviving split image resolved from the first carrier item (wrong/missing across items) resolved from its own item
assistant array-content item (text unrewritten) stringified on rebuild byte-exact array round-trip

Unchanged: single-core rendering, string-content assistants, rewritten text (ref-tag/demoted-thinking falls back to synthesized form), the patch lane (patchResponsesInput untouched), and the main proxy lane (coreToResponsesWithToolImages wraps per-message — zero change there).

Coordination

Stacked on #4151 (responses wire judge): this PR flips judge P1 back to the verbatim assertion and drops P1b's stringification expectation. If #4151 lands first, this diff shrinks to just the fix.

Tests

  • New regression suite kernel/tests/issue4150-responses-split-rebuild.test.ts (7 tests: issue matrix, cross-image assignment, marker leak, partial folds, rewritten-byte locality, assistant shapes, cross-lane agreement on the full state).
  • Kernel suite 1404/1404 pass; verif-subset-guard green (wire/responses stays inside the translatable sublanguage); judge test 8/8 green with flipped pins; targeted responses-lane bili batch (76) + toolargs-adversarial (674) green locally; typecheck + build clean. Full-suite verdict = remote CI.

ework-agent and others added 3 commits October 21, 2026 18:36
…ge drop)

New judge tests/verif-wire-responses-stability.test.ts over kernel/src/wire/responses.ts
(responsesToCore / coreToResponses / patchResponsesInput / normalizeEasyInputItem):

- P1 pinned #4150: string-content user item directly before a multi-part
  (text+image) user item → flushUserRun picks the STRING item as rawCarrier
  → imageParts=[] → image silently dropped; items also merge. Documented
  today-broken pin, flips when fixed.
- P1b: round-trip without the #4150 shape — user multi-part verbatim
  (image kept), function pair verbatim, assistant array-content
  stringified (cross-lane inconsistency with patchResponsesInput,
  recorded in #4150 comment), trailing user verbatim.
- P2 idempotence + input non-mutation of the full codec.
- P3 Easy-stamp stability: type-less items stamped {..,type:"message"}
  from turn 1 — consistent across folds (append-stability of survivors).
- P4 fold locality: folding image/text cores keeps exactly the surviving
  part; no sidecar resurrection of folded parts (#3642 semantics).
- P4b anchor semantics: insertions land before surviving anchors.
- P5 append-stability of patchResponsesInput under tail growth.
- P6 both-parts-folded: slot drops from layout without touching others.

Full suite 73/73 green with solver; tsc clean.
flushUserRun took the FIRST sidecar core as THE carrier for a whole multi-
core run, so a string-content user item ahead of a multi-part item starved
the image cores of their parts (image silently dropped) and the items
merged into one. The run now groups by source item (wireIndex, sidecar-
identity fallback) and rebuilds each group: full coverage + unchanged
bytes => the original item verbatim; partial survival => a filtered view
of that item's own surviving parts (images resolved from the group's own
item, never resurrecting folded content); text-only split groups without
a sidecar synthesize from the cores. Single-core rendering extracted
verbatim into flushUserCore (byte-identical).

Assistant lane gets the same raw-sidecar verbatim fast path: an array-
content item whose text survived unrewritten round-trips byte-exact
(shape included) instead of being stringified — closing the cross-lane
divergence vs patchResponsesInput's slot.original (#3837 family).

Regression suite: kernel/tests/issue4150-responses-split-rebuild.test.ts
(issue matrix, cross-image assignment, marker leak, partial folds,
rewritten-byte locality, assistant shapes, cross-lane agreement).

kernel-change: bugfix in wire/responses.ts rebuild path — image parts
lost on outbound reconstruction when a plain-text user item is adjacent
to a multi-part one (#4150); owner-approved via issue triage.
P1 was the inverted control while broken (image-drop shape); with the
per-source-item grouping landed it asserts deepEqual(STAMPED) again.
P1b drops the assistant-stringification expectation — the array-content
assistant item now round-trips byte-exact like every other item.
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-21_issue4150-responses-image-drop (42f1ccf)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4950-pr.4155.8bb4201

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4155.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: responses rebuild keeps images when a plain-text user item sits before a multi-part one (#4150) [PR] fix: responses rebuild keeps images when a plain-text user item sits before a multi-part one (#4150) Oct 21, 2026
d9110fb bumped kernel/package.json (CI-enforced pairing) but left the kernel lockfile at 0.0.147; the lock-sync gate's idempotent regen diffs and fails (#3782 drift class), skipping every full test lane. Regenerated with npm install --package-lock-only in kernel/.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 检索 open+closed issue/PR(input_image、coreToResponses、responses rebuild):最接近的 #3290(工具结果图片上游丢失,机制不同,已关闭)、#3701(WS 1009 体积,已关闭)、#3412(媒体计数,已关闭)、#3676(split 边缘字节漂移而非内容丢失,已关闭),均与「纯文本 user item 紧邻多部件 item 时重建丢图」这一症状+机制不同;无并行修复 PR 在途(#4151 是 judge 钉桩的 stacked base,不是竞争修复)。

标签: bug · codex · opencodev2 · scope/kernel · p2 —— Responses 车道缺陷(codex/opencodev2 宿主),改动在 kernel/src/wire 编解码层;p2:特定消息形状下的功能性缺陷,主代理车道不受影响,非会话丢失/缓存泄露类。

状态:审核进行中,先报一个已顺手修掉的问题。 CI 上 lock-sync 红导致全部 test/e2e 车道被跳过——根因是 d9110fb 把 kernel/package.json 提到 0.0.148 但漏了 kernel/package-lock.json(仍 0.0.147,#3782 同类版本漂移)。已在 PR 分支补了独立标注 commit 5fc174bea(lockfile 重生成,仅两行版本号),push 完成,CI 正在新 head 上重跑。完整八问审核结论在本地定向验证 + 远程 CI 绿灯后给出。

Companion to 5fc174b: the root lockfile's file:-link record for kernel/ also carries acp-kernel's version, so the lock-sync gate's root regen diffs on it too. Same #3782 drift class.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #4155 审核结论(独立审核,非复读 issue 侧分析)

查重:不重复。检索 open+closed issue/PR(input_image、coreToResponses、responses rebuild):最接近的 #3290(tool-result 图片在上游丢失,机制不同)、#3701(WS 1009 尺寸)、#3412(media 计数)、#3676(split-edge 字节漂移而非内容丢失),均不同症状+不同机制;无并行修复 PR(#4151 是堆叠基座,不是竞争修复)。

标签:bug · codex · opencodev2 · scope/kernel · p2(保留原有 pr 标签)。Responses 车道宿主;kernel/src/wire 编解码面;p2 = 特定消息形状下的功能性缺陷(主代理车道不受影响,不属于会话不可恢复/缓存泄露类 p1)。

结构化八问

  1. 是什么问题:Responses 车道的会话里,当一个纯文本(string-content)user item 紧挨在多部分(text+image)item 前面时,压缩后的出站重建会静默丢掉图片——模型从此再也看不到那张图。
  2. 什么原因导致的(我独立走读 parse+rebuild 两侧确认):旧 flushUserRun 把整段相邻 user core 的第一个 sidecar core 当作整段的唯一载体。当领头 item 是 string content 时它的 imageParts=[],后面 image core 从载体上解析图片得到 undefined 被跳过 → 图片丢失。同一段跨 item 还会被合并成一个 wire item;full-cover 判定只数个数对某一个 raw item(count-only),文本被改写后可能把旧字节当原样复活。
  3. 影响范围与严重程度:Responses 协议宿主(codex / opencode-v2);触发条件 = user 消息序列出现 [纯文本, [文本+图]] 这类邻接形状且发生折叠;主代理车道不受影响(coreToResponsesWithToolImages 按单条消息包裹,行为不变)。特定条件下有问题的功能性缺陷,非核心能力停摆 → p2。
  4. 怎么修复的:重建改为按源 item 分组——cores 按 wireIndex(缺省时按 sidecar 对象同一性)分组,每组携带自己的 raw item:全覆盖且逐 part 字节未变 ⇒ 原样 verbatim;部分存活 ⇒ 用该组自己 item 的存活 parts 渲染(图片从本 item 解析);无 sidecar 的纯文本 split 组从 cores 合成。单 core 渲染路径原样抽成 flushUserCore(逐字节等价)。assistant 车道补了同样的 raw-sidecar verbatim 快路径:数组内容且文本未被改写时逐字节回环(含形状),不再 stringify。
  5. 是否有回归问题(逐项过既有行为清单):
    • 单 core 渲染:flushUserCore 与旧路径逐行比对,字节级一致 ✓
    • string-content assistant / 被改写文本(ref-tag、demoted-thinking):仍走合成路径,有负向测试钉住 ✓
    • patch 车道(patchResponsesInput)完全未触碰 ✓
    • 主代理车道:零改动(无 src/ 文件变更;单 item run 走 single-core 路径不变;未变状态的多 core 组现在命中 verbatim,与旧 fullCover 结果一致)✓
    • 有意变更(已披露并验证):多 core full-cover 判定从 count-only 收紧为逐 part 字节相等——这是纪律收紧不是回归(与单 core 和 patch 车道的口径对齐,杜绝改写后复活旧字节);[text,text]/[multipart,text] 从合并成一个数组 item 变为按 item 分开、各自字节精确——出站形状变化但内容保真度更高,属本次修复目标行为。
  6. 是否破坏缓存:不涉及缓存失效策略;verbatim 快路径反而保证未变状态出站字节稳定(前缀缓存友好)。
  7. 是否改了设计方向:没有,codec 局部修复,不动架构/边界/数据流。
  8. 是否引入新配置:无。

测试有效性(④)

新套件 kernel/tests/issue4150-responses-split-rebuild.test.ts 7 个测试,断言有牙:5 形状矩阵 deepEqual 对照原始输入(含两个本就正确的形状作对照)、跨 item 图片归属(fold item0 后 item1 保留自己的图、绝不复活外来的)、[image] marker 不外泄、4 种部分折叠(图片从本 item 解析 / 存活文本 core 独立渲染一致性 / 全折则整项消失 / 改写文本走过滤视图拿新字节)、assistant 数组 verbatim + 改写 assistant 保持字符串形态(负向门)、以及跨车道一致性:patchResponsesInput(proj,proj.msgs) === coreToResponses(proj.msgs) 在全状态上逐字节相等。门的双向性(#3617)满足。

本地定向验证(仅跑窄范围,全量交给 CI):kernel 新套件 7/7、judge 回放测试 9/9、verif-subset-guard 2/2、typecheck + build 干净。全量裁决 = 远程 CI。

顺手修掉的 CI 红(lock-sync,#3782 漂移类)

d9110fb 把 kernel/package.json 升到 0.0.148 但两处 lockfile 都没跟着动,lock-sync 门禁红、test 车道全被 needs 跳过。已在 PR 分支上以独立标注 commit 修掉:

  • 5fc174bea — kernel/package-lock.json 两行版本号同步(0.0.147→0.0.148);
  • 42f1ccf1e — root package-lock.json 的 kernel 记录一行版本同步(第一次 push 后 CI 日志显示 root 检查仍差这一行,补上)。

CI 状态与 windows 车道 flake 说明

CI @ 42f1ccf:除一条车道外全绿(ubuntu×{22,24} 全部 shard、windows/24、以及所有 e2e 工作流:real codex / native dsh·opencode·pi / fake-codex / pi-codex-ws / local-registry / ci-image / codex-persona / build artifact)。唯一失败 test (windows-latest, 22, 1):tests/issue4079-egress-window-fuse.test.ts 两个 e2e 用例报 failureType: 'cancelledByParent'("Promise resolution is still pending but the event loop has already resolved")——是父套件在子用例 pending 时被中断,不是断言失败。判为环境性 flake 的依据:(a) 同一代码在本 run 的 ubuntu22/ubuntu24/windows24 车道全过;(b) 本 PR 只改 kernel/src/wire/responses.ts + lockfile + responses 测试,与 egress fuse/body-size 路径零交集;(c) CI 脚本注释写明 windows shard 墙钟 ~1110s 贴着 step 上限,而失败的两个用例恰是该文件里最重的 I/O(8.68MB 构造体 + 1.63MB 转发体)。已对该 run 触发 rerun-failed-jobs:重跑通过——最终全量 CI @ 42f1ccf 32 success / 14 skipped / 0 failure,与 flake 判定一致(该车道第二次跑就绿了)。

合并顺序说明

本 PR 堆叠在 #4151(judge-pinning)之上:#4151 先合则本 diff 自动缩小到纯修复;judge 翻转(P1 回到 verbatim 断言、P1b 去掉 stringify 预期)依赖修复落地才成立,所以两者不能颠倒单独合。master 领先 3 个提交仅为 release-notes v0.1.203 critical entry 的陈旧基座产物,无冲突风险(mergeable=true)。

判定

可合——全量 CI @ 42f1ccf 已全绿(kernel 层变更按规约由人工执行合并,合并顺序注意 #4151)。修复方向正确、打到根因(载体选择错误而非症状遮蔽)、回归面逐项核过、测试有牙且有跨车道一致性钉。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):issue [4150] 已由已合并 PR [4151] 修复。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant