Skip to content

[PR] fix(image): preserve restored originals in PTC results (#4070) - #4102

Closed
qiangu2020 wants to merge 2 commits into
ranxianglei:masterfrom
qiangu2020:2026-10-20_ptc-original-image
Closed

qiangu2020 wants to merge 2 commits into
ranxianglei:masterfrom
qiangu2020:2026-10-20_ptc-original-image

Conversation

@qiangu2020

@qiangu2020 qiangu2020 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #4070

Problem

Restoring an original image through a PTC host leaves the outer tool named ipython/run_code. Both normal image rescaling and byte-pressure shedding then treat the restored bytes as ordinary images.

Change

  • Append a versioned SHA256 footer to successful original-image restores; preserve the existing human-readable first line.
  • Match only the decoded bytes of an image beside that receipt in the same tool output. Keep unrelated images eligible for rescaling/shedding.
  • Apply the predicate to Anthropic and Responses (steady and rebuilt output paths), including both byte-shed rungs.

The footer is a caller-fabricable non-security convention, not authentication. There is no exemption registry, history mutation, new configuration, dependency/version change, or blanket tool-name exemption. Hosts must forward the complete receipt beside the image. This does not by itself update a host SDK or establish every API's carrier behavior. OpenAI Chat Completions sidecars and other serializers are not covered by this change.

Checks

  • npm run typecheck: passed.

  • node --import tsx --test tests/issue3400-image-rescale.test.ts tests/responses-tool-images.test.ts tests/issue3701-ws-byte-budget.test.ts: 40/40 passed, no skips.

  • git diff --check: passed.

  • Independent three-file review: no actionable findings.

  • Actual candidate CLI/plugin roundtrip: passed 8 checks across Responses and Anthropic. Each original was indexed through a real inbound request, discovered via the public restoration list, restored through nativeCaller=true, then forwarded with its full returned receipt. Egress retained the original SHA256 while ordinary and mixed unrelated images still shrank. A 1600px control stayed unchanged. This used synthetic host carriers, not a live Prime kernel.

  • Additional host acceptance (outside this three-file PR): actual Rust daemon + freshly built Python kernel + proxy CLI against a loopback fake provider passed 8/8 checks twice. Official 0.1.202 explicitly refused legacy nested-image receipts; the isolated PR candidate retained the original 2000x1625 image SHA256 through the real kernel on Anthropic. This is candidate validation, not an installed production release or a claim of Chat support.

No production installation or release was changed. Remote full-suite CI remains authoritative.

@ranxianglei ranxianglei changed the title fix(image): preserve restored originals in PTC results (#4070) [PR] fix(image): preserve restored originals in PTC results (#4070) Oct 21, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

PR #4102 审核结论(独立审核,未照抄 issue/作者描述作依据)

查重:不重复。全仓仅本 PR 引用 #4070;相邻 image 类 PR(#4061 outbound reserve 计费、#4016 sidecar 计价、#3735 fold-footer recarry、#3577 docs)机制与失败域均不同,无重叠。

审核独立性:我独立 rebase 到当前 master(5c2f24a4e)、逐行读两个源文件 diff、本地跑 typecheck + 定向测试。master 自 PR base 起 32 个提交均未触碰这 3 个文件或 responses-tool-output,rebase 干净(diff 恰为 3 文件)。本地(rebase 后 head)typecheck 绿、定向 40/40 绿无 skip——比 PR 原始 base 的 CI 更强。


结构化八问

  1. 是什么问题:PTC 宿主在外部 Python 工具(ipython/run_code)内部调 acp_read_original_image,把恢复出的原图字节塞回该工具的 tool_result;但可见的外层工具名仍是 ipython,既有的「按工具名豁免」(toolName===acp_read_original_image / call-id→name map)匹配不到 → 恢复的原图被当普通图再次降采样/字节丢弃(复现:2000×1625 PNG 4,896,543B → 1600×1300 3,405,591B)。
  2. 什么原因导致的:根因是「恢复原图的豁免」绑的是外层工具名,而 PTC 真实外层名不是 acp_read_original_image——用**名字(症状层)而非字节(内容层)**做身份判定,在嵌套绑定下必然失效。
  3. 影响范围与严重程度:受影响面 = 走 Responses/Anthropic 车道、经 PTC 宿主(外层名 ipython/run_code)转发恢复原图的用户;触发条件明确(必须嵌套在非标准外层工具名下)。严重度 p2:能用但体验受损——用户明确要原分辨率却拿到降采样副本,细字/颜色/对齐失真;有绕行(直接调 acp_read_original_image 控制路径仍逐字节保留)。核心压缩不停摆,不属 p1。
  4. 怎么修复的(机制层面):把豁免从「名字」改为「内容」——executeReadOriginalImage 成功时在原图旁追加一行确定性回执 Billion original image receipt v1 sha256=<原图sha256>;新增 matchesOriginalImageReceipt(b64, parts) 只在同一 tool output 的兄弟 part里找该行,命中才豁免该张图;宿主须把完整回执与原图一起转发。无豁免注册表、无历史改写、无新配置、无依赖/版本变更、无整类工具名豁免。
  5. 是否有回归问题:无意回归未发现。逐项核对:egress-shed 把 anthropic 顶层图片收集从 per-part else 挪到前置 addAnthropicImages(content,false) 预扫、per-part else 只留 tool_result 容器——顶层图片仍被收集(toolOutput=false,不豁免)、tool_result 图片走 true,两者互斥无双收;openai/responses/google 的 content part 无 tool_result type,不受 else if 影响;rescaleResponsesItem 的 message 分支逐字节一致;两个 shed rung(A archive/B downscale ladder)因豁免项不入 occ 列表而自动保护、无关图照常 shed;structural 名守卫保留并与新回执守卫叠加;golden wire-contract 只 pin 工具 schema description、不含运行时 result 文本,追加回执不触发 golden 变更。
  6. 是否破坏缓存:不破坏,反而消除一处缓存接缝。回执行是原图 sha256 的确定性函数,跨重启稳定;rescale 编码本就按 ref+bytes 幂等缓存。本 PR 让 steady 路径开始降采样 Responses tool output,对齐了早已如此处理的 rebuilt 路径——此前 steady 保大、rebuilt 缩小,同一历史重建前后出站字节跳变(prefix-cache 接缝),现两路一致,接缝消失。
  7. 是否改了设计方向:是,一处。「恢复原图的身份」从结构名判定升级为内容指纹判定(名字守卫保留共存作辅助兜底)。这正是 issue PTC original-image results are rescaled again when the outer tool is ipython/run_code #4070 明确要求的方向(CONTENT-BOUND + restart-stable,禁 blanket 名豁免/注册表/knob/全局关)。Google 车道与 OpenAI chat-completions sidecar 刻意不在本次范围(PR 已声明),PTC 实际落在 Responses/Anthropic,核心 bug 已覆盖。
  8. 是否引入了新的配置:无。无新 env/config/默认值/依赖/版本变更。

行为变更披露(Deliverables 规则 4,旧值→新值+原因)

  • ① acp_read_original_image 成功结果文本末尾新增一行 …\nBillion original image receipt v1 sha256=<64hex>。旧值=单行人话;新值=人话+回执行(首行逐字不变)。原因=给宿主一个可转发的内容指纹载体。
  • ② steady 路径 Responses tool output(function_call_output/custom_tool_call_output)图片从「不处理(保大)」→「参与降采样;命中回执者豁免」。旧值=steady 不动 tool output 图片;新值=steady 与 rebuilt 一致缩小(除回执原图)。原因=闭合 steady/rebuilt 分歧、消除 prefix-cache 接缝,属一致性收敛而非功能回退。

测试有效性(§④):4 个新测试断言有牙——对象恒等(out[0]===input[0])+JSON 逐字节+正负成对+两个 shed rung(tail 0/9)+两种协议+两种 Responses output 类型+reset 后无状态+作用域边界(别处/用户消息里的回执不授予豁免)。回归时可抓住,非恒真/mock-only。

Auto-merge 门禁(§7.4)判定:不合 auto-merge,须人工 review。触碰 wire/message shape(①result 文本加行、②steady 新降采样 tool output 图片并往 tool text 追加 note),违反第 4 条「no change to wire protocol/message shape」;且引入新处理面偏 feat/refactor。其余项(回归测试绿、rebase head 绿、不碰 kernel/update/security)达标。

CI 现状:全部静态/guard gate success(kernel-guard、pi-subagents-guard、config-surface-guard、version-guard、docs-sync、lock-sync、unused-exports、test-import-graph、lane-guard、build-artifact 等);test 分片(ubuntu/windows × node22/24 × 3)仍在跑,为最终权威判据。

一处非阻塞小观察(可选精修):回执前缀串 Billion original image receipt v1 sha256= 在写入端(executeReadOriginalImage)与读取端正则(matchesOriginalImageReceipt)各手写一份——版本若升级(v1→v2)两端需同步改,存在漂移风险。建议抽一个共享常量钉死两端(2 行级)。当前形式清晰且有注释,不阻塞合并。

结论:可合(待 CI test 分片绿 + 人工点头)。方向正确、范围精准、测试扎实、无无意回归,两处行为变更均已披露。唯一残留:Google/OpenAI-chat sidecar 不在本次覆盖(PTC 未落这两条,可接受;后续确有该场景再单列)。请 owner 合并。

ranxianglei#4070)

Reviewer-added completion of PR ranxianglei#4102: the receipt predicate landed for Anthropic and Responses but not Google, so a restored original attached by a PTC host beside its entry-level inlineData part would still be rescaled/shed on that wire. Same mechanism, same fix shape:

- rescaleGoogle skips inlineData whose decoded bytes match a receipt line in the same content-entry parts; the egress-shed google collector applies the same skip before archiving/downscaling.
- matchesOriginalImageReceipt now accepts any string .text part (the Google text part is untyped {text}); typed-wire behavior is unchanged.
- Images nested inside functionResponse.parts were already invisible to both passes and remain so (unchanged).

Tests: positive+negative rescale rows plus both shed rungs added to tests/issue3400-image-rescale.test.ts. Local: typecheck clean, targeted files green.
@ranxianglei

Copy link
Copy Markdown
Owner

(ai: zhipuai-lb/glm-5.3)

已并入合并列车 #4525(master 6d7c587,合并方式 merge commit,提交保留原作者署名)。本 PR 关单收尾;若列车中对应提交出问题,回滚点为 #4525 单提交 revert。

感谢 @qiangu2020 的贡献——你的提交以原作者署名随列车进入 master(合并方式为 merge commit),仓库贡献者列表可见。

z0d0dOr pushed a commit to z0d0dOr/billion-context that referenced this pull request Oct 24, 2026
ranxianglei#4070)

Reviewer-added completion of PR ranxianglei#4102: the receipt predicate landed for Anthropic and Responses but not Google, so a restored original attached by a PTC host beside its entry-level inlineData part would still be rescaled/shed on that wire. Same mechanism, same fix shape:

- rescaleGoogle skips inlineData whose decoded bytes match a receipt line in the same content-entry parts; the egress-shed google collector applies the same skip before archiving/downscaling.
- matchesOriginalImageReceipt now accepts any string .text part (the Google text part is untyped {text}); typed-wire behavior is unchanged.
- Images nested inside functionResponse.parts were already invisible to both passes and remain so (unchanged).

Tests: positive+negative rescale rows plus both shed rungs added to tests/issue3400-image-rescale.test.ts. Local: typecheck clean, targeted files green.
(cherry picked from commit 9534fb5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PTC original-image results are rescaled again when the outer tool is ipython/run_code

2 participants