Skip to content

[PR] fix: restore byte-triggered image re-encoding - #7147

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-18_restore-byte-triggered-image-encoding
Nov 20, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-18_restore-byte-triggered-image-encoding

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Fixes #7146.

Problem

The ordinary image pass admits a candidate when its dimensions or decoded byte size exceed the existing thresholds. The encoder then rejects every within-limit image when force=false, so the byte-size trigger never reaches a normal re-encode.

On current master, a real 256 × 256 PNG written with PNG compression level 0 remains 262,503 bytes on the wire. The same installed Jimp codec can re-encode it to 757 bytes at unchanged dimensions with identical RGBA pixels.

Change

Restore the pre-#3701 byte-size gate in doRescale. The production change is one condition: ordinary within-limit images above 100 KiB reach the existing encoder and 20% keep-smaller check.

The forced encoder's truth table is unchanged. The existing format, quality, dimension limit, cache, original-image exemption and outbound-copy behavior are retained.

This reduces request bytes for images with encoding headroom. It does not reduce pixel-based vision token counts for same-dimension images. PNG remains lossless in the regression; JPEG uses the existing quality-80 policy. Newly admitted candidates pay the existing encode cost once per cache lifetime; low-benefit results still pass through unchanged.

Validation

  • 17 new real-codec and real-HTTP regressions/controls: unchanged master 12 failed, 5 passed; fixed 17 passed.
  • Five core image carriers and both Responses item shapes; PNG pixel/dimension equality, JPEG policy, decoded-size threshold on both sides, unhelpful BMP output, disabled rescale, original-image tool exemption, repeated output with/without cache, and unchanged forced-mode behavior.
  • Actual proxy and plugin requests deliver the 757-byte image to a local HTTP upstream while their remembered session views retain the original 262,503-byte image without synthetic note messages.
  • New suite plus existing image-rescale, encode-cache, ref-byte, WS byte-budget and hermetic image-billing suites: 73 passed, 0 failed, 0 skipped.
  • TypeScript typecheck, complete build, import-graph guard and git diff --check passed.
  • An independent read-only review checked the condition's complete truth table and the actual codec/HTTP assertions. Full platform regression is left to the repository CI on this head.

Prior work

Audited all 258 prior fork heads, five distinct image-rescale source versions, and full file history. The gate was last changed by 6e39a65d; its #3706 review corrected forced-mode comments, not the ordinary branch. Existing cache-key, cache-budget and original-ref work addresses different mechanisms. No configuration, kernel, persistence, or prompt strings change.

Restore ordinary same-dimension encodes above the existing byte threshold.
Keep the forced path and keep-smaller policy unchanged, and verify real
codec output, original-byte preservation and proxy/plugin HTTP requests.

Fixes ranxianglei#7146
@ranxianglei ranxianglei changed the title fix: restore byte-triggered image re-encoding [PR] fix: restore byte-triggered image re-encoding Nov 18, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Review: mergeable. I independently re-derived the gate's truth table, read all 17 tests, and checked CI live on head 8f078956.

What I verified (independent of the PR's claims):

  • The bug is real. On master, src/image-rescale.ts:166 is if (scale >= 1 && (!force || origBytes <= RESCALE_TRIGGER_BYTES)) return null;. For the ordinary path (force=false), !force is true, so the sub-clause is always true and every within-dimension-limit image is rejected regardless of byte size. The byte-size trigger at line 199 (origBytes > RESCALE_TRIGGER_BYTES) admits the candidate, then line 166 throws it back — the ordinary byte trigger is unreachable. Confirmed by code reading.
  • The fix is correct and the forced path is provably unchanged. New condition if (scale >= 1 && origBytes <= RESCALE_TRIGGER_BYTES) return null;. Forced path (force=true): old = scale >= 1 && (false || origBytes <= trigger) = scale >= 1 && origBytes <= trigger; new = scale >= 1 && origBytes <= trigger — bit-identical. Ordinary path: old rejected all within-limit images; new rejects only within-limit and below-trigger, restoring the byte trigger. The scale < 1 resize branch, keep-smaller 20% floor, encode cache key, original-image exemption, and annotation text are all untouched.
  • Tests are real, not tautological. 17 tests: all 5 carriers + both Responses item shapes, PNG pixel/dimension equality (asserts decoded.bitmap.data deep-equals the source), JPEG quality-80 policy, the 100 KiB threshold on both sides (199×128 below / 200×128 above, using the real byte length as ground truth), unhelpful-BMP passthrough, disabled-rescale, original-image tool exemption (both message and item form), repeated output byte-identical with and without the cache (determinism), unchanged forced-mode gate, and real proxy + plugin HTTP requests asserting the 757-byte image reaches the upstream while session memory keeps the 262,503-byte original with no synthetic acp_imgnote_ message. Real Jimp codecs, real startServer + fake upstream.
  • CI is green. Live check on 8f078956: 37 success, 13 skipped, 0 failed, 0 in progress. The skips are the file-gated real-client e2e lanes (correctly not triggered by this diff) plus the hermetic image-billing e2e, which is gated behind the ci:full label on PRs (runs on the 30-min cron) and tests the feat: dual-channel image/text accounting — pixel prior by default, learned per-route cost (#1843) #1857 billing surface, not rescale — not a gap. The new unit file runs in the main test shards and passed.
  • No config / kernel / persistence / prompt change. Diff is one production line + the new test file. Reuses the existing RESCALE_TRIGGER_BYTES = 100 * 1024.

Non-blocking observation (pre-existing, not introduced here): the force doc comment at src/image-rescale.ts:152-155 says "skips the 100 KiB trigger gate" / "A within-maxDim image is NOT skipped here", but the code still evaluates origBytes <= RESCALE_TRIGGER_BYTES for the within-limit forced case. It's accurate for the production path (the egress-shed caller pre-filters maxDim-fitting images, making that branch dead), but the wording overstates it. Out of scope for this PR (different path); flagging so it isn't lost.


结构化八问

  1. 是什么问题: 普通(非强制)图片通道的字节触发器失效——doRescale 门控把「尺寸在限内」的图一律拒绝,即使解码字节超过 100 KiB 触发线;rescaleImage 按字节放行的候选到 doRescale 又被挡回,普通出站路径的字节触发器永远到不了编码器。
  2. 什么原因导致的: [Bug] Jimp is active, but image accumulation still triggers Responses WS 1009; duplicate request size not reduced (v0.1. | [Bug] Jimp 已生效,图片累积仍触发 Responses WS 1009;重复请求未缩小(v0.1.200) #3701 引入强制字节 shedding 路径时,把普通路径的字节门控与强制路径的「跳过触发门」合并进同一条件 (!force || origBytes <= trigger);force=false 时 !force 恒真,尺寸在限内的图被无条件拒绝。是条件合并时的逻辑错误,非设计意图。
  3. 影响范围与严重程度: 所有普通出站请求中,尺寸在 maxDim 限内但解码字节 >100 KiB 且有编码余量的图(如 deflate-0 PNG、高质量 JPEG)不被重编码,白传 ~99% 字节(实测 262,503 → 757)。触发条件:限内 + 超 100 KiB + 有编码余量。严重程度:能用但请求字节膨胀(成本/体验受损),p2;不影响正确性(原图照常转发,只是没省字节)。
  4. 怎么修复的: 门控改回 scale >= 1 && origBytes <= RESCALE_TRIGGER_BYTES——仅对「限内且字节低于触发线」返回 null,恢复普通字节触发器。强制路径条件改动前后逐位相同,行为零变化;尺寸 resize 路径、keep-smaller 20%、缓存键、原图豁免、注释文本均不动。
  5. 是否有回归问题: 无。唯一行为变化即目标修复本身。强制路径、尺寸触发、keep-smaller、缓存键、原图豁免、注释格式、出站副本语义逐项核对均未变。PNG 重编码无损(像素逐位相等,有测试钉死);JPEG 沿用 quality-80。确定性保持:同输入→同输出(with/without cache 重复输出测试)。
  6. 是否破坏缓存: 不破坏。编码缓存键 mime:maxDim:b64 不变,同图同 maxDim 命中同一项;出站字节对同一历史逐轮字节稳定。唯一一次性影响:版本升级时这些限内大图前缀字节 262,503→757,触发一次 provider 前缀缓存 miss——但这是修复目的(省字节),净减少,稳态更瘦。
  7. 是否改了设计方向: 没有。无架构/边界/数据流转向,仅恢复 [Bug] Jimp is active, but image accumulation still triggers Responses WS 1009; duplicate request size not reduced (v0.1. | [Bug] Jimp 已生效,图片累积仍触发 Responses WS 1009;重复请求未缩小(v0.1.200) #3701 之前的既有字节触发器语义。
  8. 是否引入了新的配置: 没有。复用既有 RESCALE_TRIGGER_BYTES = 100 * 1024,无新 env/config/默认值。

判定: 可合。 修复正确、强制路径可证不变、测试有牙、CI 绿、无回归、无新配置。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Follow-up on the non-blocking observation above: the forced-path doc tangle (src/image-rescale.ts:152-155 vs shedDownscaleImage's doc, both contradicting the line-166 code on within-maxDim handling) needs a design decision — is the within-maxDim trigger gate in the forced path intended or a bug? Filed it as a separate drive-by issue so it isn't lost and doesn't expand this PR's scope: #7151 (mirror #7151).

This PR (#7147) itself is unaffected and remains mergeable — it provably does not touch the forced path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image byte-size trigger skips compressible images within the dimension limit

2 participants