Skip to content

[PR] fix: bound retained image encode cache payloads - #6848

Closed
Afloat16 wants to merge 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-15_image-cache-budget
Closed

Afloat16 wants to merge 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-15_image-cache-budget

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

Fixes #6846.

The process-wide image encode cache retains completed base64 output behind a 512-entry cap. Large image encodes can consume hundreds of MiB long before reaching that count, even after the originating request/session stops using them.

Changes

  • Keep the existing 512-entry cap and also cap completed base64 payloads at 64 MiB.
  • Return an oversized result to its caller without retaining it or evicting all the smaller cached results.
  • Keep in-flight entries available during byte-based eviction, so concurrent callers still share an encode.
  • Ignore an old promise's completion after its entry was cleared or evicted; it cannot charge bytes against a replacement entry.
  • Reuse the existing cache-reset test seam for small-budget fixtures. A no-argument reset restores the production budget; no user-facing setting is added.

The resize engine, dimensions, quality, MIME keys, original-image storage, and annotations are unchanged. Evicted images can require another deterministic encode. This is a bound on retained cache payloads, not on codec working memory or total process RSS.

Reproduction and measured result

Tested from master 2e3cfc9 (0.1.215), the parent of this commit. Head: c39ace7.

The included benchmark runs twenty distinct 2400×1600 PNG images through the real rescale pass at the default 1600-pixel longest edge. It warms the codec first, then runs three garbage collections with event-loop turns before each memory snapshot. The fixture uses deterministic high-entropy pixels; it does not represent the size of every screenshot.

Measurement Unchanged master Candidate
Last image's encoded base64 length 7,346,668 bytes 7,346,668 bytes
External memory after warmup/cache reset 22.1 MiB 22.1 MiB
External memory after 20 images 162.2 MiB 85.1 MiB
External memory after clearing the cache 22.1 MiB 22.1 MiB
Retained heap + external memory released by clearing the cache 140.1 MiB 63.1 MiB

Reproduce with:

node --expose-gc --import ./tests/preload-test-env.mjs --import tsx scripts/bench-image-encode-cache.mjs 20

These are local measurements. CI uses deterministic cache-admission assertions, without timing/RSS thresholds or large image allocations.

Validation

53 focused tests passed, run with the repository preload, tsx, and test-concurrency=1:

  • tests/image-rescale-cache-budget.test.ts
  • tests/issue3400-image-rescale.test.ts
  • tests/egress-shed-downscale-tail.test.ts
  • tests/egress-shed-media.test.ts
  • tests/issue3701-ws-byte-budget.test.ts

The new tests cover shared byte eviction across normal and forced encodes, oversized-result admission, byte-identical re-encoding, concurrent requests, completion after cache reset, pending entries under byte pressure, negative-result count limits, and restoring the default test budget. The original controls include real image codecs and HTTP paths, original-image recovery exemptions, and WS oversize/1009 retries.

On unchanged master, the original large-payload regression fails; its in-flight-sharing control passes. The small-fixture regressions also fail for byte eviction and oversized retention while their three initial controls pass.

Also passed:

  • node node_modules/typescript/bin/tsc --noEmit --project tsconfig.json
  • npm run build
  • node scripts/check-test-import-graph.mjs
  • git diff --check

The live Codex four-phase E2E has not been run: the built distribution and Codex CLI are present, but E2E_CHECK=1 reports the configured http://127.0.0.1:8199/v1/models upstream unreachable. The real-codec and HTTP/WS tests listed above are hermetic.

Related changes checked: #6086 (resize-policy cache key) and open #6109 (shed MIME key) address different behavior.

@ranxianglei ranxianglei changed the title fix: bound retained image encode cache payloads [PR] fix: bound retained image encode cache payloads Nov 15, 2026
@Afloat16

Copy link
Copy Markdown
Contributor Author

The failed CI job is test (windows-latest, 22, 2). Its two failures are the existing abort x proxy-openai-sse and abort x proxy-responses-sse cases in tests/wire-exit-matrix.test.ts; both timed out waiting for upstream closure with closed=0. These use text-only request fixtures and do not exercise the image encoding cache.

The same two abort cases passed locally on this PR head c39ace7aabe8c15f5d924f8162e6d727cac58fe8 (2/2) and on the unchanged base 2e3cfc973b8620b1f7bc8a415ddcbd31e7f4aaee (2/2), with Linux and Node 24.22.1. That comparison does not establish that the Windows/Node 22 failure is resolved.

Could a maintainer rerun this failed Windows job? No abort behavior or timeout threshold has been changed in this PR.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Investigated before acting — this is a known flaky cell, not a regression from the PR. Reran the failed job; it's green now.

What failed — only test (windows-latest, 22, 2), two subtests in tests/wire-exit-matrix.test.ts:

  • abort x proxy-openai-sse — upstream socket(s) must be destroyed after the client abort (closed=0), 22.2s
  • abort x proxy-responses-sse — upstream socket(s) destroyed after client abort (closed=0), 21.7s

Both are the client-abort → upstream-teardown cells. In the same run the sibling cells were green (abort x proxy-anthropic-sse 376ms, abort x proxy-json 423ms) and every other shard passed (2105/2141).

Not a regression from this PR. The diff touches only src/image-rescale.ts (adds a retained-payload byte budget to the image-encode cache), a new bench script, and a new test. I confirmed it does not touch tests/wire-exit-matrix.test.ts or any abort/SSE/teardown/timeout code, and the two failing cells use text-only fixtures (content: "hi", no images) — they never exercise the image-encode cache. The abort/teardown path is byte-for-byte the same as on base.

Known flake. The test header (lines 31–64) documents this exact signature as a repeat Windows flake — closed=0 after the full teardown poll on windows-latest node22 with siblings green, attributed to runner-side event-loop starvation, not a teardown hang (a mandated audit found the destroy/abort wiring in relay.ts is synchronous and untouched since d244fff). The 20s win32 budget + measured-lag extension (capped at 2×base) is the final robustness design; there's no smaller mechanical fix to add without a design change.

Action + result. Reran the failed job(s) of run 43728334421 (rerun-failed-jobs, HTTP 201). The rerun (run_attempt 2) passed — test (windows-latest, 22, 2) is now success, and the whole run is green (0 failures). Note the CI harness deliberately did not auto-retry this failure ("failing WITHOUT auto-retry so a real regression is never masked"); I overrode that default on the strength of the analysis above (PR provably out of the code path + documented flake signature + siblings/shards green). If it flakes again on this PR, that points at a persistent Windows-node22 issue and I'll escalate with options.


结构化八问

  1. 是什么问题:windows-latest node22 shard 2 上 tests/wire-exit-matrix.test.ts 的两个 abort 用例(proxy-openai-sse / proxy-responses-sse)在客户端中止后等待上游 socket 销毁超时,closed=0。
  2. 什么原因导致的:不是本 PR 引入的回归。PR 只改 src/image-rescale.ts(图片编码缓存加保留字节预算)+ 新增 bench 脚本 + 新增测试,完全不碰 abort/SSE/teardown/timeout 代码;这两个用例用纯文本 fixture(无图片),根本不会走到图片编码缓存路径。失败签名(closed=0、win32 node22、同 run 兄弟用例与其余 shard 全绿)与该测试头部注释记录的 Windows 负载 runner 事件循环饥饿 flake 完全一致——既有审计已确认 destroy/abort 接线是同步的、自 d244fff 后未动过。
  3. 影响范围与严重程度:仅单个 Windows 测试 shard 的偶发;不影响功能/用户面,纯 CI 噪音,体验瑕疵级。
  4. 怎么修复的:触发该 run 的失败 job 重跑(rerun-failed-jobs,HTTP 201);未改任何代码。重跑(run_attempt 2)已通过,全 run 转绿。
  5. 是否有回归问题:无。PR 未触碰失败路径,重跑不改任何行为。
  6. 是否破坏缓存:不涉及(重跑不碰任何缓存/重试/游标/去重逻辑)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:非本 PR 回归,系已知 Windows flake,已重跑并转绿,可正常走合并流程。

@Afloat16

Copy link
Copy Markdown
Contributor Author

Thanks for investigating and rerunning the Windows Node 22 job; the current head is green now.

The merge panel still waits for three Required checks:

  • test (windows-latest, 24, 1)
  • test (windows-latest, 24, 2)
  • test (windows-latest, 24, 3)

The plan job selected core, which does not create Windows/Node 24 jobs. Those jobs retain these exact names in the existing FULL matrix. The workflow comment documents this legacy required-check mismatch.

Could you apply the existing ci:full label to this PR so the missing checks run on the same head? The workflow is identical on this head and current master, so rebasing would not address the missing checks. The same Windows/Node 24 contexts are also absent from #6843, #6845, #6853, and #6854, whose emitted checks have passed or intentionally skipped; could those use ci:full as well?

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

@Afloat16

Copy link
Copy Markdown
Contributor Author

Thanks for confirming the Windows Node 22 rerun. I checked the merge panel again after your update: it still shows 13 skipped, 3 expected, 37 successful checks on c39ace7a. The three Windows/Node 24 checks listed above are still marked Required — Expected: Waiting for status to be reported.

The remaining step is to create those missing check results, using the workflow's existing FULL matrix. Could you apply ci:full specifically to this PR (and the four related PRs listed above)? The label has not been applied yet, and the ordinary CORE run cannot emit these three contexts.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

🔶 需要决策

Background (one line): the PR's code is reviewed and ready, but the merge panel is blocked by a pre-existing branch-protection mismatch — not a defect in this PR.

What I verified (live, mirror check cache): the core-matrix run on c39ace7a is fully green — 30 checks, 0 failed, 0 pending (all 11 core test legs + static-gates + every guard). The three required checks test (windows-latest, 24, 1/2/3) were never emitted, because the plan job selected core scope and the CORE matrix has no windows/Node-24 legs (they exist only in the FULL matrix). I confirmed this against the workflow itself: the plan-job comment in .github/workflows/ci.yml states "core produces 11 of the 16 current names (windows node-24 legs appear only in FULL scope)" and "master's required-checks list still carries the legacy 12 FULL-matrix leg names". So your analysis is correct: the ordinary CORE run cannot emit those three contexts.

The decision needed: master's required-status-check list is stale — it references windows/Node-24 shard 1/2/3 from the pre-#4351 3-shard era, while the current FULL matrix emits windows/Node-24 shards 1–5. This blocks 5 open PRs (#6848 plus #6843, #6845, #6853, #6854 — the four you confirmed are missing the same contexts), not just this one. Two ways out:

  • Option A — quick unblock (per-PR): apply the ci:full label to [PR] fix: bound retained image encode cache payloads #6848. The labeled event re-runs the plan job with full scope, which emits the windows/Node-24 legs (1–5) and satisfies the three required checks. Cost: one full-matrix run (~16 legs) per PR, repeated on all 5 affected PRs. Does not fix the root cause.
  • Option B — root fix (systemic): update master's branch-protection required-status-check list to match the CORE matrix (the intended blocking layer) — drop the legacy windows/Node-24 names and align to the current leg names (the workflow comment already notes the new windows shard-4/5 names "must be added to branch protection before they gate merges"). Cost: a one-time branch-protection edit (owner/admin); fixes all 5 PRs and every future PR.

My recommendation: Option B is the correct root fix — the required-check list is stale infrastructure, and per-PR ci:full is a band-aid that re-occurs on every PR. If you need #6848 merged immediately, apply ci:full to it now (Option A) and do Option B separately.

Default if no decision: the PR stays blocked on the three "expected" checks; nothing merges. I have not applied the label: per my operating rules I don't mutate the source PR's labels (that's a maintainer/owner action), and the branch-protection list is owner/admin territory. I also won't touch the 4 related PRs — they're separate threads outside this task's scope; Option B fixes them all at once.


代码评审(结构化八问)

  1. 是什么问题:进程级图片编码缓存此前只有 512 条目的数量上限,没有对"保留下来的 base64 载荷总字节"设限。少量大图解码(每张几 MiB base64)就能在远未到 512 条时占用数百 MiB,且原请求/会话早已不再使用它们。
  2. 什么原因导致的:缓存按条目数(FIFO 到 512)淘汰,与载荷大小正交。大图片每张持有数 MiB base64,保留内存随图片尺寸增长而非随条目数增长——数量上限管不住字节总量。
  3. 影响范围与严重程度:任何需要缩放大量/大尺寸图片(截图、高分辨率上传)的进程都受影响;保留内存只被 512 条数量上限约束,可涨到数百 MiB。属重度图片负载下的内存压力/OOM 风险,不是正确性 bug(线上输出字节不变)。
  4. 怎么修复的:在既有 512 条数量上限之上叠加 64 MiB 字节预算。每个完成的编码按 base64 字节长度记账;字节超限时淘汰最旧的已完成载荷(在途/未完成条目保留,继续做去重共享)。单张图若本身超过 64 MiB 预算,则只返回给调用方、不保留(避免一张巨图挤掉所有小的有用条目)。被清除/淘汰/替换的旧 promise 完成时不得对替换条目记账(防陈旧完成污染新条目)。
  5. 是否有回归问题:无。逐项核对:①线上输出字节不变——缓存是纯优化,淘汰只触发确定性重编码(同字节);②现有 clearRescaleEncodeCache() 调用方(issue3400 测试)全部无参调用 → 取生产 64 MiB 预算,不受影响;③fresh 标志与在途去重语义不变;④共享的 cacheEncode 反而消除了原先两处重复的淘汰循环。已知取舍(非回归):大图片现在会被字节预算更早淘汰,因而可能重编码(确定性、同字节)、压缩日志可能再次触发——这是内存上限的既定取舍,数量上限时代本就存在。
  6. 是否破坏缓存:本 PR 约束的就是这个图片编码缓存本身(这正是目的)。未触碰其他任何缓存/重试/游标/去重;前缀缓存与消息身份缓存不受影响(线上字节不变)。
  7. 是否改了设计方向:无架构转向。同一模块内的有界缓存细化(数量上限 → 数量+字节双上限)。刻意不加新配置项(符合 config-surface 纪律)。
  8. 是否引入了新的配置:无。64 MiB 是硬编码常量 ENCODE_CACHE_MAX_BYTES(产品决策,非旋钮)。测试缝 clearRescaleEncodeCache(maxBytesForTest?) 新增可选参数(无参恢复生产值)——仅测试用,无用户面。

质量:外部提交,按 ⑤b 从严过了一遍——命名达意、无死代码、无重复(还顺带消掉了重复的淘汰逻辑)、错误处理完整(陈旧完成/超大结果/负结果各有分支)、无魔法值、注释讲为什么、无新依赖、测试有真断言(字节数、读取次数、字节级重编码、并发/竞态)而非恒真。代码质量达标,无需对齐重构。

判定:代码正确、测试扎实、质量达标,可合;唯一阻塞是上面那条既有的分支保护 required-check 失配(非本 PR 缺陷),需 owner 在 A/B 中拍板。64 MiB 默认值属产品决策,按 §7.4 走人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

Closing status (full review + decision request are in the comment above):

Match the issue3400 suite's BILI_CONFIG_FILE isolation so the
admission/eviction tests run on the default (enabled) rescale posture
regardless of any host config file.
@ranxianglei

Copy link
Copy Markdown
Owner

This fix is included verbatim (original commits and authorship preserved) in the merged batch PR #7074 — the changes are already on master. Closing to avoid a duplicate merge. 该修复已随批量 PR #7074 原样合入 master,为避免重复合并,关闭此 PR。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image rescale cache retains over 140 MiB after only 20 images

2 participants