Skip to content

[PR] fix(egress): count protected originals against image caps - #7141

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-18_count-retained-originals
Nov 20, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-18_count-retained-originals

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

Fixes #7140.

Image-count rescue currently removes protected originals from the occurrence list before calculating the remaining image count. Those images still occupy slots in the upstream request. When the eligible images alone appear to fit, rescue leaves the complete request above the provider's limit and returns the rejection to the caller.

A real relay reproduction sends two retained original images plus three ordinary images to an upstream that accepts at most four images. Unchanged master forwards five images once and returns HTTP 400. The existing rescue target is three; the complete request must retain the two originals and one ordinary image.

Change

Keep protected originals in the collected occurrences and mark their exemption explicitly. Count all of those images toward the remaining-image budget, while skipping exempt occurrences when choosing images to archive.

The byte-pressure path still excludes protected originals before its archival/downscaling work. Existing wire-format support, original-image exemptions, configuration, and prompt text are preserved.

Validation

  • Added 13 regression cases: unchanged master 9 failed, 4 passed.
  • Eight focused suites: 78 passed, 0 failed, 0 skipped. Coverage includes Anthropic, Chat Completions, Responses function/custom outputs, Google inline images, and receipt-bound original-image exemptions.
  • Real proxy and plugin-mode relays each observe upstream image counts 5 → 3 → 3 across an initial request, its successful retry, and a subsequent turn. Both return HTTP 200, preserve the original image bytes, and leave the caller-owned five-image history intact.
  • Negative controls cover already-fitting requests, unattainable caps, original-image byte-pressure exemptions, and Google audio remaining outside the image budget.
  • npm run typecheck, npm run build (including distribution guards), and git diff --check pass. An independent source/test review found no blocking issues.
  • Full regression remains the remote CI verdict, as required by the repository workflow.

Prior work and scope

Compared all 258 earlier fork heads, the five resulting src/egress-shed.ts variants, its 15-commit file history, and relevant open/closed upstream work. The previous image-count implementation (#4496) and tool-result exemption work address different behavior; the earlier variants still omit retained originals from this count.

Three files change: src/egress-shed.ts, the existing #4124 count-cap regression, and the new #7140 regression suite. No kernel, dependency, configuration, or persistence-format changes.

Primary contributor: Afloat16.

Keep retained original images in the remaining-image budget while preserving their byte-pressure exemptions. Archive eligible images until the shared target is met and reuse the learned cap on subsequent proxy and plugin relay turns.

Fixes ranxianglei#7140
@ranxianglei ranxianglei changed the title fix(egress): count protected originals against image caps [PR] fix(egress): count protected originals against image caps Nov 18, 2026
@ranxianglei ranxianglei added bug Something isn't working p1 scope/wiring labels Nov 18, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 No other open issue or PR targets #7140, and no open [PR] row in the mirror covers this thread. The related work is all closed and addresses different behavior: #4496 (introduced the count-cap convergence this PR fixes), #5754 (64-image hard cap), #6185 (downscale worklist), #7033 (sibling-result exemption). The open #5093 (invariant-judge false verdict on archived images) shares the egress-shed area but is a different symptom and a different mechanism.

标签: bug + p1 + scope/wiring — p1: under a provider image cap, a session whose history contains restored originals (acp_read_original_image) becomes permanently unusable (every request 400s, both rescue paths fail to converge) — this breaks the core promise of #4496 ("no more 400 loops"). scope/wiring: the fix lives in outbound wire adaptation to the provider API boundary (src/egress-shed.ts). Applied add-only to both PR #7141 and source issue #7140.

Review verdict: mergeable (pending the remote CI test shards, which were still running at check time). No blocking issues found.

What I verified

The bug is real (reproduced on unchanged master). I ran the PR's new regression suite tests/issue7140-retained-image-budget.test.ts against unmodified master: 9 failed, 4 passed — exactly matching the PR's claim. Root cause confirmed in src/egress-shed.ts: collectOccurrences() dropped protected originals (name-based or receipt-based acp_read_original_image matches) from the occurrence list, and shedWireImagesToCount() used that filtered length as the total image count. With 2 protected + 3 ordinary images and a cap of 4 (target 3), the filtered list (3) appeared to fit, so rescue no-oped and the full 5-image request was sent → upstream 400. Both call sites in src/server/relay.ts fail: the on-rejection retry (relay.ts:1896) and the proactive pre-send convergence (relay.ts:920) — the session is stuck on every turn.

The fix logic is correct (full diff read). Occurrence gains exempt: boolean; collectOccurrences now keeps exempt images with the flag set across all four wire paths; the byte-pressure path (shedWireImages) filters them out right after collection — byte-identical to master (same set, same order); the count path counts all occurrences toward the budget but skips exempt ones when choosing what to archive. The documented best-effort boundary (exempt-alone-over-cap → nothing archivable → rejection stays visible) is preserved.

Regression review — every existing behavior on the touched paths:

  1. Byte-pressure shedding (shedWireImages) — unchanged: the post-collection .filter((o) => !o.exempt) yields exactly master's collected set in the same order; all byte-shed suites green (issue3701, issue3909, downscale-tail, issue5745, image-restore, image-ref-byte-match).
  2. Count path under-cap passthrough — unchanged: occurrences.length <= maxImages → no-op; verified by the fitting-request negative control (no resolveRef calls, byte-identical body).
  3. Tail-yield / oldest-first walk — unchanged: walk order is still chronological occurrence order; the new if (o.exempt) continue; only skips exempt entries; tail protection and allowTail semantics intact.
  4. No-op when nothing archivable — unchanged: all-exempt-over-cap still returns the body untouched (archived=0), so the relay warn at relay.ts:1899 now fires only in the genuinely-unarchivable case — its parenthetical ("images sit in exempt tool outputs?") is now accurate.
  5. invalidateSeparatorOwnership() + metadata.imageCountShed — now fire in the rescue case that previously no-oped; correct, because the outbound copy is actually rewritten in that case now.
  6. Client history — untouched (outbound-copy-only discipline; the relay tests assert the client's five-image history stays intact).
  7. The changed No per-prompt image count cap: upstream "Too many images in request: 165 > 30" is forwarded as-is | 缺少按 prompt 的图片张数上限:上游 "Too many images in request: 165 > 30" 被原样转发 #4124 assertion — hand-verified: cap 1, 1 exempt + 3 ordinary; the old expectation (archived=2) left the body at 2 images, over cap — it had encoded the bug; the new one (archived=3) leaves exactly 1.
  8. No kernel / config / persistence / wire-format / model-facing text changes (placeholder note format unchanged); git diff --check clean; PR is on current master (ancestor check passed).

Test validity (④). The new suite uses an independent wire-shape oracle (countImages) rather than production logic; it asserts exact counts, original bytes surviving every send, oldest-first archival order, byte-stable converged resends, and no ref lookups under cap. The relay tests use a counting fake upstream (400 iff >4 images — not a scripted success) in both proxy and plugin mode and assert the upstream actually receives 5 → 3 → 3 across initial request, successful retry, and the next turn. Port 0 (deterministic). Both gate directions present per #3617 (triggers when it should; does not trigger for fitting requests, unattainable caps, byte-shed exclusion, and Google audio). These are real regression tests: on unmodified master, 9 of 13 fail.

Local runs on the PR head: typecheck clean, build clean (dist guards OK), and 124 targeted tests green across 13 egress-shed-related suites (72 + 52). The full-suite verdict is the remote CI's.

CI (mirror cache, fetched 2026-11-18T10:23:54Z): 30 checks, 0 failed, 11 pending (test shards in progress); all 19 completed lanes green (static-gates ×2, gate, lock-sync, kernel-guard, version-guard, config-surface-guard, mysql-contract); e2e lanes skipped (gated).

结构化八问

  1. 是什么问题:带「原图恢复」结果(acp_read_original_image 工具输出)的会话,当总图片数超过上游 provider 的图片上限时,图片数量救援机制完全不收敛——每次请求都被上游 400 拒绝,会话永久卡死。
  2. 什么原因导致的:collectOccurrences() 在收集图片时把受保护原图(按工具名或回执匹配)整个跳过,而 shedWireImagesToCount() 用这个被过滤后的列表长度当作总图片数。受保护原图仍占用上游请求的槽位,但预算计算看不见它们——当「可归档图片单独看放得下」时,救援静默 no-op,完整请求(含全部原图)原样发出 → 400。relay.ts 的两条救援路径(拒绝后重试 + 发送前主动收敛)都因此失效。
  3. 影响范围与严重程度:触发条件——会话历史含恢复出的原图,且总图片数超过该 origin 学到的/已知的 provider 图片上限(如 4 张)。一旦发生,该会话每一轮请求都硬 400,用户无法继续使用,只能手动清理历史。严重度 p1:核心承诺([PR] Learn upstream per-prompt image-count caps and converge outbound images — no more 'Too many images' 400 loops #4496「不再 400 循环」)在此场景下失效,属可用性缺陷。
  4. 怎么修复的:给 Occurrence 增加 exempt 标记——受保护原图计入图片预算(占槽位就要数),但永不被归档(保留原图字节是既有豁免语义)。预算不够时按最旧优先归档普通图片,使出站副本总图片数 ≤ cap−1。字节压力路径(shedWireImages)在收集后过滤掉 exempt,行为与 master 逐字节一致。
  5. 是否有回归问题:审过触碰路径的全部既有行为——字节压力 shed(逐字节一致,6 个相关套件全绿)、低于上限直通(no-op 不变)、最旧优先/tail 让位遍历(顺序与语义不变)、无可归档时 no-op(保留,拒绝仍可见)、relay 警告语(现在只在真正无可归档时触发,措辞反而更准确)、invalidateSeparatorOwnership 触发面扩大(正确,因为出站副本确实被改写了)。结论:无回归。
  6. 是否破坏缓存:不破坏。遍历顺序仍是时间序(确定性),归档选择可复现;低于上限的请求字节不变,前缀缓存不受影响;会话级学到的 cap 机制不变。
  7. 是否改了设计方向:没有。仍在「出站副本只改、客户端历史不动」的既有框架内,只修正了预算口径(受保护原图从「看不见」变为「占槽位」)。
  8. 是否引入了新的配置:没有。无新 env、无新 config 字段、无新默认值。

行为变更披露(Deliverable 4):旧行为——受保护原图不计入图片预算,救援可能在「可归档图片单独放得下」时静默 no-op,完整请求(总图片数 > cap)原样发出并被 400;新行为——受保护原图计入预算,普通图片被归档直至总数 ≤ cap−1,请求收敛后成功。这是本 PR 的修复目标本身;字节压力路径行为不变。

结论:可合(等远程 CI 的 test 分片跑完;已完成的 19 个 check 全绿、0 失败)。无阻塞问题,无需顺手修。合并由人工执行。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image-count rescue omits protected originals from the remaining budget

2 participants