Skip to content

[PR] fix(egress-shed): archive past the 64-image cap so image-dense bodies reach the byte budget - #5754

Merged
ranxianglei merged 2 commits into
masterfrom
2026-11-05_image-shed-lift-cap
Nov 7, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-11-05_image-shed-lift-cap

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

A session with more than 64 inline history images on the plain/proxy HTTP relay lane could never be brought under the egress byte budget (network.httpOutboundBytes, default 10 MiB). The pre-send byte-shed capped its work list at MAX_SHED_IMAGES (64) for both rungs, so once a session had >64 sheddable images the excess bytes were permanently un-removable: the body stayed over budget, upstream rejected it (400 with no overflow marker), nothing self-healed, and the session retried the identical over-budget body every round — shed to X MiB but still over budget … sending anyway.

Reported repro (#5745): ZCode / openai wire, 582 images, 39.6 MiB out → shed to 31.4 MiB → permanent 400 loop, session unusable.

Fix

Rung A (archiving an image into a [image: … omitted] placeholder note) is a cheap string swap — it does not decode/re-encode through jimp — so the per-request CPU bound does not apply to it. Rung A now walks every non-tail occurrence oldest-first until the estimate fits; the 64-image bound stays only on rung B (downscale), where it bounds decode/re-encode CPU. This mirrors what the count-shed sibling (shedWireImagesToCount) already does and documents as intentional ("that bound caps jimp CPU, while archiving is a cheap string swap").

The protected-tail invariant is unchanged: the last 8 entries are still never archived, only downscaled. Yielding the tail when even that falls short is a separate design decision (raised on the issue thread), not part of this fix.

Behavior change (intended, disclosed)

  • ≤64 sheddable images: byte-for-byte identical to before (the loop already stopped at the cap or earlier).
  • >64 sheddable images: the outbound copy now archives more old images than before so the body actually reaches the budget. This changes the outbound bytes for those oversized sessions, which breaks their prefix cache once — but those sessions were hard-400'ing on every round before, so no usable cache existed. Net effect: availability restored (session works again; archived old images stay reachable via read_original_image).

Old → new: rung A work list was occurrences.slice(0, 64) (capped); now occurrences (all non-tail, stopping as soon as the estimate fits). Rationale: the cap was applied to a non-CPU-bound action by accident of sharing the variable with rung B.

Tests

New tests/issue5745-image-shed-cap.test.ts:

  1. 100 images over a 3 MiB budget → asserts archived > 64 and finalBytes <= budget. Fails on the old code (archived 64 images), passes here — verified by stashing the source fix.
  2. Budget reachable via non-tail archiving → protected tail keeps its images (invariant preserved).
  3. Boundary: when non-tail archive + downscale cannot reach an impossibly tight budget → only the 4 non-tail images are archived, tail intact, still-over reported (pins the edge vs. the pending yield-tail decision).

Existing shed suites stay green locally (targeted runs): issue3701-ws-byte-budget + issue3909-http-byte-budget + issue4124-image-count-cap + issue3400-image-rescale (all pass). Full-suite verdict = remote CI. Typecheck clean.

Fixes #5745

… reach the byte budget

The pre-send egress byte-shed capped its work list at MAX_SHED_IMAGES (64) for BOTH rungs. A session with more than 64 inline history images could therefore never shed enough bytes to get under network.httpOutboundBytes: the body stayed over budget, upstream rejected it (400, no overflow marker), nothing self-healed, and the session retried the identical over-budget body forever ('shed to X MiB but still over budget ... sending anyway').

Rung A (archiving) is a cheap string swap, not a jimp pass, so the per-request CPU bound does not apply to it. Rung A now walks every non-tail occurrence oldest-first until the estimate fits; the 64-image bound stays only on rung B (downscale), where it bounds decode/re-encode CPU. The protected-tail invariant is unchanged (tail is still never archived, only downscaled).

Regression test: tests/issue5745-image-shed-cap.test.ts (fails on the old code with 'archived 64 images', passes here).

Fixes #5745
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-05_image-shed-lift-cap (114bfd4)

Latest build — install this exact version

npm install -g billion-context@0.0.8284-pr.5754.fe7c77e

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
114bfd4 billion-context@0.0.8284-pr.5754.fe7c77e build
6aed3b8 billion-context@0.0.8269-pr.5754.baf8409 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5754.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 —— 在途 [PR] 线程里只有本线程针对 #5745 的图片 shed 上限问题,无并行 PR。

标签: 不打([PR] 镜像线程惯例不打标签;底层缺陷在上游 #5745 已带 bug/p1/zcode/scope/wiring)。

中期状态(代码审完、CI 红灯已定位,结论随后补):

  1. 代码审查:改动最小且正确——rung A(归档为占位符,纯字符串替换、不过 jimp)改为遍历全部非尾部出现点、按最旧优先、估算达标即停;64 张上限只保留在 rung B(jimp 缩放,CPU 有界)。保护尾部不变量(最后 8 条永不归档)未动。≤64 张的会话出站字节逐字不变;>64 张的会话此前每轮硬 400,现在能真正降到预算内(代价是该次前缀缓存失效一次,但那些会话本来就没有可用缓存)。测试有牙:断言 archived > 64 且 finalBytes <= budget,旧码下可复现失败。
  2. CI 红灯 = 继承性失败,与本次改动无关:10/11 个 test shard 挂在同一个用例 tests/issue2447-dsh-workspace-instructions.test.ts:243(dsh workspace-instructions 基线守卫断言 BLOCKED ref),该文件在 PR 基线之后被 ci-fixes 上的 0470bec62(realign five dsh regression pins,CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599)修过;PR 分支落后 master 27 个提交,没带上这个修复。ci-fixes tip 全绿。
  3. 处置:把 origin/ci-fixes merge 进 PR 分支并推送,本地先定向复现(issue2447 用例在 PR head 上应红、merge 后应绿)+ 跑新测试与四个兄弟 shed 套件,然后看 CI 重跑结果出最终八问结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

最终结论(结构化八问):

  1. 是什么问题:plain/proxy HTTP relay 车道上,会话内联历史图片超过 64 张且出站体超过字节预算(默认 10 MiB)时,永远降不到预算内——上游每轮硬 400、无自愈,会话不可用([Bug][media] Cannot compress image replay: shed only processes the first 64 images, imageCompression has been removed, b | [Bug][media] 图片重放无法压缩:shed 只处理前 64 张、imageCompression 已移除、字节计费把 582 张图算成 3.56M tok → 39.6MiB 请求体被上游 400 拒 → 会话永久不可用 #5745 复现:ZCode/openai wire,582 张图,39.6 MiB)。
  2. 什么原因导致的:shedWireImages() 里 rung A(归档为占位符,纯字符串替换,不过 jimp)和 rung B(jimp 缩放,CPU 有界)共用同一个 64 张工作列表上限(MAX_SHED_IMAGES)。这个上限本意是限制 jimp 解码/重编码的 CPU,却连带把非 CPU 受限的归档也截断了——超过 64 张后多余的字节变成永久不可移除。
  3. 影响范围与严重程度:仅「>64 张可 shed 图片 + 出站体超预算」的 plain/proxy 车道会话;严重度 p1——这类会话完全不能用(每轮硬 400)。≤64 张的会话完全不受影响。
  4. 怎么修复的:rung A 改为遍历全部非尾部出现点(最旧优先,估算达标即停);64 张上限只保留在真正有 CPU 成本的 rung B。保护尾部不变量(最后 8 条永不归档、只缩放)未动。这与计数版兄弟函数 shedWireImagesToCount(No per-prompt image count cap: upstream "Too many images in request: 165 > 30" is forwarded as-is | 缺少按 prompt 的图片张数上限:上游 "Too many images in request: 165 > 30" 被原样转发 #4124)已文档化的意图一致。
  5. 是否有回归问题:逐项审过——①≤64 张会话:旧循环本来就在上限或更早处停,行为逐字节不变;②保护尾部不变量:test 2/3 钉住;③WS 车道([Bug] Jimp is active, but image accumulation still triggers Responses WS 1009; duplicate request size not reduced (v0.1. | [Bug] Jimp 已生效,图片累积仍触发 Responses WS 1009;重复请求未缩小(v0.1.200) #3701)与 HTTP 车道(Image session permanent 503: re-sending 48.6 MiB outbound body every round (fold blocked by cadence, archived images sti | 图片会话永久 503:每轮重发 48.6 MiB 出站体(fold 被 cadence 挡住,已归档图片仍在 wire 上) #3909)共用入口,两套件全绿;④无 fail-fast 变吞错、无阈值/时序漂移。CI 首跑 10/11 shard 红与本改动无关:是继承性 born-red(DSH: If the first request of a session originates from a bypass call, compression for the entire session is permanently | DSH: If the first request of a session originates from a bypass call, compress for the entire session is permanently inv | DSH:会话首个… #5656/[PR] fix: unknown dsh one-shot sidecars steal the raw-key anchor and kill compression (context-explosion root cause) #5687 在分支基线之后合入,5 个 dsh 回归钉失效:issue2447 BLOCKED-ref 断言、issue2817 差一 8!==9、issue5403 A/B turn-0 戳为 null),我已在旧 head 本地逐字复现同一断言;修复在 ci-fixes 0470bec62,哨兵已把分支同步 master 至 114bfd459,新 head 全仓 CI 全绿(含全部 test shard 与 e2e)。本地定向复验(新 head):新测试 + 4 个兄弟 shed 套件 + 3 个重对齐钉,8 文件 67/67 通过。
  6. 是否破坏缓存:有意变更(已在 PR 描述披露)——>64 张会话现在会归档更多旧图,出站字节变化一次 → 前缀缓存失效一次;但这些会话此前每轮硬 400,本来就没有可用缓存。缓存/重试/增量游标机制本身不受影响;归档图仍可通过 read_original_image 取回。
  7. 是否改了设计方向:没有。双档结构、不变量、协议形状都不变,只是把工作列表上限移到它理由成立的位置(jimp CPU 侧)。
  8. 是否引入了新的配置:没有。

收工判定:可以合并。按 §7.4 属人工合并层(出站字节行为变更),请 owner 自行合并:#5754 。备注:我推送时发现分支已被哨兵先行同步 master(non-fast-forward),遂放弃重复 merge、改在新 head 上做验证,未产生多余提交。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 修复完成:PR #5754 转绿 @ 114bfd459(18 success / 12 env-gated skip / 0 failed / 0 pending),可合。

七问:

  1. 是什么问题:PR 首跑(run 42049555651 @ 6aed3b840)10/10 test shard 全红(ubuntu 22×4+24×3、windows 22×3),阻塞合并;static-gates、mysql-contract 均绿。
  2. 什么原因导致的:继承性失败——跑测时 master 自身红(DSH: If the first request of a session originates from a bypass call, compression for the entire session is permanently | DSH: If the first request of a session originates from a bypass call, compress for the entire session is permanently inv | DSH:会话首个… #5656 sidecar residual 五钉家族)。抽样 ubu22-s1 / ubu24-s1 / win22-s1 三份 job 日志签名逐项对上:issue2446/2447 BLOCKED-ref guard-stamp(expected true actual false)、issue2817 8 !== 9、issue5403/[Bug][dsh] Plugin lane performs a full re-parse and resends the entire body every round (81 MiB → hundreds of MB AST), w | [Bug][dsh] plugin 车道每轮全量解析重发 body(81 MiB→数百 MB AST)是 #5403 OOM 的生产者:用水位线断言替换逐条复核,服务端只解析新增尾段 #5418 watermark A/B null !== 2;win22-s1 另有 dsh-native cancelledByParent 负载级联(Promise resolution is still pending,同级用例前一刻刚过)——非断言失败。本 PR diff(src/egress-shed.ts 9 行 + 新测试 tests/issue5745-image-shed-cap.test.ts)与全部失败路径零交集,零 PR 自致。
  3. 怎么修复的:master 已吸收 ci-fixes(realignment 0470bec62),故直接 git merge origin/master(bfabcd1ca,ci-fixes 超集)进分支——干净合并、无冲突、零代码改动,纯 base 对齐。本地定向验证:六个继承钉文件 + 本 PR issue5745 回归测试 21/21 绿,npm run build 绿。已推送,CI 在新 head 全矩阵重跑。
  4. 是否有回归问题:无。零代码改动,只移动了 base;PR 自身行为(egress-shed 归档越过 64 图上限)未动,其回归测试在新 head 通过;既有行为清单(继承钉)逐项复验全绿。
  5. 是否破坏缓存:不涉及。
  6. 是否改了设计方向:不涉及。
  7. 是否引入了新的配置:不涉及。

协调留痕:已在 #4599 评论一行(继承红判定 + merge sha)。本轮无 🔶 需要决策项。等 owner 合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

APPROVE — 审的是 PR #5754(head 114bfd459,分支 2026-11-05_image-shed-lift-cap,base master)。单行语义改动:rung A(归档)从遍历 work(=occurrences.slice(0, MAX_SHED_IMAGES))改为遍历 occurrences 全量,外加注释;新增一个回归测试文件。共 2 文件(+96/-1):src/egress-shed.ts、tests/issue5745-image-shed-cap.test.ts。方向正确、机制成立、行为变更已披露。

八问

  1. 是什么问题:纯代理 HTTP relay(及 WS)车道上,历史内联图片 >64 张的会话,发送前字节 shed 的两档都把工作表截在 64 张,多余图片的字节永远删不掉 → body 一直超 network.httpOutboundBytes(默认 10 MiB)→ 上游每轮 400(无 overflow marker)、不自愈、原样重发 → 会话永久不可用([Bug][media] Cannot compress image replay: shed only processes the first 64 images, imageCompression has been removed, b | [Bug][media] 图片重放无法压缩:shed 只处理前 64 张、imageCompression 已移除、字节计费把 582 张图算成 3.56M tok → 39.6MiB 请求体被上游 400 拒 → 会话永久不可用 #5745 复现:582 图 / 39.6 MiB)。
  2. 什么原因:「64 张」本意是给 rung B(jimp 解码+重编码)设 per-request CPU 上限,却因复用同一个 work 变量被误加到了 rung A 头上——而 rung A 归档是纯字符串替换、不走 jimp。超限会话里 rung A 只归档了前 64 张,剩下几百张照发。根因是「把一个 CPU 上限错套到一个非 CPU 受限动作上」。
  3. 影响范围与严重程度:仅纯代理/pi-native HTTP relay 与 WS 车道上「>64 张可 shed 图片」的会话;之前是完全不能用(每轮硬 400)。≤64 张会话零影响。属可用性缺陷(p1 级症状),本 PR 恢复可用性。
  4. 怎么修的:rung A 改为遍历全部非 tail 出现点、oldest-first、估算值一降到预算内即停;64 张上限只留在 rung B。这与同文件 count-shed 兄弟函数 shedWireImagesToCount 早已采用并写明为有意的设计("该上限 cap jimp CPU,归档是廉价字符串替换")一致——本 PR 把 byte-shed 对齐过去。保护尾不变式不动:最后 8 条仍永不归档、只可降采样;tail 是否也参与归档列为独立决策、不在本 PR。
  5. 是否有回归(逐点过既有行为):①≤64 张——循环本就在 cap 或更早处停,逐字一致;②保护尾——仍 if (o.protectedTail) continue,从不归档;③确定性——oldest-first + 同一 est 阈值,相同历史重复重发产出稳定字节,前缀缓存不抖动;④两个调用点(shedEgressIfOverBudget 预发送门 src/egress-shed.ts:436、WS 1009 恢复重发 src/server/relay.ts:933)一致受益、无副作用;⑤rung B 仍 cap 64,jimp CPU 有界。性能面:rung A 现对每张非 tail 图各调一次 shedNote(decodeImageDims 只解析头部几十字节、b64Bytes O(1)、refForItemBytes 索引一遍廉价比较+偶发 readFile),最坏 O(n×索引大小)次廉价操作且预算可达时提前 break——与已上线的 count-shed 完全同构,非新增风险。
  6. 是否破坏缓存:>64 图超限会话出站字节会变(多归档旧图)→ 前缀缓存一次性失效;但这些会话此前每轮硬 400、本就无可用缓存,净效果恢复可用(旧图仍可经 read_original_image/decompress 取回)。≤64 图会话字节不变、缓存不受影响。PR 正文「Behavior change」段已披露旧值→新值+原因。
  7. 是否改了设计方向:无。仍是 outbound-copy-only、两档 shed、保护尾不变式;只是把误加的 cap 从 rung A 摘掉。
  8. 是否引入新配置:无。无 config/env/默认值/schema/持久化/wire 结构/kernel 改动。

测试:tests/issue5745-image-shed-cap.test.ts 三例——①100 图 / 3 MiB 预算断言 archived>64 且 finalBytes<=budget(老代码停在 64、剩 ~4.8 MiB,两断言必挂;作者已 stash 验证);②30 图经非 tail 归档达预算后保护尾 8 条仍持图(不变式钉);③12 图极紧预算只归档 4 张非 tail、tail 不被让渡、报 still-over(边界钉,对应待决策)。断言有牙,能兜住「重新给 rung A 加 cap」这类回归。

不涉 kernel/(无 kernel-change 需求)。diff 收敛到 2 文件、无无关改动,CI 绿。证据基础:静态审读全函数 + 两个调用点 + 关键 helper(decodeImageDims/b64Bytes/refForItemBytes)+ 真实回归测试 + 远端 CI 绿(本地未重跑全量,遵本地测试纪律)。

结论:可合(APPROVE)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

✅ 协调回执:PR #5754(head 114bfd459)review 完成 —— APPROVE。无顺手小修需补;diff 收敛到 2 文件(+96/-1)、不涉 kernel/、无新配置面、CI 绿。完整八问分析见上方评论。

一点供合并参考:本 PR 带 ework-agent-pr 标记(agent 自提、同一账号),GitHub 禁止对同账号 PR 发正式 APPROVE review(返回 422),故结论以评论形式记录;owner 可直接合并。

@ranxianglei
ranxianglei merged commit 078379f into master Nov 7, 2026
47 checks passed
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1059-master.078379f (rolling master tag).

npm install -g billion-context@0.0.1059-master.078379f  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Louie0806 added a commit to Louie0806/billion-context that referenced this pull request Nov 7, 2026
Louie0806 added a commit to Louie0806/billion-context that referenced this pull request Nov 7, 2026
…ed cap, render-snapshot plane

Upstream 2919794..ce8f92e (57 commits) lands three large planes:

- ranxianglei#5652 fastpath-v6 identity lane (c295f90 storage layer, d55a50c
  composite identity, 9c1fb86 full-list wire, da6885d fixed tail window,
  plus the review follow-ups)
- ranxianglei#5754 image-shed 64-item cap (6aed3b8)
- ranxianglei#5689 render-snapshot plane (acca4db schema v12, 6d3ca30 O(1) steady
  turns, 49ba14d seal-shut probe)
- ranxianglei#5721 witnessed rebase supplement

Overlap with this lane was four files. src/plugin.ts auto-merged (upstream
touches the handoff-reply import at :31 and handlePluginTool at :2337; this
lane owns the strip entry points at :22 / :61 / :2640..:4554). The two
CONFIGURATION files are generated and were regenerated from the seed.

One real conflict, in tests/issue4666-migration-dedup.test.ts line 158:
upstream re-swept the sqlite user_version pin to the literal 13, this lane had
already derived it. Kept the derived form -- that is the whole point of the
earlier sweep, and the literal is what goes stale on the next bump.

That conflict resolution exposed a latent bug of mine. 3eb22b1 ("pin the
mysql lock lane to the CURRENT schema version") derived the mysql stamp
assertions from STORAGE_SCHEMA_VERSION as well. When it was written both
numbers were 9; ranxianglei#5652/ranxianglei#5689 have since pushed the sqlite unified store to 13
while the MySQL backend stayed at 11, because the v12 identity plane and the
v13 render-snapshot plane are sqlite-only. The mysql-contract lane therefore
asserted "13" against a migrator that correctly stamps "11".

Fixed by exporting MYSQL_SCHEMA_VERSION from src/storage/mysql.ts and pointing
the four mysql-side assertions at it; the sqlite assertions keep deriving from
STORAGE_SCHEMA_VERSION. Verified String(MYSQL_SCHEMA_VERSION) is "11", i.e.
byte-identical to the literal upstream asserts.

Verification on the merge tree: issue5175 58/0, tag-echo 79/0,
issue2190-degen-close 32/0, golden-storage-render 5/0, storage-engine-proofs
12/0, native-compact-terminal 1/0, plugin-passthrough-tag-strip-chat 20/0,
issue4666-migration-dedup 2/0 (the three mysql cases skip without
BILI_MYSQL_TEST_DSN; no local MySQL or docker socket in this sandbox).
tsc --noEmit clean, all four gates exit 0, generated docs byte-identical.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant