Skip to content

feat!: remove image pre-compression and the image_full tool (#3266) - #3295

Merged
ranxianglei merged 6 commits into
masterfrom
2026-10-15_remove-image-compression
Oct 15, 2026
Merged

ranxianglei merged 6 commits into
masterfrom
2026-10-15_remove-image-compression

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

What

Removes the entire compress.imageCompression feature and the image_full tool — host and kernel — per the decision on #3266 (fixes shipped as #3270 first, then the call was made to delete instead of redesign).

Why delete (not fix)

What's removed

  • kernel: image-compress.ts (downscale pass, restore, notes), tool declarations, config schema/state/persist keys — kernel 0.0.125 → 0.0.126
  • host: src/image-compress.ts, 4 prepare lanes' pass + tool injection + trailing note, handle stamping, session stats/Maps, persist fields, loop/stream tool dispatch, side-request + pi-subagent allowlists, ingest whitelist, config schema, compress-settings plumbing
  • sharp is no longer an optional dependency (package-lock −583 lines)
  • docs (CONFIGURATION.md/zh-CN), website tooltip rows, wire-contract goldens, storage corpus regenerated

Back-compat

Sessions persisted by older versions load unchanged: the legacy imageShrinks/imageFullRestored state keys and all imageShrunk*/imageFull* stats are dropped on load (whitelisted readers). One nuance pinned in review: metadata is an opaque host bag and passes through as-is, so a legacy metadata.imageShrinks entry survives in memory as inert data nothing reads. The storage corpus pins this as a REAL fixture — the generator re-injects the pre-#3266 keys into the on-disk record after write (scripts/update-storage-corpus.ts), and tests/storage-corpus.test.ts asserts each key lands where the contract says (dropped: stats/state; passed through: metadata).

Tests

  • kernel: 1268/1268
  • host targeted (wire-contract, goldens, corpus, persist, session, loop, fold, side-request, absorb, conflict, prepare-lane families): 259/259 + 159/159, tsc --noEmit clean, npm run build OK

Closes #3266 (redesign tracked there if it ever comes back).

kernel-change: image-compression removal — kernel/src/image-compress.ts deleted (downscale pass, image_full tool, restore + fail-streak), config schema/state/persist keys removed; kernel 0.0.125 → 0.0.126. Part of the #3266 feature deletion (PR title and body carry the full rationale).

The opt-in compress.imageCompression pass and its image_full restore
channel are structurally broken where they matter most:

- hosts that reject bili-injected tools (codex) can never deliver the
  restore call — the downscale is permanent for them (#3270 disarmed
  the lane, which proved the tool was dead weight there);
- where the tool IS injectable, the model has no reliable way to learn
  the ref of the downscaled message (no ref annotations on the wire),
  producing the 38-round guess-reject-resend loop from #3266 (cache
  hit 75% -> 28%).

Remove the whole feature from host and kernel: arrival-time downscale
pass, image_full injection + sticky restore + fail-streak guardrail,
image stats, persistence keys, wire-contract tool variants, config
schema, docs, and the sharp optional dependency. Every image now passes
through byte-identical at original resolution.

Sessions persisted by older versions load unchanged; legacy
imageShrinks/imageFullRestored state keys and imageShrunk* stats are
ignored on load (storage corpus pins that as a fixture).

kernel 0.0.126.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 检索了 open+closed 的 issue/PR(关键词 image compression / image_full):相关条目只有本 PR 的直接上游 —— #3266(原始 bug 报告,已关闭)与 #3270(先行合入的 codex-lane 关断修复)。没有其他在途 PR 或 issue 在做同一删除或重设计。进入正常审核流程,稍后给结构化结论。

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-15_remove-image-compression (3c815e5)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3844-pr.3295.a9b37fa

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3295.tgz
npm install -g package

Install instructions are refreshed on each push.

ework-agent and others added 4 commits October 15, 2026 22:41
…tats fields + stale comments

Review of the main removal commit left dead code behind: ImageFormat/
ImageBillingMode type aliases and the CompressionStats imagesShrunk/
imageBytesSaved/imageTokensSaved fields (seeded by createInitialState,
asserted in the kernel persist test) had zero remaining readers after
src/image-compress.ts went away. Removed them and reworded three
comments that still described the image_full restore channel as live.
The first commit regenerated the corpus after stripping the legacy
image keys, so tests/storage-corpus.test.ts asserted 'keys must NOT
materialize' against a fixture that never carried them — a vacuous pin
while the PR body claimed the corpus pinned back-compat.

The generator now re-injects the pre-#3266 keys into the on-disk record
after the normal write (writeAndPin mutateFiles hook), so the fixture
carries exactly what an older version persisted. Verified load behavior
pins honestly: whitelisted readers DROP stats.imageShrunk*/imageFull*
and state.imageShrinks/imageFullRestored; metadata is an opaque host
bag and passes THROUGH (legacy imageShrinks survives as inert data —
the digest covers the loaded view, so it pins that split). All case
digests shift because the kernel state.stats blob lost its three image
fields; folded/zstd pair stays byte-identical.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 结论:PR #3295 — 删除图片预压缩与 image_full 工具

查重:不重复(首条评论已给:仅祖先 #3266/#3270,均已关闭)。

八问:

  1. 是什么问题:feat! 删除型 PR——按 image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 的决定(删而非修),移除整个 compress.imageCompression 功能与 image_full 工具(host + kernel)。
  2. 什么原因导致的 / 为何删:两条独立机制证明该功能不成立——① codex lane:宿主拒绝 bili 注入的工具并 fork 历史,再正确的 ref 也到不了工具(fix: image_full 死循环(#3266)——codex lane 上 restore 通道结构性不可用:按宿主门控 + 提示/回执列出可恢复 ref #3270 已把该 lane 整体拆除,证明是死重);② 其他宿主:模型没有可靠渠道学到降采样消息的 ref,只能猜邻近 id 被拒,image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 实测 38 轮 guess→reject→resend 循环(cache hit 75%→28%)。方向判断成立:修的是根因(功能本身不可用),不是症状。
  3. 影响范围与严重程度:功能自出生默认关(enabled: false),仅显式 opt-in 用户受影响;变更后图片按原始分辨率字节透传。默认用户零行为变化。属有意 breaking(feat!),范围有限。
  4. 怎么修复的:62 文件完整删除(+81/−3702):host 4 条 prepare lane(pass+工具注入+trailing note)、handle stamping、session stats/Maps、persist 字段、loop/stream dispatch、side-request+pi-subagent allowlist、ingest 白名单、config schema、compress-settings;kernel 侧 image-compress.ts、工具声明、config/state/persist(0.0.125→0.0.126);sharp optional dep 移除(package-lock −585);docs/goldens/corpus 同步。逐段通读完毕,无残留引用。
  5. 是否有回归问题:既有行为逐项清点——wire 工具面 −image_full ×4 lanes(有意,已披露);[acp-usage] 日志失去 img 后缀(有意);旧配置里 compress.imageCompression.* 今后静默忽略——与既有 unknown-key 处理一致(parseCompressSettings 本就逐 key 校验,拼错的 key 历来被忽略),无新失效模式;forkParentRevisionLegacy 的 state spread 不再含 image 键——对带旧键的持久化会话,fork hash 输入与老版本构建不同,但 forkHash 用于同进程谱系内的 parent-revision 匹配,跨版本连续性风险低(记录在案);加载旧会话实测验证:whitelisted readers 丢弃 legacy stats/state 键,而 metadata 是不透明 host bag 原样透传——legacy metadata.imageShrinks 会以 inert 数据形式留在内存(无任何读取方)。此细节与原 PR body「ignored on load」的笼统表述不符,已修正 body 措辞并做实 corpus pin(见下)。高危模式逐项过:无 fail-fast 变吞错、无阈值/时序漂移、无缓存策略静默失效。结论:无意回归为零。
  6. 是否破坏缓存:不涉及——nudge cadence、前缀缓存、压缩触发机制均未触碰;corpus 的 folded/zstd 双份仍字节一致(digest 相同),证明加载语义未变。
  7. 是否改了设计方向:无转向,纯删除,未引入新架构/边界/数据流。
  8. 是否引入新配置:无新增;移除 compress.imageCompression 段(config-surface-guard 通过:136 leaf fields,25 CLI flags)。

审阅中发现并已直接修复的问题(2 个 commit 已推 PR 分支):

  1. 缺 kernel-change: 声明——PR 触碰 kernel/** 但 body 没有强制标记,kernel-guard CI 必红。已补进 PR body(含理由:image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 上「删而非修」的决定)。
  2. 假 back-compat pin(fake test)——首 commit 重新生成 corpus 后 fixture 不再含任何 legacy 键,而 tests/storage-corpus.test.ts 声称「corpus record predates image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 and still carries imageShrinks metadata」,断言恒真;PR body「storage corpus pins that as a fixture」与事实不符。修复:generator 加 mutateFiles hook,写盘后向 image case 的记录注入 pre-image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 legacy 键(stats/state/metadata 三处),fixture 真正携带旧格式;测试改为断言各键落在契约规定的位置(stats/state 被丢、metadata 透传)——pin 从恒真变为有牙。
  3. 删除遗留死代码——kernel 孤儿类型 ImageFormat/ImageBillingMode 与 CompressionStats 嵌套字段 imagesShrunk/imageBytesSaved/imageTokensSaved(createInitialState 播种、persist 测试断言)在删除后零引用;另有 3 处注释把已删的 restore channel 描述为存活。一并清理(同机制收尾)。

作者侧在我审阅期间自行落了 da58486(website yaml 陈旧种子清理),与我发现的 stale-docs 问题重合,未重复修。

观察项(不拦):corpus 全量再生成含既有漂移——biliVersion 戳 0.1.191→0.1.197、storageMode 字段新增、folded fixture tail 顺序(master 自 v0.1.191 以来有 #3050/#3194 等多个 persist 提交,goldens 只是追平,非本 PR 引入);我的第 2 个 commit 使所有 case digest 因 kernel state.stats blob 缩 3 字段而平移(有意,commit message 已注明)。

验证:定向本地测试 storage-corpus + wire-contract 29/29、kernel persist 45/45、tsc --noEmit clean、config-surface-guard 通过。CI(head 9c0efb9):已完成 12/27 全绿(pi native lane、#2195 线程保持、pi WS lane、codex fake upstream 等),guard/e2e 余量排队中,暂无红;全量判定以 CI 为准。

判定:可合(CI 全绿后)。 方向成立、删除彻底、back-compat 真正钉住、死代码已收尾、行为变更全部披露。

…the PR body after the last push

kernel-guard snapshots github.event.pull_request.body at dispatch time
(ci.yml PR_BODY), so the runs triggered by the review-fix commits still
carry the pre-marker body and would fail on a red herring. This empty
commit produces a fresh pull_request event whose payload includes the
declaration.
@ranxianglei
ranxianglei merged commit e479ed3 into master Oct 15, 2026
27 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.463-master.e479ed3 (rolling master tag).

npm install -g billion-context@0.0.463-master.e479ed3  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
…am (#3252)

The compress/absorb loop's round-2 replay rebuilt its system prompt
without the project attribution line, so the trigger turn and the
replay diverged in the pre-summary head — every cache-friendly,
cache-proof and seam-probe leg went red. Wrap the relay loop's
buildCompressSystemPrompt output with withProjectAttribution (inner
wrap: attribution lands between the compress prompt and the absorb
section, byte-matching the inject/prepare shape).

Storage-render goldens regenerated via
scripts/update-storage-render-goldens.ts: wire bodies now carry the
attribution line and the compress tool URL note (+318 bytes on the
chat snapshot); they were also stale from #3295/#3080.
ranxianglei added a commit that referenced this pull request Oct 16, 2026
…olden-rebaseline

test: rebaseline storage-render goldens after #3295 dropped dead image-stats fields from state.stats
Zhishulo pushed a commit to Zhishulo/billion-context that referenced this pull request Oct 23, 2026
…nglei#4351)

The suite grew ~399 -> ~860 files in three weeks while the core matrix
shape stayed frozen at its ~650-file design point (ranxianglei#3295), so its
heaviest legs became the per-PR wall-clock floor: the ubuntu-24
single-leg whole suite measured ~21 min and the windows 2-shard halves
~13-17 min (up to ~35 min tails on flake re-runs).

All three core combos now shard /3. partitionAll stays total for any
i/N, so the union of shards is still every file — zero coverage loss;
the wall drops to the slowest shard (~7-12 min instead of 20-25). The
win node-24 legs stay FULL-scope only. Check-run names are unchanged
(explicit job name template), and all nine core leg names already sit
in master's required-checks list.
ranxianglei pushed a commit that referenced this pull request Nov 1, 2026
…3295)

Review follow-up on this PR (owner-directed in-PR fix): after dropping master
from ci.yml's push arm, the janitor's "never cancel master" guard only matched
event==push && branch==master, so it no longer covered master's new verification
signal — the :03/:33 schedule runs. Extend the guard to also exempt master
schedule runs (push stays covered for master-build/release, which remain
push-triggered), and correct the now-stale "head's required checks" comment.
qiangu2020 pushed a commit to qiangu2020/billion-context that referenced this pull request Nov 1, 2026
…mers (ranxianglei#3295)

Owner decision 2026-11-01: every merge into master re-ran the CORE matrix
(~30 heavy shards across two OSes) whose results nobody consumes — nothing
blocks on them and nobody reads their red, while they burn minutes other
PRs are waiting for (the queue pile-up of this week was fed by exactly
these runs).

Master coverage does not lapse:
- pre-merge blocking stays intact (pull_request arm + the 14 required
  test-shard checks on PRs are untouched)
- the :03/:33 schedule cron, which re-runs the FULL matrix whenever
  master moved since the last green scheduled run, becomes master's
  standing verifier (bounded ~30 min window)
- release-ci-gate --wait already accepts green runs at EXTRA_MATCH_SHAS
  (PR head HEAD^2), and release-branch releases ride the UNCHANGED push
  arms
- master-build.yml / release.yml fire on their own master-push triggers
  independent of CI green

Release/bugfix push arms stay: bot-authored PR checks sit in
action_required, so head sha checks must come from real-token ref pushes
(ranxianglei#772). Verified: YAML parse OK; tests/release-ci-gate.test.ts 16/16
(all mock API).
@github-actions github-actions Bot mentioned this pull request Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant