Skip to content

feat: image pre-compression routing/downsample + image_full restore channel - #357

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-09-23_image-precompression
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-09-23_image-precompression

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #353

What

Kernel-side half of image pre-compression (billion-context#1095 analysis): pure deterministic routing/downsample decisions + an image_full restore channel. Zero native deps, zero file I/O — encode execution, sidecar I/O, and wire injection stay host-side.

New module src/image-compress.ts, exported from the barrel:

Routing & estimation

  • decideImageRoute(meta, config, classifier?) — pure deterministic decision fn; check order: disabled → not-an-image → below-min-tokens → not-screenshot-like → downsample (with resolved recipe {maxDimension, quality, format}).
  • Billing-aware token estimator ported from billion-context src/image-tokens.ts: header-only dimension parsing (PNG/GIF/BMP/WebP VP8X|VP8L|VP8/JPEG SOF scan, budgeted base64 head) + OpenAI high-detail tile model (short side scaled up to 768, long side capped at 2048, 85 + 170·tiles); unknown dims → 16384 fallback; bytes billing → ceil(b64len/4).
  • Pluggable ScreenshotClassifier; v1 default heuristic: aspect ratio band [1.4, 2.6] + short-side floor 720px. A trained router can slot in later without touching pipeline code.

image_full restore channel

  • Tool contract ×3 wire shapes (IMAGE_FULL_TOOL / _OPENAI / _RESPONSES), args {ref: "mNNNNN"}.
  • applyImageFull({ref, state, config}) — success / idempotent-already-restored / failure texts ([image_full FAILED: ...]); sticky per-session restored-ref set.
  • Byte-stable buildImageFullSystemNote(count) fragment for host prompt injection.
  • Shrink records + cumulative stats counters on session state (imagesShrunk / imageBytesSaved / imageTokensSaved); resetImageFullState() clears ref-keyed entries (hosts MUST call it on compaction reset / fork recovery / session restart — refs re-issue from m00001).
  • Store entry-kind contract ImageStoreEntry {kind:"image", ...} for the not-yet-merged content store (#1097).

Config

  • imageCompression?: {enabled, minTokens=512, maxDimension=1280, quality=80, format="webp"} — defaults + validation in src/config.ts, sub-field merge exactly like absorb; host-side 3-level merge unchanged.

Answers to the issue's open questions

  1. Placement — arrival-time substitution vs per-forward: arrival-time. Determinism + prefix-cache stability: the substitution happens once when the message enters the session, so every downstream forward sees identical bytes.
  2. v1 heuristic params: kept the aspect band [1.4, 2.6] + floor per the owner-approved spec (non-screenshot originals must pass through untouched; size-only gating would violate that). Known limitation documented in the module header: 3:2 photos (ratio 1.5) false-positive until a trained router slots into the classifier interface. Floor 720px = HD short side; excludes small crops the tile model would upscale.
  3. Estimator location: canonical copy moves to the kernel (kernel owns the math); hosts keep their body-scanning wrapper calling the kernel's per-image estimator.

Behavior changes (disclosure)

  • None user-visible. Feature is OFF by default ⇒ byte-identical pass-through (regression-tested: full pass-through battery under enabled:false).
  • Additive exports only; no existing signature changed.
  • Fresh states now seed 3 zero-valued stats counters + 2 empty arrays, carried through mergeCompressionState. Old persisted states untouched (all readers use ??); one pre-existing test literal updated to match the new seeded zeros.

Verification

  • npm run typecheck clean · npm test 800/800 green (incl. new suite: routing matrix, crafted container headers, determinism loop, image_full failure modes, config/state/persist plumbing) · npm run build clean.

Host follow-ups stay in billion-context: sharp encode execution, sidecar I/O, wire injection ×4 protocols, logging, packaging (#1095/#1097 integration).

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewing this PR now (agent-completed work, not redoing): pulling branch 2026-09-23_image-precompression, verifying the diff against #353's acceptance criteria, running typecheck/tests/build locally, and checking CI status. Full review conclusion (diff cleanliness + regression pass) coming next.

@ranxianglei
ranxianglei force-pushed the 2026-09-23_image-precompression branch from 141563b to 5095ec7 Compare September 22, 2026 18:39
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-23_image-precompression (5095ec7)

Option A — Install from npm PR tag (recommended)

In your adapter project:

npm install acp-kernel@pr-357

Each push to this PR publishes a new version under the pr-357 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf acp-kernel-pr357.tgz
npm install ./package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — rebuilt on current master, CI green, ready to merge.

What I verified

  • Audited the full diff (9 files, +1336/−2) against Image pre-compression: routing/downsample strategy + image_full restore channel (billion-context#1095) #353's acceptance criteria: routing order (disabled → not-an-image → below-min-tokens → not-screenshot-like → downsample), estimator math, image_full state machine (success / idempotent-already-restored / failure texts, sticky restored-ref set, reset contract), config defaults + validation + sub-field merge, state seeding + persist carry, barrel exports. All present and correct.
  • Cross-checked the WebP dimension parser against canonical libwebp (vp8_dec.c VP8GetHeaders, vp8l_dec.c ReadImageInfo) and the billion-context reference (src/image-tokens.ts): bit layouts match exactly (frame tag precedes sync code, u16le width/height @ offsets 26/28 masked 0x3fff; VP8L signature → w−1 (14b) → h−1 (14b) → alpha → version). PNG/GIF/BMP/JPEG offsets match spec layouts; length guards exact. No bugs found.
  • Local run at the original head: typecheck clean · 800/800 tests · build clean (matches the PR's claim).

Blocker found & fixed directly

Stale base. The branch was based on v0.0.73 (8ee1801); master had moved 57 commits to v0.0.82 (b30723f) and all 6 of the PR's support files had changed on master (rules / terminal-escape / Gemini wire shapes) → GitHub reported mergeable_state: dirty and no CI ran on the original head.

  • Rebased onto b30723f. All 6 conflicts were benign both-sides-appended-at-the-same-spot (master's features vs the image feature); resolved by keeping both sides. Git ate shared context lines at 4 seams (tsc caught each): missing export { in src/index.ts, missing } closing RulesConfig in src/types.ts, two missing braces in src/config.ts, missing }; closing ABSORB_TOOL_GOOGLE in src/compress-tools.ts — all restored.
  • Also fixed one nit: src/image-compress.ts:147 comment said VP8X "flags(4) reserved(3)" — corrected to describe the actual 4-byte flags field (code offsets were already correct).
  • Post-rebase local: typecheck clean · 903/903 tests (master grew the suite 800→903; includes both the rules/terminal-escape and image-compression suites, proving no semantic clobber) · build clean.
  • Pushed rebased head 5095ec7; GitHub CI on 5095ec7: CI ✅ + PR Build Artifact ✅; mergeable_state: clean.

Regression pass (existing behaviors on touched paths)

  • src/compress-tools.ts — ACP_TOOLS_* arrays untouched; master's ACP_TOOLS_GOOGLE intact; image_full constants are opt-in, host-injected only when enabled. Wire-artifact formats unchanged.
  • src/config.ts — existing defaults/validation unchanged (truncate.terminalEscapeAfter, rules.*, protectedLatestTools all preserved); new validation fires only when the imageCompression key is present.
  • src/state.ts / src/persist/state-merge.ts — additive seeding/carry with ?? fallbacks; master's rules/nextRuleId/terminalStreak coexist with the PR's imageFullRestored/imageShrinks; old persisted states read safely (all readers use ??).
  • src/types.ts / src/index.ts — purely additive types/exports; CompressionStats gains 3 required numeric fields, seeded zero in fresh states (intentional, disclosed in the PR; one pre-existing test literal updated to match).
  • Untouched: pipeline node ordering, nudge, prompts, wire codec, ref allocation, protected-tool filtering — all §7.3 kernel contracts intact.
  • Coexistence spot-checked: both feature sets present in every merged file.

Behavior-change disclosure

  • My rebase introduced zero behavior drift — the final diff vs master is exactly the PR's original 9-file diff (+1336/−2); conflict resolution = union of intent.
  • The PR's own disclosure stands and is accurate: feature OFF by default ⇒ byte-identical pass-through; additive exports only; fresh states seed 3 zero counters + 2 empty arrays; old persisted states untouched.

Not blocking, noted separately

Per AGENTS.md §7.4 this stays human-gated (feat + state-schema change), so: ready for you to merge.

修了什么问题 / 为什么修 / 是否可以合并:PR 代码本身正确(WebP estimator 与 libwebp 及 billion-context 参考实现逐一核对无误),唯一问题是分支基于旧 master(v0.0.73)导致无法合并——已直接 rebase 到 v0.0.82、解决 6 个文件的纯叠加冲突并修复 4 处合并接缝,本地与 GitHub CI 全绿(903/903)、无行为回归,可以合并。

PR: #357 (head 5095ec7)

@ranxianglei

Copy link
Copy Markdown
Owner Author

Rebased onto current master (v0.0.83 + #359 crush gate) and resolved all 6 conflict hunks as a union — both crush and imageCompression coexist in config defaults, merge, validation, types, and exports.

Pre-flight on the rebased head: typecheck ✅ / 952 tests pass ✅ / build ✅

Fixed by #363 (supersedes the stale #353 branch).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image pre-compression: routing/downsample strategy + image_full restore channel (billion-context#1095)

1 participant