Repository navigation
feat: image recovery channel for stripImages (#1995) - #2022
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2428-pr.2022.bfa4adeThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2022.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 PR #2022 审查完成。审查中发现两处首轮遗漏(均已按「小问题直接修」以追加 commit 修到 PR 分支),CI 现已全绿。判定:可合并(人工合并)。 结构化七问
需要 owner 拍板的三件事
验证证据:本地 typecheck 干净 + 定向套件(image-restore 7/7、wire-contract、strip-images×2、decompress-empty-range、proxy-decompress-cache、wire-schema-portable、fix-decompress-reaper/tofile)全绿; |
64172c5 to
75259ca
Compare
|
Follow-up from review — cross-session collision fix (blocker): mNNNNN refs are per-session sequence numbers, so the flat
Branch rebased onto current master ( Still open from review (not addressed in this commit): tool-result-nested images (anthropic |
[bot] 🏷 第二轮审查完成(head ① 是什么问题 ② 什么原因导致的 ③ 怎么修复的(机制层面,已读 diff 核验)
④ 是否有回归问题:无。既有行为清点:wire-contract goldens 相对新 master 的 delta 只有 decompress 描述尾注 + imageRef 属性(5 个变体全覆盖,无杂项);持久化格式未动(incomingImageIndex 仍不落盘);非 stripImages 会话字节级不变(stripImages off 时索引构建整段跳过并清空);剥离行为本身未变。rebase 后 ⑤ 是否破坏缓存:不涉及前缀缓存(索引构建不改 wire 字节,剥离时机不变);磁盘侧 skip-if-exists 使文件稳定不重写。已知限制: ⑥ 是否改了设计方向:没有,同一机制族原位加固。tool_result 嵌套图(anthropic ⑦ 是否引入新配置:不涉及。 两个次要发现(均不阻塞):
判定:blocker 修复正确且有牙测试钉住,round-1 内存问题已消解,CI 全绿。可以合入——按 auto-merge 门禁(feat + wire 形状变更)需人工合并;建议先做上面第 2 点的最终 rebase。 |
|
Second review finding fixed (
Tests: per-wire nested extraction (multi-image order, pdf part untouched, fileData skip) + end-to-end anthropic tool_result recovery through the index builder — the screenshot is spilled under the tool message's ref and the pixels round-trip. image-restore 10/10; strip-images / image / wire-contract / decompress family green; tsc clean. With this, every shape the strip side removes has a recovery path except the two no-bytes classes (non-data URLs, Files-API fileData). |
[bot] 🏷 Round-3 review — 1. 是什么问题 2. 什么原因导致的 3. 怎么修复的 4. 是否有回归问题 5. 是否破坏缓存 6. 是否改了设计方向 7. 是否引入了新的配置 证据:本地 typecheck clean;定向套件 56/56(image-restore 10/10 含新增 nested 用例);CI 在 仍 open(round-2 的 advisory 小项,未处理,非阻塞):① sanitized-id 碰撞残差(两个不同 raw id 净化后可映射同目录,如 合并判定:机制本身审查通过,可合并。但分支现在落后 master 15 个 commit,其中 #2271(persona continuity,+44/−8)动了 |
|
Follow-up to the two review commits (75259ca session-namespacing, 0a50040 nested tool-result indexing): gap 2 is now implemented on this branch (ecbe6b4) — the strip boundary is fold-anchored on anthropic instead of sliding, so the forwarded body's image-strip prefix is byte-stable between compression events and the prompt cache stops re-billing the window every turn. Design notes in the PR body (kernel changes: |
[bot] 🏷 第四轮审查结论(gap 2:折叠锚定剥离边界,ecbe6b4a) 机制审查通过。逐层核对了实现与 kernel 侧实际代码( ① 是什么问题 ② 什么原因导致的 ③ 怎么修复的 ④ 是否有回归问题
⑤ 是否破坏缓存 ⑥ 是否改了设计方向 ⑦ 是否引入了新的配置 本轮我直接修掉的两个小项(已推分支,独立标注 commit):
修后复验:typecheck clean + 相关套件 25/25 绿。push 后新 head 仍 open 的小项(不阻塞,延续前几轮清单): ① sanitize 后 session id 碰撞残差( 合并判定:机制审查通过,测试有牙,本地定向验证绿,CI 在新 head 上跑。但含 kernel 变更 → 永远 human-gate(§7.4 + Kernel Boundary),且分支落后 master 15 个 commit,热文件重叠( |
stripImages drops historical image parts with no way back: decompress is text-only and image_full (#1095) only restores downsampled originals, so once stripped the pixels are gone from the model's view forever — while the sliding keep-recent window rewrites one message per turn and busts the prompt cache on the most expensive bytes (the pi#593 cost source). Either way the user pays. Add a recovery channel. Each request indexes the images still present in the client's resent raw history by their mNNNNN ref (via stable message ids), gated on stripImages so non-strip sessions pay nothing. decompress gains an optional imageRef: "list" enumerates what is restorable; an mNNNNN ref writes the original bytes to a host-readable file under <stateDir>/retrieve/img/ and returns the path for the model's read tool. File-first delivery is uniform across all four wires — no per-protocol inline-image rendering. The index is built from the UNSTRIPPED body before strip drops the parts, and cleared when stripImages is off. Wires imageRef into every decompress variant (range and no-range alike; it is CCR-independent, unlike startId/endId). Regenerates the wire-contract goldens accordingly — verified the diff is exactly the decompress description note plus the imageRef property, nothing else. Proxy-mode coverage: the proxy never mutates client state, so every inbound request carries full raw history and stripped pixels are always recoverable. Native agents own their own decompress (their self-folded images are theirs, cf pi#594); this recovers what the PROXY strips. Gap2 (reducing cache-bust frequency via a compression-anchored strip boundary) stays tracked as the remaining facet of #1995 — a token-vs-cache tradeoff pending owner sign-off, not shipped here. Closes #1995
…ageRef addition The PR added an optional imageRef param plus an exact description note to decompress in all wire shapes but left the #1179 drift guard in tests/search-context-cross-session.test.ts pinning 'kernel + startId/endId only', so CI failed on all four test-matrix jobs. The guard now treats imageRef as a third sanctioned bili-only optional param and strips the note by exact suffix match (wording drift still fails the test).
…bits)
writeRestoredImage passes {mode:0o600} which NTFS does not apply, so the
restored-file permission check failed both Windows test-matrix jobs with
438 !== 384. Byte round-trip and idempotency assertions stay platform-wide.
…x-time The review of #1995 found incomingImageIndex pinned every resent image's base64 in session memory for the session's lifetime — for screenshot-dense sessions that is ~cumulative image bytes, unbounded. Fix at the root: spill each image to <stateDir>/retrieve/img/ at index-time (skip-if-exists) and retain only metadata + path (IndexedImage), so residency is O(refs) regardless of image volume. decompress({ imageRef }) now returns the stored path (a pure lookup, never re-writing bytes); an externally-evicted file degrades to an explicit 're-spilled on next request' message instead of a dangling path. Non-strip sessions still pay nothing (index build gated off). No tool-schema change, so wire-contract goldens are untouched.
mNNNNN refs are per-session sequence numbers, so the flat
<stateDir>/retrieve/img/<ref>.png layout let two sessions' same-numbered
refs collide: with skip-if-exists, whichever session spilled first owned
the file forever, and every other session's decompress({imageRef}) for
that number returned the FIRST session's pixels — with listing metadata
(dims/bytes) describing the caller's own image, so the mismatch was
invisible to the model. Adjacent CCR exports avoid this by rewriting on
every retrieve; recovery here stays skip-if-exists (refs are
deterministic within one session) and takes the other half of the fix:
the session id is now part of the path.
- restoreExportDir(sessionId) -> <stateDir>/retrieve/img/<sessionId>/
- buildIncomingImageIndex / writeRestoredImage take the session id
- server.ts passes session.id at the index call site
- regression test: two sessions, identical ref numbering, distinct
bytes -> distinct files, each restores its own pixels
Also resolves the rebase onto master's ProxyToolResult migration:
resolveImageRestore now returns toolOk/toolFail outcomes.
…erable (#1995 review) stripNestedImages removes images from anthropic tool_result.content, Responses function_call_output/custom_tool_call_output .output, and Gemini functionResponse.parts — but messageImageBytes only walked the plain-content sidecars, so that whole class was stripped with no recovery path (the exact #1995 failure, just for nested shapes). - anthropic: walk tool_result.content[] image blocks (base64 + data-URL) - responses: pick the parts array by item type (.output for function_call_output / custom_tool_call_output, .content otherwise), mirroring the strip side's key choice exactly - google: descend into functionResponse.parts for image-mime inlineData; nested fileData stays unindexable (remote reference, no bytes transit the proxy — same disclosed class as URL-sourced images) Tests: per-wire nested extraction (incl. pdf part untouched, fileData skip) + end-to-end anthropic tool_result recovery through the index builder (spilled under the tool message's ref, pixels round-trip).
…store-file eviction (#1995 gap 2) Kernel: - stripHistoricalImages gains opts.cutoffIndex (protocol-neutral): an explicit wire-message index replaces the sliding len-keepRecent cutoff, so a host can pin the strip boundary and keep the stripped prefix byte-stable between compression events (prompt-cache stability — the sliding window breaks the cache at the image bytes every turn). - anthropicToCore tags every flattened core message with its source wire ordinal (BiliMessage.wireIndex) so hosts can map fold coverage (core ids) back to wire positions without re-deriving codec split behavior. - image-only anthropic tool_results collapse to the empty string instead of a "[image]" text part: the derived id is seeded from concatenated TEXT parts, so "" keeps it byte-identical across the strip ("[image]" would flip it). - kernel 0.0.104 -> 0.0.105. Host: - foldAnchoredCutoff (anthropic only — the other wires flip ids on strip and keep the sliding window until placeholders are id-neutral): with an active fold, the strip cutoff anchors to the last covered wire message; a partially-covered boundary wire message (live image in the same message) is never stripped. Falls back to the sliding window when nothing is covered. - server wires the anchor into stripHistoricalImages; count_tokens requests keep the same cutoff (token parity) but skip the bookkeeping index rebuild. - summary image notes now carry the ref ([image: png 1024x768 · m00042]) so the fold's own text points at decompress({imageRef}) long after the wire dropped the pixels. - pruneRetrieveImgExports: best-effort 7-day TTL eviction for spilled restore files (throttled to once a minute in the request path; emptied dirs removed). Tests: kernel wire-strip-cutoff (5), host strip-cutoff-anchor (7); existing kernel pins updated for the empty-string tool_result collapse.
…d cutoff + recovery channel
…not the artifact
…ns, GC the image-spill tree, mirror the Gemini $ref guard, ref footers on model-driven folds
f2ced6f to
59c49cf
Compare
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.91-master.433def4 # this exact build
npm install -g billion-context@master # always the newest merged build
|
What & why
compress.stripImages(opt-in,src/config.ts, defaultstripImagesKeepRecent=5) drops historical image parts from every forwarded request so image-dense sessions stay lean. Today that drop has no way back:decompressis text-only andimage_full(#1095) only restores downsampled originals — a strip-removed pixel is gone from the model's view forever. Meanwhile the sliding keep-recent window rewrites one message each turn, busting the prompt cache on the most expensive bytes (the billion-context-pi#593 cost source). The user is forced to pick between burning context or burning cache and going permanently blind to old images.This adds the recovery channel so stripping becomes reversible: pick lean context, recover any old image on demand.
What this PR does
mNNNNNref (buildIncomingImageIndex, newsrc/image-restore.ts). Built from the unstripped body beforestripHistoricalImagesdrops the parts. Gated onstripImagesand cleared when it's off, so non-strip sessions pay nothing. Each indexed image is spilled to disk immediately (skip-if-exists) and only its metadata + path is retained, so session memory stays proportional to the number of images, not their byte size.decompressgains an optionalimageRef:"list"enumerates what is restorable; anmNNNNNref writes the original bytes to a host-readable file under<stateDir>/retrieve/img/and returns the path for the model'sreadtool.The proxy never mutates client state, so every inbound request carries full raw history and stripped pixels are always recoverable. Native agents own their own
decompress(their self-folded images are theirs, cf pi#594); this recovers what the proxy strips.Config surface
No new config keys. v1 deliberately adds none: the restore location is fixed under the existing
<stateDir>/retrieve/tree (same home asacp_retrieve's spill), and the inline-vs-file split is not exposed as a knob yet — file-first is the safe default and a budget knob can be added later once the shape is proven.stripImages/stripImagesKeepRecentsemantics are unchanged.Behavior changes
decompress.imageRef; omitting it preserves today's behavior exactly.<stateDir>/retrieve/img/at index-time and keeps only metadata + path, capping residency at O(#images). Restore-by-ref is now a pure path lookup (no re-write); an externally-evicted file reports "re-spilled on next request" instead of returning a dangling path. Non-strip sessions are unaffected (index build gated off).decompresstool description changed (doc-only): one sentence appended advertisingimageRef. Byte-visible in the served schema, no semantic change.stripImagesbehavior DID change in gap2 (correcting an earlier draft that said otherwise): with an active fold on the anthropic wire, the strip boundary is now fold-anchored instead of purely count-driven (see "Gap2" below) and summary image notes carry refs — output format changed for fold-anchored anthropic sessions. No-active-fold and non-anthropic sessions keep the sliding window with byte-identical behavior.Both compression modes
Wire-contract goldens (#1304)
Regenerated
bili.json+ both manifests viascripts/update-wire-contract-goldens.ts. Verified the diff is exactly thedecompressdescription note plus the newimageRefproperty (and its closing brace) — no other field moved.kernel.jsonis unchanged (no kernel tools touched).Gap2 — fold-anchored strip boundary (implemented, anthropic)
The sliding window moved the strip boundary every turn, re-billing the window's image bytes as cache writes forever (billion-context-pi#593). With an active compression fold the cutoff now anchors to fold coverage (
foldAnchoredCutoff,src/image-restore.ts): strip exactly the wire messages the fold covers, keep everything after it verbatim — the stripped prefix is byte-stable between compression events, so the cache break stays put. A partially-covered boundary wire message (a live image in the same message the fold stopped before) is never stripped. No active fold → sliding-window fallback;count_tokensuses the same cutoff (token parity) but skips the index rebuild.This is anthropic-only on purpose: the other three wires flip kernel message ids when stripping (openai/google collapse to "" vs "[image]", responses concatenates text+"[image]"), so anchoring there would orphan fold coverage at T+1. They keep the sliding window until their strip placeholders are made id-neutral — the kernel side needed for anthropic is in this PR:
stripHistoricalImagesopts.cutoffIndex(protocol-neutral explicit boundary)anthropicToCoretags each flattened core message with its source wire ordinal (BiliMessage.wireIndex)Plus the two remaining review asks: summary image notes now carry the ref (
[image: png 1024x768 · m00042]) so the fold's own text points atdecompress({ imageRef })after the wire drops the pixels, and spilled restore files get best-effort 7-day TTL eviction (pruneRetrieveImgExports, throttled to 1/min; emptied dirs removed).kernel-change: wire strip-images cutoff option + anthropic wireIndex tagging + empty-string tool_result collapse; kernel 0.0.104 → 0.0.105.
Review round 4 follow-ups (all five open items addressed)
restoreExportDirsalted: any session id the sanitizer changes (or truncates at 100 chars) gets-<8 hex of sha256(raw id)>appended, soa/banda-bno longer share a spill dir. Deterministic → restart-stable; already-pure ids keep their historical path (test:restoreExportDirName).session-gc.ts): deleting a session file co-deletes itsretrieve/img/<sessionId>/dir (same lifecycle as the [Bug] Session-file GC (#1082) never removes #1097 .content-store.json companions — orphaned ref files accumulate #1180 content-store companions), and a stale-mtime pass sweeps orphaned dirs past the same age gate. Spills are a pure cache — a live session that loses its dir self-heals by re-spilling on the next request (item 3's mechanism, now explicit). Opt-inBILI_SESSION_GCas always;session-gc.test.tsnow isolatesXDG_STATE_HOMEfor the whole file so sweeps under test never touch a real spill tree.stream.tsapplyRanges): preflight summaries already render ref-carrying image notes; a MODEL-written fold covering sidecar images (e.g. responses tool-output screenshots — foldable by design) now gets a compact host-appended footer —[folded images: [image: png 1x1 · m00002] — decompress({ imageRef }) restores pixels]— so the restore channel stays discoverable in that path too. Appended after the kernel's length validation; no footer when the fold covers no images.$refguard mirrored (messageImageBytes): functionResponse parts whoseresponseuses{"$ref": displayName}are skipped at index time, matching the strip side's guard (wasted indexing, never corruption).Also from the review: rebased onto current master (was 15 behind,
src/server.ts/src/session.tsoverlap) — clean, no conflicts.Closes #1995