fix(compression): omit serialized base64 media from compression prompts

[ian-de-marcellus]

Problem

Imported tool output can carry a whole data:<mime>;base64,... URL as text. For example, the Codex rollout importer stores function_call_output as JSON.stringify(item.output), and Codex image output is image_url: "data:image/png;base64,…". The compression image-byte cap only sees native image blocks, and the tokenizer treats base64 as prose, so a single image can turn into hundreds of thousands of tokens in an L1 or merge prompt.

Fix

• One final projection, projectCompressionWireMessages (serialized-media replacement, then the image-byte cap), applied by every compression request builder to the post-split list it actually ships: L1 canonical, L1 source-only fallback, merge, and the split-stitch rung.
• Scans text blocks and tool results in both forms (string and block array).
• Replaces data URLs of 4096+ base64 characters with a marker naming the media type and size. Matches MIME parameters before ;base64 and JSON-escaped slashes. The matcher is safe on V8 at multi-megabyte sizes.
• Prompt only: Chronicle is never mutated (copy-on-write).

Not covered

• Line-wrapped (MIME) base64.
• Bare base64 under a JSON key without a data: prefix (e.g. Anthropic-shape "source":{"data":"iVBOR…"}).

Tests

• test/compression-serialized-media-wire.test.ts captures the real membrane.complete requests: L1 canonical (and Chronicle unchanged), string-form tool_result, merge, and the split-stitch rung (forced by refusing the canonical and source-only requests). Plus matcher coverage and a ~9M-character payload under node/V8.
• On the previous head, the L1, string-form, split-stitch and V8 tests fail. The merge test passes there (that path was already correct) and guards the shared projection.
• test/compression-serialized-media.test.ts: helper-level cases.
• Full suite: 857/857 (node 22).

🤖 Generated with Claude Code

[Anarchid]

Verdict: changes requested. On the default L1 path the fix never reaches the wire

Reviewed at 2c05207 against main eca70da (0.11.0).

The idea is right: data URLs carried as text need handling in the prompt only. But the stripper is applied to an array that was already copied into the request, it skips the most common tool-result shape, and its regex can throw under Node. The merge path works. The L1 path, the one this PR describes, does not.

Blocker

1. The L1 strip rewrites a dead copy. (Sol) autobiographical.ts:5517-5526 builds split → collapsed → cleaned from llmMessages, and each step allocates new message wrappers. The new call at :5582 then reassigns m.content on the old llmMessages wrappers. mintMessages and the dispatched request are built from cleaned (:5594-5621), so they never see the change.

Repro (below; main + #114–#118 merged): an L1-only store with a 20k-char data:image/png;base64,… in one message.

build	compression requests	carrying the raw payload	carrying the marker
main	6	1	0
+ this PR, L1 only	6	1	0
+ this PR, mergeThreshold: 3	8	1 (the L1)	1 (the merge)

So "before every compression image cap (L1, …)" in the changelog is not true for L1.

Fix: apply the projection to cleaned, or run it before splitMixedToolMessages. Add a test that captures the real membrane.complete request for L1, then assert both that the marker is present and that the Chronicle source is unchanged.

Inherited, worth its own issue: main's own capCompressionImageBytes call at the same site has the same aliasing bug. With maxCompressionImageBytes: 40000 and two 30 KB images, main logs replaced 1 older image(s) with placeholders, yet the request still carries both images. Since 2026-07-12, membrane's shedOversizeImages has been the only thing actually capping L1 image bytes.

repro115.mjs (node repro115.mjs <cm-root> <tmp-store-dir> [mergeThreshold])

import { rmSync } from 'node:fs';
const [root, store, mt] = process.argv.slice(2);
const { ContextManager, AutobiographicalStrategy } = await import(`${root}/dist/src/index.js`);
rmSync(store, { recursive: true, force: true });
const requests = []; let calls = 0;
const membrane = { complete: async (req) => { requests.push(JSON.stringify(req)); calls++;
  return { stopReason: 'end_turn', content: [{ type: 'text', text: `Summary #${calls}.` }], usage: { inputTokens: 500, outputTokens: 100 } }; } };
const strategy = new AutobiographicalStrategy({ compressionModel: 'm', targetChunkTokens: 200, headWindowTokens: 0, recentWindowTokens: 0,
  autoTickOnNewMessage: false, minChunkCharsForLLM: 0, summaryParticipant: 'Claude', hierarchical: true, mergeThreshold: Number(mt ?? 1000) });
const cm = await ContextManager.open({ path: store, strategy, membrane });
const payload = 'Q'.repeat(20000);
cm.addMessage('User', [{ type: 'text', text: `look: data:image/png;base64,${payload}` }]);
for (let i = 0; i < 40; i++) cm.addMessage(i % 2 ? 'Claude' : 'User', [{ type: 'text', text: `M-${i} ${'word '.repeat(30)}` }]);
for (let r = 0; r < 40; r++) { await cm.compile(); await cm.tick(); }
console.log(`requests=${requests.length} carrying-raw-payload=${requests.filter((r) => r.includes(payload)).length} ` +
  `carrying-marker=${requests.filter((r) => r.includes('omitted from compression prompt')).length}`);
await cm.close();

The image-cap variant swaps the data-URL message for [{type:'text',text:'two pics'}, img('X'), img('Y')] with 30 KB base64 each and sets maxCompressionImageBytes: 40000.

Major

2. String-form tool_result.content is never inspected. membrane's ToolResultContent.content is string | ContentBlock[] (membrane types/content.ts:142-150), but the helper recurses only when it's an array (:10466). The shapes that actually carry this media are strings:

• connectome-host scripts/import-codex-rollout.ts:138 stores function_call_output as JSON.stringify(item.output), and Codex image output is image_url: "data:image/png;base64,…";
• the claude.ai importer (import-claudeai-export.ts:354) stringifies non-text arrays;
• AF's direct path (agent.ts:1107) stores strings.

Probe on the PR build: string tool_result → replaced=0; the same payload in array form → replaced=1. Fix: handle the string case with the same copy-on-write replacement, and test both forms through a real compression request.

3. Under Node, the regex throws RangeError: Maximum call stack size exceeded on one large data URL. V8's irregexp overflows on [A-Za-z0-9+/_=-]{4096,} once a matching run reaches ~8M characters (4M ok, 8M throws; node 22.21 / V8 12.4). That's ~6 MB of media, exactly the size class this PR targets. Bun 1.3.10 (JSC) doesn't throw, and neither does a + quantifier on either runtime. In executeMerge the throw is neither a MergeDispositionRejection nor retryable: false, so it rethrows with no attempt accounting and the entry retries on every tick. Before this PR, the same prompt would have gone out and failed with a bounded context_length quarantine. connectome-host runs on Bun, but CM's CI, AF's tests and any Node host run on V8.

const re = /data:([A-Za-z0-9][A-Za-z0-9!#$&^_.+-]*\/[A-Za-z0-9][A-Za-z0-9!#$&^_.+-]*);base64,([A-Za-z0-9+/_=-]{4096,})/g;
('data:image/png;base64,' + 'A'.repeat(8e6)).replace(re, 'x');   // node: RangeError; bun: ok

Fix: match with + and check payload.length >= minChars inside the replacer, or use an indexOf-based scanner.

4. The split-stitch rung rebuilds raw prompts with neither projection. (Sol) buildSub (:6195-6208) rebuilds from chunk.messages and applies neither the strip nor the image cap. It's opt-in (compressionSplitFallback, which connectome-host passes through), and it runs exactly when a source-only request has already failed, so it's the rung most likely to meet an oversized leaf. Fix: centralise one final compression-request projection and call it from L1, source-only, merge and buildSub.

Minor

5. Matcher coverage. It misses data:image/png;charset=…;base64,, \/- and /-escaped JSON, and MIME line-wrapped base64. Anthropic-shape image JSON ("source":{"data":"iVBOR…"}, no data: prefix) is also out of scope. That's fine if deliberate, but the body's "a foreign harness's image JSON" suggests it's covered.
6. Wiring isn't tested. All three tests call the protected helper directly, so nothing proves any call site reaches the wire. That's how #1 got through.

Verified fine

• Merge (:7268) and source-only-final (:5652) project the object that actually goes out. The source-only messages are structuredCloned first (:5509), and Chronicle-backed blocks are never mutated.
• There's no catastrophic backtracking on non-matching input: 20 MB of near-miss text runs in ~30 ms.
• generateTransitionSummary is the only other prompt builder, and it caps raw head entries at 8,000 chars. No tool-prose-hoist builder exists on main or in #114–#118.
• main + #114–#118 merged together builds and passes 852/852 (node 22).

Method

Two reviewers in parallel. Sol (Codex gpt-5.6-sol, xhigh, read-only, no network; ~8 min, 28 commands, 2.9M input tokens (2.8M cached), 25k output) read the PR worktree against main and the sibling PRs. Claude built main and the five-PR merge, ran the repros above, and checked every Sol citation. Sol only: #1 (Claude had checked that the call sites existed but not the data flow), #4, and the charset// cases in #5. Both: #2. Claude only: #3 (Sol timed only non-matching input, and the overflow needs a match), the importer evidence for #2, and the runtime proof of #1 and of the inherited image-cap bug.

[ian-de-marcellus]

Thanks, especially for #1, which helper-level tests could never have caught. Addressed in 45533f0. The branch is rebased onto current main (d1d17b0), which includes 3b324ca's fix for the inherited image-cap aliasing.

1. One wire projection. Rather than moving the call, every compression request builder (L1 canonical, source-only, merge, split-stitch) now calls a single projectCompressionWireMessages (serialized-media replacement, then the image cap) on the post-split list it actually ships. So the next builder can't repeat the bug.
2. String tool_result.content is scanned as well as the block-array form.
3. V8: the matcher uses +, with the 4096 floor checked in the replacer. A test runs a ~9M-character payload under node.
4. Split-stitch rung: now projected. The test forces the rung by refusing the canonical and source-only requests, then inspects the sub-requests.
5. Coverage: MIME parameters before ;base64 and JSON-escaped slashes are now handled. Line-wrapped base64 and bare base64 under a JSON key (Anthropic "source":{"data":…}) are out of scope, and the body now says so.
6. Wiring: new compression-serialized-media-wire.test.ts captures the real membrane.complete requests per path and checks Chronicle is unchanged. On the previous head the L1, string-form, split-stitch and V8 tests fail. The merge test passes there (that path was already right) and now guards the shared projection.

Full suite: 857/857 (node 22).

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 3/5

[Medium risk] Refactors compression prompt building to strip embedded media.

The PR should not merge until recall-curve fallbacks use a consistent projected representation.

Findings

1.  Projected answers hide recall pairs ▶
2.  Fallbacks restore omitted media ▶

Fix with agent prompt

### Issue 1
src/strategies/autobiographical.ts:5588
When a stored L2 summary contains a data URL of at least 4096 characters, this call replaces the URL in the canonical request. The recall-curve builder then searches that request for the original, unprojected answer and cannot find it. If the canonical request is refused, its recall-expansion fallbacks are unavailable, so a chunk that could otherwise be compressed may be quarantined. Compare the pair in the same projected form.

### Issue 2
src/strategies/autobiographical.ts:10427-10429
When a recall-curve fallback inserts a stored child summary containing a large data URL, it does so after the canonical messages have been projected. The completed variant is sent without this projection, so a canonical refusal can put the full base64 payload back on the wire. Project each completed variant before admission and dispatch.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR adds a copy-on-write projection that removes large serialized data URLs from compression messages and caps native image bytes on the messages sent by the principal L1, merge, and split-stitch builders.

• Recall-curve fallbacks remain inconsistent with that projection: stored parent media can prevent a fallback from being built, and stored child media can return unprojected.
• Split-stitch targeting and the source-only wire assertion need follow-up.

_{Reviews (1) · Last reviewed commit: "fix(compression): one wire projection fo..."}

[greptile-apps]

Projected answers hide recall pairs When a stored L2 summary contains a data URL of at least 4096 characters, this call replaces the URL in the canonical request. The recall-curve builder then searches that request for the original, unprojected answer and cannot find it. If the canonical request is refused, its recall-expansion fallbacks are unavailable, so a chunk that could otherwise be compressed may be quarantined. Compare the pair in the same projected form.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/strategies/autobiographical.ts
Line: 5588

Comment:
**Projected answers hide recall pairs** When a stored L2 summary contains a data URL of at least 4096 characters, this call replaces the URL in the canonical request. The recall-curve builder then searches that request for the original, unprojected answer and cannot find it. If the canonical request is refused, its recall-expansion fallbacks are unavailable, so a chunk that could otherwise be compressed may be quarantined. Compare the pair in the same projected form.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Fallbacks restore omitted media When a recall-curve fallback inserts a stored child summary containing a large data URL, it does so after the canonical messages have been projected. The completed variant is sent without this projection, so a canonical refusal can put the full base64 payload back on the wire. Project each completed variant before admission and dispatch.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/strategies/autobiographical.ts
Line: 10427-10429

Comment:
**Fallbacks restore omitted media** When a recall-curve fallback inserts a stored child summary containing a large data URL, it does so after the canonical messages have been projected. The completed variant is sent without this projection, so a canonical refusal can put the full base64 payload back on the wire. Project each completed variant before admission and dispatch.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.