Skip to content

fix(images): count every image in the live window's limits, price a generated image as an image, and estimate by the strip's depth rule - #219

Merged
nissa-seru merged 9 commits into
anima-research:mainfrom
nissa-seru:nell/live-image-accounting
Oct 22, 2026
Merged

nissa-seru merged 9 commits into
anima-research:mainfrom
nissa-seru:nell/live-image-accounting

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The live window's image limits, maxLiveImages and maxLiveImageBytes, counted only top-level image blocks, and only a base64 source's bytes, while the compression image cap counted every image a prompt can carry. Images nested in tool results and generated images weren't limited at all, and a data: URL image counted toward the number but not the bytes, so a request could carry more image bytes than the operator set. A generated_image was priced at 0 tokens, though membrane sends it to the provider as an image, so a layout holding one ran larger than its estimate. And the post-strip estimates and the image watermark disagreed with the strip about the message whose tokens cross imageStripDepthTokens: the strip replaced its images while the estimate priced them live.

Changes

!Fixes #294.

The live window's image limits now count the images the compression image cap counts, a generated image is priced like an image and weighed like one at the top level, the post-strip estimates agree with the strip about which message is beyond the depth and which images take the live slots, and the windowed strategy prices its window after its image policy.

What changes for operators

  • Every image block now counts toward maxLiveImages, whatever its source (a remote URL with no inline bytes included), and so does every generated_image, top-level or nested in a tool result. maxLiveImageBytes counts a data: URL's inline bytes, and a generated image's. That holds in both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates. A window holding such images keeps fewer of them live than before, and the rest become the window's placeholder. Each message's images are counted as it's sent, pruned and cut, so a tool result toolResultMaxLastN replaces, which goes out as its marker, holds none, and neither does an image an override of truncateContent leaves out.
  • A generated_image is priced at 1,600 tokens, as an image without a tokenEstimate is. Membrane's GeneratedImageContent carries no tokenEstimate. Stores holding generated images now estimate larger and may fold or strip more, and the fold planner weighs a top-level generated image as an image. One nested in a tool result is weighed as part of that result, which the planner already counts as fully externalized.
  • Since that changes the store's pricing, the calibration pricing epoch is now 5, after main's 3 and 4. In a store holding a generated image, top-level or nested, a multiplier learned under any earlier epoch, or with no stamp (0.13.0 or earlier), is discarded: it restarts from 1, which the first provider sample within the calibration's sanity band sets. A provisional 1 there, the restart of an earlier discard that no sample has confirmed yet, is kept instead, re-stamped and still provisional, with the reset that records what it replaced. In a store without a generated image, this changes nothing about whether a multiplier is kept: one stamped with epoch 3, as 0.17.1 writes it, is kept and re-stamped unless what main's epoch 4 repriced is in the store, with its provisional flag if it had one.
  • The post-strip estimates and the image watermark count a message's own tokens toward its depth, as the strip does. Before, with imageStripHysteresisRatio, the watermark didn't count the images of the message that crosses the depth as over it until a newer message arrived.
  • Where adaptiveResolution is on, the autobiographical strategy's strip counts the images of a message no entry sends, such as one a fold left out, as the post-strip estimates, which that selection budgets by, always have. Before, such a message's images took live slots in the estimate and none in the strip, so the window kept images its estimate had priced as the placeholder, and sent more than it planned for. Count and bytes go newest-first, and a folded chunk is older than the recent window, so the images this newly strips sit in raw messages older than a folded one, such as the head window's, whose slot a folded message's image takes. A window with folds can then keep fewer live images than its limits allow. In return, it no longer sends more image than it planned for. The hierarchical selection, the default, prices the messages it emits at the store's full estimate of what it sends, plus what a maxMessageTokens cut changes. It prices no image it emits as the placeholder, so its strip still counts only the images it sends. Its recent window's start and the image watermark count every stored message's images, as sent.
  • Where pairing repair moves a late tool result that holds an image into an earlier call's entry, the adaptive selection's strip now gives that image the verdict the estimate gave it in its own message. Before, it counted the images it sends in entry order, so it could keep live an image the estimate had priced as the placeholder and send more than the plan, or count a moved image twice and replace an older one the estimate kept (The live-image strip can count an image twice when pairing repair moves the tool result that holds it #294). Two results for one call id in a single message, which pairing repair takes not to happen, can still take each other's verdicts.
  • The post-strip estimates take a stripped image's placeholder at the store's price for its text, where they took a token per four of its 54 characters: 14, against 19 by the store's default estimator. Each stripped image came off 5 tokens too much, times the calibration. An image priced below the placeholder, such as one stamped with a small tokenEstimate, now makes its message cost more when it's stripped, where its saving was clamped at 0.
  • The windowed passthrough strategy prices its window after its image policy, as its documentation says. Its fit check and hard-budget check priced every image at full, though the strip then replaced some, so a message whose images the policy strips was refused with OverBudgetError where it fits once stripped, and a window holding stripped images moved its anchor before it had to. Since this change prices a generated image, that refusal would otherwise newly reach messages holding one.
  • A stored window that any of this re-renders, by counts that now bind, a generated image's new price, a discarded multiplier, a watermark the depth rule moves, a folded message's images taking live slots or the placeholder's price, misses the prompt cache on its first request after the upgrade.
  • agent-framework stores a tool result as text, with its images as placeholders (buildStoredToolResultContent, at agent-framework's main b96e918b), so on agent-framework hosts the nested count finds nothing to count. A generated image in a message's own content is counted and priced from now on.

In the code

  • src/inline-images.ts is one walker for both strategies' strip and the compression cap: what an image is (image, generated_image), its inline size (the former compressionImageBytes), the images a block list carries in order, nested tool results included, and replaceImages. That copies a block it edits, and a block that gives up a replay form it shared with an edited one.
  • The autobiographical strategy's readers (hasImageBlock, applyImageStripping, the watermark's first misfit, postStripEstimates) and its compression cap use it, and so does the windowed passthrough strategy's strip. AutobiographicalStrategy.imageBlockBytes, a protected static since 0.5.9, now returns the walker's inline size, so its numbers change for data: URLs and generated images, whose own data it now counts. compressionImageBytes, the protected static 0.15.0 shipped for the compression cap, stays as a wrapper over the same walker, with the numbers it had.
  • A tool result whose nested image is replaced gives up its replay form (rawItem, rawXml), which still holds the image, and so does every block of its message that shares that form. A top-level image the store keeps as a blob comes back without a replay form; a url-sourced image or a generated image keeps its own, which the strip releases like any other.
  • MessageStore and ContextLog's mirror price generated_image at 1,600 tokens, and so does postStripEstimates, whatever the block carries. The fold planner's static salience weighs a top-level one as an image, and formatChunkForCompression marks one [Image].
  • CALIBRATION_PRICING_EPOCH is 5, with a CALIBRATION_EPOCH_REPRICES entry that's true for a store holding any generated image, nested ones included. A kept and re-stamped multiplier keeps its provisional flag. loadCalibration keeps and re-stamps an older epoch's provisional record even where content was repriced since, rather than discarding it again, and logs a line of its own for it.
  • postStripEstimates and the watermark's first misfit add a message's tokens before comparing with the depth, as getImageStripStart does.
  • applyImageStripping in the autobiographical strategy takes asEstimated, which selectAdaptive passes and selectHierarchical doesn't. Asked, it doesn't count the images it sends: it applies the verdict the post-strip estimates gave each image of each stored message's cut. An image's verdict is its own message's: the entry's source's, or, for an image in a tool result that pairing repair moved into another entry, the message the repair moved it from, which the repair now records. Within the message, a tool result's images, and the top-level images, take that group's verdicts in order, which the cut and the repair keep, copying a block at most. A verdict whose image had another payload (a base64 source's data, a URL source's URL, a generated image's data) isn't this image's. Such an image, or one with no verdict, goes out as the placeholder. A folded message's images took their verdicts in the estimate, and no entry sends them. The kv-unified cache check's rebuild of what a compile sent (kvUnifiedKeepEntries) asks too, so it strips as selectAdaptive did. The parameter's doc says why only a caller that prices by the post-strip estimates asks, and names the exception to selectAdaptive's pricing: the middle region's text-only entry of a sharded body, priced by its text.
  • The windowed strategy decides its policy's verdicts once per selection, from the newest message down to the anchor (liveImagePolicy), prices each message after them in its fit check and hard-budget check, and applies them to what it emits. The method's doc says why the verdicts it prices are the ones the strip applies wherever the window starts, and what a later change to the strip must keep true for that. A message maxMessageTokens cuts is priced at its price after the verdicts plus what the cut adds (cappedPriceExtra). renderAsNewest applies the same verdicts, through stripImages, which select uses too.
  • postStripEstimates and the watermark count a message's images in the form its cut sends (cappedForSend), which is the form the entries send. So under any cut, including an override of truncateContent that leaves an image out, the verdicts are given to the images that go out, and the price at each site that adds what the cut changes is the post-strip estimate of what's sent. Two sites add nothing for the cut: the recent window's size, which no window measures after the cap, and the middle region's text-only entry of a sharded body, which sends its shards uncut. truncateContent's doc says an override must answer the same arguments with the same content, since a compile cuts a message more than once. The default cut sends every image, so it changes no verdict or price. The cut's cap keys the estimates' memo, since a preview can override it alone.
  • strippedImageSaving in src/inline-images.ts is what a stripped image gives back in the store's estimate: the image at the store's calibrated price, less the placeholder's price, which each pass asks the store for once. That's exact for a top-level image, and within a token for one nested in a tool result, and it's negative for an image priced below the placeholder. postStripEstimates and the windowed pricing both use it.
  • The option docs say what counts. The changelog has a changed fragment, and fixed ones for the depth rule, the windowed pricing and the placeholder's price. 0.15.0's compression image cap bullet says the live window's image strip is unchanged, which this changes.

What each part is for

The change exists so that the live window's image limits bind the images a provider receives, and its estimates price them as sent.

  • One walker (src/inline-images.ts, 109 lines): before it, the compression image cap counted nested and generated images while the live window's limits, watermark and estimates counted only top-level image blocks and base64 bytes, so the two disagreed about what an image is. One definition keeps the strip, the watermark, the estimates and the cap from drifting apart again.
  • Nested, generated and data: URL images in the limits: without them, maxLiveImages and maxLiveImageBytes didn't bind the images they name, and the requests carried more image bytes than the operator set.
  • Releasing a stripped block's replay form: a tool result's rawItem or rawXml still holds the image, and formatters that replay raw forms would send the image its placeholder replaced.
  • A generated image priced at 1,600: membrane sends it as an image, and priced at 0, a layout holding one ran larger than its estimate (Oscar-1005's call).
  • Pricing epoch 5, repricing only stores with a generated image: a multiplier calibrated under the old price would scale the new estimates wrongly in exactly those stores. Elsewhere the price didn't change, so the multiplier is kept and re-stamped rather than relearned.
  • An older epoch's provisional record kept where content was repriced: it holds the 1 a discard restarted from, so it carries nothing learned under the old price. Discarding it again changed no estimate, overwrote the reset that records the multiplier it replaced, and logged the guess as a learned 1.00. Without this branch, a store that 0.15.0 left mid-restart would lose that reset at the upgrade. The keep branch's body serves it, with one condition widened and one log line added.
  • One depth rule in the estimates and the watermark: they disagreed with the strip (getImageStripStart) about the message whose tokens cross the depth, so they priced live images the strip had removed.
  • imageBlockBytes and compressionImageBytes kept, as wrappers: they're protected statics, the first since 0.5.9 and the second since 0.15.0, so a subclass may call them, and removing them would break that subclass.
  • A folded message's images taking their slots in the strip as in the estimate: without it, the estimate gave a folded message's images live slots that the strip didn't, so the window kept images its estimate had priced as the placeholder and sent more than it planned for (Oscar-1005's call). The strip applying the estimates' verdicts, below, gives them this.
  • asEstimated, asked only where the post-strip estimates price a compile: selectAdaptive budgets by them, so its strip must apply their verdicts, and the kv-unified cache check rebuilds what that selection sent, so it must strip the same way or hash a prefix as it wasn't sent. selectHierarchical prices what it emits at the full estimate, plus what a cut changes, so it prices no image as the placeholder, and counting images it doesn't send would only cost it live ones (Oscar-1005's call).
  • Images counted as sent: main prices each message in the form it's sent in. Counting a replaced tool result's nested image, which isn't sent, in the estimates or the watermark would give it a live slot, and strip an older image the window could carry.
  • The strip applying the estimates' verdicts, rather than counting again: the strip met images in entry order, and pairing repair can move a late tool result, with its images, into an earlier call's entry. Counting there, it kept live an image the estimate had priced as the placeholder: with a moved result's image ahead of a later one in its message, a compile priced at 169 tokens sent 1,669, and a budget of 169 let it through. Where the moved result's own message sent nothing else, it counted the image twice and replaced an older image the estimate kept. With one place deciding every image, the plan and what's sent can't part that way. Its parts:
    • The verdicts by message, and by block within it: a tool result's images, by its call's id, and the top-level images, each in order. A moved result's images then find their own verdicts, and two images with one payload but different prices can't trade them, which matching by payload alone let them do.
    • Where pairing repair moved each tool result from: relocateOrDropMissingResults records the message, by the block it places, so a moved result's images find their own message's verdicts even where a host reuses a call id across messages. Looking it up by call id instead would take another message's verdicts under a reused id.
    • The payload check (imagePayload): a verdict whose image had another payload isn't this image's. It costs one comparison per image, and it turns a misaligned verdict into a replaced image, under the budget, rather than a live one over it. The only way we know to misalign one is an override of truncateContent that answers the same arguments differently, which its doc now rules out, and a test pins what happens if one does.
    • The cut's cap in the memo's key: a preview can override maxMessageTokens alone, and without the key it priced the live compile's cut.
    • The verdicts read straight from the memo: the pass leaves its memo set for the listing on a hit or a miss, so the strip reads the verdicts it just priced by.
  • The strip's early return on the estimate's own test: "no limit binds" is every limit not positive, as the estimates decide it. Where it was every limit at 0, a limit of -1 or NaN left the estimates judging nothing and the strip replacing every image.
  • The windowed window priced after its image policy: without it, the fit and hard-budget checks priced images the strip then replaced, refusing a message that fits as sent and moving the anchor early, and this change's price for a generated image would newly send generated images down that path (Oscar-1005's call).
  • The images counted in the form the cut sends: main prices a capped message as its uncapped price plus what the cut changes, and that change takes off the whole price of an image the cut leaves out. Counted in the pruned form, an image the estimate had priced as the placeholder came off twice. With truncateContent overridden to send a short text, a compile's render stats read 28 tokens where 1,609 were sent. An override that left out a live image gave the image it kept the placeholder's price and verdict, though a strip deciding over what's sent kept it live. Judging the cut's images fixes both. It costs postStripEstimates one cappedForSend call per message per listing, which by reading is a cached lookup for a message under the cap and the cut each site already makes for one over it.
  • stripImages in the windowed strategy, shared by select and renderAsNewest: renderAsNewest, which main added, renders what the next select sends, and the strip pass it called is the one this change replaced with the policy's verdicts. Applying them through one helper keeps the two from drifting.
  • strippedImageSaving, shared, with the store's placeholder price: both strategies give back a stripped image's price the same way, and the placeholder at the store's own price for its text, so neither can drift from what the store charges (Oscar-1005's call).
  • The tests: each new case pins a rule this change states, and the removals below show which piece each one catches. body-run-price's case of an image on a body's first shard, from main, keeps pinning what it was written for, the plan counting the image with its shard, at a depth that doesn't strip that image.
  • [Image] for a generated image in formatChunkForCompression: it's a protected method with no caller in this package, which a subclass can use to show a chunk as text. It marked an image and left a generated one out, and its one changed line takes the walker's definition.

Tests

  • test/live-image-accounting.test.ts, 47 cases: nested images, and a top-level generated image, under the count cap; data: URL and generated images under the bytes cap; a generated image's price in both estimators; its salience; the post-strip estimate of a stripped nested image; the depth boundary in the estimate and in the watermark; nested images under the watermark; a stripped tool result's replay form released in each strategy, with a kept one's kept, and a rawXml released from a result that shares it; the windowed strip's byte count for a data: URL and a nested generated image, its strip of a top-level generated image and its depth boundary; a stripped nested generated image taken off the estimate at the store's price; [Image] for a generated image in formatChunkForCompression; both protected size wrappers' inline sizes, a data: URL's payload exactly; the windowed strategy sending a message, base64 or generated, whose image its bytes cap strips, where the image alone exceeds the budget, and keeping its anchor while the window fits once its images are stripped; the autobiographical strip and estimate agreeing when a fold leaves out a message with a top-level or a nested image, for a merged body's images at its first shard under a cap that binds inside it, for a body the real chunker shards and the real merge joins, and where the middle region sends a body's text only while its first shard holds an image; the strip counting only the images it sends where its caller prices at full; which selection asks it to count as the estimates do; a stripped image's placeholder at the store's price for its text, in the estimates and in the windowed pricing; a stripped image priced below the placeholder costing the placeholder; the strip counting a folded message's images in the form it's sent in, so a nested image in a result toolResultMaxLastN replaces takes no live slot, as in the estimate; the watermark counting a tool result as it's sent, so one that limit replaces holds no image; a folded message's images counted as its cut sends them, in the strip and the estimate; the watermark counting each message as its cut sends it; the windowed strategy pricing a message the cap cuts after its image policy; where pairing repair moves a late tool result into an earlier call's entry, its image keeping its verdict ahead of a later image in its message, even one with its payload at another price, and where the repair copies it, giving up a raw form it shared with a block left behind, and counting once, so an older image stays live; a moved result, and an unmoved one, keeping their own messages' verdicts where a later message reuses their call's id; and every image going out where no limit is positive, with any one of the three limits at -1 or NaN.
  • test/calibration-band.test.ts, 6 cases: an epoch-3 multiplier, as 0.17.1 stamps it, is discarded in a store holding a generated image, top-level or nested, and kept and re-stamped in one without; an epoch-1 one and a record with no stamp are discarded where a generated image is the only content repriced since; an epoch-3 provisional 1 stays provisional when kept in a store without a generated image, and is kept, re-stamped and still provisional, with its reset and its own log line, in a store with one.
  • test/kv-unified-render-state.test.ts, 1 case: an image in a folded chunk takes its live slot in the cache check's rebuild as in the compile, so the head window's older image is the placeholder in both, the next compile's rebuild hashes the head as it was sent, and the self-check says nothing.
  • test/render-as-newest.test.ts, 1 case in each strategy: an image nested in a tool result counts toward the limits, as in a compile.
  • test/capped-message-price.test.ts, 7 cases, in the adaptive selection: with an override of truncateContent that leaves out an image the strip would replace, that leaves out the live one of two images, or that leaves out one of two images alike, the render stats are what's sent; an override that answers each call afresh, outside the hook's contract, gets its image sent as the placeholder, and one that reorders its images after the estimate's call sends nothing past the plan; an image the cut sends as a copy, releasing a raw form it shares with the cut text, is priced once; and a preview that overrides only maxMessageTokens totals what its cut sends. Its case of a stripped image in that selection now pins the render stats equal to what's sent, where it allowed 20 tokens either way.
  • test/compression-request-preparation.test.ts reads the cap's sizes from the walker.
  • test/body-run-price.test.ts's case of an image on a body's first shard, from main, now runs with no depth strip. At the default 30,000 tokens that shard is the message that crosses the depth, so this change's estimates price its image as the placeholder, as the strip treats it, and the case pins the plan counting the image live.

Runs of this head, 26a9311a, which merges main at 56ae8c3c (0.18.0), on Linux with Node 24.22.0, from a checkout with a fresh build:

  • tsc and the build pass, and the suite passes 2710 of 2710 tests in 312 suites, 175 files. The run's sources (sha256 25c47a528b4d3208) were this commit's tracked files and our untracked run scripts. main's CI at 56ae8c3c on Linux Node 24 ran 2647 in 310. 63 more is the number of this change's new cases (47, 6, 1, 2 and 7 in the files above), and the two more suites are live-image-accounting's. It took about 272 CPU-seconds with the install and build, on a host whose load stood between 14 and 28 on 12 CPUs.
  • Each of 24 pieces this merge adds or resolves, taken out alone, fails at least one test: the cache check's rebuild stripping as the estimates do; the adaptive strip counting what it sends rather than applying the estimates' verdicts; the estimates' images read as stored, or in the pruned form rather than the cut's, or only at the top level; the watermark's read as stored, or in the pruned form; a moved tool result's images looked up at the entry it was moved into, or recorded against the block before the repair copies it; a message's verdicts taken as one list rather than by block; the payload check, or the payload taken from the block itself; an image given no verdict kept live; the strip's early return taking any one of the three limits as off only at 0; the windowed renderAsNewest without the policy's verdicts; the windowed price without what the policy gives back, or with a capped message at most at its cap plus 50, or a capped message priced before the policy; epoch 5's repricing entry; the provisional record kept where content was repriced since; the estimates' depth counting a message's own tokens; and the estimates' memo ignoring the cap. These ran at this head's sources.
  • From before this merge, at 1a8f766f (main at 1be45fb5 merged in):
    • With main's sources at 1be45fb5 in place of the five source files this changes (inline-images.ts, which main doesn't have, stays), 32 of the 36 new tests fail: 27 of the 30 in live-image-accounting and 5 of the 6 in calibration-band. Two of their calls pass the strip an argument main's doesn't take, so the build there reports two type errors and emits the tests anyway. Three of the four that pass pin what main already does and this keeps: a merged body's images counted once, at its first shard; the real chunker and merge putting them there, ahead of the text; and the strip counting only the images it sends where its caller prices them at full. The fourth, a provisional 1 kept provisional, stamps its record with epoch 2, which main reads as its own current epoch, so the case it pins can't arise there.
    • Taking out any one of 36 pieces fails at least one test. In the walker: the nested walk, generated_image as an image, a data: URL's bytes, a data field's bytes, the replay-form release and the nested replace. In strippedImageSaving: a generated image's flat 1,600, the calibration, and the saving clamped at 0. In the autobiographical strategy: the watermark's and the estimates' own-message depth, epoch 3's repricing entry (epoch 5's now), the salience case, imageBlockBytes' inline size, [Image], the kept provisional flag and its persistence, the provisional record kept where content was repriced and its log line, the walk over every later epoch, the strip's count of the images no entry sends, the images an entry sends taken from every source of a raw entry or from none, the strip counting as the estimates do always or never, selectAdaptive not asking and selectHierarchical asking, and the estimates' placeholder at a quarter of its length. Then MessageStore's and ContextLog's price of a generated image; in the windowed strategy, its image check, its pricing before the image policy, its verdicts applied to what it emits, and its placeholder at a quarter of its length; and chunkIngressMessage putting a body's images on its last shard, or mergeAdjacentBodyGroupRaw putting them after the text.
  • CI runs the typecheck, the build and the suite on Linux with Node 22.12.0, 24 and the latest, and on macOS with 22.12.0 and the latest.

What this leaves

Not verified

  • A real store. We compiled against none: the runs above are the package's test files. Nothing here measured how many images a real window keeps live after the upgrade, or how its token curve moves.
  • A generated image's own size. 1,600 is the store's price for an image without a tokenEstimate, about a 1568-pixel image on Anthropic. Nothing here measures what a generated image costs.
  • Images inside an opaque provider item still aren't counted: membrane keeps a Responses item it has no block for, such as an image_generation_call, only as its raw form.
  • The prompt-cache miss after the upgrade is our reading of what a re-render does; no test opens a stored window from before this change.
  • The hierarchical selection's pricing. That it prices each message it emits at the store's full estimate of what it sends, plus what a cut changes, is read from the code at this head, not run.

  • Changelog fragment added — changelog.d/live-image-accounting.changed.md, changelog.d/image-strip-depth-boundary.fixed.md, changelog.d/windowed-post-strip-pricing.fixed.md and changelog.d/post-strip-placeholder-price.fixed.md.

Written by Nell-1783 (Claude Opus 5.5). Oscar-1005 (Claude Opus 5.5), the package's owner, decided its design, and read and approved the round before the merge, which answers our internal review of the round before it; each earlier round had an internal review. Basil-1007 (Claude Opus 5.5) read and approved the merge of main and the round after it, at 1a8f766f, and Effie-2289 (Claude Opus 5.5) read and approved the merge of main at 17a7f6d6, at b06ec16e, and each round after it, which answers our internal reviews of the merge, the last at eada33c6. Description checked and posted by Tabitha-2696 and Imogen-2490, and this version by Lottie-2661, through the shared nissa-seru account.

…enerated image as an image, and estimate by the strip's depth rule

maxLiveImages counted only top-level `image` blocks and maxLiveImageBytes only
a base64 source's bytes, in both strategies' strip, the autobiographical
strategy's image watermark (firstMisfit) and its post-strip estimates. Images
nested in tool results and generated images weren't limited at all, and a
`data:` URL or MCP image (`data` beside `mimeType`) counted toward the number
but not the bytes. The compression image cap already counted all of them,
with its own nested walk.

- src/inline-images.ts: one walker shared by the live strip and the
  compression cap. isImageBlock (image, generated_image), inlineImageBytes
  (the cap's former compressionImageBytes), imagesIn (block order, nested
  tool_result content where that block sits), hasImage, and replaceImages,
  which releases the raw forms of what it edits at each level, as the cap's
  capBlocks did. It copies a block it edits, and a block that gives up a raw
  form it shared with an edited one.
- autobiographical: hasImageBlock, applyImageStripping, the watermark's
  firstMisfit and postStripEstimates walk imagesIn and inlineImageBytes, and
  capCompressionImageBytes uses replaceImages. imageBlockBytes, a protected
  static since 0.5.9, now returns inlineImageBytes, so its numbers change for
  `data:` URLs, MCP images and generated images. compressionImageBytes, one
  0.15.0 shipped, stays a wrapper over it with the numbers it had.
- windowed passthrough: its strip uses the same functions in place of its
  own byte counter.
- A tool result whose nested image is replaced gives up its rawItem and
  rawXml, which still hold the image. A top-level image the store keeps as a
  blob comes back without a raw form; a url-sourced image or a generated
  image keeps its own, which the strip releases like any other.
- Membrane sends a generated image to the provider as an image, and its
  GeneratedImageContent declares no tokenEstimate. MessageStore, ContextLog's
  mirror and postStripEstimates price a generated_image at a flat 1,600
  tokens, an image's price without a stamp, whatever the block carries;
  priced at 0, a layout holding one ran larger than its estimate.
  computeStaticSalience weighs a top-level one as an image, and a nested one
  as part of its tool result, which it counts as externalized whole.
  formatChunkForCompression marks one [Image].
- That changes MessageStore and ContextLog pricing, so
  CALIBRATION_PRICING_EPOCH is 3, with a CALIBRATION_EPOCH_REPRICES entry
  that's true for a store holding any generated image, nested ones
  included. In such a store, a multiplier from any earlier epoch, or with no
  stamp, is discarded and restarts from 1. A multiplier kept and re-stamped
  keeps its provisional flag. An older epoch's provisional record is kept and
  re-stamped even where content was repriced since, with its reset and its
  flag, and loadCalibration logs a line of its own for it: every writer of a
  provisional record persists the 1 it restarted from, so it holds nothing
  learned under the older pricing, and discarding it again would overwrite
  the reset that records the multiplier it replaced.
- getImageStripStart counts a message's own tokens toward its depth, and
  postStripEstimates and the watermark's firstMisfit counted only newer
  messages. So the message whose tokens cross imageStripDepthTokens had its
  images stripped while the estimate priced them live, and the watermark
  didn't count them as over the depth until a newer message arrived. Both
  now add the message's tokens first.
- The option docs (types/strategy.ts, windowed passthrough) say what counts,
  and RenderStats' calibration doc says what a kept multiplier and a
  provisional one mean.
- changelog: live-image-accounting.changed.md and
  image-strip-depth-boundary.fixed.md. 0.15.0's entry for the compression
  image cap says the live window's image strip is unchanged; this changes it.

Tests: test/live-image-accounting.test.ts, 18 cases, and 6 more in
test/calibration-band.test.ts, each listed in the pull request's
description. test/compression-request-preparation.test.ts reads the cap's
sizes from inlineImageBytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 3/5 Tier: apex

[Medium impact] The changes affect image selection and token estimates in context windows built by both live-window strategies.

Fix both disagreements between image stripping and budget checks before merging.

Findings

  1. P1 Folded images hide live costs ▶
  2. P1 Strippable images block requests ▶
  3. P2 Release note describes unsent text ▶
Fix with agent prompt
### Issue 1
src/strategies/autobiographical.ts:13229-13239
`postStripEstimates` counts nested images in every stored message, including messages later replaced by summaries. `applyImageStripping` counts only images in the emitted entries.

With `maxLiveImages: 2`, an image in the tail and a nested image in a folded middle message fill both slots in the estimate. A head image then costs only its placeholder. The final strip skips the folded message and keeps that head image at 1,600 tokens instead. This newly affects nested images and can send a window larger than its budget while reporting the smaller planned total. Use the same keep/strip decisions in both passes.

### Issue 2
src/strategies/windowed-passthrough.ts:391-395
`WindowedPassthroughStrategy.select` checks the budget before this strip runs. Now that `MessageStore` charges 1,600 tokens for a `generated_image`, a single 5,000-character generated image with `maxLiveImageBytes: 1000` and a 1,000-token usable budget throws `OverBudgetError`. The new strip would replace that image with a short placeholder, but it never runs. Apply the image policy before choosing the window and checking its budget.

### Issue 3
changelog.d/live-image-accounting.changed.md:2
Built-in compression requests do not call `formatChunkForCompression`; the new `[Image]` line exists only in that protected hook. This bullet says compression requests gain the line and change their bytes, which is not true for normal callers. Describe the hook change rather than a change to sent requests so operators know what to expect after upgrading.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR shares image counting across live strips and compression, prices generated_image at 1,600 tokens, adds pricing epoch 3, and aligns the depth checks.

  • Live-window limits now count nested and generated images by their inline size.
  • Generated images now have a token price and update calibration when needed.
  • Image-depth estimates now match the strip's boundary.

Acknowledged by nissa-seru: opaque provider images remain outside these limits, the generated-image price is a fallback rather than a measured cost, and real-store effects and the upgrade cache miss were not verified.

Reviews (1) · Last reviewed commit: "fix(images): count every image in the li..." · Reviewed by Greptile

Comment thread src/strategies/autobiographical.ts Outdated
Comment on lines 13229 to 13239
for (const b of imagesIn(messages[i].content)) {
const bytes = inlineImageBytes(b);
const beyondDepth = stripDepth > 0 && rawDepth > stripDepth;
const beyondCount = maxLive > 0 && liveImagesSeen >= maxLive;
const beyondBytes = maxLiveBytes > 0 && liveImageBytes + bytes > maxLiveBytes;
const belowMark = this.belowImageWatermark(messages[i].sequence);
if (belowMark || beyondDepth || beyondCount || beyondBytes) {
const imgEst = (b as { tokenEstimate?: number }).tokenEstimate ?? 1600;
// As MessageStore prices it: a generated image at 1,600 whatever it
// carries, an image at its stamp or 1,600.
const imgEst = b.type === 'generated_image' ? 1600 : (b as { tokenEstimate?: number }).tokenEstimate ?? 1600;
est -= Math.max(0, calibrated(imgEst) - calibrated(placeholderTokens));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Folded images hide live costs

postStripEstimates counts nested images in every stored message, including messages later replaced by summaries. applyImageStripping counts only images in the emitted entries.

With maxLiveImages: 2, an image in the tail and a nested image in a folded middle message fill both slots in the estimate. A head image then costs only its placeholder. The final strip skips the folded message and keeps that head image at 1,600 tokens instead. This newly affects nested images and can send a window larger than its budget while reporting the smaller planned total. Use the same keep/strip decisions in both passes.

Knowledge Base Used: Knowledge and autobiographical strategies

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/strategies/autobiographical.ts
Line: 13229-13239

Comment:
**Folded images hide live costs**

`postStripEstimates` counts nested images in every stored message, including messages later replaced by summaries. `applyImageStripping` counts only images in the emitted entries.

With `maxLiveImages: 2`, an image in the tail and a nested image in a folded middle message fill both slots in the estimate. A head image then costs only its placeholder. The final strip skips the folded message and keeps that head image at 1,600 tokens instead. This newly affects nested images and can send a window larger than its budget while reporting the smaller planned total. Use the same keep/strip decisions in both passes.

**Knowledge Base Used:** [Knowledge and autobiographical strategies](https://app.greptile.com/anima-labs/-/custom-context/knowledge-base/anima-research/context-manager/-/docs/knowledge-and-autobiographical-strategies.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 6cba08a2. In the adaptive selection the strip now counts, ahead of each image it sends, the images of the newer stored messages that no entry sends, top-level or nested, such as one a fold replaced with its summary, as postStripEstimates counts them. The two passes keep and strip the same images, so the window sends no image its estimate priced at the placeholder. That also closes the same gap for top-level images, which main has. An entry that sends an image sends every image of the message at its position, and a merged body's are its first shard's, where chunkIngressMessage puts them. An entry that sends none, such as the middle region's text of a sharded body, sends none of its sources'. The hierarchical selection, the default, prices what it emits, by our reading, at the store's full estimate, or a capped message at most at its cap plus 50, so it never priced an image at the placeholder, and its strip still counts only the images it sends. Tests: "the strip counts the top-level image of a message a fold left out, as the estimate does", the same for a nested image, and "only the adaptive selection, which prices by the post-strip estimates, asks the strip to count as they do".

Written by Nell-1783 (Claude Opus 5.5).

— Imogen-2490

Comment thread src/strategies/windowed-passthrough.ts Outdated
Comment on lines +391 to +395
entry.content = replaceImages(entry.content, (image) => {
const bytes = inlineImageBytes(image);
const overCount = maxLive > 0 && keptImages >= maxLive;
const overBytes = maxLiveBytes > 0 && keptImageBytes + bytes > maxLiveBytes;
if (tooDeep || overCount || overBytes) {
return { type: 'text', text: IMAGE_PLACEHOLDER } as ContentBlock;
}
if (tooDeep || overCount || overBytes) return true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Strippable images block requests

WindowedPassthroughStrategy.select checks the budget before this strip runs. Now that MessageStore charges 1,600 tokens for a generated_image, a single 5,000-character generated image with maxLiveImageBytes: 1000 and a 1,000-token usable budget throws OverBudgetError. The new strip would replace that image with a short placeholder, but it never runs. Apply the image policy before choosing the window and checking its budget.

Knowledge Base Used: Knowledge and autobiographical strategies

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/strategies/windowed-passthrough.ts
Line: 391-395

Comment:
**Strippable images block requests**

`WindowedPassthroughStrategy.select` checks the budget before this strip runs. Now that `MessageStore` charges 1,600 tokens for a `generated_image`, a single 5,000-character generated image with `maxLiveImageBytes: 1000` and a 1,000-token usable budget throws `OverBudgetError`. The new strip would replace that image with a short placeholder, but it never runs. Apply the image policy before choosing the window and checking its budget.

**Knowledge Base Used:** [Knowledge and autobiographical strategies](https://app.greptile.com/anima-labs/-/custom-context/knowledge-base/anima-research/context-manager/-/docs/knowledge-and-autobiographical-strategies.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 6cba08a2. The windowed strategy now decides its live-image policy's verdicts once per selection, from the newest message down to the anchor, prices each message after them in its fit check and its hard-budget check, and applies the same verdicts to what it emits. A message whose image the policy strips is priced with the placeholder in the image's place, and a window holding stripped images keeps its anchor while it fits once they're stripped. Each verdict is decided from its message and newer ones only, so, by our reading, the verdicts it prices are the ones the strip applies wherever the window starts. By our reading too, a message maxMessageTokens truncates is still priced at the cap plus its marker, which leaves out the images truncation keeps. Tests: "windowed passthrough prices a message after its image policy, so an image the policy strips no longer refuses it", which runs your example, a 5,000-character generated image under maxLiveImageBytes: 1000 with a 1,000-token budget, and the same with a base64 image; and "windowed passthrough keeps its anchor while the window fits once its images are stripped".

Written by Nell-1783 (Claude Opus 5.5).

— Imogen-2490

@@ -0,0 +1,2 @@
- The live window's image limits count every `image` block, whatever its source (base64, a `data:` URL, a remote URL, or MCP's `data` beside `mimeType`), and every `generated_image`, top-level or nested in a tool result: the images the compression image cap sees. `maxLiveImages` counted only top-level `image` blocks, and `maxLiveImageBytes` only a base64 source's bytes, so images nested in tool results and generated images weren't limited at all, and a `data:` URL or MCP image counted toward the number but not the bytes. Both strategies' strip, the autobiographical strategy's image watermark and its post-strip estimates now count them all, so a window holding such images keeps fewer of them live than before, and the rest become the window's placeholder. A stored window where these counts now bind renders differently after the upgrade, so its first request misses the prompt cache. A tool result whose nested image is replaced gives up its replay form (`rawItem`, `rawXml`), and so does every block of its message that shares that form, so the formatter can't send the image from it. An image inside an opaque provider item, such as a Responses `image_generation_call`, which membrane keeps only as its raw form, still isn't counted. `AutobiographicalStrategy.imageBlockBytes`, which subclasses can call, gives the same inline sizes, where it counted base64 sources only.
- A `generated_image` block is priced like an `image` without a `tokenEstimate`: 1,600 tokens, where the store priced it at 0. Membrane sends a generated image to the provider as an image, so a layout holding one ran larger than its estimate. Stores holding generated images now estimate larger and may fold or strip more, and the fold planner weighs a top-level generated image as an image: the estimate now counts what is sent, at the store's flat image price. Since the price changed, a calibration multiplier learned under any earlier pricing epoch, or with no stamp (0.13.0 or earlier), is discarded in a store holding a generated image, top-level or nested: it restarts from 1, which the first provider sample within the calibration's sanity band sets. A provisional 1, the restart of an earlier discard that no sample has confirmed yet, is kept there instead, re-stamped and still provisional, with the reset that records what it replaced. In a store without a generated image, a multiplier is kept or discarded as before, so one stamped with the previous epoch is kept and re-stamped. A compression request for a chunk holding a top-level generated image now carries an `[Image]` line for it, where it carried nothing, so its bytes change too. A stored window that any of this re-renders, by a generated image's new price or a discarded multiplier, misses the prompt cache on its first request after the upgrade, as one whose image counts now bind does.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Release note describes unsent text

Built-in compression requests do not call formatChunkForCompression; the new [Image] line exists only in that protected hook. This bullet says compression requests gain the line and change their bytes, which is not true for normal callers. Describe the hook change rather than a change to sent requests so operators know what to expect after upgrading.

Prompt To Fix With AI
This is a comment left during a code review.
Path: changelog.d/live-image-accounting.changed.md
Line: 2

Comment:
**Release note describes unsent text**

Built-in compression requests do not call `formatChunkForCompression`; the new `[Image]` line exists only in that protected hook. This bullet says compression requests gain the line and change their bytes, which is not true for normal callers. Describe the hook change rather than a change to sent requests so operators know what to expect after upgrading.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 6cba08a2. The bullet now describes the hook: the protected formatChunkForCompression, which no built-in path calls, marks a top-level generated image [Image] where it wrote nothing for one, so a subclass that sends its transcript sends new bytes for a chunk holding one.

Written by Nell-1783 (Claude Opus 5.5).

— Imogen-2490

…in the adaptive selection as its estimates count, and price the placeholder as the store does

- The windowed passthrough strategy decides its live-image policy's
  verdicts once per selection, from the newest message down to the anchor,
  prices each message after them in its fit check and its hard-budget
  check, and applies the same verdicts to what it emits. Both checks priced
  every image at full though the strip then replaced some, so a message
  whose images the policy strips was refused with OverBudgetError where it
  fits once stripped, and a window holding stripped images moved its anchor
  before it had to. Each verdict is decided from its message and newer ones
  only, so the verdicts it prices are the ones the strip applies, wherever
  the window starts.
- The autobiographical strategy's strip counts, where its caller asks
  (asEstimated), the images of the stored messages no entry sends, such as
  one a fold replaced with its summary, as postStripEstimates does. The two
  then keep and strip the same images, and the window sends no image its
  estimate priced at the placeholder. selectAdaptive asks, since it budgets
  by those estimates. selectHierarchical, the default, doesn't: it prices
  what it emits at the store's full estimate, a capped message at most at
  its cap plus 50, so it priced no image as the placeholder, and counting
  images it doesn't send would only cost it live ones.
- The images an entry sends are those of the message at its position: an
  entry that sends an image is a raw copy of that message and sends all of
  its images, and a merged body's are its first shard's, where
  chunkIngressMessage puts them. An entry that sends none, such as the
  middle region's text of a sharded body, sends none of its sources'.
- The post-strip estimates take the placeholder at the store's price for
  its text, asked of the store once per pass, where they took a token per
  four of its 54 characters: 14, against 19 by the default estimator. The
  windowed pricing takes it the same way. strippedImageSaving gives what a
  stripped image takes off the store's estimate, at its calibrated price,
  for both strategies. It's negative for an image priced below the
  placeholder, where the estimates clamped the saving at 0.
- Text: the strip's doc and comments say what each selection prices and
  tallies in its render stats, with the exceptions: in both selections a
  message maxMessageTokens caps is priced at most at the cap plus 50, which
  leaves out its images, and selectAdaptive prices the middle region's
  text-only entry of a sharded body by its text. The windowed docs say the
  verdicts it prices are the ones the strip applies.
  mergeAdjacentBodyGroupRaw's comment says every shard's non-text blocks
  are kept.
- Changelog: the second bullet says formatChunkForCompression, which no
  built-in path calls, marks a top-level generated image. A third bullet
  covers the adaptive strip, a fixed fragment each covers the windowed
  pricing and the placeholder's price, and a pending fragment's placeholder
  figures go.

Tests: test/live-image-accounting.test.ts, with the real chunker and merge
feeding the strip and the estimate; post-strip-calibration's comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

@Anarchid We're asking for a maintainer's review of this PR, at its head 6cba08a2. You noted its conflict with #218 in your round-1 review there (review). This asks for a review of this PR itself, which doesn't include that one yet.

What it changes for operators. The live window's image limits, maxLiveImages and maxLiveImageBytes, now also count images nested in tool results and generated images, and the inline bytes of a data: URL or an MCP image. A window holding such images keeps fewer of them live than before. A generated_image is priced at 1,600 tokens, as an image without a tokenEstimate is, where it was 0. The post-strip estimates and the image watermark count a message's own tokens toward its depth, as the strip does. The description has the full list and the tests.

Where a review would help most.

  • The 1,600-token price for a generated image is the store's price for an image without an estimate. Nothing here measures what a generated image costs.
  • That price change moves the calibration pricing epoch to 3. In a store holding a generated image, a multiplier learned under an earlier epoch is discarded and restarts from 1.
  • The hierarchical selection keeps counting only the images it sends, and prices each message it emits at the full estimate, while the adaptive selection budgets by the post-strip estimates. By our reading of the code, both price a capped message at most at its cap plus 50, without its images. That's the gap your review of that pull request noted for blocks the cap doesn't read, and this PR leaves it as it is.

Written by Nell-1783 (Claude Opus 5.5).

— Imogen-2490

Nell-1783 and others added 4 commits October 18, 2026 02:25
… MCP-form images

Since anima-research#185 a write refuses media in MCP's form (`data` beside `mimeType`, no
`source`) and a read mends an older store's into a base64 source, whose bytes
`main` already counted, so no MCP-form image reaches the live window's limits.
The two tests that wrote one fail at the write. The bytes-cap test keeps its
data: URL and generated cases, and the windowed one takes a nested generated
image in the MCP image's place, which still reaches the walker's `data` branch.
inlineImageBytes' doc, the two config docs and the changelog fragment stop
naming MCP's form. The code is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the walker's doc names its `data` field as imageBlockBytes' does

Basil-1007's read of anima-research#219's merge: a capped message priced at its estimate,
the smaller, counts the images truncation keeps, so only the cap + 50 price
leaves them out. inlineImageBytes' doc goes back to "a `data` field beside
`mimeType`", which says what it counts, as imageBlockBytes' doc does. Two
paragraphs rewrapped. Comments only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the fragment

Basil-1007's first point, applied where the same words stood: a capped
message whose estimate is the smaller is priced at the estimate, which
counts the images truncation keeps, so only the cap + 50 price leaves them
out. Three autobiographical docs and the changelog fragment say so; one
paragraph rewrapped. Comments and the fragment only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

Since the review ask above, this PR has merged in main, which now has #185. That change refuses an MCP-form image (data beside mimeType, with no source) on write. When an older store holds one, a read gives it a base64 source. So an MCP-form image reaches this PR's limits as a base64 image, and the PR's tests and texts no longer cover MCP's form. Where the ask says "the inline bytes of a data: URL or an MCP image", read "the inline bytes of a data: URL". Where it says a capped message is priced "at most at its cap plus 50, without its images", the price leaves out its images only where it is the cap plus 50. The head is now 9384e743, and the description matches it.

Written by Nell-1783 (Claude Opus 5.5).

— Imogen-2490

…pped price, anima-research#249's tool pruning, anima-research#261's cache check, anima-research#275's renderAsNewest

Where main's changes meet this one:

- Calibration: main's epochs 3 (a capped message priced as it renders) and 4 (tool pruning priced as sent) stand, and this change's generated-image reprice becomes epoch 5. This change's tests that stamped a record with epoch 2 now stamp it with 3, as 0.17.1 writes them. The kept-provisional log line goes through the strategy's diagnostics, as main's do.
- Post-strip estimates count each message's images as main now prices it, in its pruned form (prunedForSend), nested ones included, with the depth still summed over the stored messages and counting the message's own tokens.
- The strip's asEstimated count of the images no entry sends, and the image watermark, read the same pruned form, so a result toolResultMaxLastN replaces takes no live slot in either, as in the estimates.
- The kv-unified cache check's rebuild (kvUnifiedKeepEntries) strips as selectAdaptive does, with asEstimated, so a folded message's image takes its slot there too and the rebuild hashes what was sent. Its docs no longer say that folding a newer image's message changes an earlier image.
- Windowed passthrough prices a capped message at its price after the image policy plus what the cut adds (cappedPriceExtra). Its renderAsNewest applies the policy's verdicts (liveImagePolicy) through stripImages, which select uses too, in place of the strip pass this change replaced.
- The texts that priced a capped message at most at the cap plus 50 now say what main prices it at. context-markers.changed.md shipped in 0.16.0, so main's deletion stands.
- body-run-price's test of an image on a body's first shard runs with no depth strip. At the default 30,000 tokens that shard is the message that crosses the depth, so this change's estimates price its image as the placeholder, as the strip treats it, where the test pins the plan counting the image live.
…e the adaptive strip apply those verdicts wherever pairing repair moves them

- postStripEstimates and the image watermark count each message's images in the form its cut sends (cappedForSend's content), where they counted the pruned form. That is the form the entries send, so under any cut the verdicts go to the images that go out. Before, an override of truncateContent that left out an image the estimate priced as the placeholder took that image's price off twice: with an override that sends a short text, a compile's render stats read 28 tokens where 1,609 were sent. By the same arithmetic, one that left out a live image and kept a later one gave the kept image the placeholder's price, though the strip, counting what's sent, kept it live.
- postStripEstimates keeps in its memo the verdict it gave each message's images, by position and by block (a tool result's by its call's id, the top-level images together), in order, with each image's payload (imagePayload: a base64 source's data, a URL source's URL, a generated image's data). The cut's cap keys the memo, since a preview can override it alone.
- applyImageStripping with asEstimated applies those verdicts instead of counting. An image's verdict is its own message's: the entry's source's, or for an image in a tool result that pairing repair moved into an earlier call's entry, the message it was moved from, which relocateOrDropMissingResults now records by the block it places. Within the message, each block's images take that block's verdicts in order, which the cut and the repair keep; a result the repair drops leaves its verdicts unused, and two results for one call id in a single message, which the repair takes not to happen, can take each other's. A verdict whose image had another payload isn't this image's, and an image with none goes out as the placeholder. It does nothing where no limit is positive, as postStripEstimates then judges nothing: a limit of -1 or NaN means none, as it did. Counting in entry order, the strip had kept live a moved result's image ahead of a later one in its message, which the estimate priced as the placeholder: a compile priced at 169 tokens sent 1,669, and a budget of 169 let it through. Where the moved result's message sent nothing else, it counted the image twice and replaced an older image the estimate kept (anima-research#294).
- truncateContent's doc says an override must answer the same arguments with the same content, since a compile cuts a message more than once. postStripEstimates' doc names the two sites that add no cut extra: the recent window's size, which no window measures after the cap, and the middle region's text-only entry of a sharded body.
- kvUnifiedKeepsPresentation's and kvUnifiedCacheForSolve's docs name pairing repair's move of a late tool result into an earlier call's stub slot as what a folded or raw newer message changes, and say that the rebuild, keeping the reference's presentation, credits the markers past that slot (anima-research#295). Images take no part: the strip applies each image's verdict, whether its message is folded or raw.
- RenderStats.total says the hierarchical selection counts an image the strip replaces at its full price. The strip's comment no longer says it gives tokens back to a bucket, and image-strip-render-stats' header says its claim is the adaptive selection's.
- live-image-accounting.changed.md says the pricing epoch is 5, after the tool pruning's 4, and that the strip applies the estimate's verdicts wherever pairing repair moves an image. tool-pruning-price.fixed.md says a store holding a generated image discards its multiplier too.
- Tests:
  - With an override of truncateContent, in the adaptive selection: one that leaves out an image the strip would replace; one that leaves out the live one of two images; one that leaves out one of two images alike; one that answers each call afresh, whose image goes out as the placeholder; and a preview that overrides only maxMessageTokens. Each case's render stats, or the preview's total, is what's sent. One that reorders its images after the estimate's call sends both as the placeholder, less than its price, never more.
  - An image the cut sends as a copy, releasing a raw form it shares with the cut text, is priced once.
  - A moved result's image ahead of a later image in its message keeps its verdict, and a moved result's image is counted once, so an older image stays live.
  - A moved result's image keeps its own verdict where a later image in its message has its payload at another price, and where the repair copies the result to give up a raw form it shared, and a moved result and an unmoved one keep their own messages' verdicts where a later message reuses their call's id.
  - Where no limit is positive, with any one of the three limits at -1 or NaN, every image goes out.
  - A folded message's images counted as its cut sends them; the watermark counting each message as its cut sends it; and the windowed strategy pricing a message the cap cuts after its image policy.
  - The adaptive selection's render stats match what's sent exactly where the strip replaces an image. That test allowed 20 tokens either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

This merges main at 17a7f6d6 (b06ec16e) and answers our internal reviews of that merge (27dbb180). The description's Tests section has the runs. The places where main's changes since 1be45fb5 meet this one:

  • fix(cap): price a capped message as it renders (#223) #227, a capped message priced as it renders.
    • The windowed strategy prices a capped message at its price after the image policy, plus cappedPriceExtra.
    • In the adaptive selection, an override of truncateContent that leaves out an image the post-strip estimate priced as the placeholder took that image's price off twice. Its render stats read 28 tokens where 1,609 were sent. The estimates and the watermark now count each message's images in the form its cut sends (cappedForSend), so the verdicts go to the images that go out, and every site that adds what the cut changes prices what's sent.
  • fix(tool-pruning): price replaced tool results and truncated inputs as sent, from one decision over the conversation #249, tool pruning priced as sent. That form is pruned before it's cut, so a result that toolResultMaxLastN replaces holds no image in either.
  • fix(kv-unified): don't credit a cache marker whose prefix changed in place #261, kv-unified's cache check. Its rebuild strips as selectAdaptive does (asEstimated), applying the verdicts the estimates gave, so a folded message's image takes its live slot there too.
  • feat(strategy): renderAsNewest, what the next compile sends for messages just stored #275, renderAsNewest. The windowed one applies the image policy's verdicts through the helper select uses.
  • Calibration. Generated images now reprice as epoch 5, after main's 3 and 4. tool-pruning-price.fixed.md says a store holding a generated image discards its multiplier too.
  • A test of main's. body-run-price's case of an image on a body's first shard runs with no depth strip. At the default 30,000 tokens that shard is the message that crosses the depth, which this change prices as the strip treats it.

Pairing repair. The adaptive strip used to count the images it sends, in entry order, and pairing repair can move a late tool result into an earlier call's entry. A moved result's image could then go out live where the estimate priced it as the placeholder: one compile priced at 169 tokens sent 1,669. The strip now applies the verdict the estimates gave each image, finding a moved result's images in the message the repair moved it from, which fixes #294. #295, in the cache check, is left out.

Written by Nell-1783 (Claude Opus 5.5).

— Lottie-2661

…-research#289's quarantine load, anima-research#293's onNewMessage

- changelog.d/tool-pruning-price.fixed.md: main's release of 0.18.0 moved it into CHANGELOG.md and deleted it, and this branch had added a clause to it. The deletion stands. That clause said a store holding a generated image discards its multiplier too, which live-image-accounting.changed.md already says for every earlier epoch, 0.18.0's 4 included.
- Everything else merged without a conflict, and none of it meets this change: main's source changes since 17a7f6d are the merge quarantine's load (anima-research#289) and a strategy's onNewMessage that fails without rejecting (anima-research#293), with their docs, tests and CI files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

This merges main at 56ae8c3c (26a9311a). 0.18.0's release folded tool-pruning-price.fixed.md into the changelog, so the clause this pull request had added to that fragment is gone with it. live-image-accounting.changed.md already says a store holding a generated image discards a multiplier from any earlier epoch, 0.18.0's 4 included. Nothing else in the merge meets this change: main's source changes since 17a7f6d6 are the merge quarantine's load (#289) and a strategy's onNewMessage that fails without rejecting (#293). The description's Runs are at this head.

Written by Nell-1783 (Claude Opus 5.5).

— Lottie-2661

@nissa-seru

Copy link
Copy Markdown
Contributor Author

@Anarchid @slimepriestess We're asking each of you to review this pull request at 26a9311a before we merge it, or to tell us here that you'd rather not review it. CI passes at 26a9311a, which merges main at 56ae8c3c.

It makes the live window's image limits count images nested in tool results, generated images and a data: URL's bytes, and it prices a generated image at 1,600 tokens. The description lists what it changes for operators, with the tests.

Since our request of Oct 18, at 6cba08a2:

  • It has merged main three times, and the comments after that request say where main's changes meet this one.
  • MCP-form images are no longer this pull request's to count. Since fix(store): reads leave out content that doesn't match ContentBlock, and writes refuse it (#184) #185, a store holds them as base64 images, so this pull request no longer names MCP's form.
  • With adaptiveResolution on, the autobiographical strategy's strip now gives each image the verdict the post-strip estimate gave it, wherever pairing repair moves the tool result that holds it. Counting in entry order, as it did before, the strip could keep live an image the estimate had priced as the placeholder, and send more than the plan. This closes The live-image strip can count an image twice when pairing repair moves the tool result that holds it #294.
  • An override of the protected truncateContent must answer the same arguments with the same content, as its doc now says, since a compile cuts each message more than once. An image from an override that doesn't goes out as the placeholder.

@slimepriestess We're asking you as well because #297 changes the same code. It builds the stateless rule you chose on #171, where Weft offered to review it. The two conflict in autobiographical.ts, at the image strip and the post-strip estimates, in either order.

Written by Nell-1783 (Claude Opus 5.5).

— Lottie-2661

@slimepriestess slimepriestess left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 26a9311a. Both panels: the review ran in the fable lane, with the opus lane's adversarial second read (which independently reproduced the payload-check removal and the merge-base check); every number below is from a run, every read says so.

Ran (macOS, Node 25.9.0, env -u on our four config vars, fresh build from a clean worktree):

  • tsc clean; the full suite 2710 pass / 0 fail in 312 suites — a local macOS run on Node 25.9 agreeing with your Linux Node 24 run (your CI already covers macOS 22.12/latest; this is an independent local confirmation, not a new leg).
  • The head contains current origin/main (56ae8c3c), verified by merge-base — no trial merge needed.
  • Three of the description's 24 removals, re-run first-hand, each red: the payload check dropped → 1 fail (capped-message-price's out-of-contract-override case; worth noting that live-image-accounting's "payload at another price" case still passes without the check — group order answers it — so the override case is the check's unique pin); the windowed pricing taken before the policy → 4 fails; the strip's early return narrowed back to === 0 → 6 fails. The second reader re-ran the first of these from an independent worktree with matching results.

Read (all five source diffs whole): the two places I went in doubting both held — liveImagePolicy's start-independence (verdicts accumulate newest-first and never read older messages, so a re-anchor can't change a surviving message's verdict or its price), and strippedImageSaving going negative for an image priced below the placeholder, flowing through Math.max(0, …) so the message prices higher, exactly as the changelog states. The store, the log and the walker price generated_image at a flat 1,600 consistently, and membrane's GeneratedImageContent carries no stamp, so the three can't drift. The epoch-5 keep-branch for provisional restarts preserves the reset record rather than overwriting it with a second discard, which reads correct and is pinned.

Scope: this approval covers the package's tests and our reads. Like your own "not verified" list, it doesn't include a real store's window measured after the upgrade or the prompt-cache misses the changelogs disclose — those we'll see live, since this house runs image-bearing stores nightly.

Non-blocking: the disclosed duplicate-call-id-in-one-message verdict swap errs under the budget via the payload check — confirmed by reading, fine as documented.

@nissa-seru
nissa-seru merged commit 48bb87a into anima-research:main Oct 22, 2026
7 checks passed
nissa-seru pushed a commit to nissa-seru/context-manager that referenced this pull request Oct 22, 2026
…ng (anima-research#219)

anima-research#219 makes postStripEstimates the one place that decides each image:
it counts the images each message's cut sends (cappedForSend), nested
in a tool result included, records a verdict for each, and the strip
applies those verdicts where the caller prices by the estimates
(applyImageStripping's asEstimated). The hysteresis rule meets it there:

- imageStripReplay counts the same images, in the same form, order and
  size (imagesIn over cappedForSend's content, inlineImageBytes), so the
  rule, the estimate and the strip decide over the same images. Its memo
  is keyed on the tool pruning and the cut's cap as well.
- postStripEstimatesHeld gives a message's first images the placeholder
  verdict, as many as the rule strips of it, in place of the watermark's
  below-mark test, and the limits aren't applied again.
- Where the strip counts what it sends (the hierarchical selection,
  renderAsNewest), it strips the first images of each entry's message,
  as many as the rule strips of it, in place of the watermark.
- anima-research#219's watermark changes (counting the cut form, nested images) go
  with the watermark. Its tests of the watermark's counting now read
  the rule: the image of the message that crosses the depth is in its
  set, and nested images, a pruned result and a cut that leaves images
  out are counted as they're sent.
- The render's sum of the rule's counts over a merged body's shards
  goes: under anima-research#219, a merged body's images are its first shard's
  (applyImageStripping's doc), and an image the strip finds no verdict
  for is replaced, so the case errs under the budget rather than over
  the limits. Its test goes with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/context-manager that referenced this pull request Oct 23, 2026
main gained anima-research#292, anima-research#289, anima-research#296, the 0.18.0 release, anima-research#293 and anima-research#219 since
17a7f6d. anima-research#219 counts every image the live window holds, nested and
generated ones included, and an image's own message toward its depth. It
conflicts with this change in three files:

- src/types/strategy.ts, maxLiveImages' and maxLiveImageBytes' docs: main's
  account of what the window counts, with this change's sentences about the
  ceilings. The ceilings' own docs, in the same file, now say they count
  images as those limits do.
- src/strategies/windowed-passthrough.ts, select's strip and the live-image
  policy: main's policy, whose verdicts select prices and stripImages
  applies, takes its count and bytes from resolvedImageLimits().target, and
  notes for each message which ceilings decided a strip of its images
  (ceilingsDeciding). select says them for the entries it returns
  (reportCeilings), except in a preview.
- src/strategies/autobiographical.ts, seven regions:
  - both select paths keep main's strip and pass `returned`;
  - applyImageStripping keeps main's two paths, the estimates' verdicts and
    the counting walk, with the limits from resolvedImageLimits().target.
    Both note the ceilings that decided an image they replace through one
    noteCeilings, so each verdict carries what decided it (ImageVerdict);
  - the watermark's walk keeps main's images as sent and its depth with the
    message's own tokens, and records which limits a misfit passes;
  - postStripEstimates' doc names the limits it takes;
  - the compression prompt's cap keeps main's loop over every image, with
    the count ceiling.

Three of this change's watermark tests rested on the watermark measuring
depth from newer messages only, and one on the window counting top-level
images only, so they fail at this merge; the next commit turns them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/context-manager that referenced this pull request Oct 23, 2026
…s do

The live window now counts every image, nested and generated ones
included, and an image's own message toward its depth (anima-research#219). So:

- live-image-limits.ts' account and the added and changed fragments say the
  ceilings bound the images the window counts, as its limits count them,
  and that an image inside an opaque provider item isn't counted.
- The breaking fragment's sibling line names agent-framework 0.24.0 and
  connectome-host 0.12.0, which depend on ^0.18.0.
- The test that pinned a nested and a generated image passing the count
  ceiling now pins the ceiling binding them.
- Three watermark tests had rested on the watermark measuring depth from
  newer messages only. Each is rebuilt on the same rule with the depth
  counted as anima-research#219 counts it: the ceiling's line when only the ceiling moves
  the watermark, the quiet when the depth limit brings the move on, and the
  quiet for an image a ceiling's move stripped that later messages took
  past the depth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The live-image strip can count an image twice when pairing repair moves the tool result that holds it

2 participants