[nissa-seru]

With foldingStrategy: 'kv-unified', the solver credits a cached prefix through the accepted request's cache markers, matching the reference's layout units by kind and key: a raw unit by its message id, the head window by head, a recall pair by its summary id. What renders under an unchanged key can still differ from the request that cached it, and a provider serves a marker's prefix only while every byte before it is unchanged. The solver kept crediting every marker past such a change. This PR records what each marker's prefix was as sent, and leaves out of the reference the solver weighs each marker whose prefix no longer renders that way in the presentation the reference kept.

Problem

A message changes in place under its key when:

• ContextManager.editMessage rewrites it, as agent-framework does to a refusal-rewind marker, to a module's editMessages and to a Discord message that was edited;
• its image is stripped, once newer images reach maxLiveImages or leave it no room in maxLiveImageBytes, or once the conversation from it on runs past imageStripDepthTokens, all on by default;
• its tool result is replaced, once newer results of the same tool reach toolResultMaxLastN;
• a late result moves up over the stub pairing repair had put after its call;
• or a different maxMessageTokens cuts it.

A recall pair changes in place under a different summaryContextLabel or summaryParticipant, and by our reading under a different maxMessageTokens, recallEnvelope or carrierPolicy.

Measured at main 0129fb22 by a probe that compiles a kv-unified store, accepts the request with one wire marker per breakpoint, appends, and compiles again. The wire allows the prefix through the last accepted breakpoint before the first message whose bytes changed.

Case	Changed in place	Solver credited	Wire allows	Solver's churn
Everything in the recent window, maxLiveImages 1	the oldest message's image, stripped when a newer one arrived	6,021	0	0
Six messages raw in the middle	the fifth message's image	12,025	8,004	0
The same, toolResultMaxLastN 1	the fifth message's tool result, replaced when a second result of the same tool arrived	10,557	6,029	0
An image in the head window	the head message's image	10,024	0	0
Control: no new image	nothing	13,611	13,611	0

By our reading, every engine (the terminal evaluator, the Pareto and packed solvers, the exact policy) matches the reference by kind and key, and the immutable prefix hash agent-framework passes covers the tools and the system prompt. The tests below fail the same way at 7f963c75, this PR's base.

Changes

• An accepted request's cache reference carries a render record (ProviderCacheReference.render, a CacheRenderRecord). It's taken from the entries the compile emitted, after every pass that runs before markers are placed. For each marked entry, the record keeps:

  • its identity, the message it ends on or its layout key;
  • a hash of every entry up to it, by participant and whole content, raw replay forms included.

  Acceptance carries it into the reference. A compile without kvUnifiedImmutablePrefixHash takes none, since acceptance keeps no reference without one.

• Before each solve, the check rebuilds the entries of the continuation that keeps the reference's presentation, with this compile's windows:

  • folded what the reference's recall units cover, through each summary's sources down to the L1s, unless it was a raw unit;
  • raw everything else: the reference's raw units, and what none of its recall units covers, which is what came after it or has left the head window since;
  • cut by cappedForSend;
  • a body's raw shards in the middle joined as the middle's shard path joins them, and shards elsewhere through mergeAdjacentBodyGroupRaw;
  • each of the reference's recall units with a message still present rendered where its folded run starts, by recallPairEntries, which the middle's two emission paths now share;
  • then postProcessToolEntries (trim, pairing repair, last-N) and applyImageStripping.

  It hashes them the same way. It leaves out a marker whose prefix differs, whose entry it can't find, or that sits on the reference's head unit while its message no longer ends this compile's head window. A recall pair whose summary throws when the rebuild renders it is left out, so the markers past it are dropped rather than the compile refused. So is one none of whose messages remain, or whose summary no longer exists, since the emission sends neither.

• A reference without a record that describes its markers, such as one a release before this one wrote, is credited as before until the next request is accepted.

• A self-check where a compile keeps the reference's presentation, that is, the reference's units (or the first of them, where the newest messages were removed) and then a raw unit per newer chunk. There, the record also hashes what was sent at the reference's marker identities and compares them with the check's rebuild. A difference is the rebuild's, and it's logged once per strategy instance, which then stops comparing: [kv-unified] the cache check rebuilt the prefix through … differently from what this compile sent ….

What each part is for

• The record from what was sent: the hash compares against the bytes that went out, not a model of them. A rebuild that renders a message differently at every compile then drops markers rather than crediting them, and the tie cases below catch that. One that misses a change the emission makes, such as a pass it lacks, credits the changed prefix. The cut cases catch that for each change they make. For a recall pair's cap, envelope and carrier policy, which no case changes, it rests on our reading.
• The self-check: the rebuild is a second rendering path that has to agree with the emission. A compile that keeps the reference's presentation is the one place where what it should have rebuilt is at hand, so the check is compared there, in either direction. It adds up to four prefix hashes to the walk the record already makes.
  • In CI, every test in kv-unified-render-state fails on the self-check's warning unless it expects one. A test takes only the warnings it names, so one that expects another warning can't take a self-check warning along with it. Elsewhere in the suite a warning fails nothing, so a later change to the emission is caught in CI only where those tests reach it.
  • In production, the warning is what shows it.
• Whole-content fingerprints of every entry: every way the strategy's bytes change shows in an entry's content. That covers an edit, a strip, a replacement and the N its text names, a stub, a cut, a raw form released, and a recall pair's cap, envelope, carrier policy, label and participant. So no rule has to list them. The hash is of each object as the strategy holds it, key order included, so it errs toward dropping. By our reading, a summary reloaded after a restart with its keys in another order can drop the markers past it once, until the next acceptance's record.
• The rebuild: to price the change at the compile where it appears, the check needs the bytes before the solve.
  • It reuses the emission's own helpers and passes, recall pairs included. So the entry list, and with it the pairing repair's look-ahead, is the emission's wherever nothing changed: by our reading, and in every tie case below.
  • It copies only the middle's walk and its raw-shard composite, which selectAdaptive builds inline.
  • The recall units come from the reference's layout, each under the raw unit before it, since by our reading the layout and the emission both place a folded run's pairs where the run starts.
• The head window's end: the head is one unit, so a marker on it stands for the head's end, and it holds only while its message still ends the head window. A head grown past that message would otherwise be credited through messages the provider never served.
• Checking the record prefix by prefix: a record persisted short, malformed or in another shape is credited as before rather than read, since a null prefix would otherwise throw on every compile.
• A recall pair the check can't render (its summary throws) is left out, with a warning once per strategy instance. A fault then drops the markers past it rather than refusing the compile. A summary that no longer exists isn't the check's fault, and it's left out silently.
• Leaving out a recall unit none of whose messages remain: no candidate can render it, so the cache model, which credits only up to the first unit where a candidate parts from the reference, credits nothing past it either way. It keeps the self-check true where the newest messages were removed and that unit shared its place with an earlier one that still renders. There, pushing its pair would make the rebuild differ from what was sent.
• Classing a message by coverage, not position: removing messages, the newest ones included, as agent-framework does when a Discord message is deleted or an operator suppresses messages, leaves the rest classed as the reference had them. A cutoff at the reference's newest raw unit moved back when that message went, and rendered folded history raw.
• Grouping recall units under their anchor in place, and finding summaries through summaryById: the rebuild's setup is linear in the layout and in the messages the reference's recall units cover, and doesn't scan every summary the resident has made. Each recall unit's own walk of its summaries says whether any of its messages remain, with a set of the summaries that walk has reached, so none is walked twice within it.
• A hash and an identity per marker: that's what a provider's cache needs, a prefix through a marker, unchanged. It persists as at most four hashes and four ids.
• Memoized block and content hashes: an entry whose content is its message's own, as most are, costs a lookup. The memo keeps only what this compile and the one before fingerprinted, so its size follows the raw population, not the history.
• Filtering the reference before the solve: every engine gets the same corrected credit from one place, and none of them changes.
• Left out:
  • Carrying the render in the unit keys. It would reach every engine's matching, reconcile, and the terminal evaluator's reuse of earlier unit codes, for the same credit.
  • A check per candidate. It would reach the engines too. The check renders the presentation the reference kept, and by our reading two kinds of candidate differ from it:
    • One that folds a newer image's or result's message can render an earlier message as the reference had it. It's credited less than the provider would serve.
    • One that renders raw a chunk the reference showed folded can tip a newest-first count over an earlier raw message. It's credited as before this PR.
      • For images, that needs a middle image within imageStripDepthTokens. The depth rule rules that out wherever the recent window is larger than the depth, as at connectome-host's default sizes.
      • For tool results, it needs fewer than toolResultMaxLastN newer results of that tool raw in the continuation.
  • The head window's members. A message that has left the head window for the middle since acceptance renders raw in the rebuild, as the head rendered it, unless one of the reference's recall units covers it. By our reading, the cache model's walk parts from the reference after the head unit anyway, since the units after it have moved.
  • Refusing a compile, or dropping the reference, on a self-check difference. A difference means the cost model's credit may be off, and refusing would stop a resident over that. The warning says where they first differ.
  • Message-position injections. compile splices beforeUser and afterUser injections in after the strategy's selection, so neither the layout nor the record sees them. They move to each new last user message, and the provider then rewrites the previous request's tail. main credits it by key already, and this PR doesn't change that. At this head, a probe with eight messages of about 2,000 tokens gave the solver's churn as 0 where the wire allows 6,903, and 18,409 with everything in the recent window, for an injection before or after the last user message.

What doesn't change

• What a compile sends: the passes after selection strip, replace and repair as before. The check runs them again on entries of its own, and by our reading they copy what they change, so a compile's entries and the store's content are left as they are.
• The engines, the layout and its keys, marker placement and reconcile.
• Prices and plans: nothing here estimates tokens.
• Stored data across versions: a reference with a record loads in a release before this one as it does today. The record is a field that release doesn't read, kept until its next acceptance replaces the reference. Run at 0547be76, 0.16.0's main, with a record added to a persisted reference in the shape this PR writes: the reference loaded, the next compile's unchanged prefix was credited whole (churn 0), and the next acceptance wrote a reference without a record.

Tests

• kv-unified-render-state, new.

  • The cut cases: each compiles, accepts with one wire marker per breakpoint, changes something and compiles again. It requires the selected cut's churn to be what the wire allows: what the reference carried, less the prefix through the last accepted breakpoint before the first message whose bytes changed, with the reference's markers tied to the wire's breakpoints. The changes:
    • an image stripped by count in the recent window, in the middle and in the head window;
    • one stripped by depth, and one by maxLiveImageBytes;
    • a tool result replaced by last-N, in the middle and in the head window;
    • an edit, and an edit to a block's raw replay form alone;
    • an edit before a later strip, where the edit decides the cut;
    • a late result moving up over its stub;
    • a recall pair rendered under another summary label, and a recall answer under another participant;
    • a message cut by a lower maxMessageTokens after a reopen;
    • a control where nothing changes.
  • The tie cases: each accepts, appends without changing anything in place, and compiles again. Every marker must survive the check, and the churn must be the wire's. They cover:
    • a stripped image, a replaced result, a pairing stub, a cut message and a cut tool input;
    • a body's raw shards in the middle, and in the recent window;
    • the head window with an image and a tool call, a head window ending in a body's merged shards, and a head window that starts past the first message after a reset;
    • with history folded, a marker on a recall answer, one after a raw unit with a message pinned raw, and a body whose first shards are folded and the rest raw.
  • The head window changing:
    • reset after acceptance, where nothing is credited;
    • grown by config over an edited message, where the old head marker isn't credited for the larger head;
    • shrunk so its old last message is in the recent window, where the old head marker isn't credited either.
  • Records:
    • a compile without kvUnifiedImmutablePrefixHash takes none;
    • a dry compile leaves the record as the real compile made it;
    • acceptance keeps the record taken when its submission began;
    • a reference reloaded from the store keeps an unchanged prefix's markers and drops a changed one;
    • a reference persisted with no record, with one shorter than its markers, with one of another shape, with a null prefix first or later, or with hashes that aren't strings is credited as before once. That over-credits by what the wire allows, and the test says so. It still weighs the cache, and the next acceptance's record catches the next change;
    • a recall pair the check can't render, in the middle of the layout, drops the markers from it on and keeps those before it, says so once over two compiles, and the compile goes on;
    • the fingerprint memo reuses what an append leaves unchanged, and forgets what no longer renders.
  • The self-check:
    • a rebuild made to lack the image strip credits the stripped prefix, and the self-check says so once over two compiles;
    • a rebuild made to render the last marker's message differently, with several markers, drops that marker alone, and the self-check says so once;
    • a record taken where the self-check compared holds only the markers' prefixes, so after a second acceptance the next strip is still caught.
  • Messages removed after acceptance, and summaries:
    • with the newest messages removed, the markers on what remains are credited, and the self-check compares the reference's first units and finds nothing;
    • with every raw unit after the last recall unit removed and a new turn added, the marker on that recall unit is still credited, since its messages are still classed as folded;
    • with one message of a recall unit removed, every marker past it is still credited, since the unit renders through its other messages;
    • with the messages of the first of several recall units removed, no marker past it is kept, since its pair is sent nowhere;
    • with the reference naming a summary that no longer exists, the markers before it are credited and none from it on, and the compile goes on;
    • a marker past an L2 recall answer is credited when nothing changed in place.
  • Every test in the file fails on a self-check warning it doesn't expect.

• Runs: this PR's CI runs the typecheck, the build and the suite on its legs. On Linux x86_64 with Node 24.22.0, at this head's tree, 5ef1a797, from a clean checkout, the typecheck and the build are clean, and npm test passes 2,258 of 2,258. main 7f963c75's CI (run 39258345454) passes 2,205 of 2,205 on each leg, and the difference is this PR's 53 new tests.

• Red-first, in the same run, with the base's autobiographical.ts (7f963c75's) and this head's tests: 37 of the new file's 53 tests fail. The 16 that pass are those where nothing the accepted request sent changes, which the base credits whole. They are the control, the nine tie cases, the three folded ties, the L2 tie, the partial-removal case and the reset head's tie. kv-unified-marker-boundaries and kv-unified-strategy-integration pass 10 of 10 and 26 of 26.

• Removals, in the same run, each piece put back alone. All are caught but one:

  • the solver given the reference as it stands, or acceptance not carrying the record: every change case, the head changes, the reload test and the record tests;
  • a reference without a usable record having every marker left out: the record variants. Its shape, length and element checks, each removed alone: the other shape, the short record, and the null prefixes and the hashes that aren't strings;
  • a marker on the head unit counted wherever its message has gone: the grown head and the shrunk head;
  • the rebuild rendering raw only the reference's raw units: the image and tool-result change cases, and the record variants;
  • the rebuild classing by position after the reference's newest raw unit, as it once did: the test with every raw unit after the last recall unit removed;
  • a summary above L1 covering nothing: the L2 tie;
  • a recall unit rendering only while every one of its messages remains: the partial-removal case;
  • a recall unit none of whose messages remain still rendered: the removed first recall unit, and the reference naming a gone summary;
  • a summary that no longer exists walked as one that does: the reference naming a gone summary;
  • the rebuild not cutting raw messages: the cut tie case and the lower cap after a reopen;
  • the recall units after a raw unit getting no pair, or a keyed placeholder standing for each pair: the pinned case and the folded cases;
  • a recall pair the rebuild can't render refusing the compile: the unrenderable recall pair's test;
  • the rebuild sending a middle body's raw shards one by one, or skipping the head's and recent window's merge: the middle composite tie case, and the recent-window body and the head ending in a body;
  • the rebuild skipping trim, pairing repair and last-N: both tool-result change cases, the late result, and the replaced-result, stub and cut-input tie cases;
  • the rebuild skipping image stripping: the image change cases, the stripped-image and head-window tie cases, the record-split test, and the reload and record tests;
  • only entries holding a stored message hashed: the late-result case, the summary label and participant, the removed first recall unit, and the unrenderable recall pair;
  • the self-check finding no difference, or comparing only the first marker's prefix: the second drift test and the unrenderable recall pair's test. Saying it at every compile: both drift tests;
  • the self-check comparing whatever the presentation: the removed raw units after the last recall unit, the reference naming a gone summary, and the memo test, each through the file's afterEach;
  • the record keeping the self-check's prefixes as well as the markers': the record-split test;
  • a record taken without the immutable prefix hash, the memo never forgetting, or the memo never reusing the last compile's hashes: their own tests.

  Not caught: the self-check comparing a layout whose units past the reference's aren't all raw. By our reading, the rebuild renders newer messages raw, and their images or results could strip or replace an earlier one that the emission, folding them, leaves as it was, which would draw a false warning. No test reaches a reference old enough for newer messages to fold. A run that tried, with 24 newer messages folded at seven budgets in 14 runs, found each later layout re-cut the reference's units rather than keeping them as a prefix.

Other open pull requests

None of the 18 open pull requests gains a conflict region from this one: each merged onto main (7f963c75) and onto this head, counting the conflict regions in each file. This head is one commit on that main, and its two warnings go through this.diag, which #245 added, so each names its namespace.

What it costs a running host

No price, plan or key changes. At a compile after a message changes in place inside a credited prefix, the solver now prices the churn the provider charges, where it counted none, so its choice there can differ: it may fold what follows the changed message, since that prefix is lost either way. The most one request can cost is still one window's write. The upgrade itself moves nothing: a reference written before it is credited as before, once.

Per compile, measured on Linux x86_64 with Node 24.22.0, on 12 cores under a load of 45 to 55 from other work, at this head. The stores rendered every message raw, which the check fingerprints whole. Each store had an image every 25th message, whose data is four bytes except where said, and a tool call and result every 10th pair. Figures are the median of five warm compiles after a first. Under that load single compiles vary widely, and at 6,000 messages the check ranged from 16 to 60 ms.

• 2,000 messages: the check took 5.8 ms and the record 5.3 ms of a 92 ms compile.
• 6,000 messages: 29 ms and 6.3 ms of a 214 ms compile.
• 2,000 messages whose images are 1.5 MB of base64 each, six of them live: 4.6 ms and 2.5 ms of a 63 ms compile.
  • The first compile hashes the live images' data once, and its record took 40 ms.
  • After an edit, by our reading the store maps every message anew, so the next check hashes every block again. It took 45 ms of a 98 ms compile.

The check's walk is linear in the stored history, as the emission's is, and its passes run over the entries the emission's do. So it repeats work the compile already does once. A folded store also renders each of the reference's recall pairs at every check, as the emission renders the pairs it sends. A long store whose history is mostly folded wasn't measured.

A persisted reference grows by at most four hashes and four ids.

Not verified

• A live host. No compile ran against a provider. The wire receipt is the probe's, one marker per breakpoint, as membrane's computeCacheWireReceipt counts them under cm-owned.
• Production stores: how often a message changes in place inside a credited prefix wasn't measured.

---

• Changelog fragment added: changelog.d/kv-unified-render-state.fixed.md

— Greta-2425 (Claude Opus 5.5). Oscar-1005 (Claude Opus 5.5) agreed the design, and the record taken from what was sent is his. Esther-2584 (Claude Opus 5.5) read the change and its rounds, and found the dropped recall markers with a control of her own. Description checked and posted by Lottie-2661 (Claude Opus 5.5) through the shared nissa-seru account.

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

The PR records hashes of the prefixes an accepted request sent. Before the next solve, it removes cache markers whose rebuilt prefixes no longer match.

• The solver checks rendered prefixes before counting them as cached.

_{Reviews (1) · Last reviewed commit: "fix(kv-unified): don't credit a cache ma..." · Reviewed by Greptile}