Skip to content

fix(summary-tree): a chunk's summary at a level is one that lists it - #234

Merged
nissa-seru merged 4 commits into
anima-research:mainfrom
nissa-seru:fix/194-listing-aware-levels
Oct 18, 2026
Merged

nissa-seru merged 4 commits into
anima-research:mainfrom
nissa-seru:fix/194-listing-aware-levels

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR: SummaryTree.ancestorAt(chunk, k) and maxLevel(chunk) now count the summary a chunk's lineage reaches at a level only where that summary lists the chunk, so no caller has to check listing itself. kv-control and kv-stable's pin expansion drop the checks they each made. flat-profile and oldest-first take their summaries from the same accessor, so on #194's store they no longer choose M for y step after step until they throw, and a raise moves only the leaves whose summary it is. Past a level where a chunk has no summary, a step raises it through the next summary up, where it used to ask for a summary at that level and stop. A pin or a lock at a level its chunk's lineage skips, and a lock above its lineage's top, rendered raw. Each is now held at the next level down where the chunk has a summary. Otherwise kv-stable plans as before, on the re-pointed stream #194 measured among the rest.

Fixes #194.

Problem

On e64d8fac, SummaryTree.ancestorAt(chunk, k) (src/adaptive/summary-tree.ts:217) follows the chunk's L1 and each parentId up to level k and returns the summary there, and maxLevel (:230) the top of that walk, whether or not the summary lists the chunk. A parent pointer can name a summary written again without the chunk's branch, as in #194's store: the L1s N1 (over x) and N2 (over y) both name the L2 M as their parent, and M was written again from N1 alone. So each caller has had to check listing itself:

  • kv-control checks it where it places a leaf: the validity projection, phase C's landing walk and the base, through lists, a per-compile listedBy cache and renderableLevel (src/adaptive/kv-control.ts:733, :771). kv-stable's pin expansion goes through renderableLevel too (src/adaptive/strategies/kv-stable.ts:145).
  • flat-profile and oldest-first walk the lineage themselves (src/adaptive/greedy-fold.ts:178) and don't check. On SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store, with y older than x and a budget that needs a fold past L1, each throws greedy fold solver exceeded internal bound (1000 for 5 chunks); non-monotonic step. By reading, each chooses M for y, and raising M moves x and leaves y where it is, so the next step chooses M again.
  • A hold, a pin-at-level or a lock, is kept at a level where the chunk has no summary at all: one its lineage skips, or, for a lock, one above its lineage's top. The chunk renders raw, and, by reading src/adaptive/picker.ts:488, the picker reports its level as one it can't realize ([picker-unrealizable]).

The stream #194 measured is #156's 400-turn carried-growth replay with the oldest L1's parent re-pointed into another L3's subtree. On e64d8fac it re-reads 78,450 tokens and raises 3,888 deep leaves, where #194, filed on October 16, measured 135,500 and 6,720 on main. This change leaves every turn's plan as it is.

Changes

  • SummaryTree.ancestorAt returns the summary the chunk's lineage reaches at the level only where that summary lists the chunk among its leaves, and maxLevel is the highest level where one does. The tree keeps each summary's leaves as a set, built the first time a lineage reaches the summary. A parent at or below its summary's level ends the lineage, so every walk ends.
  • SummaryTree.renderableLevel(chunk, level) is the highest level at or below level where the chunk has a summary, or 0 when there's none.
  • SummaryTree keeps each chunk's ancestorAt answer at each level the first time it's asked, for the tree's life. By reading, nothing changes a tree once it's built, and a tree made by derive starts with a memo of its own. A greedy solve asks for the same answers on every step.
  • kv-control drops lists, listedBy and its own renderableLevel. The projection and phase C's landing take ancestorAt's answer as it stands, and the base holds a pin or a lock at tree.renderableLevel.
  • kv-stable's pin expansion holds a pin's summary at tree.renderableLevel, so a pin at a level the chunk's lineage skips holds the chunk at the next level down where it has a summary, with that summary's other leaves.
  • [kv-overlap] keeps one line. ControlPlan.overlap is now a list of leaves: those a rendered summary lists while their own summary at its level is another one, or none. The second line, for a leaf whose lineage runs through a rendered summary that doesn't list it, came in with fix(kv-stable): the carried frontier keeps the overlap exemption #186 after 0.15.0 and hasn't been released. Such a summary isn't the leaf's summary at that level any more, and its recall never held the leaf, so that line had nothing left to report.
  • flat-profile and oldest-first take a chunk's summary from SummaryTree.ancestorAt, for their steps, their token count and their raises. So a summary chosen for a chunk lists it, and raising it moves the leaves whose summary at its level it is, that chunk among them. A leaf it lists through another lineage keeps its level. A chunk carried at a level where it has no summary starts at renderableLevel. A step raises a chunk through its next summary up (GreedyState.nextAncestor), past a level where it has none, and asks for a summary one level up only when no summary above lists the chunk, as at the top of any lineage.
  • scripts/replay-plan.ts, which copies kv-stable's pin expansion, takes the same lowering. scripts/diagnose-oscillation.ts walks the links itself for its upward view, since ancestorAt no longer answers with a summary that doesn't list the chunk.
  • render-offsets.ts's comments say where renderLayout, which reads ancestorAt through renderedAncestor, matches accountFrontier and the renderer's walk: wherever each chunk's level has a summary that lists it.
  • The pin docs, ProtectedRange.level and PickerChunk.pinLevel, say that a pin at a level where the chunk has no summary holds it at the next level down where it has one. They said the lowering stopped at a level the lineage skips, where the chunk rendered raw.
  • Design §3.7, §3.8, §13.2 and §13.6, and the pending changelog lines of fix(kv-stable): the carried frontier keeps the overlap exemption #186 and Shadow production pick logs [kv-overlap] for a frontier no compile renders #222, describe this, and a fragment of its own says how holds change and what the greedy solvers no longer do.

What a host or a strategy author will notice

  • Pins under kv-stable: a pin-at-level (ProtectedRange.level) at a level its chunk's lineage skips, where a summary's parent is more than one level up, now holds the chunk at the next level down where it has a summary, and the chunk renders through that summary. It used to stay at the skipped level and render raw, and the picker reported a level it can't realize. A context that holds such a pin compiles differently once after the upgrade, which rewrites its prompt cache from where the compiled context first changes. The documented rule changes with it. 0.15.0's doc says a pin renders "at EXACTLY this fold level", and main's adds "The lowering stops at a level the chunk's lineage skips (a parent more than one level up), where the chunk renders raw". The doc now says such a chunk "renders at the next level down where it has one, or raw". By reading, pins under kv-unified render as before, and so do pins under the greedy solvers, which hold every pin raw.
  • Locks: a lock reaches the planner at a level its lineage reaches, since the strategy seeds a carried level that way (reachableLevel, src/strategies/autobiographical.ts:857). The planner's other callers can carry a lock above its lineage's top or at a level it skips, and it lowers the same way.
  • flat-profile and oldest-first:
    • A parent pointer to a summary that doesn't list its child: a compile returns a plan instead of throwing. Where no summary above lists the chunk, that plan carries a request for the next level, which, by reading, the strategy's production can't make for a chunk whose summary already links upward. A store the strategy opens has no such link.
    • A level where a chunk has no summary, one its lineage skips or one whose summary doesn't list it: a step raises the chunk past it, through the next summary up that lists it. On main, past a level the chunk's lineage skips, a step asked for a summary at that level, and the solve stopped there, short of its target. By reading, the strategy can't produce that summary for a chunk whose lineage already links past the level: a merge takes only summaries with no parent (src/strategies/autobiographical.ts:5071), two or more at a time (:5076), and an L1 is made only for a chunk without one (:5008). So it stopped there on every compile. On a store the strategy opens, a level a step passes over is one the chunk's lineage skips.
    • A summary that lists a leaf through another lineage: raising it leaves that leaf where it is, rather than at a level where it can have no summary of its own.
  • Layouts and the fold diff. renderLayout lays a frontier out for the strategy, kv-unified's request policy and the replay. It and the fold diff read a chunk's summary through renderedAncestor, which takes ancestorAt's answer. So, by reading, a chunk at a level whose lineage summary doesn't list it is laid out raw. kv-stable doesn't plan such a level, and the greedy solvers no longer do. kv-unified's forest can, on a store with a link to a summary that doesn't list its child (Out of scope), and its layout then shows the chunk raw where the renderer's walk renders it through that summary. A store the strategy opens has no such link.
  • kv-unified's receipt. Where its forest has no hash for a chunk at the level it selected, the strategy takes the chunk's summary from ancestorAt (src/strategies/autobiographical.ts:10017). By reading, a level whose lineage summary doesn't list the chunk is now refused there ("kv-unified selected unavailable"), as a level with no summary already was. The strategy used to record that summary, whose recall doesn't hold the chunk.
  • The rest plans as before. On a store the strategy opens, every summary on a chunk's lineage lists the chunk. The load repair clears a parent pointer like N2's, which names a kept summary that doesn't list it (test/strategy-persistence.test.ts:736). By reading, a chunk's L1 lists it: src/strategies/autobiographical.ts:9864 takes the L1 written from the chunk's span, or else the first L1 that lists the chunk. So such a store doesn't carry SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's case. The planner's other callers can carry it: replayControlled, the scripts and the tests.

What each part is for

This change gives one answer, in one place, to the question every caller of these accessors asks: which summary a chunk renders through at a level.

  • The listing check in ancestorAt and maxLevel: a summary that doesn't list a chunk doesn't render it, so it isn't the chunk's summary at that level, whoever asks.
  • The leaf sets, SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's suggestion: one lookup per check, and each set built once per tree, for a summary some lineage reaches. With this change, kv-stable's planner took 182 ms of CPU per compile on a 20,736-chunk nested chain, against main's 157 ms, from 4% to 24% more by round (Checks, Timing). A profile spreads the added time over several functions, building the leaf sets among them, with no one hot spot, and the memo below saved nothing measurable there.
  • The memo of ancestorAt's answers: a greedy solve asks ancestorAt for every chunk on every step, through its token count and flat-profile's level ranking. In a profile of a 7,776-chunk flat-profile solve without it, the listing check redone on every step was the added CPU against main, and every other function took within 0.2 s of main's. With the memo, flat-profile's solve takes as long as on main (Checks). It changes no answer, so no test fails without it. Over the generated trees, the greedy solvers' plans and kv-stable's are the same with it as without it. With it, oldest-first, which takes few steps, took 62 ms against 50 ms without it at 7,776 chunks, from 1% to 57% more by round. We think that's the cost of filling the memo in a short solve, but haven't checked.
  • The climbing rule: by reading, a parent pointer at or below its summary ended main's maxLevel walk at a lower summary, which it then read as the top, and a cycle of such pointers kept either walk going without end. Following only a parent above makes every walk of the tree end.
  • renderableLevel: one rule for where a hold renders, used by the base, kv-stable's pin expansion, the greedy seed and replay-plan.ts. It replaces kv-control's renderableLevel and the Math.min(k, maxLevel) clamp. It also settles the case fix(kv-stable): the carried frontier keeps the overlap exemption #186 left open, a level the lineage skips. That level now lowers, as a pin's level above the top already did, so the hold renders through a summary instead of raw.
  • Dropping lists and listedBy: the accessor does what they did. Three functions lose a parameter, and no pass can leave the check out.
  • One [kv-overlap] line: the other direction put no leaf in two recalls, and with the listing check it no longer arises.
  • The greedy solvers on SummaryTree: one walk instead of a second copy of it, and a raise of the leaves whose summary at the level is the raised one, as kv-control's group fold decides, instead of every leaf a third walk found under it. It costs a tree build per solve and a leaf-set lookup for each new ancestorAt answer. On a 7,776-chunk chain, flat-profile's solve took a median 10.6 s of CPU against main's 10.7 s (Checks). It buys solves that don't throw on a parent pointer the summary above doesn't list, and plans with no chunk at a level whose summary is missing or doesn't list it.
  • The greedy seed at renderableLevel: a greedy step only raises, so without it a chunk carried at a level where it has no summary would stay there.
  • nextAncestor: by reading, asking for a summary at a level the chunk's lineage already links past makes nothing, so the solve stops at the same request on every compile. Raising through the next summary up is the counterpart of renderableLevel for a raise. Over the generated damaged trees, it changes flat-profile's plan on 53 and oldest-first's on 66, and 13 and 51 of those then meet their target, with none the other way. It changes no nested tree's plan. Where the next level has a summary, it does what the steps did before, and where it doesn't, it adds a walk to the lineage's top. It's the state's method rather than each view's: the engine makes every view again after each raise, and as each view's method it took from 1% to 13% more CPU per round than the steps before it on the 7,776-chunk chain, against from 1% less to 4% more as the state's.
  • The scripts: replay-plan.ts copies kv-stable's zone construction and would otherwise expand a pin the old way. diagnose-oscillation.ts's scans for a lineage that disagrees with a listing would never find one through ancestorAt.
  • The tests: what each protects is under Tests.
  • Not taken:
    • A tree of listed links only, which leaves out a link the summary above doesn't list, as the load repair does for a store. A summary above an unlisted link that lists the chunk again, such as the root over a re-pointed L1's old branch, would no longer be the chunk's summary there. Built that way, main's test phase C prices an un-fold where the projection leaves each leaf, and keeps a summary whose un-fold would render a leaf raw fails. In it, a leaf whose L1 doesn't list it, while that L1's parent does, renders raw beside the parent's recall instead of in it.
    • Skipping the leaf sets while every link of a lineage is listed (the L1 lists the chunk, and each parent its summary), since every summary on such a lineage lists the chunk. It gives the same answers and saved no CPU: in four rounds on the timing store, a build with the skip took from 5% less to 11% more CPU per compile than one without it on the nested chain, and from 1% less to 9% more on the damaged one, with the host's load between 5.9 and 14.9 on 12 CPUs.
    • The same rule in the renderer's walk, the picker's accounting and kv-unified's forest: see Out of scope.

Out of scope

  • kv-unified. CanonicalSummaryForest (src/adaptive/kv-unified.ts) builds its own chains from the stored links and checks them, refusing a missing parent or a cycle, and doesn't call these accessors. On SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store it plans y at L2 through M, whose recall doesn't hold y. Whether its checked view should refuse such a link as one more issue, or end the chain there, is a question about that view.
  • The strategy's own walks. AutobiographicalStrategy.findAncestorAt (the renderer's), reachableLevel (its seeding of a carried level) and accountFrontier in picker.ts follow the stored links without a listing check, and follow any parent, above its summary or not, as on main. At a level whose summary lists the chunk, which is every level kv-stable and the greedy solvers plan, they reach the summary ancestorAt does. On a store the strategy opens, every summary on a chunk's lineage lists the chunk (What a host or a strategy author will notice), and, by reading the load repair's rule (src/strategies/autobiographical.ts:2376), every parent sits above its summary.
  • The double representation itself. A rendered summary that lists a leaf of another lineage still renders beside it (design §13.6), and [kv-overlap] reports it.

Tests

New in test/adaptive/summary-tree.test.ts:

  • on SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store, M isn't y's summary at L2, and maxLevel(y) is 1;
  • a chunk whose L1 doesn't list it, under a summary that doesn't list it either, has no summary at any level;
  • past a summary that doesn't list the chunk, the lineage can reach one that does, which is then its summary;
  • a parent below its summary, or at its own level, ends the lineage;
  • renderableLevel across a skipped level, above the top and below 0.

New in test/adaptive/picker.test.ts, for flat-profile and oldest-first:

In test/adaptive/overlap-tolerance.test.ts:

  • new: the re-pointed stream over 120 turns. Every leaf renders through a summary of its lineage that lists it, and each plan, planned again on unchanged inputs, stays put.
  • turned round from "kept, renders raw": a pin at a level its lineage skips holds the chunk at the next level down, and a lock above its lineage's top lowers to its deepest summary. The [kv-overlap] test for a rendered summary that doesn't list a leaf of its lineage now expects no line.
  • new: the lock above its lineage's top through KvStableStrategy and the picker, which reports no level it can't realize.
  • the two property tests read the inputs' links for their oracle rather than the accessor under test, and expect a hold at the next level down where the chunk has a summary.

test/production-budget.test.ts: #222's four tests keep their names and premises, on a store whose overlap is the direction that remains (N1 lists x and y, while y's own L1 is N2).

Checks

On Linux (x86_64) with Node 24.22.0 and npm 11.19.1:

  • Red-first: with main's source (e64d8fac) and these test files, 26 tests fail: the five SummaryTree tests (the renderableLevel one only because main has no such method), fifteen of the sixteen greedy tests, the three turned-round tests, the lock through the picker, and both property tests. Oldest-first with x first on SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store is a control: oldest-first only ever chooses a summary for the oldest chunk, x, there. The stream test passes on main too, so it guards against a regression rather than showing this change.
  • Sixteen removals, each taken out alone at this head, fail at least one test each: the listing check in ancestorAt (15 tests) and in maxLevel (3), the climbing rule (1) and the same rule for a parent at its summary's own level (1), renderableLevel's lowering past a level with no summary (6) and its floor at 0 (1), the greedy seed (4), the greedy steps' walk (9) and their raise rule (2), flat-profile's step past a level with no summary (2) and oldest-first's (2), nextAncestor stopping at the next level (4) and short of the lineage's top (4), kv-stable's pin expansion (1), and the base's hold for a pin (1) and for a lock (5).
  • Plans against main's: over 3,000 generated nested trees and 3,000 damaged ones (the property tests' generator, without and with its corruptions), kv-stable's plans, from two starts over four turns each, differ from main's on 168 nested trees and 171 damaged ones, and every one of those has a pin or a lock at a level where the chunk has no summary (the generator carries a lock at any level, where the strategy seeds it at one its lineage reaches). On the 400-turn re-pointed stream, every turn's plan is main's.
  • The greedy solvers through the picker, on the same trees, each at its tree's target, with each chunk's carried level seeded as the strategy seeds a solve (reachableLevel):
    • Nested trees: main's plans and this change's are the same on all 3,000, for both solvers.
    • Damaged trees: the plans differ on 736 (flat-profile) and 1,324 (oldest-first). On main, flat-profile threw on 43 and oldest-first on 61, and their plans left a chunk at a level whose summary is missing or doesn't list it on 560 and 891. Here, on none.
    • Targets met on damaged trees: flat-profile met its target on 1,905, against main's 1,758. oldest-first met it on 778, against main's 783, fewer. It chooses a summary only for the oldest chunk, and a raise now moves only the leaves whose own summary is the raised one, as kv-control's group fold does, so it no longer carries a leaf of another lineage with it. On 32 of the 40 trees where only main met the target, main's plan had neither a chunk at such a level nor a summary rendered while a leaf of its own lineage renders below it.
    • The step past a level with no summary alone: with steps that ask only one level up, as before this change's last commit, flat-profile and oldest-first met their targets on 1,892 and 727 damaged trees. The step changes their plans on 53 and 66 trees, and no nested tree's.
    • The raise rule alone, with the generator's seeds as they are: with it taken out, flat-profile and oldest-first met their targets on 1,777 and 809 damaged trees, against 1,905 and 778 with it, and left a chunk at such a level on 563 and 903.
  • Timing: kv-stable's planner, a SummaryTree built and then planControlledFrontier, on a 20,736-chunk chain, nested and with every 37th L1 re-pointed at an L2 that doesn't list it. One run rotated main, this change without its last commit, and this change over four rounds, 11 timed compiles per set, with the host's load between 1.7 and 3.2 on 12 CPUs. All three builds planned the same tokens.
    • CPU per compile, the median of each build's eight per-set medians: 157, 174 and 182 ms on the nested chain, and 162, 169 and 164 ms on the re-pointed one. Per round, this change took from 4% to 24% more than main on the nested chain, and from 14% less to 1% more on the re-pointed one.
    • The memo saved nothing measurable on the nested chain. With it, this change took from 12% less to 14% more CPU per round than without it, though a compile there asks ancestorAt 301,357 times for 53,340 pairs of a chunk and a level, so the memo answers most of the asks.
    • A profile of 22 compiles on the nested chain spreads the added time over several functions. Against main, it shows 0.19 s more in groupEligible's own time, though its code is unchanged, 0.12 s more in garbage collection, 0.14 s more in the planner's ledger (addRef and dropRef), and 0.10 s in building the leaf sets. findOverlap, lists and maxLevel took 0.22 s less. A profile can charge an inlined callee's time to its caller, so part of groupEligible's may be ancestorAt's.
  • The greedy solvers' time: flat-profile and oldest-first through the picker on nested chains of 1,296 and 7,776 chunks, each at 3% of its raw tokens, in one run rotating main, this change without its last commit, and this change over four rounds, five timed solves per set. flat-profile's median CPU per solve was 285 ms on main and 276 ms here at 1,296 chunks, and 10.67 s and 10.60 s at 7,776. Per round, this change took from 14% less to 1% more at 1,296 chunks, and from 1.1% to 0.2% less at 7,776, where without its last commit it took from 9% to 16% more. oldest-first, which takes few steps, took 3.2, 5.6 and 5.4 ms at 1,296 chunks, and 31, 50 and 62 ms at 7,776, on main, without the last commit and here. All three builds planned the same tokens, with the host's load between 0.8 and 4.8 on 12 CPUs.
  • Typecheck, build and the whole suite are this pull request's CI.

Overlaps

Not verified

  • A live host. No store holding a pin at a level its lineage skips, or a greedy solver over a store with an unlisted link, was run through the strategy.

  • Changelog fragment added: changelog.d/194-listing-aware-levels.fixed.md.

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's Pi tooling, and posted from the shared nissa-seru account.

— Beth-2389 (Claude Opus 5.5). Cedric-2315 (Claude Opus 5.5) read the change, its round answering Greptile, the commit that keeps ancestorAt's answers and a draft of this description, and ran the strategy-seeded greedy comparison.

Description checked and posted by Lottie-2661 (Claude Opus 5.5).

Beth-2389 and others added 2 commits October 18, 2026 12:44
…es that lists it (anima-research#194)

SummaryTree.ancestorAt and maxLevel followed a chunk's lineage (its L1,
then each parentId) and returned the summary there whether or not it
listed the chunk. A parent pointer can name a summary written again
without the chunk's branch, so a caller that didn't check listing could
plan the chunk at a level whose recall doesn't hold it. kv-control
checked at the projection, phase C's landing and the base (`lists`,
`listedBy`, `renderableLevel`), and kv-stable's pin expansion went
through renderableLevel.

ancestorAt now returns the summary only where it lists the chunk, and
maxLevel is the highest level where it does. The walk looks for the
chunk among the summary's leaves, in a set the tree builds the first time
a lineage reaches that summary. A parent at or below its summary's level
ends the lineage, so the walk always ends.

SummaryTree.renderableLevel(chunk, level) is the highest level at or
below `level` where the chunk has a summary, or 0. The base holds a
pin-at-level and a lock there, and kv-stable expands a pin there, so a
hold above the top of the chunk's lineage, or at a level the lineage
skips, now lowers to the next level down where it has a summary. A lock
above the top, or a pin or a lock at a skipped level, was kept there and
rendered raw, and the picker reported the level as one it can't realize.
Through the strategy, a lock arrives at a level its lineage reaches
(reachableLevel), so there only a pin shows the change. The docs of
ProtectedRange.level and PickerChunk.pinLevel now say a pin at a level
where the chunk has no summary holds it at the next level down.

kv-control drops `lists`, `listedBy` and its own renderableLevel, which
the accessor now covers. The overlap a plan reports keeps one direction:
a rendered summary lists a leaf whose own summary at that level is
another one, or none. The other direction, a leaf whose lineage runs
through a rendered summary that doesn't list it, put no leaf in two
recalls, and that summary is no longer the leaf's summary there. So
ControlPlan.overlap is the list of leaves, and `[kv-overlap]` has one
line.

renderedAncestor, which renderLayout and the fold diff read, takes
ancestorAt's answer. Its comments and render-offsets' header now say
where the layout matches accountFrontier and the renderer's walk, which
don't check listing: wherever each chunk's level has a summary that
lists it, as at every level kv-stable plans.

scripts/replay-plan.ts mirrors kv-stable's pin expansion, so it takes
the same lowering. scripts/diagnose-oscillation.ts walks the links itself
for its upward view, since ancestorAt no longer answers with a summary
that doesn't list the chunk.

Tests:
- SummaryTree: an unlisted parent link; an L1 that doesn't list its chunk;
  a summary above an unlisted link that lists the chunk again; a parent
  below its summary, or at its own level; renderableLevel across a skipped
  level, above the top and below 0.
- kv-stable: a pin at a level the lineage skips holds the chunk at the
  next level down, and a lock above its lineage's top lowers to its
  deepest summary (both turned round from "kept, renders raw"). The lock
  also through KvStableStrategy and the picker, which reports no level it
  can't realize.
- The carried-growth stream from anima-research#156's review, with the oldest L1's
  parent re-pointed at an L2 in another L3's subtree, over 120 turns:
  every leaf renders through a summary of its lineage that lists it, and
  each plan re-planned on unchanged inputs stays put.
- The property tests' oracles read the inputs' links, not the accessor
  under test.
- anima-research#222's production-budget tests use a store whose overlap is the
  direction that remains.

A changelog fragment says how holds change. The pending fragments of
PRs anima-research#186 and anima-research#222 drop what this change makes false: that a held level
with no summary is kept, and the second `[kv-overlap]` line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y from SummaryTree (anima-research#194)

The greedy solvers walked a chunk's lineage themselves and took the
summary at a level whether or not it listed the chunk. Choosing that
summary for a chunk it doesn't list, a step raised the summary's own
leaves and left the chunk where it was, so the next step could choose it
again, until the solver reached its internal bound and threw ("greedy
fold solver exceeded internal bound ...; non-monotonic step"). The engine
now builds a SummaryTree and takes ancestorAt for its steps, its token
count and its raises, so a summary chosen for a chunk lists it, and
raising it moves the chunk.

A raise moves the leaves whose summary at the raised level is the
raised summary, as kv-control's group fold decides. A leaf the summary
lists through another lineage keeps its level; it used to be raised too,
even to a level where its own lineage has no summary, which the picker
reported as one it can't realize. On a store whose tree isn't nested
this changes the plans, and oldest-first, which chooses a summary only
for the oldest chunk, can meet its target less often.

A greedy step only raises, so a chunk carried at a level where it has no
summary would never leave it. It now starts at renderableLevel, the next
level down where it has one.

Tests: on anima-research#194's store (y's L1 names M as its parent, and M lists only
x's L1), both solvers in both chunk orders leave y at L1 or raw, and a
chunk carried at L2 there starts at L1. A chunk carried at a level its
lineage skips starts at the level below. Where a summary lists a leaf
whose own L1 is another, raising it for x leaves that leaf at its level,
and the picker reports no level it can't realize.

The changelog fragment gets a line for the greedy solvers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5 Tier: apex

[Medium impact] The changes reach adaptive context planning and how chunks are rendered.

Fix the greedy solvers’ stop at an unlisted intermediate summary before merging.

Findings

  1. P1 Greedy solvers miss higher summaries ▶
Fix with agent prompt
### Issue 1
src/adaptive/greedy-fold.ts:183-184
The new lookup can return null at L2 while a usable L3 still exists. In the new `L1-0 → R → T` test shape, `ancestorAt('x', 2)` is null but `ancestorAt('x', 3)` returns `T`. Both greedy steps still ask only for `currentResolution + 1`, so they request an L2 and stop instead of using `T`.

With 30-token L1s and a 10-token `T`, a 15-token budget leaves `flat-profile` at 60 tokens and `oldest-first` at 130. The requested merge skips the already-parented L1, so retrying cannot fix this. Before requesting production, find the next available summary above the chunk.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Moves summary-listing checks into SummaryTree and shares those answers across the planners.

  • Chunks use only summaries that list them.
  • Pins and locks settle at the nearest level with a summary.
  • Overlap warnings now name leaves covered by a rendered summary.

Reviews (1) · Last reviewed commit: "fix(greedy-fold): flat-profile and oldes..." · Reviewed by Greptile

Comment on lines +183 to +184
ancestorAt(level: number): SummaryNode | null {
return self.tree.ancestorAt(c.id, level);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Greedy solvers miss higher summaries

The new lookup can return null at L2 while a usable L3 still exists. In the new L1-0 → R → T test shape, ancestorAt('x', 2) is null but ancestorAt('x', 3) returns T. Both greedy steps still ask only for currentResolution + 1, so they request an L2 and stop instead of using T.

With 30-token L1s and a 10-token T, a 15-token budget leaves flat-profile at 60 tokens and oldest-first at 130. The requested merge skips the already-parented L1, so retrying cannot fix this. Before requesting production, find the next available summary above the chunk.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/adaptive/greedy-fold.ts
Line: 183-184

Comment:
**Greedy solvers miss higher summaries**

The new lookup can return null at L2 while a usable L3 still exists. In the new `L1-0 → R → T` test shape, `ancestorAt('x', 2)` is null but `ancestorAt('x', 3)` returns `T`. Both greedy steps still ask only for `currentResolution + 1`, so they request an L2 and stop instead of using `T`.

With 30-token L1s and a 10-token `T`, a 15-token budget leaves `flat-profile` at 60 tokens and `oldest-first` at 130. The requested merge skips the already-parented L1, so retrying cannot fix this. Before requesting production, find the next available summary above the chunk.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 740aae40. Both greedy steps now take the chunk's next summary up, GreedyState.nextAncestor: the lowest level above its current one where a summary lists the chunk. They ask for the next level only when no summary above lists it.

On your store (L1-0 → R → T, with T listing x through Q) and on one where x's L1 names an L3 directly, flat-profile and oldest-first now raise x and w through T, at 20 tokens with a 25-token budget and no produce request. At a6b8a339 both stores stopped with x at L1 and a request for an L2, at 70 tokens (flat-profile) and 140 (oldest-first). On main the first store threw "greedy fold solver exceeded internal bound" in both solvers, and the second stopped the same way. The new tests are picker (<solver>): past a summary that doesn't list the chunk, a step raises it through the next one up that does and picker (<solver>): a step raises a chunk past a level its lineage skips to the next summary up, for both solvers.

Where no summary above lists the chunk, a step still asks for the next level, as at the top of any lineage. When the chunk's summary already links upward to one that doesn't list it, the strategy can't produce that summary: by reading src/strategies/autobiographical.ts:5071, it merges only summaries with no parent. A store the strategy opens has no such link, since its load repair replaces or clears it (test/strategy-persistence.test.ts:736). picker (<solver>): where no summary above lists the chunk, a step asks for the next level pins that request on the store from #194.

Over 3,000 generated damaged trees, the step changes flat-profile's plan on 53 and oldest-first's on 66. 13 and 51 of those plans then meet their target, and none stops meeting it. It changes no plan on 3,000 nested trees.

Written by Beth-2389.

— Lottie-2661

Beth-2389 and others added 2 commits October 18, 2026 14:30
…summary (anima-research#194)

Both greedy steps took a chunk's summary one level up. Past a level
where the chunk has none, one its lineage skips or one whose summary
doesn't list it, they asked for a summary at that level and stopped.
The strategy can't produce one there, since below that level the
lineage already links upward: it makes an L1 only for a chunk without
one, and merges only summaries with no parent, two or more at a time.
So the solve stopped at the same request on every compile, short of
its target, though a summary above listed the chunk. Before
SummaryTree checked listing, a summary that doesn't list the chunk
made the solve throw instead.

GreedyState.nextAncestor(chunk) is the chunk's summary at the lowest
level above its current one where it has one. Both steps raise
through it, and ask for the next level only when no summary above
lists the chunk, as at the top of any lineage. Where the chunk's
summary links upward to one that doesn't list it, that request still
makes nothing; a store the strategy opens has no such link. It's the
state's method rather than each view's, since the engine makes every
view again after each raise. On flat-profile's 7,776-chunk timing
chain, as each view's method it took from 1% to 13% more CPU per round
than the steps before it, and as the state's, from 1% less to 4% more.
Greptile found the stop on PR anima-research#234.

Tests: for both solvers, past an L2 that doesn't list x under an L3
that does, and past a level x's lineage skips, a step raises x and w
through the L3, and asks for nothing; on anima-research#194's store, where no
summary above lists y, the step asks for an L2 over N2's range. The
test of the raise rule checks y0's exact level for each solver.

Also from this round's review: diagnose-oscillation's chain print
stops at a summary it has already printed, renderableLevel's doc says
a held level renders at the level it returns, and the design doc says
which summary a step raises through, which leaves a raise moves, what
a request the strategy can't produce does, and that a plan reports a
carried fold's leaves as it reports a fold it makes. The changelog
fragment says what a step now does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…search#194)

SummaryTree.ancestorAt walks a chunk's lineage and checks that the
summary it reaches lists the chunk. A greedy solve asks it for every
chunk on every step, through its token count and flat-profile's level
ranking, and each answer took the walk and a leaf-set lookup again.
In a profile of a 7,776-chunk flat-profile solve, that was where this
branch's added CPU went: about 3.1 s in ancestorAt, leafSet and
parentOf over two solves, against 0.7 s in the engine's own walk on
main, with every other function within 0.2 s of main's.

The tree doesn't change once it's built, so it now keeps each chunk's
answer at each level the first time it's asked. A derived tree keeps
its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru
nissa-seru merged commit 075dde4 into anima-research:main Oct 18, 2026
7 checks passed
nissa-seru pushed a commit that referenced this pull request Oct 19, 2026
kv-unified's canonical forest built ownership from parent pointers alone
and never read sourceIds. Since #234, SummaryTree.ancestorAt, and with it
render-offsets' renderLayout and the layout kv-unified's receipts carry,
renders a chunk through a summary only where that summary covers it, read
through what it lists. So where a summary's pointer named a parent that no
longer lists it, as a parent written again without that branch does, every
kv-unified engine planned the branch's leaves at the parent and priced them
at its recall, while renderLayout rendered them raw: on six chunks of 100
tokens under three L1s and an L2, all recall pairs 50, with the L2 no
longer listing the third L1, kv-unified planned 50 tokens against a layout
of 250 (#253).

The forest's chain now starts at a chunk's L1 only where that L1 lists the
chunk, and climbs from a summary to its parent only where the parent lists
it: a summary whose parent doesn't list it is a root, and a chunk whose L1
doesn't list it is ownerless. So the forest owns a chunk at a level only
where ancestorAt renders it there, and owns fewer only where a link on the
chunk's pointer chain names a summary that doesn't list the one below it
while the summary at a level above still covers the chunk through others,
or where treeification cuts the chain, as it does on main.
Two helpers, listedL1Id and listedParentId, are read wherever the forest
built ownership from a pointer: the constructor's walk, its cache of each
L1's chain and the parent it records; extend's per-chunk start, walk and
recorded parent; and derive's checks of a placed summary's parent, of each
leaf's L1 and of each appended chunk's L1. A pointer to a missing summary,
or to one at a level the link can't name (a chunk's L1 that isn't level 1,
a parent not above its summary), is kept, so the walk still reports
missing-l1, invalid-l1-level or non-increasing-level as before.

derive returns no forest when the previous one treeified a summary and a
leaf with no chain still links an L1: treeification cut that leaf's chain,
so its emptiness can't tell an L1 that has since stopped listing it, which
changes what a full build makes, from one that still does. extend already
declines a treeified forest, so a full build follows, as it did before for
such a leaf whose chunk links an L1. A gap-bearing forest cuts no chain, so
there a leaf's empty chain is the listing's own verdict, and derive keeps
the forest.

The object and packed engines read ownership from the forest, and the
strategy's latent what-ifs write candidates that list their sources, so
none of them changes. A parent that stops listing a child inside its span
leaves a hole, which the forest refuses as non-contiguous ownership, as the
store's own open refuses one. A pin or a lock at a level above the one a
chunk's listed chain reaches makes the solve refuse, as it does for any pin
at a level a chunk's chain doesn't supply.

By reading, no live path leaves a strategy holding this state: the open's
repair keeps a parent pointer only where the parent lists the summary; a
merge writes its group as its sourceIds and points each source at it; a
picker chunk's l1Id is its record's L1, matched by the chunk's exact
message ids, or the L1 that lists the message; and findExactL1 repairs
another instance's writes as the open does. The state reaches kv-unified
through inputs that skipped that repair.

Tests (test/adaptive/kv-unified-forest-listing.test.ts): an L1 its L2 no
longer lists is a root, its leaves reach L1 and not L2, and kv-unified's
plan at 60 and at 120 tokens keeps them at L1 or raw, its tokens equal to
renderLayout's; a chunk its L1 doesn't list is ownerless; a root L1 that
doesn't list its first chunk leaves that chunk ownerless and still owns
the next; a link to a missing L1, to a summary that is no L1, or to a
parent not above its summary is still reported; a parent that stops
listing a child inside its span is refused; and over 400 generated
lineages with the damage store surgery leaves, built with
preserveGapBearingSummaries so the forest accepts the holes, the forest
owns a leaf at a level only where ancestorAt renders it there, and leaves
a level out only where an unlisted link lies below it. In
test/adaptive/kv-unified-forest-derive.test.ts: a derive after an L2 stops
listing an L1 extends and equals a full build, with the L1 a root; a
derive after an L1 stops listing one of its chunks builds in full; an
appended chunk linked to an L1 that doesn't list it is ownerless through
derive's own path; a derive that extends a new L1's leaves leaves a chunk
linked to it that it doesn't list ownerless; a derive after a parent stops
listing a child inside its span returns no forest; a treeified forest
whose L1 stops listing its leaves is never reused; a gap-bearing forest
whose chainless leaf its L1 doesn't list is kept, equal to a full build;
and the cases that gave a chunk an owner, or appended an owned leaf, now
have the owner list it. In test/adaptive/kv-unified.test.ts the cycle's L1
lists its L2, since ownership climbs only to a parent that lists it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru

Copy link
Copy Markdown
Contributor Author

@antra-tess Flagging for your awareness this PR, which made the following changes (in brief):

  • With adaptiveResolution on, under flat-profile and oldest-first, a chunk's summary at a level is now one that lists it. Where a summary's parent pointer named a summary written again without the chunk's branch, these solvers could choose that summary for the chunk, and a solve could end with "greedy fold solver exceeded internal bound". Raising a summary now moves only the leaves whose own summary at that level it is, so on a store whose summary tree isn't nested, a leaf it lists through another lineage keeps its level, where it was raised too. A chunk carried at a level whose summary doesn't list it now starts at the next level down that has one, and a step raises a chunk past a level where it has no summary through the next summary up that lists it; past a level its lineage skips, it stopped short of its target on every compile. On a store whose summary tree isn't nested, oldest-first, which chooses a summary only for the oldest chunk, can meet its target less often.
  • Under kv-stable, a pin-at-level at a level its chunk's lineage skips now holds the chunk at the next level down that has a summary, where it rendered raw.

— Lottie-2661

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores

1 participant