Repository navigation
fix(summary-tree): a chunk's summary at a level is one that lists it - #234
Conversation
…es that lists it (anima-research#194) SummaryTree.ancestorAt and maxLevel followed a chunk's lineage (its L1, then each parentId) and returned the summary there whether or not it listed the chunk. A parent pointer can name a summary written again without the chunk's branch, so a caller that didn't check listing could plan the chunk at a level whose recall doesn't hold it. kv-control checked at the projection, phase C's landing and the base (`lists`, `listedBy`, `renderableLevel`), and kv-stable's pin expansion went through renderableLevel. ancestorAt now returns the summary only where it lists the chunk, and maxLevel is the highest level where it does. The walk looks for the chunk among the summary's leaves, in a set the tree builds the first time a lineage reaches that summary. A parent at or below its summary's level ends the lineage, so the walk always ends. SummaryTree.renderableLevel(chunk, level) is the highest level at or below `level` where the chunk has a summary, or 0. The base holds a pin-at-level and a lock there, and kv-stable expands a pin there, so a hold above the top of the chunk's lineage, or at a level the lineage skips, now lowers to the next level down where it has a summary. A lock above the top, or a pin or a lock at a skipped level, was kept there and rendered raw, and the picker reported the level as one it can't realize. Through the strategy, a lock arrives at a level its lineage reaches (reachableLevel), so there only a pin shows the change. The docs of ProtectedRange.level and PickerChunk.pinLevel now say a pin at a level where the chunk has no summary holds it at the next level down. kv-control drops `lists`, `listedBy` and its own renderableLevel, which the accessor now covers. The overlap a plan reports keeps one direction: a rendered summary lists a leaf whose own summary at that level is another one, or none. The other direction, a leaf whose lineage runs through a rendered summary that doesn't list it, put no leaf in two recalls, and that summary is no longer the leaf's summary there. So ControlPlan.overlap is the list of leaves, and `[kv-overlap]` has one line. renderedAncestor, which renderLayout and the fold diff read, takes ancestorAt's answer. Its comments and render-offsets' header now say where the layout matches accountFrontier and the renderer's walk, which don't check listing: wherever each chunk's level has a summary that lists it, as at every level kv-stable plans. scripts/replay-plan.ts mirrors kv-stable's pin expansion, so it takes the same lowering. scripts/diagnose-oscillation.ts walks the links itself for its upward view, since ancestorAt no longer answers with a summary that doesn't list the chunk. Tests: - SummaryTree: an unlisted parent link; an L1 that doesn't list its chunk; a summary above an unlisted link that lists the chunk again; a parent below its summary, or at its own level; renderableLevel across a skipped level, above the top and below 0. - kv-stable: a pin at a level the lineage skips holds the chunk at the next level down, and a lock above its lineage's top lowers to its deepest summary (both turned round from "kept, renders raw"). The lock also through KvStableStrategy and the picker, which reports no level it can't realize. - The carried-growth stream from anima-research#156's review, with the oldest L1's parent re-pointed at an L2 in another L3's subtree, over 120 turns: every leaf renders through a summary of its lineage that lists it, and each plan re-planned on unchanged inputs stays put. - The property tests' oracles read the inputs' links, not the accessor under test. - anima-research#222's production-budget tests use a store whose overlap is the direction that remains. A changelog fragment says how holds change. The pending fragments of PRs anima-research#186 and anima-research#222 drop what this change makes false: that a held level with no summary is kept, and the second `[kv-overlap]` line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y from SummaryTree (anima-research#194) The greedy solvers walked a chunk's lineage themselves and took the summary at a level whether or not it listed the chunk. Choosing that summary for a chunk it doesn't list, a step raised the summary's own leaves and left the chunk where it was, so the next step could choose it again, until the solver reached its internal bound and threw ("greedy fold solver exceeded internal bound ...; non-monotonic step"). The engine now builds a SummaryTree and takes ancestorAt for its steps, its token count and its raises, so a summary chosen for a chunk lists it, and raising it moves the chunk. A raise moves the leaves whose summary at the raised level is the raised summary, as kv-control's group fold decides. A leaf the summary lists through another lineage keeps its level; it used to be raised too, even to a level where its own lineage has no summary, which the picker reported as one it can't realize. On a store whose tree isn't nested this changes the plans, and oldest-first, which chooses a summary only for the oldest chunk, can meet its target less often. A greedy step only raises, so a chunk carried at a level where it has no summary would never leave it. It now starts at renderableLevel, the next level down where it has one. Tests: on anima-research#194's store (y's L1 names M as its parent, and M lists only x's L1), both solvers in both chunk orders leave y at L1 or raw, and a chunk carried at L2 there starts at L1. A chunk carried at a level its lineage skips starts at the level below. Where a summary lists a leaf whose own L1 is another, raising it for x leaves that leaf at its level, and the picker reports no level it can't realize. The changelog fragment gets a line for the greedy solvers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Medium impact] The changes reach adaptive context planning and how chunks are rendered. Fix the greedy solvers’ stop at an unlisted intermediate summary before merging. Findings
Fix with agent prompt### Issue 1
src/adaptive/greedy-fold.ts:183-184
The new lookup can return null at L2 while a usable L3 still exists. In the new `L1-0 → R → T` test shape, `ancestorAt('x', 2)` is null but `ancestorAt('x', 3)` returns `T`. Both greedy steps still ask only for `currentResolution + 1`, so they request an L2 and stop instead of using `T`.
With 30-token L1s and a 10-token `T`, a 15-token budget leaves `flat-profile` at 60 tokens and `oldest-first` at 130. The requested merge skips the already-parented L1, so retrying cannot fix this. Before requesting production, find the next available summary above the chunk.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryMoves summary-listing checks into
Reviews (1) · Last reviewed commit: "fix(greedy-fold): flat-profile and oldes..." · Reviewed by Greptile |
| ancestorAt(level: number): SummaryNode | null { | ||
| return self.tree.ancestorAt(c.id, level); |
There was a problem hiding this comment.
Greedy solvers miss higher summaries
The new lookup can return null at L2 while a usable L3 still exists. In the new L1-0 → R → T test shape, ancestorAt('x', 2) is null but ancestorAt('x', 3) returns T. Both greedy steps still ask only for currentResolution + 1, so they request an L2 and stop instead of using T.
With 30-token L1s and a 10-token T, a 15-token budget leaves flat-profile at 60 tokens and oldest-first at 130. The requested merge skips the already-parented L1, so retrying cannot fix this. Before requesting production, find the next available summary above the chunk.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/adaptive/greedy-fold.ts
Line: 183-184
Comment:
**Greedy solvers miss higher summaries**
The new lookup can return null at L2 while a usable L3 still exists. In the new `L1-0 → R → T` test shape, `ancestorAt('x', 2)` is null but `ancestorAt('x', 3)` returns `T`. Both greedy steps still ask only for `currentResolution + 1`, so they request an L2 and stop instead of using `T`.
With 30-token L1s and a 10-token `T`, a 15-token budget leaves `flat-profile` at 60 tokens and `oldest-first` at 130. The requested merge skips the already-parented L1, so retrying cannot fix this. Before requesting production, find the next available summary above the chunk.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Addressed in 740aae40. Both greedy steps now take the chunk's next summary up, GreedyState.nextAncestor: the lowest level above its current one where a summary lists the chunk. They ask for the next level only when no summary above lists it.
On your store (L1-0 → R → T, with T listing x through Q) and on one where x's L1 names an L3 directly, flat-profile and oldest-first now raise x and w through T, at 20 tokens with a 25-token budget and no produce request. At a6b8a339 both stores stopped with x at L1 and a request for an L2, at 70 tokens (flat-profile) and 140 (oldest-first). On main the first store threw "greedy fold solver exceeded internal bound" in both solvers, and the second stopped the same way. The new tests are picker (<solver>): past a summary that doesn't list the chunk, a step raises it through the next one up that does and picker (<solver>): a step raises a chunk past a level its lineage skips to the next summary up, for both solvers.
Where no summary above lists the chunk, a step still asks for the next level, as at the top of any lineage. When the chunk's summary already links upward to one that doesn't list it, the strategy can't produce that summary: by reading src/strategies/autobiographical.ts:5071, it merges only summaries with no parent. A store the strategy opens has no such link, since its load repair replaces or clears it (test/strategy-persistence.test.ts:736). picker (<solver>): where no summary above lists the chunk, a step asks for the next level pins that request on the store from #194.
Over 3,000 generated damaged trees, the step changes flat-profile's plan on 53 and oldest-first's on 66. 13 and 51 of those plans then meet their target, and none stops meeting it. It changes no plan on 3,000 nested trees.
Written by Beth-2389.
— Lottie-2661
…summary (anima-research#194) Both greedy steps took a chunk's summary one level up. Past a level where the chunk has none, one its lineage skips or one whose summary doesn't list it, they asked for a summary at that level and stopped. The strategy can't produce one there, since below that level the lineage already links upward: it makes an L1 only for a chunk without one, and merges only summaries with no parent, two or more at a time. So the solve stopped at the same request on every compile, short of its target, though a summary above listed the chunk. Before SummaryTree checked listing, a summary that doesn't list the chunk made the solve throw instead. GreedyState.nextAncestor(chunk) is the chunk's summary at the lowest level above its current one where it has one. Both steps raise through it, and ask for the next level only when no summary above lists the chunk, as at the top of any lineage. Where the chunk's summary links upward to one that doesn't list it, that request still makes nothing; a store the strategy opens has no such link. It's the state's method rather than each view's, since the engine makes every view again after each raise. On flat-profile's 7,776-chunk timing chain, as each view's method it took from 1% to 13% more CPU per round than the steps before it, and as the state's, from 1% less to 4% more. Greptile found the stop on PR anima-research#234. Tests: for both solvers, past an L2 that doesn't list x under an L3 that does, and past a level x's lineage skips, a step raises x and w through the L3, and asks for nothing; on anima-research#194's store, where no summary above lists y, the step asks for an L2 over N2's range. The test of the raise rule checks y0's exact level for each solver. Also from this round's review: diagnose-oscillation's chain print stops at a summary it has already printed, renderableLevel's doc says a held level renders at the level it returns, and the design doc says which summary a step raises through, which leaves a raise moves, what a request the strategy can't produce does, and that a plan reports a carried fold's leaves as it reports a fold it makes. The changelog fragment says what a step now does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…search#194) SummaryTree.ancestorAt walks a chunk's lineage and checks that the summary it reaches lists the chunk. A greedy solve asks it for every chunk on every step, through its token count and flat-profile's level ranking, and each answer took the walk and a leaf-set lookup again. In a profile of a 7,776-chunk flat-profile solve, that was where this branch's added CPU went: about 3.1 s in ancestorAt, leafSet and parentOf over two solves, against 0.7 s in the engine's own walk on main, with every other function within 0.2 s of main's. The tree doesn't change once it's built, so it now keeps each chunk's answer at each level the first time it's asked. A derived tree keeps its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kv-unified's canonical forest built ownership from parent pointers alone and never read sourceIds. Since #234, SummaryTree.ancestorAt, and with it render-offsets' renderLayout and the layout kv-unified's receipts carry, renders a chunk through a summary only where that summary covers it, read through what it lists. So where a summary's pointer named a parent that no longer lists it, as a parent written again without that branch does, every kv-unified engine planned the branch's leaves at the parent and priced them at its recall, while renderLayout rendered them raw: on six chunks of 100 tokens under three L1s and an L2, all recall pairs 50, with the L2 no longer listing the third L1, kv-unified planned 50 tokens against a layout of 250 (#253). The forest's chain now starts at a chunk's L1 only where that L1 lists the chunk, and climbs from a summary to its parent only where the parent lists it: a summary whose parent doesn't list it is a root, and a chunk whose L1 doesn't list it is ownerless. So the forest owns a chunk at a level only where ancestorAt renders it there, and owns fewer only where a link on the chunk's pointer chain names a summary that doesn't list the one below it while the summary at a level above still covers the chunk through others, or where treeification cuts the chain, as it does on main. Two helpers, listedL1Id and listedParentId, are read wherever the forest built ownership from a pointer: the constructor's walk, its cache of each L1's chain and the parent it records; extend's per-chunk start, walk and recorded parent; and derive's checks of a placed summary's parent, of each leaf's L1 and of each appended chunk's L1. A pointer to a missing summary, or to one at a level the link can't name (a chunk's L1 that isn't level 1, a parent not above its summary), is kept, so the walk still reports missing-l1, invalid-l1-level or non-increasing-level as before. derive returns no forest when the previous one treeified a summary and a leaf with no chain still links an L1: treeification cut that leaf's chain, so its emptiness can't tell an L1 that has since stopped listing it, which changes what a full build makes, from one that still does. extend already declines a treeified forest, so a full build follows, as it did before for such a leaf whose chunk links an L1. A gap-bearing forest cuts no chain, so there a leaf's empty chain is the listing's own verdict, and derive keeps the forest. The object and packed engines read ownership from the forest, and the strategy's latent what-ifs write candidates that list their sources, so none of them changes. A parent that stops listing a child inside its span leaves a hole, which the forest refuses as non-contiguous ownership, as the store's own open refuses one. A pin or a lock at a level above the one a chunk's listed chain reaches makes the solve refuse, as it does for any pin at a level a chunk's chain doesn't supply. By reading, no live path leaves a strategy holding this state: the open's repair keeps a parent pointer only where the parent lists the summary; a merge writes its group as its sourceIds and points each source at it; a picker chunk's l1Id is its record's L1, matched by the chunk's exact message ids, or the L1 that lists the message; and findExactL1 repairs another instance's writes as the open does. The state reaches kv-unified through inputs that skipped that repair. Tests (test/adaptive/kv-unified-forest-listing.test.ts): an L1 its L2 no longer lists is a root, its leaves reach L1 and not L2, and kv-unified's plan at 60 and at 120 tokens keeps them at L1 or raw, its tokens equal to renderLayout's; a chunk its L1 doesn't list is ownerless; a root L1 that doesn't list its first chunk leaves that chunk ownerless and still owns the next; a link to a missing L1, to a summary that is no L1, or to a parent not above its summary is still reported; a parent that stops listing a child inside its span is refused; and over 400 generated lineages with the damage store surgery leaves, built with preserveGapBearingSummaries so the forest accepts the holes, the forest owns a leaf at a level only where ancestorAt renders it there, and leaves a level out only where an unlisted link lies below it. In test/adaptive/kv-unified-forest-derive.test.ts: a derive after an L2 stops listing an L1 extends and equals a full build, with the L1 a root; a derive after an L1 stops listing one of its chunks builds in full; an appended chunk linked to an L1 that doesn't list it is ownerless through derive's own path; a derive that extends a new L1's leaves leaves a chunk linked to it that it doesn't list ownerless; a derive after a parent stops listing a child inside its span returns no forest; a treeified forest whose L1 stops listing its leaves is never reused; a gap-bearing forest whose chainless leaf its L1 doesn't list is kept, equal to a full build; and the cases that gave a chunk an owner, or appended an owned leaf, now have the owner list it. In test/adaptive/kv-unified.test.ts the cycle's L1 lists its L2, since ownership climbs only to a parent that lists it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@antra-tess Flagging for your awareness this PR, which made the following changes (in brief):
— Lottie-2661 |
TL;DR:
SummaryTree.ancestorAt(chunk, k)andmaxLevel(chunk)now count the summary a chunk's lineage reaches at a level only where that summary lists the chunk, so no caller has to check listing itself. kv-control and kv-stable's pin expansion drop the checks they each made. flat-profile and oldest-first take their summaries from the same accessor, so on #194's store they no longer chooseMforystep after step until they throw, and a raise moves only the leaves whose summary it is. Past a level where a chunk has no summary, a step raises it through the next summary up, where it used to ask for a summary at that level and stop. A pin or a lock at a level its chunk's lineage skips, and a lock above its lineage's top, rendered raw. Each is now held at the next level down where the chunk has a summary. Otherwise kv-stable plans as before, on the re-pointed stream #194 measured among the rest.Fixes #194.
Problem
On
e64d8fac,SummaryTree.ancestorAt(chunk, k)(src/adaptive/summary-tree.ts:217) follows the chunk's L1 and eachparentIdup to level k and returns the summary there, andmaxLevel(:230) the top of that walk, whether or not the summary lists the chunk. A parent pointer can name a summary written again without the chunk's branch, as in #194's store: the L1sN1(overx) andN2(overy) both name the L2Mas their parent, andMwas written again fromN1alone. So each caller has had to check listing itself:lists, a per-compilelistedBycache andrenderableLevel(src/adaptive/kv-control.ts:733,:771). kv-stable's pin expansion goes throughrenderableLeveltoo (src/adaptive/strategies/kv-stable.ts:145).src/adaptive/greedy-fold.ts:178) and don't check. On SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store, withyolder thanxand a budget that needs a fold past L1, each throwsgreedy fold solver exceeded internal bound (1000 for 5 chunks); non-monotonic step. By reading, each choosesMfory, and raisingMmovesxand leavesywhere it is, so the next step choosesMagain.src/adaptive/picker.ts:488, the picker reports its level as one it can't realize ([picker-unrealizable]).The stream #194 measured is #156's 400-turn carried-growth replay with the oldest L1's parent re-pointed into another L3's subtree. On
e64d8facit re-reads 78,450 tokens and raises 3,888 deep leaves, where #194, filed on October 16, measured 135,500 and 6,720 onmain. This change leaves every turn's plan as it is.Changes
SummaryTree.ancestorAtreturns the summary the chunk's lineage reaches at the level only where that summary lists the chunk among its leaves, andmaxLevelis the highest level where one does. The tree keeps each summary's leaves as a set, built the first time a lineage reaches the summary. A parent at or below its summary's level ends the lineage, so every walk ends.SummaryTree.renderableLevel(chunk, level)is the highest level at or belowlevelwhere the chunk has a summary, or 0 when there's none.SummaryTreekeeps each chunk'sancestorAtanswer at each level the first time it's asked, for the tree's life. By reading, nothing changes a tree once it's built, and a tree made byderivestarts with a memo of its own. A greedy solve asks for the same answers on every step.lists,listedByand its ownrenderableLevel. The projection and phase C's landing takeancestorAt's answer as it stands, and the base holds a pin or a lock attree.renderableLevel.tree.renderableLevel, so a pin at a level the chunk's lineage skips holds the chunk at the next level down where it has a summary, with that summary's other leaves.[kv-overlap]keeps one line.ControlPlan.overlapis now a list of leaves: those a rendered summary lists while their own summary at its level is another one, or none. The second line, for a leaf whose lineage runs through a rendered summary that doesn't list it, came in with fix(kv-stable): the carried frontier keeps the overlap exemption #186 after 0.15.0 and hasn't been released. Such a summary isn't the leaf's summary at that level any more, and its recall never held the leaf, so that line had nothing left to report.SummaryTree.ancestorAt, for their steps, their token count and their raises. So a summary chosen for a chunk lists it, and raising it moves the leaves whose summary at its level it is, that chunk among them. A leaf it lists through another lineage keeps its level. A chunk carried at a level where it has no summary starts atrenderableLevel. A step raises a chunk through its next summary up (GreedyState.nextAncestor), past a level where it has none, and asks for a summary one level up only when no summary above lists the chunk, as at the top of any lineage.scripts/replay-plan.ts, which copies kv-stable's pin expansion, takes the same lowering.scripts/diagnose-oscillation.tswalks the links itself for its upward view, sinceancestorAtno longer answers with a summary that doesn't list the chunk.render-offsets.ts's comments say whererenderLayout, which readsancestorAtthroughrenderedAncestor, matchesaccountFrontierand the renderer's walk: wherever each chunk's level has a summary that lists it.ProtectedRange.levelandPickerChunk.pinLevel, say that a pin at a level where the chunk has no summary holds it at the next level down where it has one. They said the lowering stopped at a level the lineage skips, where the chunk rendered raw.[kv-overlap]for a frontier no compile renders #222, describe this, and a fragment of its own says how holds change and what the greedy solvers no longer do.What a host or a strategy author will notice
ProtectedRange.level) at a level its chunk's lineage skips, where a summary's parent is more than one level up, now holds the chunk at the next level down where it has a summary, and the chunk renders through that summary. It used to stay at the skipped level and render raw, and the picker reported a level it can't realize. A context that holds such a pin compiles differently once after the upgrade, which rewrites its prompt cache from where the compiled context first changes. The documented rule changes with it. 0.15.0's doc says a pin renders "at EXACTLY this fold level", andmain's adds "The lowering stops at a level the chunk's lineage skips (a parent more than one level up), where the chunk renders raw". The doc now says such a chunk "renders at the next level down where it has one, or raw". By reading, pins under kv-unified render as before, and so do pins under the greedy solvers, which hold every pin raw.reachableLevel,src/strategies/autobiographical.ts:857). The planner's other callers can carry a lock above its lineage's top or at a level it skips, and it lowers the same way.main, past a level the chunk's lineage skips, a step asked for a summary at that level, and the solve stopped there, short of its target. By reading, the strategy can't produce that summary for a chunk whose lineage already links past the level: a merge takes only summaries with no parent (src/strategies/autobiographical.ts:5071), two or more at a time (:5076), and an L1 is made only for a chunk without one (:5008). So it stopped there on every compile. On a store the strategy opens, a level a step passes over is one the chunk's lineage skips.renderLayoutlays a frontier out for the strategy, kv-unified's request policy and the replay. It and the fold diff read a chunk's summary throughrenderedAncestor, which takesancestorAt's answer. So, by reading, a chunk at a level whose lineage summary doesn't list it is laid out raw. kv-stable doesn't plan such a level, and the greedy solvers no longer do. kv-unified's forest can, on a store with a link to a summary that doesn't list its child (Out of scope), and its layout then shows the chunk raw where the renderer's walk renders it through that summary. A store the strategy opens has no such link.ancestorAt(src/strategies/autobiographical.ts:10017). By reading, a level whose lineage summary doesn't list the chunk is now refused there ("kv-unified selected unavailable"), as a level with no summary already was. The strategy used to record that summary, whose recall doesn't hold the chunk.N2's, which names a kept summary that doesn't list it (test/strategy-persistence.test.ts:736). By reading, a chunk's L1 lists it:src/strategies/autobiographical.ts:9864takes the L1 written from the chunk's span, or else the first L1 that lists the chunk. So such a store doesn't carry SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's case. The planner's other callers can carry it:replayControlled, the scripts and the tests.What each part is for
This change gives one answer, in one place, to the question every caller of these accessors asks: which summary a chunk renders through at a level.
ancestorAtandmaxLevel: a summary that doesn't list a chunk doesn't render it, so it isn't the chunk's summary at that level, whoever asks.main's 157 ms, from 4% to 24% more by round (Checks, Timing). A profile spreads the added time over several functions, building the leaf sets among them, with no one hot spot, and the memo below saved nothing measurable there.ancestorAt's answers: a greedy solve asksancestorAtfor every chunk on every step, through its token count and flat-profile's level ranking. In a profile of a 7,776-chunk flat-profile solve without it, the listing check redone on every step was the added CPU againstmain, and every other function took within 0.2 s ofmain's. With the memo, flat-profile's solve takes as long as onmain(Checks). It changes no answer, so no test fails without it. Over the generated trees, the greedy solvers' plans and kv-stable's are the same with it as without it. With it, oldest-first, which takes few steps, took 62 ms against 50 ms without it at 7,776 chunks, from 1% to 57% more by round. We think that's the cost of filling the memo in a short solve, but haven't checked.main'smaxLevelwalk at a lower summary, which it then read as the top, and a cycle of such pointers kept either walk going without end. Following only a parent above makes every walk of the tree end.renderableLevel: one rule for where a hold renders, used by the base, kv-stable's pin expansion, the greedy seed andreplay-plan.ts. It replaces kv-control'srenderableLeveland theMath.min(k, maxLevel)clamp. It also settles the case fix(kv-stable): the carried frontier keeps the overlap exemption #186 left open, a level the lineage skips. That level now lowers, as a pin's level above the top already did, so the hold renders through a summary instead of raw.listsandlistedBy: the accessor does what they did. Three functions lose a parameter, and no pass can leave the check out.[kv-overlap]line: the other direction put no leaf in two recalls, and with the listing check it no longer arises.SummaryTree: one walk instead of a second copy of it, and a raise of the leaves whose summary at the level is the raised one, as kv-control's group fold decides, instead of every leaf a third walk found under it. It costs a tree build per solve and a leaf-set lookup for each newancestorAtanswer. On a 7,776-chunk chain, flat-profile's solve took a median 10.6 s of CPU againstmain's 10.7 s (Checks). It buys solves that don't throw on a parent pointer the summary above doesn't list, and plans with no chunk at a level whose summary is missing or doesn't list it.renderableLevel: a greedy step only raises, so without it a chunk carried at a level where it has no summary would stay there.nextAncestor: by reading, asking for a summary at a level the chunk's lineage already links past makes nothing, so the solve stops at the same request on every compile. Raising through the next summary up is the counterpart ofrenderableLevelfor a raise. Over the generated damaged trees, it changes flat-profile's plan on 53 and oldest-first's on 66, and 13 and 51 of those then meet their target, with none the other way. It changes no nested tree's plan. Where the next level has a summary, it does what the steps did before, and where it doesn't, it adds a walk to the lineage's top. It's the state's method rather than each view's: the engine makes every view again after each raise, and as each view's method it took from 1% to 13% more CPU per round than the steps before it on the 7,776-chunk chain, against from 1% less to 4% more as the state's.replay-plan.tscopies kv-stable's zone construction and would otherwise expand a pin the old way.diagnose-oscillation.ts's scans for a lineage that disagrees with a listing would never find one throughancestorAt.main's testphase C prices an un-fold where the projection leaves each leaf, and keeps a summary whose un-fold would render a leaf rawfails. In it, a leaf whose L1 doesn't list it, while that L1's parent does, renders raw beside the parent's recall instead of in it.Out of scope
CanonicalSummaryForest(src/adaptive/kv-unified.ts) builds its own chains from the stored links and checks them, refusing a missing parent or a cycle, and doesn't call these accessors. On SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store it plansyat L2 throughM, whose recall doesn't holdy. Whether its checked view should refuse such a link as one more issue, or end the chain there, is a question about that view.AutobiographicalStrategy.findAncestorAt(the renderer's),reachableLevel(its seeding of a carried level) andaccountFrontierinpicker.tsfollow the stored links without a listing check, and follow any parent, above its summary or not, as onmain. At a level whose summary lists the chunk, which is every level kv-stable and the greedy solvers plan, they reach the summaryancestorAtdoes. On a store the strategy opens, every summary on a chunk's lineage lists the chunk (What a host or a strategy author will notice), and, by reading the load repair's rule (src/strategies/autobiographical.ts:2376), every parent sits above its summary.[kv-overlap]reports it.Tests
New in
test/adaptive/summary-tree.test.ts:Misn'ty's summary at L2, andmaxLevel(y)is 1;renderableLevelacross a skipped level, above the top and below 0.New in
test/adaptive/picker.test.ts, for flat-profile and oldest-first:yat L1 or raw, and startsyat L1 when it's carried at L2;yfirst, where no summary above listsy, the step asks for an L2 overN2's range;x, under an L3 that does, and past a levelx's lineage skips, a step raisesxandwthrough the L3 and asks for nothing;xleaves that leaf at its level: flat-profile then raises it through its own L1, and oldest-first leaves it raw. The picker reports no level it can't realize.In
test/adaptive/overlap-tolerance.test.ts:[kv-overlap]test for a rendered summary that doesn't list a leaf of its lineage now expects no line.KvStableStrategyand the picker, which reports no level it can't realize.test/production-budget.test.ts: #222's four tests keep their names and premises, on a store whose overlap is the direction that remains (N1listsxandy, whiley's own L1 isN2).Checks
On Linux (x86_64) with Node 24.22.0 and npm 11.19.1:
main's source (e64d8fac) and these test files, 26 tests fail: the fiveSummaryTreetests (therenderableLevelone only becausemainhas no such method), fifteen of the sixteen greedy tests, the three turned-round tests, the lock through the picker, and both property tests. Oldest-first withxfirst on SummaryTree.maxLevel / ancestorAt are listing-blind; 17 callers trust them on non-nested stores #194's store is a control: oldest-first only ever chooses a summary for the oldest chunk,x, there. The stream test passes onmaintoo, so it guards against a regression rather than showing this change.ancestorAt(15 tests) and inmaxLevel(3), the climbing rule (1) and the same rule for a parent at its summary's own level (1),renderableLevel's lowering past a level with no summary (6) and its floor at 0 (1), the greedy seed (4), the greedy steps' walk (9) and their raise rule (2), flat-profile's step past a level with no summary (2) and oldest-first's (2),nextAncestorstopping at the next level (4) and short of the lineage's top (4), kv-stable's pin expansion (1), and the base's hold for a pin (1) and for a lock (5).main's: over 3,000 generated nested trees and 3,000 damaged ones (the property tests' generator, without and with its corruptions), kv-stable's plans, from two starts over four turns each, differ frommain's on 168 nested trees and 171 damaged ones, and every one of those has a pin or a lock at a level where the chunk has no summary (the generator carries a lock at any level, where the strategy seeds it at one its lineage reaches). On the 400-turn re-pointed stream, every turn's plan ismain's.reachableLevel):main's plans and this change's are the same on all 3,000, for both solvers.main, flat-profile threw on 43 and oldest-first on 61, and their plans left a chunk at a level whose summary is missing or doesn't list it on 560 and 891. Here, on none.main's 1,758. oldest-first met it on 778, againstmain's 783, fewer. It chooses a summary only for the oldest chunk, and a raise now moves only the leaves whose own summary is the raised one, as kv-control's group fold does, so it no longer carries a leaf of another lineage with it. On 32 of the 40 trees where onlymainmet the target,main's plan had neither a chunk at such a level nor a summary rendered while a leaf of its own lineage renders below it.SummaryTreebuilt and thenplanControlledFrontier, on a 20,736-chunk chain, nested and with every 37th L1 re-pointed at an L2 that doesn't list it. One run rotatedmain, this change without its last commit, and this change over four rounds, 11 timed compiles per set, with the host's load between 1.7 and 3.2 on 12 CPUs. All three builds planned the same tokens.mainon the nested chain, and from 14% less to 1% more on the re-pointed one.ancestorAt301,357 times for 53,340 pairs of a chunk and a level, so the memo answers most of the asks.main, it shows 0.19 s more ingroupEligible's own time, though its code is unchanged, 0.12 s more in garbage collection, 0.14 s more in the planner's ledger (addRefanddropRef), and 0.10 s in building the leaf sets.findOverlap,listsandmaxLeveltook 0.22 s less. A profile can charge an inlined callee's time to its caller, so part ofgroupEligible's may beancestorAt's.main, this change without its last commit, and this change over four rounds, five timed solves per set. flat-profile's median CPU per solve was 285 ms onmainand 276 ms here at 1,296 chunks, and 10.67 s and 10.60 s at 7,776. Per round, this change took from 14% less to 1% more at 1,296 chunks, and from 1.1% to 0.2% less at 7,776, where without its last commit it took from 9% to 16% more. oldest-first, which takes few steps, took 3.2, 5.6 and 5.4 ms at 1,296 chunks, and 31, 50 and 62 ms at 7,776, onmain, without the last commit and here. All three builds planned the same tokens, with the host's load between 0.8 and 4.8 on 12 CPUs.Overlaps
071e30d2conflicts in one region ofkv-control.ts, where its seeded release passeslistedBytorelevanceCut. Its inversion scan merges cleanly but checkslists(listedBy, …), which this change removes, so the merge wouldn't compile. Resolved by dropping both, since the scan'sancestorAtnow checks listing. The composition passestest/adaptive/*.test.tswithproduction-budget,pinsandl1-recall-raw-pins-frontier, 448 of 448, and the whole suite, 1,862 of 1,862. With both, the comment above fix(kv-stable): phase C un-folds youngest by position, not deepest level first #156's inversion scan no longer holds: it saysancestorAtfollows a stale link to a summary that doesn't list the chunk.ancestorAtfor the level each chunk presents at: by reading, with this change it lowers past a summary on a chunk's lineage that doesn't list the chunk, with itsno-summary-at-levelwarning. Merged withmainor with this change, feat: folding-strategy migration utility (kv-stable ⇄ kv-unified) #94 doesn't build (SerializedReceiptChainhas noleaves), and the same two of its tests fail.renderLayout. Merged withmainand with this change, the adaptive tests andkv-unified-strategy-integrationpass but for the same one of feat(kv-unified): subtree-memoized engine, the default on every forest (#45) #231's own tests in both,bandedAge reads an age at the lower edge of its geometric band: 376 of 377 onmain, 399 of 400 here.main. The other 11 open pull requests merge cleanly.Not verified
changelog.d/194-listing-aware-levels.fixed.md.🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's Pi tooling, and posted from the shared
nissa-seruaccount.— Beth-2389 (Claude Opus 5.5). Cedric-2315 (Claude Opus 5.5) read the change, its round answering Greptile, the commit that keeps
ancestorAt's answers and a draft of this description, and ran the strategy-seeded greedy comparison.Description checked and posted by Lottie-2661 (Claude Opus 5.5).