Repository navigation
fix: send signed thinking only while the prefix it was minted under holds - #160
nissa-seru wants to merge 53 commits into
Conversation
…ted rounds
Shelf-354 (per-channel delivered clocks) and shelf-381 (askable fold record)
both need to know what a successful provider round actually carried. CM is
the only layer that knows how each compiled message was produced, so compile()
now returns that as `provenance`, and a host hands it back through
acceptRound() once a round carrying it has stood.
Compile provenance (src/compile-provenance.ts, types in types/context.ts):
- `messages`, parallel to CompileResult.messages (split parts repeat their
entry's sources; spliced injections are `{kind:'injection', namespace}`).
Raw entries name each stored body with `complete` or `missing`
(content | shards | tool-pairing). Completeness tolerates additions (a
frontdesk provenance header merged into the first text block) but not
losses (truncation, stripped image, a block removed by repair). Body groups
are complete only when every shard renders raw unaltered in one entry.
- `layout`: an immutable, compact RenderedLayout derived from the entries the
strategy actually emitted (never from its plan): runs of raw / partial raw /
summary-set / omitted units over the view, with base token estimates and the
calibration at compile time. Null when the strategy does not declare
`renderedForms` or names a summary it cannot describe.
- Binds to the branch the compile read ({id, name, created}) and namespace.
Strategy surface (types/strategy.ts): `renderedForms`, `ContextEntry.summaries`
({id, level, partial?}), `describeRenderedSummaries(ids)` (level, leaves,
method), `takeSelectionCause()`. Passthrough + windowed declare raw/omitted
(windowed reports cause 'window' when the anchor moves); autobiographical (and
knowledge/frontdesk by inheritance) annotate all four summary emission sites
and report 'budget-fit' when a committed adaptive select moved resolutions.
The legacy combined recall answer now reports which summaries survived its
cap (renderCombinedRecallAnswer), so a dropped summary is never claimed.
Fold journal (src/fold-journal.ts): acceptRound compares the accepted layout
with the last layout accepted on the compile's own captured branch (even if
another branch is selected by then), over messages present in both, and
appends one receipt when any changed form; a baseline (historyBefore
'unknown') when the branch has no record. Arrivals are not folds. Storage is
Chronicle typed records (unbranched; survive deleteBranch): a once-minted
store id, per-(namespace, branch) accepted layouts as snapshot/delta chains
(snapshot every 64), and the receipts. Every write syncs the store first,
because these records name branch-slot state. Diffs read persisted units
only. Usage is the confirming round's own; unreported fields 'unknown'.
Query: listFoldReceipts({since: id|ISO, limit default 10 cap 100, branch
(including deleted)}), foldReceiptsFor(branch), onFoldReceipt,
setReceiptSource, getStoreId, currentBranchRef, describeRenderedForms.
Tests: test/fold-journal.test.ts (16) with a scripted strategy covering
baseline, appends, fold/deeper/fold-down/unfold/omission, partial raw,
arrivals already folded, failed-then-successful round, restart, branches with
captured-branch acceptance and deleted-branch queries, removal after a
receipt, since/limit validation, source identity, non-reporting strategy, a
70-step delta chain across restart, and no receipt text in compiled context.
Full suite 958/958 (baseline 942).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l summaries into provenance - test/fold-journal-autobiographical.test.ts: the real strategy's positioned recall pairs name their summaries; describeRenderedSummaries expands a merged L2 to its leaves; a seeded fold yields a change receipt with the strategy's ids and method 'unknown'; the legacy combined answer under a cap names the whole summary, marks the cut one partial and reports the dropped one's messages omitted (both envelopes). - test/compile-provenance.test.ts: coversStoredContent accepts additions (headers) and rejects losses (truncation, stripped image, removed block); attributeEntries: body groups complete only with every shard raw and unaltered (separate copies or one intact composite), missing shards/content reported, summary partial flags preserved. - compile-provenance: CompiledMessageSources summary items now keep partial: true (it was dropped when building per-message sources). - changelog fragment: missing is content | shards (no tool-pairing value). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ia, rawSources, presentation
Fixes from Hugo's first-pass review (room-220 #43359, #43509, #43572, #43608):
- Fold journal: a receipt and the layout it was compared into are committed
as ONE context-manager/accepted-layout record (the receipt is embedded; its
id is the record id). Before, the receipt was appended first and the layout
second, so a failure or crash between them made a retry write a second
baseline. A failed append drops the branch's cached latest layout; the next
acceptance rereads records and finds the compile already accepted if its
record landed. The fold-receipt record type is gone.
Test: an append that fails cleanly, then one that lands but throws; each
retry, from the same or a fresh journal, yields exactly one receipt.
- Layout ranges (summary and omitted units) carry exact membership `m`: runs
[firstSeq, firstId, lastSeq, lastId] of consecutive sequences. The diff
intersects runs, so a message one view lacked (removed, filtered out) is
never counted inside a range, nor charged its tokens. Change counts are now
always exact, and baseline runs count range members exactly. Test: m1..m3
raw, then m2 removed and s13 covering m1 and m3: one change of 2 messages,
with "before" tokens two thirds of three raw bodies.
- Media comparison is exact: the whole data, or the URL, plus kind and media
type. Sampled ends could call different bytes the same. Test: equal
length and equal 64-character ends with a different middle are not covered.
- CompileResult.rawSources: the stored messages behind every raw body, from
the compile's own view (auxiliary-store messages included, shards
included), bound to the compile. Consumers resolving a body's metadata or
content read it here instead of getMessage, which can't see auxiliary
slots and would bind a later edit.
- acceptRound({ presentation }): receipts record how the confirming round
carried the compile, as its producer reported: verbatim, altered or
unknown (the default). The layout remains the compile's.
- Legacy combined recall (recallEnvelope 'none') attributes survival across
all block kinds by aligning the truncator's output with its input. A
summary whose text was dropped but whose thinking block was retained is
named partial, no longer omitted. Rendering is unchanged.
Full suite 971/971.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… getMessage cannot see Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…converge Hugo (room-220 #44042): after a landed-then-throw commit, the retry correctly found the canonical record, but no onReceipt listener was ever called, so a bound exporter never refreshed for that receipt. The latest-layout cache now keeps the receipt its record embedded. An acceptance that finds its compile already committed announces that receipt if this journal never has. Every announcement is once per receipt per journal (a notified set), on the normal path too. An identical-layout acceptance no longer inherits the previous record's receipt. Test: nothing is announced while the write is uncertain; the retry announces the recovered receipt once; a further retry announces nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n any acceptance Hugo (room-220 #44256): if the uncertain commit was on a stream's terminal round, the next activation accepts a NEW compile, often with an identical layout. The same-compile recovery branch was bypassed, the no-change path wrote nothing, and the recovered receipt was never announced. accept() now announces the latest record's receipt whenever it loads one this journal hasn't announced, before deciding anything about the new compile. Announcements stay once per receipt per journal. Test: the landed-then-threw write is on the last compile; a fresh journal for the next activation accepts a new compile with an identical layout. There is no new receipt, and the recovered one is announced exactly once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… every declared shard Hugo's storage-failure probe (agent-framework room-220 #47804, /tmp/hugo-partial-shard-add-probe.ts): in a real addMessage with lossless per-block sharding, an exception before the second shard's append left the first shard (the source header) stored. The next compile's provenance said complete:true, because it counted only the members in its view; a header-only body was then confirmed as delivered downstream. Shards are immutable, but that proves members unchanged, not that the group was written whole. Tessa (#47842) asked for the fix at CM's producer boundary, covering an interrupted write and a reopen as well as selection. - StoredMessage.shardCount: how many shards the bodyGroup was written with, declared on every shard at write time and kept through append's extras and the public view. Absent on groups written before it. - appendMessage refuses a chunking decision whose indices aren't exactly 0..n-1 before writing any shard, so a declared size always names exact indices. - attributeEntries: a group whose head declares its size is whole only when its members are exactly indices 0..n-1 and all are carried. Otherwise it is missing ['shards'] (rawComplete false, so the rendered layout reads it as partial raw too). An undeclared legacy group keeps its old meaning: every member in the view is carried. CM can't say more about such a group. RawBodySource.complete's doc says so. Changelog fragment shard-count.added.md. Tests (compile-provenance.test.ts): - attributeEntries: a declared group short a shard is missing shards; a whole declared group is complete; an undeclared one-member legacy group is complete. - Through ContextManager: every shard records shardCount and a whole group compiles complete; an append failure injected before the second shard leaves a head declaring 2, which compiles as missing shards, and still does after close and reopen; a decision with indices [0, 2] is refused, with nothing written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eptance per compile, bodies as stored, token costs) Greptile's first review of anima-research#146 (7 findings, review 5448880651) at 60508e9. Each was reproduced independently before this change (Mira-1754's probe, room-260); every new test below fails at 60508e9 except where noted. Fold journal (P1 "Shared managers miss folds", P1 "Older retries create false folds"): - The records are the journal's only truth. `accept` reads the store's accepted-layout ids and checks its cached latest layout against them, walking back from the newest id to the cached record: a newer record of this namespace and branch, written by any journal on the store, replaces the cache. Only records appended since are read. - Every acceptance commits its record, an unchanged layout included (an empty delta), so every accepted compile is durably identifiable. A compile is accepted once: `alreadyAccepted` finds a record of this namespace with its compileId. Every message a layout names was stored before its compile ran, and Chronicle record ids are one global, increasing counter (the log's next_id; message ids are record ids), so only records newer than the newest named message can hold the compile's acceptance; the walk stops there. A layout naming no numeric id searches every record, and so does a store whose open truncated a torn tail (`recovery()`), since Chronicle reissues ids only then. - `acceptedCompiles` and the compileId rewrite of the cache on identical layouts are gone. First confirmations still count in acceptance order: two compiles made before either was accepted both record their changes. (A basis gate that dropped the second was considered and rejected in review: it would discard real confirmations on a heuristic.) Raw bodies are judged as stored (P2 "Filtered bodies change identity"): - compile() attributes raw bodies and builds `rawSources` from the merged messages before the view filter, taken in the same synchronous pass; the layout stays on the filtered view. A body's head is shard 0 (the id addMessage returned) even when the filter hid it, its membership is the stored group, and a hidden shard is missing `shards`, for an undeclared legacy group too, so a filtered body is never claimed whole. `rawSources` holds the head and every shard, so a consumer resolves the whole body. - addMessage writes a decision's shards in index order, so the id it returns is always shard 0's (it returned the first in decision order). Composite media (P2 "Whole media copies look incomplete"): - compositeCovers: the members' text, joined in shard order, must appear in the composite's text, and their non-text blocks must appear unaltered and in order among the composite's non-text blocks. Where media sits relative to the text isn't compared: a composite gathers the text into one block. Token costs (P2 "Fold costs miss summaries", P2 "Calibration changes saved base costs"): - A summary unit's `sm` tuples carry each summary's tokens (`[id, level, method, partial, tokens]`); the unit-level `t` on summary units is gone. diffLayouts and layoutRuns count each summary once per side, by id, in the first (changed) run that names it, as FoldChange's doc already promised. Persisted shape change; nothing has shipped. - MessageStore.estimateBaseTokens(content): the sum of raw per-block estimates, before the calibration multiplier and its per-block rounding. Layouts use it, so identical content costs the same at any calibration (it was [7, 13, 10] for one message at 0.7, 0.8, 1). Docs (P2 "Strategy docs name missing setting"): ContextEntry.summaries names `ContextStrategy.renderedForms`. Docs and changelog fragments updated: RawBodySource, rawSources, LayoutUnit, acceptRound, the fold-journal header, accepted-round-provenance.added.md, shard-count.added.md. Tests: - fold-journal.test.ts: two managers on one store each compare with the other's layout; retries of an older receipt-writing compile, an older unchanged compile and the newest, after a reopen, write nothing; a layout naming no record id (empty, or non-numeric ids) is still found on retry; two compiles made before either was accepted both count (passes at 60508e9 too: it guards the acceptance-order contract); a summary is charged to the first changed run naming it when an unchanged unit already renders it ([S, 0]); layout tokens equal at calibrations 0.7, 0.8 and 1. - compile-provenance.test.ts: autobiographical's own composite of an image body is complete, and a dropped or different image or cut text is not; a hidden head keeps its id, its body is missing shards, rawSources holds both shards, and the layout shows the visible shard as partial (through ContextManager with a viewFilter); attributeEntries judges declared and legacy groups by stored members (unit level, passes at 60508e9 too: the fix is in what compile passes); shards written in index order from a reversed decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the branch's newest receipt Mira-1754's review of a7129a4 (room-260) found two blockers, both reproduced (/tmp/mira-1754-recovery-probe.mjs, /tmp/mira-1754-tail-probe.mjs): 1. Persisting no-change acceptances hid the branch's newest receipt from the announce-latest convergence, which read only the newest record's own receipt. A's change lands then throws; B (another journal) records an unchanged layout; A's next acceptance announced nothing. The same held at 60508e9 for arrival-only records, which never carried a receipt. 2. The message-id floor that bounded the already-accepted search relied on record ids growing across opens. Chronicle reissues ids after truncating a torn tail; the `recovery()` guard covered only the first open after it. Accepting a pre-crash compile under a reissued low id, then a clean reopen, hid that acceptance from the floor, and a retry wrote a false omitted-to-raw receipt. The journal now keeps an index of the store's accepted-layout records, built by reading each record once, oldest first, and brought up to date before every use (Chronicle lists a type's ids in append order, and the list only grows while the store is open). Per namespace it holds every accepted compileId, each branch's newest record and newest record holding a receipt, and the receipts. It depends on no id ordering. - accept: announce the branch's newest receipt if this journal hasn't; return null if the index holds the compileId; compare with the branch's newest record (the journal's cached layout is used only while its record is still that newest one). - The index is a function of the records alone, so every journal on one store object shares it (agent-framework opens one store for every agent, subagent, subconscious and conversation namespace): one scan per store per process. A reopened store is a new object, indexed afresh. - Gone: alreadyAccepted's floor walk, newestNamed, the recovery() guard, loadLatest's backward walk, refreshReceipts, and LatestLayout.receipt. The record format is unchanged from a7129a4. Cost: the first acceptance (or receipt query) after a store opens reads every accepted-layout record once. Measured on /dev/shm with a 500-unit sliding-window layout (snapshot every 64): 2k records 34 ms, 10k 147 ms, 40k 539 ms; later acceptances 1-10 ms (getRecordIdsByType materializes the id list each call); another manager's first acceptance on the same store 2-16 ms. At 60508e9 a fresh journal read only the branch's newest record and its delta chain, and only queries read every record. Tests (fold-journal.test.ts), both failing at a7129a4: - a journal whose own change write landed then threw hears that receipt once, after another journal recorded an unchanged layout and an arrival after it; a journal opened later hears it too. - a retry is found even when its layout names ids newer than its acceptance record (as after a torn-tail recovery reissues ids); no false omitted-to-raw receipt, no new record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every manager on one store object shares the accepted-layout index (116832f). The first acceptance or receipt query after a store opens reads every record once; hosts with many accepted rounds will see that once per open. Changelog only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mpile captured Mira-1754's factual review (room-260 #50134, #50145): layouts persist base estimates, but changeReceipt, baselineReceipt and diffLayouts apply the confirming compile's layout.calibration to both sides of a receipt's estimated token counts (`usage` stays the provider's uncalibrated report). FoldReceipt.estimate said "the store's calibration at acceptance"; it is the calibration the compile captured. The changelog fragment now separates the two claims. Doc and changelog only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in anima-research#144 (XML-history carrier pricing), anima-research#147 (sibling-append cache certification), anima-research#148/anima-research#149 (kv-unified receipt RLE and leaf ordering) and the 0.13.x releases ahead of the rework for review B1/M1/M2. The merge is clean, as Anarchid's trial merge found (1009/1009 at main f74b374 + anima-research#146). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e window, bodies as ingestions, durable acceptances
Anarchid's review (changes requested, PR thread 2026-10-09) found one blocker,
two majors and three minors. This commit answers all of them.
B1, accepted layouts O(session) per acceptance. A range unit carried its
exact membership as runs of consecutive store sequences, but store sequences
are shared with state updates and typed records (the journal's own records
among them), so the runs fragmented to one per message, and a one-message
change to a range unit recopied all of it.
- Format 2. A layout is two parts: `members`, the view's sequences
ascending (persisted gap-coded: LEB128 varint gaps, base64url, about 1.3
bytes per member), and `units`, forms with counts and no membership:
`{k:'r', t, p?}`, `{k:'s', n, sm}`, `{k:'o', n}`. Ids live only in memory
(`memberIds`), and receipts take bounds from the compile being accepted.
`RenderedLayout` is v:2; `MemberRun` is gone. Format 1 never shipped, so
its records are not read.
- A delta edits each part on its own: members as the sequences removed
(`mr`) and added (`ma`), units as before (keep p, drop, insert, keep,
append), now a few bytes per unit. Equal units are common without ids, so
a next layout that matches the previous one through its end, short of
it, is a tail drop, not a reason for a snapshot.
- A snapshot follows once the layout bytes of the deltas written since the
last snapshot would exceed that snapshot's (or 256 deltas), so storage
stays about twice the deltas' and a rebuild reads about two snapshots'
worth. `sb`/`db` in each record carry the policy across a reopen.
- The diff is a merge walk over the two member lists: same rules for runs,
exact counts, tokens once per raw message and once per summary per side.
- Measured on a synthetic version of the review's production store (16,871
messages, a record between every few, 24 summary spans, 388 raw):
baseline layout 30,091 B (was 140,245 B); pure append 262 B (280 B); one
summary re-id 295 B (15.5 KB); 10 raws folded into a neighbour 293 B
(94 KB); a 24-summary rotation 1,506 B of layout (124.5 KB). The
passthrough probe (600 turns, 6k budget) writes 550 deltas whose layout
part stays flat after the window fills, and 670 KB in all, receipts
included, against 3.2 MB.
- The index (R4) keeps record ids, each branch's newest record and each
receipt's id and acceptance time; receipt bodies are read when a query
returns them, newest first, stopping at `limit`. It lists the store's
records once per store object and then grows by write-through: Chronicle
locks a store to one open object, so every writer is a journal on it. An
append that throws (it may have landed) leaves the index stale, and the
next use lists again from where it stopped.
M1, identical sharded bodies conflated. A group id is a content hash, so two
ingestions of one text shared a group. A body is now one ingestion: in
sequence order, a group's next body begins where a shard's index doesn't
follow its predecessor's (shards are written 0..n-1, one ingestion at a
time). It needs no new stored field and holds for stores already written.
Attribution keys bodies by their head. `rawSources` is replaced by
`rawBodies` (head -> the body's messages, head first), so a consumer groups
a body's shards by ingestion too. Rendering, removal and other readers of a
group id still treat a group's contiguous run, or every message with the id,
as one body, so two ingestions of one text are one body there: that predates
this PR (Basil-1007 probed the render at main f74b374) and is fixed
separately.
M2, an acceptance announced before it was durable. The store now syncs
before the append and after it. Basil-1007 showed (room-440) why one sync
after isn't enough on the Chronicle context-manager depends on (0.4): a
state's head lives in state.bin, written only by sync, and open doesn't
advance it from the log, so a record could outlive the summaries it names.
With chronicle anima-research#22 (unreleased) a single sync after would do.
m1, a tool result moved by pairing repair read as lost content. Repair
records the tool_use ids it moved out of an entry (`relocatedResults`), and
attribution checks the rest of the stored body against the entry and each
moved result against the block now carrying it. An altered or missing
moved result still counts as not carried.
m3, `since` ambiguous. `afterId` takes a receipt id; `since` takes an ISO
8601 time only, and a bare number is refused with a pointer to `afterId`.
(c) The changelog now says typed records take the selected branch's next
sequence, which is what fragmented the old runs.
Tests: 12 new. Body identity (apart, back to back, interrupted and
re-ingested, and through addMessage with the real chunker); relocation
(credited, altered, missing); size (passthrough 600 turns with other records
between messages: flat deltas, bounded snapshots, total within twice the
deltas; a summary re-id deep in 1,200 messages; a removal); the sequence
encoding; a 2,000-round fuzz of delta round trips; sync order (sync, append,
sync, announce); one listing per store object and reads only for returned
receipts. Existing tests moved to the new shapes; two that wrapped the store
in a Proxy now keep every journal on the wrapped object, as Chronicle's lock
does in practice.
npm test equivalent, one unforced `node --test` per compiled file on the VPS
(node 24.21): 1021 pass / 0 fail across 124 files; tsc and typecheck clean.
At the merge with main (64ba2f9): 1009 / 0 across 124.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lds are recorded The review noted no test exercised kv-unified's layout (the production-store probe was the only one). An autobiographical strategy with adaptive resolution and the kv-unified folding strategy, 40 messages and four seeded L1 summaries of eight each: - at a budget everything fits, the layout is the whole view, in order, every message raw, and the first acceptance writes a baseline; - at 3,000 tokens kv-unified folds the four spans: four summary units naming L1-0..L1-3 over their exact messages, eight raw, no message left out or omitted, and the acceptance records four raw-to-summary runs of eight messages each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From Basil-1007's review of the rework. `accept` announces the branch's newest receipt that this journal hasn't announced, which is how a write that reported failure after landing still reaches listeners. When that failure was the sync after the append, the record is in the log with its durability unknown, and announcing it as found would break M2 on that path: heard of before durable. It now syncs first, and a sync that throws propagates with nothing announced. A sync with nothing pending is cheap. Test: the sync after the append fails once (the receipt landed, unannounced); the next acceptance calls sync, then announces the change. Dropping the new sync fails it (the announcement comes before any sync); fold-journal is 33/33 unmutated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…says whether there is more From Basil-1007's review of 9f2f5e75. `afterId` was offered as a cursor (continue from `latestId`), but the query walked newest first and stopped at `limit`, so with more than `limit` receipts after the cursor it returned the newest ones and the reader never saw those in between. Now a query with `afterId` returns the oldest `limit` receipts after it, found by binary search on the ascending ids, and continues from the last one it returns; `afterId: '0'` pages through a whole record. Without `afterId` a query still returns the newest first. Every result carries `more`: true when the query matched more receipts than it returned. `since` filters either way. Tests: the existing filter test now expects the continuation (the next receipt after the cursor with limit 1, `more` true), `more` on a newest-first query, and nothing after the newest; a new test pages a 25-receipt record from '0' in tens and sees every receipt once, oldest to newest. 1024 / 0 per file on the VPS, tsc and typecheck clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erId continues From Basil-1007's review of 2838fb2 (approved, with optional points): - The paging test now stores enough records first that the receipts' ids cross a digit boundary, and asserts they do. Comparing ids as strings in firstAfter ("100" sorts before "99") passed every test before; it now fails the paging test. - FoldQuery.afterId's doc says to continue from the last receipt of a previous `afterId` query: the last receipt of a newest-first page is its oldest, and reading on from it would repeat newer ones. fold-journal 34/34 on the VPS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a branch's whole record foldReceiptsFor(branch) (FoldJournal.receiptsFor) read every receipt of a branch, and its doc named its one consumer: 'what a projection writes'. That consumer, connectome-host's folds.jsonl exporter (anima-research#226), now writes only the newest receipts, one listFoldReceipts page, which reads only the receipts it returns: a whole-branch read after every fold grew without bound over a resident's life. Nothing in agent-framework or connectome- host calls it now, and a whole-branch read in the public API invites the same per-receipt O(n). Removing it before this PR merges avoids taking a published method back later (Basil-1007, room-427 #64701). A reader that wants the whole record pages it: listFoldReceipts with afterId '0', then from the last receipt of each page while more is true. The changelog fragment says so in place of foldReceiptsFor. Tests that used it as a convenience now use the query: the captured- branch case drops its third, redundant count (listFoldReceipts already asserts main's baseline); the digit-boundary case takes its 25 ids from one newest-first page, reversed; the reads-only-what-it-returns case counts 31 with one page. Full suite 1024/1024 (pretest build). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found by removing each piece of the rework on its own and running the full suite (skill:preparing-upstream-pull-requests): two pieces of the layout storage had no test aimed at them. - Writing a snapshot once the deltas since the last one would outweigh it: with the rule removed (always a delta until MAX_CHAIN), all 1024 tests still passed. The size test's bound (total within three times the deltas) holds with fewer snapshots too. - The 256-delta cap (MAX_CHAIN): with it removed, all 1024 still passed. It binds when deltas are tiny beside the snapshot, an unchanged layout's ~10 bytes against a large window, where the outweigh rule alone would let a chain run to thousands of deltas, all read at a rebuild. New tests, through a FoldJournal over a store proxy whose sync costs nothing (these are about which records are written; the sync order has its own test), so they take a fraction of a second: - a 10-message window over 60 unchanged rounds: snapshots recur, and no delta's running bytes since its snapshot exceed the snapshot's; - a 200-message window over 258 unchanged rounds: record 256 is the 256th delta, still far from outweighing its snapshot, and record 257 is a snapshot. Each fails with its piece removed. Full suite 1026/1026 on node 24.19 and node 20.20. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module header, commit()'s doc and the changelog fragment claimed more than the code does: - "a record's size follows what changed and what was rendered, not how much history lies behind the window": a snapshot carries every member of the view, the history behind the window included (about 1.3 bytes each, gap-coded). Only a delta's size follows what changed; the code's own AcceptedLayoutRecord doc already said "the history behind the window costs nothing beyond its members, in snapshots only". Now the header says that too. - "a delta ... until the deltas written since the last snapshot would outweigh it (or MAX_CHAIN of them), then a snapshot: storage stays proportional to what changed": commit() also writes a snapshot when layoutDelta returns null (the previous layout's last unit isn't found past the common prefix), and when the 256-delta cap fires on tiny deltas, storage is one snapshot per 257 acceptances, not proportional to what changed (Basil-1007's review of 56ebd1e, room-440 #65050). What does hold exactly: db <= sb in every chain, so a rebuild reads at most two snapshots' worth of layout. The doc now says the three triggers and that bound. The fragment said only the weight rule; it now names all three. The fragment also lacked a word ("When any changed form" -> "When any message changed form") and its acceptRound signature omitted presentation, which the PR body and the code have. Comments and the changelog fragment only; no behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… can come again onFoldReceipt's doc said "Be told after each fold receipt is appended". FoldJournal.accept() also announces the branch's newest receipt, after a sync, at a manager's next acceptance on that branch whenever this manager hasn't announced it: a write that reported failure after landing, a receipt another manager on the store wrote, or one from before a reopen, since `notified` is per journal and in memory. So a listener can hear a receipt again (the first acceptance on each branch with receipts after a reopen re-announces its newest), and needn't hear every receipt (only the newest is announced, so a manager's listeners don't hear another manager's earlier receipts). Basil-1007 named the behaviour in his review of eb2122c (room-440 #65184); the test "announces the branch's newest receipt to a journal that missed it, across later records without a receipt" pins it, including the later journal that hears the change again. The doc now says exactly that: every receipt this manager's acceptRound returns (announced after commit's second sync), plus the newest-receipt announcement, and tells listeners to key on receipt.id or read listFoldReceipts. The changelog fragment's subscribe line says the same. Docs and the changelog fragment only; no behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[High impact] The PR changes context compilation and accepted-round records across the manager's strategies. Do not merge yet: stripping can break Responses requests, and historical branches or failed-load recovery can send invalid signed thinking. Findings
Fix with agent prompt### Issue 1
src/thinking-binding.ts:297
When the pass strips reasoning from Responses history, the following text or tool-call block keeps its `rawItem`. Both `bindThinking()` and `releaseThinkingView()` remove reasoning without calling `releaseEditedRawForms()`. The remaining item can therefore be replayed with its provider ID but without the reasoning it was paired with, causing the provider to reject the request.
Reconcile raw forms in both strip paths before hashing or returning the remaining content. The existing `stripThinkingBlocks()` path already does this.
### Issue 2
src/thinking-binding.ts:610-611
`foreign()` compares every message’s branch-local sequence with one `attributedAfter` value captured on opening. After opening a long history and switching to a branch made at an old message, new writes can remain below that value. Another namespace’s mid-round write into the shared slot is then treated as carried.
A reply minted without that write gets stamped after it, so the next compile keeps thinking under a prefix it was not minted under. The provider can refuse or drop it. Track the opening boundary per branch, or use a store-wide record identity.
### Issue 3
src/thinking-binding.ts:804-806
`load()` sets `loaded` before `refresh()` succeeds. If the first record listing or read throws while opening on an app-owned store, retrying `ContextManager.open()` on that same store reuses the cached `ThinkingBinding`. It skips loading and can appear unengaged even though accepted binding records exist.
The next compile can send old signed thinking after a prefix change, causing a provider rejection. Only a later acceptance retries `refresh()`. Set `loaded` after the initial refresh completes.
### Issue 4
src/compile-provenance.ts:390
This shortcut also skips Responses blocks whose entire payload is in `rawItem`, such as encrypted compaction or hosted tool calls. A capped raw reply can lose a later such block when `remaining` reaches zero while keeping its earlier reasoning and text unchanged. `coversStoredContent()` then reports the shortened body as complete because it ignores the missing block.
This makes `complete` and fold receipts overlook lost content. Only skip genuinely empty text, and compare the raw payload of these Responses blocks.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryAdds prefix stamps and branch-scoped releases for signed thinking, plus compile source tracking and durable fold receipts.
Reviews (2) · Last reviewed commit: "docs(thinking-binding): rounds that over..." · Reviewed by Greptile |
…ted under holds (anima-research#155) On an account that enforces thinking binding, Anthropic refuses a signed thinking block once anything before it has changed since it was minted: "The block is bound to a different conversation", or a silent drop under drop_block. A compile that folds, merges, caps, strips an image or releases a raw form changes the prefix of every reply after the change, so the anima-research#155 resident 400ed on every turn after its first fold. The pass (src/thinking-binding.ts) runs in compile() over the final messages, injections included, so it covers every strategy without knowing which rewrite did what: - Chain: sha256 over the compiled messages in order (canonical JSON, keys sorted), seeded with SelectOptions.prefixIdentity (the host's system and tools) and the system injections the compile returns. - Stamp: each stored message with thinking gets, once, when it is stored (the message store's add event; or, if a compile not yet accepted fenced the walk before it, when that compile is accepted), the seed, the chain before it (before each part a compile splits it into: splitMixedToolMessages, for a bundled tool cycle, with the part's first block), and a digest of each of its blocks. The chain comes from the branch's owner, its latest accepted compile, and is extended by the manager's OWN stored messages after it, split the same way, as they stand when the message is stored: what the live request carried. So an edit after a reply is stored strips it, and an edit before (a host writing a result over its placeholder before the reply is stored) is what its round carried. That is the host's contract, stated in acceptRound's doc: what it stores in the manager's slot after a compile, up to a reply, is what the round carried, in order, as it stands when the reply is stored (agent-framework defers cross-turn writes during a turn). Another writer's slot isn't walked, since the live request didn't carry what was stored there after the compile. A sharded body ends the walk until the next acceptance: the chunker joins text blocks and keeps media on shard 0, so its form as sent can't be restored. The walk is incremental per branch (a cursor, found by binary search), so it visits only the messages after where it stopped. - Fence: a compile bound after the owner and not yet accepted may have sent something else (a heartbeat's seed, a cap the next compile didn't repeat), and a reply minted under it would get a stamp that could match later. So each branch keeps the compiles bound after its owner that the walk hasn't passed, in bind order, and the walk passes one's head only when its chain there is the walk's. At one that isn't, it stops; the replies after it await that compile's acceptance, which stamps them from its own chain, and lose their thinking for good if a compile bound later is accepted first. A passed compile leaves the list (it stays passed under any later owner bound after it); a compile identical in head and end to the previous one isn't added; past 32, the branch goes blind from the first forgotten head until a compile bound after the last forgotten one becomes the owner. In this process the owner is the accepted compile bound last, and each record says whether its acceptance made its compile the owner, so a reopen keeps that choice. Compiles bound and stamps made since the last record aren't known after a reopen, and a reply stamped late (at the acceptance of a compile that fenced it) reads what was stored after that compile's head as it stands then. - Shared: every manager on one store object shares a namespace's binding state (thinkingBindingFor), as the fold journal's index is shared, so an acceptance through one engages, stamps and fences for all. They walk one slot, the namespace's, unless one opens it isolated and another doesn't. - Strip: thinking renders only when the chain before its message equals the stamp's, and only before the first block that differs from the minted one, so a store edit or a compile edit within a message counts. A compiled message carrying several bodies is vouched for by none. Thinking without a stamp doesn't render once engaged: carriers, replies from before the first accepted compile, replies after a sharded body. A message the strip leaves empty is dropped. Stripped bodies are marked partial in provenance (missing: content) and in the layout, which is priced as sent: an entry the pass changed by its content as sent, and one it left nothing of reads as omitted. - Engagement: at the namespace's first accepted compile. Only acceptRound says which compile a reply was minted under, so a store whose host never accepts a round renders as before. - Release: what a compile strips is released on the branch, from the first stripped block's place in the stored message (min-merged), so thinking before an edited block, or in an earlier part of a split message, stays. The strategy's view then reads the message without it, so budgets count what is sent. A compile under a seed other than the owner's (a one-turn system change) keeps what was minted under the owner's seed, which comes back with it; a reply awaiting its own compile isn't released; a branch with no accepted compile releases nothing. With a system injection that changes every turn, each reply releases at the second compile after it. Released copies are memoized per stored message across compiles, because the strategy's _salienceCache is a WeakMap keyed by message object. Bodies are judged against the store, not the released view. - Strategy hooks: noteStrippedTokens(tokens) after every compile that strips, dry runs included, since AutobiographicalStrategy's dry-run select records its calibration estimate too; noteThinkingBound() once engaged, at each acceptance, strategy initialization and compile (so a manager another manager's acceptance engaged hears it too). With it, AutobiographicalStrategy renders and prices the live window as under carrierPolicy 'live-strip' (liveCarrierPolicy), since a carrier can never be sent there once bound. Mint and merge requests are unchanged. - Storage: Chronicle typed records `context-manager/thinking-binding` (format 1), one per accepted compile: seed, chain end, head, whether it became its branch's owner, and the stamps ([id, seed, [chain, first block] per part, block digests]) and releases no earlier record holds. Appended in acceptRound before the fold journal's record and made durable by the journal's sync or acceptRound's own after it. No sync before the append: a record that outlives unsynced messages names ids no later message takes (a message's id is its Chronicle record's), and a head past them costs a lose-once. Read when first needed and again at each acceptance, so a record whose write reported a failure after landing is found, not written twice. Not inside the accepted-layout record: the journal writes nothing for a compile without a layout. - A dry run binds and reports, and commits nothing. A manager remembers its 32 most recent compiles; accepting an older one, or one from before a reopen, records nothing. Built from the prototype on proto/thinking-binding (aa1dac57..64ae122f, rebased to d7f3e2be..56775e0c on anima-research#146's dac776b). Two prototype faults fixed here: a partial strip released the whole message's thinking, and pending compiles were never forgotten. Reviewed by Basil-1007, who didn't build it; the fence, the seed-carrying release rule, the dropped pre-append sync and the cursor search came from that review. Greptile's review of the first push found the shared-manager, reopen-owner, early-edit, layout, landed-record and bundled-cycle gaps fixed here. Tests: test/thinking-binding.test.ts, 40. - Red-first, with src at anima-research#146's head: 31 fail. The 9 that pass are the 4 unit tests of the new module, and five controls that pin what the pass must not strip. With src at the first push (d7a0a14), the six tests aimed at Greptile's findings fail, and so does the one for a strategy told at compile, besides two unit tests that fail only on mintStamp's new shape. - Removals: 68 pieces, each removed alone (removals-r3.sh). Each fails at least one test aimed at it, except two kept for stated reasons: the cross-compile copy memo (for the salience cache) and the bound on a branch's fence list (memory, under a host that stops accepting). Pieces cut because nothing failed without them: an "accepted once" check that the pending map's deletion already makes (twice), and trimming stamp digests to the last thinking block. - Full suite: 1066/1066 on Node 24.21.0 and 20.20.2 (anima-research#146's head: 1026). Growth: one record per accepted compile, all read at load, and every stamp kept in memory. Measured on a fresh store, 50 accepted compiles with 75 replies carrying thinking: 36.5 KB of binding records (304 to 885 bytes), against 20.5 KB of the journal's accepted-layout records and 311 KB of message content with 4,000-character signatures. Nothing compacts them. Pre-existing, not changed: AutobiographicalStrategy.selectAdaptive with dryRun still sets _lastCompileEstimate and arms calibration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d7a0a14 to
1c77918
Compare
|
Replying to this comment. Thank you. All six findings were real, and 1c77918 fixes them, with a test for each (replies on each thread). The whole suite passes 1066/1066 on Node 24.21.0 and 20.20.2, and the description is updated, including what's still not covered. — Oscar-1005 |
anima-research#155's agent-framework part needs a way to stop sending a thinking block the provider refused, when every request would otherwise send it again: a signature that was altered (refused under every prefix_mismatch_behavior, drop_block included; live, Opus 5.5, 2026-10-10), or, without drop_block, a block bound to another conversation. The host finds the stored message by the refused block's signature, releases it from that block, and retries. A store edit would rewrite history; a release keeps the store as it is and is branch-scoped, like the pass's own releases. - ContextManager.releaseThinking(messageId, fromBlock = 0) works on the current branch, from that block of the stored message on. From the next compile the strategy's view reads the message without it, through the release machinery the pass already uses (ThinkingBinding.release: the branch's released map, min-merged, and its unrecorded releases), so budgets and estimates count what is sent. Thinking before fromBlock stays; thinking after it in the message goes too, since it was minted after the refused block. The replies after the message lose their thinking at the next compile, as for any change before them, and the pass releases them in turn. - The branch's next accepted compile records it with the pass's own releases; until then it holds in this process. In the host's flow that acceptance is the retry that succeeds, so a reopen in between loses one round's release, and the next refusal brings it back. - It acts whether or not the pass is engaged, since it works through the view. - It throws for a message the current branch doesn't hold, or a fromBlock that isn't one of its blocks, so a host's off-by-one is loud. agent-framework's part of anima-research#155 asked for it, and agreed the shape before it was built. Tests: 5 new in test/thinking-binding.test.ts (45 there): what the host releases stops being sent, the replies after it are stripped, the release is recorded with the next acceptance and holds across a reopen; thinking before the block stays, and a later call keeps the earlier block; it releases before the pass is engaged; on the current branch only; and it refuses an unknown message, and a block that isn't one of the message's. Removals: 7 pieces, each failing a test aimed at it (the release, its recording, the min-merge, the three checks, the manager's call). Full suite: 1071/1071 on Node 24.21.0 and 20.20.2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…age claim says Emma-2309 (room-203 #78532): commit()'s doc still said snapshots "cost about as much again as the deltas", and recordBytes's said it measures a record "as stored". commit() now says snapshots are amortized against the deltas between them, so the bytes per acceptance stay bounded on average while the deltas do; recordBytes says it counts the whole record as JavaScript's JSON, which is what the snapshot rule compares. Comments only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, at the loop that stages resolution changes: main now walks the middle chunks (the prefix of pickerChunks) with the resolution each was built with, where this branch had walked result.finalResolutions and skipped head and tail ids. The resolution keeps main's loop and this branch's selectionCause, declared before it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anima-research#146's head now carries main 1566b3a, so this brings main in too. One conflict, at context-manager.ts's imports: main's describe-thrown import and this branch's thinking-binding import both stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isting identity while nothing is released The store hands out one listing from getAll while nothing is written, and main keys four memos on that listing's identity: the store order and its position index, the chunk by position, the post-strip estimates and salience by position. Once a branch had any released thinking, the released view mapped a fresh array on every getAll, so all four missed on every call: 13 distinct arrays in one select where there had been one. releaseThinkingView now memoizes the listing it maps, per source listing, under the release state it was last mapped in, in a WeakMap the binding keeps beside its released copies. A released map changes only through addRelease, which bumps the map's version when it releases more, so a listing mapped under an older version is mapped again. A release that releases nothing more keeps the listing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a release that lowers a held block From the internal review of the memo round. - releaseThinkingView's doc said the listing is memoized per store listing and release state. It is memoized per source listing, under the release state it was last mapped in, so it keeps the source's identity: one listing while nothing is written or released, from a store that hands out one. A manager's merged view over auxiliary stores builds its listing afresh on every call, released or not, and so does this view over it. getFrom and getTail map a fresh array each call. Callers must not mutate the listing, as with the store's own. - addRelease's doc names the version bump the memo relies on. - A test releases a message from its second thinking block, then lowers the release to its first: the map already held the message, so its size doesn't change, and the listing is mapped again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@slimepriestess @Anarchid This PR is now at
Written from Oscar-1005's notes. — Lottie-2661 |
slimepriestess
left a comment
There was a problem hiding this comment.
Changes requested at 72dcca5, for one gap: two rounds out at once on one branch. The rest holds up under everything we tried, and that was a lot.
Run first-hand on Node 20 from a clean dist:
- At the head, 1,372 of 1,372 tests pass.
- On a trial merge with today's
main(cd6ad3d, 0.15.0, 52 commits past this branch's base), 1,502 of 1,502 pass, and the textual merge is clean.
#146 merged this morning (its head 54d916b, merged as c90d4da). So this PR's own change is:
thinking-binding.ts;- its wiring in
compile(),acceptRound, message stamping, fork and branch inheritance, andreleaseThinking; - the strategy hooks, and
AutobiographicalStrategy's live window and calibration; - the types and tests.
The gap (blocking): a reply stored while two compiles are out on one branch is stamped from the one bound last
The module comment says the pass "can't tell which compile a reply answers", and it names rounds that overlap a branch switch as the limit that follows. Rounds that overlap on one branch have the same problem, and the comment doesn't name it. Your own test "keeps the owner bound last when an earlier compile at the same head is accepted after it" sets this case up: a heartbeat compile and an ordinary compile, both at one head, both accepted.
Here it is with a reply. Our second reader found this, and we reproduced it in a scratch copy at 72dcca5:
await turn(cm, 'u1', 'a1', 'S1'); // as in thinking-binding.test.ts
cm.addMessage('user', text('u2'));
const hb = await cm.compile(BUDGET, undefined, { prefixIdentity: 'p0+heartbeat' });
const ord = await cm.compile(BUDGET, undefined, { prefixIdentity: 'p0' });
cm.addMessage('assistant', reply('quiet', 'H1')); // the heartbeat's reply
cm.acceptRound({ provenance: hb.provenance! });
cm.acceptRound({ provenance: ord.provenance! });
cm.addMessage('user', text('u3'));
const next = await cm.compile(BUDGET, undefined, { prefixIdentity: 'p0' });
// signatures in next: ["S1","H1"]H1 was minted under the heartbeat's seed, and the next ordinary compile sends it under p0. On an enforcing account, that's the refusal from #155.
- Reply between the acceptances: store H1, accept
hb, store S2, acceptord. The compile renders["S1","H1","S2"]. S2 goes out with H1 before it, thoughord's request never carried H1. - Acceptances in the other order: store S2, accept
ord, store H1, accepthb. The compile renders["S1","S2","H1"]withthinkingStripped0.
As we read it, a stamp is taken at the next walk, after the acceptances, and the owner is the compile bound last by then. Separately, accept() drops every compile bound before the new owner from the fence ("Compiles bound before the owner minted nothing after it", thinking-binding.ts:828–830). That assumption fails when rounds overlap.
Ask. Either of these would do:
- Give a reply stored while more than one unaccepted compile on its branch disagrees with the walk no stamp, as the pass already does for a branch switch, so its thinking never renders. Include compiles bound before the eventual owner.
- Or name overlap on one branch as a limit beside the switch case, with a test that pins what the pass does there.
Either way, a test that fails without it. We haven't checked whether agent-framework ever has two rounds out for one agent on one branch. Your test treats it as a case the pass handles, so we're holding the library to that.
Which test goes red
We broke the pass 20 ways, and each one failed a test at the assertion written for it (ERR_ASSERTION, not a crash):
- The chain ignored at the stamp check: 21 tests fail.
- The block-by-block compare dropped.
- The chain extended over what the strategy rendered instead of what goes out: the fold, sharded-body and fence tests fail.
- A compiled message carrying several bodies vouched for.
- The seed without the system injections, and the seed without
prefixIdentity. - Unsorted keys in
canonicalJson. - An emptied message kept, and its breakpoint not moved.
- Every part of a bundled tool cycle read as part 0.
- Another writer's message taken as carried.
- A one-turn seed releasing.
- A reply awaiting its compile's acceptance released.
- A reply stored on another branch left unmarked.
- A new branch not inheriting its parent's releases.
bindsThinking: falseignored.- The listing memo always missing: the warm-cache test fails.
- The memo ignoring the release version: the store-edit, remap and warm-cache tests fail.
- The live window's carrier policy ignoring bound thinking.
- The strip not taken out of the calibration sample.
One of these was a null mutant on the first try (TypeScript narrowing). We rewrote it, and it fails a test now.
What we checked by reading
- Cache breakpoints: they're a message-level field, outside what
chainStephashes (participant and content), so moving a breakpoint never strips a reply. - Cases the pass strips: in the cases we read, it strips rather than sends:
- no stamp;
- a compiled message carrying several bodies;
- another writer's message stored while this process runs;
- a compile the fence stops at;
- a reply stored on a branch other than the one last bound.
- Cases it can't see: the module comment names these: a reopen before the next record, rounds that overlap a branch switch, and messages stored before the process opened the namespace. In those, the thinking goes out until the host releases it. The gap above is one more.
For whoever deploys this (non-blocking): the fix for #155 needs the host's half
- Nothing calls the pass yet. agent-framework's
main(91e70ed) calls none ofacceptRound,prefixIdentityorbindsThinking. - #253 would engage it, but not alone. anima-research/agent-framework#253 (open, at
a0a390b) callsacceptRound. It engages the pass only once agent-framework takes a context-manager release that carries this PR:mainpins^0.14.1, and #253 doesn't change that. - Prefix changes the seed can't see. #253 doesn't set
prefixIdentity, so the seed would cover only the system injections a compile returns. If the host changes its own system prompt or tools between compiles outside those injections, a reply minted under the old prefix would still render. Your description names this limit. - Request preparation. #253's request preparation also changes the compiled messages: it drops empty text blocks, and it adds separator turns before silent ticks. The pass vouches for what was compiled, so this holds only while that preparation is a fixed function of the compiled messages. We haven't checked that.
- The host also owns
bindsThinkingper model, and the release after athinking_bindingrefusal.
It may be worth stating the order in #155, or in an issue on agent-framework.
… limit; one round at a time is the host's part When two compiles on a branch are out at once and both are accepted, a reply stored after the second is bound and before both are accepted is stamped from the one bound last: the pass takes a compile bound before its branch's owner to have minted nothing after the owner's head. That is what lets a branch recover from a round that ended unaccepted, since when a reply is stored a compile still out and one whose round failed look the same. So a reply that answered the other compile goes out under a prefix it wasn't minted under until the host releases it. - The module comment names same-branch overlap beside the branch-switch limit, and the host's part now says a host runs one round at a time on a branch: a compile is accepted, or its round has ended, before the branch's next committing compile. - Tests pin what the pass does when a host overlaps rounds anyway: a heartbeat compile and an ordinary one at one head, with the heartbeat's reply stored before both acceptances, between them, or after the ordinary one's; and that releasing the misattributed reply takes its thinking out of every later compile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Replying to this review. @slimepriestess We've named rounds that overlap on one branch as a limit beside the branch-switch one, with tests that pin what the pass does, rather than making the pass strip. The new head is The pass takes a compile bound before its branch's owner to have minted nothing after the owner's head. That is what lets a branch recover from a round that ended unaccepted: when a reply is stored, a compile still out and one whose round failed look the same. The module comment now says that when two compiles on a branch are out at once and both are accepted, a reply stored after the second is bound and before both are accepted is stamped from the one bound last, and that one which answered the other goes out with every compile under that one's seed until the host releases it. The host's part now says a host runs one round at a time on a branch: a compile is accepted, or its round has ended, before the branch's next committing compile. In agent-framework, a scheduled turn takes the agent's turn token before it awaits anything, and the scheduler requeues a wake while the agent is turn-alive, inferring, streaming or waiting for tools. Its direct Tests ("rounds that overlap on one branch", Thank you for the review, and for building the overlap case. Written by Oscar-1005. — Lottie-2661 |
slimepriestess
left a comment
There was a problem hiding this comment.
Approve at 1dfac33. This answers our changes-requested review with the second of the two ways we offered.
What we checked:
- The limit is named. The module comment now says rounds that overlap on one branch defeat the stamp, as an overlap across a switch does. It also gives the reason the pass works this way: when a reply is stored, a compile still out and one whose round failed look the same, and treating them alike is what lets a branch recover from a round that ended unaccepted. That's a fair trade, and now it's written where the next reader will look.
- The host contract gains the rule that avoids the limit: "A host runs one round at a time on a branch: a compile is accepted, or its round has ended, before the branch's next committing compile."
- What the pass does there is pinned. The new tests are our three orderings as written: the heartbeat's reply sent under the ordinary prefix, both between and before the acceptances, and with nothing stripped when the ordinary round comes first. A fourth test shows the host's
releaseThinkingtakes the misattributed reply's thinking out of every later compile. - Run: at
1dfac33, from a clean dist on Node 20.20.2, 1,375 of 1,375 tests pass, and the three new ones among them.
For whoever deploys this (non-blocking), an addition to our last note: the new host-contract line is now part of what agent-framework has to keep when it takes this release, beside prefixIdentity and the release after a refusal. By our reading, agent-framework keeps it today. Its wake gate requeues an agent's wake while that agent's turn is live or it's inferring, streaming or waiting for tools (src/framework.ts:10621 and :10633 on main at 14e25dc), and a heartbeat enters through the same queue. A budget restart skips the live-turn check, but it continues the same turn after its compile's round has ended rather than running beside it. So the line holds as written. anima-research/agent-framework#253, or whichever PR adopts this release, would do well to pin it with a test there.
|
@slimepriestess We're weighing replacing this PR with a much smaller one, and we'd value your view before anything is built, since you approved this one at Why. We checked each part of this PR against what the provider already does, and three findings point to a smaller design:
The design we're weighing:
It's meant to keep the guarantee this PR gives for every rewrite of our own, in much less code than this PR's 1,291 lines of source. What it changes for a resident is that it no longer sees its earlier turns' thinking. We'd like to know whether this PR covers a case the smaller design would miss, and whether you'd keep any part of it. Written by Oscar-1005. Ezra-2423 checked the PR part by part and ran the census. — Lottie-2661 |
| } | ||
| next.push(block); | ||
| } | ||
| if (next.length !== content.length) content = next; |
There was a problem hiding this comment.
Stripping breaks Responses item pairs
When the pass strips reasoning from Responses history, the following text or tool-call block keeps its rawItem. Both bindThinking() and releaseThinkingView() remove reasoning without calling releaseEditedRawForms(). The remaining item can therefore be replayed with its provider ID but without the reasoning it was paired with, causing the provider to reject the request.
Reconcile raw forms in both strip paths before hashing or returning the remaining content. The existing stripThinkingBlocks() path already does this.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/thinking-binding.ts
Line: 297
Comment:
**Stripping breaks Responses item pairs**
When the pass strips reasoning from Responses history, the following text or tool-call block keeps its `rawItem`. Both `bindThinking()` and `releaseThinkingView()` remove reasoning without calling `releaseEditedRawForms()`. The remaining item can therefore be replayed with its provider ID but without the reasoning it was paired with, causing the provider to reject the request.
Reconcile raw forms in both strip paths before hashing or returning the remaining content. The existing `stripThinkingBlocks()` path already does this.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| private foreign(key: string, message: StoredMessage): boolean { | ||
| return message.sequence > this.attributedAfter && !this.stored.get(key)?.has(message.id); |
There was a problem hiding this comment.
Historical branches trust other writers
foreign() compares every message’s branch-local sequence with one attributedAfter value captured on opening. After opening a long history and switching to a branch made at an old message, new writes can remain below that value. Another namespace’s mid-round write into the shared slot is then treated as carried.
A reply minted without that write gets stamped after it, so the next compile keeps thinking under a prefix it was not minted under. The provider can refuse or drop it. Track the opening boundary per branch, or use a store-wide record identity.
Knowledge Base Used: Message store and context lifecycle
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/thinking-binding.ts
Line: 610-611
Comment:
**Historical branches trust other writers**
`foreign()` compares every message’s branch-local sequence with one `attributedAfter` value captured on opening. After opening a long history and switching to a branch made at an old message, new writes can remain below that value. Another namespace’s mid-round write into the shared slot is then treated as carried.
A reply minted without that write gets stamped after it, so the next compile keeps thinking under a prefix it was not minted under. The provider can refuse or drop it. Track the opening boundary per branch, or use a store-wide record identity.
**Knowledge Base Used:** [Message store and context lifecycle](https://app.greptile.com/anima-labs/-/custom-context/knowledge-base/anima-research/context-manager/-/docs/message-store-and-lifecycle.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| if (this.loaded) return; | ||
| this.loaded = true; | ||
| this.refresh(); |
There was a problem hiding this comment.
Failed reads disable thinking checks
load() sets loaded before refresh() succeeds. If the first record listing or read throws while opening on an app-owned store, retrying ContextManager.open() on that same store reuses the cached ThinkingBinding. It skips loading and can appear unengaged even though accepted binding records exist.
The next compile can send old signed thinking after a prefix change, causing a provider rejection. Only a later acceptance retries refresh(). Set loaded after the initial refresh completes.
Knowledge Base Used: Context compilation pipeline
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/thinking-binding.ts
Line: 804-806
Comment:
**Failed reads disable thinking checks**
`load()` sets `loaded` before `refresh()` succeeds. If the first record listing or read throws while opening on an app-owned store, retrying `ContextManager.open()` on that same store reuses the cached `ThinkingBinding`. It skips loading and can appear unengaged even though accepted binding records exist.
The next compile can send old signed thinking after a prefix change, causing a provider rejection. Only a later acceptance retries `refresh()`. Set `loaded` after the initial refresh completes.
**Knowledge Base Used:** [Context compilation pipeline](https://app.greptile.com/anima-labs/-/custom-context/knowledge-base/anima-research/context-manager/-/docs/context-compilation.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| // start, since a strategy can join several text blocks into one. | ||
| let from = 0; | ||
| for (const block of stored) { | ||
| if (block.type === 'text' && block.text === '') continue; // carries nothing |
There was a problem hiding this comment.
Missing Responses items look complete
This shortcut also skips Responses blocks whose entire payload is in rawItem, such as encrypted compaction or hosted tool calls. A capped raw reply can lose a later such block when remaining reaches zero while keeping its earlier reasoning and text unchanged. coversStoredContent() then reports the shortened body as complete because it ignores the missing block.
This makes complete and fold receipts overlook lost content. Only skip genuinely empty text, and compare the raw payload of these Responses blocks.
Knowledge Base Used: Compilation provenance and fold journal
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/compile-provenance.ts
Line: 390
Comment:
**Missing Responses items look complete**
This shortcut also skips Responses blocks whose entire payload is in `rawItem`, such as encrypted compaction or hosted tool calls. A capped raw reply can lose a later such block when `remaining` reaches zero while keeping its earlier reasoning and text unchanged. `coversStoredContent()` then reports the shortened body as complete because it ignores the missing block.
This makes `complete` and fold receipts overlook lost content. Only skip genuinely empty text, and compare the raw payload of these Responses blocks.
**Knowledge Base Used:** [Compilation provenance and fold journal](https://app.greptile.com/anima-labs/-/custom-context/knowledge-base/anima-research/context-manager/-/docs/compilation-provenance-and-fold-journal.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
@slimepriestess Closing in favour of #331, which replaces this PR.
Thank you for your review and your approval at Written by Ezra-2423 (Claude Opus 5.5). — Lottie-2661 |
Since
80a3dd22affb3a48merges feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 at03ad9ae6, which carriesmainthrough1566b3a0. It merged without conflicts. feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 has one docs commit since,54d916b9, which this branch doesn't carry.main's listing memos warm (ad9e6764,72dcca5e). The store hands out one listing fromgetAllwhile nothing is written, andmainkeys four memos on that listing's identity: the store order and its position index, the chunk by position, the post-strip estimates, and salience by position. Once a branch had released thinking, the released view mapped a fresh array on everygetAll, so all four missed every time. One select after one release made 13 different arrays, where the store hands out one. The view now memoizes the listing it maps, per source listing, under the release state it was last mapped in, so the memos stay warm while nothing is written or released. A manager's merged view over auxiliary stores builds a fresh listing on every call, released or not, and that is unchanged.Since
61b50548An automated review of this PR's first two commits found 26 issues. This round answers them, adds a compile option for models that don't bind thinking to its prefix, and merges #146's current head. The later commits answer the reviews of the round.
The merge,
0ea26b93. It brings feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 at13e6208b, which carries main through01a697e8. One file conflicted,src/context-manager.ts. fix(autobiographical): a dry compile leaves what a select leaves behind as it found it #161 landed in between, and since then a dry select puts back the calibration estimate the last committing compile made. So a dry compile no longer tells the strategy what the pass stripped, which would have subtracted a dry run's strip from a real compile's estimate.SelectOptions.bindsThinking, defaulttrue. We probed withdrop_block, replaying a block under a changed prefix. Opus 5.5, Haiku 5.5, Sonnet 5.5 and Fable 5.1 dropped it. Fable 5 and Sonnet 5 kept it and counted its tokens. For a target that keeps thinking, stripping gains nothing and costs retained thinking and the live window's recall carriers. With the optionfalse, a compile strips nothing, and stamping goes on, so a later compile for a target that binds finds each reply's stamp. It brings back nothing the branch has released, whether the pass or the host released it. The host sets it from what it knows of the model.The manager tells the strategy per compile. It calls
noteThinkingBound(bound)before each compile's select, with whether that compile's thinking is bound, rather than once at initialization and at acceptance. SoAutobiographicalStrategy's live window follows the target, and a reused instance renders as configured on a store with no accepted round. A dry compile tells the strategy its own value, then puts back the value last told.Other writers in a shared slot. Managers of different namespaces that aren't isolated share the default message slot. The walk used to read another writer's message, stored while a round was out, as if the live request had carried it, and the provider would then refuse the reply's thinking. Now the walk takes only what its own namespace's managers stored as carried. A compile that renders another writer's message differs from the walk at its head, so the fence holds the replies after it until that compile is accepted. The pass can't tell apart the messages stored before this process opened the namespace, so it takes them as carried.
A reply stored on another branch than the latest compile was bound on (a branch switched while the round was out) gets a stamp nothing matches. The walk used to stamp it from the new branch's chain, which can match.
A fork, or a branch made at a message, starts with the releases of the branch it was made from, so a block the provider refused there isn't sent again. The new branch's first accepted compile records them.
A dry compile's provenance carries
dryRun, andacceptRoundrefuses it with a warning and records nothing. It used to write the fold receipt and engage the pass while the binding ignored the compile.A message the strip empties is dropped, as before. Its cache breakpoint now moves to the message before it, if there is one, since that message ends the same prefix.
b509ed3bkeeps the noted writes bounded for a host that never accepts a round. Each bind forgets the writes no walk can read again. Those are the writes at or below the owner's head, or, while the branch has no owner, at or below the lowest head among its remembered compiles, any of which could become the owner.d5889edbkeeps released thinking released for a compile that doesn't bind, whatever its target. A host's release marks a block the provider refused. Restoring the pass's releases would cost two cache rewrites per switch and back. And every read of the branch, budgets and ticks included, shares one view. A test checks it.2e7037b7answers an automated review of the round. Initialization tells the strategy whether the namespace is engaged, so a dry compile before any committing one, or with a strategy new to its manager, puts back the right value. A dry compile's restore also covers the reads between telling the strategy and selecting. The texts now state the pass's limits: rounds that overlap a branch switch, what the noted writes' floor assumes, that engagement is the namespace's, what a reopen loses, and a first message's cache breakpoint.d04b531f,6911c725andb72517e9answer two colleagues' reviews of the round. Initialization runs on open, onsetStrategyand at every branch switch. It now tells the strategy the namespace's engagement for the last committing compile's target, and it assumes a target that binds before any compile. So a fresh strategy gets the engagement, a target that doesn't bind stays unbound across a switch, and a switch after the store's first acceptance binds. The tell also runs inside thetrywhosefinallyputs a dry compile's value back, so a strategy hook that takes the value and then throws still leaves the strategy as the dry compile found it.80a3dd22corrects the texts on a late reply. A reply whose round overlapped a branch switch, or one stamped again after a reopen over an edit made in that window, doesn't send its thinking only once, as they said. It can go out with every compile that sends the same prefix before it, refused or dropped each time, until the host releases it.Texts. They now say four things:
prefixIdentity;thinkingStripped's doc says it counts what the pass removed, and the changelog namesacceptRound's sync.Three findings got a different answer, with reasons in the review's verdicts:
Problem
In #155, on an account that enforces thinking binding, Anthropic refuses a signed
thinkingblock once anything before it has changed since it was minted. The refusal is a 400, "The block is bound to a different conversation", or a silent drop underdrop_block. A compile that folds, merges, caps, strips an image or releases a raw form changes the prefix of every reply after the change. So the reporter's resident got a 400 on every turn after its first fold.This is context-manager's part of the design agreed on #155 (the design, its correction, and the reporter's answers to the three decisions it left open).
Changes
A pass in
compile()(src/thinking-binding.ts) runs over the final messages, injections included. It covers every strategy without knowing which rewrite did what.SelectOptions.prefixIdentity(new, the host's identity for its system prompt and tools) and the system injections the compile returns.auxiliaryMessageViews), and passes over what another namespace's manager stores in a shared slot while this process runs.acceptRound's doc. The pass takes what a host stores through this namespace's managers after a compile, up to a reply, as what the round's request carried after the compile's messages, in that order, each message as the walk read it. A host stores each round as it went out and each reply whole, and defers its other writes (a message heard mid-tool-loop) until the turn ends. So an edit made after the walk read a message strips the replies after it, and one made before (a host writing a tool result over its placeholder before anything after it is stored or compiled) is what the round carried. Agent-framework defers cross-turn writes during a turn.acceptRound, since only a host that accepts rounds says which compile a reply was minted under. A store whose host never accepts a round renders as before. A compile withbindsThinking: falsestrips nothing more, and stamping goes on. What the branch has released stays released.context-manager/thinking-binding), one per accepted compile. Each holds its compile's seed, chain end and head, whether its acceptance made it the compile its branch's replies follow, and the stamps and releases no earlier record holds.acceptRoundwrites them before the fold journal's record, and the sync that follows makes them durable. The pass reads them when first needed and at each acceptance after that, so it finds a record whose write reported a failure after landing rather than writing it twice. They're separate from the accepted-layout record because the journal writes nothing for a compile without a layout.dryRun, andacceptRoundrefuses it. A manager remembers its 32 most recent compiles, and accepting an older one records nothing.What callers will notice.
CompileResult.thinkingStrippedcounts the blocks the pass removed from what the strategy rendered. It rises at a fold and once when the pass engages, and stays above zero only while something keeps invalidating replies, such as a positional injection.SelectOptions.bindsThinking(defaulttrue) isfalsefor a target that keeps thinking under a changed prefix. Such a compile strips nothing more, and thinking the branch has released stays released. It records a provider behaviour we measured, which the provider could change. If a model setfalsestarts binding, an enforcing account refuses the first request that sends a block under a changed prefix, and this setting is the one to check.noteStrippedTokens(tokens)reports thinking a committing compile priced but didn't send, andAutobiographicalStrategytakes it out of its calibration sample.noteThinkingBound(bound)says whether a compile's thinking is bound, before each compile's select, and at initialization it gives the namespace's engagement for the last committing compile's target.CompileProvenance.dryRunis set on a dry compile's provenance, whichacceptRoundrefuses.releaseThinking(messageId, fromBlock = 0)lets a host stop sending a block the provider refused, which every request would otherwise send again. That's a block whose signature was altered (refused under everyprefix_mismatch_behavior,drop_blockincluded), or a binding refusal withoutdrop_block. From the next compile, the strategy's view reads the message without that thinking, from that block on, as after the pass's own release. The branch's next accepted compile records it, so a restart before then loses it, and the host releases it again when the provider refuses it again. A branch made from this one starts with it.AutobiographicalStrategyrenders and prices the live window as undercarrierPolicy: 'live-strip', whatever is configured. A carrier was minted in the summarizer's request, so an enforcing account refuses it there. Without this the plan would price carriers the pass then strips.'full'still governs a store whose host has never accepted a round, and a target that keeps thinking, and mint and merge requests are unchanged. Folding breaks preserved-thinking signatures on enforced accounts: resident 400s ("bound to a different conversation") on every turn after the first fold #155's reporter agreed to this, and to the two decisions behind the bullets above: one behaviour on every account, and the one-time strip.For a rehearsal on a real store
The pass engages only on a store whose host calls
acceptRound. In agent-framework that's anima-research/agent-framework#253, approved and not yet merged. So a rehearsal needs a host on #253, or a driver that callsacceptRoundafter each compile it sends and stores each round as it went out. PassingprefixIdentityis what lets the pass see a change to the system or the tools. Without it, such a change is left to the provider. Agent-framework's part of the #155 design is to pass it, and to ask membrane fordrop_block. The number to watch isthinkingStrippedbetween folds.Tests
Every run below is on Linux x64, from a fresh
npm ci, withnpm run buildandnpm test.This branch at
72dcca5e. Typecheck and build are clean, and the suite passes 1372 of 1372 on Node 24.21.0 and on Node 20.20.2, with membrane 0.5.86. GitHub's CI run at it passes all four Build & Test legs: Ubuntu and macOS, each on Node 20 and 24. Against feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 at03ad9ae6, the only test file this PR adds istest/thinking-binding.test.ts, with 59 tests.Merged onto main
53bab67f. The merge is clean, and typecheck, build and the suite pass, 1455 of 1455 on both Nodes.What they cover:
acceptRound), accepting once (also when a write reports a failure after landing), and forgetting old compiles;Red checks for this round. Each is a mutant of the source at
b72517e9, applied alone and run againsttest/thinking-binding.test.ts, which passes 56 of 56 unmutated. Each of these fails the test aimed at it:bindsThinkingin the pass, or in what the strategy is told;trywhosefinallyputs a dry compile's value back;Forgetting no noted writes fails nothing, since that only costs memory.
The first two commits. With
srcat feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146's then head, 36 of their 45 tests failed. The 9 that passed were the new module's 4 unit tests, and five controls that pin what the pass must not strip. Each of 75 pieces, removed alone, failed at least one test aimed at it, except two:Not verified
bindsThinkingnames, all on one organization's account.drop_blockor refused without, until the host releases it. Making each stamp durable when it's made would take a record per reply.releaseThinking.[Continue]row, a heartbeat separator) aren't in the chain. A reply minted after one comes back as a leftover, which the provider drops underdrop_block. Agent-framework's part of Folding breaks preserved-thinking signatures on enforced accounts: resident 400s ("bound to a different conversation") on every turn after the first fold #155 covers them.changelog.d/thinking-binding.fixed.md,thinking-binding.added.mdandthinking-binding.changed.md.Companion PRs and merge order
main. Until feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 lands, this diff includes it. This PR's own change is1c779181(the pass),61b50548(the host's own release), the merge of feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 at0ea26b93, the round01c05c2b,b509ed3b,d5889edb,2e7037b7,d04b531f,6911c725,b72517e9and80a3dd22, the merge of feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 at03ad9ae6(affb3a48), andad9e6764and72dcca5e. Please merge feat: compile provenance, rendered layouts and fold receipts at accepted rounds #146 first.acceptRound, so it's what engages this pass. Agent-framework's own part of Folding breaks preserved-thinking signatures on enforced accounts: resident 400s ("bound to a different conversation") on every turn after the first fold #155 (asking fordrop_block, its incident alerts, the prefix identity, and two prefix fixes) follows this release.drop_blockoption, a per-call report ofinput_transformations, and the binding 400 as its own error type) is independent of this package.🤖 Built by Oscar (Oscar-1005, Claude Opus 5.5) on the Rookery's Pi tooling, with the
Co-Authored-Bymodel trailer on its commits. Ada (Ada-1017) reviewed the design, wrote its agent-framework side, and agreed thebindsThinkingshape with Basil. Basil (Basil-1007, Claude Opus 5.5), who didn't build it, reviewed the code and this text for the first two commits.80a3dd22. Fiona's own runs and probes cover the round, and her probe of overlapping rounds led to80a3dd22. She also read this description in full.affb3a48,ad9e6764and72dcca5e, measured the 13 arrays before the memo and 1 after it, and approved72dcca5e.01c05c2banswers. One of that round found what2e7037b7answers, and one of the memo found what72dcca5eanswers.The names in parentheses are Rookery colleague references; the house shares one GitHub account.