Skip to content

fix(tune-out): store the subconscious's last looks - #939

Open
nissa-seru wants to merge 1 commit into
anima-research:mainfrom
nissa-seru:hilda/tune-out-looks-stored
Open

nissa-seru wants to merge 1 commit into
anima-research:mainfrom
nissa-seru:hilda/tune-out-looks-stored

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

A persistent subconscious's last look at each tuned-out channel is now stored, so a coordinator built after a restart reads it back. At main 48f26ce56c the coordinator keys each look ${serverId}\0${channelId}, and chronicle's state write refuses an object key that holds a NUL byte, on both published 0.5.x releases. The coordinator logs the refusal and keeps the look in memory only, so a coordinator built after a restart reads no looks. This change keys a channel by JSON.stringify([serverId, channelId]). Its one new test fails with 48f26ce56c's coordinator and passes at this head. This addresses #917.

@antra-tess, we're asking you to review this before it's merged, since it changes what a persistent subconscious's next look counts after a restart: only what was held since its last look, now that a coordinator built after a restart reads that look back.

Problem

#917 reports that the looks are never stored, found against chronicle at anima-research/chronicle#62's head. A probe outside this change ran main's coordinator at 48f26ce56c over a real JsStore, on Linux x86_64 with Node 24.22.2, once with chronicle 0.5.0 and once with 0.5.1, each pinned exactly:

  • A state value whose key holds a NUL (disc\0#a) is refused with "nul byte found in provided data at position: 4". The same value keyed disc|#a is written and read back, from the same store and from the store reopened from disk.
  • markLook logs once: "[tune-out] the subconscious's last looks (tune-out/scout/last-looks) could not be stored: nul byte found in provided data at position: 4".
  • A second coordinator over the same store reads the looks as {}, and so does one over the store reopened from disk.

Both runs printed the same result. At 48f26ce56c, key() builds the NUL form (src/tune-out/coordinator.ts:391–393), the all-except scope's key is '\0all-except' (:383), and writeLooks catches the refusal and logs it (:1595–1602).

The coordinator's documentation already says the looks are "Durable, so that after a restart its next look counts only what was held since" (agent-framework 0.30.0's published dist/src/tune-out/coordinator.d.ts:229–231). At v0.29.0, the release before, the coordinator's source names no last-looks, and the release takes chronicle ^0.4.0. 0.30.0 takes chronicle ^0.5.0, and at v0.30.0 its coordinator builds the same NUL key (src/tune-out/coordinator.ts:335–337).

Changes

Line numbers are this head's, in src/tune-out/coordinator.ts.

  • A channel's key is JSON.stringify([serverId, channelId]) (key(), :395–397). The coordinator uses it for its in-memory maps and for the stored looks alike. channelOf() (:399–401) parses it back where the coordinator needs the pair, at the four places that split the NUL form on main (:512, :1099, :1198, :1239).
  • The all-except scope's timer key is 'all-except' (:384). A channel's key starts with [, so the two can't be equal, whatever ids a host uses.
  • handOffKey() (:404), which keys the readers' hand-off bookkeeping, is unchanged.
  • Looks stored under the old key have no reader. Each write of them was refused on chronicle 0.5.0 and 0.5.1 (under Problem), and 0.30.0's range for chronicle is ^0.5.0.

On October 30 at 06:07 UTC, this head merged into main at 48f26ce56c with no conflict, and git merge-tree of each of the 41 open pull requests with that merge gave the same conflicted files, with the same number of conflict blocks, as with main alone.

Tests

This head, 2ff23f90aa, is one commit on main at 48f26ce56c. It changes src/tune-out/coordinator.ts, adds test/tune-out-last-looks-stored.test.ts, and adds the changelog fragment.

  • The new test, "the subconscious's last looks are read back from the store, by a coordinator over the same store and over the store reopened": one coordinator marks looks for three epochs, one of them for the same channel id on two servers. A second coordinator over the same store, and a third over the store reopened from disk, each read them back whole.
  • The consequence after a restart is pinned already, over a stand-in registry that keeps any key in memory: test/tune-out-reader-queued.test.ts:537, "after a restart, the persistent subconscious's cadence look counts only what was held since its last look". The new test pins the round trip through chronicle's own store, which that test's stand-in skips.
  • The runs, at a clean 2ff23f90aa, on Linux x86_64 with Node 24.22.2: the repository has no lockfile, and npm install resolved chronicle 0.5.1, context-manager 0.24.1 and membrane 0.8.0. tsc, the build and pretest exit 0. Run one file at a time, the new test passes, and so do the 45 other test files whose source names TuneOutCoordinator, tune_out, tuneOut, cancelTuneOut or all-except: 1,103 tests in 46 files, none failing.
  • Red first: with 48f26ce56c's src/tune-out/coordinator.ts in place of this head's, in the same run, the new test fails at "read back over the same store".

Not verified


  • Changelog fragment added: changelog.d/917-tune-out-last-looks-stored.fixed.md.

Hilda-2681 made the change and ran its probes, on the shape Ada-1017 set. Crispin-2406 read it. Beth-2709 wrote this description.

The coordinator keyed a channel `${serverId}\0${channelId}` in its maps
and in the persistent subconscious's stored last looks
(tune-out/<primary>/last-looks). Chronicle's state write refuses an
object key that holds a NUL byte ("nul byte found in provided data"), so
every write of the looks failed. writeLooks logged the error and kept
the look in memory only, so a restart forgot every look, and the
subconscious's first cadence look after it counted every held message as
new.

A channel's key is now JSON.stringify([serverId, channelId]), parsed
back where the coordinator needs the pair, and the all-except scope's
timer key is 'all-except', which no channel's key, starting with '[',
can equal. No store holds looks under the old key, since every such
write was refused, so nothing reads it.

handOffKey, which keys the reader forks' hand-off bookkeeping in memory
and is never stored, is unchanged.

Test: coordinators over the same store, and over the store reopened from
disk, read back the looks another coordinator marked for three epochs,
one of them for the same channel id on two servers. Before this change
both read {}.
@nissa-seru nissa-seru self-assigned this Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant