Repository navigation
fix(memory): stop layout divergence and serve the graph as map tiles - #586
Merged
Merged
Conversation
The Memory graph stopped updating and drew five thousand memories as 35 marks. Every background layout pass failed with "NOT NULL constraint failed: graph_layout.x", so nothing recorded since late October was ever placed, and the overview binned the whole store into a handful of cells that had to be clicked open one at a time. Once that was fixed the map drew its lobes as solid blobs under a grey mesh of lines. Warm passes re-ran ForceAtlas2 from the stored positions and then pushed the lobes apart again, but the stored positions were already pushed apart and the simulation only pulled part of that back. On an unchanged store each pass grew the map by about 1.4x; over weeks that compounded to 1e38, where squaring a coordinate overflows to Infinity and the pass produced NaN. The binned overview divided that astronomically wide extent into a grid, which is why nearly every memory fell into the same few cells. Layout: a warm pass now moves nothing that is already placed. Arrivals are dropped next to what they connect to and relaxed by a bounded local solver scaled to the map's own neighbour spacing. The map is re-arranged from scratch only on a version change, a diverged arrangement, or once a fifth of the store arrived since the last cold pass; the result is rigidly aligned onto the previous map and keeps its community numbers, so colours and orientation hold. A cold pass now spreads each lobe's dense core to a minimum spacing and packs the lobes toward the centre without overlap, which took the median distance to a nearest neighbour from 46 to 413 units on a real store. Non-finite rows are never written. Archived and superseded memories get positions too. View: the graph is served as map tiles over a fixed world square. The server shares one screen-wide budget between the tiles on screen and folds cells into clusters only when the screen cannot draw everything; zooming in resolves them without clicks. A store that fits the budget is fetched once as a single tile, and any complete tile stands in for every deeper level. Derived edges, node degree and the neighbour spacing are persisted by the pass (migration 089). The meta carries a layout stamp, so reinforcement no longer refetches tiles. Canvas: nodes are sized from the map's neighbour spacing and grow with the zoom up to a ceiling, so they neither merge at the overview nor shrink into the gaps when zoomed. At rest only each memory's strongest links are drawn; hovering or selecting a memory shows all of its edges. The selection, its connections and search hits are always drawn. Framing is computed from the world box, because sigma never builds its matrix while the graph is empty and the first framing happened then. Sigma's index is settled after every structural change, which removes the "can't be repaint" fallback. Restores the selection/search highlight, theme repaint, hover from the results list and inset resize effects lost in #483.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the memory layout pass that diverged to NaN and failed on every run, replaces the binned "click to open" overview with a zoom-driven tiled map that shows every memory whenever the screen can draw it, and makes the map readable: spread-out lobes, density-aware node sizes, and a sparse edge backbone.
Why
Every layout pass since late October failed with
NOT NULL constraint failed: graph_layout.x. Warm passes re-settled the map from its own previously spread result, inflating it ~1.4x per pass until coordinates reached 1e38 and overflowed. New memories were never placed, and the overview binned 5,171 memories into ~35 marks that had to be clicked open. After that fix, lobes still drew as solid blobs (neighbours ~1.6 px apart at fit, nodes 2–9 px) under 23k fully drawn edges.Changes
LAYOUT_VERSION8 rebuilds oncegraph_layout.degreeandgraph_layout_edges(derived edges the pass laid out on)memory:graphreturns meta (world, epoch, stamp, spacing, counts, extent); newmemory:graph-tiles(screen-wide budget water-filled across tiles, edges deduped per request) andmemory:locateTest plan
bun run check,bun run lintbackend/memory/layout.test.ts(13 tests, stable over repeated runs): exact stability across passes, arrivals move nothing and land at their partners' centre, divergence recovery, re-cold keeps colours, Procrustes alignment, spacing and stamp, tile budget/folding/stale/dedupe/filter/locatebun run test: same pre-existing failures as a cleanHEAD(shared~/.clopen-teststate in messages/git tests)Breaking changes
WebSocket:
memory:graphnow returns view meta instead of the full graph; tiles come frommemory:graph-tiles. Only the bundled frontend consumes it.Migration
Migration 089 runs automatically. The first layout pass after upgrade rebuilds the arrangement from scratch (seconds for a few thousand memories).