Skip to content

perf(runtime): matching ANode views the block instead of copying it (draft) - #2019

Draft
nicolas-abril wants to merge 1 commit into
mainfrom
ai/1991-array-views
Draft

nicolas-abril wants to merge 1 commit into
mainfrom
ai/1991-array-views

Conversation

@nicolas-abril

Copy link
Copy Markdown
Collaborator

Refs #1991. A draft to look at, not ready to merge: comp.ts is 713 tokens over its cap, so gates/repo.ts fails, and no perf bench uses array splits yet. This doesn't fix #1991 either; see the numbers below.

Matching ANode copies both halves today. Views were removed in 64adb37db because a block changed size class on every split and leaked (#804). This brings them back on the host and merges the pieces when they're freed, so nothing leaks. The device still copies.

How it works

  • Views: blk_half returns a view of the same block. Its aux has bit 5 set and the root block's span in bits 6-10.
  • Merging: freeing a view flips its parent's midpoint bit in a global bitmap (VIEW_BITS). The second sibling to die climbs a level, and the last one frees the root. This is why large blocks must be aligned to their own size.
  • Alignment: heap_alloc_miss takes only the gap it needs, with a CAS on the bump pointer. Only large host blocks use the CAS. Using it on every miss made GPU time much worse (merkle 3.16x).
  • Bounded retention: a piece more than 16x smaller than its root is copied into its own block, except under 8 words, where the copy costs more than it saves.
  • Inlining: blk_half is forced inline and its copy path moved out to blk_part. Otherwise clang stops inlining it and loses the constant arr and hi.

Results

All local on an ARM Mac, one thread, best of 3. main is 017f83fd, which differs from this branch's base only in bend.ts.

Program main this PR
#1991 repro, Array.to_list 0.52 s, 10.14 B instr, 29.7 MB 0.35 s, 9.49 B instr, 21.8 MB
#1991 repro, Array.get loop 0.12 s, 2.27 B instr, 21.4 MB 0.12 s, 2.27 B instr, 21.5 MB
Walk 2^16 Array<U32> by matching, 1600 times 0.53 s 0.45 s
Split 2^16 once, 16000 times 0.10 s 0.03 s
Rebuild 2^16 by matching (map), 400 times 0.37 s 0.32 s
Small arrays, cycles 0.21 B 0.24 B
Slice kept 12 levels down, 250 arrays 2.3 MB 1.8 MB
Slice kept 4 levels down, 250 arrays 6.3 MB 65 MB
bfs peak memory, 8 threads 1.91 MB 1.91 MB

On the #1991 repro Array.to_list gets about 31% faster but still takes 2.8x the time and 4.2x the instructions of the Array.get loop, so a flat walk is still the direct fix for that issue.

Gates

  • gates/test.ts: 1819/1819 on this branch. It can't see leaks: the old leaking views passed it too. No test is added here.
  • gates/repo.ts: fails, bend2/comp.ts: 66713 > 66000 ttok.
  • gates/perf.ts, 5 rounds interleaved with main, run on 017f83fd: CPU time 1.000-1.003x, GPU time 0.995x, memory within 0.2%, compile time 1.035x. It passed 124/124 three times and 123/124 twice, against main's five clean passes. The miss is merkle compile at about 1.16x against the 1.15x limit, and main reaches 1.15x there too.

Known costs

  • Tokens: 713 over a permanent cap.
  • Small arrays: about 14% more cycles and 8% more instructions, not yet explained.
  • Retention: a slice kept 4 levels down or fewer pins its whole block, up to 16x its own size. Main doesn't have this.
  • Compile time: tree-radix goes from 0.48 s to 0.68 s, not yet explained; it still passes its pin. Without it the compile geomean is about 1.015x.
  • Startup: every binary reserves 128 GB of virtual memory for the bitmap with MAP_NORESERVE and aborts if the mapping fails. This has only run on macOS, so a Linux box with strict overcommit is a real question.

🤖 Generated with Claude Code

blk_half returns a view on the host. Freed views merge through midpoint
bits and the last one frees the root, so splits no longer leak. Large
blocks are aligned to their size for that, and a piece over 16x smaller
than its root is copied so a slice can't pin a large block unbounded.

comp.ts is over its ttok cap: this is a draft.

Refs #1991

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

C backend: Base's Array.to_list copies both halves at every level, ~190 instructions a cell (a loop of Array.get takes ~35)

1 participant