Repository navigation
eager: 2.1–2.5× faster reduction, same answers - #39
Merged
Conversation
A continuation frame was four 4-byte fields: pushed through a stack temporary GCC filled field by field and then copied as one 16-byte load, and popped as one 16-byte (or 8-byte) load of what four 4-byte stores had just written. Neither load can be forwarded from the store buffer, so each waited for the stores to retire — and a frame is usually popped a few instructions after it is pushed (a MEMOIZE under a step that answers outright). perf put a fifth of apply()'s samples on the instructions right after those loads. The frame is now two 8-byte words, built in registers, constructed in place and read back at the width it was written. Same fields, same semantics. compile_file, reduce ms (runner-stats), min of 3, base memo policy: source before after 1.26 MB forest file 1439 908 (-37%) compiler.lamb, 66 KB 701 448 (-36%) 5 MB forest file (min of 1) 6845 4560 (-33%) Output sha unchanged on all three (whole and code-only); test.sh passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
The MEMOIZE frame's `meta` was never set (the pushes left it 0), so the
"skip steps that resolved in under 16 rules" filter compared the whole
step count against 16 and filtered nothing: every recursing step was
memoized. The frames now carry the step count they were pushed at.
Switching the filter on at 16, as documented, doubles the steps of
compile_file on the 5 MB source (47.6M -> 95.1M): a character-class test
is a few rules and is asked of every character, and its hits were the
point. Swept on both workloads, on this machine, one run at a time:
MEMO_MIN_STEPS replay of the arboretum compile_file reduce ms
build's runner sessions 5 MB 1.26 MB 66 KB
off (as before) 159.4 s 2637M steps 3891 723 382
2 158.9 s 2628M 3992 785 375
4 152.6 s 2658M 3456 666 367
8 152.4 s 2737M 3642 717 408
16 (3006M steps; 5 MB file: 95.1M steps vs 47.6M)
4 is best or tied on every column. Replay output identical throughout.
Memo size was swept the same way and is left budget-scaled (2^23 slots at
the default 512 MB budget): 2^24 cut steps (2495M) but not time (154.8 s,
and 4356 ms on the 5 MB file); 2^22 cost 2972M steps and 177.7 s; at 2^16
the build's heaviest tests did not finish in 15 min, although that size is
20-30% faster on a single compile_file. Letting the memo grow from 2^16 on
evidence (a sampled shadow of a full-size table) kept that gain on single
files but not on the build's compile session (4.0 -> 4.75 s, the memo is
full-size by its 22nd source), so it is not included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
Every step hashes a pair of indices before it can load anything — the
hash-consing slot of the node it builds, the memo slot of the redex it
looks up — so the hash's latency is paid in series with the cache miss it
leads to. The 64-bit Murmur finalizer is five dependent operations, two
of them multiplies; Fibonacci hashing is one multiply, with the high half
folded onto the low bits the tables index by.
Probe lengths do not suffer (mean slots examined per hash-consing search:
1.26 MB source 2.72 -> 2.68, compiler.lamb 2.56 -> 2.41, three heaviest
build tests 2.03 -> 1.96), and neither do memo hits.
replay of the arboretum build's runner sessions, back to back:
152.4 s -> 140.8 s (-8%)
compile_file reduce ms, 5 interleaved runs (3 for 5 MB), min / median:
1.26 MB file 744 / 823 -> 683 / 778
compiler.lamb 384 / 408 -> 363 / 389
5 MB file 3979 / 4255 -> 3821 / 3836
Replay output identical (3b8f647b617e24b3); benchmark shas unchanged;
test.sh passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
Most memo lookups are for functions whose applications never recur — a
loop applied to each suffix of its input, a closure over one position.
Counted per function (the `a` of apply(a, b)) on compile_file:
source lookups by functions that never hit share of hits
5 MB forest 25.9M 74.9% 0.00%
compiler.lamb 3.8M 69.5% 0.01%
Each such lookup is a cache miss into a table sized for a module build,
and each is followed by a MEMOIZE frame whose write, when the step is
done, evicts an entry that might have hit. So a small per-function table
(64K one-byte counters, cache-resident) counts consecutive misses, and a
function that has missed MEMO_COLD = 64 times in a row is neither looked
up nor recorded; one lookup in sixteen for a cold function is still made,
and a hit warms it. Reset whenever the memo is re-sized or collected
(swept indices are reused by other functions). The memo stays exact: this
only decides what is asked of it.
Effect on the 5 MB source: 15.9M memo writes -> 1.2M, hits unchanged
(6.33M), steps unchanged (47.62M).
Threshold swept on the build replay (steps / time):
off 2651M / 136.5 s, 32: 2842M / 125.5 s, 64: 2759M / 122.3 s,
127: 2700M / 123.3 s; 64 was also best on single files.
replay of the arboretum build's runner sessions, back to back:
136.5 s -> 125.3 s (steps 2651M -> 2711M)
compile session alone, 4 interleaved runs: 3.78-4.38 s -> 3.70-3.93 s
compile_file reduce ms, interleaved, min / median:
1.26 MB file (5) 628 / 1159 -> 360 / 522
compiler.lamb (5) 343 / 359 -> 296 / 312
5 MB file (3) 3475 / 4113 -> 2272 / 2353
Replay output identical (3b8f647b617e24b3); benchmark shas unchanged;
test.sh passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
A node is made after its children, so nothing can have the node made
last as a child yet: a fork or stem over it cannot already exist. That
is list()'s argument a cell at a time, and it holds for whatever a
reduction builds on what it just built — counted per source, 26% (5 MB
file) to 49% (compiler.lamb, the build's heaviest tests) of the nodes a
reduction interns. Those skip the search and just take a slot; with no
search to wait on, the table write's cache miss overlaps with the steps
that follow instead of stalling them. `_newest` is reset by a sweep (the
node may have been freed) and by clear().
Exactness checked directly after each benchmark run and the replayed
compile session, heavy tests (one collection) and repeated binds: every
live node's (u, v) unique and found by the table.
replay of the arboretum build's runner sessions, back to back:
121.2 s -> 115.2 s, 116.0 s (steps unchanged, 2711M)
compile session alone: 3.04 s -> 2.69 s
three heaviest build tests, 2 interleaved runs: 30.4-33.2 s -> 28.8-29.4 s
compile_file reduce ms, interleaved, min / median:
compiler.lamb (5) 266 / 273 -> 258 / 265
5 MB file (3) 2208 / 2245 -> 2065 / 2234
1.26 MB file (5) 360 / 378 -> 360 / 382
Replay output identical (3b8f647b617e24b3); benchmark shas unchanged;
test.sh passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
Transparent huge pages (MADV_HUGEPAGE on the arena and both tables) were
measured and are not taken. Back to back they win — the build replay of its
recorded runner sessions 142.2 -> 125.0 s, 12% — because every large region
here is read at random and 4 KiB pages leave the TLB covering a few MB of
hundreds. Started from cold memory they lose, and on this VM that is the usual
start: it reports free 2 MiB blocks to its host (virtio-balloon free page
reporting, order 9), the host drops them, and a huge-page fault is served from
exactly those blocks, while 4 KiB faults still find warm fragments. First touch
of 96 MB, alone: 4 KiB pages 49-70 ms; huge pages 16-18 ms right after another
huge-page process exited, 342 ms otherwise. So a run's cost hung on what ran in
the seconds before it; 6 of 15 bare starts stalled 0.2-0.65 s (none at 4 KiB
pages: 113-131 ms).
compile_file, load + bind + reduce ms, median of 3, each run after 4 s idle:
huge pages -> 4 KiB
5 MB file 4498 -> 3243
1.26 MB file 1226 -> 694
compiler.lamb 930 -> 534
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
Both tables are now mapped once, at the most they can reach (2^32 slots for the hash-consing table, MAX_MEMO for the memo), and re-laid in the memory they already have, as the arena always was. Growing a vector-backed table mapped a fresh copy and filled it: every page faulted in again, with the old copy still held. rebuild_interned() reads the arena, not the old table, so nothing needs that copy. It is also what lets the memo grow in place (next commit). Peak RSS: compile_file of the 5 MB file 474 -> 444 MB, 1.26 MB 205 -> 188 MB, compiler.lamb 151 -> 137 MB; the build's compile session 413 -> 344 MB (minor faults 125K -> 89K), its test session 1258 -> 1134 MB (425K -> 293K). Time, compile_file load + bind + reduce ms, median of 3 cold (4 s idle before each) / 4 back to back: 5 MB file 3243 -> 3118 / 3174 -> 3156 1.26 MB file 694 -> 671 / 695 -> 677 compiler.lamb 534 -> 499 / 507 -> 515 The build's test session (the replay's long runner), median of 3 interleaved: 137.1 -> 138.4 s, within its noise. Outputs unchanged (5 MB sha 92c528b6bcf22b16 whole and code-only; replay 3b8f647b617e24b3); test.sh and the C++ suite (test.cpp) pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
size_memo() laid the memo out at its bound up front: 2^23 slots at the default budget, 96 MB written before the first request, which was most of a runner's start-up and all of it paid again by every process however little it went on to reduce. The memo now starts at MIN_TABLE slots and doubles in place once it has taken as many entries as it has slots, up to the same bound. Doubling keeps every entry: at twice the size an entry hashes to its own slot or to the one `half` above it, in the half being added. Start plus loading compile_file.dag: 114-163 -> 34-41 ms. Peak RSS: compile_file of the 5 MB file 444 -> 360 MB, 1.26 MB 188 -> 95 MB, compiler.lamb 137 -> 53 MB; the build's compile session 344 -> 296 MB. compile_file, load + bind + reduce ms, median of 3 cold (4 s idle before each) / 4 back to back: 5 MB file 3118 -> 3047 / 3156 -> 2902 1.26 MB file 671 -> 595 / 677 -> 579 compiler.lamb 499 -> 346 / 515 -> 356 The build's three heaviest tests, which do fill the memo to its bound: min of 3 interleaved 34.25 -> 34.16 s, steps 525.9M -> 529.4M (the memo is smaller while it grows). Replay of the build's sessions, two rounds: 148.8 / 144.7 -> 148.2 / 146.9 s. Outputs unchanged (5 MB sha 92c528b6bcf22b16 whole and code-only; replay 3b8f647b617e24b3); test.sh and the C++ suite pass, and an audited build of this commit's successor (every memo hit and entry checked against a Merkle hash of its nodes, entries checked to sit in their slot after growing) ran the whole replay at a 64 MB budget and the heaviest tests at 16 MB (959 collections) without a finding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
recall() made one lookup in sixteen for a cold function by a single counter shared by all of them, so a function looked up between the same even number of other cold lookups each time kept landing on the same residue: it could go unsampled for good, never be recorded, never hit, never warm up again. Now the count is the function's own — its byte of _cold, past MEMO_COLD — so every sixteenth of its own lookups is made. One member less. Checked with a scratch harness that makes a function cold, then looks it up on one fixed argument each round between k lookups of other cold functions, completing each sampled miss the way apply() does: with the shared counter it never hit again in 8, 12, 14 and 15 of 16 starting phases for k = 1, 3, 7, 15; with its own, in none. What a cold function pays once it does need the memo stays linear: a recursion that calls itself twice on one argument (2^n steps without the memo) takes, for n = 1000 / 2000, 21367 / 42367 steps with every lookup made, 43186 / 96034 before this commit, 46861 / 89855 after. Time: compile_file load + bind + reduce ms, median of 3 cold / 4 back to back: 5 MB 3047 -> 3056 / 2902 -> 3054, 1.26 MB 595 -> 637 / 579 -> 542, compiler.lamb 346 -> 338 / 356 -> 362 — within their noise. Where the memo matters most it costs a little: the build's three heaviest tests 34.16 -> 35.43 s (min of 3; steps 529.4M -> 532.8M), its test session 138.4 s two commits back and 139.3 s here (median of 3), replay steps 2709.8M -> 2720.9M. Outputs unchanged (5 MB sha 92c528b6bcf22b16 whole and code-only; replay 3b8f647b617e24b3); test.sh passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
…rops The header still said every step that recurses is memoized; it now names the two exceptions, MEMO_MIN_STEPS and recall()'s cold functions. hash() now says what its fold leaves out: bit 32 + j of the product depends on bits 0..j of u only, so in a table of 2^k slots (u, v) and (u + 2^k, v) always share a slot — rare in the hash-consing table (sized to the live nodes; probes average 2.3-2.8 slots a search on the benchmark sources), a standing eviction pair in the memo. It stays, measured: folding a 128-bit product instead keeps every bit and took 0.15% fewer steps on the build's three heaviest tests (530.1M against 530.9M), but 35.9 s against 35.4 s. Comments only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
The collector, the hash-consing table's re-lay and the memo's growth were all inlined into apply(), which made it 1,558 instructions long — and which of its small helpers the compiler still inlined then hung on any edit to the file: a variant of a later change (marshalling long strings as runs) that happened to leave intern() out of line took 1,070M instructions on the benchmark below; that change, on top of this one, takes 924M. collect(), rebuild_interned() and grow_memo() are now [[gnu::noinline]]; apply() is 910 instructions, and what a change to the hot path costs can be measured again. Instructions (cachegrind, compile_file of lambada's compiler.lamb with the base compiler, 9.67M steps): 937.1M -> 919.2M. Time, min / median of 3 interleaved runs, no change beyond noise: the build's compile session (61M steps) 3.71 / 3.80 -> 3.78 / 3.80 s loading the build's library (158M steps) 10.01 / 10.38 -> 10.00 / 10.07 s its three heaviest tests (2.56G steps) 136.5 / 140.0 -> 134.2 / 135.4 s Outputs unchanged (replay 5556b28b64d74f28 and 6856a56e08e6ffec). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
This was referenced Sep 28, 2026
olydis
merged commit Sep 28, 2026
0d2342b
into
claude/tc-performance-optimizations-rqtyzl-marshal
1 check passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eager evaluator: 2.1–2.5× faster reduction, same answers. arboretum full build 449 → 270 s.
compiler.lamb(66 KB)MEMO_MIN_STEPSactually filters (metawas never set), 4 by sweepnoinlineStacked on #38 (marshalling).
runner -s(eager), min of 5 interleaved runs, 4-CPU VM; outputs byte-identical to main. Build: arboretum main./build.shfrom empty caches, 2 interleaved runs each (449 / 465 → 270 / 292 s),git statusclean after.🤖 Generated with Claude Code
https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
Generated by Claude Code