Repository navigation
eager: marshal long strings as runs (5 MB bind: 204 → 65 ms) - #42
Merged
olydis merged 2 commits intoSep 28, 2026
Conversation
With skip_line a jet, binding the source is most of compile_file on a
file that is mostly comments: 121-163 ms for the 5 MB source's bind,
against 15 ms of reduction. Of that, growing the hash-consing table to
fit a cell per character (21 ms, 2^19 -> 2^23 slots) and inserting them
(71-97 ms, arena writes included) was the most, and decoding the text
into a vector first another 13-18.
list() now takes its heads from a callable, in order, and writes each
cell as it comes, at the high-water mark from the list's end, so
of_string decodes straight into the arena — one pass first to count the
characters and build their nats, since a head may not build a node
(list() checks). Cells that existed already are looked for as before,
dropped, and the first new one takes the old suffix for its tail. 2^16
or more new cells are a run: not inserted, because where a cell is says
what it is — the one cell that can be (u, v) is at v + 1, or the first
if v is the run's tail. intern() and list() look there after the table
(in_run, one well-predicted branch when there is no run), so
hash-consing stays exact; rebuild_interned() leaves the run's cells out,
collect() ends a run whose first cell it freed (every other cell reaches
it), and the next run puts the previous one's cells in the table.
compile_file, runner -s, ms, min / median of 5 interleaved:
bind reduce
5 MB source 121-163 -> 44 / 45 15 -> 15 / 16
1.2 MB source 32-43 -> 12 / 12 7 -> 7 / 7
The same with the comment and blank lines taken out: bind 1 ms, reduce
4 ms. Instructions where no string is long enough for a run
(compile_file of compiler.lamb with the base compiler): 920.3M ->
924.2M, and time, min / median of 3 interleaved:
the build's compile session 3.71 / 3.72 -> 3.57 / 3.64 s
loading the build's library 10.28 / 10.34 -> 10.33 / 10.56 s
its three heaviest tests 137.3 / 137.4 -> 136.2 / 137.1 s
(the runtime before these commits: 4.00 / 4.01, 10.65 / 10.70, 137.4 /
138.0 s). Outputs unchanged, the replay's and compile_file's.
Exactness re-verified: an audit build (every live node found by the
search intern() makes, no two live nodes of one shape, Merkle-shadowed
memo entries) after every command and after a forced collection, on the
5 MB and 1.2 MB compiles at the default budget and at 16 MB (a
collection with the run live), and on sessions binding a run, another,
the first again and suffixes of both; the differential fuzzer against
the base evaluator (list() against a fork chain, collections at a 4096
node budget, clear()) on 6 seeds at runs of 1 and of 3 cells and up,
30,000 operations each; the 164 sources byte for byte as before, the
lazy runner's included.
test-runner.sh: an 80,000-character string round-trips, and binding it
again, a suffix of it, and a reduction rebuilding one of its cells make
no node; without in_run in list(), or in intern(), those tests fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
The long-string test sends a payload that spells △, three bytes in UTF-8,
and gave its length as ${#rebuild}: characters under a UTF-8 locale, as on
CI's runner, so the payload came up short and its tail was read as a
command (err unknown command: ~r). LC_ALL=C makes every ${#…} in the
script a byte count, which is what the runner frames by. Reproduced with
LC_ALL=C.UTF-8 ./test-runner.sh; passes under both C and C.UTF-8 now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr
olydis
merged commit Sep 28, 2026
7557ccd
into
claude/tc-performance-optimizations-rqtyzl-jets
2 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Marshal a string of ≥ 2^16 characters as a "run": cells written straight into the arena, not inserted into the hash-consing table, and found where they lie (a cell (u, v) of a run can only be at v + 1). Hash-consing stays exact.
bindWith #38–#41 and the compiler of lambada-llc/lambada#41, the 5 MB source compiles whole in 82 ms (bind 65 + reduce 17), from 11,742 ms on main.
Stacked on #41.
runner -s(eager), min of 5 interleaved runs, 4-CPU VM; outputs byte-identical.🤖 Generated with Claude Code
https://claude.ai/code/session_018ffv5AybPTpV56D5vuayQr