Repository navigation
uorc v0.4: per-frame ncm profiles, learned context masks, strategy portfolio, wasm SIMD - #133
Merged
Merged
Conversation
…xt buckets, squash LUT, reused next-byte hashes A profile is a set of the 17 contexts; a left-out context is never looked up or adapted and would feed every mixer 0, so the payload is exactly the model with that input held at 0. The full profile is byte-identical to v0.3. Mixer rows are 16 wide (one cache line) when the profile has at most 13 contexts. Byte-identical for every profile: a context bucket's 16 slots pack probability and count into one 64-byte line, squash is a 4096-entry integer table, and the context hashes prefetch_next computes for the expected byte are reused when that byte comes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
…-gated Profile bit 17 adds a second match model keyed on the 4 bytes before a position (found from 4 agreeing bytes). Profile bit 18 adds a repeat model: the byte at the distance of the last match found keeps being predicted after a mismatch, with its run length as confidence, a hashed context of its expected byte, run and the previous byte (context 13 under this bit), and the match state that selects mixer 1 and the first APM. Profile 0x1ffff is the v0.3 coder byte for byte; both bits are 6.2% smaller on eight test files (3 MB of .rlib: 744,349 -> 674,978 B; xz -9e 685,552). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
…rofiled coder Profile word: bits 0-16 keep contexts, F_SHORT (17) and F_REP (18) the match features; V03 = 0x1ffff is the v0.3 coder byte for byte, FULL = 0x7ffff. Native equals lane 1's native for every feature combination and equals the IR (client/gen/ncm_model.py) for masked profiles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
…s start at weight 0 Bit 19 (F_SEL) now carries the selection of mixer 1 and the first APM by the two match models' states: it pays on long inputs (+0.6% without it on 9.5 MB) and costs on short ones (-1.0% without it on 150 files under 32 KiB). The short match and repeat inputs start at weight 0 so they only count once they predict. Profile 0x7ffff (bits 17, 18) is now 0.4% smaller than v0.3 on the short files; FULL is 0xfffff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
…ta family A strategy is a transform (none or the x86 filter) and an ncm profile (FULL, MID = FULL without match-state selection, V03). A frame's family (tiny, small, x86 code, text, other) orders every strategy by a bounded 1..16 weight; the encoder codes the first two of that schedule and keeps the smallest archive. The weights are learned by the rule30 rule (+2 winner, -1 others) over measured outcomes of 328 training files, committed with the test that re-learns them. On those files the budget-2 schedule is within 0.01% of trying all six strategies and 3.7% smaller than the v0.3 rule. Client.lex.tex states the same rule (portfolio/schedule/family) over Archive.profileCandidate; archive.rs carries interim profile forms until the profile templates land. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
…ep their meaning
The standard basis gains ncmRunProfile (the coder of client/gen/ncm_model.py
under a profile word) after every v0.3 block, which stay byte for byte, as
the v0.2 ones do. A profiled entry carries its profile word as a natural
constant after the payload. v0.3 archives name the v0.3 identity and decode
with their frozen suffixes; the native client, Archive.lex.tex and
NcmDecode/NcmEncode (runProfile) read every era.
Archive.profileCandidate and archive::{linked,window}_archive_profile are
what the strategy portfolio calls. inspect reports ncm_profile.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
…h profileCandidate Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
…to claude/project-thread-8f7s6w
A lone frame now compresses in about the time of one strategy (3 MB .rlib: 11.4 s -> 5.8 s on 4 cores; v0.3 5.4 s). The choice does not depend on the order, so the archives are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
… for LexLean Profile 0x1ffff is the v0.3 coder byte for byte, so profileCandidate (and the native linked/window profile archives) write the old-shape v0.3 entry over the v0.3 identity: byte-identical to v0.3 and without a profile word. Client.weight and its callers bind fam, since a binder spelled family would capture the declaration (LLT4001). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
A v0.3 stream's reference frame names its frame's κ-label over the v0.3 library. The decoder's store now offers each decoded frame under both labels (archive::frame_labels), so v0.3 streams with reference frames keep decoding under the appended library. Checked end to end: a 16 MiB A+A stream compressed by v0.3.0 decodes byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
The first scheduled strategy codes with learning on: every coded bit charges each kept context the cost (an integer log LUT, 256 log2(4096/p)) of the bit with that context's term left out of the three mixers, less the cost with it. The contexts that saved nothing are dropped, the learned profile is coded too, and that frame is kept when it is no larger: fewer contexts, faster decode, never a larger archive. Model: ncm_model encodeLearn (GN gains, COST table), NcmEncode.learnProfile, Archive.learnedCandidate, Client.portfolioLearned; native ncm::encode_learn, byte-identical to the model on the checked inputs. The decoder and the library are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
examples/portfolio_outcomes.rs regenerates src/portfolio-outcomes.tsv (one line per training file, sorted by name), from which the weight table is learned and checked. portfolio_streams_do_not_depend_on_threads codes frames of all five families with 1-4 threads and requires identical streams. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
…to claude/project-thread-8f7s6w
The wasm build enables simd128 (.cargo/config.toml) and ncm dispatches to a Simd128 kernel: 4 i32 lanes, products widened to i64 pairs as the scalar kernel computes them. In node (V8) on 1 MB samples, the archives are identical and decode runs 1.4-1.55x faster (text.rs 0.20 -> 0.28 MB/s, docs.tar 0.23 -> 0.36, rlib 0.24 -> 0.36). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
ncm::encode_below abandons a payload as soon as it reaches a caller's bound; a payload it finishes is the unbounded one. The portfolio bounds each strategy by the raw archive and, when strategies run one after another (--threads 1, a full batch, wasm), the second by the first's archive too; the two run side by side only when the batch leaves a core free, so --threads 1 uses one core. Archives are unchanged: all 332 training files give identical bytes. Single-thread CPU on 8 test files 80.9 -> 78.9 s, 2 MB random 15.6 -> 13.1 s; incompressible input cannot stop early without a guess. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
…to claude/project-thread-8f7s6w
…masks, portfolio) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
…to lane 2 ncm v4's mixers have NC + 7 = 24 inputs: Neuromorphic.mixerInputs is 24 and weightInitialAdded (0) records the short match and repeat inputs' start. The decoder's runs label a whole frame under the v0.3 library too, so v0.3 reference frames keep decoding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
Fixtures written by the release binaries for one input (8 MiB frame, the same frame, a 4 KiB frame): v0.2.0 ulz and ulz window frames; v0.3.0 ncm, a reference frame labelled over the v0.3 library and an ncm window frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
…0.4.0 The export identity is re-recorded under #131's closure rule (the exported client sources are unchanged). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY
afflom
pushed a commit
that referenced
this pull request
Nov 11, 2026
…hives New acceptance case decompress-released (feature decompressArchive): every archive in crates/uorc/tests/compat is restored in the browser to its original bytes. Measured against ncm v4 (#133): 18 cases pass; compress 178 KB/s, decompress 467 KB/s, worker memory 215 MiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk
afflom
pushed a commit
that referenced
this pull request
Nov 11, 2026
…nt landed in #133) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3PhocgkyQBi9JbhCuH
This was referenced Nov 11, 2026
afflom
added a commit
that referenced
this pull request
Nov 11, 2026
…h browser acceptance (#132) * Prism model: ncm's plasticity as the neuromorphic standard; Pages closure v2 Uorc.Neuromorphic (src/Uorc/Neuromorphic.lex.tex) states the plasticity of every learning part of the ncm coder as the Prism neuromorphic standard, in the codec's own integer (offset) form, with kernel-checked bounds: - hebbianBoundsHold: every mixer weight stays within +-2^22 under the bounded Hebbian rule; potentiation never lowers and depression never raises a weight. - plasticityBoundsHold: the plasticity 24 + 104*4096/(4096+pos) lies in [24, 128] per 65536 (closed vectors 128, 76, 25, 24 by decide). - laneFitHolds: |x * err * plasticity| <= 1,073,217,536 < 2^31, so the SIMD kernels' 32-bit lanes cannot overflow, and one bit changes a weight by at most 16,376. - slot plasticity: rates 131072/(2n+3) lie in [510, 43690], a slot never steps past its gap to the coded bit, and consolidates at count 127. - refinementBoundsHold: probability-map points stay within 16 bits. The products are proved through `scale`, the product by its recursion (scaleIsMultiply bridges it to `*`), with each bound carried in one induction, since omega has no nonlinear arithmetic. The native coder reads the model's constants from ncm::standard (byte identical output); tests/neuromorphic.rs holds them equal to the model's and the theorem inventory complete. Registered as UC-NEU-01 (kernel_theorem). The Pages site becomes a Prism browser closure v2 (PrismPM SPEC 1.2.1): index.html, app.css, app.js, uorc.js, uorc_bg.wasm, provenance.json and the ten Hologram brand files (fonts, licences, icons, manifest.webmanifest), flat and same-origin, installable, with no remote load. The brand files are vendored byte-identical to PrismPM's (vendor/hologram-brand). scripts/site.mjs assembles and checks the closure (provenance.json names every file's SHA-256); release.yml uses it and `just test` runs its tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3MmVJi8mhN3UZ6rgwH * Pages client: Prism product model of the page, generated from it, with browser acceptance src/Uorc/Page.lex.tex states the page in Prism's product-modeling vocabulary (Foundation.Arch stakeholders, features, journeys and UML state machine; Foundation.Qual requirements and measures; the text/3 workbench's operations, fields, examples, error hints, sections, usage, links and labels), registered as UC-WEB-01. repo-model checks it as PrismPM checks its own (PP2014-PP2017, one initial state, closed triggers, reachability, closed sets) and renders web/index.html from it; app.js interprets the embedded model. The codec runs in a module worker (uorc.js), with progress, Cancel, typed error hints and recovery from a stopped worker. In-memory decoding no longer spawns threads, which trapped WebAssembly on Verify, every example and every folder archive. The wasm client builds with SIMD128 (1.34x, same bytes). scripts/page-acceptance.mjs drives the built site in headless Chrome over the DevTools protocol: every journey, all 17 acceptance cases, all 11 transitions, controls checked against the model in every state, gated quality measures. CI runs it in pages-client.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Pages client acceptance is a factory obligation keyed on its own inputs The page acceptance runs as obligation `pages` (factory/graph.json): its identity covers the client set, the wasm crate, web/, the brand files and the two site scripts, so other changes find its receipt and skip it. The release gate needs it, and factory.yml records its receipts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * pages-client: read the single-obligation route outputs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Pages client: folders stream through the frame operations compressTreeBlob and decompressTreeBlob run a file tree a frame at a time (FrameEncoder::objects over the tree layout; decoded frames split at their object lengths), so a folder of any size never sits whole in wasm memory. The wasm test checks the streamed tree equals the library's compress_tree bytes and restores the same entries. The model's features name the new calls, and the acceptance checks that every uorc.* call a feature names exists in the shipped uorc.js. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Page acceptance: all-zero frames are reference frames again (#134) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * factory: the formal set holds web/index.html, a projection of the page model #131's legs run on a checkout materialized to their inputs, and the formal set excluded web/, so reproducibility found the check-model projection web/index.html missing (URP004). The page is rendered from src/Uorc/Page.lex.tex, so it belongs with the formal inputs; the rest of web/ stays out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Page acceptance: the page restores the released v0.2.0 and v0.3.0 archives New acceptance case decompress-released (feature decompressArchive): every archive in crates/uorc/tests/compat is restored in the browser to its original bytes. Measured against ncm v4 (#133): 18 cases pass; compress 178 KB/s, decompress 467 KB/s, worker memory 215 MiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: report /tmp use before PrismPM's native regeneration The rest legs fail with ENOSPC while PrismPM's template check builds the regeneration copy in the SDK container's 2 GiB /tmp; locally the copy and its build peak at 0.72 GB both at the last green revision and now. Print what holds /tmp at that point so the cause is measured, not guessed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: run PrismPM's template check before the xtask audit Measured in the rest leg: when PrismPM's native regeneration starts, /tmp (2 GiB tmpfs in the SDK container) already holds 1.2 GB, all of it /tmp/prismpm-cargo-1001, the debug xtask build audit-bootstrap leaves there. The regeneration copies the tree (0.13 GB) and builds it from scratch (0.72 GB at peak, measured in a 2 GiB tmpfs), so the lexlean rlib write hits ENOSPC. Running the template check first gives it an empty /tmp; both checks still run and must pass. Drops the /tmp report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: report /tmp, tree sizes and cargo environment before the template check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: build PrismPM's native regeneration in the workspace target Measured in the rest leg: the SDK container's /tmp is a 2 GiB tmpfs and CARGO_HOME (/tmp/prismpm-cargo-1001) already holds 1.1 GiB of it when the template check starts. Native regeneration copies the tree there (0.13 GB) and built it from scratch beside the copy (0.69 GiB at peak), which ran out of space writing the lexlean rlib. With CARGO_TARGET_DIR at the workspace's target on disk the build reuses audit-bootstrap's compiled dependencies, rebuilds the workspace crates from the staged sources, and /tmp holds only the copy (measured in a 2 GiB tmpfs: 0.13 GB, 4 s instead of 43 s). Restores the original order and drops the diagnostics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * model-write: build into the target the template check names, not beside the staged copy PrismPM runs model-write in a copy of the tree staged in the SDK's /tmp (a 2 GiB tmpfs of which CARGO_HOME holds 1.1 GiB) with a cleared environment that keeps only PATH, the toolchain homes, HOME and XDG_CACHE_HOME, so CARGO_TARGET_DIR (8032234) never reached it. The template check now leaves .tmp/model-write-target, which is copied with the tree, and model-write builds into that directory (the workspace's target on disk) when the note is there and into target otherwise. Simulated with the same cleared environment in a 2 GiB tmpfs: /tmp holds only the 0.13 GB copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: the staged regeneration builds in a target directory of its own On e6399ae the template check passed, but the next recipe (validate) ran the xtask binary the staged copy had built into the workspace's target: cargo took it as current, and it read sources at the copy's deleted /tmp path. The note now names target/regeneration, still on the workspace's disk and shared with nothing else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk --------- Co-authored-by: Claude <noreply@anthropic.com>
afflom
added a commit
that referenced
this pull request
Nov 11, 2026
…ounded decode memory (#140) * Prism model: ncm's plasticity as the neuromorphic standard; Pages closure v2 Uorc.Neuromorphic (src/Uorc/Neuromorphic.lex.tex) states the plasticity of every learning part of the ncm coder as the Prism neuromorphic standard, in the codec's own integer (offset) form, with kernel-checked bounds: - hebbianBoundsHold: every mixer weight stays within +-2^22 under the bounded Hebbian rule; potentiation never lowers and depression never raises a weight. - plasticityBoundsHold: the plasticity 24 + 104*4096/(4096+pos) lies in [24, 128] per 65536 (closed vectors 128, 76, 25, 24 by decide). - laneFitHolds: |x * err * plasticity| <= 1,073,217,536 < 2^31, so the SIMD kernels' 32-bit lanes cannot overflow, and one bit changes a weight by at most 16,376. - slot plasticity: rates 131072/(2n+3) lie in [510, 43690], a slot never steps past its gap to the coded bit, and consolidates at count 127. - refinementBoundsHold: probability-map points stay within 16 bits. The products are proved through `scale`, the product by its recursion (scaleIsMultiply bridges it to `*`), with each bound carried in one induction, since omega has no nonlinear arithmetic. The native coder reads the model's constants from ncm::standard (byte identical output); tests/neuromorphic.rs holds them equal to the model's and the theorem inventory complete. Registered as UC-NEU-01 (kernel_theorem). The Pages site becomes a Prism browser closure v2 (PrismPM SPEC 1.2.1): index.html, app.css, app.js, uorc.js, uorc_bg.wasm, provenance.json and the ten Hologram brand files (fonts, licences, icons, manifest.webmanifest), flat and same-origin, installable, with no remote load. The brand files are vendored byte-identical to PrismPM's (vendor/hologram-brand). scripts/site.mjs assembles and checks the closure (provenance.json names every file's SHA-256); release.yml uses it and `just test` runs its tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3MmVJi8mhN3UZ6rgwH * Pages client: Prism product model of the page, generated from it, with browser acceptance src/Uorc/Page.lex.tex states the page in Prism's product-modeling vocabulary (Foundation.Arch stakeholders, features, journeys and UML state machine; Foundation.Qual requirements and measures; the text/3 workbench's operations, fields, examples, error hints, sections, usage, links and labels), registered as UC-WEB-01. repo-model checks it as PrismPM checks its own (PP2014-PP2017, one initial state, closed triggers, reachability, closed sets) and renders web/index.html from it; app.js interprets the embedded model. The codec runs in a module worker (uorc.js), with progress, Cancel, typed error hints and recovery from a stopped worker. In-memory decoding no longer spawns threads, which trapped WebAssembly on Verify, every example and every folder archive. The wasm client builds with SIMD128 (1.34x, same bytes). scripts/page-acceptance.mjs drives the built site in headless Chrome over the DevTools protocol: every journey, all 17 acceptance cases, all 11 transitions, controls checked against the model in every state, gated quality measures. CI runs it in pages-client.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Pages client acceptance is a factory obligation keyed on its own inputs The page acceptance runs as obligation `pages` (factory/graph.json): its identity covers the client set, the wasm crate, web/, the brand files and the two site scripts, so other changes find its receipt and skip it. The release gate needs it, and factory.yml records its receipts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * pages-client: read the single-obligation route outputs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Pages client: folders stream through the frame operations compressTreeBlob and decompressTreeBlob run a file tree a frame at a time (FrameEncoder::objects over the tree layout; decoded frames split at their object lengths), so a folder of any size never sits whole in wasm memory. The wasm test checks the streamed tree equals the library's compress_tree bytes and restores the same entries. The model's features name the new calls, and the acceptance checks that every uorc.* call a feature names exists in the shipped uorc.js. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Page acceptance: all-zero frames are reference frames again (#134) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * factory: the formal set holds web/index.html, a projection of the page model #131's legs run on a checkout materialized to their inputs, and the formal set excluded web/, so reproducibility found the check-model projection web/index.html missing (URP004). The page is rendered from src/Uorc/Page.lex.tex, so it belongs with the formal inputs; the rest of web/ stays out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Page acceptance: the page restores the released v0.2.0 and v0.3.0 archives New acceptance case decompress-released (feature decompressArchive): every archive in crates/uorc/tests/compat is restored in the browser to its original bytes. Measured against ncm v4 (#133): 18 cases pass; compress 178 KB/s, decompress 467 KB/s, worker memory 215 MiB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: report /tmp use before PrismPM's native regeneration The rest legs fail with ENOSPC while PrismPM's template check builds the regeneration copy in the SDK container's 2 GiB /tmp; locally the copy and its build peak at 0.72 GB both at the last green revision and now. Print what holds /tmp at that point so the cause is measured, not guessed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: run PrismPM's template check before the xtask audit Measured in the rest leg: when PrismPM's native regeneration starts, /tmp (2 GiB tmpfs in the SDK container) already holds 1.2 GB, all of it /tmp/prismpm-cargo-1001, the debug xtask build audit-bootstrap leaves there. The regeneration copies the tree (0.13 GB) and builds it from scratch (0.72 GB at peak, measured in a 2 GiB tmpfs), so the lexlean rlib write hits ENOSPC. Running the template check first gives it an empty /tmp; both checks still run and must pass. Drops the /tmp report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: report /tmp, tree sizes and cargo environment before the template check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: build PrismPM's native regeneration in the workspace target Measured in the rest leg: the SDK container's /tmp is a 2 GiB tmpfs and CARGO_HOME (/tmp/prismpm-cargo-1001) already holds 1.1 GiB of it when the template check starts. Native regeneration copies the tree there (0.13 GB) and built it from scratch beside the copy (0.69 GiB at peak), which ran out of space writing the lexlean rlib. With CARGO_TARGET_DIR at the workspace's target on disk the build reuses audit-bootstrap's compiled dependencies, rebuilds the workspace crates from the staged sources, and /tmp holds only the copy (measured in a 2 GiB tmpfs: 0.13 GB, 4 s instead of 43 s). Restores the original order and drops the diagnostics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * model-write: build into the target the template check names, not beside the staged copy PrismPM runs model-write in a copy of the tree staged in the SDK's /tmp (a 2 GiB tmpfs of which CARGO_HOME holds 1.1 GiB) with a cleared environment that keeps only PATH, the toolchain homes, HOME and XDG_CACHE_HOME, so CARGO_TARGET_DIR (8032234) never reached it. The template check now leaves .tmp/model-write-target, which is copied with the tree, and model-write builds into that directory (the workspace's target on disk) when the note is there and into target otherwise. Simulated with the same cleared environment in a 2 GiB tmpfs: /tmp holds only the 0.13 GB copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * template-check: the staged regeneration builds in a target directory of its own On e6399ae the template check passed, but the next recipe (validate) ran the xtask binary the staged copy had built into the workspace's target: cargo took it as current, and it read sources at the copy's deleted /tmp path. The note now names target/regeneration, still on the workspace's disk and shared with nothing else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * uorc: code a planned frame one strategy at a time, with a progress tick The portfolio's strategy coding is lifted out of compress_pieces as compress_strategy and choose (same bounds, same tie rule, same raw fallback), and FrameEncoder gains plan/plan_end, which cut frames and finish repeats without coding the rest (Planned::code codes one). The wasm crate exports plan_frame, plan_end, code_frame (one strategy or the whole frame) and choose_frame, so the page can code frames and strategies on several workers and write the library's bytes; a test checks that stream equals uorc::compress. ncm calls a thread-local progress hook once per 64 KiB of each coding or decoding pass; the wasm crate forwards it to the page's uorc.progress import. All 64 uorc tests pass unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Pages: a worker pool on the visitor's cores, progress within a frame The page's worker plans each frame with the library and a pool of workers (one per core, at most 16, two per GiB of device memory, one shared module) codes the frame's two portfolio strategies side by side; the library's rule keeps one, so archives equal uorc::compress. ncm reports progress every 64 KiB, so the bar moves within an 8 MiB frame. The page model adds requirements useTheCores and progressWithinFrame and gated measures coresUnused (0) and progressGap (at most 2 s), and states the throughput before and after on the same 10 MiB (4 vCPU, headless Chromium): compress 168 -> 369 KB/s, decompress 563 -> 561 KB/s (every frame of that input is a window frame, so decoding stays sequential, as natively). Acceptance: 18 cases pass, 10 measures within their gates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * ncm: decode a window frame without copying the window, and keep only the output The archive layer joined the window's frames into one buffer and the model copied that buffer again into its history. The model now takes the history as pieces and copies the frames once, and the decoded output gives back the history's share of its buffer (the next frame's window holds the output). Same bytes: the uorc tests pass. Measured in Node (WebAssembly, SIMD128), streams of distinct 8 MiB frames: decode memory 241/257/257 MiB before for 3/6/12 frames, 237/237 MiB after for 3/6 frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk * Pages: bound decode memory across frames, and keep the progress bar moving ncm: in WebAssembly, whose memory never shrinks, a model's tables (about 100 MiB for an 8 MiB frame) are kept for the next model of the same size instead of being allocated again; that had left the allocator's free space in pieces and memory grew with the frame count. Measured in Node on distinct 8 MiB frames: decode 226 / 250 / 250 MiB for 3 / 6 / 12 frames (241 / 257 / 257 before). Same bytes. Progress (gate progressGap, 2 s): the acceptance found two stalls over 2 s. The page's worker now codes alone until two pool workers have loaded, so the first file does not wait on their start; a short last frame, coded when the stream ends, now reports its passes; ncm reports every 16 KiB; and a frame's share keeps moving through passes beyond the usual three. Longest stall now 552 ms; 18 cases pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Asa3N6Mcveqrwnu6XBiYk --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Alex · project thread
This PR carries v0.4 lanes 1 and 2, the decode-cost and ratio lanes, through a single export. Lane 1's commits on claude/project-thread-2twygm are merged in.
What changes
Profile word per frame (lane 2).
prefix uleb(len) payload 00 uleb(profile) tail.9272ab03….Earlier releases keep their meaning.
client/gen/v03/, as an Era (Current/V03/V02) in the model and natively.chunk::runs,archive::frame_labels) label a whole frame under both libraries.Learned context masks (lane 2).
256 log2(4096/p), computed by normalisation and squaring.ncm_model.encodeLearn(GN/COST tables),NcmEncode.learnProfile,Archive.learnedCandidate,Client.portfolioLearned. Native:ncm::encode_learn.Strategy portfolio (lane 1).
Byte-identical speedups, R4-style integer LUTs.
.cargo/config.tomlenables simd128 for wasm32.Neuromorphic standard (#129).
mixerInputsis 24 andweightInitialAddedis 0. No bound depends on the input count.Release.
generated/rust/factory.identity(b84a73b1…).native_archives_are_the_model_archivesand the window model tests pass against it.Backward decode: which test covers what
All fixtures in
crates/uorc/tests/compat/were written by the v0.2.0 and v0.3.0 release binaries.archive::tests::earlier_releases_decode(text-v0.2.0.uorc)earlier_releases_decode(text-v0.2.0-ncm.uorc)earlier_releases_decode(x86-v0.2.0.uorc)tests::released_streams_decode(stream-v0.2.0.uorc)earlier_releases_decode(text-v0.3.0.uorc)earlier_releases_decode(x86-v0.3.0.uorc)released_streams_decode(stream-v0.3.0.uorc)released_streams_decode(stream-v0.3.0.uorc)Both streams code the same input: an 8 MiB frame, the same frame again, then a 4 KiB slice of it.
Measured
Lane 1 net, 8 files vs v0.3.0: -7.8% bytes and -3.5% single-thread decode CPU. On the 3 MB .rlib concatenation: 676,337 B (v0.3: 744,443; xz -9e: 685,552).
Full sample files vs v0.3.0. Single-thread decode, release binaries, 4 vCPU VM shared with the export, so wall times are noisy:
--threads 1it now really uses one core.Checks run locally
cargo test --release -p uorc: all lib and integration tests pass against the export, including neuromorphic.cargo clippy --testsis clean, natively and for wasm32.lexlean checkof the client closure passes (77 modules), and of Neuromorphic.gen_basis.py --check: every profiled codec decodes on the graph mirror.client/gen/build.shregenerates byte-identically.🤖 Generated with Claude Code
https://claude.ai/code/session_013Asa3Mtk8bNt1Y8AGfBfPY