Skip to content

[v0.4] #27 stage 8: the production compiler authored in .lex.tex and compiled by LexLean to production software #92

Open

Description

@afflom

Child of #27; plan in this comment, re-scoped by the maintainers' decision of 2026-10-14 (#27 comment). Delivery stage 8, the last.

Decisions (made by the maintainers)

  • The concern is LexLean → production software. Lean 4 verifies LexLean and has nothing to do with the production environment. The earlier recommendation on this issue (the compiler's execution engine is generated Lean driven by a typed host adapter) is withdrawn: the compiler is a LexLean program compiled by LexLean's own production pipeline to production Rust, and that Rust is the compiler that runs.
  • .lex.tex is the only authorized authoring surface of LexLean. The compiler's logic is committed as .lex.tex; Rust builders, Rust modules and JSON generated by tools are not sources.
  • "Production IR" is the semantic IR (the maintainers did not object to the recommendation, and it is what the pipeline consumes today); to be confirmed in the first sub-stage.

Scope

The production pipeline is Rust: production/ (23,522 lines: certificate.rs 8,796, lower.rs 3,682, eligibility.rs 3,350, rust_cert.rs 2,348, preserve.rs 1,725, lcnf.rs 1,727, source.rs 893, mod.rs 626, rust_term.rs 375) and the target calculus calculus/ (about 11k lines with calculus/rust/): eligibility, lowering, certificate generation, the Rust AST, lowering, validation, printing, packaging, and the compiler's own front end. After this issue that logic is authored in language 1.2 .lex.tex, certified as a production root, and rendered to a Rust package by the compiler it implements.

Self-hosting acceptance (the bootstrap)

  1. Stage 0: today's Rust compiler (the oracle, kept under tests/ until the fixed point).
  2. Stage 1: stage 0 compiles the .lex.tex compiler to a certified Rust package; its provenance binds the source, certificates A, B, E and the runtime.
  3. Stage 2: stage 1 compiles the same .lex.tex compiler.
  4. Fixed point: stage 2's package is byte-identical to stage 1's, and stage 1 equals stage 0 on every committed project, corpus, fixture and golden (differential, byte for byte), and on the Atlas verification records. Only then are the Rust modules removed.

What it needs (inventory, to be refined in the first sub-stage)

Proposed order inside this stage

  1. Boundary and effects design (a short design issue, then implementation); the capability set.
  2. Library needs as stdlib members: strings, bytes, maps, sorting, JSON, SHA-256.
  3. Pilot: the preservation lexer and token audit (preserve.rs), and the axiom-audit parser: small, closed, with a differential against the oracle.
  4. calculus::check and rust::{lower, validate, print, package}.
  5. The production registry, eligibility, source and lowering stages.
  6. Certificates (after [v0.4] #27 stage 5: certificates A, B, E and the audit modules as typed data, with no Lean text template in the production compiler #89).
  7. Front end (lexicon, grammar, linking), then the fixed point and removal of the Rust modules.

DoD (whole stage)

  • The production compiler's logic exists only as .lex.tex; the Rust of those modules is absent from the release tree.
  • The fixed point above holds on the normative host, and the compiler that ran just vv is the one LexLean built.
  • The differential against stage 0 is byte-identical on every committed root and corpus before removal.
  • Lean verifies the compiler's certificates; no Lean text is produced by or consulted at compile time of production output.

Acceptance criteria

Per sub-stage: equivalence evidence, planted defects turn named gates red, languages 1.0/1.1 byte-identical, 1.2 identities regenerated, full just vv.

Activity

  1. added this to the LexLean v0.4 milestone on Oct 14, 2026
  2. changed the title [-][v0.4] #27 stage 6: the production compiler's logic authored in language 1.2[/-] [+][v0.4] #27 stage 8: the production compiler authored in .lex.tex and compiled by LexLean to production software[/+] on Oct 14, 2026
  3. afflom commented on Oct 15, 2026

    @afflom
    MemberAuthor

    #92 spike: what a compiler component needs today, and the boundary/effects design

    Measured on main at 2fd1c1e9 (after #103), with the release binary, in a scratch project (not committed). The question the spike answers: can a pure piece of this compiler be authored in .lex.tex, verified by Lean, compiled by LexLean's own production pipeline to Rust, and agree with the Rust it replaces? The answer is yes for byte-level algorithms today. The rest is the list of what is missing, with an owner for each item.

    1. The pilot

    Three production roots in one language-1.2 module, authored in the readable notation (no JSON, no Rust):

    root replaces shape
    isClosedLeanName(raw : Bytes) : Bool ir::core::validate_name (ASCII Lean-name grammar; used by the axiom-output parser, §22.5) a 3-state byte machine, iterate_until over raw
    splitOnByte(subject : Bytes, delimiter : UInt8) : List Bytes str::split / the line splitter iterate with (position, start, pieces) state, list_fold to reverse
    startsWith(subject, prefix : Bytes) : Bool starts_with / strip_prefix iterate over the prefix

    Results, all with the pinned Lean 4.32.1:

    • lexlean check: eligible for rust-std with effects: [allocation, overflow]. rust-core is refused with LLT4005 ("type.bytes requires heap allocation"), so a compiler is a rust-std program.
    • lexlean verify: 16 theorems by reflexivity{} (accepts/rejects of names, splits, prefixes) pass the kernel; the axiom audit is exact per declaration (Quot.sound, propext for the UInt8 range checks and what depends on them, none for the rest).
    • lexlean production build: 3 certified Rust packages (certificates A, B, E accepted). Wall clock 3m25s for one root and 4m30s for three, almost all Lean.
    • Differential against plain Rust, on the generated crates: 1,000,000 random names against a hand-written validate_name (alphabet biased to the grammar's boundaries, plus every byte in first and later position), 1,000,000 splits against slice::split, 1,000,000 prefix tests against starts_with: no disagreement. This is the shape of evidence [v0.4] #27 stage 8: the production compiler authored in .lex.tex and compiled by LexLean to production software #92's stage 0 oracle gives.

    2. What is missing, by owner

    A. A defect found, filed as #105 (part of the path, needs to land before #92 stage 3). iterate, iterate_until and list_fold (and the other 17 library templates) are realized as self-recursive Rust functions, yet the registry rows say recursion = false. A root that scans a 80,000-byte name aborts with a stack overflow on an 8 MiB stack while declaring no recursion effect. A compiler whose loops are all folds over 48 MB modules cannot run until the templates are loops, or the effect is honest and the host provides the stack. #105 has the repro, and the fix in two steps (true now; cheap later with a loop construct, which is #91 stage 3).

    B. Primitive semantics the pilot had to work around (stdlib stage, with #90/#91's calculus work; no hurry, none blocks a pilot):

    1. No widening conversion. checked_convert must target a fixed width, so a UInt8 cannot become a Nat. Character classes were written as checked_subtract byte low / checked_subtract high byte both some, which works and is how the stdlib must say it until there is a to_nat.
    2. No Ordering literal or match. compare_bytes a b can only be compared with another compare_bytes (compare_bytes 0x 0x as "equal").
    3. Strings are opaque. No scalar iteration, only utf8_encode/utf8_decode/split_exact/join. Components should work on Bytes; the stdlib's text functions (startsWith, endsWith, trim, lines, splitOn, joinWith, find) are then ordinary members over bytes.
    4. Bytes and text primitives are not kernel-reducible. rfl fails on equal of Bytes, compare_bytes, utf8_encode, split_exact, join, format_decimal, parse_decimal (it succeeds on length, index, slice, append, the fixed-width operations, Nat, String equality). Cause: LexLeanRuntime.compareBytes is compare left.toList right.toList, and ByteArray.toList, ByteArray.instBEq and String.toUTF8 are core definitions that the module system does not expose to a downstream module; the runtime's own definitions are @[expose] but bottom out in them. So a theorem about text processing stated as an equation cannot be discharged by evaluation. Remedy: define those operations in the runtime over exposed structural definitions (fuel-indexed over data[i]?, as LexLeanCollections does for its key order), which is a runtime-content change (identity regenerates) that must agree with the calculus' compare. Until then the compiler's text logic is validated by differential against stage 0, not by in-language theorems.
    5. Authoring friction worth fixing in the surface, not the language: Nat subtraction only as prim subtract, + needs parentheses, lambda captures are explicit and must be sorted, and an exact axiom policy reports one violation per verify, so aligning N declarations costs N runs (a mode that lists every mismatch would be one run).

    C. Not expressible at all, by design: effects. Everything above is a function from data to data. The compiler also reads and writes files, lists directories, reads a few environment variables, runs lean/lake/leanchecker under a time and output limit, and exits with a code. #92 asks for "a defined, minimal, typed entry".

    3. Proposal: the effects boundary (for the maintainers' decision)

    The program is a pure state machine; a generated driver performs the requests. No effect enters the calculus.

    Request  := ReadFile path | WriteFile path bytes | ListDir path | Stat path | MakeDir path
              | Rename from to | Remove path | Environment name
              | Run argv env limits       -- limits = the config's child_timeout_ms, max_child_output_bytes
              | Emit stream bytes         -- stdout / stderr
    Response := Contents bytes | Listing (List entry) | Info stat | Done | Missing | Denied reason
              | Exited code stdout stderr | Value (Option bytes) | Failed reason
    Step S   := Finished code | Ask Request S
    root start  : Arguments -> Step S                         -- S is first-order data
    root resume : S -> Response -> Step S
    
    • A root takes and returns first-order data (§17.13 rule 1), so the continuation is data: a defunctionalized state, which is how the compiler already organizes its phases (a record of what is done and what is next).
    • The driver (perform : Request -> Response in a loop) is the only code that touches the host. It is part of the Rust machine: a runtime profile item in RustSyntax, authored in .lex.tex as [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91 stage 4 makes the runtime source, with the Rust standard-library calls it uses on the closed list of machine primitives [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91 stage 1 fixes (file read/write, std::process::Command with a timeout, env::var). This is not a new trust category: it is the same kind of item as tick.
    • Semantics. TargetSemantics and the preservation certificates cover the pure functions start and resume, for every response sequence (the compiler is correct against any host behaviour, including Denied and Failed). The claim that the driver performs the request is a table of machine primitives with stated meanings and a differential, at level build, never some-true: honest in the same way the work counter is.
    • Determinism (R10). No clock, no randomness, no ambient state: the environment is read only through Environment name for a closed list of names. Output order is the order of requests. The existing deterministic-artifacts gates therefore apply unchanged.
    • Effect registry. allocation, overflow and recursion stay what they are (resource failures of the pure function). I/O is not a fourth effect; it is the Ask data a root returns. A host profile is a new registry entry beside rust-core/rust-std (call it rust-host: rust-std plus the driver), so a root that does not use the entry stays a library and the driver exists only where a program asks for it.
    • Parallelism is not semantics. Today's compiler uses all cores in places (reading the identity of language 1.2: 0.8 s on four cores). The first realization is sequential; a map over independent items that the driver may evaluate in parallel is a later, separate addition with its own determinism statement. [v0.4] #27 stage 8: the production compiler authored in .lex.tex and compiled by LexLean to production software #92's performance gate should be a measurement (a regression bound on just vv), not a promise.

    Alternatives considered: (a) an effect monad inside the calculus (rejected: it puts state and failure into TargetSemantics and re-proves every certificate); (b) a typed host-function table callable from any closure member (rejected: it makes purity a convention, and the eligibility analysis could not tell a pure member from an effectful one); (c) world-passing (World -> (a, World)): equivalent to the state machine but needs a nominal World and linearity, which the calculus does not have.

    4. Stdlib and order

    The stdlib is source (.lex.tex, language 1.2, certified), a package of modules the compiler project imports: bytes and text over Bytes (item B3), sorting and Map/Set-based dedup (the collections exist; they are templates, so #105 first), canonical JSON, decimal and hex, SHA-256 (expressible today over UInt32 bit operations and iterate; to be measured, because the digest is on the hot path of every build), a diagnostic record and the closed error model's constructors.

    Proposed order, replacing my earlier list:

    1. Land production: the collection and iteration primitives are realized by self-recursive functions but registered recursion = false; a root overflows the stack at about 40k iterations #105 (honest effects; loops after [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91 stage 3) and [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91 stage 1 (feat(rust): close the runtime's library boundary (RB-09, stage 1 of #91) #102).
    2. Boundary design agreement on this comment; then one PR: SPEC §17.x for the host profile, register rows, the Request/Response/Step types as a language-1.2 stdlib module, a gate that no handwritten driver text exists.
    3. Stdlib members in the order the pilot needs them: bytes and text, sorting, JSON, hex/decimal, SHA-256 (each with a differential against its Rust counterpart on a seeded corpus).
    4. First real pilot: the axiom-output parser (verify/axiom.rs, 210 lines, §22.5): closed, pure, one Result of data, with committed golden vectors (tests/golden/axiom-parser/: accepted, wrapped, rejected) as the oracle and LLV7004 as its only error. Then the preservation lexer/token audit.
    5. calculus::check, rust::{lower, validate, print, package}, the production stages, certificates, front end (as the issue says), each under the differential and, at the end, the fixed point.

    Questions

    1. Is the state-machine boundary (§3) the direction? If yes I draft its SPEC section and register rows as the first PR of this issue once production: the collection and iteration primitives are realized by self-recursive functions but registered recursion = false; a root overflows the stack at about 40k iterations #105 and feat(rust): close the runtime's library boundary (RB-09, stage 1 of #91) #102 are in.
    2. rust-host as a third target profile, or the driver as a feature of rust-std? I recommend a profile: eligibility then states, per root, whether it may ask.
    3. Item B4 (making the bytes and text primitives kernel-reducible) is a runtime change that regenerates the language-1.2 identity. It can ride with [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91 stage 2 (which changes the runtime's content address anyway) or stand alone; I recommend riding with it.

    Generated by Claude Code

  4. humuhumu33 commented on Oct 15, 2026

    @humuhumu33
    Contributor

    Note: SHA-256, the last stdlib member in the spike comment (§4, step 3), is being delivered under #126 (claim posted there; stage 1 = module + differential against sha2 + production-build measurement, stage 2 = switching every identity).

  5. humuhumu33 commented on Oct 15, 2026

    @humuhumu33
    Contributor

    Claiming the stdlib lane of this issue (step 3 of the order above): the members, in the order the pilot needs them, starting with bytes and text. Not the effects boundary (§3), which stays with afflom.

    Member 1: bytes and text over Bytes. A language-1.2 project stdlib/ beside compiler/ (module prefix LexLeanStd; the examples leg verifies it as it does compiler/), first module Text, seven production roots for rust-std, written in the notation:

    root oracle (Rust std)
    startsWith (subject prefix : Bytes) : Bool <[u8]>::starts_with
    endsWith (subject suffix : Bytes) : Bool <[u8]>::ends_with
    trim (subject : Bytes) : Bytes <[u8]>::trim_ascii
    lines (subject : Bytes) : List Bytes str::lines on UTF-8 input (\n and \r\n end a line, the last line needs no ending)
    splitOn (subject : Bytes) (delimiter : UInt8) : List Bytes <[u8]>::split
    joinWith (pieces : List Bytes) (separator : Bytes) : Bytes <[Vec<u8>]>::join
    find (subject needle : Bytes) : Option Nat str::find on UTF-8 input

    Written as the spike did (B1–B3): components over Bytes, character classes by equal on UInt8, no String scalar iteration; iterate, iterate_until and list_fold carry recursion (#129).

    Evidence (R3): one register row, SL-01 in a new suite stdlib, scenario and failing test first. The test (a) checks and builds the project and reads the seven roots' eligibility, (b) runs each root's lowered program through the calculus interpreter and its rust-std rendering through the pinned rustc on a seeded corpus against the oracle above, zero disagreements, (c) detects a planted defect in a member, and (d) under the pinned Lean, exports the certified packages (certificates A, B, E) through lexlean::api::production, committed under tests/production-export/stdlib/ and compared by check-production-export. Theorems that need Bytes equality by evaluation wait for B4 (#91 stage 2); until then the module's text logic is validated by the differential only, as §2.B4 says.

    Branch claude/issue-92-stdlib-bytes-text; draft PR to follow. Sorting, JSON and hex/decimal come after, one PR each, unless the layout is objected to (SHA-256 is claimed under #126 and stays out of this lane).

  6. humuhumu33 commented on Oct 16, 2026

    @humuhumu33
    Contributor

    Member 1 (bytes and text over Bytes) is up as draft PR #153: stdlib/ beside compiler/, seven rust-std roots in LexLeanStd.Text, SL-01 (suite stdlib, SPEC §17.22, register 382) green locally in 156 s including the Lean-backed export; fast vv legs green, the long legs run in its CI. Sorting is next, in its own PR, unless the layout draws an objection there.

  7. afflom commented on Oct 16, 2026

    @afflom
    MemberAuthor

    Maintainer decisions on the three questions of the spike comment, so that the effects boundary has an answer and the stdlib lane (#153, #126) is not waiting on one. Each follows from what the issue already requires; none needs more information.

    1. The state-machine boundary of §3 is the direction. The program is a pure start/resume pair over first-order data and a generated driver performs the requests. The alternatives fail the issue's own text: an effect monad puts state and failure into TargetSemantics and re-proves every certificate; a host-function table makes purity a convention, so eligibility could not tell a pure member from an effectful one; world-passing needs a nominal World and linearity the calculus lacks. The one thing this fixes for the stdlib: no member may call a host function, and effects are the Ask data a root returns.
    2. rust-host is a third target profile, rust-std plus the driver. Eligibility then states per root whether it may ask, which a feature of rust-std would hide.
    3. B4 rides with [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91 stage 2, which changes the runtime's content address anyway: one regeneration of the language-1.2 identity instead of two. Until it lands, theorems that need Bytes equality by evaluation wait, and the stdlib's text logic is validated by its differential, as the spike says.

    What this leaves in order: the boundary's SPEC section, register rows, the Request/Response/Step module and the gate come after #91 stage 4, because the driver is a runtime profile item that is authored in .lex.tex and rendered by the Rust backend, and it cannot exist as handwritten Rust (this issue's DoD and #91's). I will not land a boundary text with no driver behind it: that would be a register row claiming what nothing implements. The stdlib members (#153 and the ones after it) do not wait for it: they are pure.


    Generated by Claude Code

  8. afflom commented on Oct 16, 2026

    @afflom
    MemberAuthor

    Claiming step 4 of the order above, the first real pilot: the exact axiom-output parser (verify/axiom.rs, §22.5), authored as .lex.tex production roots, certified (A, B, E) and compared with the Rust it replaces. A contributor session has it, stacked on #153's Text members (branch claude/issue-92-pilot-axiom-parser, from claude/issue-92-stdlib-bytes-text; nothing is pushed to #153's branch). It will post its project layout here before building, and open a draft PR (Refs #92) with one register row (BS-01), the golden vectors of tests/golden/axiom-parser/ and a seeded differential as the evidence. The Rust parser stays as the stage-0 oracle until the fixed point.

    @humuhumu33: this is the user of #153's startsWith, trim, lines, splitOn, find. If the pilot finds a member missing or wrong, it will say so on #153, not patch your lane. It does not move the language-1.2 identity; if it needs to, it stops and reports here first.

    The boundary (SPEC section, register rows, the Request/Response/Step module and its gate) is still after #91 stage 4, as decided above.


    Generated by Claude Code

  9. afflom commented on Oct 16, 2026

    @afflom
    MemberAuthor

    Pilot layout (step 4, axiom-output parser). Proceeding unless the maintainer objects here.

    Cross-project imports are impossible today. A project reads only its own source_roots; config::is_project_relative rejects .., and symlinks are refused (project.rs). So bootstrap/ cannot \importmodule LexLeanStd.Text.

    Least-bad layout:

    One member does not fit: Text.trim is trim_ascii, by design. The Rust parser uses str::trim and char::is_whitespace (Unicode White_Space: VT, U+0085, U+00A0, U+2000-200A, ...), so the pilot defines the Unicode variants locally and the differential includes those characters. I will say so on #153 rather than change it there.

    No runtime or calculus change is planned; the language-1.2 identity is not touched. Branch claude/issue-92-pilot-axiom-parser.


    Generated by Claude Code

  10. afflom commented on Oct 16, 2026

    @afflom
    MemberAuthor

    Maintainer: the pilot layout above is accepted; proceed.

    The vendored Text.lex.tex is acceptable because the case asserts on every run that it equals #153's file with the production: clauses stripped, so it can only be a copy, not a fork. It is temporary and has to say so in its header and in the SPEC paragraph.

    The finding behind it is bigger than the pilot, and it is on this issue's path: a project cannot import another project's modules (source_roots stay inside the project, .. and symlinks are refused). Step 2 of the order here ("a package of modules the compiler project imports") has no mechanism today, and the lexicon_source kinds (builtin, path, git) are for glossaries, not source modules. Before the compiler's own modules (steps 5 to 7) import the stdlib at scale there must be one. Options, for whoever takes it (I will write the design comment when #153 and #146 have landed, so it can rest on the stdlib as it is then): (a) one project holds both, the stdlib being modules of the compiler project and stdlib/ its published slice; (b) a source_package kind of lexicon_source that pins another project's modules by tree hash, read into the importing project's graph and identity; (c) a project-level [[dependency]] with the same pinning. (b) and (c) change the identity and the lock format, so they belong with #91 stage 2's regeneration, not before.

    Not in the pilot's scope; no change to its plan.


    Generated by Claude Code

  11. 73 remaining items

  12. afflom commented on Oct 24, 2026

    @afflom
    MemberAuthor

    #92 step 4, BS-13 scope finding (loops: tail::looped and tail::loops)

    In Lowering::lower, each function runs expr (takes fresh names), succeed, fold, then tail::looped (takes 1 + |changed parameters| more fresh names when the function loops), then moves::apply, one function after the other on one counter. So the names of function i+1's terms come after the loop names of function i; BS-11's lowerBodies threads the counter across all functions from 0 and is right only because its corpus has no looping function.

    Cut I will take, unless told otherwise:

    • lowerBodies of RustBodies takes the counter it starts from as a parameter (today the literal 0); with every function but one marked dead it lowers one function and returns the counter after its terms. This is a one-parameter change to BS-11's root (its certificate grows by a few hundred bytes, and BS-11/BS-12 need one more export each when it lands on their branches).
    • A new root loopFunction of RustLoops ports looped (occurs, tails, passes, retarget, the state type and the new parameters) over one function: parameters, result type, body block, start counter; it returns whether it loops, the new parameters and body, and the counter after.
    • The host composes, per function in order, lowerBodies (one function, counter in) → succeedBodies → foldBodies → loopFunction (counter in, counter out) → all functions to applyMoves, and compares with the Function items of rust::lower. The host threads one number between certified roots and decides nothing else; the order of the functions is the oracle's.
    • Held to the oracle: the programs in which some function loops (the corpus filter becomes rust::loops true for at least one function), and tail::loops is checked against the root's found flag on every vector, as the oracle itself asserts at lowering.
    • Outside: the crate around the bodies and the assembly of all of this into one entry point (BS-14), which is where the per-function interleaving that the host does here becomes the claim.

    I start now; if you want the counter handled differently (for example a root that returns the loop-name counts so lowerBodies is not touched), say so.


    Generated by Claude Code

  13. afflom commented on Oct 24, 2026

    @afflom
    MemberAuthor

    #92 step 4, BS-13 (loops: tail::looped): figures

    One production root for rust-std, RustLoops.loopFunction, ported from tail::looped and held to the Function items of rust::lower to the byte, function by function: for each live function the terms from the counter of fresh names (BS-11's lowerBodies, which now takes the counter it continues from and returns the one after its bodies), succeed (BS-12), the fold, then loopFunction (which takes 1 + |changed parameters| more names); the counter passes to the next function; then all items go to BS-08's applyMoves. loopFunction finds a loop exactly where calculus::rust::loops does on every function, and the case fails otherwise.

    Root Cert A Cert B Cert E
    loopFunction 744,534 (11.8% of 6,291,456) 623,883 160,793
    • Budget in the row: 3,145,728. Export: 30 min 32 s. Case up to the comparison with the committed packages: 2,270 s.
    • 88 vectors (43 within the evaluator's 300 nodes; 87 without the new package), 107 seeded mutants, 328 rustc runs agree; 12 planted defects detected. Four hand-built rules reach the paths the corpus did not (self call in a binding, condition or scrutinee; an unread parameter).
    • Removed rather than claimed: three guards of the oracle that no program reaches (the condition and the scrutinee calling the function, a block that is itself a tail).
    • lowerBodies (BS-11) changed shape by the counter parameter: part A 5,328,431 (84.7%, within the row's 5,505,024); BS-11's row and figures are updated on this branch.
    • Outside the claim: the crate around the bodies and the assembly (BS-14); parameters and result types of a function are inputs (the host reads a looping function's pre-loop parameters back from the oracle's item).

    Generated by Claude Code

  14. afflom commented on Oct 24, 2026

    @afflom
    MemberAuthor

    #92 step 4, BS-14 scope finding (the crate around the bodies)

    What Lowering::lower still does outside the certified roots: the order of the items, the parameters of each function (binder: a name when the body mentions it and the type is not unit, a wildcard otherwise), its result type (wrapped in Fallible when the function can overflow), and the body of a function with an uninhabited parameter (an empty match on that parameter, with the origin type:<kind>; such a function is the dead one). Until now the cases took those from the oracle's own items (the parameters of every function, the bodies of the dead ones, and the whole of the types items were not compared).

    Cut I take:

    • RustAssemble.functionShells: for each function in order, the Function item with its parameters, its result type, and, for a dead function, its complete body; a live function's body is a placeholder. Inputs are the program and the frame, as lowerBodies takes them. The host reads the parameters and the result of a function from here (so loopFunction no longer takes the pre-loop parameters from the oracle).
    • RustAssemble.assembleItems: the items of lowerTypes (BS-07), the shells, and for each live function what the loop pass returned (parameters and body), as one chain of items.
    • The host then composes: frame → shells → per function (terms with counter, success, fold, loop) → assembleItems → applyMoves, and compares the whole item chain of rust::lower's crate (the types items included), not only its Function items; the profile is the host's input.
    • Outside: the checks rust::lower runs after lowering (validate, correspond), which are the roots of BS-05/BS-06's Rust validator and are not composed here; the profile (an input).

    I start now.


    Generated by Claude Code

  15. afflom commented on Oct 25, 2026

    @afflom
    MemberAuthor

    #92 step 4, BS-14 (the crate around the bodies): figures

    Two production roots for rust-std, RustAssemble.functionShells (a Function item of each function: parameters by the binder rule, the result type wrapped Fallible when the function can overflow, and the complete empty-match body of a function with an uninhabited parameter) and RustAssemble.assembleItems (the items of lowerTypes, the shells, and what loopFunction returned for each live function, as one chain, the types' first). Held to the whole chain of items of rust::lower's crate, the types items included, to the byte, by composition with BS-07 and BS-11..13 and applyMoves.

    Root Cert A Cert B Cert E
    functionShells 1,195,444 (19.0% of 6,291,456) 960,728 279,092
    assembleItems 107,569 109,542 169,057
    • Budget in the row: 3,145,728. Export of the two roots: 30 min 48 s. Case up to the comparison with the committed packages: 2,836 s.
    • 324 vectors (276 within the evaluator's 300 nodes; 322 without the new packages), 537 seeded mutants (560 without), 552 rustc runs agree; 11 planted defects detected.
    • What the Rust host still does, as the row states it: threads the counter of the fresh names from one live function to the next, supplies the profile, and calls applyMoves. Three planted hosts that do more are each caught by the comparison with the crate: the counter starting again at each function, the functions lowered last to first, and the items of the first live function left out.
    • The set of inhabited types that decides a dead function must be the one the lowering reaches after it drops the closures of dead functions; the frame of BS-11 assumes all functions live, so functionShells recomputes it (vector 9 caught the difference).
    • A second kernel defect on the way, certificate A this time: Certificate A is rejected by the kernel for a recursive function with a list parameter before its recursive argument #314 (a recursive function with a list parameter before its recursive argument, matching that list in the arm; minimal one-module repro attached). Worked around by putting the recursive argument first; sibling of Certificate B is rejected by the kernel for a match arm that returns a binder of the matched recursive type #310.
    • Outside the claim: the checks rust::lower runs on the crate (validate, correspond), which are the roots of BS-05/BS-06 and are not composed here.

    Generated by Claude Code

  16. afflom commented on Oct 25, 2026

    @afflom
    MemberAuthor

    #92 step 4 is complete (#307 → #311 → #312 → #313 → #315); the boundary for step 5: what the Rust host still does

    The roots below, composed in this order, give the whole chain of items of rust::lower's crate to the byte (BS-14, crates/conformance/src/cases/bootstrap/rust_assemble.rs). Names are Module.root; P is the program as calculus data (spec : Bytes, adts, parameters : List (List Nat), types, results, bodies) (the checker's encoding, program_arguments in the cases).

    # Root Takes Returns
    1 CalculusCheck.siteTrees P sites (a type chain)
    2 CalculusCheck.useTrees P uses
    3 RustLower.deadFunctions P, sites dead : List Bool
    4 RustLower.fallibleFunctions P, sites, dead fallible : List Bool
    5 RustBodies.lowerFrame P, sites, dead Frame {known, edges, inhabited, ordered}
    6 RustLower.lowerTypes P, sites, dead, fallible Def chain: the Enum and Apply items
    7 RustAssemble.functionShells spec, adts, parameters, types, results, bodies, fallible, Frame Def chain: a Function item per function; complete for a function with an uninhabited parameter
    8 RustBodies.lowerBodies P, sites, uses, dead, fallible, Frame, start : Nat Lowered {expr, next}: the bodies as lower holds them before succeed, a dead function's place being the empty chain
    9 RustSucceed.succeedBodies results, fallible, bodies bodies, Ok pushed into the tails of the functions that can overflow
    10 RustBodies.foldBodies bodies bodies, folded
    11 RustLoops.loopFunction me : Nat, parameters (Par), result Ty, body, start LoopOut {found, params, body, next}
    12 RustAssemble.assembleItems Def of 6, Def of 7, List LoopOut of the live functions in order, dead all the items, the types' first
    13 RustMoves.applyMoves Def Def

    What the Rust host still does, exactly:

    1. Orders and threads the counter. For each live function in position order: root 8 with every other function marked dead (so one body is lowered) and start = the counter so far, then 9, 10, then 11 with start = the next of 8, whose own next is the counter for the following function. lowerBodies over all functions at once is right only when no function loops (BS-11's corpus); the loop pass takes 1 + |changed parameters| names between two functions' terms.
    2. Supplies the pieces roots take from other roots: the parameters and result type of function i for root 11 are read from the shell of root 7; the list of LoopOuts for root 12 is the live functions' results, in order.
    3. Supplies the profile (Std or Core; the Crate is (core : Bool, items)), and calls root 13 on the chain of 12.
    4. Not composed at all: the checks rust::lower runs afterwards, RustValidate.validateCrate(krate) and RustValidate.correspondCrate(krate, elements) (BS-05/BS-06's roots); and the realization elements correspondCrate takes (elements(program, lowering): program_elements, function, type:adt, overflow, indirection), which no root computes yet.
    5. The refusals. The cases keep only programs rust::lower accepts. The errors Lowering::new/lower return (an ADT with no constructor, a computation of a value no type inhabits, a heap type under rust-core, failures()) are not ported: no root says why a program has no rendering.
    6. In the cases (not a design limit), dead and fallible are taken from the oracle (lowered(), rust::fallible_functions), while roots 3 and 4 exist and are held to the same oracle in BS-07; a driver should feed 5–12 from 3–4.

    Two things that bit during step 4, for whoever composes these: (a) the set of inhabited types that decides a dead function is the one reached after the closures of dead functions are dropped (the Frame's inhabited assumes all live; root 7 recomputes it); (b) certificate defects #310 (an arm that is a bare binder of the matched type) and #314 (list parameter before the recursive argument), both worked around in the sources.


    Generated by Claude Code

  17. humuhumu33 commented on Oct 25, 2026

    @humuhumu33
    Contributor

    Proposal for the owner: tag v0.4.0 at the downstream boundary; steps 5 and 7 of this issue and #91's H7 go to the next release

    Posted here as the 2026-10-22 ruling asks (comment): a proposal only, the owner decides. No milestone, label or title is changed.

    What is new since the 10-21 and 10-22 rulings

    1. The downstream train waits on this tag. PrismPM v0.4.0: everything prism-code v0.1 needs prismpm#211 fixes the policy: "PrismPM v0.4.0 is held until LexLean v0.4.0 is released and imported here", and prism-code v0.1.0 is in turn held until it imports PrismPM v0.4.0.
    2. What that train needs from LexLean is already on release/v0.4. PrismPM's v0.4 list asks LexLean for language 1.2 and the 1.1→1.2 migration (Migrate a language-1.1 data-form project to language 1.2 from the CLI #257). Add lexlean migrate --to 1.2 for language-1.1 data-form projects (#257) #261 delivered the migration at fbfaa73fe. No item on feat(rust): a loop over a state with a break value in the declared machine (SP-19, #91 stage 3) #211's list depends on steps 5 to 7 of this issue or on [v0.4] #27 stage 6: complete the closed Rust AST so the runtime is generated from the calculus #91's H7.
    3. The remaining time is now mostly unwritten rows. Step 4 is complete as drafts (BS-10: the literal values of the Rust lowering (RustTerms.lowerValueAt) #307 → BS-11: the terms of a function body of the Rust lowering (RustBodies, useTrees) #311 → BS-12: the value a function that can overflow returns (RustSucceed) #312 → BS-13: a function whose self calls are all tail calls as a loop (RustLoops) #313 → BS-14: the crate around the bodies of the Rust lowering (RustAssemble) #315, comment), and two of the heap half's five PRs have landed (#91 stage 4, H4a: the first four std runtime functions as source — append_list, length_list, index_list, slice_list (SP-29, RB-20) #293, #91 stage 4, H4b: the byte and text functions of the std runtime as source (SP-29, RB-20) #297). Step 5 has the 5a figure (comment), with 5b and 5c not started. Step 7 (7a to 7c) is not started and its lane is unassigned (question). Each of those six rows needs a measured figure, a review, a draft and a landing with its own regeneration. That is most of the time between today and the tag.

    Proposal

    This is the maintainer's option (a) of 2026-10-21 (comment), drawn later: the heap half and step 4 are in, only the unwritten rows move.

    Estimate: about ten landings remain under the proposal. With stacked PRs landed together (offer 1 below), that is about three to five days of landings. This is an extrapolation from the 2 to 3 hour full gates of this week, not a measurement.

    Offers, whatever the decision

    1. Fewer regenerations. Where a stack is accepted as a whole, land it as one regeneration: #91 stage 4, H5a: the types, impls, and helpers of the std runtime as source (RB-20, RB-19, BS-09) #298 + #91 stage 4, H6: the number functions of the std runtime as machine calls with their refinement theorems, and every runtime item has its refinement statement (SP-29, RB-20, SP-30) #304 + #91 H7-audit: the crate header, module wrapper, and definitions as RuntimeSyntax nodes; RB-21 and RB-10 tightened; the shared-write plants #305, and the eight step-4 PRs. The maintainer decides which stacks qualify.
    2. Regeneration work. Our session prepares the regeneration for any landing the maintainer assigns, and the maintainer reviews and merges. Precedent: #91 stage 4, H2: the runtime syntax validator is bootstrap source (BS-09) #259, prepared from the landing procedure, full gate green on 6e8e4e9a1 (run 40030579344), merged unchanged.
    3. If the scope stays. We continue step 5 in our lane and start 7a to 7c if the lane is given to us.

    🤖 Generated with Claude Code

  18. afflom commented on Oct 25, 2026

    @afflom
    MemberAuthor

    I agree that features and releases need to be focused on downstream requirements instead of arbitrary/self-referential aspirations. That's not to say the self-compilation/self-expression is not needed, but no release should be held if the downstream is blocked and the only remaining issues in the release are not requirements driven.

  19. afflom commented on Oct 25, 2026

    @afflom
    MemberAuthor

    Maintainer ruling, implementing the owner's note above (6399822402) and the owner's direction to me today: v0.4.0 is tagged at the downstream boundary, now. The downstream train (PrismPM#211, prism-code behind it) waits on the tag; what it needs, language 1.2 and the 1.1→1.2 migration (#257, landed in #261), is on release/v0.4; every other open v0.4 item is self-hosting or hygiene, not a downstream requirement, and a release is not held for those.

    v0.4.0 = release/v0.4 as it stands (201519a1b, register 456) plus #309 (version 0.4.0 and the release notes). Nothing else lands before the tag. The release notes say exactly which bootstrap stages ship: the register carries BS-01 to BS-06 and BS-09 (the printed std runtime through H4b); BS-07, BS-08, BS-10 to BS-14 and the rest of the runtime-as-source ship in the patch release.

    v0.4.1 (new milestone, a patch release that follows directly on the same line, one regeneration at a time, in this order): #298 (H5, in flight; its regeneration is redone on the bumped line), #304 (H6), #305 (H7-audit), the bootstrap stack through #315, #247 (#30 slice 4), #256 (#251, #252), #284 with #292. All of it is written and accepted; none of it holds downstream. Issues #27, #30, #91 and #92 move to that milestone.

    v0.5 keeps what the earlier rulings put there: steps 5 to 7 of this issue, #91's H7 (#306), and everything without a milestone.

    On the proposal of 6399769811: it is the right direction and the owner has agreed with it; the cut here is tighter than offered because the owner's rule is "not requirements-driven, not in the release", and the heap half and step 4 are not. Offer 1 is already how the stack lands (#315). Offer 2: the regeneration that moves the identity stays the maintainer's step; thank you. Offer 3 does not apply.

    Mechanics: #309 marked ready today and gated; merge; one release.yml rehearsal (the owner's go of #34 covers it); tag v0.4.0; main receives the line at the tag; release/v0.4 forward-merges into release/v0.5. Then v0.4.1 landings resume at once. The moratorium of #15 stands until the tag; after it, issues and PRs open only against a milestone.


    Generated by Claude Code

  20. afflom commented on Oct 26, 2026

    @afflom
    MemberAuthor

    LexLean v0.4.0 is released.

    Next: release/v0.4 is merged into main by PR (the line at its tag, SPEC §9.4), the hold on release/v0.4 landings is lifted (patch work on the line continues as 0.4.x; feature work goes to release/v0.5), and the second forward merge into release/v0.5 follows.


    Generated by Claude Code

  21. added 3 commits that reference this issue on Nov 11, 2026
  22. humuhumu33 commented on Nov 12, 2026

    @humuhumu33
    Contributor

    Proposal on #273: certifying a compiler-sized root (modular certification)

    Posted in full on #273: #273 (comment). It is a proposal for the owner and the maintainer to decide. No milestone, label or title is changed, and nothing is built or pushed.

    Why it concerns this issue. Stage 1 is one command root whose closure is the whole compiler. A root's certificates restate its whole closure (§17.17).

    On the 62 committed bootstrap roots at aaf845507 (counted):

    • the roots' programs hold 4,698 functions, of which 2,221 are distinct;
    • program bytes, summed over the roots, are 1.95 times the distinct bytes;
    • the certificates total 156,902,122 bytes, and the largest is 5,320,581 bytes, against max_file_bytes 6,291,456.

    Scaled to the compiler (an estimate), one root's certificates A and B would be about 340 MB and 440 MB.

    The proposal.

    • Lower one program per project and target. Each function's certificate rows are emitted once, in shard modules cut along the call graph, and each root keeps a small A, B and E that points into them.
    • The library's proofs and the calculus semantics stay as they are. Only the entry index of A's root theorem moves.
    • A measurement spike on the bootstrap comes first, with a stated go/no-go gate.

    The #273 comment also covers the stage-1 decisions that follow: where the command root lives, how the compiler reaches its 36 MB language/ tree, rust_renderer until removal, and whether the self-compile's certificates are committed. It ends with six questions.


    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    self-hostingLexLean compiling itself: compiler in .lex.tex, certified, built by its own pipeline

    Type

    No type

    Projects

    • Status
      Todo

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions