Skip to content

Self-hosting: the effects boundary, a typed minimal entry for a production root (arguments, files, child processes, exit) #271

Open

Description

@humuhumu33

Child of #92 (stage 8 of #27); named in #92 under "What it needs". Part of v0.4's objective: LexLean compiles itself.

Scope

A LexLean-built compiler must read sources, write outputs, run Lean for verification and exit with a status. Production roots today are pure. This issue defines, as part of the language rather than a handwritten shim, a minimal typed entry for a production root: its arguments, a file-system capability, a child-process capability and its exit, with the effects declared and checked like the existing ones (§17.13), realized in the Rust runtime of #91.

Done when

Each part follows the standing R3 discipline of #15: register rows in model/ids.toml and SPEC §31, one scenario and one conformance_<id> test per row, positive and negative fixtures, a planted defect recorded in VERIFICATION.md, and the Rust module kept as the stage-0 oracle (differential to the byte) until the fixed point of #92 removes it.


Generated by Claude Code

Activity

  1. added this to the LexLean v0.4 milestone on Oct 22, 2026
  2. added
    enhancementNew feature or request
    self-hostingLexLean compiling itself: compiler in .lex.tex, certified, built by its own pipeline
    on Oct 22, 2026
  3. humuhumu33 commented on Oct 22, 2026

    @humuhumu33
    ContributorAuthor

    Claim. I take this issue. Design first; nothing is built until you confirm, since it changes the language and the registry.

    The shape: the entry is a pure step function, and the host only carries out what it returns

    A command root never performs an effect. It returns a request as data; a generated driver performs exactly that request and passes the outcome back as the next event. The step stays an ordinary production root (rule 1 of §17.13: first-order data in and out), certificates A, B and E apply to it unchanged, and each capability is a constructor that the eligibility walk already sees. Base: release/v0.4 at 9796067; I read none of your open drafts as touching effects or a host entry (#254, #255, #259, #262, #264, #265 are the runtime; #253, #258, #266 the Rust backend).

    1. Surface

    A closed core module LexLeanHost (§17.10), three inductives:

    Event   = start (arguments : List Bytes)
            | read (outcome : Result Bytes Failure)
            | listed (outcome : Result (List Bytes) Failure)
            | wrote (outcome : Result Unit Failure)
            | ran (outcome : Result Ran Failure)          -- Ran: status : Option Int32, output, errors : Bytes
    Request = read (path : Bytes) | list (path : Bytes) | write (path : Bytes) (contents : Bytes)
            | run (program : Bytes) (arguments : List Bytes) (input : Bytes)
            | exit (status : UInt8) (output : Bytes) (errors : Bytes)
    Failure = notFound | denied | invalid | other
    

    Failure is closed so an outcome is the same bytes on every host (no OS message text). list is there because a project is found by reading a directory; write creates missing parent directories. An entry is a root declared with entry: command:

    definition{
      name: step,
      parameters: [state : (Option Session), event : LexLeanHost::Event],
      result: (Session * LexLeanHost::Request),
      executable: true,
      production: {effects: [allocation, files, overflow, process], entry: command, targets: [rust-std]}
    }
    

    The first call gets none and start. An entry whose parameters and result are not exactly this shape, for one first-order Session, is refused (a new code, LLT4017, or a named violation of LLT4005: your call).

    2. Typing and checking of the capabilities

    The registry (production-1.2.toml) gains two effects, files and process (order: allocation, files, overflow, process, recursion), a target property host (rust-std true, rust-core false), and one construct row per request constructor, keyed as constructor.nat_succ is (constructor.host_read, …): read, list, write realize files, run realizes process, exit nothing. Rule 5 then checks them as it checks allocation today: a closure that builds Request.run realizes process, and a root that does not admit it is LLT4005. A new clause beside rule 4 refuses either effect on a target without host. This is sound without a flow analysis: no event carries a request, so the step can return only requests its closure builds.

    3. Realization in the Rust runtime of #91

    An entry's package gains src/main.rs and a [[bin]]. The driver is source in the manner of RuntimeSupport: compiler/src/RuntimeHost.lex.tex in RuntimeSyntax, whose closed table of outside names grows by exactly the calls it needs (std::env::args_os, std::fs::{read, read_dir, create_dir_all, write}, std::process::Command with output, std::process::exit, std::io::Write::write_all, and std::thread::Builder::stack_size so the step runs under the root's measured stack bound of PD-08 and #105). The driver has an arm only for the declared effects; any other request maps to a fixed refusal status, so a binary whose root does not admit process contains no call that starts a process. A failure of the step (overflow, allocation) exits with a fixed status and fixed text. Paths and arguments are bytes; on Windows they go through UTF-8 and UTF-16, and an ill-formed one is Failure.invalid.

    In RuntimeSemantics the outside calls are one parameter, a world Request -> Event. The kernel-checked statement is that the driver's run under any world equals LexLeanHost.trace step world, so the driver adds nothing to the step. What stays trusted is that the standard library realizes the world's contract; that part is held by a differential, not a proof, and the row says so.

    4. Register rows (numbers at landing)

    • PD-‹n›: the files and process effects, the host property, the entry shape, and the checks of section 2, each with positive and negative fixtures.
    • RB-‹n›: an entry renders a binary whose driver is the validated RuntimeHost source; on every host, a fixture command root (lists a directory, copies a file, runs a child built from a second fixture root, exits with its status) gives the stated files, output and status, and the driver agrees with a reference interpreter on scripted worlds.
    • SP-‹n›: the driver refines LexLeanHost.trace, kernel-checked.

    5. Planted defects

    The registry row of run names files instead of process (a root that runs a child while admitting only files is accepted: PD red). The driver keeps the run arm for a root without process (RB red, by the absence check on the binary). The driver reports a signal-killed child as status 0, or swaps output and errors (RB red). The driver passes the old state to the next step (SP refuses the planted module).

    6. PRs, smallest first

    1. LexLeanHost, the registry rows, the entry shape and the eligibility clauses, PD-‹n›. Check only, no Rust. It moves the compiler-semantics ID (the registry is hashed), so it lands in your regeneration order after the #91 stage 4, PR 5: the core runtime's support items as source — RuntimeSyntax, RuntimeSupport, RuntimeSemantics (RB-17, SP-25) #254 to #91 stage 4, H4-0: the meaning of the buffer, share, and replace primitives over the model, kernel-checked (SP-28) #265 stack.
    2. RuntimeHost, the outside-name entries, the package's [[bin]], RB-‹n›. Stacked on H4a (the driver walks List Bytes). It touches package.rs, the oracle of Self-hosting: complete the certified Rust packager beyond the BS-15 first slice #272 (BS-08), so it goes before or after BS-08 as you order.
    3. SP-‹n›, the refinement theorem of the driver.

    7. For you to decide

    1. The shape: a pure step plus a generated driver, rather than capability values inside the language. Confirm or redirect.
    2. Output only at exit (exit status output errors, as above), or a print request so a long run can stream.
    3. A child's environment and working directory: inherited (simplest, ambient) or explicit in run (deterministic, more to carry).
    4. LLT4017 or a violation of LLT4005 for a malformed entry.
    5. Anything else the compiler needs at the boundary that I have missed (stdin, removal, time, environment variables are left out on purpose).

    Generated by Claude Code

  4. changed the title [-][v0.4] Self-hosting: the effects boundary, a typed minimal entry for a production root (arguments, files, child processes, exit)[/-] [+]Self-hosting: the effects boundary, a typed minimal entry for a production root (arguments, files, child processes, exit)[/+] on Oct 22, 2026
  5. removed this from the LexLean v0.4 milestone on Oct 22, 2026
  6. humuhumu33 commented on Nov 1, 2026

    @humuhumu33
    ContributorAuthor

    Design update. This replaces the claim above, which nobody answered. Ilya accepted this design on 2026-11-01. Base: release/v0.4 at ccbf038. The first slice is written and has not yet passed a build.

    The shape is unchanged

    A command root is a pure step function. It returns a request as data, and a generated driver carries out the request and passes the outcome back as the next event. The step stays an ordinary production root, so eligibility and certificates A, B and E apply unchanged. There is no new syntax and no new error code.

    The entry

    A root is a command exactly when its signature is:

    step (state : Option Session) (event : LexLeanStd.Host::Event) : Session * LexLeanStd.Host::Request
    

    for one first-order Session. The type is the declaration. A root of another shape is an ordinary pure root and gets no binary.

    LexLeanStd.Host is an ordinary module of stdlib/ that holds types only. Event.start carries the arguments, working directory, executable path, environment, and whether stderr is a terminal. These are inputs, like arguments, so they need no effect.

    There are 14 requests, and each stands for one call of Rust's standard library:

    Effect Requests
    files read, write, list, stat, createDir, removeFile, removeTree, rename, canonical, lock
    process spawn, wait, stop
    none exit status output errors

    Text sent to the host is String, so an ill-formed path cannot exist. Text from the host is Bytes. A failure is a closed kind plus the host's message as bytes. list returns names ascending by bytes. A child's environment is always explicit, stdin is null, there is no shell, and its output goes to two files the step names. The oracle is UTF-8 only too, so this loses nothing.

    There is no blocking run request. The order, width, memory budget (§22.11), timeout and output cap (§25.5) are semantic and produce diagnostics, so they belong in the certified step.

    How effects are checked

    The registry gains the effects files and process, and a fourth closed section, [[request]], with one row per constructor of Request naming the effect it realizes. Rule 5 of §17.13 applies: a closure that builds Request.spawn realizes process, and a root that does not admit it is LLT4005. This is sound without a flow analysis, because no event carries a request and nothing else makes one, so a step can return only the requests its closure builds.

    Requests are a section of their own rather than constructor.host_* construct rows. Construct rows must match the realization table, the coverage corpus and the oracle corpus one to one, and a request is not a construct kind of the IR.

    rust-core needs no new rule, because Request holds heap storage and rule 4 already refuses it on a target without allocation. An effect is a kind of authority, not a sandbox: files reaches every path, and a child has the authority of the process that started it. Path confinement stays in the step, certified.

    The driver

    A command root's package gains src/main.rs, printed from a new module RuntimeHost in RuntimeSyntax, as RuntimeSupport is. lib.rs and the runtime text do not change. Program ADTs are nameless, so the renderer finds them by walking the export's signature against the shape of Host, refuses any mismatch, and prints aliases. A project that fakes a module named LexLeanStd.Host is held to the same shape and the same rows, so it gains nothing.

    An arm is printed only for an admitted effect, and a new validator rule hidden-effect refuses a host name in a package that does not admit its effect. No closures or macros are needed. RuntimeSyntax grows by the standard-library names, one closed cfg attribute (for process_group), and the package's own crate as a use root, and the twin BS-09 learns the same.

    What the no-outside-libraries rule changes

    Crate Replacement
    fs4 std File::lock
    tempfile, walkdir composed in LexLean from createDir, list, stat
    rustix std cannot signal a process group, so the driver runs the host's kill (taskkill on Windows, as the oracle already does)
    signal-hook std has no signal API and packages forbid unsafe, so a watchdog (below) replaces it
    same-file dev and ino under cfg(unix); on Windows the call is unstable, so LLC0104 does not detect hard links there

    Watchdog. When the driver starts its first child, it re-executes itself as a small process in its own group, holding a pipe from the driver, and writes each child's group to it. When the pipe ends, however the driver died, the watchdog kills those groups. No handler is installed, so ignored signals stay ignored and the exit status stays the signal's, which keeps §17.17's tested contract. It also covers SIGKILL. One gap: a compiler running as PID 1 with no handler ignores SIGTERM, so docker stop waits for SIGKILL.

    Also lost, none of it stated by the SPEC: permission bits, the mode fs::copy keeps, and threads over pure computation, because values are Rc. The thread loss is a cost for #277.

    Row 1466's public Rust function end_live_children cannot exist in a command binary. That belongs to #274 with the Engine API.

    The trusted base

    Added to the Lean kernel, leanchecker, rustc and the host:

    1. The driver text, an estimated 300 lines of validated RuntimeSyntax with no semantic logic. It is held by the validator and its Lean twin, by plants, and by a differential against scripted events. It is not kernel-checked, and the design does not include proving its loop: the theorem needs RuntimeSemantics to give meaning to let, loop and assignment, and the per-request code would stay outside it regardless.
    2. std's realization of each of the 14 calls, which is part of rustc and the host.
    3. The host's kill or taskkill.

    For #273: "the only trusted base" is not literally true even without this issue. The SPEC already says the RuntimeStd walks and the number text are evidence, not proof (SP-29). #273 should state those and the driver text explicitly. The compiler's own logic stays certified, since it all sits in the step.

    Slices

    1. PD-10: the effects, the request rows and the checks, with no Rust rendered. It moves the compiler-semantics ID and touches the stdlib, so it regenerates after BS-15: the Rust package renderer as language-1.2 source #371.
    2. RB-24: the driver and a fixture command root that lists a directory, copies a file, runs itself as a child and exits with the child's status. It also touches the package renderer and the BS-15 twin from BS-15: the Rust package renderer as language-1.2 source #371.
    3. RB-25: the watchdog, tested with real signals.

    Each has its scenario, conformance_<id> test, planted defects and VERIFICATION.md entry, and each is one regeneration and one full gate. The rows skip PD-09 and RB-23 because draft #295 holds them.


    Generated by Claude Code

  7. afflom commented on Nov 4, 2026

    @afflom
    Member

    The Host declarations requested in #382 are now published on release/v0.4 through accepted #389, merge c613290: exact stdlib/src/Host.lex.tex. The combined delivery passed all44 final CI jobs plus honesty/repro, and the actual merge tree matches the tested tree. #382 is explicitly closed.

    This publishes the request/event/failure interface and its reviewed standard-library delivery. It does not claim the command driver or all remaining primitives/laws are implemented. This parent remains open; #419 and the other scoped work retain their existing owners and unfinished acceptance requirements. Please use the exact landed declarations when reconciling those sources rather than maintaining a guessed interface.

  8. added a commit that references this issue on Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestself-hostingLexLean compiling itself: compiler in .lex.tex, certified, built by its own pipeline

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions