From c21a0566082edf90a1628248f84adc84ebc74aa2 Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Mon, 9 Nov 2026 17:13:20 +0000 Subject: [PATCH 1/3] BS-26: the register row, scenario, and case of the LRE reader and the facts of a render MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The second slice of the lexicon as source (#269), on BS-25: the row (register 479 -> 480, the §31 table, the RP-07 check and its case), the scenario, the §17.23 prose, the CHANGELOG entry, the README's claim, the CI part, and the case, conformance_bs_26, which holds LexiconLre.parseLre and renderFacts to lexicon::lre::parse and the walkers of Render on hand-built vectors that reach every error of the reader and the limit on nesting, on every math render of the committed lexicon entries, and on seeded renders and edits, with 20 planted defects each named by its vector. Co-Authored-By: Claude Opus 5.5 --- .github/bdd-parts.json | 3 + .github/vv-rows.json | 9 + CHANGELOG.md | 6 + CONFORMANCE.md | 1 + README.md | 4 +- SPEC.md | 27 +- crates/conformance/src/cases/bootstrap.rs | 2 + .../src/cases/bootstrap/lexicon_lre.rs | 1020 +++++++++++++++++ crates/conformance/src/cases/repository.rs | 2 +- crates/conformance/tests/conformance.rs | 5 + features/suites/bootstrap.feature | 7 + model/ids.toml | 6 + xtask/src/spec_links.rs | 4 +- 13 files changed, 1090 insertions(+), 6 deletions(-) create mode 100644 crates/conformance/src/cases/bootstrap/lexicon_lre.rs diff --git a/.github/bdd-parts.json b/.github/bdd-parts.json index 9b99d3d59c..976d9ab839 100644 --- a/.github/bdd-parts.json +++ b/.github/bdd-parts.json @@ -22,6 +22,9 @@ "bootstrap-gnaf": [ "BS-16" ], + "bootstrap-lexicon-lre": [ + "BS-26" + ], "bootstrap-lexicon-lse": [ "BS-25" ], diff --git a/.github/vv-rows.json b/.github/vv-rows.json index d464cfef70..7f094cc320 100644 --- a/.github/vv-rows.json +++ b/.github/vv-rows.json @@ -62,6 +62,15 @@ "memory_kib": "", "hints": "" }, + { + "part": "bdd-bootstrap-lexicon-lre", + "recipes": "bdd", + "examples_part": "", + "bdd_part": "bootstrap-lexicon-lre", + "width": "", + "memory_kib": "", + "hints": "" + }, { "part": "bdd-bootstrap-lexicon-lse", "recipes": "bdd", diff --git a/CHANGELOG.md b/CHANGELOG.md index a6b6a71662..7593c11ab2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,12 @@ workspace carried at those versions without a tag. ## Unreleased +- The reader of LexLean Rendering Expressions and the facts of a render as + language-1.2 source, the second slice of the lexicon (SPEC.md §17.23, §13.9, + `BS-26`, #269; no new diagnostic code, no change to the language-1.2 + identity). `parseLre` and `renderFacts` of `LexiconLre` return what + `lexicon::lre::parse` and the walkers of `Render` return; neither admits + `recursion`. The register is 480 IDs. - The reader and the canonical printer of LexLean Semantic Expressions as language-1.2 source, the first slice of the lexicon (SPEC.md §17.23, §13.8, `BS-25`, #269; no new diagnostic code, no change to the language-1.2 diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 2b22435c0f..de80b7fcb3 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -78,6 +78,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `BS-16` | `build` | The GNAF request validation of SPEC.md §17.15 (`gnaf::validate`, with `universe_size`) is language-1.2 source in the `bootstrap/` project: `validateRequest` of `GnafValidate` is a production root eligible for `rust-std` with exactly its declared effects, over the projection of a request that validation reads (the length of the domain, the machine, the carrier's kind with each plan's preparations and the thresholds, the completeness evidence, the objective, the claim's class, and the scope) as first-order data, and returns the first violated rule in the model's order as the oracle's `serde_json` bytes, or nothing; on every host, its rendered `rust-std` package under the pinned rustc agrees with the Rust oracle `gnaf::validate` on the acceptance and on the rejection, to the byte, on the committed requests of `compiler/gnaf/`, the requests the committed certificate fixtures name, hand-built requests that reach each rejection, and seeded mutants of the committed requests; on the normative host, the root's lowered program under the calculus evaluator agrees with the oracle on the same corpus, each planted defect in the source is detected, and the certified package the production export publishes for the project is the committed one; loading a request (`gnaf::load`), the universe identity, the answer, and certification are outside the claim. | | `BS-20` | `build` | The canonical LaTeX of a tooling module of SPEC.md §17.20 (`backend::tooling::render_latex`: the fixed document frame and, for each declaration in order, its title, its kind, and the lines the Lean printer of the same file spells for its types, signatures, and terms, every line escaped by `tex_escape`) is language-1.2 source in the `bootstrap/` project: `renderToolingLatex` of `ToolingLatex`, over the tooling module as first-order data with each list carried as a chain of its own `Nil` and `Cons` constructors and with the typings the checker infers as an argument, writing through the standard library's `Buffer` with an explicit capacity, is a production root eligible for `rust-std` with exactly its declared effects; its rendered `rust-std` package under the pinned rustc and its lowered program under the calculus evaluator agree with the Rust oracle on the text, to the byte, and on the refusal of a write past the capacity, on the committed tooling modules (whose committed `.tex` is equal to the text), on hand-built modules that reach every declaration, type, signature, and term form and every byte class of the escaper, and on seeded mutants, every planted defect is detected, and the certified package the production export publishes for the project is the committed one; the checker, the decoder of the module, and the source map of the emitted pieces stay in the host. | | `BS-25` | `build` | The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim. | +| `BS-26` | `build` | The reading of LexLean Rendering Expressions of SPEC.md §13.9 and the facts of a render (`lexicon::lre::parse`: a text, tokenized and read as one S-expression by the reader of `BS-25`, bounded by the configured `max_scope_depth`, and converted into a render, a form or self-form reference, a slot, a sequence, a space, a renderer token, a group, parentheses, brackets, a subscript, a superscript, a fraction, or an operator name, or the kind and message of the first error, in the order the oracle finds them; and the walkers of `Render`: the slots, the renderer tokens, the form and self-form references and the operator names in occurrence order, the first construct the text channel cannot render, and the first script or fraction operand that renders nothing or is a bare script) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLre` over `LexiconLse`: `parseLre` and `renderFacts`, each eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `math` render of the committed lexicon entries read and walked, and on seeded renders and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the checks of a render against its entry (the slots against `surface_arity`, the tokens against the registry, the references against the glossary) stay in the entry validator and are outside the claim. | ## calculus diff --git a/README.md b/README.md index 9520a8b11e..0034829ad2 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ just vv # the complete normative acceptance gate (SPEC.md §9.2) just release # vv, then the §30 release criterion; refused until 1.0.0 ``` -All 479 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. +All 480 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. The differential against the independent generator `lean4-prod` (SPEC.md §17.19, `OR-01`..`OR-08`) acquires it at its pinned commit: from the checkout `LEXLEAN_ORACLE_DIR` names, or by fetching the commit with `git` into the Cargo target directory (`LEXLEAN_ORACLE_CACHE` overrides), and builds it with the pinned Lean and Rust toolchains. It is validation evidence only and no input to any identity. An oracle that cannot be acquired, is not the pinned files, or fails is unavailable: that fails the gate on Linux x86-64 and is printed as `NOT VALIDATED` elsewhere, and is never reported as agreement. @@ -197,7 +197,7 @@ Every row is validated by `just vv`; the IDs link the claim to its register row, | The language-1.2 authoring notation: a readable `.lex.tex` text for a semantic module, in bijection with its canonical JSON, whose every position and variant is read from the schema, with readable forms for types and terms and a generic form for every other value, formatted canonically by `fmt` failing closed, the only form in which a language-1.2 source of the product is written (the data form is refused under language 1.2, and `lexlean::sdk` is the explicit conversion), the same for the compiler-tooling modules in the generic notation, with no compiler-semantics ID that depends on which form an embedded module is written in, with no product source owned by a generator but five named records, read and written by programs only through the authoring API (`lexlean::sdk`), and, for a native core module, a readable program over the shared table of kernel terms whose numbering is the order of its text | `AU-01`..`AU-13` | `build` | | The runtime of generated Lean as native modules: the portable, collection, model, and reasoning runtimes of language 1.2 are LexLean source that every generated module imports, migrated losslessly and audited against their exact axioms, the released runtime text of languages 1.0 and 1.1 is frozen data equal to their released output, and no other Lean runtime text remains in the compiler but the decoder of core module 2 | `RT-01`..`RT-04` | `build` | | The standard library: language-1.2 source beside the compiler whose members are certified production roots, each validated against its oracle by a seeded differential of its evaluator and rustc realizations; bytes and text over `Bytes`, an output buffer of a given capacity written in order; decimal and hexadecimal; the byte order and stable sorts over it; canonical JSON reading and writing; maps and sets over byte keys; SHA-256 against FIPS 180-4, the NIST CAVP vectors, and the `sha2` crate; and the `std-models` and `std-reasoning` source packages, generic over types, whose laws are kernel-checked theorems and whose application is the `helpdesk` example | `SL-01`, `SL-02`, `SL-03`, `SL-04`, `SL-05`, `SL-09`, `SL-10`, `SL-11`, `SL-13`, `SL-14` | `build` | -| The bootstrap project: the compiler's own components as language-1.2 source compiled by LexLean's production pipeline to certified Rust packages, the exact axiom-output parser and its closed Lean-name check, the primitive scanner of §12.2, the preservation lexer and token audit, the target calculus's checker, the Rust crate validator, the Rust printer, the runtime syntax validator, the failure typing and the types of the Rust lowering, its last-use moves, its literal values and its terms, the package renderer, the GNAF request validation, the LaTeX of a tooling module, and the reader and printer of LexLean Semantic Expressions, each held to the Rust it replaces (the stage-0 oracle) by the committed vectors and a seeded differential of its evaluator and rustc realizations, with planted defects detected and the production export committed | `BS-01`..`BS-25` | `build` | +| The bootstrap project: the compiler's own components as language-1.2 source compiled by LexLean's production pipeline to certified Rust packages, the exact axiom-output parser and its closed Lean-name check, the primitive scanner of §12.2, the preservation lexer and token audit, the target calculus's checker, the Rust crate validator, the Rust printer, the runtime syntax validator, the failure typing and the types of the Rust lowering, its last-use moves, its literal values and its terms, the package renderer, the GNAF request validation, the LaTeX of a tooling module, the reader and printer of LexLean Semantic Expressions, and the reader of LexLean Rendering Expressions, each held to the Rust it replaces (the stage-0 oracle) by the committed vectors and a seeded differential of its evaluator and rustc realizations, with planted defects detected and the production export committed | `BS-01`..`BS-26` | `build` | | Source packages: a language-1.2 project imports the modules of a locked, content-addressed package of its workspace: `[[source_package]]` and `lexlean-workspace.toml`, the pins in `lexlean.lock` recomputed by every command, imports closed at link, no re-export of a package's production roots, and the package's modules replayed by the importer's `verify` with their origin in the records | `PK-01`..`PK-05` | `build` | | The literal `nat-add-zero` example, the Lean-verified feature examples, and the complete negative fixture suite | `EX-01`..`EX-09` | `build` | | Language-1.2 text components: tokenizers and their round-trip laws, with the laws carried as one native core module the generated modules import rather than authored Lean text | `LM-01`, `LM-02`, `LM-03`, `LM-13` | `build` | diff --git a/SPEC.md b/SPEC.md index 90b28595fa..083a9e3247 100644 --- a/SPEC.md +++ b/SPEC.md @@ -8760,6 +8760,30 @@ its type, value, and body, as the oracle's work stack does. The scope check agai universes, the conservative checker of signatures and values, and the rendering expressions (§13.9) stay in the host. +The reader of LexLean Rendering Expressions (§13.9) and the facts a render yields are the +second component of the lexicon as source (`BS-26`, issue 269), the module `LexiconLre` over +`LexiconLse`, with two production roots for `rust-std`: + +| Root | Takes | Returns | Oracle | +| --- | --- | --- | --- | +| `LexLeanBoot.LexiconLre.parseLre` | a text and the depth bound | the render, or the kind and message of the first error | `lexicon::lre::parse` | +| `LexLeanBoot.LexiconLre.renderFacts` | a render and its number of nodes | its slots, tokens, form and self-form references, operator names, first math-only construct, and first script-operand problem | the walkers of `Render` | + +`parseLre` tokenizes and reads the text with `BS-25`'s tokenizer and tree reader, so the lexical +and structural errors are theirs, and converts the tree with a work stack that visits the +operands of a form in the oracle's order (the first operand of a `sub`, `sup`, or `frac` before +the second) and builds the form when they are on the value stack, so it admits no `recursion`. +A slot index is a canonical decimal within 32 bits; a form ID and a renderer-token ID are +`[a-z][a-z0-9-]*` and an operator name `[A-Za-z][A-Za-z0-9_]*`, each checked by a scan over the +bytes. `renderFacts` walks the render once, pre-order over an explicit stack whose steps the +host bounds by the number of nodes, and collects what the entry validator reads in one pass: the +slot indices, renderer tokens, form references, self-form references, and operator names in +occurrence order; the first `sub`, `sup`, `frac`, or `operator-name`, which the text channel +cannot render; and the first operand of a `sub`, `sup`, or `frac` that renders nothing (a space, +or a sequence of nothing but spaces, found by a bounded walk of its own) or that is a bare script +under a `sub` or `sup`, with the oracle's message. The checks of those facts against the entry +and the glossary stay in the host. + ## 18. Lean backend ### 18.1 Output contract @@ -11682,6 +11706,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `BS-16` | `bootstrap` | The GNAF request validation of SPEC.md §17.15 (`gnaf::validate`, with `universe_size`) is language-1.2 source in the `bootstrap/` project: `validateRequest` of `GnafValidate` is a production root eligible for `rust-std` with exactly its declared effects, over the projection of a request that validation reads (the length of the domain, the machine, the carrier's kind with each plan's preparations and the thresholds, the completeness evidence, the objective, the claim's class, and the scope) as first-order data, and returns the first violated rule in the model's order as the oracle's `serde_json` bytes, or nothing; on every host, its rendered `rust-std` package under the pinned rustc agrees with the Rust oracle `gnaf::validate` on the acceptance and on the rejection, to the byte, on the committed requests of `compiler/gnaf/`, the requests the committed certificate fixtures name, hand-built requests that reach each rejection, and seeded mutants of the committed requests; on the normative host, the root's lowered program under the calculus evaluator agrees with the oracle on the same corpus, each planted defect in the source is detected, and the certified package the production export publishes for the project is the committed one; loading a request (`gnaf::load`), the universe identity, the answer, and certification are outside the claim. | §17.15, §17.23 | | `BS-20` | `bootstrap` | The canonical LaTeX of a tooling module of SPEC.md §17.20 (`backend::tooling::render_latex`: the fixed document frame and, for each declaration in order, its title, its kind, and the lines the Lean printer of the same file spells for its types, signatures, and terms, every line escaped by `tex_escape`) is language-1.2 source in the `bootstrap/` project: `renderToolingLatex` of `ToolingLatex`, over the tooling module as first-order data with each list carried as a chain of its own `Nil` and `Cons` constructors and with the typings the checker infers as an argument, writing through the standard library's `Buffer` with an explicit capacity, is a production root eligible for `rust-std` with exactly its declared effects; its rendered `rust-std` package under the pinned rustc and its lowered program under the calculus evaluator agree with the Rust oracle on the text, to the byte, and on the refusal of a write past the capacity, on the committed tooling modules (whose committed `.tex` is equal to the text), on hand-built modules that reach every declaration, type, signature, and term form and every byte class of the escaper, and on seeded mutants, every planted defect is detected, and the certified package the production export publishes for the project is the committed one; the checker, the decoder of the module, and the source map of the emitted pieces stay in the host. | §17.20, §17.23 | | `BS-25` | `bootstrap` | The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim. | §17.23, §13.8 | +| `BS-26` | `bootstrap` | The reading of LexLean Rendering Expressions of SPEC.md §13.9 and the facts of a render (`lexicon::lre::parse`: a text, tokenized and read as one S-expression by the reader of `BS-25`, bounded by the configured `max_scope_depth`, and converted into a render, a form or self-form reference, a slot, a sequence, a space, a renderer token, a group, parentheses, brackets, a subscript, a superscript, a fraction, or an operator name, or the kind and message of the first error, in the order the oracle finds them; and the walkers of `Render`: the slots, the renderer tokens, the form and self-form references and the operator names in occurrence order, the first construct the text channel cannot render, and the first script or fraction operand that renders nothing or is a bare script) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLre` over `LexiconLse`: `parseLre` and `renderFacts`, each eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `math` render of the committed lexicon entries read and walked, and on seeded renders and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the checks of a render against its entry (the slots against `surface_arity`, the tokens against the registry, the references against the glossary) stay in the entry validator and are outside the claim. | §17.23, §13.9 | | `PK-01` | `packages` | A language-1.2 project that declares a source package by its identifier, version, kind `path`, and the member directory a `lexlean-workspace.toml` lists locks it by the tree digest of its manifest and source files, its manifest digest, its version, its module prefix, and its own packages, and the configuration and the lock read back to the bytes written; a project without a source package keeps its bytes, and a package of another kind, a source package in another language, a changed byte or version (LLC0105), an undeclared package (LLC0106), a module prefix claimed twice (LLC0107), a project or a package outside the workspace (LLC0108), a path that escapes it, and a symlink (LLS8001) are each refused with the named code. | §10.5, §11.1, §25.1 | | `PK-02` | `packages` | A module of a project imports only modules of its own project and modules of the packages the project declares, and a module of a package only modules of itself and of the packages its own manifest declares: a package module is a module of the importing project under its own module prefix, a module of a package the project does not declare fails with LLR3009, and an import cycle through a package fails with LLR3003. | §10.5, §15.1 | | `PK-03` | `packages` | An importing project builds and verifies under the pinned Lean offline, and its package modules are generated, elaborated, replayed with `leanchecker`, and audited for their exact axioms under the package's module prefix with the package as their origin in the records; the source ID of the project changes with the pins, and no file of the package is written by `fmt`, `build`, or `verify`. | §10.5, §21.3, §22 | @@ -11698,7 +11723,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `LM-09` | `language-models` | No prompt is code: no member of the closed language maps `string` or `bytes` to a function, a term, or source — every operation of the primitive registry that takes a `string` or `bytes` operand is classified by the audit with a data result, the registry's schema enumeration and the production registry's primitive rows equal one another and equal that classified set, and the text declarations of `lexlean/semantic-module/2` state no `string`-typed member, so a prompt reaches a model only as the tokens `encodeText` makes of it; a planted operation or text member of that shape is caught by the audit with its name. | §17.12 | | `LM-13` | `language-models` | The text laws a tokenizer elaborates through are LexLean source and not authored text: the module that states them is one language-1.2 native core module of the embedded `language/text-1.2` project, the project holds no file of Lean source, no independently authored Lean file anywhere in the release tree declares the `LexLeanText` namespace or lies under its module root, and the one-time migration input is absent; the registry lists only that module, the declarations and their exact axiom policies are read from its native data, which the compiler embeds and hashes into the language-1.2 semantics, and a module that declares a text component imports the generated module rather than splicing any Lean text. | §17.10, §17.12 | -**Total required capability IDs:** 479. +**Total required capability IDs:** 480. No row may be downgraded to `some-true` or `open`. Upstream Lean facts are ledger/authority rows, not substitutions for these build behaviors. diff --git a/crates/conformance/src/cases/bootstrap.rs b/crates/conformance/src/cases/bootstrap.rs index 0303f0fa7a..aeb8aa3c41 100644 --- a/crates/conformance/src/cases/bootstrap.rs +++ b/crates/conformance/src/cases/bootstrap.rs @@ -22,6 +22,7 @@ use crate::rust_packages; use crate::support::{self, P}; mod gnaf_validate; +mod lexicon_lre; mod lexicon_lse; mod runtime_validate; mod rust_assemble; @@ -2459,6 +2460,7 @@ fn case(id: &str) { // §17.23, §17.20: the canonical LaTeX of a tooling module. "BS-20" => tooling_latex::case(), "BS-25" => lexicon_lse::case(), + "BS-26" => lexicon_lre::case(), other => panic!("no bootstrap case is wired for {other}"), } } diff --git a/crates/conformance/src/cases/bootstrap/lexicon_lre.rs b/crates/conformance/src/cases/bootstrap/lexicon_lre.rs new file mode 100644 index 0000000000..3f98975e44 --- /dev/null +++ b/crates/conformance/src/cases/bootstrap/lexicon_lre.rs @@ -0,0 +1,1020 @@ +//! `BS-26` (SPEC.md §17.23, §13.9): the reader of LexLean Rendering Expressions +//! and the facts the entry validator reads from a render, as language-1.2 +//! source, `LexLeanBoot.LexiconLre`, held to the byte to +//! `lexlean::lexicon::lre::parse` and the walkers of `Render`. +//! +//! `parseLre` tokenizes and reads a text with the reader of `BS-25` and +//! converts the tree with a work stack, so that it admits no `recursion`. +//! `renderFacts` walks a render once, pre-order over an explicit stack, and +//! returns what `slots`, `tokens`, `form_refs`, `self_form_refs`, +//! `operator_names`, `math_only_construct`, and `script_operand_error` return; +//! the host passes the number of nodes, which bounds the walk. +//! +//! Each planted defect names the one vector that catches it. + +use std::collections::{BTreeMap, BTreeSet}; +use std::sync::OnceLock; + +use lexlean::calculus::Value; +use lexlean::lexicon::lre::{parse, Render}; +use lexlean::lexicon::lse::{ParseError, QualifiedId}; +use serde_json::Value as Json2; + +use super::rust_validate::{eligibility_of, rustc_differential}; +use super::{list, option, pair, text, Run, Seeded}; +use crate::production_export as harness; +use crate::support::{self, repo_root}; + +const MODULE: &str = "LexLeanBoot.LexiconLre"; +const SOURCE: &str = "bootstrap/src/LexiconLre.lex.tex"; +const ROOTS: [(&str, &[&str]); 2] = [ + ("parseLre", &["allocation", "overflow"]), + ("renderFacts", &["allocation", "overflow"]), +]; + +/// The most a part of a certificate of a root may be, as the row states. +const CERTIFICATE_BUDGET: u64 = 3_145_728; +/// The depth the committed renders and the seeded ones are read at. +const DEPTH: u64 = 64; +/// Seeded renders under the evaluator, and under rustc. +const EVALUATOR_SEEDED: usize = 300; +const RUSTC_SEEDED: usize = 1_500; + +// ---------------------------------------------------------------- the sorts + +/// The constructors of the module's `Render`, in declaration order, read from +/// its text. +fn constructors() -> &'static Vec { + static TABLE: OnceLock> = OnceLock::new(); + TABLE.get_or_init(|| { + let source = std::fs::read_to_string(repo_root().join(SOURCE).as_std_path()) + .expect("the module's source"); + let block = source + .split("inductive{") + .nth(1) + .and_then(|rest| rest.split("\n}\n").next()) + .expect("the first inductive"); + assert!( + block.contains("name: Render,"), + "the first inductive is Render" + ); + block + .split("{name: ") + .skip(1) + .map(|rest| { + rest.split(',') + .next() + .expect("a constructor") + .trim() + .to_owned() + }) + .collect() + }) +} + +fn adt(constructor: &str, fields: Vec) -> Value { + let found = constructors() + .iter() + .position(|name| name == constructor) + .unwrap_or_else(|| panic!("`Render` has no constructor `{constructor}`")); + Value::Adt { + constructor: u64::try_from(found).expect("an index"), + fields, + } +} + +// ------------------------------------------------------------- the encoders + +fn nat_u64(value: u64) -> Value { + Value::Nat { + value: value.to_string(), + } +} + +/// A list of renders as the module's chain of `RItem` cells. +fn chain(items: &[Render]) -> Value { + items + .iter() + .rev() + .fold(adt("RNil", Vec::new()), |rest, item| { + adt("RItem", vec![enc(item), rest]) + }) +} + +/// A render as the module's `Render`. +fn enc(render: &Render) -> Value { + match render { + Render::Form { entry, form } => adt( + "RForm", + vec![text(&entry.package), text(&entry.entry), text(form)], + ), + Render::SelfForm(form) => adt("RSelfForm", vec![text(form)]), + Render::Slot(index) => adt("RSlot", vec![nat_u64(u64::from(*index))]), + Render::Seq(items) => adt("RSeq", vec![chain(items)]), + Render::Space => adt("RSpace", Vec::new()), + Render::Token(id) => adt("RToken", vec![text(id)]), + Render::Group(inner) => adt("RGroup", vec![enc(inner)]), + Render::Paren(inner) => adt("RParen", vec![enc(inner)]), + Render::Bracket(inner) => adt("RBracket", vec![enc(inner)]), + Render::Sub(a, b) => adt("RSub", vec![enc(a), enc(b)]), + Render::Sup(a, b) => adt("RSup", vec![enc(a), enc(b)]), + Render::Frac(a, b) => adt("RFrac", vec![enc(a), enc(b)]), + Render::OperatorName(name) => adt("ROperatorName", vec![text(name)]), + } +} + +/// The number of nodes of a render, the bound the host passes to the walk. +fn nodes(render: &Render) -> usize { + let mut count = 0; + let mut stack = vec![render]; + while let Some(node) = stack.pop() { + count += 1; + match node { + Render::Seq(items) => stack.extend(items.iter()), + Render::Group(inner) | Render::Paren(inner) | Render::Bracket(inner) => { + stack.push(inner); + } + Render::Sub(a, b) | Render::Sup(a, b) | Render::Frac(a, b) => { + stack.push(a); + stack.push(b); + } + _ => {} + } + } + count +} + +fn texts(items: &[String]) -> Value { + list(items.iter().map(|item| text(item)).collect()) +} + +/// What the oracle's walkers say of a render, as the module's `Facts`. +fn facts(render: &Render) -> Value { + Value::Adt { + constructor: 0, + fields: vec![ + list( + render + .slots() + .iter() + .map(|index| nat_u64(u64::from(*index))) + .collect(), + ), + texts(&render.tokens()), + list( + render + .form_refs() + .iter() + .map(|(entry, form): &(QualifiedId, String)| { + pair(text(&entry.package), pair(text(&entry.entry), text(form))) + }) + .collect(), + ), + texts(&render.self_form_refs()), + texts(&render.operator_names()), + option(render.math_only_construct().map(text)), + option(render.script_operand_error().as_deref().map(text)), + ], + } +} + +/// The run of `parseLre` on a text at a depth, and the oracle's message when +/// it refuses. +fn parse_run(source: &str, depth: u64) -> (Run, Option) { + let (value, refusal) = match parse(source, depth) { + Ok(render) => ( + Value::Ok { + value: Box::new(enc(&render)), + }, + None, + ), + Err(error) => { + let kind = match error { + ParseError::Syntax(_) => 0, + ParseError::DepthExceeded(_) => 1, + }; + let message = error.to_string(); + ( + Value::Error { + value: Box::new(Value::Adt { + constructor: 0, + fields: vec![nat_u64(kind), text(&message)], + }), + }, + Some(message), + ) + } + }; + ( + ( + vec![text(source), nat_u64(depth)], + serde_json::to_value(value).expect("a value is JSON"), + ), + refusal, + ) +} + +/// The run of `renderFacts` on a render. +fn facts_run(render: &Render) -> Run { + ( + vec![ + enc(render), + nat_u64(u64::try_from(nodes(render)).expect("a count")), + ], + serde_json::to_value(facts(render)).expect("a value is JSON"), + ) +} + +// ------------------------------------------------------------------ vectors + +enum Subject { + Parse(String, u64), + Facts(Render), +} + +struct Vector { + label: String, + subject: Subject, +} + +impl Vector { + fn walking(&self) -> bool { + matches!(self.subject, Subject::Facts(_)) + } + + fn run(&self) -> (Run, Option) { + match &self.subject { + Subject::Parse(source, depth) => parse_run(source, *depth), + Subject::Facts(render) => (facts_run(render), None), + } + } +} + +fn parsed(source: &str) -> Render { + parse(source, DEPTH).unwrap_or_else(|error| panic!("`{source}` parses: {error}")) +} + +#[allow(clippy::too_many_lines)] +fn named() -> Vec { + let mut out: Vec = Vec::new(); + let mut add = |label: &str, subject: Subject| { + out.push(Vector { + label: label.to_owned(), + subject, + }); + }; + let reading = |source: &str| Subject::Parse(source.to_owned(), DEPTH); + + // Reading: what is accepted. + for (label, source) in [ + ("a form", "(form lexlean.core::the the-l)"), + ("a form of a nested entry", "(form a.b::c.d e-f9)"), + ("a self form", "(self-form x)"), + ("a self form with digits and hyphens", "(self-form x0-y)"), + ("a slot", "(slot 0)"), + ("a slot of many digits", "(slot 4294967295)"), + ("a space", "(space)"), + ("a token", "(token plus)"), + ("a token with digits and hyphens", "(token comma-1)"), + ("a group", "(group (slot 0))"), + ("a paren", "(paren (slot 0))"), + ("a bracket", "(bracket (slot 0))"), + ("a sub", "(sub (slot 0) (slot 1))"), + ("a sup", "(sup (slot 0) (slot 1))"), + ("a frac", "(frac (slot 0) (slot 1))"), + ("an operator name", "(operator-name succ_1)"), + ("an operator name of capitals", "(operator-name Succ)"), + ("a seq of one", "(seq (slot 0))"), + ("a seq", "(seq (slot 0) (space) (token plus) (space) (slot 1))"), + ( + "every production", + "(seq (form lexlean.core::the the-l) (space) (self-form x) (slot 0) (token plus) (group (slot 1)) (paren (slot 2)) (bracket (slot 3)) (sub (slot 4) (slot 5)) (sup (slot 6) (slot 7)) (frac (slot 8) (slot 9)) (operator-name succ_1))", + ), + ("white space of both kinds", "(seq\n(slot 0) (space)\n)"), + ("atoms touching parentheses", "(seq(slot 0)(space))"), + ( + "the oracle's own render", + "(seq (operator-name mulassoc) (paren (seq (slot 0) (token comma) (space) (slot 1) (token comma) (space) (slot 2))))", + ), + ] { + add(label, reading(source)); + } + + // Reading: the limit on nesting. + add( + "nesting at the depth", + Subject::Parse("(group (slot 0))".to_owned(), 2), + ); + add( + "nesting beyond the depth", + Subject::Parse("(group (slot 0))".to_owned(), 1), + ); + + // Reading: what is refused. + for (label, source) in [ + ("the empty text", ""), + ("an unbalanced close", ")"), + ("an unclosed open", "(seq (slot 0)"), + ("trailing tokens", "(slot 0) (slot 1)"), + ("a tab", "(slot\t0)"), + ("a bare atom", "x"), + ("an empty list", "()"), + ("a list headed by a list", "((slot 0))"), + ("an unknown form", "(raw \\relax)"), + ("a form of one", "(form a::b)"), + ("a form of three", "(form a::b c d)"), + ("a form whose entry is a list", "(form (a::b) c)"), + ("a form with a bad entry", "(form nat the-l)"), + ("a form whose form is a list", "(form a::b (c))"), + ("a form with a bad form ID", "(form a::b Bad)"), + ("a self form of nothing", "(self-form)"), + ("a self form that is a list", "(self-form (x))"), + ("a self form with a bad ID", "(self-form Bad)"), + ("a slot of nothing", "(slot)"), + ("a slot that is a list", "(slot (0))"), + ("a slot with a leading zero", "(slot 01)"), + ("a slot with a sign", "(slot +1)"), + ("a slot of a word", "(slot x)"), + ("a slot beyond 32 bits", "(slot 4294967296)"), + ("a slot beyond 64 bits", "(slot 18446744073709551616)"), + ("a space with an operand", "(space x)"), + ("a token of nothing", "(token)"), + ("a token that is a list", "(token (plus))"), + ("a token with a bad ID", "(token \\relax)"), + ("a token of capitals", "(token Plus)"), + ("an operator name of nothing", "(operator-name)"), + ("an operator name that is a list", "(operator-name (x))"), + ("an operator name with a hyphen", "(operator-name a-b)"), + ("an operator name of a digit first", "(operator-name 1a)"), + ("an empty seq", "(seq)"), + ("a group of two", "(group (slot 0) (slot 1))"), + ("a paren of nothing", "(paren)"), + ("a bracket of two", "(bracket (slot 0) (slot 1))"), + ("a sub of one", "(sub (slot 0))"), + ("a sup of three", "(sup (slot 0) (slot 1) (slot 2))"), + ("a frac of nothing", "(frac)"), + ("a bad item inside a seq", "(seq (slot 0) x)"), + ( + "the first operand's error comes before the second's", + "(sub (foo) (bar))", + ), + ("a bad item inside a group", "(group (bar))"), + ] { + add(label, reading(source)); + } + + // The facts of renders. + for (label, source) in [ + ("the facts of every production", "(seq (form lexlean.core::the the-l) (space) (self-form x) (slot 0) (token plus) (group (slot 1)) (paren (slot 2)) (bracket (slot 3)) (sub (slot 4) (slot 5)) (sup (slot 6) (slot 7)) (frac (slot 8) (slot 9)) (operator-name succ_1))"), + ("the facts of a slot", "(slot 3)"), + ("the facts of a space", "(space)"), + ("slots in order", "(seq (slot 2) (group (seq (slot 0) (slot 1))) (slot 2))"), + ("tokens and forms in order", "(seq (token b) (form p::e f) (self-form g) (token a) (form q::r s))"), + ("a sub is math only", "(seq (slot 0) (sub (slot 1) (slot 2)))"), + ("a frac before an operator name", "(seq (frac (slot 0) (slot 1)) (operator-name f))"), + ("an operator name before a sup", "(seq (operator-name f) (sup (slot 0) (slot 1)))"), + ("a sub whose script is a space", "(sub (slot 0) (space))"), + ("a sup whose base renders nothing", "(sup (seq (space) (space)) (slot 0))"), + ("a frac whose operands render", "(frac (seq (space) (slot 0)) (slot 1))"), + ("a sup whose script is a sub", "(sup (slot 0) (sub (slot 1) (slot 2)))"), + ("a sub whose base is a sup", "(sub (sup (slot 0) (slot 1)) (slot 2))"), + ("a grouped nested script", "(sup (slot 0) (group (sub (slot 1) (slot 2))))"), + ("a frac of scripts", "(frac (sub (slot 0) (slot 1)) (sup (slot 2) (slot 3)))"), + ("the first problem of two", "(seq (sub (slot 0) (sub (slot 1) (slot 2))) (sup (slot 3) (space)))"), + ("a sub whose operands fail differently", "(sub (space) (sub (slot 1) (slot 2)))"), + ("a problem inside a group", "(group (seq (slot 0) (sub (space) (slot 1))))"), + ("a nested seq of spaces", "(frac (seq (seq (space)) (space)) (slot 0))"), + ] { + add(label, Subject::Facts(parsed(source))); + } + out +} + +// ------------------------------------------------------------------- seeded + +const TOKENS: &[&str] = &["plus", "comma", "times-1", "eq"]; +const FORMS: &[(&str, &str)] = &[("lexlean.core", "the"), ("p.q-r", "s.t-u")]; +const FORM_IDS: &[&str] = &["the-l", "x", "f0"]; +const OPERATORS: &[&str] = &["succ_1", "mulassoc", "Op"]; + +fn drawn(seeded: &mut Seeded, depth: usize) -> Render { + let kind = if depth == 0 { + seeded.below(6) + } else { + seeded.below(13) + }; + match kind { + 0 => { + let (package, entry) = *seeded.pick(FORMS); + Render::Form { + entry: QualifiedId::parse(&format!("{package}::{entry}")).expect("an ID"), + form: (*seeded.pick(FORM_IDS)).to_owned(), + } + } + 1 => Render::SelfForm((*seeded.pick(FORM_IDS)).to_owned()), + 2 => Render::Slot(u32::try_from(seeded.below(4)).expect("a slot")), + 3 => Render::Space, + 4 => Render::Token((*seeded.pick(TOKENS)).to_owned()), + 5 => Render::OperatorName((*seeded.pick(OPERATORS)).to_owned()), + 6 | 7 => { + let count = 1 + seeded.below(4); + Render::Seq((0..count).map(|_| drawn(seeded, depth - 1)).collect()) + } + 8 => Render::Group(Box::new(drawn(seeded, depth - 1))), + 9 => Render::Paren(Box::new(drawn(seeded, depth - 1))), + 10 => Render::Bracket(Box::new(drawn(seeded, depth - 1))), + 11 => Render::Sub( + Box::new(drawn(seeded, depth - 1)), + Box::new(drawn(seeded, depth - 1)), + ), + _ => { + if seeded.chance(50) { + Render::Sup( + Box::new(drawn(seeded, depth - 1)), + Box::new(drawn(seeded, depth - 1)), + ) + } else { + Render::Frac( + Box::new(drawn(seeded, depth - 1)), + Box::new(drawn(seeded, depth - 1)), + ) + } + } + } +} + +/// The text of a render, as the entries write them. +fn written(render: &Render) -> String { + match render { + Render::Form { entry, form } => format!("(form {entry} {form})"), + Render::SelfForm(form) => format!("(self-form {form})"), + Render::Slot(index) => format!("(slot {index})"), + Render::Seq(items) => format!( + "(seq {})", + items.iter().map(written).collect::>().join(" ") + ), + Render::Space => "(space)".to_owned(), + Render::Token(id) => format!("(token {id})"), + Render::Group(inner) => format!("(group {})", written(inner)), + Render::Paren(inner) => format!("(paren {})", written(inner)), + Render::Bracket(inner) => format!("(bracket {})", written(inner)), + Render::Sub(a, b) => format!("(sub {} {})", written(a), written(b)), + Render::Sup(a, b) => format!("(sup {} {})", written(a), written(b)), + Render::Frac(a, b) => format!("(frac {} {})", written(a), written(b)), + Render::OperatorName(name) => format!("(operator-name {name})"), + } +} + +const PIECES: &[&str] = &[ + "(", + ")", + " ", + "\n", + "seq", + "slot", + "space", + "token", + "group", + "sub", + "frac", + "form", + "x", + "0", + "01", + "::", + "-", + "(slot 0)", + "(space)", + "(sub (slot 0) (slot 1))", +]; + +fn edited(seeded: &mut Seeded, source: &str) -> String { + let mut scalars: Vec = source.chars().collect(); + for _ in 0..=seeded.below(3) { + let at = seeded.below(scalars.len() + 1); + let piece: Vec = seeded.pick(PIECES).chars().collect(); + match seeded.below(3) { + 0 if at < scalars.len() => { + scalars.remove(at); + } + 1 => { + drop(scalars.splice(at..at, piece)); + } + _ if at < scalars.len() => { + drop(scalars.splice(at..=at, piece)); + } + _ => scalars.extend(piece), + } + } + scalars.into_iter().collect() +} + +/// `count` seeded renders: the runs of `renderFacts` on them and of `parseLre` +/// on their texts and on edits of those. +fn seeded_runs(count: usize, bases: &[String]) -> (Vec, Vec) { + let mut seeded = Seeded(0x0B55_0026); + let mut walking = Vec::with_capacity(count); + let mut reading = Vec::with_capacity(2 * count); + while walking.len() < count { + let depth = 1 + seeded.below(4); + let render = drawn(&mut seeded, depth); + let source = written(&render); + walking.push(facts_run(&render)); + reading.push(parse_run(&source, DEPTH).0); + let base = if seeded.chance(30) && !bases.is_empty() { + seeded.pick(bases).clone() + } else { + source + }; + reading.push(parse_run(&edited(&mut seeded, &base), DEPTH).0); + } + (walking, reading) +} + +/// Every render a committed lexicon entry carries: the `math` field of the +/// `[render]` table of every `entries/*.toml`, with its path. +fn committed() -> Vec<(String, String)> { + let root = repo_root(); + let walker = walkdir::WalkDir::new(root.as_std_path()) + .sort_by_file_name() + .into_iter() + .filter_entry(|entry| { + let name = entry.file_name().to_string_lossy(); + !(entry.file_type().is_dir() + && (matches!(name.as_ref(), "target" | ".git" | ".lexlean") + || support::holds_cache_tag(entry))) + }); + let mut out = Vec::new(); + for entry in walker.flatten() { + let path = entry.path(); + let shown = path.to_string_lossy().replace('\\', "/"); + if !entry.file_type().is_file() || !shown.ends_with(".toml") || !shown.contains("/entries/") + { + continue; + } + let Ok(content) = std::fs::read_to_string(path) else { + continue; + }; + let relative = path + .strip_prefix(root.as_std_path()) + .expect("under the root") + .to_string_lossy() + .replace('\\', "/"); + for line in content.lines() { + for field in ["math = \"", "text = \""] { + if let Some(rest) = line.strip_prefix(field) { + if let Some(inner) = rest.strip_suffix('"') { + if inner.starts_with('(') { + out.push((relative.clone(), inner.to_owned())); + } + } + } + } + } + } + out +} + +// ------------------------------------------------------------------- plants + +struct Plant { + label: &'static str, + root: usize, + site: &'static str, + replacement: &'static str, + vector: &'static str, +} + +const PLANTS: &[Plant] = &[ + Plant { + label: "a form takes three", + root: 0, + site: "(if ((prim length arguments : Nat) =? 2) then (match (LexiconLse::atomOf (LexiconLse::nodeAt arguments 0) (LexiconLse::lit \"a qualified ID\") fuel)", + replacement: "(if (2 <=? (prim length arguments : Nat)) then (match (LexiconLse::atomOf (LexiconLse::nodeAt arguments 0) (LexiconLse::lit \"a qualified ID\") fuel)", + vector: "a form of three", + }, + Plant { + label: "a form ID need not be an ID", + root: 0, + site: "| Result.ok form => (if (isFormId form) then (rvPush st rest (ctor Render.RForm (first parts) (second parts) form))", + replacement: "| Result.ok form => (if true then (rvPush st rest (ctor Render.RForm (first parts) (second parts) form))", + vector: "a form with a bad form ID", + }, + Plant { + label: "a token ID admits a capital", + root: 0, + site: "(if (LexiconLse::isLowerNum (LexiconLse::byteNum text 0)) then ((first (prim iterate_until (fun [n, text] (position : Nat) => (scanTokenId text n position)) n 1 : (Nat * Bool))) =? n) else false)", + replacement: "(if (LexiconLse::isAlphaNum (LexiconLse::byteNum text 0)) then ((first (prim iterate_until (fun [n, text] (position : Nat) => (scanTokenId text n position)) n 1 : (Nat * Bool))) =? n) else false)", + vector: "a token of capitals", + }, + Plant { + label: "an operator name admits a hyphen", + root: 0, + site: "(if (if (LexiconLse::isAlnumNum value) then true else (value =? 95)) then", + replacement: "(if (if (LexiconLse::isAlnumNum value) then true else (if (value =? 95) then true else (value =? 45))) then", + vector: "an operator name with a hyphen", + }, + Plant { + label: "a slot index may exceed 32 bits", + root: 0, + site: "(if (4294967295 (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted digits) (LexiconLse::lit \" is not a slot index\"))))", + replacement: "| Option.some fault => (rvPush st rest (ctor Render.RSlot (LexiconLse::decimalValue digits)))", + vector: "a slot with a leading zero", + }, + Plant { + label: "a space takes an operand", + root: 0, + site: "(if (argc =? 0) then (rvPush st rest (ctor Render.RSpace))", + replacement: "(if (argc <=? 1) then (rvPush st rest (ctor Render.RSpace))", + vector: "a space with an operand", + }, + Plant { + label: "an empty seq is a seq", + root: 0, + site: "(if (argc =? 0) then (rvFail st (LexiconLse::failed (LexiconLse::lit \"`seq` has at least one item\")))", + replacement: "(if false then (rvFail st (LexiconLse::failed (LexiconLse::lit \"`seq` has at least one item\")))", + vector: "an empty seq", + }, + Plant { + label: "a group takes two", + root: 0, + site: "(if (argc =? 1) then (record Rv {work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RBuild head 1) rest))", + replacement: "(if (1 <=? argc) then (record Rv {work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RBuild head 1) rest))", + vector: "a group of two", + }, + Plant { + label: "a sub takes one", + root: 0, + site: "(if (argc =? 2) then (record Rv {work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 1))", + replacement: "(if (1 <=? argc) then (record Rv {work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 1))", + vector: "a sub of one", + }, + Plant { + label: "a sup is a sub", + root: 0, + site: "(if (prim equal head (LexiconLse::lit \"sub\") : Bool) then (ctor Render.RSub left right) else", + replacement: "(if (isBinary head) then (ctor Render.RSub left right) else", + vector: "a sup", + }, + Plant { + label: "a paren is a group", + root: 0, + site: "(if (prim equal head (LexiconLse::lit \"paren\") : Bool) then (ctor Render.RParen inner) else", + replacement: "(if false then (ctor Render.RParen inner) else", + vector: "a paren", + }, + Plant { + label: "the second operand is visited first", + root: 0, + site: "work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 1)) (cons (ctor RWork.RBuild head 2) rest))", + replacement: "work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 1)) (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RBuild head 2) rest))", + vector: "the first operand's error comes before the second's", + }, + Plant { + label: "the slots come last first", + root: 1, + site: "slots := (LexLeanStd.Order::reverseList [Nat] (project facts slots))", + replacement: "slots := (project facts slots)", + vector: "slots in order", + }, + Plant { + label: "the tokens come last first", + root: 1, + site: "tokens := (LexLeanStd.Order::reverseList [Bytes] (project facts tokens))", + replacement: "tokens := (project facts tokens)", + vector: "tokens and forms in order", + }, + Plant { + label: "a frac is not math only", + root: 1, + site: "| Render.RFrac a b => (withFacts (cons a (cons b rest)) (binaryFacts facts (LexiconLse::lit \"frac\") a b fuel))", + replacement: "| Render.RFrac a b => (withFacts (cons a (cons b rest)) (record Facts {slots := (project facts slots), tokens := (project facts tokens), formRefs := (project facts formRefs), selfForms := (project facts selfForms), operatorNames := (project facts operatorNames), mathOnly := (project facts mathOnly), scriptError := (firstProblem (project facts scriptError) (scriptProblem (LexiconLse::lit \"frac\") a b fuel))}))", + vector: "a frac before an operator name", + }, + Plant { + label: "the last math construct wins", + root: 1, + site: " body: (match current\n | Option.some kept => current\n | Option.none => (ctor Option.some [Bytes] candidate)),", + replacement: " body: (ctor Option.some [Bytes] candidate),", + vector: "an operator name before a sup", + }, + Plant { + label: "a seq of spaces renders something", + root: 1, + site: "| Render.RSeq items => (ctor Option.some [((List Render) * Nat)] ((prim list_fold (fun (acc : (List Render)) (item : Render) => (cons item acc)) rest (renderItems items fuel) : (List Render)), 0))", + replacement: "| Render.RSeq items => (ctor Option.some [((List Render) * Nat)] (rest, 1))", + vector: "a sup whose base renders nothing", + }, + Plant { + label: "a frac's script may be a script", + root: 1, + site: "(if (if (prim equal name (LexiconLse::lit \"frac\") : Bool) then false else (isScript operand)) then", + replacement: "(if false then", + vector: "a sup whose script is a sub", + }, + Plant { + label: "the second operand is checked first", + root: 1, + site: " body: (match (operandProblem name left fuel)\n | Option.some problem => (ctor Option.some [Bytes] problem)\n | Option.none => (operandProblem name right fuel)),", + replacement: " body: (match (operandProblem name right fuel)\n | Option.some problem => (ctor Option.some [Bytes] problem)\n | Option.none => (operandProblem name left fuel)),", + vector: "a sub whose operands fail differently", + }, +]; + +fn plants(vectors: &[Vector]) { + let mut missed = Vec::new(); + for plant in PLANTS { + let vector = vectors + .iter() + .find(|vector| vector.label == plant.vector) + .unwrap_or_else(|| panic!("{}: no vector `{}`", plant.label, plant.vector)); + assert_eq!( + vector.walking(), + plant.root == 1, + "{}: the vector is a run of the root planted", + plant.label + ); + let planted = support::P::bootstrap(); + let source = planted.read_raw("src/LexiconLre.lex.tex"); + assert_eq!( + source.matches(plant.site).count(), + 1, + "{}: the plant site occurs once", + plant.label + ); + planted.write( + "src/LexiconLre.lex.tex", + &source.replacen(plant.site, plant.replacement, 1), + ); + let checked = support::checked_project(&planted); + let low = super::lower(&checked, &planted, MODULE, ROOTS[plant.root].0); + match super::evaluate(&low, &[vector.run().0], 1).first() { + Some(first) => eprintln!( + "BS-26: plant detected: {}: {}", + plant.label, + first.chars().take(160).collect::() + ), + None => missed.push(plant.label), + } + } + assert!( + missed.is_empty(), + "planted defects their own vector does not detect: {missed:?}" + ); +} + +fn export(lowered: &BTreeMap<&str, super::Lowered>) { + let exported = harness::exported("bootstrap"); + let bundle = exported.bundle(); + let mut realized = 0usize; + for exported_root in &exported.output.result.roots { + let Some((member, low)) = exported_root + .root + .strip_prefix(&format!("{MODULE}.")) + .and_then(|member| lowered.get(member).map(|low| (member, low))) + else { + continue; + }; + let largest = exported_root + .outputs + .iter() + .flat_map(|output| &output.certification.certificates) + .map(|certificate| certificate.file.byte_length) + .max() + .expect("a certificate"); + eprintln!( + "BS-26: {}: largest certificate part {largest} bytes", + exported_root.root + ); + assert!( + largest <= CERTIFICATE_BUDGET, + "{}: a certificate part of {largest} bytes is beyond {CERTIFICATE_BUDGET}", + exported_root.root + ); + for output in &exported_root.outputs { + let package: Json2 = serde_json::from_slice( + &std::fs::read( + bundle + .join(&output.package.directory) + .join("package.json") + .as_std_path(), + ) + .expect("package.json"), + ) + .expect("package JSON"); + assert_eq!( + package["program"], + serde_json::to_value(&low.program).expect("a program is JSON"), + "{member}: the exported package realizes the program the differential ran" + ); + realized += 1; + } + } + assert_eq!(realized, ROOTS.len(), "one certified package per root"); + let packages = harness::tree(&bundle.join("packages")); + let committed_packages = harness::tree(&harness::fixture_dir("bootstrap").join("packages")); + assert_eq!( + packages.keys().collect::>(), + committed_packages.keys().collect::>(), + "the exported package files are the committed ones" + ); + for (relative, content) in &packages { + assert!( + committed_packages.get(relative) == Some(content), + "packages/{relative} differs from the committed package" + ); + } +} + +/// Every error `parse` can state, by a part of its message (R2). +const REFUSALS: &[&str] = &[ + "whitespace is one or more ASCII spaces or LF", + "max_scope_depth exceeded: configured", + "trailing tokens after the expression", + "unbalanced `)`", + "unexpected end of expression", + "expected `(`, found", + "expected an LRE form", + "a form reference is `(form qualified-id form-id)`", + "expected a qualified ID, found", + "is not a qualified `package::entry` ID", + "expected a form ID, found", + "is not a form ID", + "a self form is `(self-form form-id)`", + "a slot is `(slot decimal)`", + "expected a slot index, found", + "is not a slot index", + "`(space)` takes no operand", + "a token is `(token renderer-token-id)`", + "expected a renderer-token ID, found", + "is not a renderer-token ID", + "an operator name is `(operator-name identifier)`", + "expected an operator name, found", + "is not an operator name", + "`seq` has at least one item", + "takes exactly one operand", + "takes exactly two operands", + "is not an LRE form", +]; + +#[allow(clippy::too_many_lines)] +pub(super) fn case() { + let started = std::time::Instant::now(); + let project = support::P::bootstrap(); + let checked_units = project.check_ok(); + assert!( + checked_units.units.contains_key("LexiconLre") + && checked_units.units.contains_key("LexiconLse"), + "LexiconLre and LexiconLse are linked modules of the project" + ); + eligibility_of(&project, "LexiconLre", MODULE, &ROOTS); + assert_eq!( + constructors().as_slice(), + [ + "RForm", + "RSelfForm", + "RSlot", + "RSeq", + "RSpace", + "RToken", + "RGroup", + "RParen", + "RBracket", + "RSub", + "RSup", + "RFrac", + "ROperatorName", + "RNil", + "RItem" + ], + "the constructors of Render" + ); + + let vectors = named(); + let labels: BTreeSet<&str> = vectors.iter().map(|vector| vector.label.as_str()).collect(); + assert_eq!(labels.len(), vectors.len(), "each vector has its own label"); + let mut messages = Vec::new(); + let mut reading: Vec = Vec::new(); + let mut walking: Vec = Vec::new(); + for vector in &vectors { + let (run, refusal) = vector.run(); + messages.extend(refusal); + if vector.walking() { + walking.push(run); + } else { + reading.push(run); + } + } + for refusal in REFUSALS { + assert!( + messages.iter().any(|message| message.contains(refusal)), + "no vector reaches the error `{refusal}`" + ); + } + + // The committed renders: read, and walked. + let texts = committed(); + assert!( + texts.len() >= 200, + "the committed lexicon entries carry {} renders", + texts.len() + ); + let mut committed_reading: Vec = Vec::new(); + let mut committed_walking: Vec = Vec::new(); + let mut bases: Vec = Vec::new(); + for (path, source) in &texts { + let (run, refusal) = parse_run(source, DEPTH); + assert!( + refusal.is_none(), + "{path}: `{source}` is refused: {refusal:?}" + ); + committed_reading.push(run); + committed_walking.push(facts_run(&parsed(source))); + bases.push(source.clone()); + } + let normative = support::normative_host(); + let (evaluator_seeded, rustc_seeded) = if normative { + ( + seeded_runs(EVALUATOR_SEEDED, &bases), + seeded_runs(RUSTC_SEEDED, &bases), + ) + } else { + ((Vec::new(), Vec::new()), (Vec::new(), Vec::new())) + }; + eprintln!( + "BS-26: {} vectors ({} read, {} walked), {} committed renders, {} and {} seeded renders under the evaluator and rustc ({:?})", + vectors.len(), + reading.len(), + walking.len(), + texts.len(), + evaluator_seeded.0.len(), + rustc_seeded.0.len(), + started.elapsed() + ); + + let checked = support::checked_project(&project); + let lowered: BTreeMap<&str, super::Lowered> = ROOTS + .iter() + .map(|(root, _)| (*root, super::lower(&checked, &project, MODULE, root))) + .collect(); + if normative { + let mut read_corpus: Vec = reading.clone(); + read_corpus.extend(committed_reading.iter().cloned()); + read_corpus.extend(evaluator_seeded.1); + let mut walk_corpus: Vec = walking.clone(); + walk_corpus.extend(committed_walking.iter().cloned()); + walk_corpus.extend(evaluator_seeded.0); + for (root, corpus) in [("parseLre", &read_corpus), ("renderFacts", &walk_corpus)] { + let disagreements = super::evaluate(&lowered[root], corpus, 5); + assert!( + disagreements.is_empty(), + "{root} and the Rust oracle disagree: {disagreements:?}" + ); + eprintln!( + "BS-26: {root}: the evaluator agrees on {} runs ({:?})", + corpus.len(), + started.elapsed() + ); + } + plants(&vectors); + } else { + eprintln!("BS-26: platform-bound host: the evaluator, the seeded runs, and the planted defects run on the normative host"); + } + + let mut compiled_reading = reading; + compiled_reading.append(&mut committed_reading); + compiled_reading.extend(rustc_seeded.1); + let mut compiled_walking = walking; + compiled_walking.append(&mut committed_walking); + compiled_walking.extend(rustc_seeded.0); + let total = rustc_differential( + "BS-26", + &[ + ( + "parseLre", + &lowered["parseLre"], + compiled_reading.as_slice(), + ), + ( + "renderFacts", + &lowered["renderFacts"], + compiled_walking.as_slice(), + ), + ], + "oracle", + ); + eprintln!("BS-26: {total} rustc runs agree ({:?})", started.elapsed()); + + if !support::lean_backed("BS-26") { + return; + } + export(&lowered); + eprintln!("BS-26: passed in {:?}", started.elapsed()); +} diff --git a/crates/conformance/src/cases/repository.rs b/crates/conformance/src/cases/repository.rs index aaab4a220b..f1757cfc23 100644 --- a/crates/conformance/src/cases/repository.rs +++ b/crates/conformance/src/cases/repository.rs @@ -873,7 +873,7 @@ pub(crate) fn run(id: &str) { let model = repo_model::Model::load(&root.join("model").into_std_path_buf()) .expect("the model loads"); let table = spec_table(); - assert_eq!(table.len(), 479, "§31 has 479 rows"); + assert_eq!(table.len(), 480, "§31 has 480 rows"); assert_eq!(model.ids.id.len(), table.len(), "register row count"); for ((spec_id, spec_suite, spec_statement), row) in table.iter().zip(model.ids.id.iter()) diff --git a/crates/conformance/tests/conformance.rs b/crates/conformance/tests/conformance.rs index 24bd1a7b6c..f06fff39e6 100644 --- a/crates/conformance/tests/conformance.rs +++ b/crates/conformance/tests/conformance.rs @@ -2325,6 +2325,11 @@ fn conformance_bs_25() { repo_conformance::cases::run("BS-25"); } +#[test] +fn conformance_bs_26() { + repo_conformance::cases::run("BS-26"); +} + #[test] fn conformance_pk_01() { repo_conformance::cases::run("PK-01"); diff --git a/features/suites/bootstrap.feature b/features/suites/bootstrap.feature index 7ac0ecb383..e23674746a 100644 --- a/features/suites/bootstrap.feature +++ b/features/suites/bootstrap.feature @@ -126,3 +126,10 @@ Feature: bootstrap When the project is checked and built, and the lowered program and the rendered package of each root are run on those texts and expressions Then each root is eligible for rust-std with exactly its declared effects, no certificate part is beyond 3,145,728 bytes, and `parseLse` and `printLse` equal `parse` and `print` on every run, every error of which a vector reaches And every planted defect is detected by its own vector, and the exported packages are byte-identical to the committed packages and realize the programs the differential ran + + @BS-26 @build + Scenario: The reading of LexLean Rendering Expressions of SPEC.md §13.9 and the facts of a render (`lexicon::lre::parse`: a text, tokenized and read as one S-expression by the reader of `BS-25`, bounded by the configured `max_scope_depth`, and converted into a render, a form or self-form reference, a slot, a sequence, a space, a renderer token, a group, parentheses, brackets, a subscript, a superscript, a fraction, or an operator name, or the kind and message of the first error, in the order the oracle finds them; and the walkers of `Render`: the slots, the renderer tokens, the form and self-form references and the operator names in occurrence order, the first construct the text channel cannot render, and the first script or fraction operand that renders nothing or is a bare script) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLre` over `LexiconLse`: `parseLre` and `renderFacts`, each eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `math` render of the committed lexicon entries read and walked, and on seeded renders and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the checks of a render against its entry (the slots against `surface_arity`, the tokens against the registry, the references against the glossary) stay in the entry validator and are outside the claim. + Given the committed `bootstrap` project, hand-built texts of every form and every error of the reader and renders of every shape, every `math` render of the committed lexicon entries, and seeded renders and seeded edits of their texts + When the project is checked and built, and the lowered program and the rendered package of each root are run on those texts and renders + Then each root is eligible for rust-std with exactly its declared effects, no certificate part is beyond 3,145,728 bytes, and `parseLre` and `renderFacts` equal `parse` and the walkers on every run, every error of which a vector reaches + And every planted defect is detected by its own vector, and the exported packages are byte-identical to the committed packages and realize the programs the differential ran diff --git a/model/ids.toml b/model/ids.toml index c9e92869c4..0ef3cbf653 100644 --- a/model/ids.toml +++ b/model/ids.toml @@ -2792,6 +2792,12 @@ level = "build" suite = "bootstrap" statement = "The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim." +[[id]] +id = "BS-26" +level = "build" +suite = "bootstrap" +statement = "The reading of LexLean Rendering Expressions of SPEC.md §13.9 and the facts of a render (`lexicon::lre::parse`: a text, tokenized and read as one S-expression by the reader of `BS-25`, bounded by the configured `max_scope_depth`, and converted into a render, a form or self-form reference, a slot, a sequence, a space, a renderer token, a group, parentheses, brackets, a subscript, a superscript, a fraction, or an operator name, or the kind and message of the first error, in the order the oracle finds them; and the walkers of `Render`: the slots, the renderer tokens, the form and self-form references and the operator names in occurrence order, the first construct the text channel cannot render, and the first script or fraction operand that renders nothing or is a bare script) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLre` over `LexiconLse`: `parseLre` and `renderFacts`, each eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `math` render of the committed lexicon entries read and walked, and on seeded renders and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the checks of a render against its entry (the slots against `surface_arity`, the tokens against the registry, the references against the glossary) stay in the entry validator and are outside the claim." + [[id]] id = "PK-01" level = "build" diff --git a/xtask/src/spec_links.rs b/xtask/src/spec_links.rs index 962afcafd8..ec77840783 100644 --- a/xtask/src/spec_links.rs +++ b/xtask/src/spec_links.rs @@ -61,8 +61,8 @@ fn parse_table(spec: &str) -> Result, Fail> { pub fn validate(root: &Path) -> Result<(), Fail> { let spec = std::fs::read_to_string(root.join("SPEC.md"))?; let rows = parse_table(&spec)?; - if rows.len() != 479 { - return Err(format!("RP-07: the §31 table has {} rows, not 479", rows.len()).into()); + if rows.len() != 480 { + return Err(format!("RP-07: the §31 table has {} rows, not 480", rows.len()).into()); } // The prose total under the table is part of the contract a reader // relies on; a row added without it would leave §31 contradicting itself. From dd0a3d8938cabafd793573a177926e1d997a241b Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Mon, 9 Nov 2026 17:13:20 +0000 Subject: [PATCH 2/3] BS-26: the LRE reader and the facts of a render, LexiconLre bootstrap/src/LexiconLre.lex.tex over LexiconLse: parseLre converts the tree the reader of BS-25 gives with a work stack, and renderFacts walks a render once over an explicit stack the host bounds by the node count, collecting the slots, tokens, references, operator names, the first math-only construct, and the first script-operand problem. Lean verifies the module and each root is eligible with exactly its declared effects, neither of them recursion. The entrypoint, the lock, the two export roots, and the project README follow. Co-Authored-By: Claude Opus 5.5 --- bootstrap/README.md | 3 + bootstrap/lexlean.lock | 2 +- bootstrap/lexlean.toml | 2 +- bootstrap/src/LexiconLre.lex.tex | 493 +++++++++++++++++++ tests/production-export/bootstrap/roots.json | 16 + 5 files changed, 514 insertions(+), 2 deletions(-) create mode 100644 bootstrap/src/LexiconLre.lex.tex diff --git a/bootstrap/README.md b/bootstrap/README.md index 7231ee204a..e3289b955b 100644 --- a/bootstrap/README.md +++ b/bootstrap/README.md @@ -22,3 +22,6 @@ writes the rejection of a GNAF request through it. `src/LexiconLse.lex.tex` (SPEC.md §17.23, `BS-25`) imports the standard library's `Numeral`, `Order`, and `Text`, and nothing of the bootstrap project: its expressions, universes, and S-expression trees are sorts of its own, each list of a sort's values a chain in the sort. + +`src/LexiconLre.lex.tex` (SPEC.md §17.23, `BS-26`) imports `LexiconLse`, whose tokenizer and +tree reader it reads with, and the standard library's `Order`. diff --git a/bootstrap/lexlean.lock b/bootstrap/lexlean.lock index c0f9b0b9ae..48ed5cda0d 100644 --- a/bootstrap/lexlean.lock +++ b/bootstrap/lexlean.lock @@ -1,7 +1,7 @@ spec = "lexlean/lock/2" language = "1.2" compiler_semantics = "635a25acb886587f8777525e778092e9599d1f2df26da59403ee39e9e299151a" -project_config_sha256 = "aadc7a0fdc3987e8bf71ba9f71a787366814ee58b080ec8815e53aa39997afe1" +project_config_sha256 = "5a568b9aeda9201032ce4f0fd5bd8560793dd661903edcfef0f165f2da0229fa" lean_toolchain = "leanprover/lean4:v4.32.1" [[workspace_file]] diff --git a/bootstrap/lexlean.toml b/bootstrap/lexlean.toml index b18757fe3e..72aa30fb6e 100644 --- a/bootstrap/lexlean.toml +++ b/bootstrap/lexlean.toml @@ -3,7 +3,7 @@ name = "bootstrap" language = "1.2" module_prefix = "LexLeanBoot" source_roots = ["src"] -entrypoints = ["src/AxiomOutput.lex.tex", "src/CalculusCheck.lex.tex", "src/GnafValidate.lex.tex", "src/LexiconLse.lex.tex", "src/PrimitiveScan.lex.tex", "src/RuntimeValidate.lex.tex", "src/RustAssemble.lex.tex", "src/RustAst.lex.tex", "src/RustBodies.lex.tex", "src/RustLoops.lex.tex", "src/RustLower.lex.tex", "src/RustMoves.lex.tex", "src/RustPackage.lex.tex", "src/RustPrint.lex.tex", "src/RustSucceed.lex.tex", "src/RustTerms.lex.tex", "src/RustValidate.lex.tex", "src/TokenAudit.lex.tex", "src/ToolingAst.lex.tex", "src/ToolingLatex.lex.tex"] +entrypoints = ["src/LexiconLre.lex.tex", "src/LexiconLse.lex.tex"] build_root = ".lexlean" lockfile = "lexlean.lock" lean_workspace = "." diff --git a/bootstrap/src/LexiconLre.lex.tex b/bootstrap/src/LexiconLre.lex.tex new file mode 100644 index 0000000000..f8713271c5 --- /dev/null +++ b/bootstrap/src/LexiconLre.lex.tex @@ -0,0 +1,493 @@ +\begin{lexlean}{LexiconLre} +\useglossary{lexlean.std.nat@1.2.0} +\importmodule{LexLeanStd.Order} +\importmodule{LexiconLse} +\title{Natural number addition} +\begin{semanticmodule} +inductive{ + name: Render, + type_parameters: [], + parameters: [], + constructors: [ + {name: RForm, fields: [Bytes, Bytes, Bytes]}, + {name: RSelfForm, fields: [Bytes]}, + {name: RSlot, fields: [Nat]}, + {name: RSeq, fields: [Render]}, + {name: RSpace, fields: []}, + {name: RToken, fields: [Bytes]}, + {name: RGroup, fields: [Render]}, + {name: RParen, fields: [Render]}, + {name: RBracket, fields: [Render]}, + {name: RSub, fields: [Render, Render]}, + {name: RSup, fields: [Render, Render]}, + {name: RFrac, fields: [Render, Render]}, + {name: ROperatorName, fields: [Bytes]}, + {name: RNil, fields: []}, + {name: RItem, fields: [Render, Render]} + ] +} + +inductive{ + name: RWork, + type_parameters: [], + parameters: [], + constructors: [{name: RVisit, fields: [LexiconLse::Node]}, {name: RBuild, fields: [Bytes, Nat]}] +} + +structure{ + name: Rv, + type_parameters: [], + parameters: [], + fields: [ + {name: work, type: (List RWork)}, + {name: values, type: (List Render)}, + {name: error, type: (Option LexiconLse::Failure)} + ] +} + +structure{ + name: Facts, + type_parameters: [], + parameters: [], + fields: [ + {name: slots, type: (List Nat)}, + {name: tokens, type: (List Bytes)}, + {name: formRefs, type: (List (Bytes * (Bytes * Bytes)))}, + {name: selfForms, type: (List Bytes)}, + {name: operatorNames, type: (List Bytes)}, + {name: mathOnly, type: (Option Bytes)}, + {name: scriptError, type: (Option Bytes)} + ] +} + +structure{ + name: Wk, + type_parameters: [], + parameters: [], + fields: [{name: stack, type: (List Render)}, {name: facts, type: Facts}] +} + +definition{ + name: scanOperatorName, + parameters: [subject : Bytes, n : Nat, position : Nat], + result: (Option Nat), + body: (if (position (scanOperatorName text n position)) n 1 : (Nat * Bool))) =? n) else false))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: scanTokenId, + parameters: [subject : Bytes, n : Nat, position : Nat], + result: (Option Nat), + body: (if (position (scanTokenId text n position)) n 1 : (Nat * Bool))) =? n) else false))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: isFormId, + parameters: [text : Bytes], + result: Bool, + body: (isTokenId text), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: rvFail, + parameters: [st : Rv, failure : LexiconLse::Failure], + result: Rv, + body: (record Rv {work := (project st work), values := (project st values), error := (ctor Option.some [LexiconLse::Failure] failure)}), + executable: true +} + +definition{ + name: rvPush, + parameters: [st : Rv, rest : (List RWork), value : Render], + result: Rv, + body: (record Rv {work := rest, values := (cons value (project st values)), error := (ctor Option.none [LexiconLse::Failure])}), + executable: true +} + +definition{ + name: pushVisitsR, + parameters: [items : (List LexiconLse::Node), work : (List RWork)], + result: (List RWork), + body: (prim list_fold (fun (acc : (List RWork)) (item : LexiconLse::Node) => (cons (ctor RWork.RVisit item) acc)) work (LexLeanStd.Order::reverseList [LexiconLse::Node] items) : (List RWork)), + executable: true +} + +definition{ + name: takeRenders, + parameters: [values : (List Render), count : Nat], + result: ((List Render) * (List Render)), + body: (first (prim iterate_until (fun (state : ((List Render) * (List Render))) => (match (second state) + | List.nil => (ctor Option.none [((List Render) * (List Render))]) + | List.cons top rest => (ctor Option.some [((List Render) * (List Render))] ((cons top (first state)), rest)))) count ((nil Render), values) : (((List Render) * (List Render)) * Bool))), + executable: true +} + +definition{ + name: renderChain, + parameters: [items : (List Render)], + result: Render, + body: (prim list_fold (fun (chain : Render) (item : Render) => (ctor Render.RItem item chain)) (ctor Render.RNil) (LexLeanStd.Order::reverseList [Render] items) : Render), + executable: true +} + +definition{ + name: visitForm, + parameters: [st : Rv, rest : (List RWork), arguments : (List LexiconLse::Node), fuel : Nat], + result: Rv, + body: (if ((prim length arguments : Nat) =? 2) then (match (LexiconLse::atomOf (LexiconLse::nodeAt arguments 0) (LexiconLse::lit "a qualified ID") fuel) + | Result.ok spelled => (match (LexiconLse::qualifiedId spelled) + | Result.ok parts => (match (LexiconLse::atomOf (LexiconLse::nodeAt arguments 1) (LexiconLse::lit "a form ID") fuel) + | Result.ok form => (if (isFormId form) then (rvPush st rest (ctor Render.RForm (first parts) (second parts) form)) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted form) (LexiconLse::lit " is not a form ID"))))) + | Result.error failure => (rvFail st failure)) + | Result.error failure => (rvFail st failure)) + | Result.error failure => (rvFail st failure)) else (rvFail st (LexiconLse::failed (LexiconLse::lit "a form reference is `(form qualified-id form-id)`")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: visitNamed, + parameters: [ + st : Rv, + rest : (List RWork), + arguments : (List LexiconLse::Node), + fuel : Nat, + what : Bytes, + shape : Bytes, + kind : Nat + ], + result: Rv, + body: (if ((prim length arguments : Nat) =? 1) then (match (LexiconLse::atomOf (LexiconLse::nodeAt arguments 0) what fuel) + | Result.ok text => (if (kind =? 0) then (if (isFormId text) then (rvPush st rest (ctor Render.RSelfForm text)) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted text) (LexiconLse::lit " is not a form ID"))))) else (if (kind =? 1) then (if (isTokenId text) then (rvPush st rest (ctor Render.RToken text)) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted text) (LexiconLse::lit " is not a renderer-token ID"))))) else (if (isOperatorName text) then (rvPush st rest (ctor Render.ROperatorName text)) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted text) (LexiconLse::lit " is not an operator name"))))))) + | Result.error failure => (rvFail st failure)) else (rvFail st (LexiconLse::failed shape))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: visitSlot, + parameters: [st : Rv, rest : (List RWork), arguments : (List LexiconLse::Node), fuel : Nat], + result: Rv, + body: (if ((prim length arguments : Nat) =? 1) then (match (LexiconLse::atomOf (LexiconLse::nodeAt arguments 0) (LexiconLse::lit "a slot index") fuel) + | Result.ok digits => (match (LexiconLse::decimalFault digits) + | Option.some fault => (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted digits) (LexiconLse::lit " is not a slot index")))) + | Option.none => (let value : Nat := (LexiconLse::decimalValue digits); (if (4294967295 (rvFail st failure)) else (rvFail st (LexiconLse::failed (LexiconLse::lit "a slot is `(slot decimal)`")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: isUnary, + parameters: [head : Bytes], + result: Bool, + body: (if (prim equal head (LexiconLse::lit "group") : Bool) then true else (if (prim equal head (LexiconLse::lit "paren") : Bool) then true else (prim equal head (LexiconLse::lit "bracket") : Bool))), + executable: true +} + +definition{ + name: isBinary, + parameters: [head : Bytes], + result: Bool, + body: (if (prim equal head (LexiconLse::lit "sub") : Bool) then true else (if (prim equal head (LexiconLse::lit "sup") : Bool) then true else (prim equal head (LexiconLse::lit "frac") : Bool))), + executable: true +} + +definition{ + name: visitHead, + parameters: [ + st : Rv, + rest : (List RWork), + head : Bytes, + arguments : (List LexiconLse::Node), + fuel : Nat + ], + result: Rv, + body: (let argc : Nat := (prim length arguments : Nat); (if (prim equal head (LexiconLse::lit "form") : Bool) then (visitForm st rest arguments fuel) else (if (prim equal head (LexiconLse::lit "self-form") : Bool) then (visitNamed st rest arguments fuel (LexiconLse::lit "a form ID") (LexiconLse::lit "a self form is `(self-form form-id)`") 0) else (if (prim equal head (LexiconLse::lit "slot") : Bool) then (visitSlot st rest arguments fuel) else (if (prim equal head (LexiconLse::lit "space") : Bool) then (if (argc =? 0) then (rvPush st rest (ctor Render.RSpace)) else (rvFail st (LexiconLse::failed (LexiconLse::lit "`(space)` takes no operand")))) else (if (prim equal head (LexiconLse::lit "token") : Bool) then (visitNamed st rest arguments fuel (LexiconLse::lit "a renderer-token ID") (LexiconLse::lit "a token is `(token renderer-token-id)`") 1) else (if (prim equal head (LexiconLse::lit "operator-name") : Bool) then (visitNamed st rest arguments fuel (LexiconLse::lit "an operator name") (LexiconLse::lit "an operator name is `(operator-name identifier)`") 2) else (if (prim equal head (LexiconLse::lit "seq") : Bool) then (if (argc =? 0) then (rvFail st (LexiconLse::failed (LexiconLse::lit "`seq` has at least one item"))) else (record Rv {work := (pushVisitsR arguments (cons (ctor RWork.RBuild head argc) rest)), values := (project st values), error := (ctor Option.none [LexiconLse::Failure])})) else (if (isUnary head) then (if (argc =? 1) then (record Rv {work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RBuild head 1) rest)), values := (project st values), error := (ctor Option.none [LexiconLse::Failure])}) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted head) (LexiconLse::lit " takes exactly one operand"))))) else (if (isBinary head) then (if (argc =? 2) then (record Rv {work := (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 0)) (cons (ctor RWork.RVisit (LexiconLse::nodeAt arguments 1)) (cons (ctor RWork.RBuild head 2) rest))), values := (project st values), error := (ctor Option.none [LexiconLse::Failure])}) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted head) (LexiconLse::lit " takes exactly two operands"))))) else (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::quoted head) (LexiconLse::lit " is not an LRE form")))))))))))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: rvVisit, + parameters: [st : Rv, rest : (List RWork), node : LexiconLse::Node, fuel : Nat], + result: Rv, + body: (match node + | LexiconLse::Node.NList chain => (match (LexiconLse::itemsOf chain fuel) + | List.nil => (rvFail st (LexiconLse::failed (LexiconLse::lit "expected an LRE form"))) + | List.cons headNode arguments => (match headNode + | LexiconLse::Node.NAtom head => (visitHead st rest head arguments fuel) + | LexiconLse::Node.NList inner => (rvFail st (LexiconLse::failed (LexiconLse::lit "expected an LRE form"))) + | LexiconLse::Node.NNil => (rvFail st (LexiconLse::failed (LexiconLse::lit "expected an LRE form"))) + | LexiconLse::Node.NItem h t => (rvFail st (LexiconLse::failed (LexiconLse::lit "expected an LRE form"))))) + | LexiconLse::Node.NAtom text => (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::lit "expected `(`, found ") (LexiconLse::describe node)))) + | LexiconLse::Node.NNil => (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::lit "expected `(`, found ") (LexiconLse::describe node)))) + | LexiconLse::Node.NItem head tail => (rvFail st (LexiconLse::failed (LexiconLse::cat (LexiconLse::lit "expected `(`, found ") (LexiconLse::describe node))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: wrapOne, + parameters: [head : Bytes, inner : Render], + result: Render, + body: (if (prim equal head (LexiconLse::lit "group") : Bool) then (ctor Render.RGroup inner) else (if (prim equal head (LexiconLse::lit "paren") : Bool) then (ctor Render.RParen inner) else (ctor Render.RBracket inner))), + executable: true +} + +definition{ + name: wrapTwo, + parameters: [head : Bytes, left : Render, right : Render], + result: Render, + body: (if (prim equal head (LexiconLse::lit "sub") : Bool) then (ctor Render.RSub left right) else (if (prim equal head (LexiconLse::lit "sup") : Bool) then (ctor Render.RSup left right) else (ctor Render.RFrac left right))), + executable: true +} + +definition{ + name: rvBuild, + parameters: [st : Rv, rest : (List RWork), head : Bytes, count : Nat], + result: Rv, + body: (let taken : ((List Render) * (List Render)) := (takeRenders (project st values) count); (let parts : (List Render) := (first taken); (if (prim equal head (LexiconLse::lit "seq") : Bool) then (rvPush (record Rv {work := rest, values := (second taken), error := (ctor Option.none [LexiconLse::Failure])}) rest (ctor Render.RSeq (renderChain parts))) else (if (isUnary head) then (match parts + | List.nil => (rvFail st (LexiconLse::failed (LexiconLse::lit "missing operand"))) + | List.cons inner more => (rvPush (record Rv {work := rest, values := (second taken), error := (ctor Option.none [LexiconLse::Failure])}) rest (wrapOne head inner))) else (match parts + | List.nil => (rvFail st (LexiconLse::failed (LexiconLse::lit "missing operand"))) + | List.cons left more => (match more + | List.nil => (rvFail st (LexiconLse::failed (LexiconLse::lit "missing operand"))) + | List.cons right tail => (rvPush (record Rv {work := rest, values := (second taken), error := (ctor Option.none [LexiconLse::Failure])}) rest (wrapTwo head left right)))))))), + executable: true +} + +definition{ + name: rvStep, + parameters: [fuel : Nat, st : Rv], + result: (Option Rv), + body: (match (project st error) + | Option.some stopped => (ctor Option.none [Rv]) + | Option.none => (match (project st work) + | List.nil => (ctor Option.none [Rv]) + | List.cons item rest => (match item + | RWork.RVisit node => (ctor Option.some [Rv] (rvVisit st rest node fuel)) + | RWork.RBuild head count => (ctor Option.some [Rv] (rvBuild st rest head count))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: renderOf, + parameters: [root : LexiconLse::Node, fuel : Nat], + result: (Result Render LexiconLse::Failure), + body: (let final : Rv := (first (prim iterate_until (fun [fuel] (st : Rv) => (rvStep fuel st)) ((prim multiply fuel 2 : Nat) + 2) (record Rv {work := (cons (ctor RWork.RVisit root) (nil RWork)), values := (nil Render), error := (ctor Option.none [LexiconLse::Failure])}) : (Rv * Bool))); (match (project final error) + | Option.some failure => (ctor Result.error [Render, LexiconLse::Failure] failure) + | Option.none => (match (project final values) + | List.nil => (ctor Result.error [Render, LexiconLse::Failure] (LexiconLse::failed (LexiconLse::lit "malformed render expression"))) + | List.cons value more => (match more + | List.nil => (ctor Result.ok [Render, LexiconLse::Failure] value) + | List.cons another rest => (ctor Result.error [Render, LexiconLse::Failure] (LexiconLse::failed (LexiconLse::lit "malformed render expression"))))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: parseLre, + parameters: [text : Bytes, maxDepth : Nat], + result: (Result Render LexiconLse::Failure), + body: (match (LexiconLse::tokenize text) + | Result.error failure => (ctor Result.error [Render, LexiconLse::Failure] failure) + | Result.ok toks => (match (LexiconLse::readTree toks maxDepth) + | Result.error failure => (ctor Result.error [Render, LexiconLse::Failure] failure) + | Result.ok root => (renderOf root (prim length toks : Nat)))), + axioms: [Quot.sound, propext], + executable: true, + production: {effects: [allocation, overflow], targets: [rust-std]} +} + +definition{ + name: renderItems, + parameters: [chain : Render, fuel : Nat], + result: (List Render), + body: (LexLeanStd.Order::reverseList [Render] (second (first (prim iterate_until (fun (state : (Render * (List Render))) => (match (first state) + | Render.RItem head tail => (ctor Option.some [(Render * (List Render))] (tail, (cons head (second state)))) + | Render.RNil => (ctor Option.none [(Render * (List Render))]) + | Render.RForm p e f => (ctor Option.none [(Render * (List Render))]) + | Render.RSelfForm f => (ctor Option.none [(Render * (List Render))]) + | Render.RSlot n => (ctor Option.none [(Render * (List Render))]) + | Render.RSeq items => (ctor Option.none [(Render * (List Render))]) + | Render.RSpace => (ctor Option.none [(Render * (List Render))]) + | Render.RToken t => (ctor Option.none [(Render * (List Render))]) + | Render.RGroup inner => (ctor Option.none [(Render * (List Render))]) + | Render.RParen inner => (ctor Option.none [(Render * (List Render))]) + | Render.RBracket inner => (ctor Option.none [(Render * (List Render))]) + | Render.RSub a b => (ctor Option.none [(Render * (List Render))]) + | Render.RSup a b => (ctor Option.none [(Render * (List Render))]) + | Render.RFrac a b => (ctor Option.none [(Render * (List Render))]) + | Render.ROperatorName n => (ctor Option.none [(Render * (List Render))]))) (fuel + 1) (chain, (nil Render)) : ((Render * (List Render)) * Bool))))), + executable: true +} + +definition{ + name: nothingStep, + parameters: [fuel : Nat, state : ((List Render) * Nat)], + result: (Option ((List Render) * Nat)), + body: (if ((second state) =? 1) then (ctor Option.none [((List Render) * Nat)]) else (match (first state) + | List.nil => (ctor Option.none [((List Render) * Nat)]) + | List.cons top rest => (match top + | Render.RSpace => (ctor Option.some [((List Render) * Nat)] (rest, 0)) + | Render.RSeq items => (ctor Option.some [((List Render) * Nat)] ((prim list_fold (fun (acc : (List Render)) (item : Render) => (cons item acc)) rest (renderItems items fuel) : (List Render)), 0)) + | Render.RForm p e f => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RSelfForm f => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RSlot n => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RToken t => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RGroup inner => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RParen inner => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RBracket inner => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RSub a b => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RSup a b => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RFrac a b => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.ROperatorName n => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RNil => (ctor Option.some [((List Render) * Nat)] (rest, 1)) + | Render.RItem h t => (ctor Option.some [((List Render) * Nat)] (rest, 1))))), + executable: true +} + +definition{ + name: rendersNothing, + parameters: [render : Render, fuel : Nat], + result: Bool, + body: ((second (first (prim iterate_until (fun [fuel] (state : ((List Render) * Nat)) => (nothingStep fuel state)) (fuel + 1) ((cons render (nil Render)), 0) : (((List Render) * Nat) * Bool)))) =? 0), + executable: true +} + +definition{ + name: isScript, + parameters: [render : Render], + result: Bool, + body: (match render + | Render.RSub a b => true + | Render.RSup a b => true + | Render.RFrac a b => false + | Render.RForm p e f => false + | Render.RSelfForm f => false + | Render.RSlot n => false + | Render.RSeq items => false + | Render.RSpace => false + | Render.RToken t => false + | Render.RGroup inner => false + | Render.RParen inner => false + | Render.RBracket inner => false + | Render.ROperatorName n => false + | Render.RNil => false + | Render.RItem h t => false), + executable: true +} + +definition{ + name: operandProblem, + parameters: [name : Bytes, operand : Render, fuel : Nat], + result: (Option Bytes), + body: (if (rendersNothing operand fuel) then (ctor Option.some [Bytes] (LexiconLse::cat (LexiconLse::quoted name) (LexiconLse::lit " operand renders nothing"))) else (if (if (prim equal name (LexiconLse::lit "frac") : Bool) then false else (isScript operand)) then (ctor Option.some [Bytes] (LexiconLse::cat (LexiconLse::quoted name) (LexiconLse::lit " operand is a bare script; wrap nested scripts in `(group ...)`"))) else (ctor Option.none [Bytes]))), + executable: true +} + +definition{ + name: scriptProblem, + parameters: [name : Bytes, left : Render, right : Render, fuel : Nat], + result: (Option Bytes), + body: (match (operandProblem name left fuel) + | Option.some problem => (ctor Option.some [Bytes] problem) + | Option.none => (operandProblem name right fuel)), + executable: true +} + +definition{ + name: withFacts, + parameters: [stack : (List Render), facts : Facts], + result: (Option Wk), + body: (ctor Option.some [Wk] (record Wk {stack := stack, facts := facts})), + executable: true +} + +definition{ + name: firstSome, + parameters: [current : (Option Bytes), candidate : Bytes], + result: (Option Bytes), + body: (match current + | Option.some kept => current + | Option.none => (ctor Option.some [Bytes] candidate)), + executable: true +} + +definition{ + name: firstProblem, + parameters: [current : (Option Bytes), candidate : (Option Bytes)], + result: (Option Bytes), + body: (match current | Option.some kept => current | Option.none => candidate), + executable: true +} + +definition{ + name: binaryFacts, + parameters: [facts : Facts, name : Bytes, left : Render, right : Render, fuel : Nat], + result: Facts, + body: (record Facts {slots := (project facts slots), tokens := (project facts tokens), formRefs := (project facts formRefs), selfForms := (project facts selfForms), operatorNames := (project facts operatorNames), mathOnly := (firstSome (project facts mathOnly) name), scriptError := (firstProblem (project facts scriptError) (scriptProblem name left right fuel))}), + executable: true +} + +definition{ + name: wkStep, + parameters: [fuel : Nat, st : Wk], + result: (Option Wk), + body: (match (project st stack) + | List.nil => (ctor Option.none [Wk]) + | List.cons top rest => (let facts : Facts := (project st facts); (match top + | Render.RForm p e f => (withFacts rest (record Facts {slots := (project facts slots), tokens := (project facts tokens), formRefs := (cons (p, (e, f)) (project facts formRefs)), selfForms := (project facts selfForms), operatorNames := (project facts operatorNames), mathOnly := (project facts mathOnly), scriptError := (project facts scriptError)})) + | Render.RSelfForm f => (withFacts rest (record Facts {slots := (project facts slots), tokens := (project facts tokens), formRefs := (project facts formRefs), selfForms := (cons f (project facts selfForms)), operatorNames := (project facts operatorNames), mathOnly := (project facts mathOnly), scriptError := (project facts scriptError)})) + | Render.RSlot n => (withFacts rest (record Facts {slots := (cons n (project facts slots)), tokens := (project facts tokens), formRefs := (project facts formRefs), selfForms := (project facts selfForms), operatorNames := (project facts operatorNames), mathOnly := (project facts mathOnly), scriptError := (project facts scriptError)})) + | Render.RSeq items => (withFacts (prim list_fold (fun (acc : (List Render)) (item : Render) => (cons item acc)) rest (LexLeanStd.Order::reverseList [Render] (renderItems items fuel)) : (List Render)) facts) + | Render.RSpace => (withFacts rest facts) + | Render.RToken t => (withFacts rest (record Facts {slots := (project facts slots), tokens := (cons t (project facts tokens)), formRefs := (project facts formRefs), selfForms := (project facts selfForms), operatorNames := (project facts operatorNames), mathOnly := (project facts mathOnly), scriptError := (project facts scriptError)})) + | Render.RGroup inner => (withFacts (cons inner rest) facts) + | Render.RParen inner => (withFacts (cons inner rest) facts) + | Render.RBracket inner => (withFacts (cons inner rest) facts) + | Render.RSub a b => (withFacts (cons a (cons b rest)) (binaryFacts facts (LexiconLse::lit "sub") a b fuel)) + | Render.RSup a b => (withFacts (cons a (cons b rest)) (binaryFacts facts (LexiconLse::lit "sup") a b fuel)) + | Render.RFrac a b => (withFacts (cons a (cons b rest)) (binaryFacts facts (LexiconLse::lit "frac") a b fuel)) + | Render.ROperatorName n => (withFacts rest (record Facts {slots := (project facts slots), tokens := (project facts tokens), formRefs := (project facts formRefs), selfForms := (project facts selfForms), operatorNames := (cons n (project facts operatorNames)), mathOnly := (firstSome (project facts mathOnly) (LexiconLse::lit "operator-name")), scriptError := (project facts scriptError)})) + | Render.RNil => (withFacts rest facts) + | Render.RItem h t => (withFacts rest facts)))), + executable: true +} + +definition{ + name: renderFacts, + parameters: [render : Render, nodes : Nat], + result: Facts, + body: (let final : Wk := (first (prim iterate_until (fun [nodes] (st : Wk) => (wkStep nodes st)) (nodes + 1) (record Wk {stack := (cons render (nil Render)), facts := (record Facts {slots := (nil Nat), tokens := (nil Bytes), formRefs := (nil (Bytes * (Bytes * Bytes))), selfForms := (nil Bytes), operatorNames := (nil Bytes), mathOnly := (ctor Option.none [Bytes]), scriptError := (ctor Option.none [Bytes])})}) : (Wk * Bool))); (let facts : Facts := (project final facts); (record Facts {slots := (LexLeanStd.Order::reverseList [Nat] (project facts slots)), tokens := (LexLeanStd.Order::reverseList [Bytes] (project facts tokens)), formRefs := (LexLeanStd.Order::reverseList [(Bytes * (Bytes * Bytes))] (project facts formRefs)), selfForms := (LexLeanStd.Order::reverseList [Bytes] (project facts selfForms)), operatorNames := (LexLeanStd.Order::reverseList [Bytes] (project facts operatorNames)), mathOnly := (project facts mathOnly), scriptError := (project facts scriptError)}))), + executable: true, + production: {effects: [allocation, overflow], targets: [rust-std]} +} +\end{semanticmodule} +\end{lexlean} diff --git a/tests/production-export/bootstrap/roots.json b/tests/production-export/bootstrap/roots.json index 35097ce3a9..7c1099a37c 100644 --- a/tests/production-export/bootstrap/roots.json +++ b/tests/production-export/bootstrap/roots.json @@ -152,6 +152,14 @@ ], "version": "1.0.0" }, + { + "name": "parse_lre", + "root": "LexLeanBoot.LexiconLre.parseLre", + "targets": [ + "rust-std" + ], + "version": "1.0.0" + }, { "name": "parse_lse", "root": "LexLeanBoot.LexiconLse.parseLse", @@ -192,6 +200,14 @@ ], "version": "1.0.0" }, + { + "name": "render_facts", + "root": "LexLeanBoot.LexiconLre.renderFacts", + "targets": [ + "rust-std" + ], + "version": "1.0.0" + }, { "name": "render_package", "root": "LexLeanBoot.RustPackage.renderPackage", From 97cdb15cf508efa711ce8508b0bf353493f2ecfe Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Mon, 9 Nov 2026 17:16:00 +0000 Subject: [PATCH 3/3] BS-26: VERIFICATION.md records the first run The case's first run on BS-25 over release/v0.4: the evaluator agrees on 885 reads and 535 walks, rustc on 5,020 runs, and all 20 plants are detected after one plant was given a vector that reaches it. The case stops at the stack-bound step until the regeneration commits the two packages. Co-Authored-By: Claude Opus 5.5 --- VERIFICATION.md | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/VERIFICATION.md b/VERIFICATION.md index b46180d8d5..a62d64a65d 100644 --- a/VERIFICATION.md +++ b/VERIFICATION.md @@ -11053,6 +11053,50 @@ and `bootstrap/print_lse` packages and measures the row of `print_lse` in `tests/stack-bounds.json` (`PD-08`). The branch is based on `release/v0.4` after `SL-13` and stacks on nothing. +### the LRE reader and the facts of a render are source (BS-26) + +`BS-26` holds the two production roots of `LexLeanBoot.LexiconLre` +(`bootstrap/src/LexiconLre.lex.tex`), `parseLre` and `renderFacts`, to +`lexicon::lre::parse` and the walkers of `Render` (`slots`, `tokens`, +`form_refs`, `self_form_refs`, `operator_names`, `math_only_construct`, and +`script_operand_error`). It is the second slice of the lexicon as source and +imports `LexiconLse` (`BS-25`), whose tokenizer and tree reader it reads with, +and the standard library's `Order`. `parseLre` converts the tree with a work +stack that visits the operands of a form in the oracle's order and builds the +form when they are on the value stack; `renderFacts` walks the render once, +pre-order over an explicit stack the host bounds by the number of nodes, and +collects every fact the entry validator reads in one pass. Neither root admits +`recursion`. The case has the shape of its predecessors: hand-built texts that +reach each error of the reader and the limit on nesting, hand-built renders of +every shape and every script-operand problem, every `math` render of the +committed lexicon entries read and walked, and seeded renders and seeded edits. +Each planted defect names the one vector that catches it, and the case asserts +that the site of each plant occurs once. The row's statement carries no counts. + +Measured with the compiler binary of this tree, on a copy of the project whose +entrypoints are `LexiconLse` and `LexiconLre`: `lexlean check`, `lexlean +build`, and `lexlean verify` pass, and the eligibility report gives each root +exactly `allocation` and `overflow`. The register agrees with the §31 table on +480 IDs, by the same parse `validate-spec-links` makes. + +The case was first compiled and run on the normative host on `BS-25` over +`release/v0.4` at `4dc5b803c` (`cargo test -p repo-conformance --test +conformance -- conformance_bs_26`). Its corpus is 88 named vectors (69 read, +19 walked), the 216 committed renders (216 reads and 216 walks), and 300 +seeded renders under the evaluator (300 walks, 600 reads) and 1,500 under +rustc. The evaluator agrees with the oracle on 885 reads and 535 walks, the +rendered packages under the pinned rustc on 5,020 runs, and each of the 20 +planted defects is detected by its own vector. The first run found one plant +its vector did not reach, and no fault of the module: `the second operand is +checked first` was tried on a render whose two problems sit in different +operators, so the order within one operator never decided it (now `a sub whose +operands fail differently`, a `sub` whose base renders nothing and whose script +is a bare script). Lean's audit corrected the axiom policy of 12 definitions. +The case then stops at the stack-bound step, `bootstrap/parse_lre is no package +of the committed exports`, as `BS-21` to `BS-25` did before their regeneration, +which produces the `bootstrap/parse_lre` and `bootstrap/render_facts` packages. +The branch is stacked on `BS-25` and lands after it. + ## The `bdd` recipe as the parts of one run (Issue #131) With UOR Atlas verified as the parts of one run, the `rest` leg became the