diff --git a/.github/bdd-parts.json b/.github/bdd-parts.json index fe7db0138d..9b99d3d59c 100644 --- a/.github/bdd-parts.json +++ b/.github/bdd-parts.json @@ -22,6 +22,9 @@ "bootstrap-gnaf": [ "BS-16" ], + "bootstrap-lexicon-lse": [ + "BS-25" + ], "bootstrap-lower": [ "BS-07" ], diff --git a/.github/vv-rows.json b/.github/vv-rows.json index a0695b8575..d464cfef70 100644 --- a/.github/vv-rows.json +++ b/.github/vv-rows.json @@ -62,6 +62,15 @@ "memory_kib": "", "hints": "" }, + { + "part": "bdd-bootstrap-lexicon-lse", + "recipes": "bdd", + "examples_part": "", + "bdd_part": "bootstrap-lexicon-lse", + "width": "", + "memory_kib": "", + "hints": "" + }, { "part": "bdd-bootstrap-tooling-latex", "recipes": "bdd", diff --git a/CHANGELOG.md b/CHANGELOG.md index 68f6a6eb5e..a6b6a71662 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,12 @@ workspace carried at those versions without a tag. ## Unreleased +- The reader and the canonical printer of LexLean Semantic Expressions as + language-1.2 source, the first slice of the lexicon (SPEC.md §17.23, §13.8, + `BS-25`, #269; no new diagnostic code, no change to the language-1.2 + identity). `parseLse` and `printLse` of `LexiconLse` return what + `lexicon::lse::parse` and `Lse::print` return; the reader admits no + `recursion` and the printer admits it. The register is 479 IDs. - Canonical JSON reading in the standard library, the fourth member the self-hosted compiler needs, #276 (SPEC.md §17.22, `SL-13`). The module `LexLeanStd.JsonRead` holds the production roots `readJson` and diff --git a/CONFORMANCE.md b/CONFORMANCE.md index c9b4937578..2b22435c0f 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -77,6 +77,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `BS-15` | `build` | The package renderer of SPEC.md §17.16 (the checks `package::lower` makes of a manifest's own fields before it reads the program: the spec, the crate name against the Rust keywords and the reserved crates, the version, the profile, the form and the order of the sources, and the effects against the registry, with `allocation` refused on `rust-core`; the Cargo manifest over the scaffold record; and the provenance in canonical JSON with the digests of the runtime, the Cargo manifest, and the library) is language-1.2 source in the `bootstrap/` project, as two production roots of `RustPackage` that the host composes around the roots of `BS-04` to `BS-14`: `checkManifest`, which for the fields of a manifest and the registered effects returns whether the profile is `rust-std`, or the refusal of `package::lower` to the byte, and `renderPackage`, which for the scaffold record, the manifest's name, version, profile, sources, effects and exports, the program's identity, the compiler-semantics identity, the two runtime texts, and the library returns the `Cargo.toml` and the `provenance.json` of `package::package` to the byte; each is eligible for `rust-std` with exactly its declared effects and has no part of its certificate beyond 3,145,728 bytes; the oracle of `checkManifest` is `package::lower` on the manifest with its program and exports replaced by a pair every later check accepts, which agrees with each committed negative manifest's refusal, and the oracle of `renderPackage` is `package::package` on the 299 committed manifests it accepts and, on 137 hand-built inputs (names, exports, sources and effects that reach the escaper, and scaffold records that differ from the committed one) and 299 seeded mutants, `cargo_toml_of` and the canonical JSON of §21.7 by the recipe `package::package` is held to; on the normative host the lowered programs under the calculus evaluator agree with the oracle on the 318 committed manifests, 182 rule manifests and 2,000 seeded mutants, and on the 241 renders whose library is at most 16,384 bytes; the rendered packages agree with the oracle under the pinned rustc on all 3,190 runs; 21 planted defects in the source are each detected; and the certified packages the production export publishes for the project are the committed ones; the checks of the program, the exports and the crate, and the bytes of the library, which are the rows `BS-04` to `BS-14`, are outside the claim. | | `BS-16` | `build` | The GNAF request validation of SPEC.md §17.15 (`gnaf::validate`, with `universe_size`) is language-1.2 source in the `bootstrap/` project: `validateRequest` of `GnafValidate` is a production root eligible for `rust-std` with exactly its declared effects, over the projection of a request that validation reads (the length of the domain, the machine, the carrier's kind with each plan's preparations and the thresholds, the completeness evidence, the objective, the claim's class, and the scope) as first-order data, and returns the first violated rule in the model's order as the oracle's `serde_json` bytes, or nothing; on every host, its rendered `rust-std` package under the pinned rustc agrees with the Rust oracle `gnaf::validate` on the acceptance and on the rejection, to the byte, on the committed requests of `compiler/gnaf/`, the requests the committed certificate fixtures name, hand-built requests that reach each rejection, and seeded mutants of the committed requests; on the normative host, the root's lowered program under the calculus evaluator agrees with the oracle on the same corpus, each planted defect in the source is detected, and the certified package the production export publishes for the project is the committed one; loading a request (`gnaf::load`), the universe identity, the answer, and certification are outside the claim. | | `BS-20` | `build` | The canonical LaTeX of a tooling module of SPEC.md §17.20 (`backend::tooling::render_latex`: the fixed document frame and, for each declaration in order, its title, its kind, and the lines the Lean printer of the same file spells for its types, signatures, and terms, every line escaped by `tex_escape`) is language-1.2 source in the `bootstrap/` project: `renderToolingLatex` of `ToolingLatex`, over the tooling module as first-order data with each list carried as a chain of its own `Nil` and `Cons` constructors and with the typings the checker infers as an argument, writing through the standard library's `Buffer` with an explicit capacity, is a production root eligible for `rust-std` with exactly its declared effects; its rendered `rust-std` package under the pinned rustc and its lowered program under the calculus evaluator agree with the Rust oracle on the text, to the byte, and on the refusal of a write past the capacity, on the committed tooling modules (whose committed `.tex` is equal to the text), on hand-built modules that reach every declaration, type, signature, and term form and every byte class of the escaper, and on seeded mutants, every planted defect is detected, and the certified package the production export publishes for the project is the committed one; the checker, the decoder of the module, and the source map of the emitted pieces stay in the host. | +| `BS-25` | `build` | The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim. | ## calculus diff --git a/README.md b/README.md index 0f5db6830c..9520a8b11e 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ just vv # the complete normative acceptance gate (SPEC.md §9.2) just release # vv, then the §30 release criterion; refused until 1.0.0 ``` -All 478 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. +All 479 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. The differential against the independent generator `lean4-prod` (SPEC.md §17.19, `OR-01`..`OR-08`) acquires it at its pinned commit: from the checkout `LEXLEAN_ORACLE_DIR` names, or by fetching the commit with `git` into the Cargo target directory (`LEXLEAN_ORACLE_CACHE` overrides), and builds it with the pinned Lean and Rust toolchains. It is validation evidence only and no input to any identity. An oracle that cannot be acquired, is not the pinned files, or fails is unavailable: that fails the gate on Linux x86-64 and is printed as `NOT VALIDATED` elsewhere, and is never reported as agreement. @@ -197,7 +197,7 @@ Every row is validated by `just vv`; the IDs link the claim to its register row, | The language-1.2 authoring notation: a readable `.lex.tex` text for a semantic module, in bijection with its canonical JSON, whose every position and variant is read from the schema, with readable forms for types and terms and a generic form for every other value, formatted canonically by `fmt` failing closed, the only form in which a language-1.2 source of the product is written (the data form is refused under language 1.2, and `lexlean::sdk` is the explicit conversion), the same for the compiler-tooling modules in the generic notation, with no compiler-semantics ID that depends on which form an embedded module is written in, with no product source owned by a generator but five named records, read and written by programs only through the authoring API (`lexlean::sdk`), and, for a native core module, a readable program over the shared table of kernel terms whose numbering is the order of its text | `AU-01`..`AU-13` | `build` | | The runtime of generated Lean as native modules: the portable, collection, model, and reasoning runtimes of language 1.2 are LexLean source that every generated module imports, migrated losslessly and audited against their exact axioms, the released runtime text of languages 1.0 and 1.1 is frozen data equal to their released output, and no other Lean runtime text remains in the compiler but the decoder of core module 2 | `RT-01`..`RT-04` | `build` | | The standard library: language-1.2 source beside the compiler whose members are certified production roots, each validated against its oracle by a seeded differential of its evaluator and rustc realizations; bytes and text over `Bytes`, an output buffer of a given capacity written in order; decimal and hexadecimal; the byte order and stable sorts over it; canonical JSON reading and writing; maps and sets over byte keys; SHA-256 against FIPS 180-4, the NIST CAVP vectors, and the `sha2` crate; and the `std-models` and `std-reasoning` source packages, generic over types, whose laws are kernel-checked theorems and whose application is the `helpdesk` example | `SL-01`, `SL-02`, `SL-03`, `SL-04`, `SL-05`, `SL-09`, `SL-10`, `SL-11`, `SL-13`, `SL-14` | `build` | -| The bootstrap project: the compiler's own components as language-1.2 source compiled by LexLean's production pipeline to certified Rust packages, the exact axiom-output parser and its closed Lean-name check, the primitive scanner of §12.2, the preservation lexer and token audit, the target calculus's checker, the Rust crate validator, the Rust printer, the runtime syntax validator, the failure typing and the types of the Rust lowering, its last-use moves, its literal values and its terms, the package renderer, the GNAF request validation, and the LaTeX of a tooling module, each held to the Rust it replaces (the stage-0 oracle) by the committed vectors and a seeded differential of its evaluator and rustc realizations, with planted defects detected and the production export committed | `BS-01`..`BS-20` | `build` | +| The bootstrap project: the compiler's own components as language-1.2 source compiled by LexLean's production pipeline to certified Rust packages, the exact axiom-output parser and its closed Lean-name check, the primitive scanner of §12.2, the preservation lexer and token audit, the target calculus's checker, the Rust crate validator, the Rust printer, the runtime syntax validator, the failure typing and the types of the Rust lowering, its last-use moves, its literal values and its terms, the package renderer, the GNAF request validation, the LaTeX of a tooling module, and the reader and printer of LexLean Semantic Expressions, each held to the Rust it replaces (the stage-0 oracle) by the committed vectors and a seeded differential of its evaluator and rustc realizations, with planted defects detected and the production export committed | `BS-01`..`BS-25` | `build` | | Source packages: a language-1.2 project imports the modules of a locked, content-addressed package of its workspace: `[[source_package]]` and `lexlean-workspace.toml`, the pins in `lexlean.lock` recomputed by every command, imports closed at link, no re-export of a package's production roots, and the package's modules replayed by the importer's `verify` with their origin in the records | `PK-01`..`PK-05` | `build` | | The literal `nat-add-zero` example, the Lean-verified feature examples, and the complete negative fixture suite | `EX-01`..`EX-09` | `build` | | Language-1.2 text components: tokenizers and their round-trip laws, with the laws carried as one native core module the generated modules import rather than authored Lean text | `LM-01`, `LM-02`, `LM-03`, `LM-13` | `build` | diff --git a/SPEC.md b/SPEC.md index 93b9b32971..90b28595fa 100644 --- a/SPEC.md +++ b/SPEC.md @@ -8725,6 +8725,41 @@ the public marker and the visibility line, and the trimming of a line) are each pinned Lean the production export (§24.7) certifies the root beside the others and its package is the committed one. +The reader and the canonical printer of LexLean Semantic Expressions (§13.8) are the first +component of the lexicon (§13) as source (`BS-25`, issue 269), the module `LexiconLse`, with two +production roots for `rust-std`: + +| Root | Takes | Returns | Oracle | +| --- | --- | --- | --- | +| `LexLeanBoot.LexiconLse.parseLse` | a text and the depth bound | the expression, or the kind and message of the first error | `lexicon::lse::parse` | +| `LexLeanBoot.LexiconLse.printLse` | an expression and whether to rename its binders | its canonical text | `Lse::print` | + +`parseLse` reads a text in the three phases of the oracle, each an iteration over an explicit +state, so that it admits no `recursion`. The tokenizer splits the text at parentheses and at ASCII +spaces and LF, and refuses any other ASCII control scalar. The reader keeps a stack of the open +lists, refuses a `(` that would open a list beyond the depth bound with the depth error of +`LLS8002` (kind 1), and every other fault as a syntax error (kind 0): an unbalanced `)`, a token +after the expression, and the end of the text inside a list or before any expression. The +conversion walks the tree with a work stack that visits a form's parts in the oracle's order (a +function before its arguments, the headers of the binders of a `pi` or `lam` last to first, then +their types first to last, then the body) and builds the form when its parts are on the value +stack, so that the first error of a text is the oracle's first error, with its message to the +byte. A list of a sort's own values is a chain in the sort (the arguments of an application, the +binders of a `pi` or `lam`, the universes of a constant and of a `max`), as the tooling and the +generic notation hold theirs, so the data holds no nested inductive. A qualified ID is split at its +first `::` and each part checked by the grammar of §10.1 and §13.1; a decimal is canonical when it +has no redundant leading zero and is within 64 bits, compared against `18446744073709551615` when +it has twenty digits. + +`printLse` writes the canonical text by structural recursion over the expression, its chains, and +its universes, so it admits `recursion` and owes a row in `tests/stack-bounds.json` (`PD-08`). With +renaming, a binder's shown name is `x` and a counter that counts every binder in printing order, +a local is shown by the innermost binder of its name in scope and by its own name when none is, +a binder's type is printed before the binder enters the scope, and a `let` enters its name before +its type, value, and body, as the oracle's work stack does. The scope check against the declared +universes, the conservative checker of signatures and values, and the rendering expressions +(§13.9) stay in the host. + ## 18. Lean backend ### 18.1 Output contract @@ -11646,6 +11681,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `BS-15` | `bootstrap` | The package renderer of SPEC.md §17.16 (the checks `package::lower` makes of a manifest's own fields before it reads the program: the spec, the crate name against the Rust keywords and the reserved crates, the version, the profile, the form and the order of the sources, and the effects against the registry, with `allocation` refused on `rust-core`; the Cargo manifest over the scaffold record; and the provenance in canonical JSON with the digests of the runtime, the Cargo manifest, and the library) is language-1.2 source in the `bootstrap/` project, as two production roots of `RustPackage` that the host composes around the roots of `BS-04` to `BS-14`: `checkManifest`, which for the fields of a manifest and the registered effects returns whether the profile is `rust-std`, or the refusal of `package::lower` to the byte, and `renderPackage`, which for the scaffold record, the manifest's name, version, profile, sources, effects and exports, the program's identity, the compiler-semantics identity, the two runtime texts, and the library returns the `Cargo.toml` and the `provenance.json` of `package::package` to the byte; each is eligible for `rust-std` with exactly its declared effects and has no part of its certificate beyond 3,145,728 bytes; the oracle of `checkManifest` is `package::lower` on the manifest with its program and exports replaced by a pair every later check accepts, which agrees with each committed negative manifest's refusal, and the oracle of `renderPackage` is `package::package` on the 299 committed manifests it accepts and, on 137 hand-built inputs (names, exports, sources and effects that reach the escaper, and scaffold records that differ from the committed one) and 299 seeded mutants, `cargo_toml_of` and the canonical JSON of §21.7 by the recipe `package::package` is held to; on the normative host the lowered programs under the calculus evaluator agree with the oracle on the 318 committed manifests, 182 rule manifests and 2,000 seeded mutants, and on the 241 renders whose library is at most 16,384 bytes; the rendered packages agree with the oracle under the pinned rustc on all 3,190 runs; 21 planted defects in the source are each detected; and the certified packages the production export publishes for the project are the committed ones; the checks of the program, the exports and the crate, and the bytes of the library, which are the rows `BS-04` to `BS-14`, are outside the claim. | §17.16, §17.23 | | `BS-16` | `bootstrap` | The GNAF request validation of SPEC.md §17.15 (`gnaf::validate`, with `universe_size`) is language-1.2 source in the `bootstrap/` project: `validateRequest` of `GnafValidate` is a production root eligible for `rust-std` with exactly its declared effects, over the projection of a request that validation reads (the length of the domain, the machine, the carrier's kind with each plan's preparations and the thresholds, the completeness evidence, the objective, the claim's class, and the scope) as first-order data, and returns the first violated rule in the model's order as the oracle's `serde_json` bytes, or nothing; on every host, its rendered `rust-std` package under the pinned rustc agrees with the Rust oracle `gnaf::validate` on the acceptance and on the rejection, to the byte, on the committed requests of `compiler/gnaf/`, the requests the committed certificate fixtures name, hand-built requests that reach each rejection, and seeded mutants of the committed requests; on the normative host, the root's lowered program under the calculus evaluator agrees with the oracle on the same corpus, each planted defect in the source is detected, and the certified package the production export publishes for the project is the committed one; loading a request (`gnaf::load`), the universe identity, the answer, and certification are outside the claim. | §17.15, §17.23 | | `BS-20` | `bootstrap` | The canonical LaTeX of a tooling module of SPEC.md §17.20 (`backend::tooling::render_latex`: the fixed document frame and, for each declaration in order, its title, its kind, and the lines the Lean printer of the same file spells for its types, signatures, and terms, every line escaped by `tex_escape`) is language-1.2 source in the `bootstrap/` project: `renderToolingLatex` of `ToolingLatex`, over the tooling module as first-order data with each list carried as a chain of its own `Nil` and `Cons` constructors and with the typings the checker infers as an argument, writing through the standard library's `Buffer` with an explicit capacity, is a production root eligible for `rust-std` with exactly its declared effects; its rendered `rust-std` package under the pinned rustc and its lowered program under the calculus evaluator agree with the Rust oracle on the text, to the byte, and on the refusal of a write past the capacity, on the committed tooling modules (whose committed `.tex` is equal to the text), on hand-built modules that reach every declaration, type, signature, and term form and every byte class of the escaper, and on seeded mutants, every planted defect is detected, and the certified package the production export publishes for the project is the committed one; the checker, the decoder of the module, and the source map of the emitted pieces stay in the host. | §17.20, §17.23 | +| `BS-25` | `bootstrap` | The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim. | §17.23, §13.8 | | `PK-01` | `packages` | A language-1.2 project that declares a source package by its identifier, version, kind `path`, and the member directory a `lexlean-workspace.toml` lists locks it by the tree digest of its manifest and source files, its manifest digest, its version, its module prefix, and its own packages, and the configuration and the lock read back to the bytes written; a project without a source package keeps its bytes, and a package of another kind, a source package in another language, a changed byte or version (LLC0105), an undeclared package (LLC0106), a module prefix claimed twice (LLC0107), a project or a package outside the workspace (LLC0108), a path that escapes it, and a symlink (LLS8001) are each refused with the named code. | §10.5, §11.1, §25.1 | | `PK-02` | `packages` | A module of a project imports only modules of its own project and modules of the packages the project declares, and a module of a package only modules of itself and of the packages its own manifest declares: a package module is a module of the importing project under its own module prefix, a module of a package the project does not declare fails with LLR3009, and an import cycle through a package fails with LLR3003. | §10.5, §15.1 | | `PK-03` | `packages` | An importing project builds and verifies under the pinned Lean offline, and its package modules are generated, elaborated, replayed with `leanchecker`, and audited for their exact axioms under the package's module prefix with the package as their origin in the records; the source ID of the project changes with the pins, and no file of the package is written by `fmt`, `build`, or `verify`. | §10.5, §21.3, §22 | @@ -11662,7 +11698,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `LM-09` | `language-models` | No prompt is code: no member of the closed language maps `string` or `bytes` to a function, a term, or source — every operation of the primitive registry that takes a `string` or `bytes` operand is classified by the audit with a data result, the registry's schema enumeration and the production registry's primitive rows equal one another and equal that classified set, and the text declarations of `lexlean/semantic-module/2` state no `string`-typed member, so a prompt reaches a model only as the tokens `encodeText` makes of it; a planted operation or text member of that shape is caught by the audit with its name. | §17.12 | | `LM-13` | `language-models` | The text laws a tokenizer elaborates through are LexLean source and not authored text: the module that states them is one language-1.2 native core module of the embedded `language/text-1.2` project, the project holds no file of Lean source, no independently authored Lean file anywhere in the release tree declares the `LexLeanText` namespace or lies under its module root, and the one-time migration input is absent; the registry lists only that module, the declarations and their exact axiom policies are read from its native data, which the compiler embeds and hashes into the language-1.2 semantics, and a module that declares a text component imports the generated module rather than splicing any Lean text. | §17.10, §17.12 | -**Total required capability IDs:** 478. +**Total required capability IDs:** 479. No row may be downgraded to `some-true` or `open`. Upstream Lean facts are ledger/authority rows, not substitutions for these build behaviors. diff --git a/VERIFICATION.md b/VERIFICATION.md index ee406e9f6f..b46180d8d5 100644 --- a/VERIFICATION.md +++ b/VERIFICATION.md @@ -10999,6 +10999,60 @@ writes them); `cargo clippy -p repo-conformance -p xtask --all-targets --all-features -D warnings`, `validate-model` (461 ids), `validate-spec-links` (461 rows), `check-test-inventory`, and the five `bdd` meta-gate tests pass. +### the LSE reader and printer are source (BS-25) + +`BS-25` holds the two production roots of `LexLeanBoot.LexiconLse` +(`bootstrap/src/LexiconLse.lex.tex`), `parseLse` and `printLse`, to +`lexicon::lse::parse` and `Lse::print`. It is the first slice of the lexicon +(§13) as source, and imports the standard library's `Numeral`, `Order`, and +`Text` and nothing of the bootstrap project. `parseLse` reads a text in the +oracle's three phases, each an iteration over an explicit state (the tokenizer, +the reader with its stack of open lists bounded by the depth argument, and the +conversion with its work stack, which visits a form's parts in the oracle's +order and builds the form when they are on the value stack), so that it admits +no `recursion` and states the oracle's first error with its message to the byte, +the depth error apart from the syntax errors as `ParseError` keeps them. +`printLse` writes the canonical text by structural recursion over the +expression, its chains, and its universes, with the binders renamed in printing +order when asked, and admits `recursion`. The case has the shape of its +predecessors: hand-built texts that reach each error of the reader and the limit +on nesting, hand-built expressions of every form printed as read and renamed, +every `signature` and `value` of the committed lexicon entries read, printed +both ways, and read back, and seeded expressions and seeded edits. Each planted +defect names the one vector that catches it, and the case asserts that the site +of each plant occurs once. The row's statement carries no counts. + +Measured with the compiler binary of this tree, on a copy of the project whose +only entrypoint is `LexiconLse`: `lexlean check`, `lexlean build`, and `lexlean +verify` pass, and the eligibility report gives `parseLse` exactly `allocation` +and `overflow` and `printLse` exactly `allocation`, `overflow`, and `recursion`. +On the project the lock is current and every module checks. The register agrees +with the §31 table on 479 IDs, by the same parse `validate-spec-links` makes. + +The case was first compiled and run on the normative host on `release/v0.4` at +`4dc5b803c` (`cargo test -p repo-conformance --test conformance -- +conformance_bs_25`). Its corpus is 139 named vectors (105 read, 34 printed), +the 417 committed expressions (834 reads, of the text and of its canonical +form, and 834 prints, as read and renamed), and 300 seeded expressions under +the evaluator (300 prints, 600 reads) and 1,500 under rustc. The evaluator +agrees with the oracle on 1,539 reads and 1,168 prints, and each of the 30 +planted defects is detected by its own vector. The reader agreed on every run +of its first run. The printer did not: it closed a constant's universe list and +not the constant, so `(const pkg::entry (u))` lost its last parenthesis; the +vector `a const with universes printed` caught it. Before the case, `lexlean +verify` rejected certificate B of `parseLse`: the scanner of a package or entry +ID kept its state as nested pairs and tested a component as a condition, which +the correspondence certificate of the Rust lowering does not type; the state is +now a structure, as the other scanners of the project keep theirs. The linker +refused three binders spelled like Lean's `id`, like the declaration +`modeText`, and like Lean's `syntax`, and Lean's audit corrected the axiom +policy of 24 definitions. The case then stops at the stack-bound step, +`bootstrap/parse_lse is no package of the committed exports`, as `BS-21` to +`BS-24` did before their regeneration, which produces the `bootstrap/parse_lse` +and `bootstrap/print_lse` packages and measures the row of `print_lse` in +`tests/stack-bounds.json` (`PD-08`). The branch is based on `release/v0.4` +after `SL-13` and stacks on nothing. + ## The `bdd` recipe as the parts of one run (Issue #131) With UOR Atlas verified as the parts of one run, the `rest` leg became the diff --git a/bootstrap/README.md b/bootstrap/README.md index a9f234eb1a..7231ee204a 100644 --- a/bootstrap/README.md +++ b/bootstrap/README.md @@ -18,3 +18,7 @@ payloads a printer reads); `src/RustPrint.lex.tex` also imports the standard library's `Buffer` (`stdlib/src/Buffer.lex.tex`), which it writes through. `src/GnafValidate.lex.tex` (SPEC.md §17.15, `BS-16`) imports `Buffer` alone and writes the rejection of a GNAF request through it. + +`src/LexiconLse.lex.tex` (SPEC.md §17.23, `BS-25`) imports the standard library's `Numeral`, +`Order`, and `Text`, and nothing of the bootstrap project: its expressions, universes, and +S-expression trees are sorts of its own, each list of a sort's values a chain in the sort. diff --git a/bootstrap/lexlean.lock b/bootstrap/lexlean.lock index dec1db451b..c0f9b0b9ae 100644 --- a/bootstrap/lexlean.lock +++ b/bootstrap/lexlean.lock @@ -1,7 +1,7 @@ spec = "lexlean/lock/2" language = "1.2" compiler_semantics = "635a25acb886587f8777525e778092e9599d1f2df26da59403ee39e9e299151a" -project_config_sha256 = "b2bd695c7fad3f83d9a41fdabb6f9f5eee0538452d59609e5170eff53d78f14f" +project_config_sha256 = "aadc7a0fdc3987e8bf71ba9f71a787366814ee58b080ec8815e53aa39997afe1" lean_toolchain = "leanprover/lean4:v4.32.1" [[workspace_file]] diff --git a/bootstrap/lexlean.toml b/bootstrap/lexlean.toml index f57986a7e8..b18757fe3e 100644 --- a/bootstrap/lexlean.toml +++ b/bootstrap/lexlean.toml @@ -3,7 +3,7 @@ name = "bootstrap" language = "1.2" module_prefix = "LexLeanBoot" source_roots = ["src"] -entrypoints = ["src/AxiomOutput.lex.tex", "src/CalculusCheck.lex.tex", "src/GnafValidate.lex.tex", "src/PrimitiveScan.lex.tex", "src/RuntimeValidate.lex.tex", "src/RustAssemble.lex.tex", "src/RustAst.lex.tex", "src/RustBodies.lex.tex", "src/RustLoops.lex.tex", "src/RustLower.lex.tex", "src/RustMoves.lex.tex", "src/RustPackage.lex.tex", "src/RustPrint.lex.tex", "src/RustSucceed.lex.tex", "src/RustTerms.lex.tex", "src/RustValidate.lex.tex", "src/TokenAudit.lex.tex", "src/ToolingAst.lex.tex", "src/ToolingLatex.lex.tex"] +entrypoints = ["src/AxiomOutput.lex.tex", "src/CalculusCheck.lex.tex", "src/GnafValidate.lex.tex", "src/LexiconLse.lex.tex", "src/PrimitiveScan.lex.tex", "src/RuntimeValidate.lex.tex", "src/RustAssemble.lex.tex", "src/RustAst.lex.tex", "src/RustBodies.lex.tex", "src/RustLoops.lex.tex", "src/RustLower.lex.tex", "src/RustMoves.lex.tex", "src/RustPackage.lex.tex", "src/RustPrint.lex.tex", "src/RustSucceed.lex.tex", "src/RustTerms.lex.tex", "src/RustValidate.lex.tex", "src/TokenAudit.lex.tex", "src/ToolingAst.lex.tex", "src/ToolingLatex.lex.tex"] build_root = ".lexlean" lockfile = "lexlean.lock" lean_workspace = "." diff --git a/bootstrap/src/LexiconLse.lex.tex b/bootstrap/src/LexiconLse.lex.tex new file mode 100755 index 0000000000..4e8ad75fb8 --- /dev/null +++ b/bootstrap/src/LexiconLse.lex.tex @@ -0,0 +1,1136 @@ +\begin{lexlean}{LexiconLse} +\useglossary{lexlean.std.nat@1.2.0} +\importmodule{LexLeanStd.Numeral} +\importmodule{LexLeanStd.Order} +\importmodule{LexLeanStd.Text} +\title{Natural number addition} +\begin{semanticmodule} +inductive{ + name: Tok, + type_parameters: [], + parameters: [], + constructors: [ + {name: TOpen, fields: []}, + {name: TClose, fields: []}, + {name: TAtom, fields: [Bytes]} + ] +} + +structure{ + name: Failure, + type_parameters: [], + parameters: [], + fields: [{name: kind, type: Nat}, {name: message, type: Bytes}] +} + +inductive{ + name: Node, + type_parameters: [], + parameters: [], + constructors: [ + {name: NAtom, fields: [Bytes]}, + {name: NList, fields: [Node]}, + {name: NNil, fields: []}, + {name: NItem, fields: [Node, Node]} + ] +} + +inductive{ + name: Universe, + type_parameters: [], + parameters: [], + constructors: [ + {name: UNum, fields: [Nat]}, + {name: UVar, fields: [Bytes]}, + {name: USucc, fields: [Universe]}, + {name: UMax, fields: [Universe]}, + {name: UIMax, fields: [Universe, Universe]}, + {name: UNil, fields: []}, + {name: UItem, fields: [Universe, Universe]} + ] +} + +inductive{ + name: Lse, + type_parameters: [], + parameters: [], + constructors: [ + {name: LSortProp, fields: []}, + {name: LSortType, fields: [Universe]}, + {name: LConst, fields: [Bytes, Bytes, Universe]}, + {name: LLocal, fields: [Bytes]}, + {name: LApp, fields: [Lse, Lse]}, + {name: LPi, fields: [Lse, Lse]}, + {name: LLam, fields: [Lse, Lse]}, + {name: LLet, fields: [Bytes, Lse, Lse, Lse]}, + {name: LNat, fields: [Bytes]}, + {name: LNil, fields: []}, + {name: LArg, fields: [Lse, Lse]}, + {name: LBinder, fields: [Nat, Bytes, Lse, Lse]} + ] +} + +inductive{ + name: Work, + type_parameters: [], + parameters: [], + constructors: [{name: WVisit, fields: [Node]}, {name: WBuild, fields: [Node]}] +} + +inductive{ + name: UWork, + type_parameters: [], + parameters: [], + constructors: [{name: UVisit, fields: [Node]}, {name: UBuild, fields: [Bytes, Nat]}] +} + +structure{ + name: Tk, + type_parameters: [], + parameters: [], + fields: [ + {name: pos, type: Nat}, + {name: start, type: Nat}, + {name: inAtom, type: Bool}, + {name: toks, type: (List Tok)}, + {name: error, type: (Option Failure)} + ] +} + +structure{ + name: Rt, + type_parameters: [], + parameters: [], + fields: [ + {name: pos, type: Nat}, + {name: stack, type: (List (List Node))}, + {name: root, type: (Option Node)}, + {name: error, type: (Option Failure)}, + {name: done, type: Bool} + ] +} + +structure{ + name: Cv, + type_parameters: [], + parameters: [], + fields: [ + {name: work, type: (List Work)}, + {name: values, type: (List Lse)}, + {name: error, type: (Option Failure)} + ] +} + +structure{ + name: Uv, + type_parameters: [], + parameters: [], + fields: [ + {name: work, type: (List UWork)}, + {name: values, type: (List Universe)}, + {name: error, type: (Option Failure)} + ] +} + +structure{ + name: Header, + type_parameters: [], + parameters: [], + fields: [{name: mode, type: Nat}, {name: name, type: Bytes}, {name: ty, type: Node}] +} + +definition{ + name: lit, + parameters: [literal : String], + result: Bytes, + body: (prim utf8_encode literal : Bytes), + executable: true +} + +definition{ + name: cat, + parameters: [left : Bytes, right : Bytes], + result: Bytes, + body: (prim append left right : Bytes), + executable: true +} + +definition{ + name: quoted, + parameters: [text : Bytes], + result: Bytes, + body: (cat (cat (lit "`") text) (lit "`")), + executable: true +} + +definition{ + name: failed, + parameters: [message : Bytes], + result: Failure, + body: (record Failure {kind := 0, message := message}), + executable: true +} + +definition{ + name: byteNum, + parameters: [subject : Bytes, position : Nat], + result: Nat, + body: (match (LexLeanStd.Text::byteAt subject position) + | Option.some byte => (prim to_nat byte : Nat) + | Option.none => 256), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: isLowerNum, + parameters: [value : Nat], + result: Bool, + body: (if (value (scanIdentifier text n position)) n 1 : (Nat * Bool))) =? n) else false))), + axioms: [Quot.sound, propext], + executable: true +} + +structure{ + name: Sg, + type_parameters: [], + parameters: [], + fields: [ + {name: pos, type: Nat}, + {name: segStart, type: Bool}, + {name: firstSeg, type: Bool}, + {name: ok, type: Bool} + ] +} + +definition{ + name: sgStop, + parameters: [st : Sg], + result: (Option Sg), + body: (ctor Option.some [Sg] (record Sg {pos := (project st pos), segStart := (project st segStart), firstSeg := (project st firstSeg), ok := false})), + executable: true +} + +definition{ + name: sgStep, + parameters: [st : Sg, segStart : Bool, firstSeg : Bool], + result: (Option Sg), + body: (ctor Option.some [Sg] (record Sg {pos := ((project st pos) + 1), segStart := segStart, firstSeg := firstSeg, ok := true})), + executable: true +} + +definition{ + name: idStep, + parameters: [subject : Bytes, n : Nat, hyphenFirst : Bool, st : Sg], + result: (Option Sg), + body: (if (project st ok) then (if ((project st pos) (idStep text n hyphenFirst st)) (n + 1) (record Sg {pos := 0, segStart := true, firstSeg := true, ok := true}) : (Sg * Bool))); (if (project final ok) then (if (project final segStart) then false else true) else false))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: isPackageId, + parameters: [text : Bytes], + result: Bool, + body: (idScan text false), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: isEntryId, + parameters: [text : Bytes], + result: Bool, + body: (idScan text true), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: scanDigits, + parameters: [subject : Bytes, n : Nat, position : Nat], + result: (Option Nat), + body: (if (position (scanDigits text n position)) (n + 1) 0 : (Nat * Bool))) =? n)), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: decimalFault, + parameters: [text : Bytes], + result: (Option Failure), + body: (let n : Nat := (prim length text : Nat); (if (if (n =? 0) then true else (if (allDigits text) then false else true)) then (ctor Option.some [Failure] (failed (cat (quoted text) (lit " is not a decimal")))) else (if (if (1 value + | Option.none => 0), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: tkFlush, + parameters: [text : Bytes, st : Tk], + result: (List Tok), + body: (if (project st inAtom) then (cons (ctor Tok.TAtom (LexLeanStd.Text::sliceOr text (project st start) (prim subtract (project st pos) (project st start) : Nat))) (project st toks)) else (project st toks)), + executable: true +} + +definition{ + name: tkStep, + parameters: [text : Bytes, n : Nat, st : Tk], + result: (Option Tk), + body: (match (project st error) + | Option.some stopped => (ctor Option.none [Tk]) + | Option.none => (let position : Nat := (project st pos); (if (position (tkStep text n st)) (n + 1) (record Tk {pos := 0, start := 0, inAtom := false, toks := (nil Tok), error := (ctor Option.none [Failure])}) : (Tk * Bool))); (match (project final error) + | Option.some failure => (ctor Result.error [(List Tok), Failure] failure) + | Option.none => (ctor Result.ok [(List Tok), Failure] (LexLeanStd.Order::reverseList [Tok] (tkFlush text final)))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: chainOf, + parameters: [items : (List Node)], + result: Node, + body: (prim list_fold (fun (chain : Node) (item : Node) => (ctor Node.NItem item chain)) (ctor Node.NNil) items : Node), + executable: true +} + +definition{ + name: rtFail, + parameters: [st : Rt, failure : Failure], + result: Rt, + body: (record Rt {pos := (project st pos), stack := (project st stack), root := (project st root), error := (ctor Option.some [Failure] failure), done := false}), + executable: true +} + +definition{ + name: rtPlace, + parameters: [st : Rt, node : Node, stack : (List (List Node))], + result: Rt, + body: (match stack + | List.nil => (record Rt {pos := ((project st pos) + 1), stack := (nil (List Node)), root := (ctor Option.some [Node] node), error := (ctor Option.none [Failure]), done := false}) + | List.cons parent more => (record Rt {pos := ((project st pos) + 1), stack := (cons (cons node parent) more), root := (project st root), error := (ctor Option.none [Failure]), done := false})), + executable: true +} + +definition{ + name: rtToken, + parameters: [maxDepth : Nat, st : Rt, tok : Tok], + result: Rt, + body: (match tok + | Tok.TOpen => (if ((prim length (project st stack) : Nat) (match (project st stack) + | List.nil => (rtFail st (failed (lit "unbalanced `)`"))) + | List.cons top rest => (rtPlace st (ctor Node.NList (chainOf top)) rest)) + | Tok.TAtom text => (rtPlace st (ctor Node.NAtom text) (project st stack))), + executable: true +} + +definition{ + name: rtStep, + parameters: [toks : (List Tok), maxDepth : Nat, st : Rt], + result: (Option Rt), + body: (match (project st error) + | Option.some stopped => (ctor Option.none [Rt]) + | Option.none => (if (project st done) then (ctor Option.none [Rt]) else (match (prim index toks (project st pos) : (Option Tok)) + | Option.none => (match (project st stack) + | List.nil => (match (project st root) + | Option.some found => (ctor Option.some [Rt] (record Rt {pos := (project st pos), stack := (nil (List Node)), root := (project st root), error := (ctor Option.none [Failure]), done := true})) + | Option.none => (ctor Option.some [Rt] (rtFail st (failed (lit "unexpected end of expression"))))) + | List.cons top rest => (ctor Option.some [Rt] (rtFail st (failed (lit "unexpected end of expression"))))) + | Option.some tok => (match (project st root) + | Option.some found => (ctor Option.some [Rt] (rtFail st (failed (lit "trailing tokens after the expression")))) + | Option.none => (ctor Option.some [Rt] (rtToken maxDepth st tok)))))), + executable: true +} + +definition{ + name: readTree, + parameters: [toks : (List Tok), maxDepth : Nat], + result: (Result Node Failure), + body: (let final : Rt := (first (prim iterate_until (fun [maxDepth, toks] (st : Rt) => (rtStep toks maxDepth st)) ((prim length toks : Nat) + 2) (record Rt {pos := 0, stack := (nil (List Node)), root := (ctor Option.none [Node]), error := (ctor Option.none [Failure]), done := false}) : (Rt * Bool))); (match (project final error) + | Option.some failure => (ctor Result.error [Node, Failure] failure) + | Option.none => (match (project final root) + | Option.some node => (ctor Result.ok [Node, Failure] node) + | Option.none => (ctor Result.error [Node, Failure] (failed (lit "unexpected end of expression")))))), + executable: true +} + +definition{ + name: itemStep, + parameters: [state : (Node * (List Node))], + result: (Option (Node * (List Node))), + body: (match (first state) + | Node.NAtom text => (ctor Option.none [(Node * (List Node))]) + | Node.NList inner => (ctor Option.none [(Node * (List Node))]) + | Node.NNil => (ctor Option.none [(Node * (List Node))]) + | Node.NItem head tail => (ctor Option.some [(Node * (List Node))] (tail, (cons head (second state))))), + executable: true +} + +definition{ + name: itemsOf, + parameters: [chain : Node, fuel : Nat], + result: (List Node), + body: (LexLeanStd.Order::reverseList [Node] (second (first (prim iterate_until (fun (state : (Node * (List Node))) => (itemStep state)) (fuel + 1) (chain, (nil Node)) : ((Node * (List Node)) * Bool))))), + executable: true +} + +definition{ + name: describe, + parameters: [node : Node], + result: Bytes, + body: (match node + | Node.NAtom text => (quoted text) + | Node.NList chain => (match chain + | Node.NItem head tail => (match head + | Node.NAtom text => (cat (cat (lit "`(") text) (lit " ...)`")) + | Node.NList inner => (lit "a list") + | Node.NNil => (lit "a list") + | Node.NItem h t => (lit "a list")) + | Node.NAtom text => (lit "a list") + | Node.NList inner => (lit "a list") + | Node.NNil => (lit "a list")) + | Node.NNil => (lit "a list") + | Node.NItem head tail => (lit "a list")), + executable: true +} + +definition{ + name: nodeAt, + parameters: [items : (List Node), position : Nat], + result: Node, + body: (match (prim index items position : (Option Node)) + | Option.some node => node + | Option.none => (ctor Node.NNil)), + executable: true +} + +definition{ + name: atomOf, + parameters: [node : Node, what : Bytes, fuel : Nat], + result: (Result Bytes Failure), + body: (match node + | Node.NAtom text => (ctor Result.ok [Bytes, Failure] text) + | Node.NList inner => (ctor Result.error [Bytes, Failure] (failed (cat (cat (cat (lit "expected ") what) (lit ", found ")) (describe node)))) + | Node.NNil => (ctor Result.error [Bytes, Failure] (failed (cat (cat (cat (lit "expected ") what) (lit ", found ")) (describe node)))) + | Node.NItem head tail => (ctor Result.error [Bytes, Failure] (failed (cat (cat (cat (lit "expected ") what) (lit ", found ")) (describe node))))), + executable: true +} + +definition{ + name: qualifiedId, + parameters: [text : Bytes], + result: (Result (Bytes * Bytes) Failure), + body: (match (LexLeanStd.Text::find text (lit "::")) + | Option.none => (ctor Result.error [(Bytes * Bytes), Failure] (failed (cat (quoted text) (lit " is not a qualified `package::entry` ID")))) + | Option.some cut => (let package : Bytes := (LexLeanStd.Text::sliceOr text 0 cut); (let entry : Bytes := (LexLeanStd.Text::sliceOr text (cut + 2) (prim subtract (prim length text : Nat) (cut + 2) : Nat)); (if (isPackageId package) then (if (isEntryId entry) then (ctor Result.ok [(Bytes * Bytes), Failure] (package, entry)) else (ctor Result.error [(Bytes * Bytes), Failure] (failed (cat (quoted entry) (lit " is not a valid local entry ID"))))) else (ctor Result.error [(Bytes * Bytes), Failure] (failed (cat (quoted package) (lit " is not a valid package ID")))))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: uvFail, + parameters: [st : Uv, failure : Failure], + result: Uv, + body: (record Uv {work := (project st work), values := (project st values), error := (ctor Option.some [Failure] failure)}), + executable: true +} + +definition{ + name: pushVisitsU, + parameters: [items : (List Node), work : (List UWork)], + result: (List UWork), + body: (prim list_fold (fun (acc : (List UWork)) (item : Node) => (cons (ctor UWork.UVisit item) acc)) work (LexLeanStd.Order::reverseList [Node] items) : (List UWork)), + executable: true +} + +definition{ + name: takeUniverses, + parameters: [values : (List Universe), count : Nat], + result: ((List Universe) * (List Universe)), + body: (first (prim iterate_until (fun (state : ((List Universe) * (List Universe))) => (match (second state) + | List.nil => (ctor Option.none [((List Universe) * (List Universe))]) + | List.cons top rest => (ctor Option.some [((List Universe) * (List Universe))] ((cons top (first state)), rest)))) count ((nil Universe), values) : (((List Universe) * (List Universe)) * Bool))), + executable: true +} + +definition{ + name: universeChain, + parameters: [items : (List Universe)], + result: Universe, + body: (prim list_fold (fun (chain : Universe) (item : Universe) => (ctor Universe.UItem item chain)) (ctor Universe.UNil) (LexLeanStd.Order::reverseList [Universe] items) : Universe), + executable: true +} + +definition{ + name: uvVisit, + parameters: [fuel : Nat, st : Uv, rest : (List UWork), node : Node], + result: Uv, + body: (match node + | Node.NAtom text => (if (allDigits text) then (match (decimalFault text) + | Option.some failure => (uvFail st failure) + | Option.none => (record Uv {work := rest, values := (cons (ctor Universe.UNum (decimalValue text)) (project st values)), error := (ctor Option.none [Failure])})) else (if (isLseIdentifier text) then (record Uv {work := rest, values := (cons (ctor Universe.UVar text) (project st values)), error := (ctor Option.none [Failure])}) else (uvFail st (failed (cat (quoted text) (lit " is not a universe")))))) + | Node.NList chain => (let items : (List Node) := (itemsOf chain fuel); (match items + | List.nil => (uvFail st (failed (lit "expected a universe form"))) + | List.cons headNode arguments => (match headNode + | Node.NAtom head => (let argc : Nat := (prim length arguments : Nat); (let known : Bool := (if (prim equal head (lit "succ") : Bool) then true else (if (prim equal head (lit "max") : Bool) then true else (prim equal head (lit "imax") : Bool))); (let fits : Bool := (if (prim equal head (lit "succ") : Bool) then (argc =? 1) else (if (prim equal head (lit "max") : Bool) then (2 <=? argc) else (if (prim equal head (lit "imax") : Bool) then (argc =? 2) else false))); (if fits then (record Uv {work := (pushVisitsU arguments (cons (ctor UWork.UBuild head argc) rest)), values := (project st values), error := (ctor Option.none [Failure])}) else (if known then (uvFail st (failed (cat (cat (lit "`(") head) (lit " ...)` has the wrong arity")))) else (uvFail st (failed (cat (quoted head) (lit " is not a universe form"))))))))) + | Node.NList inner => (uvFail st (failed (lit "expected a universe form"))) + | Node.NNil => (uvFail st (failed (lit "expected a universe form"))) + | Node.NItem h t => (uvFail st (failed (lit "expected a universe form")))))) + | Node.NNil => (uvFail st (failed (lit "expected a universe form"))) + | Node.NItem head tail => (uvFail st (failed (lit "expected a universe form")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: uvBuild, + parameters: [st : Uv, rest : (List UWork), head : Bytes, count : Nat], + result: Uv, + body: (let taken : ((List Universe) * (List Universe)) := (takeUniverses (project st values) count); (let parts : (List Universe) := (first taken); (let built : Universe := (if (prim equal head (lit "succ") : Bool) then (match parts + | List.nil => (ctor Universe.UNil) + | List.cons a more => (ctor Universe.USucc a)) else (if (prim equal head (lit "imax") : Bool) then (match parts + | List.nil => (ctor Universe.UNil) + | List.cons a more => (match more + | List.nil => (ctor Universe.UNil) + | List.cons b tail => (ctor Universe.UIMax a b))) else (ctor Universe.UMax (universeChain parts)))); (record Uv {work := rest, values := (cons built (second taken)), error := (ctor Option.none [Failure])})))), + executable: true +} + +definition{ + name: uvStep, + parameters: [fuel : Nat, st : Uv], + result: (Option Uv), + body: (match (project st error) + | Option.some stopped => (ctor Option.none [Uv]) + | Option.none => (match (project st work) + | List.nil => (ctor Option.none [Uv]) + | List.cons item rest => (match item + | UWork.UVisit node => (ctor Option.some [Uv] (uvVisit fuel st rest node)) + | UWork.UBuild head count => (ctor Option.some [Uv] (uvBuild st rest head count))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: universeOf, + parameters: [node : Node, fuel : Nat], + result: (Result Universe Failure), + body: (let final : Uv := (first (prim iterate_until (fun [fuel] (st : Uv) => (uvStep fuel st)) ((prim multiply fuel 2 : Nat) + 2) (record Uv {work := (cons (ctor UWork.UVisit node) (nil UWork)), values := (nil Universe), error := (ctor Option.none [Failure])}) : (Uv * Bool))); (match (project final error) + | Option.some failure => (ctor Result.error [Universe, Failure] failure) + | Option.none => (match (project final values) + | List.nil => (ctor Result.error [Universe, Failure] (failed (lit "expected a universe"))) + | List.cons top more => (ctor Result.ok [Universe, Failure] top)))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: universesOf, + parameters: [items : (List Node), fuel : Nat], + result: (Result Universe Failure), + body: (match (prim list_fold (fun [fuel] (acc : (Result (List Universe) Failure)) (item : Node) => (match acc + | Result.ok kept => (match (universeOf item fuel) + | Result.ok level => (ctor Result.ok [(List Universe), Failure] (cons level kept)) + | Result.error failure => (ctor Result.error [(List Universe), Failure] failure)) + | Result.error failure => (ctor Result.error [(List Universe), Failure] failure))) (ctor Result.ok [(List Universe), Failure] (nil Universe)) items : (Result (List Universe) Failure)) + | Result.ok reversed => (ctor Result.ok [Universe, Failure] (universeChain (LexLeanStd.Order::reverseList [Universe] reversed))) + | Result.error failure => (ctor Result.error [Universe, Failure] failure)), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: sortError, + parameters: [], + result: Failure, + body: (failed (lit "a sort is `(sort prop)` or `(sort (type u))`")), + executable: true +} + +definition{ + name: leafSort, + parameters: [arguments : (List Node), fuel : Nat], + result: (Result Lse Failure), + body: (if ((prim length arguments : Nat) =? 1) then (match (nodeAt arguments 0) + | Node.NAtom text => (if (prim equal text (lit "prop") : Bool) then (ctor Result.ok [Lse, Failure] (ctor Lse.LSortProp)) else (ctor Result.error [Lse, Failure] (sortError))) + | Node.NList inner => (let items : (List Node) := (itemsOf inner fuel); (if ((prim length items : Nat) =? 2) then (match (nodeAt items 0) + | Node.NAtom keyword => (if (prim equal keyword (lit "type") : Bool) then (match (universeOf (nodeAt items 1) fuel) + | Result.ok level => (ctor Result.ok [Lse, Failure] (ctor Lse.LSortType level)) + | Result.error failure => (ctor Result.error [Lse, Failure] failure)) else (ctor Result.error [Lse, Failure] (sortError))) + | Node.NList deeper => (ctor Result.error [Lse, Failure] (sortError)) + | Node.NNil => (ctor Result.error [Lse, Failure] (sortError)) + | Node.NItem h t => (ctor Result.error [Lse, Failure] (sortError))) else (ctor Result.error [Lse, Failure] (sortError)))) + | Node.NNil => (ctor Result.error [Lse, Failure] (sortError)) + | Node.NItem h t => (ctor Result.error [Lse, Failure] (sortError))) else (ctor Result.error [Lse, Failure] (sortError))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: constError, + parameters: [], + result: Failure, + body: (failed (lit "a const is `(const qualified-id [universes])`")), + executable: true +} + +definition{ + name: leafConst, + parameters: [arguments : (List Node), fuel : Nat], + result: (Result Lse Failure), + body: (let argc : Nat := (prim length arguments : Nat); (match (nodeAt arguments 0) + | Node.NAtom spelled => (if (argc =? 1) then (match (qualifiedId spelled) + | Result.ok parts => (ctor Result.ok [Lse, Failure] (ctor Lse.LConst (first parts) (second parts) (ctor Universe.UNil))) + | Result.error failure => (ctor Result.error [Lse, Failure] failure)) else (if (argc =? 2) then (match (nodeAt arguments 1) + | Node.NList universes => (let items : (List Node) := (itemsOf universes fuel); (match items + | List.nil => (ctor Result.error [Lse, Failure] (constError)) + | List.cons h t => (match (qualifiedId spelled) + | Result.ok parts => (match (universesOf items fuel) + | Result.ok chain => (ctor Result.ok [Lse, Failure] (ctor Lse.LConst (first parts) (second parts) chain)) + | Result.error failure => (ctor Result.error [Lse, Failure] failure)) + | Result.error failure => (ctor Result.error [Lse, Failure] failure)))) + | Node.NAtom text => (ctor Result.error [Lse, Failure] (constError)) + | Node.NNil => (ctor Result.error [Lse, Failure] (constError)) + | Node.NItem h t => (ctor Result.error [Lse, Failure] (constError))) else (ctor Result.error [Lse, Failure] (constError)))) + | Node.NList inner => (ctor Result.error [Lse, Failure] (constError)) + | Node.NNil => (ctor Result.error [Lse, Failure] (constError)) + | Node.NItem h t => (ctor Result.error [Lse, Failure] (constError)))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: leafLocal, + parameters: [arguments : (List Node)], + result: (Result Lse Failure), + body: (if ((prim length arguments : Nat) =? 1) then (match (nodeAt arguments 0) + | Node.NAtom name => (if (isLseIdentifier name) then (ctor Result.ok [Lse, Failure] (ctor Lse.LLocal name)) else (ctor Result.error [Lse, Failure] (failed (cat (quoted name) (lit " is not a valid local ID"))))) + | Node.NList inner => (ctor Result.error [Lse, Failure] (failed (lit "a local is `(local id)`"))) + | Node.NNil => (ctor Result.error [Lse, Failure] (failed (lit "a local is `(local id)`"))) + | Node.NItem h t => (ctor Result.error [Lse, Failure] (failed (lit "a local is `(local id)`")))) else (ctor Result.error [Lse, Failure] (failed (lit "a local is `(local id)`")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: leafNat, + parameters: [arguments : (List Node)], + result: (Result Lse Failure), + body: (if ((prim length arguments : Nat) =? 1) then (match (nodeAt arguments 0) + | Node.NAtom digits => (match (decimalFault digits) + | Option.some failure => (ctor Result.error [Lse, Failure] failure) + | Option.none => (ctor Result.ok [Lse, Failure] (ctor Lse.LNat digits))) + | Node.NList inner => (ctor Result.error [Lse, Failure] (failed (lit "a nat is `(nat decimal)`"))) + | Node.NNil => (ctor Result.error [Lse, Failure] (failed (lit "a nat is `(nat decimal)`"))) + | Node.NItem h t => (ctor Result.error [Lse, Failure] (failed (lit "a nat is `(nat decimal)`")))) else (ctor Result.error [Lse, Failure] (failed (lit "a nat is `(nat decimal)`")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: leafOf, + parameters: [head : Bytes, arguments : (List Node), fuel : Nat], + result: (Result Lse Failure), + body: (if (prim equal head (lit "sort") : Bool) then (leafSort arguments fuel) else (if (prim equal head (lit "const") : Bool) then (leafConst arguments fuel) else (if (prim equal head (lit "local") : Bool) then (leafLocal arguments) else (if (prim equal head (lit "nat") : Bool) then (leafNat arguments) else (ctor Result.error [Lse, Failure] (failed (cat (quoted head) (lit " is not an LSE form")))))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: binderMode, + parameters: [text : Bytes], + result: (Option Nat), + body: (if (prim equal text (lit "explicit") : Bool) then (ctor Option.some [Nat] 0) else (if (prim equal text (lit "implicit") : Bool) then (ctor Option.some [Nat] 1) else (if (prim equal text (lit "instance") : Bool) then (ctor Option.some [Nat] 2) else (ctor Option.none [Nat])))), + executable: true +} + +definition{ + name: binderHeader, + parameters: [node : Node, fuel : Nat], + result: (Result Header Failure), + body: (match node + | Node.NList chain => (let items : (List Node) := (itemsOf chain fuel); (if ((prim length items : Nat) =? 3) then (match (atomOf (nodeAt items 0) (lit "a binder mode") fuel) + | Result.ok modeWord => (match (binderMode modeWord) + | Option.some mode => (match (atomOf (nodeAt items 1) (lit "a local ID") fuel) + | Result.ok name => (if (isLseIdentifier name) then (ctor Result.ok [Header, Failure] (record Header {mode := mode, name := name, ty := (nodeAt items 2)})) else (ctor Result.error [Header, Failure] (failed (cat (quoted name) (lit " is not a valid local ID"))))) + | Result.error failure => (ctor Result.error [Header, Failure] failure)) + | Option.none => (ctor Result.error [Header, Failure] (failed (cat (quoted modeWord) (lit " is not a binder mode"))))) + | Result.error failure => (ctor Result.error [Header, Failure] failure)) else (ctor Result.error [Header, Failure] (failed (lit "a binder is `(mode name type)`"))))) + | Node.NAtom text => (ctor Result.error [Header, Failure] (failed (cat (lit "expected a binder, found ") (describe node)))) + | Node.NNil => (ctor Result.error [Header, Failure] (failed (cat (lit "expected a binder, found ") (describe node)))) + | Node.NItem head tail => (ctor Result.error [Header, Failure] (failed (cat (lit "expected a binder, found ") (describe node))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: cvFail, + parameters: [st : Cv, failure : Failure], + result: Cv, + body: (record Cv {work := (project st work), values := (project st values), error := (ctor Option.some [Failure] failure)}), + executable: true +} + +definition{ + name: pushVisits, + parameters: [items : (List Node), work : (List Work)], + result: (List Work), + body: (prim list_fold (fun (acc : (List Work)) (item : Node) => (cons (ctor Work.WVisit item) acc)) work (LexLeanStd.Order::reverseList [Node] items) : (List Work)), + executable: true +} + +definition{ + name: binderVisits, + parameters: [binders : (List Node), work : (List Work), fuel : Nat], + result: (Result (List Work) Failure), + body: (prim list_fold (fun [fuel] (acc : (Result (List Work) Failure)) (binder : Node) => (match acc + | Result.ok pending => (match (binderHeader binder fuel) + | Result.ok header => (ctor Result.ok [(List Work), Failure] (cons (ctor Work.WVisit (project header ty)) pending)) + | Result.error failure => (ctor Result.error [(List Work), Failure] failure)) + | Result.error failure => (ctor Result.error [(List Work), Failure] failure))) (ctor Result.ok [(List Work), Failure] work) (LexLeanStd.Order::reverseList [Node] binders) : (Result (List Work) Failure)), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: isBinderForm, + parameters: [head : Bytes], + result: Bool, + body: (if (prim equal head (lit "pi") : Bool) then true else (prim equal head (lit "lam") : Bool)), + executable: true +} + +definition{ + name: isLeafForm, + parameters: [head : Bytes], + result: Bool, + body: (if (prim equal head (lit "sort") : Bool) then true else (if (prim equal head (lit "const") : Bool) then true else (if (prim equal head (lit "local") : Bool) then true else (prim equal head (lit "nat") : Bool)))), + executable: true +} + +definition{ + name: visitBinderForm, + parameters: [ + st : Cv, + rest : (List Work), + node : Node, + head : Bytes, + arguments : (List Node), + fuel : Nat + ], + result: Cv, + body: (if ((prim length arguments : Nat) =? 2) then (match (nodeAt arguments 0) + | Node.NList binderChain => (let binders : (List Node) := (itemsOf binderChain fuel); (match binders + | List.nil => (cvFail st (failed (cat (quoted head) (lit " has at least one binder")))) + | List.cons firstBinder more => (match (binderVisits binders (cons (ctor Work.WVisit (nodeAt arguments 1)) (cons (ctor Work.WBuild node) rest)) fuel) + | Result.ok work => (record Cv {work := work, values := (project st values), error := (ctor Option.none [Failure])}) + | Result.error failure => (cvFail st failure)))) + | Node.NAtom text => (cvFail st (failed (lit "expected a binder list"))) + | Node.NNil => (cvFail st (failed (lit "expected a binder list"))) + | Node.NItem h t => (cvFail st (failed (lit "expected a binder list")))) else (cvFail st (failed (cat (cat (cat (cat (lit "`") head) (lit "` is `(")) head) (lit " (binders...) body)`"))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: visitLet, + parameters: [st : Cv, rest : (List Work), node : Node, arguments : (List Node), fuel : Nat], + result: Cv, + body: (if ((prim length arguments : Nat) =? 4) then (match (atomOf (nodeAt arguments 0) (lit "a local ID") fuel) + | Result.ok name => (if (isLseIdentifier name) then (record Cv {work := (cons (ctor Work.WVisit (nodeAt arguments 1)) (cons (ctor Work.WVisit (nodeAt arguments 2)) (cons (ctor Work.WVisit (nodeAt arguments 3)) (cons (ctor Work.WBuild node) rest)))), values := (project st values), error := (ctor Option.none [Failure])}) else (cvFail st (failed (cat (quoted name) (lit " is not a valid local ID"))))) + | Result.error failure => (cvFail st failure)) else (cvFail st (failed (lit "`let` is `(let name type value body)`")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: visitForm, + parameters: [ + st : Cv, + rest : (List Work), + node : Node, + head : Bytes, + arguments : (List Node), + fuel : Nat + ], + result: Cv, + body: (if (isLeafForm head) then (match (leafOf head arguments fuel) + | Result.ok value => (record Cv {work := rest, values := (cons value (project st values)), error := (ctor Option.none [Failure])}) + | Result.error failure => (cvFail st failure)) else (if (prim equal head (lit "app") : Bool) then (if ((prim length arguments : Nat) (match (itemsOf chain fuel) + | List.nil => (cvFail st (failed (lit "expected an LSE form"))) + | List.cons headNode arguments => (match headNode + | Node.NAtom head => (visitForm st rest node head arguments fuel) + | Node.NList inner => (cvFail st (failed (lit "expected an LSE form"))) + | Node.NNil => (cvFail st (failed (lit "expected an LSE form"))) + | Node.NItem h t => (cvFail st (failed (lit "expected an LSE form"))))) + | Node.NAtom text => (cvFail st (failed (cat (lit "expected `(`, found ") (describe node)))) + | Node.NNil => (cvFail st (failed (cat (lit "expected `(`, found ") (describe node)))) + | Node.NItem head tail => (cvFail st (failed (cat (lit "expected `(`, found ") (describe node))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: takeValues, + parameters: [values : (List Lse), count : Nat], + result: ((List Lse) * (List Lse)), + body: (first (prim iterate_until (fun (state : ((List Lse) * (List Lse))) => (match (second state) + | List.nil => (ctor Option.none [((List Lse) * (List Lse))]) + | List.cons top rest => (ctor Option.some [((List Lse) * (List Lse))] ((cons top (first state)), rest)))) count ((nil Lse), values) : (((List Lse) * (List Lse)) * Bool))), + executable: true +} + +definition{ + name: argChain, + parameters: [items : (List Lse)], + result: Lse, + body: (prim list_fold (fun (chain : Lse) (item : Lse) => (ctor Lse.LArg item chain)) (ctor Lse.LNil) (LexLeanStd.Order::reverseList [Lse] items) : Lse), + executable: true +} + +definition{ + name: binderChainOf, + parameters: [binders : (List Node), types : (List Lse), fuel : Nat], + result: Lse, + body: (second (prim list_fold (fun [fuel] (state : ((List Lse) * Lse)) (binder : Node) => (match (first state) + | List.nil => state + | List.cons ty more => (match (binderHeader binder fuel) + | Result.ok header => (more, (ctor Lse.LBinder (project header mode) (project header name) ty (second state))) + | Result.error failure => (more, (second state))))) ((LexLeanStd.Order::reverseList [Lse] types), (ctor Lse.LNil)) (LexLeanStd.Order::reverseList [Node] binders) : ((List Lse) * Lse))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: cvBuild, + parameters: [st : Cv, rest : (List Work), node : Node, fuel : Nat], + result: Cv, + body: (match node + | Node.NList chain => (let items : (List Node) := (itemsOf chain fuel); (match items + | List.nil => (cvFail st (failed (lit "expected a list"))) + | List.cons headNode arguments => (match headNode + | Node.NAtom head => (if (prim equal head (lit "app") : Bool) then (let taken : ((List Lse) * (List Lse)) := (takeValues (project st values) (prim length arguments : Nat)); (match (first taken) + | List.nil => (cvFail st (failed (lit "missing function"))) + | List.cons function parts => (record Cv {work := rest, values := (cons (ctor Lse.LApp function (argChain parts)) (second taken)), error := (ctor Option.none [Failure])}))) else (if (isBinderForm head) then (match (nodeAt arguments 0) + | Node.NList binderChain => (let binders : (List Node) := (itemsOf binderChain fuel); (let taken : ((List Lse) * (List Lse)) := (takeValues (project st values) ((prim length binders : Nat) + 1)); (match (LexLeanStd.Order::reverseList [Lse] (first taken)) + | List.nil => (cvFail st (failed (lit "missing body"))) + | List.cons body typesReversed => (let bound : Lse := (binderChainOf binders (LexLeanStd.Order::reverseList [Lse] typesReversed) fuel); (record Cv {work := rest, values := (cons (if (prim equal head (lit "pi") : Bool) then (ctor Lse.LPi bound body) else (ctor Lse.LLam bound body)) (second taken)), error := (ctor Option.none [Failure])}))))) + | Node.NAtom text => (cvFail st (failed (lit "expected a binder list"))) + | Node.NNil => (cvFail st (failed (lit "expected a binder list"))) + | Node.NItem h t => (cvFail st (failed (lit "expected a binder list")))) else (let taken : ((List Lse) * (List Lse)) := (takeValues (project st values) 3); (match (first taken) + | List.nil => (cvFail st (failed (lit "missing type"))) + | List.cons ty afterTy => (match afterTy + | List.nil => (cvFail st (failed (lit "missing value"))) + | List.cons value afterValue => (match afterValue + | List.nil => (cvFail st (failed (lit "missing body"))) + | List.cons body more => (match (atomOf (nodeAt arguments 0) (lit "a local ID") fuel) + | Result.ok name => (record Cv {work := rest, values := (cons (ctor Lse.LLet name ty value body) (second taken)), error := (ctor Option.none [Failure])}) + | Result.error failure => (cvFail st failure)))))))) + | Node.NList inner => (cvFail st (failed (lit "expected a form, found a list"))) + | Node.NNil => (cvFail st (failed (lit "expected a form, found a list"))) + | Node.NItem h t => (cvFail st (failed (lit "expected a form, found a list")))))) + | Node.NAtom text => (cvFail st (failed (lit "expected a list"))) + | Node.NNil => (cvFail st (failed (lit "expected a list"))) + | Node.NItem head tail => (cvFail st (failed (lit "expected a list")))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: cvStep, + parameters: [fuel : Nat, st : Cv], + result: (Option Cv), + body: (match (project st error) + | Option.some stopped => (ctor Option.none [Cv]) + | Option.none => (match (project st work) + | List.nil => (ctor Option.none [Cv]) + | List.cons item rest => (match item + | Work.WVisit node => (ctor Option.some [Cv] (cvVisit st rest node fuel)) + | Work.WBuild node => (ctor Option.some [Cv] (cvBuild st rest node fuel))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: lseOf, + parameters: [root : Node, fuel : Nat], + result: (Result Lse Failure), + body: (let final : Cv := (first (prim iterate_until (fun [fuel] (st : Cv) => (cvStep fuel st)) ((prim multiply fuel 2 : Nat) + 2) (record Cv {work := (cons (ctor Work.WVisit root) (nil Work)), values := (nil Lse), error := (ctor Option.none [Failure])}) : (Cv * Bool))); (match (project final error) + | Option.some failure => (ctor Result.error [Lse, Failure] failure) + | Option.none => (match (project final values) + | List.nil => (ctor Result.error [Lse, Failure] (failed (lit "malformed expression"))) + | List.cons value more => (match more + | List.nil => (ctor Result.ok [Lse, Failure] value) + | List.cons another rest => (ctor Result.error [Lse, Failure] (failed (lit "malformed expression"))))))), + axioms: [Quot.sound, propext], + executable: true +} + +definition{ + name: parseLse, + parameters: [text : Bytes, maxDepth : Nat], + result: (Result Lse Failure), + body: (match (tokenize text) + | Result.error failure => (ctor Result.error [Lse, Failure] failure) + | Result.ok toks => (match (readTree toks maxDepth) + | Result.error failure => (ctor Result.error [Lse, Failure] failure) + | Result.ok root => (lseOf root (prim length toks : Nat)))), + axioms: [Quot.sound, propext], + executable: true, + production: {effects: [allocation, overflow], targets: [rust-std]} +} + +definition{ + name: modeText, + parameters: [mode : Nat], + result: Bytes, + body: (if (mode =? 1) then (lit "implicit") else (if (mode =? 2) then (lit "instance") else (lit "explicit"))), + executable: true +} + +definition{ + name: fresh, + parameters: [name : Bytes, rename : Bool, counter : Nat], + result: (Bytes * Nat), + body: (if rename then ((cat (lit "x") (LexLeanStd.Numeral::decimal counter)), (counter + 1)) else (name, counter)), + executable: true +} + +definition{ + name: lookup, + parameters: [scope : (List (Bytes * Bytes)), name : Bytes], + result: Bytes, + body: (match (prim list_fold (fun [name] (found : (Option Bytes)) (entry : (Bytes * Bytes)) => (match found + | Option.some shown => found + | Option.none => (if (prim equal (first entry) name : Bool) then (ctor Option.some [Bytes] (second entry)) else (ctor Option.none [Bytes])))) (ctor Option.none [Bytes]) scope : (Option Bytes)) + | Option.some shown => shown + | Option.none => name), + executable: true +} + +definition{ + name: universeText, + parameters: [level : Universe], + result: Bytes, + body: (match level + | Universe.UNum value => (LexLeanStd.Numeral::decimal value) + | Universe.UVar name => name + | Universe.USucc inner => (cat (cat (lit "(succ ") (universeText inner)) (lit ")")) + | Universe.UMax items => (cat (cat (lit "(max") (universeItems items)) (lit ")")) + | Universe.UIMax left right => (cat (cat (cat (cat (lit "(imax ") (universeText left)) (lit " ")) (universeText right)) (lit ")")) + | Universe.UNil => 0x + | Universe.UItem head tail => 0x), + executable: true, + mutual: Level, + recursive_argument: level +} + +definition{ + name: universeItems, + parameters: [items : Universe], + result: Bytes, + body: (match items + | Universe.UNum value => 0x + | Universe.UVar name => 0x + | Universe.USucc inner => 0x + | Universe.UMax inner => 0x + | Universe.UIMax left right => 0x + | Universe.UNil => 0x + | Universe.UItem head tail => (cat (cat (lit " ") (universeText head)) (universeItems tail))), + executable: true, + mutual: Level, + recursive_argument: items +} + +definition{ + name: constText, + parameters: [package : Bytes, entry : Bytes, universes : Universe], + result: Bytes, + body: (let heading : Bytes := (cat (cat (cat (lit "(const ") package) (lit "::")) entry); (match universes + | Universe.UNil => (cat heading (lit ")")) + | Universe.UItem head tail => (let listed : Bytes := (universeItems universes); (cat (cat (cat heading (lit " (")) (LexLeanStd.Text::sliceOr listed 1 (prim subtract (prim length listed : Nat) 1 : Nat))) (lit "))"))) + | Universe.UNum value => (cat heading (lit ")")) + | Universe.UVar name => (cat heading (lit ")")) + | Universe.USucc inner => (cat heading (lit ")")) + | Universe.UMax inner => (cat heading (lit ")")) + | Universe.UIMax left right => (cat heading (lit ")")))), + executable: true +} + +definition{ + name: showExpr, + parameters: [value : Lse, rename : Bool, scope : (List (Bytes * Bytes)), counter : Nat], + result: (Bytes * Nat), + body: (match value + | Lse.LSortProp => ((lit "(sort prop)"), counter) + | Lse.LSortType level => ((cat (cat (lit "(sort (type ") (universeText level)) (lit "))")), counter) + | Lse.LConst package entry universes => ((constText package entry universes), counter) + | Lse.LLocal name => ((cat (cat (lit "(local ") (if rename then (lookup scope name) else name)) (lit ")")), counter) + | Lse.LApp function arguments => (let shownFunction : (Bytes * Nat) := (showExpr function rename scope counter); (let shownArguments : (Bytes * Nat) := (showArgs arguments rename scope (second shownFunction)); ((cat (cat (cat (lit "(app ") (first shownFunction)) (first shownArguments)) (lit ")")), (second shownArguments)))) + | Lse.LPi binders body => (let shownBinders : ((Bytes * (List (Bytes * Bytes))) * Nat) := (showBinders binders rename scope counter); (let shownBody : (Bytes * Nat) := (showExpr body rename (second (first shownBinders)) (second shownBinders)); ((cat (cat (cat (cat (lit "(pi (") (first (first shownBinders))) (lit ") ")) (first shownBody)) (lit ")")), (second shownBody)))) + | Lse.LLam binders body => (let shownBinders : ((Bytes * (List (Bytes * Bytes))) * Nat) := (showBinders binders rename scope counter); (let shownBody : (Bytes * Nat) := (showExpr body rename (second (first shownBinders)) (second shownBinders)); ((cat (cat (cat (cat (lit "(lam (") (first (first shownBinders))) (lit ") ")) (first shownBody)) (lit ")")), (second shownBody)))) + | Lse.LLet name ty bound body => (let shownName : (Bytes * Nat) := (fresh name rename counter); (let inner : (List (Bytes * Bytes)) := (cons (name, (first shownName)) scope); (let shownType : (Bytes * Nat) := (showExpr ty rename inner (second shownName)); (let shownValue : (Bytes * Nat) := (showExpr bound rename inner (second shownType)); (let shownBody : (Bytes * Nat) := (showExpr body rename inner (second shownValue)); ((cat (cat (cat (cat (cat (cat (cat (lit "(let ") (first shownName)) (lit " ")) (first shownType)) (lit " ")) (first shownValue)) (lit " ")) (cat (first shownBody) (lit ")"))), (second shownBody))))))) + | Lse.LNat digits => ((cat (cat (lit "(nat ") digits) (lit ")")), counter) + | Lse.LNil => (0x, counter) + | Lse.LArg head tail => (0x, counter) + | Lse.LBinder mode name ty rest => (0x, counter)), + executable: true, + mutual: Show, + recursive_argument: value +} + +definition{ + name: showArgs, + parameters: [arguments : Lse, rename : Bool, scope : (List (Bytes * Bytes)), counter : Nat], + result: (Bytes * Nat), + body: (match arguments + | Lse.LArg head tail => (let shownHead : (Bytes * Nat) := (showExpr head rename scope counter); (let shownTail : (Bytes * Nat) := (showArgs tail rename scope (second shownHead)); ((cat (cat (lit " ") (first shownHead)) (first shownTail)), (second shownTail)))) + | Lse.LNil => (0x, counter) + | Lse.LSortProp => (0x, counter) + | Lse.LSortType level => (0x, counter) + | Lse.LConst package entry universes => (0x, counter) + | Lse.LLocal name => (0x, counter) + | Lse.LApp function inner => (0x, counter) + | Lse.LPi binders body => (0x, counter) + | Lse.LLam binders body => (0x, counter) + | Lse.LLet name ty bound body => (0x, counter) + | Lse.LNat digits => (0x, counter) + | Lse.LBinder mode name ty rest => (0x, counter)), + executable: true, + mutual: Show, + recursive_argument: arguments +} + +definition{ + name: showBinders, + parameters: [binders : Lse, rename : Bool, scope : (List (Bytes * Bytes)), counter : Nat], + result: ((Bytes * (List (Bytes * Bytes))) * Nat), + body: (match binders + | Lse.LBinder mode name ty rest => (let shownName : (Bytes * Nat) := (fresh name rename counter); (let shownType : (Bytes * Nat) := (showExpr ty rename scope (second shownName)); (let piece : Bytes := (cat (cat (cat (cat (cat (cat (lit "(") (modeText mode)) (lit " ")) (first shownName)) (lit " ")) (first shownType)) (lit ")")); (let inner : (List (Bytes * Bytes)) := (cons (name, (first shownName)) scope); (match rest + | Lse.LNil => ((piece, inner), (second shownType)) + | Lse.LBinder m n t r => (let shownRest : ((Bytes * (List (Bytes * Bytes))) * Nat) := (showBinders rest rename inner (second shownType)); (((cat (cat piece (lit " ")) (first (first shownRest))), (second (first shownRest))), (second shownRest))) + | Lse.LSortProp => ((piece, inner), (second shownType)) + | Lse.LSortType level => ((piece, inner), (second shownType)) + | Lse.LConst package entry universes => ((piece, inner), (second shownType)) + | Lse.LLocal other => ((piece, inner), (second shownType)) + | Lse.LApp function arguments => ((piece, inner), (second shownType)) + | Lse.LPi bs body => ((piece, inner), (second shownType)) + | Lse.LLam bs body => ((piece, inner), (second shownType)) + | Lse.LLet n t v body => ((piece, inner), (second shownType)) + | Lse.LNat digits => ((piece, inner), (second shownType)) + | Lse.LArg head tail => ((piece, inner), (second shownType))))))) + | Lse.LNil => ((0x, scope), counter) + | Lse.LSortProp => ((0x, scope), counter) + | Lse.LSortType level => ((0x, scope), counter) + | Lse.LConst package entry universes => ((0x, scope), counter) + | Lse.LLocal name => ((0x, scope), counter) + | Lse.LApp function arguments => ((0x, scope), counter) + | Lse.LPi bs body => ((0x, scope), counter) + | Lse.LLam bs body => ((0x, scope), counter) + | Lse.LLet name ty bound body => ((0x, scope), counter) + | Lse.LNat digits => ((0x, scope), counter) + | Lse.LArg head tail => ((0x, scope), counter)), + executable: true, + mutual: Show, + recursive_argument: binders +} + +definition{ + name: printLse, + parameters: [value : Lse, rename : Bool], + result: Bytes, + body: (first (showExpr value rename (nil (Bytes * Bytes)) 0)), + executable: true, + production: {effects: [allocation, overflow, recursion], targets: [rust-std]} +} +\end{semanticmodule} +\end{lexlean} diff --git a/crates/conformance/src/cases/bootstrap.rs b/crates/conformance/src/cases/bootstrap.rs index 49c5bbf14c..0303f0fa7a 100644 --- a/crates/conformance/src/cases/bootstrap.rs +++ b/crates/conformance/src/cases/bootstrap.rs @@ -22,6 +22,7 @@ use crate::rust_packages; use crate::support::{self, P}; mod gnaf_validate; +mod lexicon_lse; mod runtime_validate; mod rust_assemble; mod rust_bodies; @@ -2457,6 +2458,7 @@ fn case(id: &str) { "BS-16" => gnaf_validate::case(), // §17.23, §17.20: the canonical LaTeX of a tooling module. "BS-20" => tooling_latex::case(), + "BS-25" => lexicon_lse::case(), other => panic!("no bootstrap case is wired for {other}"), } } diff --git a/crates/conformance/src/cases/bootstrap/lexicon_lse.rs b/crates/conformance/src/cases/bootstrap/lexicon_lse.rs new file mode 100755 index 0000000000..8cc5c85c3d --- /dev/null +++ b/crates/conformance/src/cases/bootstrap/lexicon_lse.rs @@ -0,0 +1,1273 @@ +//! `BS-25` (SPEC.md §17.23, §13.8): the reader and the canonical printer of +//! LexLean Semantic Expressions as language-1.2 source, `LexLeanBoot.LexiconLse`, +//! held to the byte to `lexlean::lexicon::lse::parse` and `Lse::print`. +//! +//! `parseLse` tokenizes a text, reads the tokens into one S-expression tree +//! with an explicit stack of open lists, bounded by the configured depth, and +//! converts the tree into an expression with a work stack, so that it admits no +//! `recursion`; it returns the expression or the kind and message of the first +//! error, in the order the oracle finds them. `printLse` prints an expression +//! canonically, renaming binders `x0`, `x1`, ... in printing order when asked, +//! by the structural recursion the oracle's work stack unrolls, so it admits +//! `recursion`. +//! +//! Each planted defect names the one vector that catches it. + +use std::collections::{BTreeMap, BTreeSet}; +use std::sync::OnceLock; + +use lexlean::calculus::Value; +use lexlean::lexicon::lse::{parse, BinderMode, Lse, LseBinder, ParseError, Universe}; +use serde_json::Value as Json2; + +use super::rust_validate::{eligibility_of, rustc_differential}; +use super::{text, Run, Seeded}; +use crate::production_export as harness; +use crate::support::{self, repo_root}; + +const MODULE: &str = "LexLeanBoot.LexiconLse"; +const SOURCE: &str = "bootstrap/src/LexiconLse.lex.tex"; +const ROOTS: [(&str, &[&str]); 2] = [ + ("parseLse", &["allocation", "overflow"]), + ("printLse", &["allocation", "overflow", "recursion"]), +]; + +/// The most a part of a certificate of a root may be, as the row states. +const CERTIFICATE_BUDGET: u64 = 3_145_728; +/// The depth the committed expressions and the seeded ones are read at; the +/// vectors that reach the limit name their own. +const DEPTH: u64 = 64; +/// Seeded expressions under the evaluator, and under rustc. +const EVALUATOR_SEEDED: usize = 300; +const RUSTC_SEEDED: usize = 1_500; + +// ---------------------------------------------------------------- the sorts + +/// The constructors of an inductive of the module, in declaration order, read +/// from its text, so that the encoder names a constructor and never numbers +/// it, and one moved in the module fails here. +fn constructors(sort: &str) -> &'static Vec { + static TABLES: OnceLock>> = OnceLock::new(); + let tables = TABLES.get_or_init(|| { + let source = std::fs::read_to_string(repo_root().join(SOURCE).as_std_path()) + .expect("the module's source"); + let mut out = BTreeMap::new(); + for block in source.split("inductive{").skip(1) { + let block = block.split("\n}\n").next().expect("a block"); + let name = block + .split("name: ") + .nth(1) + .and_then(|rest| rest.split(',').next()) + .expect("the inductive's name") + .trim() + .to_owned(); + let names: Vec = block + .split("{name: ") + .skip(1) + .map(|rest| { + rest.split(',') + .next() + .expect("a constructor") + .trim() + .to_owned() + }) + .collect(); + out.insert(name, names); + } + out + }); + tables + .get(sort) + .unwrap_or_else(|| panic!("the module has no inductive `{sort}`")) +} + +fn adt(sort: &str, constructor: &str, fields: Vec) -> Value { + let found = constructors(sort) + .iter() + .position(|name| name == constructor) + .unwrap_or_else(|| panic!("`{sort}` has no constructor `{constructor}`")); + Value::Adt { + constructor: u64::try_from(found).expect("an index"), + fields, + } +} + +// ------------------------------------------------------------- the encoders + +fn nat_u64(value: u64) -> Value { + Value::Nat { + value: value.to_string(), + } +} + +fn mode_nat(mode: BinderMode) -> Value { + nat_u64(match mode { + BinderMode::Explicit => 0, + BinderMode::Implicit => 1, + BinderMode::Instance => 2, + }) +} + +/// A list of universes as the module's chain of `UItem` cells. +fn universe_chain(items: &[Universe]) -> Value { + items + .iter() + .rev() + .fold(adt("Universe", "UNil", Vec::new()), |rest, item| { + adt("Universe", "UItem", vec![enc_universe(item), rest]) + }) +} + +fn enc_universe(universe: &Universe) -> Value { + match universe { + Universe::Num(value) => adt("Universe", "UNum", vec![nat_u64(*value)]), + Universe::Var(name) => adt("Universe", "UVar", vec![text(name)]), + Universe::Succ(inner) => adt("Universe", "USucc", vec![enc_universe(inner)]), + Universe::Max(items) => adt("Universe", "UMax", vec![universe_chain(items)]), + Universe::IMax(left, right) => adt( + "Universe", + "UIMax", + vec![enc_universe(left), enc_universe(right)], + ), + } +} + +/// Arguments as the module's chain of `LArg` cells. +fn arg_chain(items: &[Lse]) -> Value { + items + .iter() + .rev() + .fold(adt("Lse", "LNil", Vec::new()), |rest, item| { + adt("Lse", "LArg", vec![enc_lse(item), rest]) + }) +} + +/// Binders as the module's chain of `LBinder` cells. +fn binder_chain(binders: &[LseBinder]) -> Value { + binders + .iter() + .rev() + .fold(adt("Lse", "LNil", Vec::new()), |rest, binder| { + adt( + "Lse", + "LBinder", + vec![ + mode_nat(binder.mode), + text(&binder.name), + enc_lse(&binder.ty), + rest, + ], + ) + }) +} + +/// An expression as the module's `Lse`. +fn enc_lse(expression: &Lse) -> Value { + match expression { + Lse::SortProp => adt("Lse", "LSortProp", Vec::new()), + Lse::SortType(universe) => adt("Lse", "LSortType", vec![enc_universe(universe)]), + Lse::Const(id, universes) => adt( + "Lse", + "LConst", + vec![ + text(&id.package), + text(&id.entry), + universe_chain(universes), + ], + ), + Lse::Local(name) => adt("Lse", "LLocal", vec![text(name)]), + Lse::App(function, arguments) => { + adt("Lse", "LApp", vec![enc_lse(function), arg_chain(arguments)]) + } + Lse::Pi(binders, body) => adt("Lse", "LPi", vec![binder_chain(binders), enc_lse(body)]), + Lse::Lam(binders, body) => adt("Lse", "LLam", vec![binder_chain(binders), enc_lse(body)]), + Lse::Let { + name, + ty, + value, + body, + } => adt( + "Lse", + "LLet", + vec![text(name), enc_lse(ty), enc_lse(value), enc_lse(body)], + ), + Lse::Nat(digits) => adt("Lse", "LNat", vec![text(digits)]), + } +} + +/// The run of `parseLse` on a text at a depth, and the oracle's message when +/// it refuses. +fn parse_run(source: &str, depth: u64) -> (Run, Option) { + let (value, refusal) = match parse(source, depth) { + Ok(expression) => ( + Value::Ok { + value: Box::new(enc_lse(&expression)), + }, + None, + ), + Err(error) => { + let kind = match error { + ParseError::Syntax(_) => 0, + ParseError::DepthExceeded(_) => 1, + }; + let message = error.to_string(); + ( + Value::Error { + value: Box::new(Value::Adt { + constructor: 0, + fields: vec![nat_u64(kind), text(&message)], + }), + }, + Some(message), + ) + } + }; + ( + ( + vec![text(source), nat_u64(depth)], + serde_json::to_value(value).expect("a value is JSON"), + ), + refusal, + ) +} + +/// The run of `printLse` on an expression. +fn print_run(expression: &Lse, rename: bool) -> Run { + ( + vec![enc_lse(expression), Value::Bool { value: rename }], + serde_json::to_value(text(&expression.print(rename))).expect("a value is JSON"), + ) +} + +// ------------------------------------------------------------------ vectors + +/// What a vector is a run of. +enum Subject { + Parse(String, u64), + Print(Lse, bool), +} + +/// One named vector. +struct Vector { + label: String, + subject: Subject, +} + +impl Vector { + fn printing(&self) -> bool { + matches!(self.subject, Subject::Print(..)) + } + + fn run(&self) -> (Run, Option) { + match &self.subject { + Subject::Parse(source, depth) => parse_run(source, *depth), + Subject::Print(expression, rename) => (print_run(expression, *rename), None), + } + } +} + +/// The oracle's reading of a text the vectors print. +fn parsed(source: &str) -> Lse { + parse(source, DEPTH).unwrap_or_else(|error| panic!("`{source}` parses: {error}")) +} + +#[allow(clippy::too_many_lines)] +fn named() -> Vec { + let mut out: Vec = Vec::new(); + let mut add = |label: &str, subject: Subject| { + out.push(Vector { + label: label.to_owned(), + subject, + }); + }; + let reading = |source: &str| Subject::Parse(source.to_owned(), DEPTH); + + // Reading: what is accepted. + for (label, source) in [ + ("a sort prop", "(sort prop)"), + ("a sort type of a number", "(sort (type 0))"), + ("a sort type of a variable", "(sort (type u))"), + ("a sort type of succ", "(sort (type (succ u)))"), + ("a sort type of max", "(sort (type (max u v 1)))"), + ("a sort type of imax", "(sort (type (imax u v)))"), + ( + "a sort type of nested universes", + "(sort (type (max (succ u) (imax 1 v) (succ (succ 0)))))", + ), + ("a const", "(const lexlean.std.nat::nat)"), + ("a const with universes", "(const pkg::entry (u 1))"), + ("a const with one universe", "(const pkg::entry ((succ u)))"), + ( + "a const of a package with segments and hyphens", + "(const a.b2.c-d::e.f-g9)", + ), + ("a const with a nested entry", "(const pkg::a.b)"), + ("a local", "(local x)"), + ("a local with digits", "(local x1)"), + ("a local of capitals", "(local Xy)"), + ("an app", "(app (local f) (local a))"), + ("an app of three", "(app (local f) (local a) (local b))"), + ( + "an app of apps", + "(app (app (local f) (local a)) (app (local g) (local b)))", + ), + ("a pi", "(pi ((explicit a (sort prop))) (local a))"), + ( + "a pi of three binders", + "(pi ((explicit a (sort prop)) (implicit b (local a)) (instance c (local b))) (app (local b) (local c)))", + ), + ("a lam", "(lam ((explicit x (const p::t))) (local x))"), + ( + "nested binders", + "(pi ((explicit a (sort prop))) (lam ((explicit b (local a))) (local b)))", + ), + ("a let", "(let x (sort prop) (local y) (local x))"), + ( + "a let in a let", + "(let x (sort prop) (sort prop) (let y (local x) (local x) (local y)))", + ), + ("a nat", "(nat 0)"), + ("a large nat", "(nat 18446744073709551615)"), + ("a nat of many digits", "(nat 123456789)"), + ("white space of both kinds", "(app\n(local f) (local a)\n)"), + ("leading and trailing white space", " (sort prop)\n"), + ("atoms touching parentheses", "(app(local f)(local a))"), + ( + "the oracle's own signature", + "(pi ((explicit a (const lexlean.std.nat::nat)) (explicit b (const lexlean.std.nat::nat))) (const lexlean.std.nat::nat))", + ), + ] { + add(label, reading(source)); + } + + // Reading: the limit on nesting. + add( + "nesting at the depth", + Subject::Parse("(app (local f) (local a))".to_owned(), 2), + ); + add( + "nesting beyond the depth", + Subject::Parse("(app (local f) (local a))".to_owned(), 1), + ); + add( + "a parenthesis at depth zero", + Subject::Parse("(sort prop)".to_owned(), 0), + ); + add("an atom at depth zero", Subject::Parse("x".to_owned(), 0)); + + // Reading: what is refused. + for (label, source) in [ + ("the empty text", ""), + ("white space only", " \n"), + ("an unbalanced close", ")"), + ("an unclosed open", "("), + ("an unclosed inner list", "(app (local f)"), + ("trailing tokens", "(sort prop) x"), + ("two expressions", "(sort prop) (sort prop)"), + ("a tab", "(sort\tprop)"), + ("a carriage return", "(sort prop)\r"), + ("a bare atom", "x"), + ("an empty list", "()"), + ("a list headed by a list", "((sort prop))"), + ("an unknown form", "(foo)"), + ("a sort of nothing", "(sort)"), + ("a sort of a word", "(sort set)"), + ("a sort type without a universe", "(sort (type))"), + ("a sort type of two universes", "(sort (type u v))"), + ("a sort of a list that is not type", "(sort (kind u))"), + ("a sort type of a bad universe", "(sort (type 01))"), + ("a universe that is a word", "(sort (type u-v))"), + ("a universe form unknown", "(sort (type (plus u v)))"), + ("a succ of two", "(sort (type (succ u v)))"), + ("a max of one", "(sort (type (max u)))"), + ("an imax of three", "(sort (type (imax u v w)))"), + ("a universe list without a head", "(sort (type ((u))))"), + ("a universe too large", "(sort (type 18446744073709551616))"), + ("a const of nothing", "(const)"), + ("a const with empty universes", "(const p::e ())"), + ("a const of two atoms", "(const p::e u)"), + ("a const without a scope", "(const nat)"), + ("a const with a bad package", "(const Pkg::e)"), + ( + "a const with a hyphen in the first package segment", + "(const a-b::e)", + ), + ("a const with a bad entry", "(const p::E)"), + ("a const with an empty entry", "(const p::)"), + ("a const with an empty package", "(const ::e)"), + ("a const with a bad universe", "(const p::e (01))"), + ("a const with a double scope", "(const p::e::f)"), + ("a local of nothing", "(local)"), + ("a local of two", "(local x y)"), + ("a local that is a list", "(local (x))"), + ("a local with a bad name", "(local 1x)"), + ("a local with an underscore", "(local x_y)"), + ("a nat of nothing", "(nat)"), + ("a nat of a word", "(nat x)"), + ("a nat with a leading zero", "(nat 01)"), + ("a nat too large", "(nat 18446744073709551616)"), + ("a nat of twenty-one digits", "(nat 100000000000000000000)"), + ("a nat that is a list", "(nat (1))"), + ("an app of one", "(app (local f))"), + ("an app of nothing", "(app)"), + ("an app with a bad argument", "(app (local f) x)"), + ("an app with a bad function", "(app x (local a))"), + ("a pi of one", "(pi ((explicit a (sort prop))))"), + ( + "a lam of three", + "(lam ((explicit a (sort prop))) (local a) (local a))", + ), + ("a pi whose binders are an atom", "(pi x (local x))"), + ("a pi of no binders", "(pi () (sort prop))"), + ("a binder that is an atom", "(pi (x) (local x))"), + ("a binder of two", "(pi ((explicit a)) (local a))"), + ( + "a binder with a bad mode", + "(pi ((strict a (sort prop))) (local a))", + ), + ( + "a binder whose mode is a list", + "(pi (((explicit) a (sort prop))) (local a))", + ), + ( + "a binder whose name is a list", + "(pi ((explicit (a) (sort prop))) (local a))", + ), + ( + "a binder with a bad name", + "(pi ((explicit 1a (sort prop))) (local a))", + ), + ( + "the last binder is checked first", + "(pi ((bad a (sort prop)) (explicit 1b (sort prop))) (sort prop))", + ), + ( + "a binder with a bad type", + "(pi ((explicit a x)) (local a))", + ), + ("a let of three", "(let x (sort prop) (local y))"), + ( + "a let whose name is a list", + "(let (x) (sort prop) (sort prop) (sort prop))", + ), + ( + "a let with a bad name", + "(let 1x (sort prop) (sort prop) (sort prop))", + ), + ( + "the function's error comes before the argument's", + "(app (foo) (bar))", + ), + ( + "a binder type's error comes before the body's", + "(pi ((explicit a (foo))) (bar))", + ), + ( + "a leaf error inside a deep form", + "(app (local f) (app (local g) (nat 01)))", + ), + ] { + add(label, reading(source)); + } + + // Printing: every form, as read and renamed. + for (label, source) in [ + ("a sort prop printed", "(sort prop)"), + ("a sort type printed", "(sort (type (max (succ u) (imax 1 v))))"), + ("a const printed", "(const lexlean.std.nat::nat)"), + ("a const with universes printed", "(const pkg::entry (u (succ 1) (max u v)))"), + ("a local printed", "(local x)"), + ("an app printed", "(app (local f) (local a) (local b))"), + ("a nat printed", "(nat 42)"), + ( + "white space canonicalized", + "(app\n(local f) (local a)\n)", + ), + ( + "a pi renamed", + "(pi ((explicit a (sort prop)) (explicit b (local a))) (app (local a) (local b)))", + ), + ( + "a lam of every mode renamed", + "(lam ((explicit a (sort prop)) (implicit b (local a)) (instance c (local b))) (local c))", + ), + ( + "shadowing", + "(pi ((explicit a (sort prop))) (lam ((explicit a (local a))) (local a)))", + ), + ("an unbound local renamed stays", "(app (local f) (local z))"), + ("a let renamed", "(let x (local x) (local x) (local x))"), + ( + "a let name shadows a binder", + "(pi ((explicit x (sort prop))) (let x (local x) (local x) (local x)))", + ), + ( + "the counter runs through siblings", + "(app (pi ((explicit a (sort prop))) (local a)) (lam ((explicit b (sort prop))) (local b)))", + ), + ( + "a binder whose type names itself", + "(pi ((explicit a (local a))) (sort prop))", + ), + ( + "the oracle's own signature printed", + "(pi ((explicit a (const lexlean.std.nat::nat)) (explicit b (const lexlean.std.nat::nat))) (const lexlean.std.nat::nat))", + ), + ] { + add(label, Subject::Print(parsed(source), false)); + add( + &format!("{label}, renamed"), + Subject::Print(parsed(source), true), + ); + } + out +} + +// ------------------------------------------------------------------- seeded + +const NAMES: &[&str] = &["a", "b", "x", "y", "f", "g", "u1", "Zed"]; +const UNIVERSES: &[&str] = &["u", "v", "w1"]; +const CONSTS: &[&str] = &[ + "lexlean.std.nat::nat", + "lexlean.std.nat::add", + "core::eq", + "p.q-r::s.t-u", +]; + +fn drawn_universe(seeded: &mut Seeded, depth: usize) -> Universe { + let kind = if depth == 0 { + seeded.below(2) + } else { + seeded.below(5) + }; + match kind { + 0 => Universe::Num(*seeded.pick(&[0u64, 1, 2, 7, 18_446_744_073_709_551_615])), + 1 => Universe::Var((*seeded.pick(UNIVERSES)).to_owned()), + 2 => Universe::Succ(Box::new(drawn_universe(seeded, depth - 1))), + 3 => { + let count = 2 + seeded.below(2); + Universe::Max( + (0..count) + .map(|_| drawn_universe(seeded, depth - 1)) + .collect(), + ) + } + _ => Universe::IMax( + Box::new(drawn_universe(seeded, depth - 1)), + Box::new(drawn_universe(seeded, depth - 1)), + ), + } +} + +fn drawn_binders(seeded: &mut Seeded, depth: usize) -> Vec { + let count = 1 + seeded.below(3); + (0..count) + .map(|_| LseBinder { + mode: *seeded.pick(&[ + BinderMode::Explicit, + BinderMode::Explicit, + BinderMode::Implicit, + BinderMode::Instance, + ]), + name: (*seeded.pick(NAMES)).to_owned(), + ty: drawn_lse(seeded, depth - 1), + }) + .collect() +} + +/// An expression of at most `depth` levels. +fn drawn_lse(seeded: &mut Seeded, depth: usize) -> Lse { + let kind = if depth == 0 { + seeded.below(5) + } else { + seeded.below(9) + }; + match kind { + 0 => Lse::SortProp, + 1 => Lse::SortType(drawn_universe(seeded, 2)), + 2 => { + let id = lexlean::lexicon::lse::QualifiedId::parse(seeded.pick(CONSTS)) + .expect("a qualified ID"); + let universes = if seeded.chance(40) { + (0..=seeded.below(2)) + .map(|_| drawn_universe(seeded, 1)) + .collect() + } else { + Vec::new() + }; + Lse::Const(id, universes) + } + 3 => Lse::Local((*seeded.pick(NAMES)).to_owned()), + 4 => Lse::Nat((*seeded.pick(&["0", "1", "42", "18446744073709551615"])).to_owned()), + 5 => { + let count = 1 + seeded.below(3); + Lse::App( + Box::new(drawn_lse(seeded, depth - 1)), + (0..count).map(|_| drawn_lse(seeded, depth - 1)).collect(), + ) + } + 6 => Lse::Pi( + drawn_binders(seeded, depth), + Box::new(drawn_lse(seeded, depth - 1)), + ), + 7 => Lse::Lam( + drawn_binders(seeded, depth), + Box::new(drawn_lse(seeded, depth - 1)), + ), + _ => Lse::Let { + name: (*seeded.pick(NAMES)).to_owned(), + ty: Box::new(drawn_lse(seeded, depth - 1)), + value: Box::new(drawn_lse(seeded, depth - 1)), + body: Box::new(drawn_lse(seeded, depth - 1)), + }, + } +} + +/// Pieces an edit inserts or substitutes. +const PIECES: &[&str] = &[ + "(", + ")", + " ", + "\n", + "\t", + "sort", + "prop", + "type", + "app", + "pi", + "lam", + "let", + "local", + "const", + "nat", + "explicit", + "x", + "0", + "01", + "::", + "-", + ".", + "(local x)", + "(sort prop)", + "(explicit a (sort prop))", +]; + +/// One seeded edit of a text at a byte boundary, up to three of them. +fn edited(seeded: &mut Seeded, source: &str) -> String { + let mut scalars: Vec = source.chars().collect(); + for _ in 0..=seeded.below(3) { + let at = seeded.below(scalars.len() + 1); + let piece: Vec = seeded.pick(PIECES).chars().collect(); + match seeded.below(3) { + 0 if at < scalars.len() => { + scalars.remove(at); + } + 1 => { + drop(scalars.splice(at..at, piece)); + } + _ if at < scalars.len() => { + drop(scalars.splice(at..=at, piece)); + } + _ => scalars.extend(piece), + } + } + scalars.into_iter().collect() +} + +/// `count` seeded expressions, with the runs of `printLse` on them, and the +/// runs of `parseLse` on their printed texts and on edits of those. +fn seeded_runs(count: usize, bases: &[String]) -> (Vec, Vec) { + let mut seeded = Seeded(0x0B55_0025); + let mut printing = Vec::with_capacity(count); + let mut reading = Vec::with_capacity(2 * count); + while printing.len() < count { + let depth = 1 + seeded.below(4); + let expression = drawn_lse(&mut seeded, depth); + let rename = seeded.chance(50); + let printed = expression.print(false); + printing.push(print_run(&expression, rename)); + reading.push(parse_run(&printed, DEPTH).0); + let base = if seeded.chance(30) && !bases.is_empty() { + seeded.pick(bases).clone() + } else { + printed + }; + reading.push(parse_run(&edited(&mut seeded, &base), DEPTH).0); + } + (printing, reading) +} + +/// Every LSE text a committed lexicon entry carries: the `signature` and +/// `value` fields of every `entries/*.toml`, with its path. +fn committed() -> Vec<(String, String)> { + let root = repo_root(); + let walker = walkdir::WalkDir::new(root.as_std_path()) + .sort_by_file_name() + .into_iter() + .filter_entry(|entry| { + let name = entry.file_name().to_string_lossy(); + !(entry.file_type().is_dir() + && (matches!(name.as_ref(), "target" | ".git" | ".lexlean") + || support::holds_cache_tag(entry))) + }); + let mut out = Vec::new(); + for entry in walker.flatten() { + let path = entry.path(); + let shown = path.to_string_lossy().replace('\\', "/"); + if !entry.file_type().is_file() || !shown.ends_with(".toml") || !shown.contains("/entries/") + { + continue; + } + let Ok(content) = std::fs::read_to_string(path) else { + continue; + }; + let relative = path + .strip_prefix(root.as_std_path()) + .expect("under the root") + .to_string_lossy() + .replace('\\', "/"); + for line in content.lines() { + for field in ["signature = \"", "value = \""] { + if let Some(rest) = line.strip_prefix(field) { + if let Some(inner) = rest.strip_suffix('"') { + if inner.starts_with('(') { + out.push((relative.clone(), inner.to_owned())); + } + } + } + } + } + } + out +} + +// ------------------------------------------------------------------- plants + +/// A defect planted in the source, the root it is in, and the named vector +/// that catches it. +struct Plant { + label: &'static str, + root: usize, + site: &'static str, + replacement: &'static str, + vector: &'static str, +} + +const PLANTS: &[Plant] = &[ + Plant { + label: "the depth admits one level more", + root: 0, + site: "(prim length (project st stack) : Nat) (rtFail st (failed (lit \"unbalanced `)`\")))", + replacement: "| List.nil => (rtPlace st (ctor Node.NNil) (nil (List Node)))", + vector: "an unbalanced close", + }, + Plant { + label: "trailing tokens are ignored", + root: 0, + site: "| Option.some found => (ctor Option.some [Rt] (rtFail st (failed (lit \"trailing tokens after the expression\"))))", + replacement: "| Option.some found => (ctor Option.some [Rt] (record Rt {pos := (project st pos), stack := (project st stack), root := (project st root), error := (ctor Option.none [Failure]), done := true}))", + vector: "trailing tokens", + }, + Plant { + label: "the end of the text closes an open list", + root: 0, + site: "| List.cons top rest => (ctor Option.some [Rt] (rtFail st (failed (lit \"unexpected end of expression\"))))", + replacement: "| List.cons top rest => (ctor Option.some [Rt] (rtToken maxDepth st (ctor Tok.TClose)))", + vector: "an unclosed open", + }, + Plant { + label: "app admits one argument fewer", + root: 0, + site: "(if ((prim length arguments : Nat) (cvFail st (failed (cat (quoted head) (lit \" has at least one binder\"))))", + replacement: "| List.nil => (record Cv {work := (cons (ctor Work.WVisit (nodeAt arguments 1)) (cons (ctor Work.WBuild node) rest)), values := (project st values), error := (ctor Option.none [Failure])})", + vector: "a pi of no binders", + }, + Plant { + label: "binders are checked first to last", + root: 0, + site: "(LexLeanStd.Order::reverseList [Node] binders) : (Result (List Work) Failure))", + replacement: "binders : (Result (List Work) Failure))", + vector: "the last binder is checked first", + }, + Plant { + label: "a binder's name need not be a name", + root: 0, + site: "(if (isLseIdentifier name) then (ctor Result.ok [Header, Failure]", + replacement: "(if true then (ctor Result.ok [Header, Failure]", + vector: "a binder with a bad name", + }, + Plant { + label: "a let's name need not be a name", + root: 0, + site: "(if (isLseIdentifier name) then (record Cv", + replacement: "(if true then (record Cv", + vector: "a let with a bad name", + }, + Plant { + label: "a local need not be a name", + root: 0, + site: "(if (isLseIdentifier name) then (ctor Result.ok [Lse, Failure] (ctor Lse.LLocal name))", + replacement: "(if true then (ctor Result.ok [Lse, Failure] (ctor Lse.LLocal name))", + vector: "a local with a bad name", + }, + Plant { + label: "a package's first segment admits a hyphen", + root: 0, + site: "(idScan text false)", + replacement: "(idScan text true)", + vector: "a const with a hyphen in the first package segment", + }, + Plant { + label: "an empty segment ends an ID", + root: 0, + site: "(if (project final segStart) then false else true)", + replacement: "(if (project final segStart) then true else true)", + vector: "a const with an empty entry", + }, + Plant { + label: "a leading zero is a digit", + root: 0, + site: "(if (if (1 (ctor Result.error [Lse, Failure] (constError))", + replacement: "| List.nil => (match (qualifiedId spelled)\n | Result.ok parts => (ctor Result.ok [Lse, Failure] (ctor Lse.LConst (first parts) (second parts) (ctor Universe.UNil)))\n | Result.error failure => (ctor Result.error [Lse, Failure] failure))", + vector: "a const with empty universes", + }, + Plant { + label: "the renamed binders count from one", + root: 1, + site: "((cat (lit \"x\") (LexLeanStd.Numeral::decimal counter)), (counter + 1))", + replacement: "((cat (lit \"x\") (LexLeanStd.Numeral::decimal (counter + 1))), (counter + 1))", + vector: "a pi renamed, renamed", + }, + Plant { + label: "a local is renamed by its outermost binder", + root: 1, + site: "(ctor Option.none [Bytes]) scope : (Option Bytes))", + replacement: "(ctor Option.none [Bytes]) (LexLeanStd.Order::reverseList [(Bytes * Bytes)] scope) : (Option Bytes))", + vector: "shadowing, renamed", + }, + Plant { + label: "a let's value is outside its scope", + root: 1, + site: "(showExpr bound rename inner (second shownType))", + replacement: "(showExpr bound rename scope (second shownType))", + vector: "a let renamed, renamed", + }, + Plant { + label: "a binder's type is in its own scope", + root: 1, + site: "(showExpr ty rename scope (second shownName))", + replacement: "(showExpr ty rename (cons (name, (first shownName)) scope) (second shownName))", + vector: "a binder whose type names itself, renamed", + }, + Plant { + label: "arguments print without a space", + root: 1, + site: "((cat (cat (lit \" \") (first shownHead)) (first shownTail)), (second shownTail))", + replacement: "((cat (first shownHead) (first shownTail)), (second shownTail))", + vector: "an app printed", + }, + Plant { + label: "a const's universes print without their parentheses", + root: 1, + site: "(cat (cat (cat heading (lit \" (\")) (LexLeanStd.Text::sliceOr listed 1", + replacement: "(cat (cat (cat heading (lit \" \")) (LexLeanStd.Text::sliceOr listed 1", + vector: "a const with universes printed", + }, + Plant { + label: "implicit prints as explicit", + root: 1, + site: "(if (mode =? 1) then (lit \"implicit\") else", + replacement: "(if (mode =? 3) then (lit \"implicit\") else", + vector: "a lam of every mode renamed", + }, + Plant { + label: "the binders of a pi print without their space", + root: 1, + site: "(((cat (cat piece (lit \" \")) (first (first shownRest)))", + replacement: "(((cat piece (first (first shownRest)))", + vector: "a pi renamed", + }, +]; + +fn plants(vectors: &[Vector]) { + let mut missed = Vec::new(); + for plant in PLANTS { + let vector = vectors + .iter() + .find(|vector| vector.label == plant.vector) + .unwrap_or_else(|| panic!("{}: no vector `{}`", plant.label, plant.vector)); + assert_eq!( + vector.printing(), + plant.root == 1, + "{}: the vector is a run of the root planted", + plant.label + ); + let planted = support::P::bootstrap(); + let source = planted.read_raw("src/LexiconLse.lex.tex"); + assert_eq!( + source.matches(plant.site).count(), + 1, + "{}: the plant site occurs once", + plant.label + ); + planted.write( + "src/LexiconLse.lex.tex", + &source.replacen(plant.site, plant.replacement, 1), + ); + let checked = support::checked_project(&planted); + let low = super::lower(&checked, &planted, MODULE, ROOTS[plant.root].0); + match super::evaluate(&low, &[vector.run().0], 1).first() { + Some(first) => eprintln!( + "BS-25: plant detected: {}: {}", + plant.label, + first.chars().take(160).collect::() + ), + None => missed.push(plant.label), + } + } + assert!( + missed.is_empty(), + "planted defects their own vector does not detect: {missed:?}" + ); +} + +fn export(lowered: &BTreeMap<&str, super::Lowered>) { + let exported = harness::exported("bootstrap"); + let bundle = exported.bundle(); + let mut realized = 0usize; + for exported_root in &exported.output.result.roots { + let Some((member, low)) = exported_root + .root + .strip_prefix(&format!("{MODULE}.")) + .and_then(|member| lowered.get(member).map(|low| (member, low))) + else { + continue; + }; + let largest = exported_root + .outputs + .iter() + .flat_map(|output| &output.certification.certificates) + .map(|certificate| certificate.file.byte_length) + .max() + .expect("a certificate"); + eprintln!( + "BS-25: {}: largest certificate part {largest} bytes", + exported_root.root + ); + assert!( + largest <= CERTIFICATE_BUDGET, + "{}: a certificate part of {largest} bytes is beyond {CERTIFICATE_BUDGET}", + exported_root.root + ); + for output in &exported_root.outputs { + let package: Json2 = serde_json::from_slice( + &std::fs::read( + bundle + .join(&output.package.directory) + .join("package.json") + .as_std_path(), + ) + .expect("package.json"), + ) + .expect("package JSON"); + assert_eq!( + package["program"], + serde_json::to_value(&low.program).expect("a program is JSON"), + "{member}: the exported package realizes the program the differential ran" + ); + realized += 1; + } + } + assert_eq!(realized, ROOTS.len(), "one certified package per root"); + let packages = harness::tree(&bundle.join("packages")); + let committed_packages = harness::tree(&harness::fixture_dir("bootstrap").join("packages")); + assert_eq!( + packages.keys().collect::>(), + committed_packages.keys().collect::>(), + "the exported package files are the committed ones" + ); + for (relative, content) in &packages { + assert!( + committed_packages.get(relative) == Some(content), + "packages/{relative} differs from the committed package" + ); + } +} + +/// Every error `parse` can state, by a part of its message, so that a corpus +/// that never reached one fails here and not silently (R2). +const REFUSALS: &[&str] = &[ + "whitespace is one or more ASCII spaces or LF", + "max_scope_depth exceeded: configured", + "trailing tokens after the expression", + "unbalanced `)`", + "unexpected end of expression", + "expected `(`, found", + "expected an LSE form", + "`app` has at least a function and one argument", + "`pi` is `(pi (binders...) body)`", + "`lam` is `(lam (binders...) body)`", + "expected a binder list", + "has at least one binder", + "expected a binder, found", + "a binder is `(mode name type)`", + "expected a binder mode, found", + "is not a binder mode", + "expected a local ID, found", + "is not a valid local ID", + "`let` is `(let name type value body)`", + "is not an LSE form", + "a sort is `(sort prop)` or `(sort (type u))`", + "is not a universe form", + "is not a universe", + "has the wrong arity", + "expected a universe form", + "a const is `(const qualified-id [universes])`", + "is not a qualified `package::entry` ID", + "is not a valid package ID", + "is not a valid local entry ID", + "a local is `(local id)`", + "a nat is `(nat decimal)`", + "is not a decimal", + "has a redundant leading zero", + "does not fit 64 bits", +]; + +#[allow(clippy::too_many_lines)] +pub(super) fn case() { + let started = std::time::Instant::now(); + let project = support::P::bootstrap(); + let checked_units = project.check_ok(); + assert!( + checked_units.units.contains_key("LexiconLse"), + "LexiconLse is a linked module of the project" + ); + eligibility_of(&project, "LexiconLse", MODULE, &ROOTS); + assert_eq!( + constructors("Lse").as_slice(), + [ + "LSortProp", + "LSortType", + "LConst", + "LLocal", + "LApp", + "LPi", + "LLam", + "LLet", + "LNat", + "LNil", + "LArg", + "LBinder" + ], + "the constructors of Lse" + ); + assert_eq!( + constructors("Universe").as_slice(), + ["UNum", "UVar", "USucc", "UMax", "UIMax", "UNil", "UItem"], + "the constructors of Universe" + ); + + let vectors = named(); + let labels: BTreeSet<&str> = vectors.iter().map(|vector| vector.label.as_str()).collect(); + assert_eq!(labels.len(), vectors.len(), "each vector has its own label"); + let mut messages = Vec::new(); + let mut reading: Vec = Vec::new(); + let mut printing: Vec = Vec::new(); + for vector in &vectors { + let (run, refusal) = vector.run(); + messages.extend(refusal); + if vector.printing() { + printing.push(run); + } else { + reading.push(run); + } + } + for refusal in REFUSALS { + assert!( + messages.iter().any(|message| message.contains(refusal)), + "no vector reaches the error `{refusal}`" + ); + } + + // The committed expressions: read, printed both ways, and read back. + let texts = committed(); + assert!( + texts.len() >= 400, + "the committed lexicon entries carry {} LSE texts", + texts.len() + ); + let mut committed_reading: Vec = Vec::new(); + let mut committed_printing: Vec = Vec::new(); + let mut bases: Vec = Vec::new(); + for (path, source) in &texts { + let (run, refusal) = parse_run(source, DEPTH); + assert!( + refusal.is_none(), + "{path}: `{source}` is refused: {refusal:?}" + ); + committed_reading.push(run); + let expression = parsed(source); + committed_printing.push(print_run(&expression, false)); + committed_printing.push(print_run(&expression, true)); + let canonical = expression.print(false); + committed_reading.push(parse_run(&canonical, DEPTH).0); + bases.push(canonical); + } + let normative = support::normative_host(); + let (evaluator_seeded, rustc_seeded) = if normative { + ( + seeded_runs(EVALUATOR_SEEDED, &bases), + seeded_runs(RUSTC_SEEDED, &bases), + ) + } else { + ((Vec::new(), Vec::new()), (Vec::new(), Vec::new())) + }; + eprintln!( + "BS-25: {} vectors ({} read, {} printed), {} committed expressions ({} read and {} printed), {} and {} seeded expressions under the evaluator and rustc ({:?})", + vectors.len(), + reading.len(), + printing.len(), + texts.len(), + committed_reading.len(), + committed_printing.len(), + evaluator_seeded.0.len(), + rustc_seeded.0.len(), + started.elapsed() + ); + + let checked = support::checked_project(&project); + let lowered: BTreeMap<&str, super::Lowered> = ROOTS + .iter() + .map(|(root, _)| (*root, super::lower(&checked, &project, MODULE, root))) + .collect(); + if normative { + let mut read_corpus: Vec = reading.clone(); + read_corpus.extend(committed_reading.iter().cloned()); + read_corpus.extend(evaluator_seeded.1); + let mut print_corpus: Vec = printing.clone(); + print_corpus.extend(committed_printing.iter().cloned()); + print_corpus.extend(evaluator_seeded.0); + for (root, corpus) in [("parseLse", &read_corpus), ("printLse", &print_corpus)] { + let disagreements = super::evaluate(&lowered[root], corpus, 5); + assert!( + disagreements.is_empty(), + "{root} and the Rust oracle disagree: {disagreements:?}" + ); + eprintln!( + "BS-25: {root}: the evaluator agrees on {} runs ({:?})", + corpus.len(), + started.elapsed() + ); + } + plants(&vectors); + } else { + eprintln!("BS-25: platform-bound host: the evaluator, the seeded runs, and the planted defects run on the normative host"); + } + + let mut compiled_reading = reading; + compiled_reading.append(&mut committed_reading); + compiled_reading.extend(rustc_seeded.1); + let mut compiled_printing = printing; + compiled_printing.append(&mut committed_printing); + compiled_printing.extend(rustc_seeded.0); + let total = rustc_differential( + "BS-25", + &[ + ( + "parseLse", + &lowered["parseLse"], + compiled_reading.as_slice(), + ), + ( + "printLse", + &lowered["printLse"], + compiled_printing.as_slice(), + ), + ], + "oracle", + ); + eprintln!("BS-25: {total} rustc runs agree ({:?})", started.elapsed()); + + if !support::lean_backed("BS-25") { + return; + } + export(&lowered); + eprintln!("BS-25: passed in {:?}", started.elapsed()); +} diff --git a/crates/conformance/src/cases/repository.rs b/crates/conformance/src/cases/repository.rs index 7df18c8c4a..aaab4a220b 100644 --- a/crates/conformance/src/cases/repository.rs +++ b/crates/conformance/src/cases/repository.rs @@ -873,7 +873,7 @@ pub(crate) fn run(id: &str) { let model = repo_model::Model::load(&root.join("model").into_std_path_buf()) .expect("the model loads"); let table = spec_table(); - assert_eq!(table.len(), 478, "§31 has 478 rows"); + assert_eq!(table.len(), 479, "§31 has 479 rows"); assert_eq!(model.ids.id.len(), table.len(), "register row count"); for ((spec_id, spec_suite, spec_statement), row) in table.iter().zip(model.ids.id.iter()) diff --git a/crates/conformance/tests/conformance.rs b/crates/conformance/tests/conformance.rs index 992ea8a7b3..24bd1a7b6c 100644 --- a/crates/conformance/tests/conformance.rs +++ b/crates/conformance/tests/conformance.rs @@ -2320,6 +2320,11 @@ fn conformance_bs_20() { repo_conformance::cases::run("BS-20"); } +#[test] +fn conformance_bs_25() { + repo_conformance::cases::run("BS-25"); +} + #[test] fn conformance_pk_01() { repo_conformance::cases::run("PK-01"); diff --git a/features/suites/bootstrap.feature b/features/suites/bootstrap.feature index 31570df912..7ac0ecb383 100644 --- a/features/suites/bootstrap.feature +++ b/features/suites/bootstrap.feature @@ -119,3 +119,10 @@ Feature: bootstrap When the project is checked and built, the lowered program of the root is run by the evaluator on those modules and the mutants, and its rendered rust-std package is run by the pinned rustc on the modules, each with the capacity that fits and capacities that do not Then the root is eligible for rust-std with exactly its declared effects and no run disagrees with the oracle text, to the byte, or with its refusal And every planted defect is detected, and the exported package is byte-identical to its committed package and realizes the lowered program + + @BS-25 @build + Scenario: The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim. + Given the committed `bootstrap` project, hand-built texts of every form and every error of the reader and expressions of every form, every `signature` and `value` of the committed lexicon entries, and seeded expressions and seeded edits of their texts + When the project is checked and built, and the lowered program and the rendered package of each root are run on those texts and expressions + Then each root is eligible for rust-std with exactly its declared effects, no certificate part is beyond 3,145,728 bytes, and `parseLse` and `printLse` equal `parse` and `print` on every run, every error of which a vector reaches + And every planted defect is detected by its own vector, and the exported packages are byte-identical to the committed packages and realize the programs the differential ran diff --git a/model/ids.toml b/model/ids.toml index 69eb399d8a..c9e92869c4 100644 --- a/model/ids.toml +++ b/model/ids.toml @@ -2786,6 +2786,12 @@ level = "build" suite = "bootstrap" statement = "The canonical LaTeX of a tooling module of SPEC.md §17.20 (`backend::tooling::render_latex`: the fixed document frame and, for each declaration in order, its title, its kind, and the lines the Lean printer of the same file spells for its types, signatures, and terms, every line escaped by `tex_escape`) is language-1.2 source in the `bootstrap/` project: `renderToolingLatex` of `ToolingLatex`, over the tooling module as first-order data with each list carried as a chain of its own `Nil` and `Cons` constructors and with the typings the checker infers as an argument, writing through the standard library's `Buffer` with an explicit capacity, is a production root eligible for `rust-std` with exactly its declared effects; its rendered `rust-std` package under the pinned rustc and its lowered program under the calculus evaluator agree with the Rust oracle on the text, to the byte, and on the refusal of a write past the capacity, on the committed tooling modules (whose committed `.tex` is equal to the text), on hand-built modules that reach every declaration, type, signature, and term form and every byte class of the escaper, and on seeded mutants, every planted defect is detected, and the certified package the production export publishes for the project is the committed one; the checker, the decoder of the module, and the source map of the emitted pieces stay in the host." +[[id]] +id = "BS-25" +level = "build" +suite = "bootstrap" +statement = "The reading and the canonical printing of LexLean Semantic Expressions of SPEC.md §13.8 (`lexicon::lse::parse`: a text, tokenized into parentheses and atoms separated by ASCII spaces and LF, read as one S-expression whose nesting the configured `max_scope_depth` bounds, and converted into an expression, a sort, a constant with its universes, a local, an application, a `pi` or `lam` with its binders, a `let`, or a natural number, or the kind and message of the first error, in the order the oracle finds them; and `Lse::print`: an expression as its canonical text, one ASCII space between atoms and no redundant grouping, with its binders renamed `x0`, `x1`, ... in printing order when asked) is language-1.2 source in the `bootstrap/` project, as two production roots of `LexiconLse`: `parseLse`, eligible for `rust-std` with exactly its declared effects, none of them `recursion`, and `printLse`, which admits `recursion`, each with no part of its certificate beyond 3,145,728 bytes; their rendered `rust-std` packages under the pinned rustc, and on the normative host their lowered programs under the calculus evaluator, agree with the oracle to the byte on the hand-built vectors, which reach every error of the reader and the limit on nesting, on every `signature` and `value` of the committed lexicon entries read, printed both ways, and read back, and on seeded expressions and seeded edits of their texts; each planted defect in the source is detected by the one vector it names, and the certified packages the production export publishes for the project are the committed ones; the scope check, the conservative checker, and the rendering expressions (`check_scopes`, `check_signature`, `check_value`, `lre`) are outside the claim." + [[id]] id = "PK-01" level = "build" diff --git a/tests/production-export/bootstrap/roots.json b/tests/production-export/bootstrap/roots.json index 4f911a0892..35097ce3a9 100644 --- a/tests/production-export/bootstrap/roots.json +++ b/tests/production-export/bootstrap/roots.json @@ -152,6 +152,14 @@ ], "version": "1.0.0" }, + { + "name": "parse_lse", + "root": "LexLeanBoot.LexiconLse.parseLse", + "targets": [ + "rust-std" + ], + "version": "1.0.0" + }, { "name": "print_crate", "root": "LexLeanBoot.RustPrint.printCrate", @@ -168,6 +176,14 @@ ], "version": "1.0.0" }, + { + "name": "print_lse", + "root": "LexLeanBoot.LexiconLse.printLse", + "targets": [ + "rust-std" + ], + "version": "1.0.0" + }, { "name": "reject_atoms", "root": "LexLeanBoot.PrimitiveScan.rejectAtoms", diff --git a/xtask/src/spec_links.rs b/xtask/src/spec_links.rs index d3b755a3d4..962afcafd8 100644 --- a/xtask/src/spec_links.rs +++ b/xtask/src/spec_links.rs @@ -61,8 +61,8 @@ fn parse_table(spec: &str) -> Result, Fail> { pub fn validate(root: &Path) -> Result<(), Fail> { let spec = std::fs::read_to_string(root.join("SPEC.md"))?; let rows = parse_table(&spec)?; - if rows.len() != 478 { - return Err(format!("RP-07: the §31 table has {} rows, not 478", rows.len()).into()); + if rows.len() != 479 { + return Err(format!("RP-07: the §31 table has {} rows, not 479", rows.len()).into()); } // The prose total under the table is part of the contract a reader // relies on; a row added without it would leave §31 contradicting itself.