From bb464d3710fc1dc3b25c8c08ad96bb5f87fb56e1 Mon Sep 17 00:00:00 2001 From: Alex Flom Date: Mon, 2 Nov 2026 23:55:44 -0700 Subject: [PATCH] Fix authoring notation nesting bounds (AU-03) Enforce the existing depth bound in recursive readable terms and types. Cover 50,000-level closed and unclosed input through the API and CLI, exact readable boundaries, and independent guard-removal plants. Closes #374. --- CONFORMANCE.md | 2 +- SPEC.md | 6 ++- VERIFICATION.md | 26 +++++++++++++ crates/conformance/src/cases/authoring.rs | 47 +++++++++++++++++++++++ crates/lexlean/src/surface/semantic.rs | 34 ++++++++++++++++ features/suites/authoring.feature | 2 +- model/ids.toml | 2 +- 7 files changed, 114 insertions(+), 5 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index ca34dedda2..2d75424772 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -44,7 +44,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | --- | --- | --- | | `AU-01` | `build` | Every value of the closed semantic-module schema has one notation and the notation reads back to the value: every alternative of the declaration, term, type, and proof unions, with and without its optional members, is written and read back equal, a node that no readable form carries exactly is written generically, the notation is its own canonical form, and every committed semantic module is written and read back equal to its data. | | `AU-02` | `build` | A language-1.2 semantic module written in the authoring notation builds, with source maps that relate each declaration to its text in the notation; fmt leaves the notation unchanged and restores its canonical layout from another layout, and the notation under language 1.1 is refused with LLP2003; its data form is not a language-1.2 source (AU-13). | -| `AU-03` | `build` | Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it. | +| `AU-03` | `build` | Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, a readable term or type nested beyond the notation bound, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it. | | `AU-04` | `build` | No language-1.2 source of the product is written as data: every committed language-1.2 source with a semantic, tooling, or core module, in the compiler, the language libraries, the examples, and the positive fixtures, is written in the authoring notation and is canonical, the data form remains only under language 1.0 and 1.1 and in refused fixtures whose payload the notation cannot spell, and a refused fixture whose payload the notation can spell is a defect. | | `AU-05` | `build` | A language-1.2 tooling module written in the authoring notation renders, with source maps that relate each declaration to its text in the notation; the notation and the canonical JSON of every committed tooling module are in bijection and the notation is its own canonical form; fmt leaves the notation unchanged; and malformed notation, and every single-token deletion and duplication of a committed tooling module, fails with a position or decodes, and nothing panics. | | `AU-06` | `build` | The compiler-semantics ID does not depend on the form a module of the embedded tree is written in: a semantic, tooling, or core module in the authoring notation contributes its data form, so the committed tree and the same tree with every such module written as data have one ID, a change to a module's meaning changes the ID in either form, the identity a preservation record states for the native library is over the same data forms, so it is the committed one, and the embedded tree holds at least one module written in the notation. | diff --git a/SPEC.md b/SPEC.md index f327f0e52c..29a7053d0b 100644 --- a/SPEC.md +++ b/SPEC.md @@ -7416,7 +7416,9 @@ survive `fmt`. exact inverses over every committed module (`AU-08`, `AU-13`). - **Failing closed (`AU-03`).** Every single-token deletion, duplication, and replacement of a committed module either fails with a registered code or links - to a module that decodes. + to a module that decodes. Readable terms and types nested beyond the + notation bound are refused with `LLP2001` at the offending token, including + when their closing delimiters are missing. - **Tooling modules (`AU-05`).** The same laws for a tooling module: read and written back equal for every committed one, rendering with a source map that names the notation, `fmt` the identity on the notation, and every @@ -11351,7 +11353,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `TL-11` | `tooling` | Tooling modules are deterministic and content-addressed: each committed fixture's published build tree equals its golden byte for byte and across two clean builds in different directories, changing any declaration, registry row, source pin, or term changes the semantic ID, and the schema lies outside `schemas/` and so outside the compiler-semantics ID, which the committed locks of the three languages still record. | §17.20, §21.2, §28.3, §28.4 | | `AU-01` | `authoring` | Every value of the closed semantic-module schema has one notation and the notation reads back to the value: every alternative of the declaration, term, type, and proof unions, with and without its optional members, is written and read back equal, a node that no readable form carries exactly is written generically, the notation is its own canonical form, and every committed semantic module is written and read back equal to its data. | §17.21 | | `AU-02` | `authoring` | A language-1.2 semantic module written in the authoring notation builds, with source maps that relate each declaration to its text in the notation; fmt leaves the notation unchanged and restores its canonical layout from another layout, and the notation under language 1.1 is refused with LLP2003; its data form is not a language-1.2 source (AU-13). | §17.21 | -| `AU-03` | `authoring` | Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it. | §17.21 | +| `AU-03` | `authoring` | Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, a readable term or type nested beyond the notation bound, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it. | §17.21 | | `AU-04` | `authoring` | No language-1.2 source of the product is written as data: every committed language-1.2 source with a semantic, tooling, or core module, in the compiler, the language libraries, the examples, and the positive fixtures, is written in the authoring notation and is canonical, the data form remains only under language 1.0 and 1.1 and in refused fixtures whose payload the notation cannot spell, and a refused fixture whose payload the notation can spell is a defect. | §17.21 | | `AU-05` | `authoring` | A language-1.2 tooling module written in the authoring notation renders, with source maps that relate each declaration to its text in the notation; the notation and the canonical JSON of every committed tooling module are in bijection and the notation is its own canonical form; fmt leaves the notation unchanged; and malformed notation, and every single-token deletion and duplication of a committed tooling module, fails with a position or decodes, and nothing panics. | §17.20, §17.21 | | `AU-06` | `authoring` | The compiler-semantics ID does not depend on the form a module of the embedded tree is written in: a semantic, tooling, or core module in the authoring notation contributes its data form, so the committed tree and the same tree with every such module written as data have one ID, a change to a module's meaning changes the ID in either form, the identity a preservation record states for the native library is over the same data forms, so it is the committed one, and the embedded tree holds at least one module written in the notation. | §17.21, §21.2 | diff --git a/VERIFICATION.md b/VERIFICATION.md index a6cf6d72f4..e6b58acebe 100644 --- a/VERIFICATION.md +++ b/VERIFICATION.md @@ -12274,3 +12274,29 @@ draws up to a fixed count. `BS-05`, `BS-06`, `BS-08` and `BS-15` read only manif and fixtures. The three failing cases assert the count before their plants, so that run did not reach the plants on the new draws. A package that adds records should set these counts with its records in the tree, on the same commit. + + +## Readable semantic notation obeys its nesting bound (AU-03, #374) + +The recursive readable term and type readers enforce the same depth-256 bound +as the generic notation reader. `conformance_au_03` constructs 50,000-level +`cons` terms and `List` types, both with and without their closing parentheses. +The public check API and the CLI in human and JSON modes must refuse each with +`LLP2001` at a position in `src/Main.lex.tex`, before decoding semantic JSON. +These are temporary negative projects inside the existing AU-03 case. + +`cargo test -p lexlean --lib readable_nesting_has_an_explicit_bound` also checks +the exact readable boundary: 255 unary terms or `List` types, and 254 `cons` +terms ending in `(nil Nat)`, read successfully; one more nesting is refused. +The leaf type of `nil` consumes the additional level. + +The test failed on the old reader because an over-bound term was accepted. +Removing only the readable term guard then failed on the 256-level unary term; +removing only the type guard failed on the over-bound `cons` term's leaf type. +Both plants returned a failing test (exit 101), without a stack abort, and were +restored. The fixed boundary test and `conformance_au_03` passed on Linux with +Rust 1.97.1. No Lean process is involved in these checks. + +The fix changes no embedded normative identity input: `build.rs` embeds +`language/`, `schemas/`, and the axiom-parser and canonical-JSON goldens, none +of which changes here. No language identity or verification oracle is rewritten. diff --git a/crates/conformance/src/cases/authoring.rs b/crates/conformance/src/cases/authoring.rs index 3ae31b3851..8989617e1b 100644 --- a/crates/conformance/src/cases/authoring.rs +++ b/crates/conformance/src/cases/authoring.rs @@ -471,6 +471,53 @@ fn fails_closed() { assert!(span.line_start >= 1, "{what}: no line"); } } + // Recursive readable forms bypass the generic reader; their own bound + // must fire before either serde's recursion limit or a host stack abort. + for (prefix, leaf, position) in [ + ("(cons 1 ", "(nil Nat)", "body"), + ("(List ", "Nat", "result"), + ] { + let nested = format!("{}{}{}", prefix.repeat(50_000), leaf, ")".repeat(50_000)); + let declaration = if position == "body" { + format!( + "definition{{name: deep, parameters: [], result: (List Nat), body: {nested}}}\n" + ) + } else { + format!("definition{{name: deep, parameters: [], result: {nested}, body: 0}}\n") + }; + for text in [ + declaration.clone(), + declaration.replacen(&nested, &nested.replace(')', ""), 1), + ] { + put_body(&project, &text); + let error = project.check_err(); + let diagnostic = &error.diagnostics[0]; + assert_eq!(diagnostic.code.as_str(), "LLP2001"); + assert!( + diagnostic.message.contains("nested more deeply"), + "{diagnostic:?}" + ); + let span = diagnostic.primary.as_ref().expect("notation position"); + assert_eq!(span.path.as_str(), MAIN); + assert!(span.column_start > 256); + let (exit, _, stderr) = project.cli(&["check"]); + assert_eq!(exit, 1, "{stderr}"); + assert!( + stderr.contains("LLP2001") && stderr.contains(MAIN), + "{stderr}" + ); + let (exit, stdout, stderr) = project.cli(&["check", "--diagnostic-format", "json"]); + assert_eq!(exit, 1, "{stderr}"); + let result: serde_json::Value = + serde_json::from_str(&stdout).expect("JSON command result"); + assert_eq!(result["diagnostics"][0]["code"], "LLP2001"); + assert_eq!(result["diagnostics"][0]["primary"]["path"], MAIN); + assert_eq!( + result["diagnostics"][0]["primary"]["column_start"], + span.column_start + ); + } + } put_body(&project, &original); project.check_ok(); diff --git a/crates/lexlean/src/surface/semantic.rs b/crates/lexlean/src/surface/semantic.rs index c6d61f9160..aeffe1343e 100644 --- a/crates/lexlean/src/surface/semantic.rs +++ b/crates/lexlean/src/surface/semantic.rs @@ -574,6 +574,9 @@ mod types { /// Read a type. pub(super) fn read(reader: &mut Reader<'_>, depth: usize) -> Result { + if depth > super::MAX_DEPTH { + return Err(reader.error("nested more deeply than the notation reads")); + } let offset = reader.offset(); match reader.peek() { Some(Token::Word(word)) => { @@ -1150,6 +1153,9 @@ mod terms { /// Read a term. pub(super) fn read(reader: &mut Reader<'_>, depth: usize) -> Result { + if depth > super::MAX_DEPTH { + return Err(reader.error("nested more deeply than the notation reads")); + } let offset = reader.offset(); match reader.peek() { Some(Token::Str(text)) => { @@ -1855,6 +1861,34 @@ mod tests { text } + #[test] + fn readable_nesting_has_an_explicit_bound() { + std::thread::Builder::new() + .stack_size(crate::READER_STACK) + .spawn(|| { + for (position, prefix, leaf, bound) in [ + (Position::Term, "(not ", "true", 255), + (Position::Term, "(cons 1 ", "(nil Nat)", 254), + (Position::Type, "(List ", "Nat", 255), + ] { + let at_bound = format!("{}{}{}", prefix.repeat(bound), leaf, ")".repeat(bound)); + assert!(parse_at(&at_bound, position).is_ok()); + let beyond = format!( + "{}{}{}", + prefix.repeat(bound + 1), + leaf, + ")".repeat(bound + 1) + ); + let error = parse_at(&beyond, position).expect_err("nesting must be bounded"); + assert_eq!(error.message, "nested more deeply than the notation reads"); + assert!(error.offset > 0 && error.offset < beyond.len()); + } + }) + .expect("reader thread") + .join() + .expect("reader must not panic"); + } + /// The readable forms print as the SPEC says they do, and parse back. #[test] fn readable_forms() { diff --git a/features/suites/authoring.feature b/features/suites/authoring.feature index 44ffc97b43..efa12a540a 100644 --- a/features/suites/authoring.feature +++ b/features/suites/authoring.feature @@ -13,7 +13,7 @@ Feature: authoring Then each builds, the source maps relate each declaration to its text in the notation, fmt leaves the notation unchanged and restores the layout, and language 1.1 refuses it @AU-03 @build - Scenario: Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it. + Scenario: Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, a readable term or type nested beyond the notation bound, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it. Given a committed language-1.2 module in the notation When it is malformed in each way named, and each of its tokens is deleted, duplicated, and replaced in turn, and numerals with a redundant leading zero are written outside and inside a string Then each is refused with a registered code at a position or links to a module that decodes, nothing panics, and only the numeral outside a string is LLL1003 diff --git a/model/ids.toml b/model/ids.toml index 48f8596be6..a37c081c5b 100644 --- a/model/ids.toml +++ b/model/ids.toml @@ -2507,7 +2507,7 @@ statement = "A language-1.2 semantic module written in the authoring notation bu id = "AU-03" level = "build" suite = "authoring" -statement = "Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it." +statement = "Malformed notation fails closed and never panics: an unclosed delimiter, a keyword used as a name, a literal that is not canonical, an unknown kind, a readable term or type nested beyond the notation bound, and every single-token deletion, duplication, and replacement of a committed module fail with a registered code at a position in the notation or link to a module that decodes, and a numeral with a redundant leading zero is LLL1003 outside a string and is not one inside it." [[id]] id = "AU-04"