Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,23 @@ workspace carried at those versions without a tag.

## Unreleased

- A semantic module nests as deeply as the project's `max_scope_depth` allows
(SPEC.md §17.17, §17.21.3, `SM-12`; no new diagnostic code; no language
identity moves). The front end read a module's JSON with a fixed 128-level
limit and its notation with a fixed 256-level one, so a list of more than
about 122 elements was refused, and past about 115 the semantic snapshot
panicked writing its own bytes. Both readers now take the project's bound
on the compile thread's stack, and every later reading of what the linker
admitted (rendering, declaration IDs, the snapshot and its canonical bytes)
reads to the same depth on a stack sized for it: a list of 1000 elements
checks, builds, verifies, and has a snapshot. Past the bound the notation is
`LLP2001` and the data `LLT4001`, each naming `max_scope_depth`. The
authoring API (`lexlean::sdk`) reads, converts, and writes a semantic module
to 1024 levels, the default `max_scope_depth`, on a stack of its own, where
it read the notation to 256 and the JSON to 128 on its caller's. A module
within the old limits reads, renders, and hashes to the same bytes as
before. This reverses the decision of #375 to keep the 128-level limit.
The register is unchanged.
- `lexlean check` builds and runs on `wasm32-wasip1`, #494 (SPEC.md §8.3,
`CL-25`). WASI has no threads, no file lock, and no processes: the check
pipeline runs on the instance's stack, a command that changes the project
Expand Down
2 changes: 1 addition & 1 deletion CONFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -594,7 +594,7 @@ the error registry does not sanction (R5, `ERRORS.md`).
| `SM-09` | `build` | Linked IR has stable schema-tagged canonical JSON with alpha-safe binder serialization. |
| `SM-10` | `build` | Source ID and semantic ID use exactly the specified framed hash inputs. |
| `SM-11` | `build` | Linked project result sets contain every selected module and imported module exactly once in stable order. |
| `SM-12` | `build` | No semantic IR node can contain opaque prose, raw backend text, or an unknown extension. Semantic modules retain the 128-level JSON nesting limit and an over-deep list is refused with LLT4001 naming smaller definitions joined by prim append as the remedy. |
| `SM-12` | `build` | No semantic IR node can contain opaque prose, raw backend text, or an unknown extension. A semantic module nests as deeply as the project's max_scope_depth allows, in its notation and in its data: a list of 1000 elements checks and builds, and its snapshot is written, and the authoring API reads and writes it, on a caller's default stack without a panic, while a list nested past the bound is refused, with LLP2001 in the notation and LLT4001 in the data, naming max_scope_depth. |
| `SM-13` | `build` | Inherited section parameters are represented explicitly and emitted only on declarations that use them. |
| `SM-14` | `build` | A numeral without a unique expected type is rejected rather than defaulted. |
| `SM-15` | `build` | A native core module is closed typed DAG data shared by both backends, carries explicit declaration policies, and accepts no backend source text. |
Expand Down
32 changes: 21 additions & 11 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -5559,14 +5559,19 @@ because they bound what verification can be given. Checking a project holds
memory that grows linearly with the number of its functions when they are
independent (1500, 3000, 7000, and 14000 trivial functions take 56, 101, 223,
and 435 MB), before any limit of this section or of `max_ir_nodes` is reached;
a chain of calls is the case above. And a semantic module is JSON read with the
reader's recursion limit of 128 levels: a term nested deeper than that (a chain
of 130 nested `let`s) is `LLT4001` for excessive JSON nesting, whatever
`max_scope_depth` says; a chain of 100 is read.
The `LLT4001` nesting diagnostic names the unchanged 128-level bound and
recommends splitting long lists into smaller definitions joined by `prim append`.
It locates the semantic-module body, not the nested term; it does not report
coordinates in generated JSON as positions in the authored source.
a chain of calls is the case above. And a semantic module nests as deeply as
the project's `max_scope_depth` allows, as the compile thread's stack hosts it
(§25.5), in its notation and in the JSON the notation reads to; a list of n
elements nests about n levels, and a chain of n nested `let`s about n. Notation
nested deeper is `LLP2001`, and JSON nested deeper is `LLT4001`, each naming
`max_scope_depth`; the JSON diagnostic also recommends splitting long lists
into smaller definitions joined by `prim append`. Both locate the
semantic-module body, not the nested term; neither reports coordinates in
generated JSON as positions in the authored source. Every later reading of
what the linker admitted (the rendering of a build, declaration IDs, the
semantic snapshot and its canonical bytes) reads it to the same depth, on a
stack sized for it, never on its caller's: a list of 1000 elements checks,
builds, verifies, and has a snapshot (`SM-12`).

**Proof.** The proof follows the source term construct by construct through
the library's compatibility lemmas. A recursive definition's relation is
Expand Down Expand Up @@ -7495,8 +7500,13 @@ with that member left out. Members are written in the order `name`,
`type_parameters`, `parameters`, `type`, `result`, `statement`, `fields`,
`constructors`, `arguments`, `body`, `proof`, `axioms`, then the others in
key order, and are read in any order; a member written twice, an integer
beyond 64 bits, a value nested deeper than 256 levels, and a word that is not
a name where a string is expected are errors. Because the generic notation
beyond 64 bits, a value nested deeper than its reader reads, and a word that
is not a name where a string is expected are errors. A project's semantic
module is read to its `max_scope_depth` (§17.17, the limits of the front end);
the authoring library (§24.8) reads a semantic module to 1024 levels, the
`max_scope_depth` a project is created with, on a stack sized for them; any
other text read with no project, to 256 levels.
Because the generic notation
covers every value, no value of any schema is unspellable, and a variant added
to a schema has a notation the day it is added.

Expand Down Expand Up @@ -11774,7 +11784,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte
| `SM-09` | `semantic-ir` | Linked IR has stable schema-tagged canonical JSON with alpha-safe binder serialization. | §17.9 |
| `SM-10` | `semantic-ir` | Source ID and semantic ID use exactly the specified framed hash inputs. | §21.3, §21.4 |
| `SM-11` | `semantic-ir` | Linked project result sets contain every selected module and imported module exactly once in stable order. | §17.5, §23.3 |
| `SM-12` | `semantic-ir` | No semantic IR node can contain opaque prose, raw backend text, or an unknown extension. Semantic modules retain the 128-level JSON nesting limit and an over-deep list is refused with LLT4001 naming smaller definitions joined by prim append as the remedy. | §6 I4, §17 |
| `SM-12` | `semantic-ir` | No semantic IR node can contain opaque prose, raw backend text, or an unknown extension. A semantic module nests as deeply as the project's max_scope_depth allows, in its notation and in its data: a list of 1000 elements checks and builds, and its snapshot is written, and the authoring API reads and writes it, on a caller's default stack without a panic, while a list nested past the bound is refused, with LLP2001 in the notation and LLT4001 in the data, naming max_scope_depth. | §6 I4, §17 |
| `SM-13` | `semantic-ir` | Inherited section parameters are represented explicitly and emitted only on declarations that use them. | §17.5, §18.3 |
| `SM-14` | `semantic-ir` | A numeral without a unique expected type is rejected rather than defaulted. | §15.5 |
| `SM-15` | `semantic-ir` | A native core module is closed typed DAG data shared by both backends, carries explicit declaration policies, and accepts no backend source text. | §17.10 |
Expand Down
58 changes: 58 additions & 0 deletions VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -13504,6 +13504,64 @@ The nesting diagnostic retains the existing `recursion limit exceeded` category,
which the `SP-02` 130-level let-chain refusal also asserts. Only its generated
JSON coordinates are replaced by the bound and workaround advice.

## A semantic module nests to `max_scope_depth` (`SM-12`)

This supersedes the record above: the 128-level limit is gone. A consumer
application whose acceptance vectors are one closed list could not carry more
than about 115 of them, and its design does not admit splitting the list.

Before the change, on a Windows host with the debug compiler, a language-1.2
project whose one definition is a list of n `Nat` cells: n = 100 and 120
checked and their `build` overflowed the main thread's stack (the renderer ran
on the caller's thread); n = 130 was `LLT4001` from the 128-level JSON reader;
and past the snapshot's own bounded re-read (`SemanticSnapshot::to_json`) a
snapshot panicked writing its bytes, which is where the consumer stopped at
about 115. The notation had a fixed 256-level bound of its own behind that.

The change bounds the notation and the JSON of a project's module by its
`max_scope_depth` as the compile thread's stack hosts it, read on that thread;
every later reading of what the linker admitted reads to the same depth on a
stack sized for it: `render_build` on a thread of the compile stack's size,
and the snapshot (`from_checked`, `to_json`, `canonical_bytes`) on one of the
largest compile stack's size, so a caller on any stack gets its bytes. The
re-reads of the linker's own data (document JSON, declaration IDs, definition
digests, the snapshot's values) read without the 128-level limit. Readers with
no project keep their bounds: `SemanticModule::parse` 127 levels, the
authoring library's notation 256, tooling modules 128 (`LLT4013`).

After it, with the same compiler: n = 130, 256, and 1000 check and build, and
n = 1100 is `LLP2001` naming `max_scope_depth` 1024. `lexlean verify` with the
pinned Lean 4.32.1 verified n = 256 (45 s) and n = 1000 (29 s), so the
ceiling is the project's bound, not Lean's. `cargo xtask check-golden` passed
for every project (no published artifact moved), and the diff touches no
identity input (`language/`, `schemas/`, the embedded goldens).

`conformance_sm_12` passes (4 s). It checks and builds the 1000-cell fixture,
writes its snapshot on a 2 MiB thread and requires the snapshot ID to be the
digest of the bytes and the bytes to nest past 1000 levels; refuses the
1100-cell fixtures with `LLP2001`; and under `max_scope_depth = 150` checks 145
cells and refuses 146 and 147 with `LLT4001` and 148 with `LLP2001`, so both
bounds are the project's. Three planted defects, each restored:

| Planted defect | `conformance_sm_12` |
| --- | --- |
| the linker reads the data with `parse` (127 levels) | fails: the 1000-cell list is refused |
| `to_json` re-reads with the bounded `Json::parse` | fails: `recursion limit exceeded` panic in the snapshot |
| `canonical_bytes` runs on its caller's stack | fails: `STATUS_STACK_OVERFLOW` on the 2 MiB thread |
| the authoring API reads the notation to 256 levels | fails: the 1000-cell list is not read |
| the authoring API runs on its caller's stack | fails: `STATUS_STACK_OVERFLOW` on the 2 MiB thread |

The authoring API had the same limits on its caller's stack (notation 256,
JSON 128), so `conformance_au_08`, which reads every committed module through
it, failed on the 1000-cell fixture in the gate. It now reads a semantic
module to 1024 levels on a stack of its own; `conformance_sm_12` reads,
converts both ways, and writes the fixture on a 2 MiB thread and refuses the
1100-cell one, and `conformance_au_08` passes.

`SP-02` asserted the 130-level let-chain refusal; it now asserts that the
chain checks. `conformance_au_01`, `conformance_au_03` (a 50,000-level
notation is still `LLP2001`), and `conformance_au_06` pass.

## Release evidence consumes the whole jobs listing (RP-18, #408)

The release gate piped the jobs listing into quiet `grep` with `pipefail`
Expand Down
16 changes: 5 additions & 11 deletions crates/conformance/src/cases/preservation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1775,17 +1775,11 @@ pub fn run(id: &str) {
}
}
}
// A semantic module is JSON, and the JSON of the front end nests
// at most 128 levels: a chain of 100 `let`s is read (a family
// above certifies it) and a chain of 130 is refused, as the
// module that is not valid JSON of this reader, before any limit
// of this section.
let deep = crate::stress::let_chain_default(130);
let error = deep.check_fails_with("LLT4001");
assert!(
error.to_string().contains("recursion limit exceeded"),
"{error}"
);
// A semantic module nests as deeply as the project's
// max_scope_depth allows (1024 here; `SM-12` holds the bound): a
// chain of 130 `let`s, past the 128 levels of the JSON reader
// the front end once had, is read.
crate::stress::let_chain_default(130).check_ok();
if let Some(stress) = stress {
let quadratic = stress
.join()
Expand Down
Loading
Loading