Skip to content

BS-24: the generic notation as language-1.2 source (#269) - #418

Closed
humuhumu33 wants to merge 54 commits into
claude/issue-269-surface-docfrom
claude/issue-269-surface-generic
Closed

humuhumu33 wants to merge 54 commits into
claude/issue-269-surface-docfrom
claude/issue-269-surface-generic

Conversation

@humuhumu33

@humuhumu33 humuhumu33 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Part of #269: the generic notation (BS-24, surface/generic.rs) as language-1.2 source, with roots parseGeneric and printGeneric, over the standard library's Json.

Draft, stacked on claude/issue-269-surface-doc. Written as source; it has not been compiled or run, and its regeneration has not run, so CI is expected red until then. It lands after the PR below it.

32 planted defects, each naming its own vector. printGeneric admits recursion, so it owes a measured row in tests/stack-bounds.json. It includes the JsonRead axiom correction and the later digit-count refusal. It still needs the reviewed SL-13 namespace follow-up from #415 before the coordinated writers; Host retains SL-12. Not run: cargo build, clippy, lexlean verify, the case.

🤖 Generated with Claude Code

Closes #425 (BS-24 delivery slice). #269 remains open.

humuhumu33 and others added 30 commits October 29, 2026 11:37
…the count mirrors (460 -> 461)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…he locks of its consumers

One recursive Json value (the spine shape of bootstrap Expr) and a mutual
structural writer: writeJson and writeJsonFile for the whole value,
escapeString for a string. An object is written with its members sorted by
Order.sortEntries and the last of a repeated key kept, as the oracle BTreeMap
keeps it; integers are a sign and a magnitude through Numeral.decimal.
Held to canonical_json::Json::to_canonical_string and to_file_bytes. The plant
keeps the first of a repeated key: only trees that repeat a key catch it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ance test

R3 first commit for #270: the BS-20 row (model/ids.toml, SPEC section 31),
its scenario, the conformance_bs_20 test (fails: no bootstrap case is wired),
the part entries in bdd-parts.json and vv-rows.json, the regenerated
CONFORMANCE.md, and the 458 -> 459 row counts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…iter

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e (first slice of #270)

ToolingAst holds the closed schema of the tooling module as data and
ToolingLatex the production root renderToolingLatex, held to
backend::tooling::latex_text (render_latex now takes its text from it).
The differential case, 17 planted defects, the SPEC 17.23 paragraph, the
VERIFICATION entry and the CHANGELOG entry are in.

Not green: the evaluator leg, the seeded corpora, the plants and the
regeneration of the committed export did not run to the end on this host
(the WSL distribution restarted under every long run); VERIFICATION.md
records what ran and what is left. bootstrap/lexlean.lock, the roots and
packages of the production export and the stack row are not in this commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The sentence said a mutual group of types has no structural recursion. The
mutual-group rule admits a recursive family over one inductive group; the
module takes the shape of Expr because it is proven on this line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…270-tooling-latex

# Conflicts:
#	CONFORMANCE.md
#	SPEC.md
…aTeX root

The render_tooling_latex root, its certified rust-std package and
provenance, the stack row measured for it, and the relocked project.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The corpus gains tokens with the characters the escaper handles, signature
applications that take an explicit @, and commands that end in a token of
multi-byte white space; the plant that removed a function's final newline was
an equivalent mutant and is replaced. The case asserts its vector and mutant
counts. VERIFICATION.md records the run on the normative host.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…270-tooling-latex

# Conflicts:
#	VERIFICATION.md
…ssue-270-tooling-latex

# Conflicts:
#	.github/bdd-parts.json
#	CONFORMANCE.md
#	SPEC.md
#	bootstrap/lexlean.lock
#	bootstrap/lexlean.toml
#	crates/conformance/src/cases/bootstrap.rs
#	crates/conformance/tests/conformance.rs
#	features/suites/bootstrap.feature
#	model/ids.toml
#	tests/production-export/bootstrap/packages/rust-std/audit_tokens/package.json
#	tests/production-export/bootstrap/packages/rust-std/audit_tokens/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/check_arguments/package.json
#	tests/production-export/bootstrap/packages/rust-std/check_arguments/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/check_program/package.json
#	tests/production-export/bootstrap/packages/rust-std/check_program/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/correspond_crate/package.json
#	tests/production-export/bootstrap/packages/rust-std/correspond_crate/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/is_closed_lean_name/package.json
#	tests/production-export/bootstrap/packages/rust-std/is_closed_lean_name/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/lex/package.json
#	tests/production-export/bootstrap/packages/rust-std/lex/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/parse_audit_output/package.json
#	tests/production-export/bootstrap/packages/rust-std/parse_audit_output/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/print_crate/package.json
#	tests/production-export/bootstrap/packages/rust-std/print_crate/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/print_expr/package.json
#	tests/production-export/bootstrap/packages/rust-std/print_expr/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/reject_atoms/package.json
#	tests/production-export/bootstrap/packages/rust-std/reject_atoms/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/scan_source/package.json
#	tests/production-export/bootstrap/packages/rust-std/scan_source/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/validate_crate/package.json
#	tests/production-export/bootstrap/packages/rust-std/validate_crate/provenance.json
#	tests/production-export/bootstrap/packages/rust-std/validate_runtime/package.json
#	tests/production-export/bootstrap/packages/rust-std/validate_runtime/provenance.json
#	tests/production-export/bootstrap/roots.json
#	tests/stack-bounds.json
…ing #371 (BS-20, #270)

The lock, the build tree (maps, modules, coverage, closures, eligibility of
the two tooling modules), the 14 rust-std packages and the golden outputs
come from `lexlean lock`, `check-production-export --write` and
`check-golden --write` on the tree merged with #371 at 32d3171.

Not final: the count constants of the cases that use the committed tree
(BS-04 to BS-15) and the statements that restate them have not yet moved by
the one package this row adds, and #371's later head ef961c7 is not merged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The register gained BS-20, so the gate in xtask and the RP-07 case count 468.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…270)

The counts are those the aarch64 job of #364 reports at 4816894, with the
closure vector #371 adds to BS-14: BS-05 and BS-06 (278 manifests, 195
programs, 254 crates, 278 packages), BS-07 (329 vectors), BS-08 (388 crates),
BS-13 (98 vectors), BS-14 (333 vectors), BS-15 (297 manifests, 278 accepted,
278 render mutants). The README capability row claims `BS-01`..`BS-20` (RP-11)
and names the LaTeX of a tooling module.

Not final: later asserts in those cases and the statements that restate the
counts are not read yet, and the two tooling modules are not yet the
canonical form of `lexlean fmt` (AU-04).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
BS-05: 492 validate and 355 correspond vectors. BS-06: 674 printCrate vectors,
508 of them from the committed crates. These are the numbers the aarch64 job
of #364 reports at 960073a.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
verify::operational tests set the variables from_environment reads, with
no lock, while tests run in parallel, so one saw the other's values. They
now hold a shared lock, and start and end with the variables unset. The
new guard unsets rather than restores, because a saved value would need
an env::var_os read that conformance's undeclared_env_reads scan refuses.

verify::child::timeout_tests wrote a script and executed it. An exec fails
with ETXTBSY while a process another test is starting holds the inherited
write descriptor, as a_zombie_in_the_group_counts_as_ended does outside the
lifeline lock. The scripts now run through /bin/sh, which only reads them.
The failed aarch64 run reported leader.pid missing about 1 ms after the
test began, so its 200 ms timeout had not fired. The leader.pid panic now
prints what the run returned, which the old message hid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The aarch64 job of #364 reports 576 at 465a97b.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…(BS-20, #270)

`lexlean fmt` re-wraps the constructor and field lists of
`bootstrap/src/ToolingAst.lex.tex` and `ToolingLatex.lex.tex`; AU-04 and the
`fmt --check` of the examples recipe require it. The meaning is unchanged:
the generated Lean and TeX, the 14 rust-std packages and the stack bounds are
byte-identical, and the maps and coverage move with the source spans
(`lexlean lock`, `check-production-export --write`, `check-golden --write`).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ssue-270-tooling-latex

# Conflicts:
#	CONFORMANCE.md
#	README.md
#	SPEC.md
#	VERIFICATION.md
#	bootstrap/expected/build/manifest.json
#	bootstrap/expected/build/maps/LexLeanBoot/AxiomOutput.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/CalculusCheck.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/PrimitiveScan.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RuntimeValidate.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustAssemble.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustAst.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustBodies.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustLoops.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustLower.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustMoves.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustPackage.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustPrint.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustSucceed.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustTerms.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/RustValidate.map.json
#	bootstrap/expected/build/maps/LexLeanBoot/TokenAudit.map.json
#	bootstrap/expected/build/maps/LexLeanStd/Buffer.map.json
#	bootstrap/expected/build/maps/LexLeanStd/Sha256.map.json
#	bootstrap/expected/build/maps/LexLeanStd/Text.map.json
#	crates/conformance/src/cases/bootstrap/rust_assemble.rs
#	crates/conformance/src/cases/bootstrap/rust_loops.rs
#	crates/conformance/src/cases/bootstrap/rust_lower.rs
#	crates/conformance/src/cases/bootstrap/rust_moves.rs
#	crates/conformance/src/cases/bootstrap/rust_package.rs
#	crates/conformance/src/cases/bootstrap/rust_print.rs
#	crates/conformance/src/cases/bootstrap/rust_validate.rs
)

The merge with release/v0.4 (#328, five Order packages) moves every count the
committed tree gives. The register statements of BS-05, BS-06, BS-07, BS-08,
BS-13, BS-14 and BS-15 and the SPEC section 17.23 paragraphs of BS-05..BS-08
state the new values in ids.toml, SPEC.md, CONFORMANCE.md and the bootstrap
feature; the maps and manifest are regenerated. Mutant counts keep release's
values.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Stop two unit test flakes from sharing process state
… count (BS-20, #270)

The examples leg needs the verify records of the two tooling modules and of the
audit modules whose id moved with them; `verify-examples --write` (every
example but UOR Atlas) rewrote only bootstrap's. The evaluator agrees on 582
crates, not 584: the constant and the register statement, SPEC section 17.23,
CONFORMANCE.md and the feature say 582.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Retain every differential input and expected outcome while charging byte payloads by their emitted Rust array elements. Keep semantic performance node counts and generated packages unchanged.

The focused regression passes and rejects the planted old weighting; targeted Clippy and formatting pass. Windows BS-15 still needs CI confirmation.
Document building generated Lean with a matching runtime in Lake
humuhumu33 and others added 8 commits November 3, 2026 10:32
…270-tooling-latex

VERIFICATION.md keeps both sections, release's first.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The conflicts were the counts, two README rows, the CHANGELOG, VERIFICATION.md,
and the lock. The register is the release's 469 plus SL-11, 470, in SPEC.md,
README.md, xtask/src/spec_links.rs and the repository case. The lock takes the
release's header: stdlib's entrypoint moved, so the locks, exports and build
oracles are regenerated once, on top of #390, in the next commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… only)

Maps and sets over byte keys, the fifth member the self-hosted compiler needs:
the front end keeps its tables in BTreeMap<String, _> and its name sets in
BTreeSet<String> (154 uses in ir/semantic.rs, 53 in link.rs). Ten production
roots for rust-std over strictly ascending lists: mapFromEntries, mapInsert,
mapGet, mapRemove, mapExtend, setContains, setRemove, setUnion,
setIntersection, setDifference. The register is 471 IDs.

The case runs each member's corpus under the evaluator and under rustc against
BTreeMap and BTreeSet, plants one defect in each member, and catches each on a
fixed vector of its own that the committed module agrees with.

Nothing was built or run beyond the prebuilt CLI's check and build on a copy:
the module checks, and all ten roots are eligible with exactly their declared
effects. Not run: the case, the Lean axioms of the definitions (a guess from
Order's), the locks, exports and build oracles, and every count that follows
the committed packages (BS-05 to BS-15), which wait for the regeneration.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ource only)

The reader of the canonical JSON of section 21.7, the fourth member the
self-hosted compiler needs: readJson and readJsonFile read a text into the Json
value of LexLeanStd.Json and refuse any text the writer would not write. The
oracle is Json::parse with the text required to be what to_canonical_string
writes of it. The reader is one machine over an explicit stack, under the
text's length as its fuel, so no member recurses. Texts are taken to be valid
UTF-8, as the writer takes them; serde_json's limit of 128 levels is not
modelled. The register is 472 IDs.

The case runs both members' corpora under the evaluator and under rustc, and
plants one defect for each rule of the canonical form (twelve), each caught on a
fixed vector of its own that the committed module and the oracle agree on.

Nothing was built or run beyond the prebuilt CLI: the module checks, both roots
are eligible with exactly their declared effects, and every planted variant
still checks. Not run: the case, so the reader's behaviour is unmeasured; the
Lean axioms of the definitions (copied from Numeral's and Json's pattern); the
locks, exports and build oracles; and every count that follows the committed
packages, which wait for the regeneration.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The register is 475. The bootstrap lock is this tree's; the regeneration
replaces it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The row is the fourth slice of #269: `surface::generic::{parse, print}` as
language-1.2 source over the standard library's Json. It is stacked on BS-22,
BS-23, and the members SL-11, SL-12, and SL-14. The case fails until the module
is an entrypoint of the project; the source follows. The statement carries no
counts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`parseGeneric` lexes with lexText and reads the tokens with an explicit stack of
frames, so it admits no recursion; the object's members are sorted when it
closes. `printGeneric` builds the document by the structural recursion the
standard library's writer uses and lays it out with renderDoc. The module is an
entrypoint and the roots are exported; the lock is this tree's, and the
regeneration replaces it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ource only)

The reader of the canonical JSON of section 21.7, the fourth member the
self-hosted compiler needs: readJson and readJsonFile read a text into the Json
value of LexLeanStd.Json and refuse any text the writer would not write. The
oracle is Json::parse with the text required to be what to_canonical_string
writes of it. The reader is one machine over an explicit stack, under the
text's length as its fuel, so no member recurses. Texts are taken to be valid
UTF-8, as the writer takes them; serde_json's limit of 128 levels is not
modelled. The register is 472 IDs.

The case runs both members' corpora under the evaluator and under rustc, and
plants one defect for each rule of the canonical form (twelve), each caught on a
fixed vector of its own that the committed module and the oracle agree on.

Nothing was built or run beyond the prebuilt CLI: the module checks, both roots
are eligible with exactly their declared effects, and every planted variant
still checks. Not run: the case, so the reader's behaviour is unmeasured; the
Lean axioms of the definitions (copied from Numeral's and Json's pattern); the
locks, exports and build oracles; and every count that follows the committed
packages, which wait for the regeneration.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

The declared axioms of every definition of Map and JsonRead were compared with
what the committed definitions declare for the features they use and the
definitions they call (to_nat carries Quot.sound and propext, compare_bytes
propext, the rest none): byteOf, digitStep and digitsEnd now declare
[Quot.sound, propext]. The other definitions agree.
humuhumu33 and others added 6 commits November 3, 2026 13:50
The member's SL-12 commit was rewritten and differs from the one merged before
only in the three axiom lists of JsonRead, so the conflicts of rows and counts
take this branch's side and JsonRead takes the member's.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The lock pins the standard library by digest; it is this tree's, and the
regeneration replaces it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
examples/helpdesk pins the stdlib by digest, and the members SL-11, SL-12, and
SL-14 change it. `just repro` of the first draft named the stale pin.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
stdlib/lexlean.toml gained src/Map.lex.tex, and its lock still held the header
of the release the merge of SL-11 took. `lexlean lock` in stdlib, bootstrap, and
examples/helpdesk moves their digests; helpdesk pins the stdlib package it
imports, so without this its check refuses with LLC0105.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
stdlib/lexlean.toml gained src/JsonRead.lex.tex. `lexlean lock` in stdlib,
bootstrap, and examples/helpdesk moves their digests; helpdesk pins the stdlib
package it imports, so without this its check refuses with LLC0105 (found by
#269 on this branch).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
humuhumu33 and others added 9 commits November 3, 2026 14:40
The merge of #364 took the release header into stdlib/lexlean.lock, which still
describes a stdlib without LexLeanStd.Json: `repro` and six stdlib cases of
`bdd` refuse it with LLC0102. `lexlean lock` in stdlib, bootstrap, and
examples/helpdesk moves their digests. The exports, build oracles and counts
are the regeneration that follows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
`SOURCE` was never used, and the lint of the first draft refused it as dead
code. It was the only error of that step, so the case compiles.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ers) into BS-24

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The first CI run on the case stopped in stack_bounds::observe, which refuses a
package whose bound is not yet committed, before the rustc outcomes were
compared and before the plants ran. The bounds of every package are now read
after both, so a bound not yet committed hides no other result. The evaluator's
differential on all ten members had already passed in that run.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… into member 5 (SL-12)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…he decimal reader

CI run on 541ecd8: the evaluator and the oracle disagreed on 2 of 300 runs of
readJson, both with the outcome overflow where the oracle refuses. The texts
were 18446744073709551616 and -82095567141444985264: Numeral.parseDecimal
overflows past the target width, so the reader must not hand it a run of digits
longer than any i64 has. readInteger now refuses past nineteen digits, a plant
removes the guard, and the case reads the stack bounds last, as SL-14's does.
The other 298 runs agreed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
readInteger changed stdlib/src/JsonRead.lex.tex, so the tree digest of the stdlib
package moved, and the locks of bootstrap and examples/helpdesk that pin it with
it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@humuhumu33

Copy link
Copy Markdown
Contributor Author

Superseded by #451, which lands BS-22, BS-23, and BS-24 together: the three were regenerated on one tree (each member's export, records, and counts depend on the packages below it), and every case passed whole there. The source of this PR is carried in #451.

@humuhumu33 humuhumu33 closed this Nov 9, 2026
@afflom afflom mentioned this pull request Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants