Skip to content

Move the Lean pin from v4.32.1 to v4.33.1 - #433

Draft
humuhumu33 wants to merge 6 commits into
release/v0.5from
claude/lean-4-33-1-pin
Draft

humuhumu33 wants to merge 6 commits into
release/v0.5from
claude/lean-4-33-1-pin

Conversation

@humuhumu33

@humuhumu33 humuhumu33 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Closes #420

This replaces the Lean pin leanprover/lean4:v4.32.1 (commit f054605aea4b840552cca2e725580bffd1e1b704) with leanprover/lean4:v4.33.1 (commit 819816b2e0a3bf405af45ae5c7af2491d8f5bee6). There is still one pin, one gate, and one set of expected outputs. The PR targets release/v0.5, per AGENTS.md's release-line rule, and should not be merged before review.

What changed

The first commit holds the hand-written sources:

  • the §8.2 chapter module, lean-toolchain, both Dockerfiles, and every pinned workspace and fixture configuration (including the 307 negative fixtures);
  • language/semantics*.toml and the schemas;
  • the compiler constants and the version-banner tests;
  • the authority and ledger modules, with IDs now *-4-33-1 and the source archive's SHA-256 re-acquired (47c32e11…; the old archive still hashes to the committed 6dec8667…, so codeload is stable).

The second commit holds the regeneration, written only by the owning commands:

  1. check-calculus --write until nothing changes;
  2. validate-model --write;
  3. lexlean lock in every project the previous compiler accepted, twice (the second pass changes nothing);
  4. check-fixtures --write, check-golden --write, check-production-export --write;
  5. verify-examples --write;
  6. a last check-calculus --write for the digests of the relocked compiler project.

The seven negative fixtures whose lock is defective by design can't be written by lexlean lock. As in the earlier identity waves, only their identity fields were substituted. No fixture's diagnostic code changed.

Compiler-semantics IDs, each moved once:

language before after
1.0 89507c61… df4317a1…
1.1 11732e9a… 1fdcc6a3…
1.2 49a667a8… (0b485471… after #396) 69fd8ec2…

What 4.33.1 broke

UOR Atlas does not verify yet, and this PR does not fix it. Atlas's source (examples/uor-atlas/src) is native kernel-term data captured from 4.32.1's elaborator: proofs produced by omega and linear-integer arithmetic, and compiled n + k match patterns. Two things in 4.33.1 break it:

  • Moved namespace: Int.Linear became Int.Internal.Linear (23 constants, 47 references in 6 modules). Lean's kernel rejects LexLeanExample.Glue first, with unknown constant 'Int.Linear.norm_eq'. On a scratch copy, the pure rename made Glue and Places pass the kernel.
  • Removed definition: Nat.elimOffset (fun h₁ h₂ => h₂ (add_right_cancel h₁)) is gone, with no counterpart. It appears once in each of Census, CensusClosure, Group and Scales.

The other 904 of the 928 external constants Atlas references exist in 4.33.1 with unchanged types. Fixing Atlas means changing its source data. The options are now with the owner:

  1. Rewrite the data: the rename, plus inlining Nat.elimOffset at its four nodes. The kernel checks the result, and MIGRATION.md records it as the 4.33.1 migration.
  2. Re-export from the original Lean implementation at checkpoint f972285f, ported to 4.33.1.

Until then the Atlas shard jobs fail.

Every other project verifies under 4.33.1 with no generator change: elaboration, leanchecker replay, and the exact axiom audit all pass.

Two Lean-version-specific inputs were re-derived rather than assumed:

  • LCNF registry (tests/lcnf/registry.json, regenerated from the pinned Lean): no signature of any used constant changed; only the SHA-256s of the defining source files did.
  • Reserved-token list (lean_tokens.rs): re-derived by probing Lean.Parser.getTokenTable over Init, Std and Lean under both toolchains. The identifier-like token tables are identical (220 tokens), so the list stands.

One generator-visible change: the new commit hash starts with a digit, so it is no longer a bare name in the tooling notation. The four hand-written tooling sources now quote it, which is the form the printer already emits in language/lcnf-1.2/tooling/src/Extract.lex.tex.

Acceptance

  • Version check: verify rejects any reported Lean other than 4.33.1 with the existing LLV7001 (see the toolchain-mismatch fixture). The required negative (the real 4.32.1 executable through the upgraded preflight, producing LLV7001 and publishing nothing) has not been run yet; the toolchain-mismatch fixture plants 4.31.0, which is not that.
  • Grep: git grep -F 4.32.1 and git grep -E '4[-_]32[-_]1|f054605' now hit only CHANGELOG.md and dated evidence records. Those are VERIFICATION.md measurement logs, the MIGRATION.md files, tests/preservation-certificates/*-STATEMENTS.md, and tests/runtime-library/LAWS.md; they record what was run under 4.32.1 at named commits, and rewriting them would falsify history. The current-state lines of VERIFICATION.md (the examples row and the authority sentence) now say 4.33.1.
  • Grep caveat: a bare git grep 4.32.1 is a regex. It also matches long numerals in kernel-term data (for example …4232318… in compiler/proofs and the Atlas core data), so it can never return only CHANGELOG.
  • just vv: CI on this PR runs the whole gate, Atlas shards included.

Verification wall time per example

Not measured yet. No timing in this PR is a completed measurement. The planned method, per the maintainer's notes on #420: freeze the 4.32.1 comparison corpus; record the source heads, machine, width, build profile and cache policy; run the same corpus under 4.32.1 and 4.33.1 back to back on an idle machine with cargo xtask verify-examples and LEXLEAN_CI_TELEMETRY; and report steps.verify_ms and wall_ms for each example. Atlas reports shard and join evidence and the critical-path elapsed time, not a sum of concurrent durations. Stale-lock refusals are not timing samples.

Downstream (separate bumps)

Each needs a LexLean release from release/v0.5 that contains this change (v0.5.0); until then each stays on v0.4.0 and Lean 4.32.1:

  • PrismPM
  • template (UOR-Foundation/template)
  • lexlean-mcp
  • uor-evm

#334

The Mathlib work for #334 exists only in open PR #362 (feat/mathlib-import), so this PR doesn't touch it. A comment on #334 and #362 points its target at the Mathlib v4.33.1 tag, commit 0df444a360eaa60ab8c11dca51a86af692955474.

🤖 Generated with Claude Code

humuhumu33 and others added 3 commits November 3, 2026 15:29
#420)

SPEC.md §8.2's chapter module, lean-toolchain, both container images, every
pinned workspace and fixture configuration, the language data, the schemas,
the authority and ledger modules (re-acquired source archive SHA-256), the
compiler constants, and the version-banner tests. Generated outputs follow
in the regeneration commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every compiler-semantics ID moves once: language 1.0 df4317a1, 1.1 1fdcc6a3,
1.2 e7451506. Written by the owning commands only, in this order:
check-calculus --write (the LCNF registry from the pinned Lean: no signature
changed, only defining-source digests) to its fixed point, validate-model
--write, lexlean lock in every project the previous compiler accepted (twice;
the second pass changes nothing), check-fixtures --write, check-golden
--write, check-production-export --write, verify-examples --write for every
example but UOR Atlas, and a last check-calculus --write for the digests of
the relocked compiler project.

The seven negative fixtures whose lock is defective by design cannot be
written by `lexlean lock`; as in the earlier identity waves only their
identity fields follow (an ID equal to its language's previous current ID,
the toolchain line, and the lean-toolchain digest). No fixture's diagnostic
code changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kappa's generated outputs take the release side and are regenerated on the
4.33.1 identity in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@afflom
afflom marked this pull request as draft November 3, 2026 17:45

afflom commented Nov 3, 2026

Copy link
Copy Markdown
Member

I have marked this draft because its own acceptance section leaves Atlas records, the actual 4.32.1 refusal and controlled before/after timings unfinished. Current head c413d76d0f8ecd0698c56db85ebc0347f2a77657 is being source-reviewed; there is no full-gate acceptance on that head. #420 remains open.

Please preserve completed compatibility outputs and record the exact currently active writer/head here. The accepted ordering in #420 was source integration with #362's reviewed follow-ups, #405 and #396, then one v0.5 toolchain/identity wave after the active v0.4 Host/#402 wave. This PR's regeneration ran ahead of that integration. Do not start another regeneration or full matrix on an incomplete tree; finish any already-running atomic writer safely and retain its logs, then reconcile the shared source before the final wave. We will use this existing PR rather than duplicate the upgrade branch.

The remaining evidence must use actual 4.32.1 through upgraded preflight (LLV7001 and no publication), a reproducible same-corpus baseline and after-run with per-example verify and total time, complete Atlas shards/join and final-head CI. Preserve frozen older-language runtime bytes and legitimate historical/wrong-version fixture exceptions. The timing section currently describes back-to-back measurements but says they are still pending; please distinguish planned methodology from completed measurements.

@afflom afflom left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bounded independent source review found no additional implementation blocker in the authored pin-migration delta. Normative sources originate in spec/src; compiler/schema/project pins and generator templates follow the new version. Frozen older-runtime files are byte-identical, all 307 negative diagnostic files retain their codes, and the LCNF registry changes exactly 179 defining-source hashes with no other parsed changes. Token-table probing and archive reacquisition were not independently executed.

This is not merge approval. The branch is based on 2849449, before #407; current v0.5 is a8ad058. KA-05/06 differences against the current base are stale integration, not an authored revert, and the early regeneration cannot establish final integrated κ compatibility. The real old-version refusal remains absent: toolchain-mismatch plants 4.31.0, not the required actual 4.32.1 executable. Atlas, integrated-source compatibility, controlled per-example verify/total timings and full final-head CI remain outstanding. Keep 'no proofs broke' scoped to the disclosed projects actually verified, and #420 open until the coordinated final integration and evidence are complete.

humuhumu33 and others added 3 commits November 4, 2026 04:30
kappa/lexlean.lock by lexlean lock, kappa/expected/build by check-golden
--write, kappa/expected/verify by verify-examples --write; check-calculus and
validate-model pass unchanged, and no other output moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every conflict is a generated output; each takes the release side and is
regenerated on the 4.33.1 identity in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#396 moves the language-1.2 compiler-semantics ID (the runtime's Portable
module), so the whole wave runs again on the merged tree, by the owning
commands only: check-calculus --write to its fixed point, validate-model
--write, lexlean lock in every project (twice), check-fixtures --write,
check-golden --write, check-production-export --write, verify-examples
--write for every example but UOR Atlas, and a last check-calculus --write.
Languages 1.0 and 1.1 keep their 4.33.1 IDs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@humuhumu33

Copy link
Copy Markdown
Contributor Author

Record of the active writer and head, as requested.

No writer is running. The current head is 7dd6361b3, pushed 2026-11-04. I did not see your 17:45 comment before acting. After it, I merged release/v0.5 twice and ran two more writer waves. Both finished without being interrupted, and they ran ahead of the integration order you set. I won't start another regeneration or matrix until the shared sources are reconciled and you call the final wave.

Commits on the branch:

The resulting IDs are 1.0 df4317a1…, 1.1 1fdcc6a3… and 1.2 69fd8ec2…. The integration with #362's follow-ups, #405 and the v0.4 Host/#402 wave is not in this head, so these outputs are not the final integrated wave.

All writer logs are kept outside the tree, with per-step exit codes and seconds: phase 1, phase 2, fixtures, golden, verify-write and calculus for the first wave, the kappa wave, and the post-#396 wave. I can attach them.

Compatibility found so far (4.33.1, Linux x86-64 container):

  • Every project but UOR Atlas verifies with no generator change: all 17 other examples, compiler, stdlib, std-models, std-reasoning, kappa and bootstrap. That covers elaboration, leanchecker replay and the exact axiom audit.
  • The LCNF registry changes only defining-source hashes.
  • I probed the token table myself with Lean.Parser.getTokenTable over Init, Std and Lean under both toolchains: identical, 220 identifier-like tokens.
  • The source archive was re-acquired: 47c32e11…. The old archive still reproduces 6dec8667….
  • Atlas fails. Its native kernel terms reference Int.Linear.*, which 4.33.1 moved to Int.Internal.Linear.* (23 constants, 47 references in 6 modules), and Nat.elimOffset, which 4.33.1 removed (one shared node in each of Census, CensusClosure, Group and Scales). A rename-only experiment on a scratch copy passed the kernel for Glue and Places; nothing is committed. Fixing this means changing Atlas source data, which needs your decision. The options are in the PR body.

Still outstanding:

  • the real-4.32.1 refusal (LLV7001, nothing published);
  • the frozen-corpus before/after timings (the PR body now gives only the planned method);
  • the Atlas shards and join;
  • the integrated final wave and final-head CI.

The frozen older-language runtime bytes are unchanged.

@afflom afflom left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refreshed source/evidence review at7dd6361b: this head now retains #407 and actual landed #396. The checked kernel/slice/join/Host sources match that landing; frozen older runtime text and 307 negative code/exit sets are preserved. Current hosted honesty proves483 model/spec rows and five BDD meta tests; reproducibility passed. The partial draft run skips seven host/Atlas stages and is not complete acceptance.

The remaining #420 requirements are still real: actual4.32.1 executable refusal with LLV7001/no publication, controlled same-corpus per-example verify and total timings, Atlas compatibility and actual shard/join critical-path evidence, then the final integrated full gate. The current Atlas authored source and347 verification records have not been repaired for the disclosed Int.Linear/Nat.elimOffset failures. Please continue the bounded Atlas SOURCE repair on this existing branch, preserving declarations/proof guarantees, before the shared freeze; do not hand-edit its generated records or start another identity/product wave yet.

Keep the complete logs from the additional completed waves, and clarify the body’s “each moved once” wording so it describes a particular wave rather than the branch’s history. The reported Atlas reference count also needs its scan scope stated: the committed authored-src scan found23 Int.Linear constants/43 references in four modules; that is narrower than the reported47/six and is not execution evidence.

The final v0.5 source freeze must include reviewed #362 follow-ups, #405’s Host/maintenance/DF-19 forward integration and new draft #437 formatting isolation, in addition to retained #396/#407. I have read your no-active-writer handoff and agreement to coordinate. Keep this PR draft and #420 open; no further matrix or regeneration is requested by this review.

afflom commented Nov 4, 2026

Copy link
Copy Markdown
Member

The integration inventory is now reconciled against the reviewed #405 + initial #362 + #437 source preparation (62db257d, 511 matching authored claims). This is source preparation only; no upgrade writer or new matrix has started.

The original pin commit has two schema conflicts with the new Mathlib contracts. Resolve those on the current schemas by changing only lean_toolchain.const to 4.33.1, retaining every closed lean_library definition. Its compiler/gnaf.manifest.json is derived: carry the producer change and let the official writer regenerate the manifest later. The remaining inventory includes new Host/DF19/import fixtures and active Mathlib pin sites that did not exist in the old upgrade source. Active Mathlib uses accepted 0df444a360eaa60ab8c11dca51a86af692955474; deliberate mismatch/tag tests and historical measurements retain their meaning and recorded versions.

Future replay-cache infrastructure has been separated from the initial Mathlib delivery without dropping its pending claims/tests. #438 is consequently a future #338 prerequisite, not a new prerequisite for this upgrade. The existing Atlas source-repair request remains with this PR; preserve the old comparison corpus and controls before retargeting active sources. Final dependency resolution, identities/products, real old-version refusal, controlled timings, Atlas and full final-head CI remain required.

afflom commented Nov 4, 2026

Copy link
Copy Markdown
Member

To make the Atlas decision explicit: use option 1 from this PR body for the bounded source repair. Rename the affected Int.Linear references and inline the old Nat.elimOffset definition at its existing nodes, preserving the original declarations, theorem statements, universe/binder arguments and axiom policies. Record the exact authored-file/reference scope and the migration rationale. Do not add a compatibility axiom, weaken a declaration or replace the corpus with a broader re-export.

Please keep the repair on this existing branch and separate the authored delta from generated records. A small direct 4.33.1 kernel check of the changed terms can establish the source repair; retain the exact toolchain/source/checker evidence and any failures. Complete Atlas shard/join and full acceptance still belong to the coordinated final tree. This does not authorize another identity/product regeneration or full matrix.

afflom commented Nov 13, 2026

Copy link
Copy Markdown
Member

Kept as the record of the 4.33.1 migration (Atlas findings: Int.Linear move, Nat.elimOffset removal). Not mergeable as is: its 4.3M-line regeneration is stale once #405 lands, and #518 moves v0.4 to 4.32.2, so lean-toolchain will conflict in the forward merge. Plan: after #405, rebase and regenerate once for v0.5's chosen pin. Left as draft.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants