Skip to content

Proposal: the kernel checks a replayed module once, in leanchecker (§22.3, §22.4) #391

Open

Description

@humuhumu33

@afflom, before I open a PR I would like your view on this, because it touches SPEC §22.3 and §22.4 and I would rather have the wording agreed than argued over a diff.

The observation

verify runs Lean's kernel twice over every replayed module: once inside lean at addDecl, and once more in the separate leanchecker process (§22.4). §22.4 already says the replay is the same kernel code run again and not an independent verifier. For the certificates of §17.17 the second pass is where most of a verification goes.

Measured on 2026-11-01 at LexLean 0b0c95d with Lean 4.32.1, on the lexlean-mcp gateway project (a language-1.2 project with production roots), local WSL, telemetry on:

width 1 width 4
full verify 2,401 s 1,548 s
certificates stage 1,834 s 1,224 s
elaboration stage 302 s 197 s
replay stage 175 s 69 s

Certificate B of gatewayRun (LexLeanPreserve.C….R2.RustStd) alone: 671 s to compile, 590 s to replay. Compiled with lean -D debug.skipKernelTC=true it takes 19 s, and its .olean is byte-identical to the normally compiled one. Compiling every module of the project on its own that way, 58 of 58 project modules and 24 of 24 comparable certificate modules were byte-identical (six certificate and four project modules were not compared because the reference file or source was not available). Project-module compile time fell from 274 s to 156 s. The sum of the per-module results suggests a full verify of about 1,450 s at width 1, but that has not been run end to end yet, and decide-heavy proofs may gain nothing. UOR Atlas is unmeasured. I will put the end-to-end numbers in the PR.

The change

Every Lean module that verify replays through leanchecker is compiled with -D debug.skipKernelTC=true: the elaborator checks it as before, the in-process kernel pass is skipped, and the replay becomes the one kernel check of its declarations. Exactly once instead of exactly twice.

  • Flagged: the project's generated modules (stage 7), the runtime modules the publishing part replays, and certificates A, B, and E (stage 12b), each of which already has its replay.
  • Never flagged: anything without a replay, because the in-process pass is its only kernel check. Today that is the text-laws library, the preservation library and calculus modules (LexLeanPreservation.*, LexLeanTarget.*), the certificate audit module, the probe, the extraction driver, and the audit members. A shard (§9.4) does not replay the runtime modules, so a shard compiles them with the pass on.
  • leanchecker never receives the option (it is not even a leanchecker argument; it refuses it).
  • A guard before publication: the part that publishes requires a successful leanchecker record for every elaboration record that carries the option, once after the replay stage over its own and imported records (so the join of §9.4 catches a shard-elaborated module nobody replayed, by name), and once more over the whole record list before the attestation is built. A hole is LLI9001, a verifier defect, and nothing is published.

What stays: every process still runs, nothing crosses runs, no cache, §9.4 and §21.8 unchanged, the attestation records every process, width invariance (§22.11) unchanged. The option enters the recorded argv, so every attestation ID and every expected/verify golden moves once.

Where a kernel refusal shows up: at the replay instead of at the compile. leanchecker names the declaration it was replaying (uncaught exception: while replaying declaration '…': (kernel) …), so LLV7013/LLV7015/LLV7016 still name the certificate's declaration and root; the three certificate-*-rejected fixtures change message text, not code or place. A generated module's kernel refusal is LLV7003 at the module. A planted false theorem added through addDecl compiles under the option and leanchecker refuses it with a kernel type mismatch; that becomes a Lean-backed half of a new row (VR-24), with the guard's planted-record refusals and a join plant.

SPEC wording I propose

§22.3 gains a paragraph: a module that §22.4 replays is compiled with the in-process kernel pass off, so the kernel checks each of its declarations once, in the replay; a module without a replay keeps the pass. §22.4 gains: the replay is the one kernel check of such a module, a declaration the kernel refuses is refused here, and publication requires a successful replay record for every module compiled without the pass. The sentence that the replay is not an independent verifier stays, with the reason made explicit: it is the same kernel, run once in a separate process rather than twice. §17.17's checking paragraph is adjusted to match. §9.4's no-cache rule and §21.8 are untouched.

Why I think this fits the line you took on #131

#131 was withdrawn because it would have carried an attestation across runs. This carries nothing across anything: the same processes run, in the same run, over the same bytes; one of two identical kernel passes is dropped and the remaining one is made a precondition of publication.

What I would like from you

  1. Whether you accept the principle: kernel once, in the replay, with the guard.
  2. The wording for §22.3/§22.4, if you want it phrased differently.
  3. Whether you want the runtime modules handled as above (pass on in a shard, off in the publishing part), or kept with the pass on everywhere.

Context: this is a SPEC-level decision Ilya is making with admin rights on the repository; it has not been reviewed by you yet, which is the point of this comment. Out of scope and not touched: splitting certificates, process reuse, content-named certificates (#353), PrismPM, lexlean-mcp and its pin.

Activity

  1. added this to the LexLean v0.4.1 milestone on Nov 2, 2026
  2. afflom commented on Nov 3, 2026

    @afflom
    Member

    Maintainer decision: accept the principle of one kernel check in the existing same-run replay for project modules and certificates A/B/E. This removes duplicate work without changing the verified claim. Keep the in-process kernel pass enabled for runtime modules in this first change, including publishing runs; we can revisit that small extra optimization if measurements justify the shard asymmetry.

    Please proceed with a PR against release/v0.4. The proposed §22.3/§22.4 wording is suitable with that runtime scope adjustment and the explicit statement that replay uses the same kernel, not an independent verifier. Keep all support-library, probe, extraction and audit-only modules checked in process.

    The publication guard must cover the actual modules compiled with the option, including imported shard records and the certificate tail. Reuse the existing same-run artifact bindings; demonstrate that a same-name record cannot accidentally discharge a different compiled module. Do not add a second hashing/audit framework. The false-declaration replay rejection and missing/failed replay and shard-join regressions should exercise the existing pipeline, with one final publication guard wherever possible.

    Report measured end-to-end gateway results and Atlas/width-invariance evidence, separating measured totals from projections, and regenerate the affected recorded argv and verification artifacts once on the final integration head. This accepts the design for implementation and review; merge still requires the complete gate. Please link the PR with Closes #391.

  3. afflom commented on Nov 3, 2026

    @afflom
    Member

    One namespace correction before implementation/writers: reserve VR-28 for this single-kernel replay/publication-guard claim. Draft #362 already contains distinct VR-24 (wrapper signature equality), VR-25 (imported axiom audit), VR-26 (import status) and VR-27 (binder/numeral hygiene) claims. Its VR-24 is not superseded by this proposal.

    The refreshed queued-source audit found no existing VR-28 reservation or implementation PR for this proposal. Use VR-28 consistently for the register, scenario, named test and prose so the eventual v0.4 → v0.5 integration preserves both claims. The accepted runtime scope, same-run artifact binding, replay/publication regressions, measurements and final-gate requirements above are unchanged.

  4. afflom commented on Nov 3, 2026

    @afflom
    Member

    Implementation review found a correction to the earlier binding assumption: current shard transport carries process records and a set of module names, but no digest associating a compile with the bytes replayed. A name-only publication check would therefore be insufficient.

    Use an optional typed .olean SHA-256 binding in the existing ChildRecord serialization and shard transport. This is smaller than a separate receipt/manifest protocol. Bind the actual compile output and actual resolved replay input, preserve certificate bindings before cleanup, and require one final publication guard over local/imported records plus the certificate tail. Successful pinned replay must match both module and bytes; project bindings must also match the staged publication artifacts. Missing, failed or mismatched associations refuse publication with LLI9001. Runtime/support/probe/extraction/audit checks remain enabled in process.

    This legitimately changes both closed process schemas and therefore all language identity partitions. That cost is accepted; do not hide it or bypass identity checks. Source preparation can proceed now, but official identity/verification regeneration stays serialized behind the existing release integrations and identity waves. VR-28, real false-declaration replay rejection, shard/join substitution plants, width invariance and measured gateway/Atlas evidence remain required.

    No implementation PR or ownership update is currently recorded. I am assigning a bounded maintainer source implementation and review; please link any unpublished active branch before duplicating this work. The eventual reviewed PR must close #391 only after complete integration, regeneration and the final gate.

  5. afflom commented on Nov 3, 2026

    @afflom
    Member

    Pinned-source inspection refined the artifact binding: LeanChecker.lean reads the main .olean, then .olean.server when present, then .olean.private when the server part exists; the last part supplies the replayed declarations. A main-file digest alone is insufficient.

    Use one typed artifact-parts member in the existing ChildRecord: required main digest, optional server/private digests using the repository's canonical omission convention. An empty companion file has its own digest and is distinct from absence; no null/one-element-array encoding or new receipt format is needed. Compare the complete captured parts at compile, replay and final publication; retain those certificate records before cleanup. The actual replay input must follow the pinned Lean/Lake search-path resolution, including workspace precedence, rather than assuming the staged module pathname.

    Bounded record/guard source preparation is proceeding, but the skip-kernel option will not be wired until that actual-path capture is reviewed. Existing replay, toolchain checks and all runtime/support checks remain unchanged meanwhile.

  6. afflom commented on Nov 3, 2026

    @afflom
    Member

    The artifact-binding source preparation now captures Lean's actual effective search path through the existing external-interface probe, with no added process/helper. The packet remains recorded through narrow typed normalization; raw roots drive first-package-root resolution, companion hashes and staged/replayed-byte comparison. Runtime/support/probe/extraction/audit kernel behavior is unchanged, and skip flags plus the final publication guard are still unwired pending final capture review.

    Independent review caught two genuine holes before enabling anything: the exact source-command allowance could fall back to prefix matching, and the packet decoder accepted escape-equivalent duplicate keys. Static exact matching and a closed typed decoder now reject both; their mutation/refusal tests pass. Actual pinned-Lean VR-04 and unchanged VR-07 noise plants pass in4.05s/4.42s after fixing a real command type-inference error. The focused run used an official temporary nat-add-zero lock refresh for the changed schema identity, then restored its exact original bytes; no test helper or expectation was bypassed. Pure binding/partition checks and correct-root VR-28 also pass.

    This is focused source evidence only. The foreign-root attempt was discarded, and no product pin/regeneration/full gate or performance saving is claimed. The Host/#402 wave retains sole writer ownership. Final flag/guard integration still owes real false-declaration rejection, shard/width/Atlas publication binding and the required measured acceptance.

  7. afflom commented on Nov 3, 2026

    @afflom
    Member

    Reviewed source preparation is published at f655d0b158 on fix/vr28-single-kernel (tree 25b4e4cbff4ed04ef7cc78e40e0be7f04b486221). The skip option is now wired only for eligible generated project modules and certificates A/B/E. Runtime, support, probe, extraction and audit compilations retain their in-process kernel checks. One final guard binds local/imported/certificate records to the actual replayed main/server/private parts and staged publication bytes; no cross-run reuse is introduced.

    The real false-declaration regression passed in 5.53s: ordinary Lean.addDecl supplies False with True.intro; unflagged compilation refuses it, flagged compilation succeeds, and the real pinned checker refuses the captured artifact. The pipeline plant independently observes successful flagged compilation followed by LLV7003 naming the bad declaration, with no publication. Three partition tests, focused Clippy and formatting also pass. Independent review checked the final executed patch and the recorded initial test/Clippy failures and corrections; no helper, checker or expectation was bypassed.

    This is source preparation, not completed acceptance. Positive production certificates and LLV7013/7015/7016 controls, real shard/join and width invariance, gateway/Atlas measurements, coordinated all-language identity/artifact regeneration and the final full gate remain required. Existing checker prefix rechecks remain, so this does not claim globally exactly-once checking. The Host/#402 wave keeps sole writer ownership; no new PR gate or competing regeneration was started. The eventual issue-closing PR will carry this source and its completed evidence.

  8. afflom commented on Nov 3, 2026

    @afflom
    Member

    The remaining certificate audit found a concrete integration defect before regeneration: failing_declaration recognized decoder/compile refusals but not pinned Lean replay's while replaying declaration envelope, and the diagnostic summary searched only for the word error. A kernel replay type mismatch could therefore lose the certificate declaration and report no error was reported, despite correctly refusing publication.

    A narrow source correction has passed independent review. It recognizes the pinned replay envelope, retains the existing compile formatting and bounded UTF-8-safe summary, and strengthens SP-05's existing production export and A/B/E rejection executions. The positive assertions bind flagged compiles to successful same-module complete artifact-part replay; the negatives retain LLV7013/7015/7016 and no-publication while checking actual replay detail, declaration and root. No duplicate certificate verification was added.

    This follow-up is still local and unexecuted while the Host writer owns the phase lock; it is not new passing evidence or a published head update. It must be formatted and tested, including those real certificate fixtures, before inclusion. Existing shard/width/Atlas, measurement, identity and final-gate obligations remain unchanged.

  9. afflom commented on Nov 4, 2026

    @afflom
    Member

    The reviewed certificate-diagnostic follow-up is now published at ec6ba36fe, tree2951f032f80f1eb8eb20046004a82af9e13036cb, extending the existing source-preparation branch.

    The three exact diagnostic units passed together. The unchanged real SP-05 case then passed in990.888s: positive A/B/E compiles carry the selected skip flag and match successful same-module replay artifact tuples; real A/B/E rejection fixtures retain LLV7013/7015/7016, replay/kernel detail, exact declaration/root and no publication. Heartbeat/resource/drift controls retain their existing codes and publication assertions. Focused Clippy, formatting and whitespace checks passed. All tested Rust bytes are unchanged in the published follow-up.

    Earlier failed setup attempts remain in the evidence. The successful run archived only proven owned empty helper directories created by official temporary lock commands, before the strict fixture loader ran. It did not weaken the loader. Seven original lock files were restored byte-for-byte; frozen binaries and actual linked roots matched before/after. No duplicate positive verify or completed-unit rerun was added.

    This discharges the focused certificate assertions, not the overall optimization acceptance. Shard/join and width invariance, gateway/Atlas measurements, coordinated all-language identities/artifacts and the full gate remain required. No new identity wave or complete CI was started; #391 stays open.

  10. afflom commented on Nov 4, 2026

    @afflom
    Member

    The next bounded acceptance follow-up is now validating on the existing source preparation. Independent review found no demonstrated binding implementation defect; the added VR-21/VR-23 assertions inspect actual complete-part bindings and kernel-check scope in their existing width/whole/shard/join results. Five copied-shard failure cases exercise main/server/private record substitution, a missing binding and a failed imported replay, with no publication on refusal. Existing assertions and positive verification calls are retained.

    Review removed one newly proposed staged-companion join before execution: ordinary Lean could reject it before the final guard, so it would not establish additional guard evidence beyond the existing byte/path plants and successful whole/join binding. This does not remove an existing case or requirement. The serial VR-21/VR-23 group runs once, without repeating completed SP-05, diagnostic units or the kernel-false regression.

    The phase holds the shared lock, reuses the frozen CLI, verifies selected harness roots/embedded inputs, and preserves/restores the two required temporary official fixture locks. The new harness is frozen by a verified transfer rather than a redundant ~226MiB copy. Build/provenance/test failures stop with evidence; no bulk cleanup or automatic rerun. No result is claimed yet. Atlas/gateway measurements, coordinated identities/artifacts and complete final acceptance remain open; this focused phase is not a new identity wave or merge clearance.

  11. afflom commented on Nov 4, 2026

    @afflom
    Member

    Focused-phase update: it stopped before any VR-21/VR-23 execution or temporary fixture lock. The first harness build succeeded but the linked probe demonstrated a stale repo-model root; its admitted targeted repair then failed linking both conformance and Cargo’s automatically selected bootstrap_perf binary with signal7 bus errors. Very low free disk was recorded, but there is no explicit ENOSPC diagnostic, so I am not claiming an established cause.

    The original logs, formatted source and provenance results are preserved. Frozen CLI and both original locks are unchanged; no functional pass is claimed. No automatic retry or cleanup ran. I am reviewing a bounded resource continuation that preserves the exact case group, profile guarantees and failure evidence; this is not an identity writer or a retry of any previously blocked experiment.

  12. afflom commented on Nov 4, 2026

    @afflom
    Member

    The bounded continuation is now running after a concrete capacity change: two completely clean, completed integration checkouts were retired using Git, while retaining their exact commits/trees, named refs and restore commands. This recovered985MB; source objects, unfinished work, failure logs and frozen binaries remain preserved. No debug-profile change or package-cache cleanup was used, and the prior bus-error cause is still not asserted.

    The unchanged-profile build now succeeded. Its selected probe reports the exact intended model/support/fixture roots, all905 embedded inputs match, and the selected harness was frozen by the reviewed byte-identical transfer. The two temporary official locks succeeded and their newly owned empty guards were archived. The exact two-case VR-21/VR-23 group has started; no functional pass is claimed until its result and the subsequent checks are complete. Original STOP evidence remains intact.

  13. afflom commented on Nov 4, 2026

    @afflom
    Member

    The independently reviewed join/width follow-up is now published at 104f755ae9, tree2c91cd770fc3020f3699eefd61f66ec1b5036e9c, extending the existing source-preparation branch.

    One exact serial VR-21/VR-23 group passed2/2 in279.03s, retaining peano width1/3, production width1/2, and whole/shard/join equality. New assertions inspect the actual complete-part bindings and kernel-check scope in existing successful outputs. Five copied-shard substitutions all refused publication: main/server/private replay tuples and missing binding reached the final binding guard (LLI9001), while failed imported replay was rejected at assembly (LLV7003). No positive verification call was added.

    Focused Clippy, formatting and whitespace checks passed. Independent terminal review verified the actual linked roots, all905 embedded inputs, frozen binaries and exact restoration of both temporary official locks. Only newly owned empty helper guards were archived; no loader override was used. Tested Rust is byte-identical in this commit. Earlier failed build evidence remains intact, with its bus-error cause unproven; the capacity recovery and later successful continuation are separately recorded. Completed SP-05, diagnostic units and kernel-false checks were not repeated.

    This is provisional functional evidence on the existing preparation pins. Atlas/gateway measurements, actual Atlas publication binding/invariance, coordinated all-language identities/artifacts and complete final CI remain required. No new identity wave or full gate ran, and #391 stays open. Checker prefix rechecks still exist; there is no globally exactly-once claim.

  14. afflom commented on Nov 4, 2026

    @afflom
    Member

    @humuhumu33, the remaining measurement audit could not locate the gateway corpus used in the original 2026-11-01 WSL measurements. Please provide its immutable repository/commit (or archive digest), project directory, config/original lock and source-package closure, plus production root selection. If available, attach the original width1/4 telemetry and invocation/toolchain/host receipts. This is an evidence/source handoff request; please do not start another writer or measurement wave.

    For a new controlled comparison, source ed1deb0 is the immediate pre-skip comparator. Against published104f755ae9, its successful-path differences are the optimization together with required final guard/rehash and scratch-retention costs; it is not the historical0b0c baseline or a flag-only comparison. A matched binary still needs to be built once after final source/corpus review. Existing Host/396 binaries have unrelated changes and cannot substitute for it.

    Atlas candidate timing and invariance evidence should reuse the already-required final all-language verification regeneration and final same-run shard/join gate. Current release.yml consumes CI evidence; it does not independently execute whole Atlas. I will not count it as such or launch a duplicate preliminary Atlas pass. Exact baseline/candidate/source/toolchain/width/host totals will be reported only after actual controlled runs; historical estimates remain historical.

  15. afflom commented on Nov 4, 2026

    @afflom
    Member

    The corpus prerequisite has advanced: the public UOR-Foundation/lexlean-mcp tree at 991ab332608727306977c9850da6be182ab80071, with its exact MCP/Prism pins, is now materialized immutably. Its selected corpus bytes also match the inspected November 1 public snapshot; this does not establish the original WSL checkout or historical timings.

    The frozen candidate's supported official lock and check commands both passed for all 11 configured entrypoints / 58 transitive modules. All 323 original inputs were preserved; only the separate scratch lock's compiler identity changed. No entrypoints or production markers were removed, and --all was not substituted for the configured selection.

    This is compatibility evidence only: no verification or benchmark ran. A minimal unchanged pre-skip ed1deb0c CLI build is being reviewed for a new controlled pair against published 104f755a. That comparison will measure the implemented skip plus required binding/guard costs, not flag-only cost or the original historical run. Remaining Atlas, all-language identities/products and complete final acceptance remain open; completed focused cases will not be repeated.

  16. afflom commented on Nov 4, 2026

    @afflom
    Member

    The new controlled public-corpus pair is complete and independently reviewed. Both full verifications succeeded over the same 11 configured entrypoints, 58 selected modules and four production roots; no --all substitution or root reduction was used.

    Observed metric Pre-skip ed1deb0 Candidate 104f755
    Complete CLI verification wall time 1,863.117 s 1,243.365 s
    Available waited-child CPU (user + system) 2,176.786 s 1,475.059 s
    Certificate stage 1,258.461 s 724.466 s
    Project elaboration stage 237.961 s 154.454 s
    Project replay stage 146.720 s 146.105 s

    That is 619.752 s (33.264%) lower observed wall time in this single pair. It measures the net skip implementation plus required binding/guard/rehash costs. Baseline ran first and candidate second with fresh project workspaces, width1 and unchanged global OS/toolchain caches; order/cache effects remain a limitation. This is neither a flag-only attribution, a general performance estimate, nor reproduction of the original WSL timings.

    Independent inspection confirmed equal official locks, source/semantic/build IDs, all 2,732 declaration policy/observation rows, toolchain/workspace records, preservation records and selected roots. Candidate's flagged compiles are exactly 58 project modules plus 12 A/B/E modules, each bound to a unique successful same-module kernel replay with its complete captured part map. Project records include main/server/private; the 12 certificate records have main and genuinely absent companions. Runtime/support/probe/extraction/audit compiles remain unflagged. All 323 original inputs, both copied source sets and both independently frozen CLIs were preserved.

    Evidence: terminal result SHA256 e5591ab5646ff2f2367df6c2c7f85784b2e6bc2c13401601d4d4ff1d061b1b42; independent review 5a074c61c36e6cfeb9f4a3796245bd385f4537238c565e34af175b33a6147eab. Raw timing/process data and successful artifacts are retained locally. No run was repeated. These are provisional preparation identities: Atlas publication/invariance and measurements, coordinated all-language identities/artifacts and complete final-head acceptance remain open. This issue stays in v0.4.1 and is not silently included in the separate #393/#390 wave.

  17. afflom commented on Nov 4, 2026

    @afflom
    Member

    The independently reviewed measurement record is now preserved in the existing preparation branch at 494fc28a9f, parent104f755ae9, tree77d74de1451f94422af480c092ae10e43d0e84f6. This changes only VERIFICATION.md, retaining every prior byte and the exact measured implementation. It includes corpus and executable identities, commands, observed timing boundaries, policy/replay-binding evidence and the comparison limitations. No completed case, build, product writer or full gate was repeated for this documentation update. Remaining acceptance described above is unchanged.

  18. afflom commented on Nov 5, 2026

    @afflom
    Member

    After #393 landed as 55be551, the existing 494fc28 preparation has been reconciled in an external source-only candidate and independently reviewed. Tree cda635374db00e4423bda93238aef7ef6657f0a2 changes 19 authored paths, preserves both exact VERIFICATION histories, Host diagnostic/policy safeguards and the accepted nested-induction/rejection work, and has 476 matching unique claim/scenario/named-test IDs. The two closed attestation schema extensions retain the existing required fields. Independent receipt SHA256: 9479812c212b6da3d21c8970946286b7afaca7ecbca07113fa3b06a34ee74377.

    This is static composition evidence only; no build, identity/product writer, case, measurement, Atlas run, commit or push was performed. The current #394 export remains the sole active producer. This preparation still needs reconciliation with its eventual accepted landing, coordinated all-language identities/products and complete final-head acceptance. Existing 494 history must survive; the external tree is not a commit parent. v0.4.1 scope is unchanged.

    Accepted 55be SPEC already fixes the old wording assigning whole Atlas verification to release.yml, so no additional normative policy edit is needed. The remaining plan captures width-1 Atlas timing/normalized records during the already-required whole verification writer and reuses the final hosted width-2 shard/join evidence; release collects successful same-tree CI evidence. Completed SP05/units/kernel-false/VR21/23 and the controlled public comparison will not be repeated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestquestionFurther information is requested

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions