From 513ea1534ff9dc0ee2a7fbd4c04c9ff567c15903 Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Thu, 12 Nov 2026 14:59:10 +0000 Subject: [PATCH 1/4] verify: every part names its toolchain by address A shard writes toolchain.json (lexlean/verify-toolchain/1) beside its output: the SHA-256 of the lean, lake and leanchecker its preflight hashed, of the running lexlean, and the address of the rustc that compiled it (its executable and rustc_driver library, hashed by the build script). A part that reads another refuses it (LLI9001, naming the part, the tool and both addresses) unless all five equal its own. A process record names only lean or leanchecker, so a part that ran another lake, lexlean or rustc was accepted before. VR-23 plants another lake address in a shard's output and, on unix, runs a join whose own lake differs from the pinned one by one byte; both are refused. Published directories, attestations and expected/verify do not change, and neither does the compiler-semantics ID. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 12 ++ CONFORMANCE.md | 2 +- SPEC.md | 6 +- crates/conformance/src/cases/operational.rs | 74 +++++++++ crates/lexlean/Cargo.toml | 5 + crates/lexlean/build.rs | 77 +++++++++ crates/lexlean/src/lib.rs | 7 + crates/lexlean/src/verify/mod.rs | 25 ++- crates/lexlean/src/verify/partition.rs | 174 ++++++++++++++++---- crates/lexlean/src/verify/toolchain.rs | 111 +++++++++++++ features/suites/verification.feature | 3 +- model/ids.toml | 2 +- 12 files changed, 455 insertions(+), 43 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 08322fa32a..5af8bd4b4e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,18 @@ workspace carried at those versions without a tag. ## Unreleased +- Every part of a verification names its toolchain by address (SPEC.md + §22.11, §9.4). A shard writes `toolchain.json` (`lexlean/verify-toolchain/1`) + beside its output: the SHA-256 of the `lean`, `lake`, and `leanchecker` its + preflight hashed, of the running `lexlean`, and the address of the `rustc` + that compiled it (its executable and `rustc_driver` library, which the build + script hashes). A part that reads another refuses it (`LLI9001`, naming the + part, the tool, and both addresses) unless all five equal its own. A record + names only the executable it ran, so a part that ran another `lake`, + `lexlean`, or `rustc` was accepted before. `VR-23` plants another `lake` + address in a shard's output, and on unix a join whose own `lake` differs by + one byte. Published directories, attestations, and `expected/verify` are + unchanged. - A one-pass writer for the counts of the bootstrap count cases, #92. Under `LEXLEAN_COUNTS=fast` each of `BS-05` to `BS-08` and `BS-11` to `BS-15` records every count it checks in `target/count-report/` instead of diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 5e0010e5e9..438caf2cef 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -675,7 +675,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `VR-20` | `build` | The verifier process width is an operational resource profile selected only by the environment, defaulting to 1 and refusing any malformed value with LLV7017, and it enters no lock, identity, or verified artifact. | | `VR-21` | `build` | A multi-module project whose imports form nontrivial frontiers verifies at width 1 and at a larger width to byte-identical verified directories and the same attestation identity, observed axiom sets, and policy rows, and no completion order orders a record or changes which failure is reported. | | `VR-22` | `build` | Performance telemetry is written only to the CI evidence directory the environment names, a verification with telemetry on publishes byte-identical artifacts to one without, and telemetry planted in a verified artifact fails the gate. | -| `VR-23` | `build` | A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, or a join reading a record of an executable other than its own, is refused. | +| `VR-23` | `build` | A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, a join reading a record of an executable other than its own, or a part that ran a toolchain other than its own, is refused. | ## Cited authorities diff --git a/SPEC.md b/SPEC.md index fb94dc902f..d2646e6557 100644 --- a/SPEC.md +++ b/SPEC.md @@ -695,7 +695,7 @@ Its image MUST install the pinned Rust and Lean toolchains at build, so that not In a run that is the whole gate, the jobs run the recipes as the rows of `.github/vv-rows.json`, each from its own clean checkout: `examples`, which verifies every example but UOR Atlas; `honesty`, which runs `fmt-check model spec-links bdd` before any expensive root; `rest`, the other recipes in the order of `vv`; and one `bdd-` row for each named part of the `bdd` plan below, which runs `bdd` alone. Each leg runs `just` with its recipes, which stops at the first that fails, as `vv` does. The part of `examples` a job runs is named by its matrix row through `LEXLEAN_EXAMPLES_PART` and by nothing else: `without:uor-atlas` on the leg, `shard:uor-atlas:` and `join:uor-atlas` on the Atlas jobs below; unset or empty, which is what `just vv` runs, the recipe verifies every example whole. After honesty succeeds, the remaining rows run as one matrix and the supported-host matrix, Intel construction and Atlas prefix may start; a failed or skipped honesty part starts none of these expensive roots. Together the legs and the Atlas jobs run every recipe of `vv`, every example, and every test of `bdd` exactly once, and the run passes only when every job passes, so it proves what one run of `just vv` proves; no recipe depends on another having run, and every recipe owns the files it builds. Its wall time is the longest path through the jobs rather than the sum. `just vv` remains the acceptance recipe of §9.2 for every other run, and release acceptance requires the complete same-tree run. -UOR Atlas is verified as the parts of one run. `.github/verify-shards.json` partitions its verification into shards, each naming the shards it needs, the modules it elaborates, and the modules it replays and audits, one of them named `join`; `cargo xtask ci-shards` checks, as the verifier would when a part ran, that every module is elaborated by exactly one shard and replayed by exactly one, that every import of a module a shard elaborates or replays is elaborated by that shard or by one it needs, directly or through another, and that `join` needs every other shard. Each shard is a job of its own clean checkout that runs the verifier's prelude whole (certificate generation, the preflights, staging, the generated-source audit, the probe), then elaborates, replays, and audits only the modules of its plan, reading the `.olean` files of the modules it imports from the output the jobs of the shards it needs uploaded in the same run, and uploads its own `.olean` files and process records as its output. The join runs the modules of its plan, reads every shard's records, requires each to name the executable the join itself hashed for the tool that produced it (`lean` for an elaboration or an audit member, `leanchecker` for a replay), judges each as the stage that produced it judges its own, and continues through policy, the attestation over every record in the canonical order of §22.11, publication, and the comparison of every normalized record with `expected/verify` (§29.5): what it publishes is what the whole verification publishes, record for record and attestation identity included (`VR-23`). A module without a record from any part, a record of a process no module owns, a record whose `executable_sha256` is not the join's digest of its tool (`LLI9001`, naming the record and both digests), or a shard whose plan leaves a module without the `.olean` it imports fails the run. Every process of the verification runs exactly once per run. The executable the Atlas jobs run is the one the `prefix` job built from its clean checkout, carried to the other jobs of the run as an artifact beside the shards' output; it, the output, and the records cross no run, and a job reads the artifacts of its own run only. The Atlas jobs run after the jobs of the shards they need, at the profile of the `examples` leg, and only in a run that is the whole gate. +UOR Atlas is verified as the parts of one run. `.github/verify-shards.json` partitions its verification into shards, each naming the shards it needs, the modules it elaborates, and the modules it replays and audits, one of them named `join`; `cargo xtask ci-shards` checks, as the verifier would when a part ran, that every module is elaborated by exactly one shard and replayed by exactly one, that every import of a module a shard elaborates or replays is elaborated by that shard or by one it needs, directly or through another, and that `join` needs every other shard. Each shard is a job of its own clean checkout that runs the verifier's prelude whole (certificate generation, the preflights, staging, the generated-source audit, the probe), then elaborates, replays, and audits only the modules of its plan, reading the `.olean` files of the modules it imports from the output the jobs of the shards it needs uploaded in the same run, and uploads its own `.olean` files and process records as its output. The join runs the modules of its plan, reads every shard's records, requires each to name the executable the join itself hashed for the tool that produced it (`lean` for an elaboration or an audit member, `leanchecker` for a replay), requires every shard's output to name the join's own toolchain addresses (§22.11), judges each as the stage that produced it judges its own, and continues through policy, the attestation over every record in the canonical order of §22.11, publication, and the comparison of every normalized record with `expected/verify` (§29.5): what it publishes is what the whole verification publishes, record for record and attestation identity included (`VR-23`). A module without a record from any part, a record of a process no module owns, a record whose `executable_sha256` is not the join's digest of its tool (`LLI9001`, naming the record and both digests), a shard whose output names no toolchain addresses or another address of any tool (`LLI9001`, naming the part, the tool, and both addresses), or a shard whose plan leaves a module without the `.olean` it imports fails the run. Every process of the verification runs exactly once per run. The executable the Atlas jobs run is the one the `prefix` job built from its clean checkout, carried to the other jobs of the run as an artifact beside the shards' output; it, the output, and the records cross no run, and a job reads the artifacts of its own run only. The Atlas jobs run after the jobs of the shards they need, at the profile of the `examples` leg, and only in a run that is the whole gate. The `bdd` recipe runs as the parts of one run as well. `.github/bdd-parts.json` names parts by conformance ID, complete integration-test targets in `targets`, and a remainder, `rest`. Cases that share a fixture built once per test process belong to one part. The `honesty` part owns the complete `bdd` meta-gate target, including any tests added to that target; no copied test-name list selects it. `cargo xtask bdd` runs `cargo test -p repo-conformance --all-features` with the Cargo target selectors and harness filters of `LEXLEAN_BDD_PART`. A target part selects its targets through `--test`; an ID part uses `--exact` and the test of each ID. The remainder discovers all integration-test targets from Cargo metadata, selects every target no target part owns, and keeps the library and binary tests and doctests; Cargo requires doctests in a separate invocation when target selectors are present. Its `--exact` and `--skip` filters omit every conformance test an ID part owns. A new test in a target belongs to that target's part; a new integration target and a new conformance ID belong to the remainder until assigned. No case decides whether it runs. `inventory` lists the tests using the same selectors and filters, proves disjointness and equality with the whole `bdd` suite, and requires a target part to own every test of its target (§9.2). Exactly one row runs every part, the remainder included. The initial honesty row names its part; each later `bdd-` row and the `rest` row name theirs. Unset or empty, as in local `just vv`, the recipe runs the whole suite unchanged. Each row builds from its own clean checkout; no build or evidence crosses runs. @@ -10224,7 +10224,7 @@ The profile changes how processes are scheduled and nothing else: For every project, the directory published at any profile equals, byte for byte, the directory published at width 1 without hints and without a budget, on the same host and toolchain; in particular the attestation ID, the observed axiom sets, the policy rows, the canonical compiler input, and every normalized verification record (§29.5) are identical. -A verification may also run as the parts of one run (§9.4): each part runs the processes of its plan at the profile of its own host, a part reads from another only the `.olean` files and process records that part's processes produced, and the join judges every record it reads as the stage that produced it judges its own, after requiring it to name the join's own digest of the executable that stage runs. The directory the join publishes equals, byte for byte, the one the whole verification publishes, attestation ID included (`VR-23`); a part is defined for a project without production roots and without PDF rendering. +A verification may also run as the parts of one run (§9.4): each part runs the processes of its plan at the profile of its own host, a part reads from another only the `.olean` files and process records that part's processes produced, and the join judges every record it reads as the stage that produced it judges its own, after requiring it to name the join's own digest of the executable that stage runs. Every part writes beside its output `toolchain.json` (`lexlean/verify-toolchain/1`): the SHA-256 of the `lean`, `lake`, and `leanchecker` its preflight hashed, of the running `lexlean`, and the address of the `rustc` that compiled that `lexlean`, which is the SHA-256 of one line ` ` per file of the compiler's `rustc` executable and `rustc_driver` library under `bin/` and `lib/`, in path order, as the build of `lexlean` found them. A part that reads another part requires its `toolchain.json` to name its own five addresses, and refuses the part otherwise, before it reads any record (`LLI9001`): a record names only the executable it ran, so a part that ran another `lake`, `lexlean`, or `rustc` is seen only there. The directory the join publishes equals, byte for byte, the one the whole verification publishes, attestation ID included (`VR-23`); a part is defined for a project without production roots and without PDF rendering. ### 22.12 Performance telemetry @@ -11805,7 +11805,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `VR-20` | `verification` | The verifier process width is an operational resource profile selected only by the environment, defaulting to 1 and refusing any malformed value with LLV7017, and it enters no lock, identity, or verified artifact. | §22.11, §23.1 | | `VR-21` | `verification` | A multi-module project whose imports form nontrivial frontiers verifies at width 1 and at a larger width to byte-identical verified directories and the same attestation identity, observed axiom sets, and policy rows, and no completion order orders a record or changes which failure is reported. | §22.3, §22.4, §22.11 | | `VR-22` | `verification` | Performance telemetry is written only to the CI evidence directory the environment names, a verification with telemetry on publishes byte-identical artifacts to one without, and telemetry planted in a verified artifact fails the gate. | §22.12 | -| `VR-23` | `verification` | A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, or a join reading a record of an executable other than its own, is refused. | §9.4, §22.11 | +| `VR-23` | `verification` | A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, a join reading a record of an executable other than its own, or a part that ran a toolchain other than its own, is refused. | §9.4, §22.11 | | `CL-01` | `cli-api` | Global options and upward project discovery obey the exact CLI contract. | §23.1, §23.2 | | `CL-02` | `cli-api` | Init creates the complete canonical skeleton only in an absent or empty destination and never overwrites. | §23.4 | | `CL-03` | `cli-api` | Lock check, local update, and explicit network acquisition obey their exact mutually exclusive behavior. | §23.4 | diff --git a/crates/conformance/src/cases/operational.rs b/crates/conformance/src/cases/operational.rs index 5719b0f8ac..2cd3da1638 100644 --- a/crates/conformance/src/cases/operational.rs +++ b/crates/conformance/src/cases/operational.rs @@ -905,6 +905,80 @@ fn parts_equal_the_whole() { text.contains("from any part of the run") && text.contains(&rest[0]), "{text}" ); + // A join refuses a part that ran another toolchain: the first shard's + // output names the address of every executable it ran or was built by + // (§22.11), and a `lake` other than the join's is refused, naming the + // part, the tool, and both addresses, although no record names `lake`. + let addresses_path = dir("a").join("toolchain.json"); + let addresses = std::fs::read_to_string(addresses_path.as_std_path()) + .expect("a shard names its toolchain addresses"); + let lake = serde_json::from_str::(&addresses).expect("the addresses parse") + ["lake"] + .as_str() + .expect("the addresses name lake") + .to_owned(); + let planted_lake = "f".repeat(lake.len()); + assert_ne!(lake, planted_lake); + std::fs::write( + addresses_path.as_std_path(), + addresses.replacen(&lake, &planted_lake, 1), + ) + .expect("plant another lake"); + let error = run_part(VerifyPart::Join { + plan: plan(&rest, &rest), + parts: vec![dir("a"), dir("b")], + }) + .map(|_| ()) + .expect_err("a part of another lake"); + support::expect_code(&error, "LLI9001"); + let text = error.to_string(); + assert!( + text.contains("ran `lake`") && text.contains(&planted_lake) && text.contains(&lake), + "{text}" + ); + std::fs::write(addresses_path.as_std_path(), &addresses).expect("restore the addresses"); + // The same refusal from a real binary: a join whose own toolchain's + // `lake` differs by one appended byte from the pinned one the shards ran + // refuses them, naming both addresses. Its preflight hashes the changed + // file, so the join never trusts a part of the pinned toolchain. + #[cfg(unix)] + { + let pinned = support::real_elan_home() + .join("toolchains") + .join(support::mangled_toolchain_name()) + .join("bin") + .join("lake"); + let mut changed = std::fs::read(&pinned).expect("the pinned lake"); + changed.push(0); + let changed_address = lexlean::Sha256Digest::of(&changed).to_hex(); + let home = support::fake_elan_home(&[("lake".to_owned(), changed)]); + let home_text = home.path().to_string_lossy().into_owned(); + let project = P::copy_example(example); + let error = support::with_env(&[("ELAN_HOME", Some(&home_text))], || { + Operational::new(1, None).expect("width 1").scoped(|| { + lexlean::verify_part( + &project.engine(), + VerifyRequest { + selection: Selection::Entrypoints, + }, + VerifyPart::Join { + plan: plan(&rest, &rest), + parts: vec![dir("a"), dir("b")], + }, + ) + }) + }) + .map(|_| ()) + .expect_err("a join whose own lake changed"); + support::expect_code(&error, "LLI9001"); + let text = error.to_string(); + assert!( + text.contains("ran `lake`") && text.contains(&lake) && text.contains(&changed_address), + "{text}" + ); + } + #[cfg(not(unix))] + support::unix_only("VR-23", "symlinked toolchain to plant a changed `lake` in"); // A join refuses a record that another `leanchecker` ran: the second // shard's replay record of a module names an executable that is not the // join's, as a part on a different toolchain would have written it. diff --git a/crates/lexlean/Cargo.toml b/crates/lexlean/Cargo.toml index ab72c8f8be..9f9576ea37 100644 --- a/crates/lexlean/Cargo.toml +++ b/crates/lexlean/Cargo.toml @@ -39,6 +39,11 @@ walkdir = { workspace = true } rustix = { workspace = true } signal-hook = { workspace = true } +# The build script records the address of the `rustc` that compiles this crate +# (SPEC.md §22.11): the SHA-256 of its executable and driver library. +[build-dependencies] +sha2 = { workspace = true } + [dev-dependencies] proptest = { workspace = true } diff --git a/crates/lexlean/build.rs b/crates/lexlean/build.rs index ddac668a2e..b8241cf791 100644 --- a/crates/lexlean/build.rs +++ b/crates/lexlean/build.rs @@ -82,6 +82,83 @@ fn main() { errors.display() ); fs::write(out_dir.join("errors.toml"), bytes).expect("write errors.toml"); + + rustc_identity(&out_dir); +} + +/// The address of the compiler that builds this crate (SPEC.md §22.11): the +/// SHA-256 of one line ` ` per +/// file of the `rustc` executable and its driver library, in path order, and +/// what `rustc -vV` reports. The `rustc` executable is a thin launcher; the +/// compiler is the driver library it loads, so both are addressed. A compiler +/// whose sysroot cannot be read is a build failure, never an empty address. +fn rustc_identity(out_dir: &Path) { + use sha2::{Digest, Sha256}; + use std::process::Command; + + println!("cargo:rerun-if-env-changed=RUSTC"); + let rustc = env::var("RUSTC").unwrap_or_else(|_| "rustc".to_owned()); + let run = |argument: &str| { + let output = Command::new(&rustc) + .arg(argument) + .output() + .unwrap_or_else(|error| panic!("running `{rustc} {argument}`: {error}")); + assert!( + output.status.success(), + "`{rustc} {argument}` failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + String::from_utf8(output.stdout).expect("rustc reports UTF-8") + }; + let version = run("-vV").replace("\r\n", "\n"); + let sysroot = PathBuf::from(run("--print=sysroot").trim()); + let mut files: Vec<(String, PathBuf)> = Vec::new(); + for directory in ["bin", "lib"] { + let Ok(entries) = fs::read_dir(sysroot.join(directory)) else { + continue; + }; + for entry in entries.filter_map(Result::ok) { + let name = entry.file_name().to_string_lossy().into_owned(); + let compiler = name == "rustc" + || name == "rustc.exe" + || name.starts_with("rustc_driver-") + || name.starts_with("librustc_driver-"); + if compiler && entry.path().is_file() { + files.push((format!("{directory}/{name}"), entry.path())); + } + } + } + files.sort(); + assert!( + files.iter().any(|(path, _)| path.starts_with("bin/rustc")), + "{}: no `rustc` executable in the compiler's sysroot", + sysroot.display() + ); + let mut listing = String::new(); + for (relative, path) in &files { + println!("cargo:rerun-if-changed={}", path.display()); + let bytes = + fs::read(path).unwrap_or_else(|error| panic!("reading {}: {error}", path.display())); + listing.push_str(&hex(&Sha256::digest(&bytes))); + listing.push_str(" "); + listing.push_str(relative); + listing.push('\n'); + } + let address = hex(&Sha256::digest(listing.as_bytes())); + fs::write( + out_dir.join("rustc_identity.rs"), + format!( + "/// What `rustc -vV` reported for the compiler that built this crate.\n\ + pub const RUSTC_VERSION: &str = {version:?};\n\ + /// The address of that compiler's executable and driver library.\n\ + pub const RUSTC_SHA256: &str = {address:?};\n" + ), + ) + .expect("write rustc_identity.rs"); +} + +fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|byte| format!("{byte:02x}")).collect() } /// Resolve a crate-relative normative path: a directory or file as such; a diff --git a/crates/lexlean/src/lib.rs b/crates/lexlean/src/lib.rs index dd7664959b..0edc6c3780 100644 --- a/crates/lexlean/src/lib.rs +++ b/crates/lexlean/src/lib.rs @@ -106,6 +106,13 @@ pub mod embedded { include!(concat!(env!("OUT_DIR"), "/embedded.rs")); } +/// The compiler that built this crate, as the build script found it +/// (SPEC.md §22.11): what `rustc -vV` reported and the address of its +/// executable and driver library. It is no input of any compiler semantics. +pub mod build_identity { + include!(concat!(env!("OUT_DIR"), "/rustc_identity.rs")); +} + /// The original language identifier. Kept as the compatibility default for /// downstream code that compiled against LexLean 1.0. pub const LANGUAGE_VERSION: &str = "1.0"; diff --git a/crates/lexlean/src/verify/mod.rs b/crates/lexlean/src/verify/mod.rs index 498053eb77..7adcaa1de0 100644 --- a/crates/lexlean/src/verify/mod.rs +++ b/crates/lexlean/src/verify/mod.rs @@ -1333,16 +1333,23 @@ pub fn run_part( // What the parts before this one wrote (§9.4): their `.olean` files into // the staged tree, and, for the part that publishes, their records. stages.enter("part-import"); + // The addresses of everything this run executes or was built by + // (§22.11): a part writes them beside its output and a reader requires + // its own. The running executable must be readable to be named. + let own_addresses = toolchain::Addresses::of( + &toolchain, + running_executable_sha256().map_err(|io_error| { + fail(internal(format!( + "reading the running executable: {io_error}" + ))) + })?, + ); let mut imported = partition::import(part.inputs(), &olean_root, part.publishes()).map_err(fail)?; - // Every record another part wrote was run by the toolchain this run's - // preflight hashed: the attestation lists them under its digests. - partition::check_toolchain( - &imported, - toolchain.lean.sha256, - toolchain.leanchecker.sha256, - ) - .map_err(fail)?; + // Every part before this one ran the toolchain this run's preflight + // hashed, and every record it wrote names it: the attestation lists them + // under its digests. + partition::check_toolchain(&imported, &own_addresses).map_err(fail)?; // The leanchecker identity (module documentation), before any replay: // the preflight's toolchain-relative path and digest line, then the @@ -1932,7 +1939,7 @@ pub fn run_part( .plan() .map(|plan| plan.elaborate.clone()) .unwrap_or_default(); - partition::export(staging.path(), output, &own).map_err(fail)?; + partition::export(staging.path(), output, &own, &own_addresses).map_err(fail)?; telemetry_run.succeeded(); return Ok(PartOutcome::Shard { output: (*output).to_owned(), diff --git a/crates/lexlean/src/verify/partition.rs b/crates/lexlean/src/verify/partition.rs index b50a1c205a..e887693394 100644 --- a/crates/lexlean/src/verify/partition.rs +++ b/crates/lexlean/src/verify/partition.rs @@ -22,9 +22,13 @@ use std::collections::{BTreeMap, BTreeSet}; use camino::{Utf8Path, Utf8PathBuf}; use crate::artifact::canonical_json::Json; -use crate::artifact::content_id::Sha256Digest; use crate::diagnostic::Diagnostic; use crate::verify::child::ChildRecord; +use crate::verify::toolchain::Addresses; + +/// The file of a part's output that names the addresses of the executables +/// the part ran and was built by (§22.11). +pub const TOOLCHAIN_FILE: &str = "toolchain.json"; /// Which modules one part elaborates, and which it replays and audits. /// @@ -116,6 +120,8 @@ pub struct Imported { pub leanchecker: BTreeMap, /// Audit records by audit member name. pub audit: BTreeMap, + /// The toolchain addresses each part named, by its output directory. + pub toolchains: Vec<(Utf8PathBuf, Addresses)>, } fn io(what: &str, error: &std::io::Error) -> Diagnostic { @@ -137,12 +143,15 @@ fn malformed(what: &str, reason: impl std::fmt::Display) -> Diagnostic { /// /// A module's `.olean` may come from one part only, and a record of one /// process from one part only: two parts that both wrote one is a plan that -/// ran a process twice, which is refused rather than resolved. +/// ran a process twice, which is refused rather than resolved. Every part +/// names the addresses of its toolchain ([`TOOLCHAIN_FILE`]), which +/// [`check_toolchain`] compares with the reader's own. /// /// # Errors /// -/// A part that cannot be read, a duplicate `.olean` or record, or a record -/// that does not parse as the verifier wrote it. +/// A part that cannot be read, a duplicate `.olean` or record, a record +/// that does not parse as the verifier wrote it, or a part without its +/// toolchain addresses. pub fn import( parts: &[Utf8PathBuf], olean_root: &Utf8Path, @@ -156,6 +165,22 @@ pub fn import( "the output directory of a part is required and is not a directory", )); } + let addresses_path = part.join(TOOLCHAIN_FILE); + let addresses = std::fs::read(addresses_path.as_std_path()) + .map_err(|error| { + malformed( + addresses_path.as_str(), + format!("a part must name the addresses of its toolchain: {error}"), + ) + }) + .and_then(|bytes| { + Json::parse(&bytes).map_err(|reason| malformed(addresses_path.as_str(), reason)) + }) + .and_then(|json| { + Addresses::from_json(&json) + .map_err(|reason| malformed(addresses_path.as_str(), reason)) + })?; + imported.toolchains.push((part.clone(), addresses)); let oleans = part.join("oleans"); if oleans.as_std_path().is_dir() { for entry in walkdir::WalkDir::new(oleans.as_std_path()) @@ -222,31 +247,49 @@ pub fn import( Ok(imported) } -/// Require every imported record to name the executable the join itself -/// runs: `lean` for an elaboration or an audit member, `leanchecker` for a -/// replay (§9.4, §22.9). +/// Require every part to name the toolchain this run's preflight hashed, and +/// every imported record to name the executable the join itself runs: +/// `lean` for an elaboration or an audit member, `leanchecker` for a replay +/// (§9.4, §22.9, §22.11). /// /// In a whole verification every record carries the digest of the toolchain /// the run's preflight hashed, by construction. Across the parts of a run it /// holds only because every job installs the same pinned toolchain, which is -/// a practice and not a check: a part whose `lean` or `leanchecker` differed -/// from the join's would otherwise contribute records the attestation then -/// lists under the join's toolchain digests. +/// a practice and not a check: a part whose `lean`, `lake`, `leanchecker`, +/// `lexlean`, or the `rustc` that built it differed from the reader's would +/// otherwise contribute records the attestation then lists under the +/// reader's toolchain. A record names only the executable it ran, so the +/// part's own addresses ([`TOOLCHAIN_FILE`]) are compared first, whole. /// /// # Errors /// -/// The first record, in the order elaborations, replays, audit members, and -/// by name within each, whose `executable_sha256` is not the join's digest of -/// the tool that produced it, naming the record and both digests. -pub fn check_toolchain( - imported: &Imported, - lean: Sha256Digest, - leanchecker: Sha256Digest, -) -> Result<(), Diagnostic> { +/// The first part, in the order given, whose addresses differ from `own`, +/// naming the part, the tool, and both addresses; then the first record, in +/// the order elaborations, replays, audit members, and by name within each, +/// whose `executable_sha256` is not the reader's digest of the tool that +/// produced it, naming the record and both digests. +pub fn check_toolchain(imported: &Imported, own: &Addresses) -> Result<(), Diagnostic> { + for (part, addresses) in &imported.toolchains { + if let Some((tool, theirs, mine)) = addresses.first_difference(own) { + return Err(malformed( + &format!("the part `{part}`"), + format!( + "it ran `{tool}` {}, but this run's `{tool}` is {}; every part of a run must run one toolchain", + theirs.to_hex(), + mine.to_hex() + ), + )); + } + } for (what, tool, expected, records) in [ - ("elaboration", "lean", lean, &imported.lean), - ("replay", "leanchecker", leanchecker, &imported.leanchecker), - ("audit", "lean", lean, &imported.audit), + ("elaboration", "lean", own.lean, &imported.lean), + ( + "replay", + "leanchecker", + own.leanchecker, + &imported.leanchecker, + ), + ("audit", "lean", own.lean, &imported.audit), ] { for (name, record) in records { if record.executable_sha256 != expected { @@ -314,7 +357,9 @@ fn read_records( /// intermediates (a later part's elaboration reads them), the elaboration /// and replay records, and the audit records. The `.olean` files the shard /// imported stay behind: they are the part that produced them's to export, -/// once. Nothing else leaves the staging tree, which is removed with it. +/// once. Beside them the part names the addresses of its toolchain +/// ([`TOOLCHAIN_FILE`]). Nothing else leaves the staging tree, which is +/// removed with it. /// /// # Errors /// @@ -323,6 +368,7 @@ pub fn export( staging: &std::path::Path, output: &Utf8Path, own: &BTreeSet, + addresses: &Addresses, ) -> Result<(), Diagnostic> { if output.as_std_path().exists() { let mut entries = std::fs::read_dir(output.as_std_path()) @@ -336,6 +382,11 @@ pub fn export( } std::fs::create_dir_all(output.as_std_path()) .map_err(|error| io("creating the part's output directory", &error))?; + std::fs::write( + output.join(TOOLCHAIN_FILE).as_std_path(), + addresses.to_json().to_file_bytes(), + ) + .map_err(|error| io("writing the part's toolchain addresses", &error))?; let oleans = staging.join("oleans"); if oleans.is_dir() { for entry in walkdir::WalkDir::new(&oleans).sort_by_file_name() { @@ -545,7 +596,15 @@ mod tests { let out = tempfile::tempdir().expect("tempdir"); let output = Utf8PathBuf::from_path_buf(out.path().join("part")).expect("utf8"); let own: BTreeSet = ["X.M".to_owned()].into_iter().collect(); - export(staging.path(), &output, &own).expect("exports"); + let digest = crate::artifact::content_id::Sha256Digest::of; + let addresses = Addresses { + lean: digest(b"lean"), + lake: digest(b"lake"), + leanchecker: digest(b"leanchecker"), + lexlean: digest(b"lexlean"), + rustc: digest(b"rustc"), + }; + export(staging.path(), &output, &own, &addresses).expect("exports"); assert!( staging.path().join("oleans/X/N.olean").exists() && !staging.path().join("oleans/X/M.olean").exists(), @@ -566,11 +625,58 @@ mod tests { assert!(imported.audit.contains_key("LexLeanAudit.X.M")); // The records name the join's `lean`; a join whose `lean` is another // executable refuses them, naming the record and both digests. - let lean = crate::artifact::content_id::Sha256Digest::of(b"lean"); - let other = crate::artifact::content_id::Sha256Digest::of(b"another lean"); - check_toolchain(&imported, lean, other).expect("the records name the join's lean"); - let refused = check_toolchain(&imported, other, other) - .expect_err("a record of another lean is refused"); + let lean = addresses.lean; + let other = digest(b"another lean"); + assert_eq!(imported.toolchains, vec![(output.clone(), addresses)]); + check_toolchain(&imported, &addresses).expect("the part and its records name the join's"); + // A part that ran another executable of any of the five is refused, + // naming the part, the tool, and both addresses, before any record. + for (tool, plant) in [ + ( + "lake", + Addresses { + lake: other, + ..addresses + }, + ), + ( + "lexlean", + Addresses { + lexlean: other, + ..addresses + }, + ), + ( + "rustc", + Addresses { + rustc: other, + ..addresses + }, + ), + ] { + let refused = check_toolchain(&imported, &plant) + .expect_err("a part of another toolchain is refused"); + let message = format!("{refused:?}"); + assert!( + message.contains("LLI9001") + && message.contains(&format!("ran `{tool}`")) + && message.contains(&other.to_hex()), + "{message}" + ); + } + // Records are compared against the reader's own `lean` once the part + // agrees: here the part's addresses are taken as the reader's, and + // the reader's `lean` differs from the one the record names. + let mut reader = imported; + reader.toolchains.clear(); + let refused = check_toolchain( + &reader, + &Addresses { + lean: other, + ..addresses + }, + ) + .expect_err("a record of another lean is refused"); let message = format!("{refused:?}"); assert!( message.contains("LLI9001") @@ -586,9 +692,21 @@ mod tests { let wrong = out.path().join("wrong/process/lean/X.N.json"); std::fs::create_dir_all(wrong.parent().expect("parent")).expect("mkdir"); std::fs::write(&wrong, record.to_json().to_file_bytes()).expect("write"); + std::fs::write( + out.path().join("wrong").join(TOOLCHAIN_FILE), + addresses.to_json().to_file_bytes(), + ) + .expect("write"); let wrong_part = Utf8PathBuf::from_path_buf(out.path().join("wrong")).expect("utf8"); let fresh = tempfile::tempdir().expect("tempdir"); let fresh_root = Utf8PathBuf::from_path_buf(fresh.path().join("oleans")).expect("utf8"); assert!(import(std::slice::from_ref(&wrong_part), &fresh_root, true).is_err()); + // A part that names no toolchain addresses is refused. + std::fs::remove_file(output.join(TOOLCHAIN_FILE)).expect("remove"); + let bare = tempfile::tempdir().expect("tempdir"); + let bare_root = Utf8PathBuf::from_path_buf(bare.path().join("oleans")).expect("utf8"); + let refused = import(std::slice::from_ref(&output), &bare_root, true) + .expect_err("a part without toolchain addresses"); + assert!(format!("{refused:?}").contains("addresses of its toolchain")); } } diff --git a/crates/lexlean/src/verify/toolchain.rs b/crates/lexlean/src/verify/toolchain.rs index 0aae60e21b..ad2ba98784 100644 --- a/crates/lexlean/src/verify/toolchain.rs +++ b/crates/lexlean/src/verify/toolchain.rs @@ -34,6 +34,117 @@ pub struct Toolchain { pub leanchecker: Tool, } +/// The format of [`Addresses`] as a part of a verification writes it. +pub const ADDRESSES_SPEC: &str = "lexlean/verify-toolchain/1"; + +/// The addresses of every executable a verification ran or was built by +/// (SPEC.md §22.11): `lean`, `lake`, and `leanchecker` as the preflight +/// hashed them, the running `lexlean`, and the `rustc` that compiled it. +/// Two parts of one verification agree on all five or neither trusts the +/// other's records. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Addresses { + /// `lean`. + pub lean: Sha256Digest, + /// `lake`. + pub lake: Sha256Digest, + /// `leanchecker`. + pub leanchecker: Sha256Digest, + /// The running `lexlean` executable. + pub lexlean: Sha256Digest, + /// The `rustc` that compiled this `lexlean` (its executable and driver + /// library, [`crate::build_identity::RUSTC_SHA256`]). + pub rustc: Sha256Digest, +} + +impl Addresses { + /// The addresses of this preflighted toolchain, the running executable + /// whose digest is `lexlean`, and the compiler that built it. + #[must_use] + pub fn of(toolchain: &Toolchain, lexlean: Sha256Digest) -> Self { + Self { + lean: toolchain.lean.sha256, + lake: toolchain.lake.sha256, + leanchecker: toolchain.leanchecker.sha256, + lexlean, + rustc: Sha256Digest::from_hex(crate::build_identity::RUSTC_SHA256) + .expect("the build script writes a SHA-256"), + } + } + + /// Every tool with its address, in name order. + #[must_use] + pub fn tools(&self) -> [(&'static str, Sha256Digest); 5] { + [ + ("lake", self.lake), + ("lean", self.lean), + ("leanchecker", self.leanchecker), + ("lexlean", self.lexlean), + ("rustc", self.rustc), + ] + } + + /// The canonical JSON form. + #[must_use] + pub fn to_json(&self) -> crate::artifact::canonical_json::Json { + use crate::artifact::canonical_json::Json; + let mut pairs = vec![("spec", Json::Str(ADDRESSES_SPEC.to_owned()))]; + for (tool, address) in self.tools() { + pairs.push((tool, Json::Str(address.to_hex()))); + } + Json::object(pairs) + } + + /// Read the canonical JSON form, refusing anything else: another + /// format, a missing or malformed address, or an unknown field. + /// + /// # Errors + /// + /// What is wrong with `json`. + pub fn from_json(json: &crate::artifact::canonical_json::Json) -> Result { + use crate::artifact::canonical_json::Json; + let Json::Obj(fields) = json else { + return Err("the toolchain addresses are an object".to_owned()); + }; + if fields.get("spec") != Some(&Json::Str(ADDRESSES_SPEC.to_owned())) { + return Err(format!("`spec` must be `{ADDRESSES_SPEC}`")); + } + let address = |tool: &str| match fields.get(tool) { + Some(Json::Str(hex)) => Sha256Digest::from_hex(hex) + .map_err(|reason| format!("the address of `{tool}`: {reason}")), + _ => Err(format!("the address of `{tool}` is required")), + }; + let addresses = Self { + lean: address("lean")?, + lake: address("lake")?, + leanchecker: address("leanchecker")?, + lexlean: address("lexlean")?, + rustc: address("rustc")?, + }; + if let Some(unknown) = fields + .keys() + .find(|key| *key != "spec" && !addresses.tools().iter().any(|(tool, _)| tool == key)) + { + return Err(format!("unknown field `{unknown}`")); + } + Ok(addresses) + } + + /// The first tool whose address differs between `self` and `other`, with + /// both addresses, in name order. + #[must_use] + pub fn first_difference( + &self, + other: &Self, + ) -> Option<(&'static str, Sha256Digest, Sha256Digest)> { + self.tools() + .into_iter() + .zip(other.tools()) + .find(|((_, mine), (_, theirs))| mine != theirs) + .map(|((tool, mine), (_, theirs))| (tool, mine, theirs)) + } +} + fn environment(message: impl Into) -> Diagnostic { Diagnostic::new(code!("LLV7001"), message) } diff --git a/features/suites/verification.feature b/features/suites/verification.feature index 98b176bec9..b222b31d9f 100644 --- a/features/suites/verification.feature +++ b/features/suites/verification.feature @@ -162,9 +162,10 @@ Feature: verification And a verified directory in which a telemetry key is planted is refused @VR-23 @build - Scenario: A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, or a join reading a record of an executable other than its own, is refused. + Scenario: A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, a join reading a record of an executable other than its own, or a part that ran a toolchain other than its own, is refused. Given a committed multi-module example project and a plan of two shards and a join over its import graph When it is verified whole and as the parts of one run Then the join publishes a verified directory byte-identical to the whole verification's, with the same attestation identity And a shard whose plan elaborates a module before its imports, and a join that is handed one shard too few, are refused + And a join reading a part whose toolchain addresses name another lake is refused And a join reading a replay record whose executable digest is not its own leanchecker's is refused diff --git a/model/ids.toml b/model/ids.toml index 7a3ca502f5..129e540642 100644 --- a/model/ids.toml +++ b/model/ids.toml @@ -1349,7 +1349,7 @@ statement = "Performance telemetry is written only to the CI evidence directory id = "VR-23" level = "build" suite = "verification" -statement = "A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, or a join reading a record of an executable other than its own, is refused." +statement = "A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, a join reading a record of an executable other than its own, or a part that ran a toolchain other than its own, is refused." [[id]] id = "CL-01" From 3f0096fdc317d5897719fbbeb6f03d4b66f3a51b Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Thu, 12 Nov 2026 18:48:30 +0000 Subject: [PATCH 2/4] ci: the bootstrap export's custody names executable addresses MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The custody record becomes lexlean/ci-bootstrap-export/2. Beside what `rustc -vV` and `lean --version` report, it names the SHA-256 of the pinned toolchain's lake, lean and leanchecker and the address of the rustc that compiled the exporting binary (SPEC.md §22.11). A leg whose own executables differ refuses the export. A version banner is the same for a rebuilt or patched binary; the bytes are not. RP-14 plants another leanchecker and another rustc. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 8 +++- SPEC.md | 2 +- crates/conformance/src/cases/repository.rs | 30 +++++++++++++++ crates/conformance/src/export_custody.rs | 43 +++++++++++++++++++--- 4 files changed, 75 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 07b3b22200..0f4c7ee698 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,8 +27,12 @@ workspace carried at those versions without a tag. names only the executable it ran, so a part that ran another `lake`, `lexlean`, or `rustc` was accepted before. `VR-23` plants another `lake` address in a shard's output, and on unix a join whose own `lake` differs by - one byte. Published directories, attestations, and `expected/verify` are - unchanged. + one byte. The bootstrap export's custody record becomes + `lexlean/ci-bootstrap-export/2`: it also names the SHA-256 of the pinned + `lake`, `lean`, and `leanchecker` and the address of the `rustc` that built + the exporting binary, and a leg whose own differ refuses the export (`RP-14` + plants another `leanchecker` and another `rustc`). Published directories, + attestations, and `expected/verify` are unchanged. - A one-pass writer for the counts of the bootstrap count cases, #92. Under `LEXLEAN_COUNTS=fast` each of `BS-05` to `BS-08` and `BS-11` to `BS-15` records every count it checks in `target/count-report/` instead of diff --git a/SPEC.md b/SPEC.md index 21cc5ca7b7..32d68ad314 100644 --- a/SPEC.md +++ b/SPEC.md @@ -703,7 +703,7 @@ The `bdd` recipe runs as the parts of one run as well. `.github/bdd-parts.json` Every normative gate job runs on the standard GitHub-hosted runner, `ubuntu-24.04`; no larger runner is part of the gate. The supported-host execution jobs run on their named hosts (§8.3); the Intel and Windows aggregates run on `ubuntu-24.04` and execute no host test. The verification resource profile (§22.11) of a job is named by its matrix row, or on an Atlas job by the job itself, and by nothing else, no repository or workflow variable and no input: the `examples` leg and every Atlas job verify at width 2, with the scheduling hints committed in `.github/verify-hints.json` (derived from a run's telemetry by `cargo xtask ci-hints`) and a memory budget of 12 GiB (12582912 KiB) for the summed expected peaks of the proof processes running at once; the other legs name no profile and run at the default. A profile is a schedule and never a verdict: every elaboration, replay, and audit member still runs on the pinned toolchain, the directory each project publishes is byte for byte the one of width 1 without hints or a budget (§22.11), and the leg and the join still require every example's normalized verification records to equal `expected/verify` (§29.5), so a profile that changed a byte would fail the gate rather than pass a different one. A stale hint reorders processes and never skips one. Local `just vv` verifies UOR Atlas whole at width 1 with no profile. The budget is below the runner's 16 GB, with the 4 GiB swap file of the job as overflow, so two processes run together only when their expected peaks fit, and a process whose expected peak alone passes the budget runs alone. The gate has no path filter, no cache of a build directory, object, verification tree, attestation, or proof result, and no condition that skips a recipe of `just vv` but the plan's, which decides only what a run is, as above; the artifacts the jobs of one run exchange are read by that run alone, and every job starts from a clean checkout with the exact pinned toolchains. Each proof or fan-out job writes its performance telemetry (§22.12) to `ci-evidence/` and uploads that directory as the artifact named after it, `acceptance-gate-evidence-` or `acceptance-gate-evidence-atlas-`: the gate log (`just --timestamp` stamps each recipe's first command on a leg, which is how a recipe is timed without changing it; an Atlas job's log is the recipe's own output), the host's memory and swap sampled every five seconds, the verifier's telemetry, and a summary of the wall time of each recipe and verification stage, the slowest modules and audit members, and the peak resident set and swap of the proof processes. The summary step also prints that summary to the job log and appends it to the step summary, and runs when the job failed, because an uploaded artifact cannot be read from the log; each table lists only its slowest rows and the part appended to the step summary is cut at 512 KiB, half of what GitHub accepts. The initial honesty job retains its recipe timing and summary in the same evidence format, without Lean processes or proof-resource samples. None of it is an input of any recipe, and a timing is never a claim about a semantic property. -The bootstrap production export (§24.7) is made once per run. Every bootstrap case compares the packages of the bootstrap project's export with the committed packages, and a test process that needs the export exports the project, which verifies it whole; each part that ran a bootstrap case ran that verification. In a run that is the whole gate, the job `bootstrap-export` runs `cargo xtask export-bootstrap` from its clean checkout after honesty, at the profile of the `examples` leg, requires every exported package to equal the committed package, and uploads the bundle as the artifact `bootstrap-export` with a custody record (`lexlean/ci-bootstrap-export/1`) that names the repository, the run and its attempt, the commit and its tree, the SHA-256 of `rust-toolchain.toml`, `lean-toolchain` and `Cargo.lock`, what `rustc -vV` and `lean --version` report, the bundle's project-relative path, and the digest of its files. The legs do not wait for that job to start: each leg of such a run starts `.github/scripts/take-bootstrap-export.sh`, which asks nothing until a case of the leg needs the export, then downloads that artifact of its own run, or records that the export job ended without it, and the case fails with that reason. The harness names the directory it waits in through `LEXLEAN_BOOTSTRAP_EXPORT`; before any case reads the bundle it requires the record to name the leg's own repository, run, commit, tree, pins and toolchains, an attempt at most the leg's (a rerun of failed jobs keeps the export that passed, on the same commit), the digest of the files it received, and a path below the project, and it replays the bundle (§24.7). A record that differs in any of these fails the case. Unset or empty, as in `just vv` and in a run that is not the whole gate, each process that needs the export runs it itself. The export crosses no run, as the Atlas parts do not; `RP-14` checks the job, the legs' fetcher, and the record against planted changes. +The bootstrap production export (§24.7) is made once per run. Every bootstrap case compares the packages of the bootstrap project's export with the committed packages, and a test process that needs the export exports the project, which verifies it whole; each part that ran a bootstrap case ran that verification. In a run that is the whole gate, the job `bootstrap-export` runs `cargo xtask export-bootstrap` from its clean checkout after honesty, at the profile of the `examples` leg, requires every exported package to equal the committed package, and uploads the bundle as the artifact `bootstrap-export` with a custody record (`lexlean/ci-bootstrap-export/2`) that names the repository, the run and its attempt, the commit and its tree, the SHA-256 of `rust-toolchain.toml`, `lean-toolchain` and `Cargo.lock`, what `rustc -vV` and `lean --version` report, the SHA-256 of the pinned toolchain's `lake`, `lean` and `leanchecker` executables and the address of the `rustc` that compiled the job's own binary (§22.11), the bundle's project-relative path, and the digest of its files. The legs do not wait for that job to start: each leg of such a run starts `.github/scripts/take-bootstrap-export.sh`, which asks nothing until a case of the leg needs the export, then downloads that artifact of its own run, or records that the export job ended without it, and the case fails with that reason. The harness names the directory it waits in through `LEXLEAN_BOOTSTRAP_EXPORT`; before any case reads the bundle it requires the record to name the leg's own repository, run, commit, tree, pins, toolchains and executable addresses, an attempt at most the leg's (a rerun of failed jobs keeps the export that passed, on the same commit), the digest of the files it received, and a path below the project, and it replays the bundle (§24.7). A record that differs in any of these fails the case. Unset or empty, as in `just vv` and in a run that is not the whole gate, each process that needs the export runs it itself. The export crosses no run, as the Atlas parts do not; `RP-14` checks the job, the legs' fetcher, and the record against planted changes. The Intel host factory is separate from the normative BDD and Atlas partitions above: it transfers only the same-run host test construction and dependency data of §8.3. Construction requires successful honesty; consumers require construction; the aggregate requires all of them and cannot skip an incomplete result. The factory's behavioral tests run in the initial honesty job, before the expensive roots, and exercise inventory completeness, custody and fail-closed result joins without needing an Intel runner. These tests do not substitute for execution of the complete workload on Intel macOS. diff --git a/crates/conformance/src/cases/repository.rs b/crates/conformance/src/cases/repository.rs index 9c227c41dd..e52b0edccf 100644 --- a/crates/conformance/src/cases/repository.rs +++ b/crates/conformance/src/cases/repository.rs @@ -3496,6 +3496,15 @@ fn bootstrap_custody_decisions() { .into_iter() .map(|(name, text)| (name.to_owned(), text.to_owned())) .collect(); + let executables: BTreeMap = [ + ("lake", "5"), + ("lean", "6"), + ("leanchecker", "7"), + ("rustc", "8"), + ] + .into_iter() + .map(|(name, digit)| (name.to_owned(), digit.repeat(64))) + .collect(); let here = Identity { repository: "UOR-Foundation/LexLean".to_owned(), run_id: "100".to_owned(), @@ -3504,6 +3513,7 @@ fn bootstrap_custody_decisions() { tree: "2".repeat(40), pins: pins.clone(), toolchains: toolchains.clone(), + executables: executables.clone(), }; let bundle = ("d".repeat(64), 162); let record = Custody { @@ -3515,6 +3525,7 @@ fn bootstrap_custody_decisions() { tree: here.tree.clone(), pins, toolchains, + executables, bundle_path: ".lexlean/production/bundle".to_owned(), bundle_sha256: bundle.0.clone(), bundle_files: bundle.1, @@ -3550,6 +3561,25 @@ fn bootstrap_custody_decisions() { .insert("rustc".to_owned(), "rustc 1.97.0".to_owned()); }), ), + // An export whose toolchain reports the pinned versions but whose + // executables are other bytes: a changed `leanchecker`, or a binary + // built by another `rustc` (§22.11). + ( + "executables", + Box::new(|record| { + record + .executables + .insert("leanchecker".to_owned(), "9".repeat(64)); + }), + ), + ( + "executables", + Box::new(|record| { + record + .executables + .insert("rustc".to_owned(), "0".repeat(64)); + }), + ), ( "bundle", Box::new(|record| record.bundle_sha256 = "f".repeat(64)), diff --git a/crates/conformance/src/export_custody.rs b/crates/conformance/src/export_custody.rs index 0f6c5fff54..a1bc4d5101 100644 --- a/crates/conformance/src/export_custody.rs +++ b/crates/conformance/src/export_custody.rs @@ -2,8 +2,9 @@ //! (SPEC.md §9.4). The `bootstrap-export` job exports the bootstrap project //! once, requires its packages to equal the committed packages, and uploads //! the bundle with a custody record that binds it to the repository, the -//! run and its attempt, the commit and its tree, the toolchain pins and the -//! toolchains that ran, and the bundle's bytes. A conformance case of the same +//! run and its attempt, the commit and its tree, the toolchain pins, the +//! toolchains that ran and the addresses of their executables, and the +//! bundle's bytes. A conformance case of the same //! run that needs the export takes it from there instead of exporting the //! project again: the harness names the directory it waits in through //! [`VARIABLE`], checks the record against its own run and checkout, replays @@ -26,7 +27,7 @@ use crate::support::{self, P}; pub const VARIABLE: &str = "LEXLEAN_BOOTSTRAP_EXPORT"; /// The custody record's format. -pub const SPEC: &str = "lexlean/ci-bootstrap-export/1"; +pub const SPEC: &str = "lexlean/ci-bootstrap-export/2"; /// The custody record, beside the bundle in the uploaded directory. pub const CUSTODY_FILE: &str = "custody.json"; @@ -49,6 +50,10 @@ pub const ARTIFACT_DIR: &str = "artifact"; /// The repository files whose bytes pin the toolchains and the dependencies. pub const PINS: [&str; 3] = ["rust-toolchain.toml", "lean-toolchain", "Cargo.lock"]; +/// The executables of the pinned Lean toolchain a custody record addresses, +/// beside `rustc` (SPEC.md §22.11). +pub const LEAN_EXECUTABLES: [&str; 3] = ["lake", "lean", "leanchecker"]; + /// The longest a case waits for the run's export: the export job's own /// limit, after which the job has failed and the fetcher says so. const WAIT: Duration = Duration::from_secs(6 * 60 * 60); @@ -73,6 +78,10 @@ pub struct Custody { pub pins: BTreeMap, /// What the toolchains report: `rustc -vV` and `lean --version`. pub toolchains: BTreeMap, + /// The address of each executable: the SHA-256 of `lake`, `lean`, and + /// `leanchecker` of the pinned toolchain, and the address of the `rustc` + /// that compiled the job's own binary (§22.11). + pub executables: BTreeMap, /// The project-relative bundle directory of the export. pub bundle_path: String, /// [`bundle_digest`] of the bundle. @@ -98,6 +107,8 @@ pub struct Identity { pub pins: BTreeMap, /// What the toolchains report. pub toolchains: BTreeMap, + /// The address of each executable. + pub executables: BTreeMap, } fn sha256_hex(bytes: &[u8]) -> String { @@ -130,7 +141,8 @@ fn run(root: &Utf8Path, program: &str, arguments: &[&str]) -> Result Result { let variable = |name: &str| -> Result { std::env::var(name) @@ -169,6 +181,19 @@ pub fn identity(root: &Utf8Path) -> Result { ("lean".to_owned(), run(root, "lean", &["--version"])?), ("rustc".to_owned(), run(root, "rustc", &["-vV"])?), ]); + let bin = lexlean::verify::toolchain::toolchain_root() + .map_err(|error| format!("the pinned toolchain: {}", error.message))? + .join("bin"); + let mut executables = BTreeMap::new(); + for name in LEAN_EXECUTABLES { + let bytes = std::fs::read(bin.join(name).as_std_path()) + .map_err(|error| format!("{bin}/{name}: {error}"))?; + executables.insert(name.to_owned(), sha256_hex(&bytes)); + } + executables.insert( + "rustc".to_owned(), + lexlean::build_identity::RUSTC_SHA256.to_owned(), + ); Ok(Identity { repository, run_id, @@ -177,6 +202,7 @@ pub fn identity(root: &Utf8Path) -> Result { tree: run(root, "git", &["rev-parse", "HEAD^{tree}"])?, pins, toolchains, + executables, }) } @@ -261,6 +287,12 @@ pub fn check(custody: &Custody, current: &Identity, bundle: &(String, u64)) -> R ¤t.toolchains, custody.toolchains == current.toolchains, ); + same( + "executables", + &custody.executables, + ¤t.executables, + custody.executables == current.executables, + ); same( "bundle", &(&custody.bundle_sha256, custody.bundle_files), @@ -330,6 +362,7 @@ pub fn publish(output: &Utf8Path) -> Result { tree: before.tree, pins: before.pins, toolchains: before.toolchains, + executables: before.executables, bundle_path: exported.output.bundle.to_string(), bundle_sha256, bundle_files, @@ -349,7 +382,7 @@ pub fn publish(output: &Utf8Path) -> Result { /// # Errors /// /// When the run has no export, the record names another run, commit, tree, -/// pin, toolchain, or bundle, or the bundle does not replay. +/// pin, toolchain, executable, or bundle, or the bundle does not replay. pub fn take(directory: &Utf8Path) -> Result { std::fs::create_dir_all(directory.as_std_path()) .and_then(|()| std::fs::write(directory.join(WANTED).as_std_path(), b"")) From 041b7ab83d43cad13908028c13e66b1aa9fb6360 Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Thu, 12 Nov 2026 22:58:37 +0000 Subject: [PATCH 3/4] lexlean: build_identity is an internal module (CL-13) Co-Authored-By: Claude Opus 5.5 --- crates/lexlean/src/lib.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/lexlean/src/lib.rs b/crates/lexlean/src/lib.rs index 0edc6c3780..2996ef667b 100644 --- a/crates/lexlean/src/lib.rs +++ b/crates/lexlean/src/lib.rs @@ -109,6 +109,7 @@ pub mod embedded { /// The compiler that built this crate, as the build script found it /// (SPEC.md §22.11): what `rustc -vV` reported and the address of its /// executable and driver library. It is no input of any compiler semantics. +#[doc(hidden)] pub mod build_identity { include!(concat!(env!("OUT_DIR"), "/rustc_identity.rs")); } From 65bfb484c093115fb969139269993d5592cb9ae1 Mon Sep 17 00:00:00 2001 From: humuhumu33 <68450279+humuhumu33@users.noreply.github.com> Date: Sat, 14 Nov 2026 01:06:33 +0000 Subject: [PATCH 4/4] model-write: regenerate CONFORMANCE.md for VR-23's new statement --- CONFORMANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index a1d178c889..47cd55c2b7 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -678,7 +678,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `VR-20` | `build` | The verifier process width is an operational resource profile selected only by the environment, defaulting to 1 and refusing any malformed value with LLV7017, and it enters no lock, identity, or verified artifact. | | `VR-21` | `build` | A multi-module project whose imports form nontrivial frontiers verifies at width 1 and at a larger width to byte-identical verified directories and the same attestation identity, observed axiom sets, and policy rows, and no completion order orders a record or changes which failure is reported. | | `VR-22` | `build` | Performance telemetry is written only to the CI evidence directory the environment names, a verification with telemetry on publishes byte-identical artifacts to one without, and telemetry planted in a verified artifact fails the gate. | -| `VR-23` | `build` | A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, or a join reading a record of an executable other than its own, is refused. | +| `VR-23` | `build` | A project verified as the parts of one run, each shard running the processes its plan names with the oleans of the shards it needs and the join reading every shard's records, publishes a verified directory byte-identical to the whole verification's with the same attestation identity, and a shard whose plan leaves a module without its imports, a join missing a record, a join reading a record of an executable other than its own, or a part that ran a toolchain other than its own, is refused. | ## Cited authorities