From 68af1055ad74e3c888cb19356829af2f015ba731 Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Thu, 12 Nov 2026 01:02:25 +0000 Subject: [PATCH] =?UTF-8?q?ci:=20fail=20stale=20generated=20files=20in=20h?= =?UTF-8?q?onesty=20(`cargo=20xtask=20check-freshness`,=20SPEC.md=20=C2=A7?= =?UTF-8?q?9.4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Of 248 ci runs this month 84 failed, many of them full gates on trees whose generated files were not regenerated; they failed hours in. The new task checks, without Lean and in about 3 s, that every input a committed build oracle (expected/build/manifest.json) records still has its digest and length, that every such project's lock passes `lexlean lock --check`, that every oracle and committed export package names this compiler's semantics, and that every committed export package names its project's committed semantic ID. Honesty runs it before its recipes, so a stale ready pull request fails before the macOS and Linux fan-out starts. It replaces no check of the gate. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 5 + CHANGELOG.md | 4 + SPEC.md | 2 +- crates/conformance/src/cases/repository.rs | 9 + crates/conformance/src/freshness.rs | 318 +++++++++++++++++++++ crates/conformance/src/lib.rs | 1 + xtask/Cargo.toml | 2 +- xtask/src/main.rs | 15 + 8 files changed, 354 insertions(+), 2 deletions(-) create mode 100644 crates/conformance/src/freshness.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 812c99c315..6c1833fbd3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -151,6 +151,11 @@ jobs: env: PYTHONDONTWRITEBYTECODE: "1" run: python3 -m unittest discover -s .github/scripts -p 'test_intel_host_tests.py' + # Stale generated files fail here in minutes, before the expensive + # roots: every input a committed build oracle records, every lock, and + # every committed export package must be current (§9.4). No Lean. + - name: Check that the committed generated files are current + run: cargo xtask check-freshness - name: Run the honesty part of the acceptance gate shell: bash run: | diff --git a/CHANGELOG.md b/CHANGELOG.md index 17a425bd30..f9347aa72f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,10 @@ workspace carried at those versions without a tag. ## Unreleased +- `cargo xtask check-freshness` (SPEC.md §9.4), run in the honesty job before the expensive roots: + every input a committed build oracle records still hashes to its recorded digest, every lock + passes `lexlean lock --check`, and every committed export package names this compiler's + semantics and its project's committed semantic ID. Stale generated files fail in minutes. - `rustix` 1.1.6 (RUSTSEC-2026-0486: the extended-attribute functions marked unwritten bytes as initialized for a zero-length buffer), so `cargo deny` passes; `errno` and `tempfile` resolve `windows-sys` 0.59, the version `fs4` 0.13 and `rustix` 1.1.6 both accept. diff --git a/SPEC.md b/SPEC.md index 6e1c370fbe..791ed065cc 100644 --- a/SPEC.md +++ b/SPEC.md @@ -703,7 +703,7 @@ Every normative gate job runs on the standard GitHub-hosted runner, `ubuntu-24.0 The Intel host factory is separate from the normative BDD and Atlas partitions above: it transfers only the same-run host test construction and dependency data of §8.3. Construction requires successful honesty; consumers require construction; the aggregate requires all of them and cannot skip an incomplete result. The factory's behavioral tests run in the initial honesty job, before the expensive roots, and exercise inventory completeness, custody and fail-closed result joins without needing an Intel runner. These tests do not substitute for execution of the complete workload on Intel macOS. -`ci.yml` MUST expose the initial `honesty` job (retaining the previous check context) separately so claim drift is distinguishable from code failure. It runs the existing format, model and specification-link recipes and the complete BDD meta-gate target, without Lean. The remaining rows, supported hosts and Atlas prefix MUST require its success; these recipes and meta tests run once in the normative gate. Both the plan's previous-run evidence and release evidence MUST require the honesty row, including its successful conclusion; missing, failed or skipped honesty never establishes full acceptance. The separate `honesty.yml` workflow is subsumed by this same-run job. +`ci.yml` MUST expose the initial `honesty` job (retaining the previous check context) separately so claim drift is distinguishable from code failure. It runs the existing format, model and specification-link recipes and the complete BDD meta-gate target, without Lean. Before them it runs `cargo xtask check-freshness`, also without Lean: every input a committed build oracle (`expected/build/manifest.json`) records still has its recorded digest and length, every such project's lock passes `lexlean lock --check`, every oracle and every committed package under `tests/production-export/` names this compiler's semantics, and every such package names its project's committed semantic ID. A tree committed without regenerating what its change moves fails there in minutes, before the expensive roots; the check adds a precondition and replaces no check of the gate, which still compares every generated byte. The remaining rows, supported hosts and Atlas prefix MUST require its success; these recipes and meta tests run once in the normative gate. Both the plan's previous-run evidence and release evidence MUST require the honesty row, including its successful conclusion; missing, failed or skipped honesty never establishes full acceptance. The separate `honesty.yml` workflow is subsumed by this same-run job. `reproducibility.yml` MUST perform two builds in distinct absolute directories and compare every platform-independent artifact byte-for-byte after path normalization. diff --git a/crates/conformance/src/cases/repository.rs b/crates/conformance/src/cases/repository.rs index 09a8125bb8..ee821a6a0d 100644 --- a/crates/conformance/src/cases/repository.rs +++ b/crates/conformance/src/cases/repository.rs @@ -2244,6 +2244,15 @@ pub(crate) fn run(id: &str) { ); // The honesty and reproducibility workflows stay separate. let honesty = job("honesty"); + // §9.4: stale generated files fail in honesty, before the gate + // command and every expensive root. + let fresh = honesty + .find(" run: cargo xtask check-freshness\n") + .expect("§9.4: honesty checks that the committed generated files are current"); + assert!( + fresh < honesty.find("just --timestamp").expect("the gate command"), + "§9.4: the freshness check runs before the honesty recipes" + ); assert!( honesty.contains("LEXLEAN_BDD_PART: ${{ matrix.bdd_part }}") && honesty.contains("fromJSON(needs.plan.outputs.honesty)") diff --git a/crates/conformance/src/freshness.rs b/crates/conformance/src/freshness.rs new file mode 100644 index 0000000000..828e6e4f5f --- /dev/null +++ b/crates/conformance/src/freshness.rs @@ -0,0 +1,318 @@ +//! Whether the committed generated files are current (SPEC.md §9.4), checked +//! by `cargo xtask check-freshness` in the initial honesty job, without Lean +//! and in seconds. A change to a source, a project file, a lock, or the +//! compiler that was committed without regenerating what depends on it fails +//! here, before the expensive roots of the gate start, instead of hours into +//! them. It adds a precondition to the gate and replaces none of its checks: +//! `golden`, `examples`, and the bootstrap and production-export cases still +//! compare every generated byte. +//! +//! What it reads: each committed build oracle `expected/build/manifest.json` +//! records the digest and length of every input its build read and the +//! compiler semantics that built it; each committed package of +//! `tests/production-export/` records, in `provenance.json`, the compiler +//! semantics and the semantic ID of the build it was exported from; each +//! project's `lexlean.lock` is what `lexlean lock --check` accepts. + +use std::collections::BTreeMap; + +use camino::{Utf8Path, Utf8PathBuf}; +use lexlean::{Engine, LockRequest}; +use serde_json::Value; +use sha2::{Digest, Sha256}; + +/// The projects of `tests/production-export/`, each with its directory. +pub const EXPORTED: [(&str, &str); 5] = [ + ("bootstrap", "bootstrap"), + ("helpdesk", "examples/helpdesk"), + ("production", "examples/production"), + ("production-coverage", "examples/production-coverage"), + ("stdlib", "stdlib"), +]; + +/// What the check read. +#[derive(Debug, Default, Clone, PartialEq, Eq)] +pub struct Summary { + /// Projects with a committed build oracle. + pub projects: usize, + /// Recorded inputs hashed. + pub inputs: usize, + /// Committed export packages read. + pub packages: usize, +} + +fn sha256_hex(bytes: &[u8]) -> String { + Sha256::digest(bytes) + .iter() + .map(|byte| format!("{byte:02x}")) + .collect() +} + +fn read_json(path: &Utf8Path) -> Result { + let bytes = std::fs::read(path.as_std_path()).map_err(|error| format!("{path}: {error}"))?; + serde_json::from_slice(&bytes).map_err(|error| format!("{path}: {error}")) +} + +/// Every project directory under `root` with a committed build oracle. +#[must_use] +pub fn oracle_projects(root: &Utf8Path) -> Vec { + let mut found = Vec::new(); + for parent in [root.to_owned(), root.join("examples")] { + let Ok(entries) = std::fs::read_dir(parent.as_std_path()) else { + continue; + }; + for entry in entries.flatten() { + let Ok(path) = Utf8PathBuf::from_path_buf(entry.path()) else { + continue; + }; + if path.join("expected/build/manifest.json").is_file() { + found.push(path); + } + } + } + found.sort(); + found +} + +/// The source packages a project names, by package, as directories. +fn source_packages( + root: &Utf8Path, + project: &Utf8Path, +) -> Result, String> { + let file = project.join("lexlean.toml"); + let text = + std::fs::read_to_string(file.as_std_path()).map_err(|error| format!("{file}: {error}"))?; + let config: toml::Value = toml::from_str(&text).map_err(|error| format!("{file}: {error}"))?; + let mut packages = BTreeMap::new(); + for row in config + .get("source_package") + .and_then(toml::Value::as_array) + .into_iter() + .flatten() + { + if let (Some(name), Some(path)) = ( + row.get("package").and_then(toml::Value::as_str), + row.get("path").and_then(toml::Value::as_str), + ) { + packages.insert(name.to_owned(), root.join(path)); + } + } + Ok(packages) +} + +/// Whether one project's committed build oracle and lock are current. Returns +/// the oracle's semantic ID and the number of inputs hashed. +/// +/// # Errors +/// +/// Every stale or unreadable input, one per line. +pub fn check_project(root: &Utf8Path, project: &Utf8Path) -> Result<(String, usize), String> { + let mut unmet = Vec::new(); + let manifest = read_json(&project.join("expected/build/manifest.json"))?; + let name = project.strip_prefix(root).unwrap_or(project); + let language = manifest["language"].as_str().unwrap_or_default(); + let compiler = lexlean::compiler_semantics_id_for(language).to_hex(); + if manifest["compiler"]["semantics_id"].as_str() != Some(compiler.as_str()) { + unmet.push(format!( + "{name}: expected/build was built by compiler semantics {}, this compiler is {compiler}", + manifest["compiler"]["semantics_id"] + )); + } + let packages = source_packages(root, project)?; + let mut hashed = 0; + for input in manifest["inputs"].as_array().into_iter().flatten() { + let path = input["path"].as_str().unwrap_or_default(); + if input["kind"] == "lexicon" { + // A lexicon's digest is its package tree's: the lock records a + // local package's (`lock --check` below), and the compiler's + // semantics ID, compared above, covers an embedded one. + continue; + } + let file = match path.split_once(':') { + Some((package, relative)) => match packages.get(package) { + Some(directory) => directory.join(relative), + None => { + unmet.push(format!( + "{name}: input `{path}` names no source package of lexlean.toml" + )); + continue; + } + }, + None => project.join(path), + }; + hashed += 1; + match std::fs::read(file.as_std_path()) { + Ok(bytes) => { + if input["sha256"].as_str() != Some(sha256_hex(&bytes).as_str()) + || input["byte_length"].as_u64() != Some(bytes.len() as u64) + { + unmet.push(format!( + "{name}: `{path}` changed since expected/build was generated" + )); + } + } + Err(error) => unmet.push(format!("{name}: `{path}`: {error}")), + } + } + let lock = Engine::load(&project.join("lexlean.toml")).and_then(|engine| { + engine.lock(LockRequest { + check_only: true, + allow_network: false, + }) + }); + if let Err(error) = lock { + unmet.push(format!( + "{name}: `lexlean lock --check` fails: {:?}", + error + .diagnostics + .iter() + .map(|diagnostic| diagnostic.message.clone()) + .collect::>() + )); + } + if unmet.is_empty() { + Ok(( + manifest["semantic_id"] + .as_str() + .unwrap_or_default() + .to_owned(), + hashed, + )) + } else { + Err(unmet.join("\n")) + } +} + +/// Whether every committed generated file under `root` is current. +/// +/// # Errors +/// +/// Every stale file, one per line, with the regeneration that renews it. +pub fn check(root: &Utf8Path) -> Result { + let mut unmet = Vec::new(); + let mut summary = Summary::default(); + let mut semantic = BTreeMap::new(); + for project in oracle_projects(root) { + summary.projects += 1; + match check_project(root, &project) { + Ok((id, hashed)) => { + summary.inputs += hashed; + semantic.insert(project, id); + } + Err(reason) => unmet.push(reason), + } + } + let compiler = lexlean::compiler_semantics_id_for(lexlean::LANGUAGE_1_2).to_hex(); + for (exported, directory) in EXPORTED { + let packages = root + .join("tests/production-export") + .join(exported) + .join("packages"); + let expected = semantic.get(&root.join(directory)); + for entry in walkdir::WalkDir::new(packages.as_std_path()) + .into_iter() + .flatten() + { + if entry.file_name() != "provenance.json" { + continue; + } + let Ok(path) = Utf8PathBuf::from_path_buf(entry.into_path()) else { + continue; + }; + summary.packages += 1; + let relative = path.strip_prefix(root).unwrap_or(&path).to_owned(); + let provenance = match read_json(&path) { + Ok(value) => value, + Err(reason) => { + unmet.push(reason); + continue; + } + }; + if provenance["compiler_semantics"].as_str() != Some(compiler.as_str()) { + unmet.push(format!( + "{relative}: exported by compiler semantics {}, this compiler is {compiler}", + provenance["compiler_semantics"] + )); + } + let sources: Vec<&str> = provenance["sources"] + .as_array() + .into_iter() + .flatten() + .filter_map(Value::as_str) + .collect(); + if expected.is_none_or(|id| sources != [id.as_str()]) { + unmet.push(format!( + "{relative}: exported from {sources:?}, {directory}/expected/build is {}", + expected.map_or("missing", String::as_str) + )); + } + } + } + if unmet.is_empty() { + Ok(summary) + } else { + unmet.sort(); + Err(format!( + "{} committed generated file(s) are stale; regenerate them (`regenerate.yml`, or the `-write` recipes):\n{}", + unmet.len(), + unmet.join("\n") + )) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn copy(from: &Utf8Path, to: &Utf8Path) { + for entry in walkdir::WalkDir::new(from.as_std_path()) + .into_iter() + .flatten() + { + let relative = entry + .path() + .strip_prefix(from.as_std_path()) + .expect("below"); + if relative.starts_with(".lexlean") { + continue; + } + let target = to.as_std_path().join(relative); + if entry.file_type().is_dir() { + std::fs::create_dir_all(&target).expect("dir"); + } else if entry.file_type().is_file() { + std::fs::copy(entry.path(), &target).expect("copy"); + } + } + } + + #[test] + fn a_committed_oracle_is_current_and_a_changed_input_is_stale() { + let root = crate::support::repo_root(); + let temp = tempfile::tempdir().expect("tempdir"); + let copy_root = Utf8PathBuf::from_path_buf(temp.path().to_owned()).expect("utf8"); + let project = copy_root.join("examples/nat-add-zero"); + copy(&root.join("examples/nat-add-zero"), &project); + let (semantic, hashed) = + check_project(©_root, &project).expect("the committed oracle is current"); + assert!(!semantic.is_empty() && hashed >= 2, "{semantic} {hashed}"); + let source = walkdir::WalkDir::new(project.join("src").as_std_path()) + .into_iter() + .flatten() + .find(|entry| entry.path().extension().is_some_and(|ext| ext == "tex")) + .expect("a source") + .into_path(); + let mut text = std::fs::read_to_string(&source).expect("source"); + text.push('\n'); + std::fs::write(&source, text).expect("write"); + let stale = check_project(©_root, &project).expect_err("a changed source is stale"); + assert!( + stale.contains("changed since expected/build was generated"), + "{stale}" + ); + std::fs::remove_file(project.join("expected/build/manifest.json")).expect("remove"); + assert!( + check_project(©_root, &project).is_err(), + "a missing oracle fails" + ); + } +} diff --git a/crates/conformance/src/lib.rs b/crates/conformance/src/lib.rs index 3a974e2dcb..302d753f7c 100644 --- a/crates/conformance/src/lib.rs +++ b/crates/conformance/src/lib.rs @@ -16,6 +16,7 @@ pub mod certification_library; pub mod differential; pub mod evidence; pub mod fixtures; +pub mod freshness; pub mod gnaf; pub mod inventory; pub mod lcnf_adapter; diff --git a/xtask/Cargo.toml b/xtask/Cargo.toml index f6090f3784..52f7929b01 100644 --- a/xtask/Cargo.toml +++ b/xtask/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "xtask" -description = "Repository gates: validate-model, validate-spec-links, verify-examples, check-golden, check-reproducibility, check-fixtures, check-calculus, check-production-export, check-package, check-test-inventory, bdd, ci-summary, ci-hints, ci-shards, release-artifacts, release-check, validate." +description = "Repository gates: validate-model, validate-spec-links, verify-examples, check-golden, check-reproducibility, check-fixtures, check-calculus, check-production-export, check-freshness, check-package, check-test-inventory, bdd, ci-summary, ci-hints, ci-shards, release-artifacts, release-check, validate." version.workspace = true edition.workspace = true rust-version.workspace = true diff --git a/xtask/src/main.rs b/xtask/src/main.rs index 9887022ba5..982df70867 100644 --- a/xtask/src/main.rs +++ b/xtask/src/main.rs @@ -35,6 +35,7 @@ fn main() -> ExitCode { "check-fixtures" => check_fixtures(&root, write), "check-calculus" => check_calculus(&root, write), "check-production-export" => check_production_export(write), + "check-freshness" => check_freshness(&root), "check-package" => check_package(&root), "check-test-inventory" => ci::check_test_inventory(&root), "bdd" => ci::bdd(&root), @@ -60,6 +61,8 @@ fn main() -> ExitCode { their generator\n\ check-production-export §24.7: the consumer's exported packages equal the committed\n\ packages under tests/production-export\n\ + check-freshness §9.4: every committed build oracle's recorded inputs, every lock, + and every committed export package are current (no Lean) check-package §30.3, RP-12: package lexlean crate, verify offline build & identity\n\ check-test-inventory RP-13: the `test` and `bdd` recipes partition the workspace tests,\n\ and the parts of .github/bdd-parts.json partition `bdd`\n\ @@ -124,6 +127,18 @@ fn check_production_export(write: bool) -> Result<(), Fail> { Ok(()) } +/// §9.4: the committed generated files are current, checked in honesty +/// before the expensive roots (`repo_conformance::freshness`). +fn check_freshness(root: &Path) -> Result<(), Fail> { + let root = camino::Utf8Path::from_path(root).ok_or("the repository path is UTF-8")?; + let summary = repo_conformance::freshness::check(root)?; + println!( + "check-freshness: {} build oracles ({} recorded inputs), their locks, and {} committed export packages are current (§9.4)", + summary.projects, summary.inputs, summary.packages + ); + Ok(()) +} + fn example_dirs(root: &Path) -> Result, Fail> { let mut dirs = Vec::new(); for entry in std::fs::read_dir(root.join("examples"))? {