From 82241baa1d9477065897395d4d5c5e13f984db02 Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Thu, 12 Nov 2026 22:56:49 +0000 Subject: [PATCH 1/3] artifact: one UOR address scheme for every content identity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A UOR address is `sha256:` and the SHA-256 of an artifact's canonical bytes (SPEC.md §21.10), the label form of the UOR Foundation's uor-addr. Every identity LexLean records is already that digest, so no recorded byte moves. content_id writes each recipe once over a sink, a SHA-256 state or a buffer (Framed, *_frames), so an identity and its canonical bytes cannot differ. artifact::address has one function per artifact kind returning its canonical bytes, Address with the sha256: label and a strict parse, and identities() for a checked project, which Engine::identities exposes. A unit test holds LexLean's canonical JSON of {"foo":"bar"} to uor-addr's published address. AR-15 recomputes with its own SHA-256 every identity the 26 committed build manifests, their locks and outputs, and the 164 committed Rust packages record (2,100 in all) and requires each to equal the recorded one; a copy whose source changed under the committed manifest has its source and build IDs named. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 11 + CONFORMANCE.md | 1 + README.md | 4 +- SPEC.md | 23 +- crates/conformance/src/cases/artifacts.rs | 324 ++++++++++++++++- crates/conformance/src/cases/repository.rs | 2 +- crates/conformance/src/support.rs | 2 +- crates/conformance/tests/conformance.rs | 5 + crates/lexlean/src/api.rs | 15 + crates/lexlean/src/artifact/address.rs | 396 +++++++++++++++++++++ crates/lexlean/src/artifact/content_id.rs | 248 +++++++++---- crates/lexlean/src/artifact/mod.rs | 1 + crates/lexlean/src/lib.rs | 51 ++- features/suites/artifacts.feature | 7 + model/ids.toml | 6 + xtask/src/spec_links.rs | 4 +- 16 files changed, 1021 insertions(+), 79 deletions(-) create mode 100644 crates/lexlean/src/artifact/address.rs diff --git a/CHANGELOG.md b/CHANGELOG.md index 20399e3a34..2360e5ea7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,17 @@ workspace carried at those versions without a tag. ## Unreleased +- One address scheme for every content identity (SPEC.md §21.10, `AR-15`). + A UOR address is `sha256:` and the SHA-256 of an artifact's canonical + bytes, the label form of the UOR Foundation's `uor-addr`; every identity + LexLean records is already that digest, so no recorded byte moves. + `lexlean::artifact::address` has one function per artifact kind returning + its canonical bytes, written by the same recipe as the identity + (`content_id::*_frames` over a hash or a buffer), and + `Engine::identities` gives a checked project's. `AR-15` recomputes with + its own SHA-256 every identity the committed build manifests, locks, + build outputs, and Rust packages record, and requires each to equal the + recorded one. The register is 494 IDs. - A one-pass writer for the counts of the bootstrap count cases, #92. Under `LEXLEAN_COUNTS=fast` each of `BS-05` to `BS-08` and `BS-11` to `BS-15` records every count it checks in `target/count-report/` instead of diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 5e0010e5e9..6f3c483abd 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -37,6 +37,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `AR-12` | `build` | Concurrent and failed builds preserve atomic content-addressed artifacts and never overwrite unexplained bytes. | | `AR-13` | `build` | Two clean builds in different absolute directories produce byte-identical platform-independent artifacts. | | `AR-14` | `build` | Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations. | +| `AR-15` | `build` | Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. | ## authoring diff --git a/README.md b/README.md index d93a9f1395..38c493e9c3 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ just vv # the complete normative acceptance gate (SPEC.md §9.2) just release # vv, then the §30 release criterion; refused until 1.0.0 ``` -All 493 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. +All 494 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. The differential against the independent generator `lean4-prod` (SPEC.md §17.19, `OR-01`..`OR-08`) acquires it at its pinned commit: from the checkout `LEXLEAN_ORACLE_DIR` names, or by fetching the commit with `git` into the Cargo target directory (`LEXLEAN_ORACLE_CACHE` overrides), and builds it with the pinned Lean and Rust toolchains. It is validation evidence only and no input to any identity. An oracle that cannot be acquired, is not the pinned files, or fails is unavailable: that fails the gate on Linux x86-64 and is printed as `NOT VALIDATED` elsewhere, and is never reported as agreement. @@ -178,7 +178,7 @@ Every row is validated by `just vv`; the IDs link the claim to its register row, | The structured proof language with pinned Lean lowerings, including closed linear arithmetic, integer order, and the fold laws | `PF-01`..`PF-23` | `build` | | Prose-free deterministic generated Lean with complete token traceability | `LN-01`..`LN-12` | `build` | | Canonical LaTeX regeneration and the optional hash-checked PDF provider | `TX-01`..`TX-12` | `build` | -| Canonical diagnostics, source maps, coverage, manifests, and reproducible builds | `AR-01`..`AR-14` | `build` | +| Canonical diagnostics, source maps, coverage, manifests, reproducible builds, and UOR addresses | `AR-01`..`AR-15` | `build` | | Sixteen-stage verification with leanchecker replay and exact axiom audit | `VR-01`..`VR-23` | `build` | | The exact CLI contract, the stable seven-method Rust `Engine` API, the host-driven end of the children the library started, the `production` subcommands, and the `migrate` command | `CL-01`..`CL-24` | `build` | | Filesystem confinement, no shell, no hidden network, closed failure model | `SE-01`..`SE-12` | `build` | diff --git a/SPEC.md b/SPEC.md index b7e6bd3a82..f9ab8f5142 100644 --- a/SPEC.md +++ b/SPEC.md @@ -9819,6 +9819,26 @@ or 1.1 build publishes no declarations file, so their output is unchanged. --- + +### 21.10 UOR addresses + +A UOR address is the label `sha256:` followed by the 64 lowercase hexadecimal digits of the SHA-256 of an artifact's canonical bytes: 71 bytes, the form of the UOR Foundation's `uor-addr` labels. Every content identity this specification defines is the SHA-256 of the canonical bytes of one artifact kind, so each recorded identity is its artifact's UOR address with the label's prefix removed. Recorded identities stay bare hexadecimal; the address is the same digest. + +| Kind | Canonical bytes | Recorded as | +| --- | --- | --- | +| file | the file's bytes as stored | lock `workspace_file` and package digests, manifest `inputs` and `outputs`, `.olean`, executable, process-stream, and PDF digests | +| tree | `lexlean-tree-v1`, NUL, then `frame(path, bytes)` per row in path order (§11.5) | tree digests of packages | +| compiler semantics | the tree of the language's embedded files (§21.2) | `compiler_semantics`, `semantics_id` | +| project configuration | the canonical `lexlean.toml` (§10.1) | `project_config_sha256` | +| source | the §21.3 recipe: `lexlean-source-v1`, NUL, then its frames | `source_id` | +| semantic | the §21.4 recipe | `semantic_id` | +| build | the §21.5 recipe | `build_id` | +| attestation | the §22.9 recipe | `attestation_id` | +| PDF recipe | the §19.8 recipe | `recipe_id` | +| Rust program | the program's canonical JSON with one final LF (§21.7) | a package's `program` | + +A framed kind's bytes begin with its own domain and a NUL, so two kinds never share an address by their framing (UOR-GNAF draft.2 §14.3 requires typed domains for addresses); a file kind is its bytes, whose kind the field that records it names. The compiler writes each framed kind's bytes and its identity with one recipe, into a buffer or into the hash (`artifact::address`, `artifact::content_id`), so the two cannot differ. `AR-15` recomputes, with its own SHA-256, the address of every identity that the committed build manifests, locks, build outputs, and Rust packages record, and requires each to equal the recorded identity. + ## 22. Verification ### 22.1 Verification stages @@ -11789,6 +11809,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `AR-12` | `artifacts` | Concurrent and failed builds preserve atomic content-addressed artifacts and never overwrite unexplained bytes. | §21.8 | | `AR-13` | `artifacts` | Two clean builds in different absolute directories produce byte-identical platform-independent artifacts. | §28.4 | | `AR-14` | `artifacts` | Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations. | §21, §22 | +| `AR-15` | `artifacts` | Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. | §21.10 | | `VR-01` | `verification` | Verification runs every specified stage in order and exposes no stage-suppression option. | §22.1 | | `VR-02` | `verification` | Lean, Lake, and leanchecker versions and executable hashes are checked and recorded before use. | §22.2 | | `VR-03` | `verification` | Lean execution uses the pinned Lake environment and never updates or fetches workspace dependencies. | §22.2 | @@ -12083,7 +12104,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `LM-09` | `language-models` | No prompt is code: no member of the closed language maps `string` or `bytes` to a function, a term, or source — every operation of the primitive registry that takes a `string` or `bytes` operand is classified by the audit with a data result, the registry's schema enumeration and the production registry's primitive rows equal one another and equal that classified set, and the text declarations of `lexlean/semantic-module/2` state no `string`-typed member, so a prompt reaches a model only as the tokens `encodeText` makes of it; a planted operation or text member of that shape is caught by the audit with its name. | §17.12 | | `LM-13` | `language-models` | The text laws a tokenizer elaborates through are LexLean source and not authored text: the module that states them is one language-1.2 native core module of the embedded `language/text-1.2` project, the project holds no file of Lean source, no independently authored Lean file anywhere in the release tree declares the `LexLeanText` namespace or lies under its module root, and the one-time migration input is absent; the registry lists only that module, the declarations and their exact axiom policies are read from its native data, which the compiler embeds and hashes into the language-1.2 semantics, and a module that declares a text component imports the generated module rather than splicing any Lean text. | §17.10, §17.12 | -**Total required capability IDs:** 493. +**Total required capability IDs:** 494. No row may be downgraded to `some-true` or `open`. Upstream Lean facts are ledger/authority rows, not substitutions for these build behaviors. diff --git a/crates/conformance/src/cases/artifacts.rs b/crates/conformance/src/cases/artifacts.rs index 01eba8d1aa..18956b5f98 100644 --- a/crates/conformance/src/cases/artifacts.rs +++ b/crates/conformance/src/cases/artifacts.rs @@ -1,4 +1,4 @@ -//! The `artifacts` suite: AR-01..AR-14. +//! The `artifacts` suite: AR-01..AR-15. use lexlean::artifact::canonical_json::Json; use lexlean::artifact::content_id::{FramedHasher, Sha256Digest}; @@ -725,6 +725,328 @@ pub(crate) fn run(id: &str) { ); } } + // §21.10, AR-15: every recorded identity is a UOR address. + "AR-15" => recorded_identities_are_addresses(), other => panic!("no artifacts case is wired for {other}"), } } + +/// The lowercase-hex SHA-256 of `bytes`, computed here and not by the +/// compiler, so a recorded identity is held to the hash of the canonical +/// bytes and not to the compiler's own digest of them. +fn sha256_hex(bytes: &[u8]) -> String { + sha2::Sha256::digest(bytes) + .iter() + .map(|byte| format!("{byte:02x}")) + .collect() +} + +/// One comparison of a recorded identity with the address of its artifact's +/// canonical bytes. +struct Addresses { + compared: usize, + unmet: Vec, +} + +impl Addresses { + fn same(&mut self, what: &str, recorded: Option<&str>, canonical: &[u8]) { + self.compared += 1; + let address = sha256_hex(canonical); + match recorded { + Some(recorded) if recorded == address => {} + other => self.unmet.push(format!( + "{what}: recorded {other:?}, address sha256:{address}" + )), + } + } +} + +/// Every committed project with a committed build manifest, relative to the +/// repository root, in path order. +fn committed_manifest_projects() -> Vec { + let root = support::repo_root(); + let mut out = Vec::new(); + for entry in walkdir::WalkDir::new(root.as_std_path()) + .sort_by_file_name() + .into_iter() + .filter_entry(|entry| { + let name = entry.file_name().to_string_lossy(); + !(entry.depth() > 0 && (name.starts_with('.') || name == "target")) + }) + .flatten() + { + if entry.file_name() == "manifest.json" + && entry.path().ends_with("expected/build/manifest.json") + { + let project = entry + .path() + .parent() + .and_then(std::path::Path::parent) + .and_then(std::path::Path::parent) + .expect("a project directory"); + if project.join("lexlean.toml").is_file() { + out.push( + camino::Utf8PathBuf::from_path_buf(project.to_path_buf()) + .expect("a UTF-8 path"), + ); + } + } + } + out +} + +/// Compare every identity a committed project records with the address of +/// the canonical bytes `lexlean::artifact::address` gives for it: the +/// manifest's source, semantic, build, and compiler-semantics IDs, the +/// lock's compiler-semantics ID and workspace files, and every build output +/// committed beside the manifest. A project that commits no lock (its case +/// writes one) is locked in a copy first, and has no lock rows to compare. +fn audit_project(dir: &camino::Utf8Path, into: &mut Addresses) { + use lexlean::{CheckRequest, Engine, LockRequest, Selection}; + let expected = dir.join("expected/build"); + let manifest: serde_json::Value = serde_json::from_slice( + &std::fs::read(expected.join("manifest.json").as_std_path()).expect("the manifest"), + ) + .expect("the manifest parses"); + let text = |value: &serde_json::Value| value.as_str().map(str::to_owned); + // A tooling fixture (§17.20) is rendered as an embedded project, not + // checked as a user project: its build ID is held to the build recipe + // over the source and semantic IDs it records, its compiler-semantics ID + // to its language's tree, and its outputs to their bytes. + if dir.starts_with(support::repo_root().join("tests/tooling")) { + let id = |key: &str| { + lexlean::Sha256Digest::from_hex(manifest[key].as_str().expect("an ID")) + .expect("a digest") + }; + into.same( + &format!("{dir}: build_id"), + text(&manifest["build_id"]).as_deref(), + &lexlean::artifact::address::build(id("source_id"), id("semantic_id")).bytes, + ); + let language = manifest["language"].as_str().expect("a language"); + into.same( + &format!("{dir}: compiler.semantics_id"), + text(&manifest["compiler"]["semantics_id"]).as_deref(), + &lexlean::artifact::address::compiler_semantics(language) + .expect("a supported language") + .bytes, + ); + audit_outputs(dir, &manifest, into); + return; + } + let committed_lock = dir.join("lexlean.lock").as_std_path().is_file(); + let copy = tempfile::tempdir().expect("tempdir"); + let engine = if committed_lock { + Engine::load(&dir.join("lexlean.toml")).expect("the project loads") + } else { + support::copy_tree(dir.as_std_path(), copy.path(), &[".lexlean", "expected"]); + let root = camino::Utf8Path::from_path(copy.path()).expect("a UTF-8 path"); + let engine = Engine::load(&root.join("lexlean.toml")).expect("the project loads"); + engine + .lock(LockRequest { + check_only: false, + allow_network: false, + }) + .unwrap_or_else(|error| panic!("{dir}: the project locks: {error}")); + engine + }; + let identities = engine + .identities(CheckRequest { + selection: Selection::Entrypoints, + }) + .unwrap_or_else(|error| panic!("{dir}: the project checks: {error}")); + for (what, recorded, canonical) in [ + ( + "source_id", + text(&manifest["source_id"]), + &identities.source, + ), + ( + "semantic_id", + text(&manifest["semantic_id"]), + &identities.semantic, + ), + ("build_id", text(&manifest["build_id"]), &identities.build), + ( + "compiler.semantics_id", + text(&manifest["compiler"]["semantics_id"]), + &identities.compiler_semantics, + ), + ] { + into.same( + &format!("{dir}: {what}"), + recorded.as_deref(), + &canonical.bytes, + ); + } + if !committed_lock { + audit_outputs(dir, &manifest, into); + return; + } + let lock: toml::Value = toml::from_str( + &std::fs::read_to_string(dir.join("lexlean.lock").as_std_path()).expect("the lock"), + ) + .expect("the lock parses"); + into.same( + &format!("{dir}: lexlean.lock compiler_semantics"), + lock.get("compiler_semantics").and_then(toml::Value::as_str), + &identities.compiler_semantics.bytes, + ); + for row in lock + .get("workspace_file") + .and_then(toml::Value::as_array) + .into_iter() + .flatten() + { + let path = row["path"].as_str().expect("a workspace file path"); + let bytes = std::fs::read(dir.join(path).as_std_path()).expect("a workspace file"); + into.same( + &format!("{dir}: lexlean.lock workspace_file {path}"), + row.get("sha256").and_then(toml::Value::as_str), + &lexlean::artifact::address::file(&bytes).bytes, + ); + } + audit_outputs(dir, &manifest, into); +} + +/// Compare every build output committed beside a manifest with the address +/// of its bytes, and its recorded length with theirs. +fn audit_outputs(dir: &camino::Utf8Path, manifest: &serde_json::Value, into: &mut Addresses) { + let expected = dir.join("expected/build"); + for output in manifest["outputs"].as_array().expect("outputs") { + let path = output["path"].as_str().expect("an output path"); + let Ok(bytes) = std::fs::read(expected.join(path).as_std_path()) else { + continue; + }; + assert_eq!( + output["byte_length"].as_u64(), + Some(bytes.len() as u64), + "{dir}: output {path}: byte length" + ); + into.same( + &format!("{dir}: output {path}"), + output["sha256"].as_str(), + &lexlean::artifact::address::file(&bytes).bytes, + ); + } +} + +/// Compare every identity a committed Rust package records (§17.23) with the +/// address of its canonical bytes: each file of the package, its program, +/// and the compiler-semantics ID it was exported under. +fn audit_package(dir: &std::path::Path, into: &mut Addresses) { + let read = |name: &str| -> serde_json::Value { + serde_json::from_slice(&std::fs::read(dir.join(name)).expect("a package file")) + .expect("a package file parses") + }; + let provenance = read("provenance.json"); + let package = read("package.json"); + let at = dir.display(); + for (path, recorded) in provenance["files"].as_object().expect("files") { + let bytes = std::fs::read(dir.join(path)).expect("a packaged file"); + into.same( + &format!("{at}: {path}"), + recorded.as_str(), + &lexlean::artifact::address::file(&bytes).bytes, + ); + } + let program = lexlean::artifact::canonical_json::Json::parse( + &serde_json::to_vec(&package["program"]).expect("the program serializes"), + ) + .expect("the program is canonical JSON"); + into.same( + &format!("{at}: program"), + provenance["program"].as_str(), + &lexlean::artifact::address::rust_program(&program).bytes, + ); + let language = lexlean::LANGUAGE_1_2; + into.same( + &format!("{at}: compiler_semantics"), + provenance["compiler_semantics"].as_str(), + &lexlean::artifact::address::compiler_semantics(language) + .expect("language 1.2") + .bytes, + ); +} + +/// §21.10: every content identity a committed artifact records is the UOR +/// address of that artifact's canonical bytes, and a recorded identity that +/// no longer matches its artifact is named. +fn recorded_identities_are_addresses() { + let projects = committed_manifest_projects(); + assert!( + projects.len() >= 26, + "every committed build manifest is audited: {projects:?}" + ); + let mut audit = Addresses { + compared: 0, + unmet: Vec::new(), + }; + for project in &projects { + audit_project(project, &mut audit); + } + let after_projects = audit.compared; + let root = support::repo_root(); + let mut packages = 0; + for entry in walkdir::WalkDir::new(root.join("tests/production-export").as_std_path()) + .sort_by_file_name() + .into_iter() + .flatten() + { + if entry.file_name() == "provenance.json" { + audit_package(entry.path().parent().expect("a package"), &mut audit); + packages += 1; + } + } + assert!( + packages >= 164, + "every committed package is audited: {packages}" + ); + assert!( + audit.unmet.is_empty(), + "{} of {} recorded identities are not the address of their canonical bytes:\n{}", + audit.unmet.len(), + audit.compared, + audit.unmet.join("\n") + ); + eprintln!( + "AR-15: {} identities of {} projects and {packages} packages are the addresses of their canonical bytes", + audit.compared, + projects.len() + ); + assert!( + after_projects > 26 * 4, + "{after_projects} project identities" + ); + + // A recorded identity that no longer names its artifact is refused: a + // copy of an example whose theorem label changed still carries the + // committed manifest, whose source and build IDs name the old source. + let project = P::copy_example("nat-add-zero"); + let committed = root.join("examples/nat-add-zero/expected/build/manifest.json"); + let copied = project.root.join("expected/build/manifest.json"); + std::fs::create_dir_all(copied.parent().expect("a parent").as_std_path()).expect("mkdir"); + std::fs::copy(committed.as_std_path(), copied.as_std_path()).expect("the manifest"); + project.edit( + "src/Main.lex.tex", + "\\begin{theorem}{add-zero}", + "\\begin{theorem}{add-zero-again}", + ); + let mut planted = Addresses { + compared: 0, + unmet: Vec::new(), + }; + audit_project(&project.root, &mut planted); + assert!( + planted + .unmet + .iter() + .any(|line| line.contains(": source_id: recorded")) + && planted + .unmet + .iter() + .any(|line| line.contains(": build_id: recorded")), + "a stale recorded identity is named: {:?}", + planted.unmet + ); +} diff --git a/crates/conformance/src/cases/repository.rs b/crates/conformance/src/cases/repository.rs index 9c227c41dd..91e9093f87 100644 --- a/crates/conformance/src/cases/repository.rs +++ b/crates/conformance/src/cases/repository.rs @@ -898,7 +898,7 @@ pub(crate) fn run(id: &str) { let model = repo_model::Model::load(&root.join("model").into_std_path_buf()) .expect("the model loads"); let table = spec_table(); - assert_eq!(table.len(), 493, "§31 has 493 rows"); + assert_eq!(table.len(), 494, "§31 has 494 rows"); assert_eq!(model.ids.id.len(), table.len(), "register row count"); for ((spec_id, spec_suite, spec_statement), row) in table.iter().zip(model.ids.id.iter()) diff --git a/crates/conformance/src/support.rs b/crates/conformance/src/support.rs index 1809bb2546..06cc6d557a 100644 --- a/crates/conformance/src/support.rs +++ b/crates/conformance/src/support.rs @@ -321,7 +321,7 @@ pub struct P { pub root: Utf8PathBuf, } -fn copy_tree(from: &Path, to: &Path, skip: &[&str]) { +pub(crate) fn copy_tree(from: &Path, to: &Path, skip: &[&str]) { for entry in walkdir::WalkDir::new(from).into_iter().flatten() { let relative = entry.path().strip_prefix(from).expect("under root"); let text = relative.to_string_lossy(); diff --git a/crates/conformance/tests/conformance.rs b/crates/conformance/tests/conformance.rs index 2056b03d10..15e3543e77 100644 --- a/crates/conformance/tests/conformance.rs +++ b/crates/conformance/tests/conformance.rs @@ -1010,6 +1010,11 @@ fn conformance_ar_14() { repo_conformance::cases::run("AR-14"); } +#[test] +fn conformance_ar_15() { + repo_conformance::cases::run("AR-15"); +} + #[test] fn conformance_vr_01() { repo_conformance::cases::run("VR-01"); diff --git a/crates/lexlean/src/api.rs b/crates/lexlean/src/api.rs index 883407efaf..b69607b896 100644 --- a/crates/lexlean/src/api.rs +++ b/crates/lexlean/src/api.rs @@ -935,6 +935,21 @@ impl Engine { self.bound("check", self.check_inner(&request)) } + /// The canonical bytes of the project's content identities (§21.10): + /// its configuration, its language's compiler-semantics tree, and the + /// messages of its source, semantic, and build IDs, whose SHA-256 are the + /// IDs a build records. + pub fn identities( + &self, + request: CheckRequest, + ) -> Result { + let (checked, _lock) = self.checked(&request.selection)?; + Ok(crate::artifact::address::identities( + &self.project.config, + &checked, + )) + } + /// Obtain the stable semantic snapshot (§24.1, Task 3). pub fn snapshot( &self, diff --git a/crates/lexlean/src/artifact/address.rs b/crates/lexlean/src/artifact/address.rs new file mode 100644 index 0000000000..6c951d777c --- /dev/null +++ b/crates/lexlean/src/artifact/address.rs @@ -0,0 +1,396 @@ +//! UOR addresses (SPEC.md §21.10): every content identity LexLean records is +//! the SHA-256 of one artifact's canonical bytes, and its UOR address is that +//! digest written `sha256:<64 lowercase hex digits>`, the form of the UOR +//! Foundation's `uor-addr` labels. +//! +//! Each artifact kind has one function here that returns its canonical +//! bytes, and [`Canonical::address`] is the only address function. The +//! framed kinds are written by the same recipe as the identity +//! (`content_id::*_frames`), into a buffer instead of a hash, so a recorded +//! identity and the address of the canonical bytes cannot differ. The kind +//! is separated inside the hashed bytes: every framed kind begins with its +//! own domain and a NUL, and the untagged kinds are file bytes, which no +//! domain-prefixed message equals by accident of construction. + +use std::fmt; + +use crate::artifact::canonical_json::Json; +use crate::artifact::content_id::{ + attestation_frames, build_frames, pdf_recipe_frames, semantic_frames, source_frames, + tree_frames, Sha256Digest, +}; +use crate::link::CheckedProject; + +/// The hash function of every LexLean address, and the label's prefix. +pub const ALGORITHM: &str = "sha256"; + +/// A UOR address: the SHA-256 of canonical bytes. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub struct Address(pub Sha256Digest); + +impl Address { + /// The address of `canonical` bytes. + #[must_use] + pub fn of(canonical: &[u8]) -> Self { + Self(Sha256Digest::of(canonical)) + } + + /// The digest, which is the identity LexLean records (bare hex). + #[must_use] + pub const fn digest(self) -> Sha256Digest { + self.0 + } + + /// Read the label form, refusing every other spelling: another algorithm, + /// upper-case hex, a missing or extra digit. + /// + /// # Errors + /// + /// What is wrong with `text`. + pub fn parse(text: &str) -> Result { + let hex = text + .strip_prefix(ALGORITHM) + .and_then(|rest| rest.strip_prefix(':')) + .ok_or_else(|| format!("an address begins `{ALGORITHM}:`: {text}"))?; + Sha256Digest::from_hex(hex).map(Self) + } +} + +impl fmt::Display for Address { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{ALGORITHM}:{}", self.0.to_hex()) + } +} + +/// Every kind of artifact LexLean identifies (§21.10). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub enum Kind { + /// A file's bytes as stored: lock workspace files, package files, + /// `.olean` files, executables, process streams, PDFs. + File, + /// The §11.5 tree of `(path, bytes)` rows. + Tree, + /// The compiler-semantics ID: the tree of a language's embedded files. + CompilerSemantics, + /// The canonical `lexlean.toml` (§10.1), `project_config_sha256`. + ProjectConfig, + /// The source ID (§21.3). + Source, + /// The semantic ID (§21.4). + Semantic, + /// The build ID (§21.5). + Build, + /// The attestation ID (§22.9). + Attestation, + /// The PDF recipe ID (§19.8). + PdfRecipe, + /// A Rust package's program (§17.23): its canonical JSON file bytes. + RustProgram, +} + +impl Kind { + /// The domain that begins the kind's canonical bytes, if it has one. + #[must_use] + pub const fn domain(self) -> Option<&'static str> { + match self { + Self::File | Self::ProjectConfig | Self::RustProgram => None, + Self::Tree | Self::CompilerSemantics => Some("lexlean-tree-v1"), + Self::Source => Some("lexlean-source-v1"), + Self::Semantic => Some("lexlean-semantic-v1"), + Self::Build => Some("lexlean-build-v1"), + Self::Attestation => Some("lexlean-attestation-v1"), + Self::PdfRecipe => Some("lexlean-pdf-recipe-v1"), + } + } +} + +/// One artifact's canonical bytes, with their kind. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Canonical { + /// What the bytes are. + pub kind: Kind, + /// The canonical bytes. + pub bytes: Vec, +} + +impl Canonical { + /// The UOR address of these bytes. + #[must_use] + pub fn address(&self) -> Address { + Address::of(&self.bytes) + } +} + +/// A file: its bytes. +#[must_use] +pub fn file(bytes: &[u8]) -> Canonical { + Canonical { + kind: Kind::File, + bytes: bytes.to_vec(), + } +} + +/// A §11.5 tree over `(path, bytes)` rows in strictly sorted path order. +#[must_use] +pub fn tree(files: &[(&str, &[u8])]) -> Canonical { + Canonical { + kind: Kind::Tree, + bytes: tree_frames(Vec::new(), files), + } +} + +/// The compiler-semantics ID of `language`: the tree of the embedded files +/// that language selects (§21.2). `None` for an unsupported language. +#[must_use] +pub fn compiler_semantics(language: &str) -> Option { + let rows = crate::compiler_semantics_files(language)?; + let files: Vec<(&str, &[u8])> = rows + .iter() + .map(|(path, bytes)| (*path, bytes.as_ref())) + .collect(); + Some(Canonical { + kind: Kind::CompilerSemantics, + bytes: tree_frames(Vec::new(), &files), + }) +} + +/// A project's configuration: its canonical `lexlean.toml`. +#[must_use] +pub fn project_config(config: &crate::config::ProjectConfig) -> Canonical { + Canonical { + kind: Kind::ProjectConfig, + bytes: config.canonical_toml().into_bytes(), + } +} + +/// The source ID's message (§21.3). +#[must_use] +pub fn source( + canonical_project_toml: &[u8], + canonical_lock: &[u8], + sources: &[(String, Vec)], +) -> Canonical { + Canonical { + kind: Kind::Source, + bytes: source_frames(Vec::new(), canonical_project_toml, canonical_lock, sources), + } +} + +/// The semantic ID's message (§21.4). +#[must_use] +pub fn semantic( + compiler_semantics: Sha256Digest, + language: &str, + linked_ir_json: &str, + lexicon_closure_json: &str, +) -> Canonical { + Canonical { + kind: Kind::Semantic, + bytes: semantic_frames( + Vec::new(), + compiler_semantics, + language, + linked_ir_json, + lexicon_closure_json, + ), + } +} + +/// The build ID's message (§21.5). +#[must_use] +pub fn build(source: Sha256Digest, semantic: Sha256Digest) -> Canonical { + Canonical { + kind: Kind::Build, + bytes: build_frames(Vec::new(), source, semantic), + } +} + +/// The attestation ID's message (§22.9). +#[must_use] +pub fn attestation(body_without_id_json: &str) -> Canonical { + Canonical { + kind: Kind::Attestation, + bytes: attestation_frames(Vec::new(), body_without_id_json), + } +} + +/// The PDF recipe ID's message (§19.8). +#[must_use] +pub fn pdf_recipe( + tex_sha256: Sha256Digest, + program_sha256: Sha256Digest, + version_stdout_sha256: Sha256Digest, + compile_argv_json: &str, + resources_json: &str, +) -> Canonical { + Canonical { + kind: Kind::PdfRecipe, + bytes: pdf_recipe_frames( + Vec::new(), + tex_sha256, + program_sha256, + version_stdout_sha256, + compile_argv_json, + resources_json, + ), + } +} + +/// A Rust package's program: its canonical JSON with one final LF. +#[must_use] +pub fn rust_program(program: &Json) -> Canonical { + Canonical { + kind: Kind::RustProgram, + bytes: program.to_file_bytes(), + } +} + +/// The content identities of one checked project, as canonical bytes. +#[derive(Debug, Clone)] +pub struct Identities { + /// The canonical `lexlean.toml`. + pub project_config: Canonical, + /// The compiler-semantics tree of the project's language. + pub compiler_semantics: Canonical, + /// The source ID's message. + pub source: Canonical, + /// The semantic ID's message. + pub semantic: Canonical, + /// The build ID's message. + pub build: Canonical, +} + +/// The canonical bytes of every content identity of `checked`, a check of +/// the project configured by `config`. +/// +/// # Panics +/// +/// When `config` names an unsupported language, which configuration refuses +/// before any project is checked. +#[must_use] +pub fn identities(config: &crate::config::ProjectConfig, checked: &CheckedProject) -> Identities { + let project_config = project_config(config); + let compiler_semantics = + compiler_semantics(&config.language).expect("a checked project has a supported language"); + let mut sources: Vec<(String, Vec)> = checked + .modules + .values() + .map(|module| { + ( + module.document.source_path.clone(), + module.normalized.clone().into_bytes(), + ) + }) + .collect(); + sources.sort_by(|a, b| a.0.cmp(&b.0)); + let source = source(&project_config.bytes, &checked.canonical_lock, &sources); + let semantic = semantic( + compiler_semantics.address().digest(), + &config.language, + &checked.linked_json().to_canonical_string(), + &checked + .closure + .closure_json("", &checked.visible_union) + .to_canonical_string(), + ); + let build = build(source.address().digest(), semantic.address().digest()); + Identities { + project_config, + compiler_semantics, + source, + semantic, + build, + } +} + +#[cfg(test)] +mod tests { + use super::{Address, Kind, ALGORITHM}; + use crate::artifact::content_id::{ + attestation_id, build_id, pdf_recipe_id, semantic_id_for, source_id, tree_digest, + Sha256Digest, + }; + + #[test] + fn every_recorded_identity_is_the_address_of_its_canonical_bytes() { + let a = Sha256Digest::of(b"a"); + let b = Sha256Digest::of(b"b"); + let sources = vec![("src/Main.lex.tex".to_owned(), b"body".to_vec())]; + let files: [(&str, &[u8]); 2] = [("a", b"1"), ("b/c", b"2")]; + for (canonical, recorded) in [ + (super::tree(&files), tree_digest(&files)), + ( + super::source(b"toml", b"lock", &sources), + source_id(b"toml", b"lock", &sources), + ), + ( + super::semantic(a, "1.2", "{}", "[]"), + semantic_id_for(a, "1.2", "{}", "[]"), + ), + (super::build(a, b), build_id(a, b)), + (super::attestation("{}"), attestation_id("{}")), + ( + super::pdf_recipe(a, b, a, "[]", "[]"), + pdf_recipe_id(a, b, a, "[]", "[]"), + ), + (super::file(b"bytes"), Sha256Digest::of(b"bytes")), + ] { + assert_eq!( + canonical.address().digest(), + recorded, + "{:?}", + canonical.kind + ); + if let Some(domain) = canonical.kind.domain() { + assert!( + canonical + .bytes + .starts_with(format!("{domain}\0").as_bytes()), + "{:?} begins with its domain", + canonical.kind + ); + } + } + assert_eq!( + super::compiler_semantics(crate::LANGUAGE_1_2) + .expect("a supported language") + .address() + .digest(), + crate::compiler_semantics_id_for(crate::LANGUAGE_1_2) + ); + assert!(super::compiler_semantics("0.9").is_none()); + // The canonical JSON of a small object is the canonical form the UOR + // Foundation's `uor-addr` labels (`tests/byte_identity.rs`), and its + // address is that crate's vector. + let object = crate::artifact::canonical_json::Json::object(vec![( + "foo", + crate::artifact::canonical_json::Json::Str("bar".to_owned()), + )]); + assert_eq!( + Address::of(object.to_canonical_string().as_bytes()).to_string(), + "sha256:7a38bf81f383f69433ad6e900d35b3e2385593f76a7b7ab5d4355b8ba41ee24b" + ); + assert_eq!(Kind::File.domain(), None); + } + + #[test] + fn the_label_is_sha256_and_lowercase_hex() { + let address = Address::of(b""); + let label = address.to_string(); + assert_eq!( + label, + "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + ); + assert_eq!(label.len(), 71); + assert_eq!(Address::parse(&label), Ok(address)); + for refused in [ + label.replacen(ALGORITHM, "blake3", 1), + label.to_uppercase(), + label[..70].to_owned(), + format!("{label}0"), + label.replacen(':', "-", 1), + ] { + assert!(Address::parse(&refused).is_err(), "{refused}"); + } + } +} diff --git a/crates/lexlean/src/artifact/content_id.rs b/crates/lexlean/src/artifact/content_id.rs index b470518847..88ad79e313 100644 --- a/crates/lexlean/src/artifact/content_id.rs +++ b/crates/lexlean/src/artifact/content_id.rs @@ -67,68 +67,126 @@ impl<'de> serde::Deserialize<'de> for Sha256Digest { } } -/// An incremental framed hasher over one labeled recipe (§21.1). -pub struct FramedHasher { - hasher: Sha256, +/// Where a recipe's bytes go: a SHA-256 state, for the identity, or a +/// buffer, for the canonical bytes the identity is the SHA-256 of +/// (`artifact::address`). One recipe written once over a sink gives both, +/// so the two never differ. +pub trait FrameSink { + /// Append bytes. + fn put(&mut self, bytes: &[u8]); +} + +impl FrameSink for Sha256 { + fn put(&mut self, bytes: &[u8]) { + self.update(bytes); + } +} + +impl FrameSink for Vec { + fn put(&mut self, bytes: &[u8]) { + self.extend_from_slice(bytes); + } +} + +/// One labeled recipe (§21.1): the domain, a NUL, then frames. +pub struct Framed { + sink: S, } -impl FramedHasher { - /// Start a recipe with its domain-separation prefix, e.g. +impl Framed { + /// Start a recipe in `sink` with its domain-separation prefix, e.g. /// `"lexlean-source-v1\0"`. #[must_use] - pub fn new(domain: &str) -> Self { - let mut hasher = Sha256::new(); - hasher.update(domain.as_bytes()); - hasher.update([0u8]); - Self { hasher } + pub fn with_sink(mut sink: S, domain: &str) -> Self { + sink.put(domain.as_bytes()); + sink.put(&[0u8]); + Self { sink } } /// Append one `frame(label, bytes)` (§21.1). pub fn frame(&mut self, label: &str, bytes: &[u8]) { let label_bytes = label.as_bytes(); - self.hasher.update( - u32::try_from(label_bytes.len()) + self.sink.put( + &u32::try_from(label_bytes.len()) .expect("label fits u32") .to_be_bytes(), ); - self.hasher.update(label_bytes); - self.hasher.update((bytes.len() as u64).to_be_bytes()); - self.hasher.update(bytes); + self.sink.put(label_bytes); + self.sink.put(&(bytes.len() as u64).to_be_bytes()); + self.sink.put(bytes); + } + + /// The sink, holding everything written. + #[must_use] + pub fn into_sink(self) -> S { + self.sink + } +} + +/// An incremental framed hasher over one labeled recipe (§21.1). +pub type FramedHasher = Framed; + +impl Framed { + /// Start a hashed recipe with its domain-separation prefix. + #[must_use] + pub fn new(domain: &str) -> Self { + Self::with_sink(Sha256::new(), domain) } /// Finish the recipe. #[must_use] pub fn finish(self) -> Sha256Digest { - Sha256Digest(self.hasher.finalize().into()) + Sha256Digest(self.sink.finalize().into()) } } -/// The §11.5 tree digest over `(relative path, bytes)` rows. The caller -/// supplies rows already restricted to participating files; this function -/// enforces bytewise-sorted path order and unique paths. -#[must_use] -pub fn tree_digest(files: &[(&str, &[u8])]) -> Sha256Digest { +impl Framed> { + /// Start a recipe whose bytes are kept, with its domain prefix. + #[must_use] + pub fn bytes(domain: &str) -> Self { + Self::with_sink(Vec::new(), domain) + } +} + +/// The recipe of the §11.5 tree digest: one frame per file, labeled by its +/// path, in the order given. +pub fn tree_frames(sink: S, files: &[(&str, &[u8])]) -> S { debug_assert!( files .windows(2) .all(|pair| pair[0].0.as_bytes() < pair[1].0.as_bytes()), "tree digest input must be strictly sorted by path bytes" ); - let mut hasher = Sha256::new(); - hasher.update(b"lexlean-tree-v1"); - hasher.update([0u8]); + let mut framed = Framed::with_sink(sink, "lexlean-tree-v1"); for (path, bytes) in files { - let path_bytes = path.as_bytes(); - hasher.update( - u32::try_from(path_bytes.len()) - .expect("path fits u32") - .to_be_bytes(), - ); - hasher.update(path_bytes); - hasher.update((bytes.len() as u64).to_be_bytes()); - hasher.update(bytes); + framed.frame(path, bytes); } - Sha256Digest(hasher.finalize().into()) + framed.into_sink() +} + +/// The §11.5 tree digest over `(relative path, bytes)` rows. The caller +/// supplies rows already restricted to participating files; this function +/// enforces bytewise-sorted path order and unique paths. +#[must_use] +pub fn tree_digest(files: &[(&str, &[u8])]) -> Sha256Digest { + Sha256Digest(tree_frames(Sha256::new(), files).finalize().into()) +} + +/// The recipe of the source ID (§21.3). +pub fn source_frames( + sink: S, + canonical_project_toml: &[u8], + canonical_lock: &[u8], + sources: &[(String, Vec)], +) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-source-v1"); + framed.frame("project", canonical_project_toml); + framed.frame("lock", canonical_lock); + for (path, bytes) in sources { + framed.frame("path", path.as_bytes()); + framed.frame("source", bytes); + } + framed.into_sink() } /// The source ID (§21.3). `sources` are `(project-relative path, normalized @@ -139,14 +197,33 @@ pub fn source_id( canonical_lock: &[u8], sources: &[(String, Vec)], ) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-source-v1"); - hasher.frame("project", canonical_project_toml); - hasher.frame("lock", canonical_lock); - for (path, bytes) in sources { - hasher.frame("path", path.as_bytes()); - hasher.frame("source", bytes); - } - hasher.finish() + Sha256Digest( + source_frames( + Sha256::new(), + canonical_project_toml, + canonical_lock, + sources, + ) + .finalize() + .into(), + ) +} + +/// The recipe of the semantic ID (§21.4). +pub fn semantic_frames( + sink: S, + compiler_semantics: Sha256Digest, + language: &str, + linked_ir_json: &str, + lexicon_closure_json: &str, +) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-semantic-v1"); + framed.frame("compiler-semantics", &compiler_semantics.0); + framed.frame("language", language.as_bytes()); + framed.frame("toolchain", crate::LEAN_TOOLCHAIN.as_bytes()); + framed.frame("linked-ir", linked_ir_json.as_bytes()); + framed.frame("lexicon-closure", lexicon_closure_json.as_bytes()); + framed.into_sink() } /// The semantic ID (§21.4). @@ -157,13 +234,17 @@ pub fn semantic_id_for( linked_ir_json: &str, lexicon_closure_json: &str, ) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-semantic-v1"); - hasher.frame("compiler-semantics", &compiler_semantics.0); - hasher.frame("language", language.as_bytes()); - hasher.frame("toolchain", crate::LEAN_TOOLCHAIN.as_bytes()); - hasher.frame("linked-ir", linked_ir_json.as_bytes()); - hasher.frame("lexicon-closure", lexicon_closure_json.as_bytes()); - hasher.finish() + Sha256Digest( + semantic_frames( + Sha256::new(), + compiler_semantics, + language, + linked_ir_json, + lexicon_closure_json, + ) + .finalize() + .into(), + ) } /// The language-1.0 semantic ID recipe retained for API compatibility. @@ -181,22 +262,58 @@ pub fn semantic_id( ) } +/// The recipe of the build ID (§21.5). +pub fn build_frames(sink: S, source: Sha256Digest, semantic: Sha256Digest) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-build-v1"); + framed.frame("source-id", &source.0); + framed.frame("semantic-id", &semantic.0); + framed.into_sink() +} + /// The build ID (§21.5). #[must_use] pub fn build_id(source: Sha256Digest, semantic: Sha256Digest) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-build-v1"); - hasher.frame("source-id", &source.0); - hasher.frame("semantic-id", &semantic.0); - hasher.finish() + Sha256Digest( + build_frames(Sha256::new(), source, semantic) + .finalize() + .into(), + ) +} + +/// The recipe of the attestation ID (§22.9). +pub fn attestation_frames(sink: S, body_without_id_json: &str) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-attestation-v1"); + framed.frame("attestation-body", body_without_id_json.as_bytes()); + framed.into_sink() } /// The attestation ID over the canonical body without its `attestation_id` /// field (§22.9). #[must_use] pub fn attestation_id(body_without_id_json: &str) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-attestation-v1"); - hasher.frame("attestation-body", body_without_id_json.as_bytes()); - hasher.finish() + Sha256Digest( + attestation_frames(Sha256::new(), body_without_id_json) + .finalize() + .into(), + ) +} + +/// The recipe of the PDF recipe ID (§19.8). +pub fn pdf_recipe_frames( + sink: S, + tex_sha256: Sha256Digest, + program_sha256: Sha256Digest, + version_stdout_sha256: Sha256Digest, + compile_argv_json: &str, + resources_json: &str, +) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-pdf-recipe-v1"); + framed.frame("tex", &tex_sha256.0); + framed.frame("program", &program_sha256.0); + framed.frame("version-output", &version_stdout_sha256.0); + framed.frame("argv", compile_argv_json.as_bytes()); + framed.frame("resources", resources_json.as_bytes()); + framed.into_sink() } /// The PDF recipe ID (§19.8). @@ -208,11 +325,16 @@ pub fn pdf_recipe_id( compile_argv_json: &str, resources_json: &str, ) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-pdf-recipe-v1"); - hasher.frame("tex", &tex_sha256.0); - hasher.frame("program", &program_sha256.0); - hasher.frame("version-output", &version_stdout_sha256.0); - hasher.frame("argv", compile_argv_json.as_bytes()); - hasher.frame("resources", resources_json.as_bytes()); - hasher.finish() + Sha256Digest( + pdf_recipe_frames( + Sha256::new(), + tex_sha256, + program_sha256, + version_stdout_sha256, + compile_argv_json, + resources_json, + ) + .finalize() + .into(), + ) } diff --git a/crates/lexlean/src/artifact/mod.rs b/crates/lexlean/src/artifact/mod.rs index 5150aa5b0e..95d7a23c3a 100644 --- a/crates/lexlean/src/artifact/mod.rs +++ b/crates/lexlean/src/artifact/mod.rs @@ -1,6 +1,7 @@ //! Canonical artifacts: JSON form, content identity, source maps, coverage, //! and build manifests (SPEC.md §20, §21). +pub mod address; pub mod canonical_json; pub mod content_id; pub mod declaration_id; diff --git a/crates/lexlean/src/lib.rs b/crates/lexlean/src/lib.rs index dd7664959b..a316fc3d70 100644 --- a/crates/lexlean/src/lib.rs +++ b/crates/lexlean/src/lib.rs @@ -237,8 +237,13 @@ pub fn identity_bytes<'a>(path: &str, bytes: &'a [u8]) -> std::borrow::Cow<'a, [ /// in four; this is sixteen times that, for modules that grow with the runtime. pub const READER_STACK: usize = 64 * 1024 * 1024; -/// The §11.5 tree digest of the files of the embedded tree selected by `keep`. -fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { +/// `(repository-relative path, bytes)` rows of the embedded tree, as the +/// §11.5 tree digest reads them, in path order. +pub type EmbeddedRows = Vec<(&'static str, std::borrow::Cow<'static, [u8]>)>; + +/// The rows of the embedded tree selected by `keep`, as the §11.5 tree digest +/// reads them, in path order. +fn embedded_rows(keep: impl Fn(&str) -> bool) -> EmbeddedRows { let selected: Vec<(&'static str, &'static [u8])> = embedded::FILES .iter() .filter(|(path, _)| keep(path)) @@ -266,11 +271,16 @@ fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { }); } }); - let rows: Vec<(&str, std::borrow::Cow<'static, [u8]>)> = selected + selected .iter() .zip(slots) .map(|((path, _), slot)| (*path, slot.into_inner().expect("every file is read"))) - .collect(); + .collect() +} + +/// The §11.5 tree digest of the files of the embedded tree selected by `keep`. +fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { + let rows = embedded_rows(keep); let files: Vec<(&str, &[u8])> = rows .iter() .map(|(path, bytes)| (*path, bytes.as_ref())) @@ -278,6 +288,26 @@ fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { tree_digest(&files) } +/// Which embedded files a language's compiler-semantics ID covers (§21.2): +/// language 1.0 excludes every file introduced for 1.1 or 1.2, and 1.1 every +/// file introduced for 1.2. `None` for an unsupported language. +fn semantics_selection(language: &str) -> Option bool> { + match language { + LANGUAGE_VERSION => Some(|path| !is_v1_1_file(path) && !is_v1_2_file(path)), + LANGUAGE_1_1 => Some(|path| !is_v1_2_file(path)), + LANGUAGE_1_2 => Some(|_| true), + _ => None, + } +} + +/// The `(repository-relative path, bytes)` rows of the embedded tree whose §11.5 +/// digest is `language`'s compiler-semantics ID, in path order (§21.10). +/// `None` for an unsupported language. +#[must_use] +pub fn compiler_semantics_files(language: &str) -> Option { + semantics_selection(language).map(embedded_rows) +} + /// The compiler-semantics ID for one selected language. Language 1.0 excludes /// all files introduced solely for 1.1 and 1.2, and Language 1.1 excludes /// all files introduced solely for 1.2, preserving historical identities. @@ -288,11 +318,16 @@ pub fn compiler_semantics_id_for(language: &str) -> Sha256Digest { static V1_1: OnceLock = OnceLock::new(); static V1_2: OnceLock = OnceLock::new(); match language { - LANGUAGE_VERSION => { - *V1_0.get_or_init(|| embedded_digest(|path| !is_v1_1_file(path) && !is_v1_2_file(path))) + LANGUAGE_VERSION | LANGUAGE_1_1 | LANGUAGE_1_2 => { + let slot = match language { + LANGUAGE_VERSION => &V1_0, + LANGUAGE_1_1 => &V1_1, + _ => &V1_2, + }; + *slot.get_or_init(|| { + embedded_digest(semantics_selection(language).expect("a supported language")) + }) } - LANGUAGE_1_1 => *V1_1.get_or_init(|| embedded_digest(|path| !is_v1_2_file(path))), - LANGUAGE_1_2 => *V1_2.get_or_init(|| embedded_digest(|_| true)), // An unsupported language never shares an identity with a supported // one; configuration rejects it before any identity is published. other => Sha256Digest::of(format!("unsupported-language\0{other}").as_bytes()), diff --git a/features/suites/artifacts.feature b/features/suites/artifacts.feature index 0c78d9cfa2..b0fd52c515 100644 --- a/features/suites/artifacts.feature +++ b/features/suites/artifacts.feature @@ -105,3 +105,10 @@ Feature: artifacts When the verified root and the content-addressed build directory are each listed Then the verified set contains a `.olean` file, `attestation.json`, and a `process/lean/` record And the build set contains no `.olean`, no `process/` entry, no `attestation.json`, and no `.pdf` + + @AR-15 @build + Scenario: Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. + Given every committed project with a build manifest and every committed Rust package + When the canonical bytes of each recorded identity's kind are hashed with SHA-256 + Then each digest equals the identity the artifact records + And a copy whose source changed while it kept the committed manifest has its source and build IDs named diff --git a/model/ids.toml b/model/ids.toml index 7a3ca502f5..3f5b9ba8b4 100644 --- a/model/ids.toml +++ b/model/ids.toml @@ -1213,6 +1213,12 @@ level = "build" suite = "artifacts" statement = "Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations." +[[id]] +id = "AR-15" +level = "build" +suite = "artifacts" +statement = "Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused." + [[id]] id = "VR-01" level = "build" diff --git a/xtask/src/spec_links.rs b/xtask/src/spec_links.rs index 231545c396..bc36890817 100644 --- a/xtask/src/spec_links.rs +++ b/xtask/src/spec_links.rs @@ -61,8 +61,8 @@ fn parse_table(spec: &str) -> Result, Fail> { pub fn validate(root: &Path) -> Result<(), Fail> { let spec = std::fs::read_to_string(root.join("SPEC.md"))?; let rows = parse_table(&spec)?; - if rows.len() != 493 { - return Err(format!("RP-07: the §31 table has {} rows, not 493", rows.len()).into()); + if rows.len() != 494 { + return Err(format!("RP-07: the §31 table has {} rows, not 494", rows.len()).into()); } // The prose total under the table is part of the contract a reader // relies on; a row added without it would leave §31 contradicting itself. From 2006e5599ba36d6fa809872ea3cb63a98c24734c Mon Sep 17 00:00:00 2001 From: humuhumu33 Date: Fri, 13 Nov 2026 01:18:21 +0000 Subject: [PATCH 2/3] api: content identities outside the seven Engine entry points (CL-13) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The canonical bytes of a project's identities are a hidden free function, api::content_identities, and not an Engine method: §24.1 fixes the Engine's public entry points at seven. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 ++-- crates/conformance/src/cases/artifacts.rs | 10 ++++--- crates/lexlean/src/api.rs | 36 +++++++++++++---------- 3 files changed, 30 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2360e5ea7b..76ed775e9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,8 +23,9 @@ workspace carried at those versions without a tag. LexLean records is already that digest, so no recorded byte moves. `lexlean::artifact::address` has one function per artifact kind returning its canonical bytes, written by the same recipe as the identity - (`content_id::*_frames` over a hash or a buffer), and - `Engine::identities` gives a checked project's. `AR-15` recomputes with + (`content_id::*_frames` over a hash or a buffer), and the hidden + `api::content_identities` gives a checked project's, outside the seven + stable `Engine` entry points (§24.1). `AR-15` recomputes with its own SHA-256 every identity the committed build manifests, locks, build outputs, and Rust packages record, and requires each to equal the recorded one. The register is 494 IDs. diff --git a/crates/conformance/src/cases/artifacts.rs b/crates/conformance/src/cases/artifacts.rs index 18956b5f98..153d79d649 100644 --- a/crates/conformance/src/cases/artifacts.rs +++ b/crates/conformance/src/cases/artifacts.rs @@ -850,11 +850,13 @@ fn audit_project(dir: &camino::Utf8Path, into: &mut Addresses) { .unwrap_or_else(|error| panic!("{dir}: the project locks: {error}")); engine }; - let identities = engine - .identities(CheckRequest { + let identities = lexlean::api::content_identities( + &engine, + &CheckRequest { selection: Selection::Entrypoints, - }) - .unwrap_or_else(|error| panic!("{dir}: the project checks: {error}")); + }, + ) + .unwrap_or_else(|error| panic!("{dir}: the project checks: {error}")); for (what, recorded, canonical) in [ ( "source_id", diff --git a/crates/lexlean/src/api.rs b/crates/lexlean/src/api.rs index b69607b896..c138fdd5cb 100644 --- a/crates/lexlean/src/api.rs +++ b/crates/lexlean/src/api.rs @@ -935,21 +935,6 @@ impl Engine { self.bound("check", self.check_inner(&request)) } - /// The canonical bytes of the project's content identities (§21.10): - /// its configuration, its language's compiler-semantics tree, and the - /// messages of its source, semantic, and build IDs, whose SHA-256 are the - /// IDs a build records. - pub fn identities( - &self, - request: CheckRequest, - ) -> Result { - let (checked, _lock) = self.checked(&request.selection)?; - Ok(crate::artifact::address::identities( - &self.project.config, - &checked, - )) - } - /// Obtain the stable semantic snapshot (§24.1, Task 3). pub fn snapshot( &self, @@ -1559,3 +1544,24 @@ pub fn verify_part( } } } + +/// The canonical bytes of a project's content identities (§21.10): its +/// configuration, its language's compiler-semantics tree, and the messages of +/// its source, semantic, and build IDs, whose SHA-256 are the IDs a build +/// records. Not one of the stable entry points of §24.1: `AR-15` holds every +/// committed identity to it. +/// +/// # Errors +/// +/// When the project does not check. +#[doc(hidden)] +pub fn content_identities( + engine: &Engine, + request: &CheckRequest, +) -> Result { + let (checked, _lock) = engine.checked(&request.selection)?; + Ok(crate::artifact::address::identities( + &engine.project.config, + &checked, + )) +} From 39de696e1816e3e8d64046b8305d3ed9275b8d96 Mon Sep 17 00:00:00 2001 From: humuhumu33 <68450279+humuhumu33@users.noreply.github.com> Date: Sat, 14 Nov 2026 01:12:46 +0000 Subject: [PATCH 3/3] cargo fmt + model-write: fix rustfmt and regenerate CONFORMANCE.md for AR-15 --- CONFORMANCE.md | 1 + crates/lexlean/src/lib.rs | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index a1d178c889..5bab5d21be 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -37,6 +37,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `AR-12` | `build` | Concurrent and failed builds preserve atomic content-addressed artifacts and never overwrite unexplained bytes. | | `AR-13` | `build` | Two clean builds in different absolute directories produce byte-identical platform-independent artifacts. | | `AR-14` | `build` | Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations. | +| `AR-15` | `build` | Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. | ## authoring diff --git a/crates/lexlean/src/lib.rs b/crates/lexlean/src/lib.rs index 46bc64d576..883550a3ba 100644 --- a/crates/lexlean/src/lib.rs +++ b/crates/lexlean/src/lib.rs @@ -333,7 +333,11 @@ pub fn compiler_semantics_id_for(language: &str) -> Sha256Digest { static V1_2: OnceLock = OnceLock::new(); match language { LANGUAGE_VERSION | LANGUAGE_1_1 => { - let slot = if language == LANGUAGE_VERSION { &V1_0 } else { &V1_1 }; + let slot = if language == LANGUAGE_VERSION { + &V1_0 + } else { + &V1_1 + }; *slot.get_or_init(|| { embedded_digest(semantics_selection(language).expect("a supported language")) })