diff --git a/CHANGELOG.md b/CHANGELOG.md index a02c2550cf..707bb7b60b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,18 @@ workspace carried at those versions without a tag. ## Unreleased +- One address scheme for every content identity (SPEC.md §21.10, `AR-15`). + A UOR address is `sha256:` and the SHA-256 of an artifact's canonical + bytes, the label form of the UOR Foundation's `uor-addr`; every identity + LexLean records is already that digest, so no recorded byte moves. + `lexlean::artifact::address` has one function per artifact kind returning + its canonical bytes, written by the same recipe as the identity + (`content_id::*_frames` over a hash or a buffer), and the hidden + `api::content_identities` gives a checked project's, outside the seven + stable `Engine` entry points (§24.1). `AR-15` recomputes with + its own SHA-256 every identity the committed build manifests, locks, + build outputs, and Rust packages record, and requires each to equal the + recorded one. The register is 497 IDs. - Every part of a verification names its toolchain by address (SPEC.md §22.11, §9.4). A shard writes `toolchain.json` (`lexlean/verify-toolchain/1`) beside its output: the SHA-256 of the `lean`, `lake`, and `leanchecker` its diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 47cd55c2b7..a318c2b927 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -37,6 +37,7 @@ the error registry does not sanction (R5, `ERRORS.md`). | `AR-12` | `build` | Concurrent and failed builds preserve atomic content-addressed artifacts and never overwrite unexplained bytes. | | `AR-13` | `build` | Two clean builds in different absolute directories produce byte-identical platform-independent artifacts. | | `AR-14` | `build` | Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations. | +| `AR-15` | `build` | Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. | ## authoring diff --git a/README.md b/README.md index db42958671..2854e65f57 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ just vv # the complete normative acceptance gate (SPEC.md §9.2) just release # vv, then the §30 release criterion; refused until 1.0.0 ``` -All 496 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. +All 497 registered conformance IDs have their named tests; `just vv` checks them with the pinned toolchain installed. The differential against the independent generator `lean4-prod` (SPEC.md §17.19, `OR-01`..`OR-08`) acquires it at its pinned commit: from the checkout `LEXLEAN_ORACLE_DIR` names, or by fetching the commit with `git` into the Cargo target directory (`LEXLEAN_ORACLE_CACHE` overrides), and builds it with the pinned Lean and Rust toolchains. It is validation evidence only and no input to any identity. An oracle that cannot be acquired, is not the pinned files, or fails is unavailable: that fails the gate on Linux x86-64 and is printed as `NOT VALIDATED` elsewhere, and is never reported as agreement. @@ -178,7 +178,7 @@ Every row is validated by `just vv`; the IDs link the claim to its register row, | The structured proof language with pinned Lean lowerings, including closed linear arithmetic, integer order, the fold laws, and decision regions | `PF-01`..`PF-24` | `build` | | Prose-free deterministic generated Lean with complete token traceability | `LN-01`..`LN-12` | `build` | | Canonical LaTeX regeneration and the optional hash-checked PDF provider | `TX-01`..`TX-12` | `build` | -| Canonical diagnostics, source maps, coverage, manifests, and reproducible builds | `AR-01`..`AR-14` | `build` | +| Canonical diagnostics, source maps, coverage, manifests, reproducible builds, and UOR addresses | `AR-01`..`AR-15` | `build` | | Sixteen-stage verification with leanchecker replay and exact axiom audit | `VR-01`..`VR-23` | `build` | | The exact CLI contract, the stable seven-method Rust `Engine` API, the host-driven end of the children the library started, the `production` subcommands, and the `migrate` command | `CL-01`..`CL-24` | `build` | | Filesystem confinement, no shell, no hidden network, closed failure model | `SE-01`..`SE-12` | `build` | diff --git a/SPEC.md b/SPEC.md index c2d7da9cc7..519c2e6e18 100644 --- a/SPEC.md +++ b/SPEC.md @@ -9900,6 +9900,26 @@ or 1.1 build publishes no declarations file, so their output is unchanged. --- + +### 21.10 UOR addresses + +A UOR address is the label `sha256:` followed by the 64 lowercase hexadecimal digits of the SHA-256 of an artifact's canonical bytes: 71 bytes, the form of the UOR Foundation's `uor-addr` labels. Every content identity this specification defines is the SHA-256 of the canonical bytes of one artifact kind, so each recorded identity is its artifact's UOR address with the label's prefix removed. Recorded identities stay bare hexadecimal; the address is the same digest. + +| Kind | Canonical bytes | Recorded as | +| --- | --- | --- | +| file | the file's bytes as stored | lock `workspace_file` and package digests, manifest `inputs` and `outputs`, `.olean`, executable, process-stream, and PDF digests | +| tree | `lexlean-tree-v1`, NUL, then `frame(path, bytes)` per row in path order (§11.5) | tree digests of packages | +| compiler semantics | the tree of the language's embedded files (§21.2) | `compiler_semantics`, `semantics_id` | +| project configuration | the canonical `lexlean.toml` (§10.1) | `project_config_sha256` | +| source | the §21.3 recipe: `lexlean-source-v1`, NUL, then its frames | `source_id` | +| semantic | the §21.4 recipe | `semantic_id` | +| build | the §21.5 recipe | `build_id` | +| attestation | the §22.9 recipe | `attestation_id` | +| PDF recipe | the §19.8 recipe | `recipe_id` | +| Rust program | the program's canonical JSON with one final LF (§21.7) | a package's `program` | + +A framed kind's bytes begin with its own domain and a NUL, so two kinds never share an address by their framing (UOR-GNAF draft.2 §14.3 requires typed domains for addresses); a file kind is its bytes, whose kind the field that records it names. The compiler writes each framed kind's bytes and its identity with one recipe, into a buffer or into the hash (`artifact::address`, `artifact::content_id`), so the two cannot differ. `AR-15` recomputes, with its own SHA-256, the address of every identity that the committed build manifests, locks, build outputs, and Rust packages record, and requires each to equal the recorded identity. + ## 22. Verification ### 22.1 Verification stages @@ -11875,6 +11895,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `AR-12` | `artifacts` | Concurrent and failed builds preserve atomic content-addressed artifacts and never overwrite unexplained bytes. | §21.8 | | `AR-13` | `artifacts` | Two clean builds in different absolute directories produce byte-identical platform-independent artifacts. | §28.4 | | `AR-14` | `artifacts` | Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations. | §21, §22 | +| `AR-15` | `artifacts` | Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. | §21.10 | | `VR-01` | `verification` | Verification runs every specified stage in order and exposes no stage-suppression option. | §22.1 | | `VR-02` | `verification` | Lean, Lake, and leanchecker versions and executable hashes are checked and recorded before use. | §22.2 | | `VR-03` | `verification` | Lean execution uses the pinned Lake environment and never updates or fetches workspace dependencies. | §22.2 | @@ -12170,7 +12191,7 @@ Every row below is normative, has honesty level `build`, and MUST be copied byte | `LM-09` | `language-models` | No prompt is code: no member of the closed language maps `string` or `bytes` to a function, a term, or source — every operation of the primitive registry that takes a `string` or `bytes` operand is classified by the audit with a data result, the registry's schema enumeration and the production registry's primitive rows equal one another and equal that classified set, and the text declarations of `lexlean/semantic-module/2` state no `string`-typed member, so a prompt reaches a model only as the tokens `encodeText` makes of it; a planted operation or text member of that shape is caught by the audit with its name. | §17.12 | | `LM-13` | `language-models` | The text laws a tokenizer elaborates through are LexLean source and not authored text: the module that states them is one language-1.2 native core module of the embedded `language/text-1.2` project, the project holds no file of Lean source, no independently authored Lean file anywhere in the release tree declares the `LexLeanText` namespace or lies under its module root, and the one-time migration input is absent; the registry lists only that module, the declarations and their exact axiom policies are read from its native data, which the compiler embeds and hashes into the language-1.2 semantics, and a module that declares a text component imports the generated module rather than splicing any Lean text. | §17.10, §17.12 | -**Total required capability IDs:** 496. +**Total required capability IDs:** 497. No row may be downgraded to `some-true` or `open`. Upstream Lean facts are ledger/authority rows, not substitutions for these build behaviors. diff --git a/crates/conformance/src/cases/artifacts.rs b/crates/conformance/src/cases/artifacts.rs index 01eba8d1aa..153d79d649 100644 --- a/crates/conformance/src/cases/artifacts.rs +++ b/crates/conformance/src/cases/artifacts.rs @@ -1,4 +1,4 @@ -//! The `artifacts` suite: AR-01..AR-14. +//! The `artifacts` suite: AR-01..AR-15. use lexlean::artifact::canonical_json::Json; use lexlean::artifact::content_id::{FramedHasher, Sha256Digest}; @@ -725,6 +725,330 @@ pub(crate) fn run(id: &str) { ); } } + // §21.10, AR-15: every recorded identity is a UOR address. + "AR-15" => recorded_identities_are_addresses(), other => panic!("no artifacts case is wired for {other}"), } } + +/// The lowercase-hex SHA-256 of `bytes`, computed here and not by the +/// compiler, so a recorded identity is held to the hash of the canonical +/// bytes and not to the compiler's own digest of them. +fn sha256_hex(bytes: &[u8]) -> String { + sha2::Sha256::digest(bytes) + .iter() + .map(|byte| format!("{byte:02x}")) + .collect() +} + +/// One comparison of a recorded identity with the address of its artifact's +/// canonical bytes. +struct Addresses { + compared: usize, + unmet: Vec, +} + +impl Addresses { + fn same(&mut self, what: &str, recorded: Option<&str>, canonical: &[u8]) { + self.compared += 1; + let address = sha256_hex(canonical); + match recorded { + Some(recorded) if recorded == address => {} + other => self.unmet.push(format!( + "{what}: recorded {other:?}, address sha256:{address}" + )), + } + } +} + +/// Every committed project with a committed build manifest, relative to the +/// repository root, in path order. +fn committed_manifest_projects() -> Vec { + let root = support::repo_root(); + let mut out = Vec::new(); + for entry in walkdir::WalkDir::new(root.as_std_path()) + .sort_by_file_name() + .into_iter() + .filter_entry(|entry| { + let name = entry.file_name().to_string_lossy(); + !(entry.depth() > 0 && (name.starts_with('.') || name == "target")) + }) + .flatten() + { + if entry.file_name() == "manifest.json" + && entry.path().ends_with("expected/build/manifest.json") + { + let project = entry + .path() + .parent() + .and_then(std::path::Path::parent) + .and_then(std::path::Path::parent) + .expect("a project directory"); + if project.join("lexlean.toml").is_file() { + out.push( + camino::Utf8PathBuf::from_path_buf(project.to_path_buf()) + .expect("a UTF-8 path"), + ); + } + } + } + out +} + +/// Compare every identity a committed project records with the address of +/// the canonical bytes `lexlean::artifact::address` gives for it: the +/// manifest's source, semantic, build, and compiler-semantics IDs, the +/// lock's compiler-semantics ID and workspace files, and every build output +/// committed beside the manifest. A project that commits no lock (its case +/// writes one) is locked in a copy first, and has no lock rows to compare. +fn audit_project(dir: &camino::Utf8Path, into: &mut Addresses) { + use lexlean::{CheckRequest, Engine, LockRequest, Selection}; + let expected = dir.join("expected/build"); + let manifest: serde_json::Value = serde_json::from_slice( + &std::fs::read(expected.join("manifest.json").as_std_path()).expect("the manifest"), + ) + .expect("the manifest parses"); + let text = |value: &serde_json::Value| value.as_str().map(str::to_owned); + // A tooling fixture (§17.20) is rendered as an embedded project, not + // checked as a user project: its build ID is held to the build recipe + // over the source and semantic IDs it records, its compiler-semantics ID + // to its language's tree, and its outputs to their bytes. + if dir.starts_with(support::repo_root().join("tests/tooling")) { + let id = |key: &str| { + lexlean::Sha256Digest::from_hex(manifest[key].as_str().expect("an ID")) + .expect("a digest") + }; + into.same( + &format!("{dir}: build_id"), + text(&manifest["build_id"]).as_deref(), + &lexlean::artifact::address::build(id("source_id"), id("semantic_id")).bytes, + ); + let language = manifest["language"].as_str().expect("a language"); + into.same( + &format!("{dir}: compiler.semantics_id"), + text(&manifest["compiler"]["semantics_id"]).as_deref(), + &lexlean::artifact::address::compiler_semantics(language) + .expect("a supported language") + .bytes, + ); + audit_outputs(dir, &manifest, into); + return; + } + let committed_lock = dir.join("lexlean.lock").as_std_path().is_file(); + let copy = tempfile::tempdir().expect("tempdir"); + let engine = if committed_lock { + Engine::load(&dir.join("lexlean.toml")).expect("the project loads") + } else { + support::copy_tree(dir.as_std_path(), copy.path(), &[".lexlean", "expected"]); + let root = camino::Utf8Path::from_path(copy.path()).expect("a UTF-8 path"); + let engine = Engine::load(&root.join("lexlean.toml")).expect("the project loads"); + engine + .lock(LockRequest { + check_only: false, + allow_network: false, + }) + .unwrap_or_else(|error| panic!("{dir}: the project locks: {error}")); + engine + }; + let identities = lexlean::api::content_identities( + &engine, + &CheckRequest { + selection: Selection::Entrypoints, + }, + ) + .unwrap_or_else(|error| panic!("{dir}: the project checks: {error}")); + for (what, recorded, canonical) in [ + ( + "source_id", + text(&manifest["source_id"]), + &identities.source, + ), + ( + "semantic_id", + text(&manifest["semantic_id"]), + &identities.semantic, + ), + ("build_id", text(&manifest["build_id"]), &identities.build), + ( + "compiler.semantics_id", + text(&manifest["compiler"]["semantics_id"]), + &identities.compiler_semantics, + ), + ] { + into.same( + &format!("{dir}: {what}"), + recorded.as_deref(), + &canonical.bytes, + ); + } + if !committed_lock { + audit_outputs(dir, &manifest, into); + return; + } + let lock: toml::Value = toml::from_str( + &std::fs::read_to_string(dir.join("lexlean.lock").as_std_path()).expect("the lock"), + ) + .expect("the lock parses"); + into.same( + &format!("{dir}: lexlean.lock compiler_semantics"), + lock.get("compiler_semantics").and_then(toml::Value::as_str), + &identities.compiler_semantics.bytes, + ); + for row in lock + .get("workspace_file") + .and_then(toml::Value::as_array) + .into_iter() + .flatten() + { + let path = row["path"].as_str().expect("a workspace file path"); + let bytes = std::fs::read(dir.join(path).as_std_path()).expect("a workspace file"); + into.same( + &format!("{dir}: lexlean.lock workspace_file {path}"), + row.get("sha256").and_then(toml::Value::as_str), + &lexlean::artifact::address::file(&bytes).bytes, + ); + } + audit_outputs(dir, &manifest, into); +} + +/// Compare every build output committed beside a manifest with the address +/// of its bytes, and its recorded length with theirs. +fn audit_outputs(dir: &camino::Utf8Path, manifest: &serde_json::Value, into: &mut Addresses) { + let expected = dir.join("expected/build"); + for output in manifest["outputs"].as_array().expect("outputs") { + let path = output["path"].as_str().expect("an output path"); + let Ok(bytes) = std::fs::read(expected.join(path).as_std_path()) else { + continue; + }; + assert_eq!( + output["byte_length"].as_u64(), + Some(bytes.len() as u64), + "{dir}: output {path}: byte length" + ); + into.same( + &format!("{dir}: output {path}"), + output["sha256"].as_str(), + &lexlean::artifact::address::file(&bytes).bytes, + ); + } +} + +/// Compare every identity a committed Rust package records (§17.23) with the +/// address of its canonical bytes: each file of the package, its program, +/// and the compiler-semantics ID it was exported under. +fn audit_package(dir: &std::path::Path, into: &mut Addresses) { + let read = |name: &str| -> serde_json::Value { + serde_json::from_slice(&std::fs::read(dir.join(name)).expect("a package file")) + .expect("a package file parses") + }; + let provenance = read("provenance.json"); + let package = read("package.json"); + let at = dir.display(); + for (path, recorded) in provenance["files"].as_object().expect("files") { + let bytes = std::fs::read(dir.join(path)).expect("a packaged file"); + into.same( + &format!("{at}: {path}"), + recorded.as_str(), + &lexlean::artifact::address::file(&bytes).bytes, + ); + } + let program = lexlean::artifact::canonical_json::Json::parse( + &serde_json::to_vec(&package["program"]).expect("the program serializes"), + ) + .expect("the program is canonical JSON"); + into.same( + &format!("{at}: program"), + provenance["program"].as_str(), + &lexlean::artifact::address::rust_program(&program).bytes, + ); + let language = lexlean::LANGUAGE_1_2; + into.same( + &format!("{at}: compiler_semantics"), + provenance["compiler_semantics"].as_str(), + &lexlean::artifact::address::compiler_semantics(language) + .expect("language 1.2") + .bytes, + ); +} + +/// §21.10: every content identity a committed artifact records is the UOR +/// address of that artifact's canonical bytes, and a recorded identity that +/// no longer matches its artifact is named. +fn recorded_identities_are_addresses() { + let projects = committed_manifest_projects(); + assert!( + projects.len() >= 26, + "every committed build manifest is audited: {projects:?}" + ); + let mut audit = Addresses { + compared: 0, + unmet: Vec::new(), + }; + for project in &projects { + audit_project(project, &mut audit); + } + let after_projects = audit.compared; + let root = support::repo_root(); + let mut packages = 0; + for entry in walkdir::WalkDir::new(root.join("tests/production-export").as_std_path()) + .sort_by_file_name() + .into_iter() + .flatten() + { + if entry.file_name() == "provenance.json" { + audit_package(entry.path().parent().expect("a package"), &mut audit); + packages += 1; + } + } + assert!( + packages >= 164, + "every committed package is audited: {packages}" + ); + assert!( + audit.unmet.is_empty(), + "{} of {} recorded identities are not the address of their canonical bytes:\n{}", + audit.unmet.len(), + audit.compared, + audit.unmet.join("\n") + ); + eprintln!( + "AR-15: {} identities of {} projects and {packages} packages are the addresses of their canonical bytes", + audit.compared, + projects.len() + ); + assert!( + after_projects > 26 * 4, + "{after_projects} project identities" + ); + + // A recorded identity that no longer names its artifact is refused: a + // copy of an example whose theorem label changed still carries the + // committed manifest, whose source and build IDs name the old source. + let project = P::copy_example("nat-add-zero"); + let committed = root.join("examples/nat-add-zero/expected/build/manifest.json"); + let copied = project.root.join("expected/build/manifest.json"); + std::fs::create_dir_all(copied.parent().expect("a parent").as_std_path()).expect("mkdir"); + std::fs::copy(committed.as_std_path(), copied.as_std_path()).expect("the manifest"); + project.edit( + "src/Main.lex.tex", + "\\begin{theorem}{add-zero}", + "\\begin{theorem}{add-zero-again}", + ); + let mut planted = Addresses { + compared: 0, + unmet: Vec::new(), + }; + audit_project(&project.root, &mut planted); + assert!( + planted + .unmet + .iter() + .any(|line| line.contains(": source_id: recorded")) + && planted + .unmet + .iter() + .any(|line| line.contains(": build_id: recorded")), + "a stale recorded identity is named: {:?}", + planted.unmet + ); +} diff --git a/crates/conformance/src/cases/repository.rs b/crates/conformance/src/cases/repository.rs index d7c2dd69d3..e6c45c8142 100644 --- a/crates/conformance/src/cases/repository.rs +++ b/crates/conformance/src/cases/repository.rs @@ -1091,7 +1091,7 @@ pub(crate) fn run(id: &str) { let model = repo_model::Model::load(&root.join("model").into_std_path_buf()) .expect("the model loads"); let table = spec_table(); - assert_eq!(table.len(), 496, "§31 has 496 rows"); + assert_eq!(table.len(), 497, "§31 has 497 rows"); assert_eq!(model.ids.id.len(), table.len(), "register row count"); for ((spec_id, spec_suite, spec_statement), row) in table.iter().zip(model.ids.id.iter()) diff --git a/crates/conformance/src/support.rs b/crates/conformance/src/support.rs index 1e5f89b902..9caf883655 100644 --- a/crates/conformance/src/support.rs +++ b/crates/conformance/src/support.rs @@ -324,7 +324,7 @@ pub struct P { pub root: Utf8PathBuf, } -fn copy_tree(from: &Path, to: &Path, skip: &[&str]) { +pub(crate) fn copy_tree(from: &Path, to: &Path, skip: &[&str]) { for entry in walkdir::WalkDir::new(from).into_iter().flatten() { let relative = entry.path().strip_prefix(from).expect("under root"); let text = relative.to_string_lossy(); diff --git a/crates/conformance/tests/conformance.rs b/crates/conformance/tests/conformance.rs index c83090918d..c09b5aebc7 100644 --- a/crates/conformance/tests/conformance.rs +++ b/crates/conformance/tests/conformance.rs @@ -1020,6 +1020,11 @@ fn conformance_ar_14() { repo_conformance::cases::run("AR-14"); } +#[test] +fn conformance_ar_15() { + repo_conformance::cases::run("AR-15"); +} + #[test] fn conformance_vr_01() { repo_conformance::cases::run("VR-01"); diff --git a/crates/lexlean/src/api.rs b/crates/lexlean/src/api.rs index 400fa59f31..ad790f8c12 100644 --- a/crates/lexlean/src/api.rs +++ b/crates/lexlean/src/api.rs @@ -1556,3 +1556,24 @@ pub fn verify_part( } } } + +/// The canonical bytes of a project's content identities (§21.10): its +/// configuration, its language's compiler-semantics tree, and the messages of +/// its source, semantic, and build IDs, whose SHA-256 are the IDs a build +/// records. Not one of the stable entry points of §24.1: `AR-15` holds every +/// committed identity to it. +/// +/// # Errors +/// +/// When the project does not check. +#[doc(hidden)] +pub fn content_identities( + engine: &Engine, + request: &CheckRequest, +) -> Result { + let (checked, _lock) = engine.checked(&request.selection)?; + Ok(crate::artifact::address::identities( + &engine.project.config, + &checked, + )) +} diff --git a/crates/lexlean/src/artifact/address.rs b/crates/lexlean/src/artifact/address.rs new file mode 100644 index 0000000000..6c951d777c --- /dev/null +++ b/crates/lexlean/src/artifact/address.rs @@ -0,0 +1,396 @@ +//! UOR addresses (SPEC.md §21.10): every content identity LexLean records is +//! the SHA-256 of one artifact's canonical bytes, and its UOR address is that +//! digest written `sha256:<64 lowercase hex digits>`, the form of the UOR +//! Foundation's `uor-addr` labels. +//! +//! Each artifact kind has one function here that returns its canonical +//! bytes, and [`Canonical::address`] is the only address function. The +//! framed kinds are written by the same recipe as the identity +//! (`content_id::*_frames`), into a buffer instead of a hash, so a recorded +//! identity and the address of the canonical bytes cannot differ. The kind +//! is separated inside the hashed bytes: every framed kind begins with its +//! own domain and a NUL, and the untagged kinds are file bytes, which no +//! domain-prefixed message equals by accident of construction. + +use std::fmt; + +use crate::artifact::canonical_json::Json; +use crate::artifact::content_id::{ + attestation_frames, build_frames, pdf_recipe_frames, semantic_frames, source_frames, + tree_frames, Sha256Digest, +}; +use crate::link::CheckedProject; + +/// The hash function of every LexLean address, and the label's prefix. +pub const ALGORITHM: &str = "sha256"; + +/// A UOR address: the SHA-256 of canonical bytes. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub struct Address(pub Sha256Digest); + +impl Address { + /// The address of `canonical` bytes. + #[must_use] + pub fn of(canonical: &[u8]) -> Self { + Self(Sha256Digest::of(canonical)) + } + + /// The digest, which is the identity LexLean records (bare hex). + #[must_use] + pub const fn digest(self) -> Sha256Digest { + self.0 + } + + /// Read the label form, refusing every other spelling: another algorithm, + /// upper-case hex, a missing or extra digit. + /// + /// # Errors + /// + /// What is wrong with `text`. + pub fn parse(text: &str) -> Result { + let hex = text + .strip_prefix(ALGORITHM) + .and_then(|rest| rest.strip_prefix(':')) + .ok_or_else(|| format!("an address begins `{ALGORITHM}:`: {text}"))?; + Sha256Digest::from_hex(hex).map(Self) + } +} + +impl fmt::Display for Address { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{ALGORITHM}:{}", self.0.to_hex()) + } +} + +/// Every kind of artifact LexLean identifies (§21.10). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub enum Kind { + /// A file's bytes as stored: lock workspace files, package files, + /// `.olean` files, executables, process streams, PDFs. + File, + /// The §11.5 tree of `(path, bytes)` rows. + Tree, + /// The compiler-semantics ID: the tree of a language's embedded files. + CompilerSemantics, + /// The canonical `lexlean.toml` (§10.1), `project_config_sha256`. + ProjectConfig, + /// The source ID (§21.3). + Source, + /// The semantic ID (§21.4). + Semantic, + /// The build ID (§21.5). + Build, + /// The attestation ID (§22.9). + Attestation, + /// The PDF recipe ID (§19.8). + PdfRecipe, + /// A Rust package's program (§17.23): its canonical JSON file bytes. + RustProgram, +} + +impl Kind { + /// The domain that begins the kind's canonical bytes, if it has one. + #[must_use] + pub const fn domain(self) -> Option<&'static str> { + match self { + Self::File | Self::ProjectConfig | Self::RustProgram => None, + Self::Tree | Self::CompilerSemantics => Some("lexlean-tree-v1"), + Self::Source => Some("lexlean-source-v1"), + Self::Semantic => Some("lexlean-semantic-v1"), + Self::Build => Some("lexlean-build-v1"), + Self::Attestation => Some("lexlean-attestation-v1"), + Self::PdfRecipe => Some("lexlean-pdf-recipe-v1"), + } + } +} + +/// One artifact's canonical bytes, with their kind. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Canonical { + /// What the bytes are. + pub kind: Kind, + /// The canonical bytes. + pub bytes: Vec, +} + +impl Canonical { + /// The UOR address of these bytes. + #[must_use] + pub fn address(&self) -> Address { + Address::of(&self.bytes) + } +} + +/// A file: its bytes. +#[must_use] +pub fn file(bytes: &[u8]) -> Canonical { + Canonical { + kind: Kind::File, + bytes: bytes.to_vec(), + } +} + +/// A §11.5 tree over `(path, bytes)` rows in strictly sorted path order. +#[must_use] +pub fn tree(files: &[(&str, &[u8])]) -> Canonical { + Canonical { + kind: Kind::Tree, + bytes: tree_frames(Vec::new(), files), + } +} + +/// The compiler-semantics ID of `language`: the tree of the embedded files +/// that language selects (§21.2). `None` for an unsupported language. +#[must_use] +pub fn compiler_semantics(language: &str) -> Option { + let rows = crate::compiler_semantics_files(language)?; + let files: Vec<(&str, &[u8])> = rows + .iter() + .map(|(path, bytes)| (*path, bytes.as_ref())) + .collect(); + Some(Canonical { + kind: Kind::CompilerSemantics, + bytes: tree_frames(Vec::new(), &files), + }) +} + +/// A project's configuration: its canonical `lexlean.toml`. +#[must_use] +pub fn project_config(config: &crate::config::ProjectConfig) -> Canonical { + Canonical { + kind: Kind::ProjectConfig, + bytes: config.canonical_toml().into_bytes(), + } +} + +/// The source ID's message (§21.3). +#[must_use] +pub fn source( + canonical_project_toml: &[u8], + canonical_lock: &[u8], + sources: &[(String, Vec)], +) -> Canonical { + Canonical { + kind: Kind::Source, + bytes: source_frames(Vec::new(), canonical_project_toml, canonical_lock, sources), + } +} + +/// The semantic ID's message (§21.4). +#[must_use] +pub fn semantic( + compiler_semantics: Sha256Digest, + language: &str, + linked_ir_json: &str, + lexicon_closure_json: &str, +) -> Canonical { + Canonical { + kind: Kind::Semantic, + bytes: semantic_frames( + Vec::new(), + compiler_semantics, + language, + linked_ir_json, + lexicon_closure_json, + ), + } +} + +/// The build ID's message (§21.5). +#[must_use] +pub fn build(source: Sha256Digest, semantic: Sha256Digest) -> Canonical { + Canonical { + kind: Kind::Build, + bytes: build_frames(Vec::new(), source, semantic), + } +} + +/// The attestation ID's message (§22.9). +#[must_use] +pub fn attestation(body_without_id_json: &str) -> Canonical { + Canonical { + kind: Kind::Attestation, + bytes: attestation_frames(Vec::new(), body_without_id_json), + } +} + +/// The PDF recipe ID's message (§19.8). +#[must_use] +pub fn pdf_recipe( + tex_sha256: Sha256Digest, + program_sha256: Sha256Digest, + version_stdout_sha256: Sha256Digest, + compile_argv_json: &str, + resources_json: &str, +) -> Canonical { + Canonical { + kind: Kind::PdfRecipe, + bytes: pdf_recipe_frames( + Vec::new(), + tex_sha256, + program_sha256, + version_stdout_sha256, + compile_argv_json, + resources_json, + ), + } +} + +/// A Rust package's program: its canonical JSON with one final LF. +#[must_use] +pub fn rust_program(program: &Json) -> Canonical { + Canonical { + kind: Kind::RustProgram, + bytes: program.to_file_bytes(), + } +} + +/// The content identities of one checked project, as canonical bytes. +#[derive(Debug, Clone)] +pub struct Identities { + /// The canonical `lexlean.toml`. + pub project_config: Canonical, + /// The compiler-semantics tree of the project's language. + pub compiler_semantics: Canonical, + /// The source ID's message. + pub source: Canonical, + /// The semantic ID's message. + pub semantic: Canonical, + /// The build ID's message. + pub build: Canonical, +} + +/// The canonical bytes of every content identity of `checked`, a check of +/// the project configured by `config`. +/// +/// # Panics +/// +/// When `config` names an unsupported language, which configuration refuses +/// before any project is checked. +#[must_use] +pub fn identities(config: &crate::config::ProjectConfig, checked: &CheckedProject) -> Identities { + let project_config = project_config(config); + let compiler_semantics = + compiler_semantics(&config.language).expect("a checked project has a supported language"); + let mut sources: Vec<(String, Vec)> = checked + .modules + .values() + .map(|module| { + ( + module.document.source_path.clone(), + module.normalized.clone().into_bytes(), + ) + }) + .collect(); + sources.sort_by(|a, b| a.0.cmp(&b.0)); + let source = source(&project_config.bytes, &checked.canonical_lock, &sources); + let semantic = semantic( + compiler_semantics.address().digest(), + &config.language, + &checked.linked_json().to_canonical_string(), + &checked + .closure + .closure_json("", &checked.visible_union) + .to_canonical_string(), + ); + let build = build(source.address().digest(), semantic.address().digest()); + Identities { + project_config, + compiler_semantics, + source, + semantic, + build, + } +} + +#[cfg(test)] +mod tests { + use super::{Address, Kind, ALGORITHM}; + use crate::artifact::content_id::{ + attestation_id, build_id, pdf_recipe_id, semantic_id_for, source_id, tree_digest, + Sha256Digest, + }; + + #[test] + fn every_recorded_identity_is_the_address_of_its_canonical_bytes() { + let a = Sha256Digest::of(b"a"); + let b = Sha256Digest::of(b"b"); + let sources = vec![("src/Main.lex.tex".to_owned(), b"body".to_vec())]; + let files: [(&str, &[u8]); 2] = [("a", b"1"), ("b/c", b"2")]; + for (canonical, recorded) in [ + (super::tree(&files), tree_digest(&files)), + ( + super::source(b"toml", b"lock", &sources), + source_id(b"toml", b"lock", &sources), + ), + ( + super::semantic(a, "1.2", "{}", "[]"), + semantic_id_for(a, "1.2", "{}", "[]"), + ), + (super::build(a, b), build_id(a, b)), + (super::attestation("{}"), attestation_id("{}")), + ( + super::pdf_recipe(a, b, a, "[]", "[]"), + pdf_recipe_id(a, b, a, "[]", "[]"), + ), + (super::file(b"bytes"), Sha256Digest::of(b"bytes")), + ] { + assert_eq!( + canonical.address().digest(), + recorded, + "{:?}", + canonical.kind + ); + if let Some(domain) = canonical.kind.domain() { + assert!( + canonical + .bytes + .starts_with(format!("{domain}\0").as_bytes()), + "{:?} begins with its domain", + canonical.kind + ); + } + } + assert_eq!( + super::compiler_semantics(crate::LANGUAGE_1_2) + .expect("a supported language") + .address() + .digest(), + crate::compiler_semantics_id_for(crate::LANGUAGE_1_2) + ); + assert!(super::compiler_semantics("0.9").is_none()); + // The canonical JSON of a small object is the canonical form the UOR + // Foundation's `uor-addr` labels (`tests/byte_identity.rs`), and its + // address is that crate's vector. + let object = crate::artifact::canonical_json::Json::object(vec![( + "foo", + crate::artifact::canonical_json::Json::Str("bar".to_owned()), + )]); + assert_eq!( + Address::of(object.to_canonical_string().as_bytes()).to_string(), + "sha256:7a38bf81f383f69433ad6e900d35b3e2385593f76a7b7ab5d4355b8ba41ee24b" + ); + assert_eq!(Kind::File.domain(), None); + } + + #[test] + fn the_label_is_sha256_and_lowercase_hex() { + let address = Address::of(b""); + let label = address.to_string(); + assert_eq!( + label, + "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + ); + assert_eq!(label.len(), 71); + assert_eq!(Address::parse(&label), Ok(address)); + for refused in [ + label.replacen(ALGORITHM, "blake3", 1), + label.to_uppercase(), + label[..70].to_owned(), + format!("{label}0"), + label.replacen(':', "-", 1), + ] { + assert!(Address::parse(&refused).is_err(), "{refused}"); + } + } +} diff --git a/crates/lexlean/src/artifact/content_id.rs b/crates/lexlean/src/artifact/content_id.rs index b470518847..88ad79e313 100644 --- a/crates/lexlean/src/artifact/content_id.rs +++ b/crates/lexlean/src/artifact/content_id.rs @@ -67,68 +67,126 @@ impl<'de> serde::Deserialize<'de> for Sha256Digest { } } -/// An incremental framed hasher over one labeled recipe (§21.1). -pub struct FramedHasher { - hasher: Sha256, +/// Where a recipe's bytes go: a SHA-256 state, for the identity, or a +/// buffer, for the canonical bytes the identity is the SHA-256 of +/// (`artifact::address`). One recipe written once over a sink gives both, +/// so the two never differ. +pub trait FrameSink { + /// Append bytes. + fn put(&mut self, bytes: &[u8]); +} + +impl FrameSink for Sha256 { + fn put(&mut self, bytes: &[u8]) { + self.update(bytes); + } +} + +impl FrameSink for Vec { + fn put(&mut self, bytes: &[u8]) { + self.extend_from_slice(bytes); + } +} + +/// One labeled recipe (§21.1): the domain, a NUL, then frames. +pub struct Framed { + sink: S, } -impl FramedHasher { - /// Start a recipe with its domain-separation prefix, e.g. +impl Framed { + /// Start a recipe in `sink` with its domain-separation prefix, e.g. /// `"lexlean-source-v1\0"`. #[must_use] - pub fn new(domain: &str) -> Self { - let mut hasher = Sha256::new(); - hasher.update(domain.as_bytes()); - hasher.update([0u8]); - Self { hasher } + pub fn with_sink(mut sink: S, domain: &str) -> Self { + sink.put(domain.as_bytes()); + sink.put(&[0u8]); + Self { sink } } /// Append one `frame(label, bytes)` (§21.1). pub fn frame(&mut self, label: &str, bytes: &[u8]) { let label_bytes = label.as_bytes(); - self.hasher.update( - u32::try_from(label_bytes.len()) + self.sink.put( + &u32::try_from(label_bytes.len()) .expect("label fits u32") .to_be_bytes(), ); - self.hasher.update(label_bytes); - self.hasher.update((bytes.len() as u64).to_be_bytes()); - self.hasher.update(bytes); + self.sink.put(label_bytes); + self.sink.put(&(bytes.len() as u64).to_be_bytes()); + self.sink.put(bytes); + } + + /// The sink, holding everything written. + #[must_use] + pub fn into_sink(self) -> S { + self.sink + } +} + +/// An incremental framed hasher over one labeled recipe (§21.1). +pub type FramedHasher = Framed; + +impl Framed { + /// Start a hashed recipe with its domain-separation prefix. + #[must_use] + pub fn new(domain: &str) -> Self { + Self::with_sink(Sha256::new(), domain) } /// Finish the recipe. #[must_use] pub fn finish(self) -> Sha256Digest { - Sha256Digest(self.hasher.finalize().into()) + Sha256Digest(self.sink.finalize().into()) } } -/// The §11.5 tree digest over `(relative path, bytes)` rows. The caller -/// supplies rows already restricted to participating files; this function -/// enforces bytewise-sorted path order and unique paths. -#[must_use] -pub fn tree_digest(files: &[(&str, &[u8])]) -> Sha256Digest { +impl Framed> { + /// Start a recipe whose bytes are kept, with its domain prefix. + #[must_use] + pub fn bytes(domain: &str) -> Self { + Self::with_sink(Vec::new(), domain) + } +} + +/// The recipe of the §11.5 tree digest: one frame per file, labeled by its +/// path, in the order given. +pub fn tree_frames(sink: S, files: &[(&str, &[u8])]) -> S { debug_assert!( files .windows(2) .all(|pair| pair[0].0.as_bytes() < pair[1].0.as_bytes()), "tree digest input must be strictly sorted by path bytes" ); - let mut hasher = Sha256::new(); - hasher.update(b"lexlean-tree-v1"); - hasher.update([0u8]); + let mut framed = Framed::with_sink(sink, "lexlean-tree-v1"); for (path, bytes) in files { - let path_bytes = path.as_bytes(); - hasher.update( - u32::try_from(path_bytes.len()) - .expect("path fits u32") - .to_be_bytes(), - ); - hasher.update(path_bytes); - hasher.update((bytes.len() as u64).to_be_bytes()); - hasher.update(bytes); + framed.frame(path, bytes); } - Sha256Digest(hasher.finalize().into()) + framed.into_sink() +} + +/// The §11.5 tree digest over `(relative path, bytes)` rows. The caller +/// supplies rows already restricted to participating files; this function +/// enforces bytewise-sorted path order and unique paths. +#[must_use] +pub fn tree_digest(files: &[(&str, &[u8])]) -> Sha256Digest { + Sha256Digest(tree_frames(Sha256::new(), files).finalize().into()) +} + +/// The recipe of the source ID (§21.3). +pub fn source_frames( + sink: S, + canonical_project_toml: &[u8], + canonical_lock: &[u8], + sources: &[(String, Vec)], +) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-source-v1"); + framed.frame("project", canonical_project_toml); + framed.frame("lock", canonical_lock); + for (path, bytes) in sources { + framed.frame("path", path.as_bytes()); + framed.frame("source", bytes); + } + framed.into_sink() } /// The source ID (§21.3). `sources` are `(project-relative path, normalized @@ -139,14 +197,33 @@ pub fn source_id( canonical_lock: &[u8], sources: &[(String, Vec)], ) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-source-v1"); - hasher.frame("project", canonical_project_toml); - hasher.frame("lock", canonical_lock); - for (path, bytes) in sources { - hasher.frame("path", path.as_bytes()); - hasher.frame("source", bytes); - } - hasher.finish() + Sha256Digest( + source_frames( + Sha256::new(), + canonical_project_toml, + canonical_lock, + sources, + ) + .finalize() + .into(), + ) +} + +/// The recipe of the semantic ID (§21.4). +pub fn semantic_frames( + sink: S, + compiler_semantics: Sha256Digest, + language: &str, + linked_ir_json: &str, + lexicon_closure_json: &str, +) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-semantic-v1"); + framed.frame("compiler-semantics", &compiler_semantics.0); + framed.frame("language", language.as_bytes()); + framed.frame("toolchain", crate::LEAN_TOOLCHAIN.as_bytes()); + framed.frame("linked-ir", linked_ir_json.as_bytes()); + framed.frame("lexicon-closure", lexicon_closure_json.as_bytes()); + framed.into_sink() } /// The semantic ID (§21.4). @@ -157,13 +234,17 @@ pub fn semantic_id_for( linked_ir_json: &str, lexicon_closure_json: &str, ) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-semantic-v1"); - hasher.frame("compiler-semantics", &compiler_semantics.0); - hasher.frame("language", language.as_bytes()); - hasher.frame("toolchain", crate::LEAN_TOOLCHAIN.as_bytes()); - hasher.frame("linked-ir", linked_ir_json.as_bytes()); - hasher.frame("lexicon-closure", lexicon_closure_json.as_bytes()); - hasher.finish() + Sha256Digest( + semantic_frames( + Sha256::new(), + compiler_semantics, + language, + linked_ir_json, + lexicon_closure_json, + ) + .finalize() + .into(), + ) } /// The language-1.0 semantic ID recipe retained for API compatibility. @@ -181,22 +262,58 @@ pub fn semantic_id( ) } +/// The recipe of the build ID (§21.5). +pub fn build_frames(sink: S, source: Sha256Digest, semantic: Sha256Digest) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-build-v1"); + framed.frame("source-id", &source.0); + framed.frame("semantic-id", &semantic.0); + framed.into_sink() +} + /// The build ID (§21.5). #[must_use] pub fn build_id(source: Sha256Digest, semantic: Sha256Digest) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-build-v1"); - hasher.frame("source-id", &source.0); - hasher.frame("semantic-id", &semantic.0); - hasher.finish() + Sha256Digest( + build_frames(Sha256::new(), source, semantic) + .finalize() + .into(), + ) +} + +/// The recipe of the attestation ID (§22.9). +pub fn attestation_frames(sink: S, body_without_id_json: &str) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-attestation-v1"); + framed.frame("attestation-body", body_without_id_json.as_bytes()); + framed.into_sink() } /// The attestation ID over the canonical body without its `attestation_id` /// field (§22.9). #[must_use] pub fn attestation_id(body_without_id_json: &str) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-attestation-v1"); - hasher.frame("attestation-body", body_without_id_json.as_bytes()); - hasher.finish() + Sha256Digest( + attestation_frames(Sha256::new(), body_without_id_json) + .finalize() + .into(), + ) +} + +/// The recipe of the PDF recipe ID (§19.8). +pub fn pdf_recipe_frames( + sink: S, + tex_sha256: Sha256Digest, + program_sha256: Sha256Digest, + version_stdout_sha256: Sha256Digest, + compile_argv_json: &str, + resources_json: &str, +) -> S { + let mut framed = Framed::with_sink(sink, "lexlean-pdf-recipe-v1"); + framed.frame("tex", &tex_sha256.0); + framed.frame("program", &program_sha256.0); + framed.frame("version-output", &version_stdout_sha256.0); + framed.frame("argv", compile_argv_json.as_bytes()); + framed.frame("resources", resources_json.as_bytes()); + framed.into_sink() } /// The PDF recipe ID (§19.8). @@ -208,11 +325,16 @@ pub fn pdf_recipe_id( compile_argv_json: &str, resources_json: &str, ) -> Sha256Digest { - let mut hasher = FramedHasher::new("lexlean-pdf-recipe-v1"); - hasher.frame("tex", &tex_sha256.0); - hasher.frame("program", &program_sha256.0); - hasher.frame("version-output", &version_stdout_sha256.0); - hasher.frame("argv", compile_argv_json.as_bytes()); - hasher.frame("resources", resources_json.as_bytes()); - hasher.finish() + Sha256Digest( + pdf_recipe_frames( + Sha256::new(), + tex_sha256, + program_sha256, + version_stdout_sha256, + compile_argv_json, + resources_json, + ) + .finalize() + .into(), + ) } diff --git a/crates/lexlean/src/artifact/mod.rs b/crates/lexlean/src/artifact/mod.rs index 5150aa5b0e..95d7a23c3a 100644 --- a/crates/lexlean/src/artifact/mod.rs +++ b/crates/lexlean/src/artifact/mod.rs @@ -1,6 +1,7 @@ //! Canonical artifacts: JSON form, content identity, source maps, coverage, //! and build manifests (SPEC.md §20, §21). +pub mod address; pub mod canonical_json; pub mod content_id; pub mod declaration_id; diff --git a/crates/lexlean/src/lib.rs b/crates/lexlean/src/lib.rs index 3cbedacdb0..2aff699669 100644 --- a/crates/lexlean/src/lib.rs +++ b/crates/lexlean/src/lib.rs @@ -245,8 +245,13 @@ pub fn identity_bytes<'a>(path: &str, bytes: &'a [u8]) -> std::borrow::Cow<'a, [ /// in four; this is sixteen times that, for modules that grow with the runtime. pub const READER_STACK: usize = 64 * 1024 * 1024; -/// The §11.5 tree digest of the files of the embedded tree selected by `keep`. -fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { +/// `(repository-relative path, bytes)` rows of the embedded tree, as the +/// §11.5 tree digest reads them, in path order. +pub type EmbeddedRows = Vec<(&'static str, std::borrow::Cow<'static, [u8]>)>; + +/// The rows of the embedded tree selected by `keep`, as the §11.5 tree digest +/// reads them, in path order. +fn embedded_rows(keep: impl Fn(&str) -> bool) -> EmbeddedRows { let selected: Vec<(&'static str, &'static [u8])> = embedded::FILES .iter() .filter(|(path, _)| keep(path)) @@ -274,11 +279,16 @@ fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { }); } }); - let rows: Vec<(&str, std::borrow::Cow<'static, [u8]>)> = selected + selected .iter() .zip(slots) .map(|((path, _), slot)| (*path, slot.into_inner().expect("every file is read"))) - .collect(); + .collect() +} + +/// The §11.5 tree digest of the files of the embedded tree selected by `keep`. +fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { + let rows = embedded_rows(keep); let files: Vec<(&str, &[u8])> = rows .iter() .map(|(path, bytes)| (*path, bytes.as_ref())) @@ -286,6 +296,18 @@ fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { tree_digest(&files) } +/// Which embedded files a language's compiler-semantics ID covers (§21.2): +/// language 1.0 excludes every file introduced for 1.1 or 1.2, and 1.1 every +/// file introduced for 1.2. `None` for an unsupported language. +fn semantics_selection(language: &str) -> Option bool> { + match language { + LANGUAGE_VERSION => Some(|path| !is_v1_1_file(path) && !is_v1_2_file(path)), + LANGUAGE_1_1 => Some(|path| !is_v1_2_file(path)), + LANGUAGE_1_2 => Some(|_| true), + _ => None, + } +} + /// The language-1.2 compiler-semantics ID of the full embedded tree, checked /// in for a build without the `production-data` feature, which lacks part of /// that tree and so cannot compute it (SPEC.md §8.4, §21.2). A build with the @@ -296,6 +318,18 @@ fn embedded_digest(keep: impl Fn(&str) -> bool) -> Sha256Digest { pub const LANGUAGE_1_2_SEMANTICS_ID: &str = "a82990b56aa082f57e3bf3e2d6494d75a8170f26e269448de19110f5459655ad"; +/// The `(repository-relative path, bytes)` rows of the embedded tree whose §11.5 +/// digest is `language`'s compiler-semantics ID, in path order (§21.10). +/// `None` for an unsupported language, and for language 1.2 without the +/// `production-data` feature, whose embedded tree cannot cover it. +#[must_use] +pub fn compiler_semantics_files(language: &str) -> Option { + if language == LANGUAGE_1_2 && !embedded::PRODUCTION_DATA { + return None; + } + semantics_selection(language).map(embedded_rows) +} + /// The compiler-semantics ID for one selected language. Language 1.0 excludes /// all files introduced solely for 1.1 and 1.2, and Language 1.1 excludes /// all files introduced solely for 1.2, preserving historical identities. @@ -306,13 +340,19 @@ pub fn compiler_semantics_id_for(language: &str) -> Sha256Digest { static V1_1: OnceLock = OnceLock::new(); static V1_2: OnceLock = OnceLock::new(); match language { - LANGUAGE_VERSION => { - *V1_0.get_or_init(|| embedded_digest(|path| !is_v1_1_file(path) && !is_v1_2_file(path))) + LANGUAGE_VERSION | LANGUAGE_1_1 => { + let slot = if language == LANGUAGE_VERSION { + &V1_0 + } else { + &V1_1 + }; + *slot.get_or_init(|| { + embedded_digest(semantics_selection(language).expect("a supported language")) + }) } - LANGUAGE_1_1 => *V1_1.get_or_init(|| embedded_digest(|path| !is_v1_2_file(path))), LANGUAGE_1_2 => *V1_2.get_or_init(|| { if embedded::PRODUCTION_DATA { - embedded_digest(|_| true) + embedded_digest(semantics_selection(language).expect("a supported language")) } else { Sha256Digest::from_hex(LANGUAGE_1_2_SEMANTICS_ID) .expect("RP-10 parses the checked-in language-1.2 ID") diff --git a/features/suites/artifacts.feature b/features/suites/artifacts.feature index 0c78d9cfa2..b0fd52c515 100644 --- a/features/suites/artifacts.feature +++ b/features/suites/artifacts.feature @@ -105,3 +105,10 @@ Feature: artifacts When the verified root and the content-addressed build directory are each listed Then the verified set contains a `.olean` file, `attestation.json`, and a `process/lean/` record And the build set contains no `.olean`, no `process/` entry, no `attestation.json`, and no `.pdf` + + @AR-15 @build + Scenario: Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused. + Given every committed project with a build manifest and every committed Rust package + When the canonical bytes of each recorded identity's kind are hashed with SHA-256 + Then each digest equals the identity the artifact records + And a copy whose source changed while it kept the committed manifest has its source and build IDs named diff --git a/model/ids.toml b/model/ids.toml index 98dab75fec..6bde4bcb49 100644 --- a/model/ids.toml +++ b/model/ids.toml @@ -1225,6 +1225,12 @@ level = "build" suite = "artifacts" statement = "Platform-independent build evidence is distinguished from platform-bound oleans, process records, PDF bytes, and attestations." +[[id]] +id = "AR-15" +level = "build" +suite = "artifacts" +statement = "Every content identity a committed artifact records, the source, semantic, build, and compiler-semantics IDs of each build manifest and lock, each lock workspace file, each committed build output, and each Rust package's files and program, is the SHA-256 of the canonical bytes of its kind, so its UOR address is `sha256:` and that digest, and a recorded identity that no longer names its artifact is refused." + [[id]] id = "VR-01" level = "build" diff --git a/xtask/src/spec_links.rs b/xtask/src/spec_links.rs index f97eb8c07c..83dab19361 100644 --- a/xtask/src/spec_links.rs +++ b/xtask/src/spec_links.rs @@ -61,8 +61,8 @@ fn parse_table(spec: &str) -> Result, Fail> { pub fn validate(root: &Path) -> Result<(), Fail> { let spec = std::fs::read_to_string(root.join("SPEC.md"))?; let rows = parse_table(&spec)?; - if rows.len() != 496 { - return Err(format!("RP-07: the §31 table has {} rows, not 496", rows.len()).into()); + if rows.len() != 497 { + return Err(format!("RP-07: the §31 table has {} rows, not 497", rows.len()).into()); } // The prose total under the table is part of the contract a reader // relies on; a row added without it would leave §31 contradicting itself.