Skip to content
Open
24 changes: 21 additions & 3 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -5141,8 +5141,15 @@ certificate generator, and their shared source reader obey §17.13's exhaustiven
audit: they name every construct and match none by default.

**Statement.** The certificate of the root `r` with parameters `x₁ … xₙ` is
the module `LexLeanPreserve.C<first-32-hex-of-semantic-id>.R<i>`, `i` the
root's position in module then report order, and proves
the module `LexLeanPreserve.C<hex>.R<i>`, `i` the root's position in module
then report order and `<hex>` the first 32 hex digits of the SHA-256 of
`lexlean/certificate-name/1`, NUL, the root's module, NUL, its name, NUL, and
the bytes of its lowered program (`preserve/program/R<i>.json`): the name is a
function of what the certificates are about, so a root whose program a change
of the project leaves as it was keeps its certificates' names. Certificate A
imports the preservation library and those generated user modules whose
constants its terms name, and no other (Lean's imports are transitive, so the
modules those constants are defined from come with them). It proves

```lean
theorem root (x₁ … xₙ) :
Expand Down Expand Up @@ -9207,6 +9214,7 @@ The profile changes how processes are scheduled and nothing else:
3. Results are consumed in the canonical order and never in completion order: the members of a frontier in module-declaration order, replays in sorted full module name, audit members in the order of the audit family. The process records, the staged files, the merge of the observed axiom sets, the policy rows, and the attestation body are functions of that order. When several processes fail, the diagnostic reported is the one of the first failing process in that order, which is the diagnostic width 1 reports; no process later in that order than a known failure is started, and every process started is finished before `verify` returns. Width 1 is exactly the sequence the verifier ran before there was a width: a process starts only after the result of the one before it has been consumed, so a failure that only the consumer can see (a missing `.olean`, a staging error) stops the stage before another process starts.
4. The verifier does not measure the host. A width above 1 is for a host whose memory envelope is known; a width the host cannot hold is a failure of that host's run, never a verification result. The normative job's `examples` leg names its profile (§9.4); every other run uses the default unless its environment selects one, and the conformance cases exercise widths above 1 on small projects.
5. Hints and the memory budget are schedule inputs, never verdicts. A hint only reorders the start of items inside a stage, and the memory budget only gates a worker from starting an item whose expected peak (plus the expected peaks of the running items) would pass it. An item whose expected peak alone passes the budget runs alone rather than waiting forever: a wrong hint is a wrong schedule, never a fence and never a kill. A process that outgrows its hinted peak runs to completion; the stage measures it for the next hints. Hints change which ready item a worker starts first; results are consumed in the canonical order exactly as at width 1, so a failure reported is still the first failure in that order.
6. Certificates (§17.17, stage 12) compile as one stage with the preservation environment they import, in the topological frontiers of their import headers, like module elaboration: the certificates A and B of every root import nothing from each other and run together, at most `W` at a time, and a certificate E starts when the A and B it composes have finished. Each certificate's `leanchecker` replay runs in the slot of its compilation, right after it. The records are listed in the stage's canonical order (the environment's compilations, then each certificate's compilation and replay, in certificate order) whatever order the processes ran in. The label of a certificate's processes, for the hints, is its module name without its `C<hex>` segment, so a root's measured cost schedules its next certificates.

For every project, the directory published at any profile equals, byte for byte, the directory published at width 1 without hints and without a budget, on the same host and toolchain; in particular the attestation ID, the observed axiom sets, the policy rows, the canonical compiler input, and every normalized verification record (§29.5) are identical.

Expand All @@ -9218,6 +9226,16 @@ Wall time, resident-set and swap measurements, scheduling and completion order,

Telemetry is write-only: no stage reads it, no ID or verified artifact contains any of it, and a failure to write it changes no result. Every key it uses is in a closed vocabulary (`wall_ms`, `started_unix_ms`, `peak_rss_kib`, `peak_swap_kib`, `mem_total_kib`, `mem_available_kib`, `swap_total_kib`, `swap_used_kib`, `host_cpus`, `process_width`, `completion_seq`), and a JSON key of that vocabulary anywhere in a verified directory fails the gate: `cargo xtask verify-examples` checks every example's verified directory for one.

### 22.13 Process store

A verification process is a function of what it reads, so the verifier may take a process another run on the same host already established instead of running it again. The store is selected only by the environment variable `LEXLEAN_VERIFY_STORE`, the path of a directory; an unset or empty variable selects none. Like the resource profile (§22.11) it is operational: it is no field of `lexlean.toml`, the lock, a request, or the command line, no artifact records it, it enters no ID, and the directory a verification publishes with a store equals, byte for byte, the one it publishes without.

The content identity of a staged Lean module is the SHA-256 of the domain `lexlean/verify-store/1`, its name, the digest of its staged text, and, for each module its import header names (the `import` commands before its first other command, with their `public`, `private`, `meta` and `all` modifiers), that module's name and content identity. A module the stage does not hold is bound only when the pinned toolchain's library holds its `.olean`; it is then identified by its name, and its bytes by the toolchain digest below. A module neither holds is unbound, and so is every module whose imports reach it: a process about an unbound module is not answered by the store and is not stored.

The context of one process is the SHA-256 of the domain, the digest of the program it starts, the toolchain digest, its normalized toolchain `bin` directory and working directory (§22.7), the digest of each of `lakefile.lean`, `lakefile.toml`, `lake-manifest.json` and `lean-toolchain` in its working directory (or that it is absent), the tree digest of the working directory's `.lake` (or that it is absent), and its environment rows with normalized values. The toolchain digest is the tree digest of the toolchain directory, and a tree digest is the SHA-256 of the domain and, in path order, the relative path and digest of every file under it a `lean`, `lake` or `leanchecker` process loads: the executables under `bin`, the shared libraries, and every `.olean`, `.olean.server`, `.olean.private` and `.ir`. The traversal follows symbolic links, as the processes reading the tree do; a link that does not resolve, or a directory reached a second time, makes the tree unreadable. A process whose context cannot be read whole, or that does not run under the toolchain's home, is not answered by the store. The key of one process is the SHA-256 of the domain, its tool, the digest of its executable, the digest of the content identity of the module (or, for an audit member, the extraction driver and the interface probe, of its source) it is about together with its context, and its normalized argument vector (§22.7). The elaborations (§22.3), the replays (§22.4), the audit members (§22.5), the interface probe (§18.8), the named-root extraction (§22.10), and the compilations and replays of the preservation environment and the certificates (§17.17) go through the store.

Only a process that exited 0 is stored, with its normalized record and every file it wrote beside its `.olean` (the companions of that file's stem). A stored process is used only when its record names this run's digest of the executable and every stored file has the digest it was stored with; its files are restored to where the process writes them and its record is judged as the stage judges its own (§22.3 to §22.6), so a stored process that a stage would refuse is refused. Anything else, an unreadable or partial entry included, is a miss and the process runs. A failure to store is dropped, since the store never decides a result. Telemetry (§22.12) reports a stored process as a `reused` event of its stage.

---

## 23. Command-line interface
Expand All @@ -9238,7 +9256,7 @@ Global options:
--help
```

No environment variable changes semantic configuration. Four operational variables exist and change no output: `LEXLEAN_VERIFY_WIDTH` selects the verification process width, `LEXLEAN_VERIFY_HINTS` names the scheduling-hints file and `LEXLEAN_VERIFY_MEMORY_KIB` the process memory budget (§22.11), and `LEXLEAN_CI_TELEMETRY` names the directory that receives performance telemetry (§22.12).
No environment variable changes semantic configuration. Five operational variables exist and change no output: `LEXLEAN_VERIFY_WIDTH` selects the verification process width, `LEXLEAN_VERIFY_HINTS` names the scheduling-hints file and `LEXLEAN_VERIFY_MEMORY_KIB` the process memory budget (§22.11), `LEXLEAN_CI_TELEMETRY` names the directory that receives performance telemetry (§22.12), and `LEXLEAN_VERIFY_STORE` names the process store (§22.13).

### 23.2 Project discovery

Expand Down
8 changes: 1 addition & 7 deletions crates/conformance/src/cases/bootstrap/calculus_check.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1382,13 +1382,7 @@ fn signature(operation: &Prim, pool: &[Ty], seeded: &mut Seeded) -> Vec<Ty> {
Prim::Join => vec![list_of(string()), string()],
Prim::FormatDecimal | Prim::Convert { .. } => vec![seeded.pick(pool).clone()],
Prim::ParseDecimal { .. } => vec![string()],
Prim::Update => vec![
bytes(),
nat(),
Ty::Fixed {
width: IntKind::U8,
},
],
Prim::Update => vec![bytes(), nat(), Ty::Fixed { width: IntKind::U8 }],
}
}

Expand Down
2 changes: 1 addition & 1 deletion crates/lexlean/src/calculus/interp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ fn primitive(operation: &Prim, values: &[Rv]) -> Out {
usize::try_from(*i)
.ok()
.filter(|at| *at < a.len())
.and_then(|at| u8::try_from(*byte).ok().map(|value| (at, value)))
.zip(u8::try_from(*byte).ok())
.map(|(at, value)| {
let mut out = a.clone();
out[at] = value;
Expand Down
8 changes: 6 additions & 2 deletions crates/lexlean/src/calculus/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,12 @@ pub enum Prim {
Join,
FormatDecimal,
Compare,
Convert { target: IntKind },
ParseDecimal { target: Ty },
Convert {
target: IntKind,
},
ParseDecimal {
target: Ty,
},
/// One byte of a byte string replaced: `none` when the position is out
/// of range (language 1.2).
Update,
Expand Down
5 changes: 4 additions & 1 deletion crates/lexlean/src/ir/semantic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9864,7 +9864,10 @@ mod tests {
};
let rejected = parse(&module("lexlean/semantic-module/1"), "1.1")
.expect_err("a language-1.1 module has no evaluation proof");
assert!(format!("{rejected:?}").contains("evaluation proof"), "{rejected:?}");
assert!(
format!("{rejected:?}").contains("evaluation proof"),
"{rejected:?}"
);
parse(&module("lexlean/semantic-module/2"), "1.2")
.expect("a language-1.2 theorem may be closed by evaluation");
}
Expand Down
19 changes: 13 additions & 6 deletions crates/lexlean/src/production/certificate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,13 @@ impl Certificate {
pub fn released(mut self) -> Self {
self.text = String::new();
self.source = None;
if let Some(data) = &mut self.data {
data.nodes = Vec::new();
data.proof_nodes = Vec::new();
match &mut self.data {
Some(data) => {
data.nodes = Vec::new();
data.proof_nodes = Vec::new();
}
// A certificate without data has no expression DAG to release.
None => {}
}
self
}
Expand Down Expand Up @@ -566,9 +570,12 @@ pub fn certificate(
generator
.within(source.len(), "certificate A")
.map_err(failure)?;
let mut imports: Vec<String> = modules
.values()
.map(|linked| linked.lean_module.to_owned())
let users: BTreeSet<&str> = modules.values().map(|linked| linked.lean_module).collect();
let mut imports: Vec<String> = data
.imports
.iter()
.filter(|import| users.contains(import.as_str()))
.cloned()
.collect();
imports.sort();
Ok(Certificate {
Expand Down
43 changes: 42 additions & 1 deletion crates/lexlean/src/production/certificate/data_a.rs
Original file line number Diff line number Diff line change
Expand Up @@ -630,10 +630,51 @@ impl Gen<'_> {
.map(|linked| linked.lean_module)
.collect();
users.sort_unstable();
let users_all: BTreeSet<String> = users.iter().map(|module| (*module).to_owned()).collect();
imports.extend(users.into_iter().map(str::to_owned));
let mut writer = Writer::new(self, name);
writer.write(root_module, root_name, report, &graph)?;
writer.finish(imports)
let mut data = writer.finish(imports)?;
// A certificate imports the user modules whose constants its terms
// name, and no other: Lean's imports are transitive, so the modules
// those constants are defined from come with them, and a module the
// root's closure never reaches is not among what the certificate
// reads (§17.17, §22.13).
let named: BTreeSet<&str> = data
.nodes
.iter()
.filter_map(|node| match node {
crate::ir::core::CoreNode::Const { n, .. } => Some(n.as_str()),
crate::ir::core::CoreNode::BVar { .. }
| crate::ir::core::CoreNode::Sort { .. }
| crate::ir::core::CoreNode::App { .. }
| crate::ir::core::CoreNode::Lambda { .. }
| crate::ir::core::CoreNode::Forall { .. }
| crate::ir::core::CoreNode::Let { .. }
| crate::ir::core::CoreNode::Nat { .. }
| crate::ir::core::CoreNode::String { .. }
| crate::ir::core::CoreNode::Mdata { .. }
| crate::ir::core::CoreNode::Projection { .. } => None,
})
.collect();
let owner = |constant: &str| -> Option<String> {
users_all
.iter()
.filter(|module| {
constant.len() > module.len()
&& constant.starts_with(module.as_str())
&& constant.as_bytes()[module.len()] == b'.'
})
.max_by_key(|module| module.len())
.cloned()
};
let reached: BTreeSet<String> = named
.iter()
.filter_map(|constant| owner(constant))
.collect();
data.imports
.retain(|import| !users_all.contains(import) || reached.contains(import));
Ok(data)
}
}

Expand Down
14 changes: 8 additions & 6 deletions crates/lexlean/src/production/certificate/data_a/prove.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1386,12 +1386,14 @@ impl<'a> Writer<'_, 'a> {
self.b.konst("Bool.false_ne_true")
};
let lemma = self.b.konst1(if polarity { "if_pos" } else { "if_neg" });
let fits_eq = self
.b
.apps(lemma, &[condition, decide, witness, bool, then_fits, else_fits]);
let src_eq = self
.b
.apps(lemma, &[condition, decide, witness, lean, then_src, else_src]);
let fits_eq = self.b.apps(
lemma,
&[condition, decide, witness, bool, then_fits, else_fits],
);
let src_eq = self.b.apps(
lemma,
&[condition, decide, witness, lean, then_src, else_src],
);
let selected_fits = self.ite(bool, side, then_fits, else_fits);
let selected_src = self.ite(lean, side, then_src, else_src);
let selected_value = self.b.app(encoder, selected_src);
Expand Down
13 changes: 10 additions & 3 deletions crates/lexlean/src/verify/leanchecker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use camino::Utf8Path;
use crate::code;
use crate::config::Limits;
use crate::diagnostic::Diagnostic;
use crate::verify::child::{run, ChildHome, ChildRecord, ChildSpec, Normalizer};
use crate::verify::child::{ChildHome, ChildRecord, ChildSpec, Normalizer};
use crate::verify::toolchain::Toolchain;

/// Run `lake env <leanchecker> <module>` (the preflighted executable by
Expand All @@ -21,9 +21,11 @@ pub fn run_leanchecker(
workspace: &Utf8Path,
limits: &Limits,
normalizer: &Normalizer,
session: Option<&crate::verify::store::Session>,
) -> Result<ChildRecord, Diagnostic> {
let bin_dir = toolchain.root.join("bin");
run(
crate::verify::store::run(
session,
&ChildSpec {
tool: "leanchecker",
module: Some(module.to_owned()),
Expand All @@ -45,6 +47,8 @@ pub fn run_leanchecker(
},
limits,
normalizer,
|session| session.prefix(module),
&[],
)
}

Expand All @@ -57,8 +61,11 @@ pub fn replay_module(
workspace: &Utf8Path,
limits: &Limits,
normalizer: &Normalizer,
session: Option<&crate::verify::store::Session>,
) -> Result<ChildRecord, Diagnostic> {
let record = run_leanchecker(toolchain, module, lean_path, workspace, limits, normalizer)?;
let record = run_leanchecker(
toolchain, module, lean_path, workspace, limits, normalizer, session,
)?;
if record.exit_code != 0 {
return Err(Diagnostic::new(
code!("LLV7003"),
Expand Down
Loading
Loading