diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index e6dafbcc2..479f84d12 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -70,7 +70,7 @@ boundaries above remain the target modular MSA architecture. | `tepp_simulation` | known-truth temporal/event data generation | | `validation_core` | RMSE, bias, coverage, graph, Monte Carlo, and exact-head claim-promotion metrics | | `tepp_api` | versioned DTO, schema, terminal-result, and export contracts | -| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution and digest-bound terminal artifacts | +| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution, digest-bound terminal artifacts, and GAP-003A scientific-acceptance validation runs (`tepp.scientific_acceptance.v1`; not implemented-main) | | `episode_membership` | event-time episode membership containment gate | | `prompt_source` | prompt boilerplate is not unique latent content and not stopword deletion | | `corpus_background` | corpus-background wording is not unique latent content and not stopword deletion | @@ -113,7 +113,7 @@ boundaries above remain the target modular MSA architecture. | `episode_membership` | episode membership cannot escape the episode event-time interval | | `membership_target` | language, episode, template, department, and opportunity-pool targets cannot collapse into entity or project | | `topic_measurement` | logistic-normal ALR/ILR coordinates and the CPU `f64` TRSL-TM reference estimator | -| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution and digest-bound terminal artifacts | +| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution, digest-bound terminal artifacts, and GAP-003A scientific-acceptance validation runs (`tepp.scientific_acceptance.v1`; not implemented-main) | | `psychometric_core` | posterior-aware structural input gates, CWC within/between OLS plus the contextual effect, event-time log-rate, unequal-interval discrete-lag remapping, constant-predictor discrete effect, time-varying-predictor discrete effect (Eq. 14), exact scalar discrete process noise (Driver et al., 2017, Eq. 3), lagged latent covariance and unconditional latent variance (Driver et al., 2017, Eq. 3–4), stationary within-subject variance (Driver et al., 2017, Eq. 4 as `Δt → ∞`; `asymDIFFUSION`), trait-plus-state variance (Driver et al., 2017, §4.3 `TRAITVAR`; not process noise), observed-indicator variance and lagged observed covariance (Driver et al., 2017, Eq. 5; Table 2 `MANIFESTVAR` is `Θ`, not `Var(y)`; `MANIFESTTRAITVAR` is not `MANIFESTVAR`; `Θ` does not enter lagged observed covariance; observed-indicator mean is `τ + λ μ`; `MANIFESTMEANS` is not `E(y)`; `CINT` is not `MANIFESTMEANS`; discrete latent mean is `exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ`; `T0MEANS` is not `μ_t`; `CINT` is not the discrete increment; evolved observed mean is `τ + λ μ_t`; `τ + λ μ_0` is not `E(y_t)`; contemporaneous `TDPREDEFFECT` impulse is `m x`, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that contemporaneous impulse is `τ + λ(μ_t + m x)`, and `τ + λ μ_t` is not that observed mean; time-independent `TIPREDEFFECT` increment is `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, not `M x`, not Voelkle Eq. 14, and not the coefficient `B`; Eq. 5 of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`; within-interval `TDPREDEFFECT` carry is `e^{A(t−u)} M x` for `t0 < u < t`, not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that carried observed mean when `u ≠ t`; first-occasion `T0TIPREDEFFECT` shift is `t0_b z` and Eq. 3 first-summand carry is `e^{A Δt} t0_b z` (`T0TIPREDEFFECT` is not `TIPREDEFFECT` `B`; `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`; `e^{A Δt} t0_b z` is not `t0_b z`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_b z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean), first-occasion `T0TDPREDEFFECT` shift is `t0_m x0` and Eq. 3 first-summand carry is `e^{A Δt} t0_m x0` (`T0TDPREDEFFECT` is not `TDPREDEFFECT` `M`; `t0_m x0` is not `M x`; `e^{A Δt} t0_m x0` is not `t0_m x0`; `e^{A Δt} t0_m x0` is not `e^{A(t−u)} M x` for `t0 < u < t`; `t0_m x0` is not `t0_b z`; an impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean; `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean; §7.2 level-change `CINT` is `κ = −a m x` with `a < 0` so `−κ / a = m x` (`−a m x` is not the dissipating Dirac, not a free `CINT`, not `TIPREDEFFECT`, and not the extra near-zero-drift latent process also named in §7.2; Eq. 3 of that setting is `(1 − e^{a Δt}) m x`, which is not `m x`, not `κ`, and not `TIPREDEFFECT`; §7.2 extra-process contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`; identification `TDPREDEFFECT` on the extra process is 1; printed extra `DRIFT` is `−0.000001`; not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`; `ε ≥ 0` fails closed; Eq. 5 of that contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)`; the extra process has `LAMBDA` 0 and is not an observed indicator; `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; the contribution is not `E(y_t)`; the evolved-plus-contribution latent mean is not `E(y_t)`; after-t0 extra-process `TDPREDEFFECT` is `a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)` for `t0 < u < t` while `μ_t` uses `Δt`; Eq. 5 of that after-t0 contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)`; the first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`; `e^{a(t−u)} m x` is a Dirac on the original process, not this `DRIFT` drive; §7.2 `asymTIPREDEFFECT` is `-B z / a` for `a < 0` (`-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`; §7.2 `addedTIPREDVAR` is `(B / a)² v`, not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`; Table 2 `asymCINT` is `-κ / a` for `a < 0` and is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`; p. 16 stationary `T0MEANS` is `-κ / a + −B z / a` and is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean; Eq. 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)`; `τ + λ μ_0` is not that observed mean; `τ + λ(−κ / a)` is not that observed mean when `B z ≠ 0`; `τ + λ μ_t` is not that observed mean; `MANIFESTMEANS` is not `E(y_0)`; the constrained latent mean is not `E(y_0)`; stationary `T0VAR` is `trait + −q / (2 a) + (B / a)² v` (not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`); lagged stationary `T0VAR` is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (trait and `addedTIPREDVAR` do not decay; contemporaneous `T0VAR` is not that lagged map; decaying the constrained total as if it were all state is not that lagged map; Eq. 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`; `Θ` does not enter; contemporaneous `Var(y_0)` is not that lagged observed covariance; the lagged latent covariance is not that observed covariance); later-occasion stationary `T0VAR` is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (trait and `addedTIPREDVAR` do not enter `Q_Δt`; under stationarity that composition equals contemporaneous `T0VAR`; evolving the constrained total as if it were all state is not that later map; the lagged covariance omits `Q_Δt`; `Q_Δt` is not that later map; Eq. 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`; lagged observed covariance omits `Q_Δt` and `θ`; `MANIFESTVAR` is not `Var(y_t)`; the later-occasion latent variance is not `Var(y_t)`))), irregular already-centered residual lag, Rubin `T` on OLS loadings, and strong-gated latent means (two-observation residual variance is identically `0` and caps at strong/scalar; Putnick & Bornstein, 2016) | | `validation_core` | RMSE, bias, coverage, graph, and Monte Carlo metrics | | `tepp_api` | versioned DTO, schema, and export contracts | diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..ccf1bacdb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- `analysis_engine` GAP-003A first slice (ADR 0026, active-PR, not implemented-main): `submit_validation_run` binds cutoff-eligible evidence identities, tenant workspace, snapshot, knowledge cutoff, CPU `f64` model, seed, backend, precision, output profile, and a pre-registered SE-gate multiplier `k` (`0 ≤ k ≤ MAX_SE_GATE_K` = 8) to a hash-stable `tepp-validation-{32 hex}` receipt that carries no scientific metrics. Receipt fields are private. `complete_validation_run` requires recovery vectors stamped to that receipt with the same `k`, records a SHA-256 of those vectors, and emits `tepp.scientific_acceptance.v1` with RMSE, bias, Wilson coverage, temporal-order accuracy, and an SE-aware gate through `validation_core` using the submitted `k`. LLM-authored recovery, a different run/tenant/seed/evidence set, a post-hoc or oversized `k`, a tampered output profile, oversized/empty/mismatched vectors, non-finite inputs, empty or duplicate evidence, snapshot mismatch, and cutoff-empty corpora fail closed. Postgres persistence remains GAP-003B. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..feb37e96e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -82,6 +82,7 @@ dependencies = [ "tepp_api", "topic_measurement", "uuid", + "validation_core", ] [[package]] diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 6fa4b9683..0d18e0e6f 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -58,6 +58,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Episode-membership identity doctoring | [`docs/research/episode-membership-identity.md`](docs/research/episode-membership-identity.md) | | Entity/project target SQL doctoring | [`docs/research/entity-project-sql.md`](docs/research/entity-project-sql.md) | | Scientific claim-promotion gate doctoring | [`docs/research/scientific-claim-promotion-gates.md`](docs/research/scientific-claim-promotion-gates.md) | +| Validation-run scientific-acceptance doctoring | [`docs/research/validation-run-scientific-acceptance.md`](docs/research/validation-run-scientific-acceptance.md) | | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) | | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) | | Relation absence is not negative evidence | [`docs/research/relation-absence-not-negative.md`](docs/research/relation-absence-not-negative.md) | @@ -147,6 +148,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Episode-membership identity doctoring | [`docs/research/episode-membership-identity.md`](docs/research/episode-membership-identity.md) | | Entity/project target SQL doctoring | [`docs/research/entity-project-sql.md`](docs/research/entity-project-sql.md) | | Scientific claim-promotion gate doctoring | [`docs/research/scientific-claim-promotion-gates.md`](docs/research/scientific-claim-promotion-gates.md) | +| Validation-run scientific-acceptance doctoring | [`docs/research/validation-run-scientific-acceptance.md`](docs/research/validation-run-scientific-acceptance.md) | | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) | | Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) | | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) | diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..0d384448b 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -22,6 +22,7 @@ tepp_api = { path = "../tepp_api", version = "0.2.0" } temporal_core = { path = "../temporal_core", version = "0.2.0" } topic_measurement = { path = "../topic_measurement", version = "0.2.0" } uuid.workspace = true +validation_core = { path = "../validation_core", version = "0.2.0" } [dev-dependencies] corpus_split = { path = "../corpus_split", version = "0.2.0" } diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..d0d544ba4 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -14,6 +14,7 @@ mod case_deletion_refit; mod lineage_criterion; mod topic_context_posterior; mod topic_lineage_artifact; +mod validation_run; use serde::Serialize; use sha2::{Digest, Sha256}; @@ -26,6 +27,7 @@ use tepp_api::{ ApiError, }; use topic_measurement::TopicMeasurementError; +use validation_core::ValidationError; /// One document admitted to exhaustive case-deletion fitting. pub use case_deletion_refit::CaseDeletionDocument; @@ -58,6 +60,14 @@ pub use topic_lineage_artifact::{ TOPIC_LINEAGE_MODEL_CONTRACT_VERSION, TOPIC_LINEAGE_OUTPUT_PROFILE, TopicLineageArtifact, TopicLineageArtifactEdge, TopicLineageExecution, execute_topic_lineage_run, }; +/// Durable scientific-acceptance validation-run contracts. +pub use validation_run::{ + MAX_RECOVERY_VECTOR_LEN, MAX_SE_GATE_K, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + ScientificAcceptanceEvidence, VALIDATION_BACKEND, VALIDATION_CPU_F64_MODEL, + VALIDATION_PRECISION, VALIDATION_RUN_ID_HEX_LEN, VALIDATION_RUN_ID_PREFIX, + ValidationRunReceipt, WILSON_Z, complete_validation_run, submit_validation_run, +}; /// Versioned artifact schema emitted by this engine. pub const ANALYSIS_ARTIFACT_SCHEMA_VERSION: &str = "tepp.temporal_evidence_readiness.v1"; @@ -248,6 +258,16 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A `validation_core` metric or report rejected the recovery vectors. + Validation(ValidationError), + /// Recovery was authored by an LLM and cannot become scientific evidence. + LlmAuthoredRecovery, + /// The corpus contained evidence, but none was available at the cutoff. + NoEligibleEvidence, + /// A completion receipt did not match the rebound scientific binding. + BindingMismatch, + /// The request did not select the scientific-acceptance profile and model. + InvalidValidationProfile, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +282,13 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::Validation(error) => return error.fmt(formatter), + Self::LlmAuthoredRecovery => { + "llm-authored recovery is not scientific acceptance evidence" + } + Self::NoEligibleEvidence => "no eligible evidence for validation run", + Self::BindingMismatch => "validation run binding mismatch", + Self::InvalidValidationProfile => "invalid scientific acceptance profile", }; formatter.write_str(message) } @@ -281,6 +308,12 @@ impl From for AnalysisEngineError { } } +impl From for AnalysisEngineError { + fn from(error: ValidationError) -> Self { + Self::Validation(error) + } +} + /// Execute the cutoff-safe temporal evidence readiness analysis. /// /// Evidence whose `available_time` is later than the request cutoff is excluded @@ -402,7 +435,7 @@ fn format_digest(digest: impl AsRef<[u8]>) -> String { output } -fn valid_identifier(value: &str) -> bool { +pub(crate) fn valid_identifier(value: &str) -> bool { !value.trim().is_empty() && value.len() <= MAX_ANALYSIS_IDENTIFIER_BYTES && !value.chars().any(char::is_control) @@ -681,6 +714,26 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::Validation(validation_core::ValidationError::InvalidInput), + "invalid validation input", + ), + ( + AnalysisEngineError::LlmAuthoredRecovery, + "llm-authored recovery is not scientific acceptance evidence", + ), + ( + AnalysisEngineError::NoEligibleEvidence, + "no eligible evidence for validation run", + ), + ( + AnalysisEngineError::BindingMismatch, + "validation run binding mismatch", + ), + ( + AnalysisEngineError::InvalidValidationProfile, + "invalid scientific acceptance profile", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); @@ -689,6 +742,9 @@ mod tests { assert_eq!(converted.to_string(), "invalid API wire payload"); let from_topic: AnalysisEngineError = TopicMeasurementError::DidNotConverge.into(); assert_eq!(from_topic.to_string(), "topic estimator did not converge"); + let from_validation: AnalysisEngineError = + validation_core::ValidationError::InvalidInput.into(); + assert_eq!(from_validation.to_string(), "invalid validation input"); assert_eq!( add_membership_count(u64::MAX, 1), Err(AnalysisEngineError::ArithmeticOverflow) diff --git a/crates/analysis_engine/src/validation_run.rs b/crates/analysis_engine/src/validation_run.rs new file mode 100644 index 000000000..2a31ae445 --- /dev/null +++ b/crates/analysis_engine/src/validation_run.rs @@ -0,0 +1,1223 @@ +//! Bind immutable evidence to a durable scientific-acceptance validation run. +//! +//! This is the first GAP-003A slice: cutoff-safe evidence, tenant workspace, +//! output profile, model, seed, backend, precision, and the SE-gate multiplier +//! hash to one durable run identity. The accepted receipt carries no scientific +//! metrics. Completion asks `validation_core` for RMSE, bias, coverage, +//! temporal-order accuracy, and an SE-aware gate, then emits +//! `tepp.scientific_acceptance.v1`. Recovery vectors must be stamped with that +//! same run identity and the pre-registered multiplier; a different run, model, +//! snapshot, seed, tenant, profile, eligible evidence set, or post-hoc `k` +//! fails closed. LLM-authored recovery, non-finite inputs, empty or duplicate +//! evidence, snapshot mismatch, oversized recovery, oversized `k`, and +//! cutoff-empty corpora fail closed. Postgres persistence remains GAP-003B. + +use crate::{AnalysisCorpus, AnalysisEngineError, valid_identifier}; +use serde::Serialize; +use sha2::{Digest, Sha256}; +use std::collections::BTreeSet; +use std::fmt::Write as _; +use temporal_core::KnowledgeCutoff; +use tepp_api::{AnalysisRunRequest, ApiError}; +use validation_core::{ + ValidationError, ValidationReport, accept_within_standard_errors, bias_standard_error, + interval_coverage, mean_bias, rmse_standard_error, root_mean_square_error, + temporal_order_accuracy, wilson_coverage_interval, +}; + +/// Versioned scientific-acceptance artifact schema. +pub const SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION: &str = "tepp.scientific_acceptance.v1"; +/// Output profile that selects this validation-run executor. +pub const SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE: &str = "scientific_acceptance_v1"; +/// CPU `f64` reference model identity for scientific acceptance. +pub const VALIDATION_CPU_F64_MODEL: &str = "validation_cpu_f64_v1"; +/// Backend identity bound into the durable run. +pub const VALIDATION_BACKEND: &str = "cpu"; +/// Numeric precision bound into the durable run. +pub const VALIDATION_PRECISION: &str = "f64"; +/// Prefix of the hash-stable durable run identity. +pub const VALIDATION_RUN_ID_PREFIX: &str = "tepp-validation-"; +/// Hex characters taken from the binding digest for `run_id`. +pub const VALIDATION_RUN_ID_HEX_LEN: usize = 32; +/// Wilson critical value for nominal 95% coverage bounds. +pub const WILSON_Z: f64 = 1.96; +/// Maximum length of one recovery, interval, or event-time vector. +pub const MAX_RECOVERY_VECTOR_LEN: usize = 10_000; +/// Largest finite SE-gate multiplier that may be pre-registered on a run. +/// +/// Conventional three-SE gates sit inside this bound. A larger `k` would make +/// `|RMSE| ≤ k · SE(RMSE)` an effectively unlimited post-hoc acceptance rule. +pub const MAX_SE_GATE_K: f64 = 8.0; + +/// Durable identity of one submitted validation run. Receipts never carry +/// scientific metrics. Fields are private so callers cannot rewrite the +/// binding or the pre-registered SE-gate multiplier after submit. +#[derive(Clone, Debug, PartialEq, Serialize)] +pub struct ValidationRunReceipt { + run_id: String, + binding_sha256: String, + tenant_workspace_id: String, + snapshot_id: String, + knowledge_cutoff: String, + model: String, + seed: u64, + backend: String, + precision: String, + output_profile: String, + eligible_evidence_count: u64, + se_gate_k: f64, +} + +impl ValidationRunReceipt { + /// Serialize the receipt after confirming it carries no metrics. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::SerializationFailure`] when JSON encoding + /// fails. + pub fn to_json(&self) -> Result { + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure) + } + + /// Return the durable run identity. + #[must_use] + pub fn run_id(&self) -> &str { + &self.run_id + } + + /// Return the canonical binding digest. + #[must_use] + pub fn binding_sha256(&self) -> &str { + &self.binding_sha256 + } + + /// Return the bound tenant workspace identity. + #[must_use] + pub fn tenant_workspace_id(&self) -> &str { + &self.tenant_workspace_id + } + + /// Return the bound snapshot identity. + #[must_use] + pub fn snapshot_id(&self) -> &str { + &self.snapshot_id + } + + /// Return the bound knowledge cutoff. + #[must_use] + pub fn knowledge_cutoff(&self) -> &str { + &self.knowledge_cutoff + } + + /// Return the bound model identity. + #[must_use] + pub fn model(&self) -> &str { + &self.model + } + + /// Return the bound seed. + #[must_use] + pub const fn seed(&self) -> u64 { + self.seed + } + + /// Return the bound backend. + #[must_use] + pub fn backend(&self) -> &str { + &self.backend + } + + /// Return the bound precision. + #[must_use] + pub fn precision(&self) -> &str { + &self.precision + } + + /// Return the bound output profile. + #[must_use] + pub fn output_profile(&self) -> &str { + &self.output_profile + } + + /// Return the eligible evidence count. + #[must_use] + pub const fn eligible_evidence_count(&self) -> u64 { + self.eligible_evidence_count + } + + /// Return the pre-registered SE-gate multiplier. + #[must_use] + pub const fn se_gate_k(&self) -> f64 { + self.se_gate_k + } + + fn identity_record(&self) -> String { + format!( + "{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{:016x}", + self.run_id, + self.binding_sha256, + self.tenant_workspace_id, + self.snapshot_id, + self.knowledge_cutoff, + self.model, + self.seed, + self.backend, + self.precision, + self.output_profile, + self.eligible_evidence_count, + self.se_gate_k.to_bits() + ) + } +} + +/// Known-truth recovery vectors offered to complete a validation run. +#[derive(Clone, Debug, PartialEq)] +pub struct RecoveryObservation { + run_id: String, + binding_sha256: String, + study_label: String, + truth: Vec, + recovered: Vec, + interval_lower: Vec, + interval_upper: Vec, + truth_times: Vec, + recovered_times: Vec, + se_gate_k: f64, + authored_by_llm: bool, +} + +impl RecoveryObservation { + /// Construct a recovery observation stamped to one submitted receipt. + /// + /// LLM authorship is recorded here and refused at completion. The SE-gate + /// multiplier must equal the pre-registered receipt value; a post-hoc `k` + /// is a binding mismatch. Empty, length-mismatched, or oversized vectors + /// fail closed immediately. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidEvidence`] for an empty, oversized, + /// or control-bearing study label or for empty/mismatched vectors, + /// [`AnalysisEngineError::LimitExceeded`] when any vector exceeds + /// [`MAX_RECOVERY_VECTOR_LEN`], [`AnalysisEngineError::BindingMismatch`] + /// when `se_gate_k` differs from the receipt, and + /// [`AnalysisEngineError::Validation`] for a non-finite or out-of-policy SE + /// multiplier. + #[allow(clippy::too_many_arguments)] + pub fn new( + receipt: &ValidationRunReceipt, + study_label: impl Into, + truth: Vec, + recovered: Vec, + interval_lower: Vec, + interval_upper: Vec, + truth_times: Vec, + recovered_times: Vec, + se_gate_k: f64, + authored_by_llm: bool, + ) -> Result { + let study_label = study_label.into(); + if !valid_identifier(&study_label) { + return Err(AnalysisEngineError::InvalidEvidence); + } + let lengths = [ + truth.len(), + recovered.len(), + interval_lower.len(), + interval_upper.len(), + truth_times.len(), + recovered_times.len(), + ]; + if lengths.iter().any(|len| *len > MAX_RECOVERY_VECTOR_LEN) { + return Err(AnalysisEngineError::LimitExceeded); + } + let n = truth.len(); + if n == 0 || lengths.iter().any(|len| *len != n) { + return Err(AnalysisEngineError::InvalidEvidence); + } + let se_gate_k = require_se_gate_k(se_gate_k)?; + if se_gate_k.to_bits() != receipt.se_gate_k.to_bits() { + return Err(AnalysisEngineError::BindingMismatch); + } + Ok(Self { + run_id: receipt.run_id.clone(), + binding_sha256: receipt.binding_sha256.clone(), + study_label, + truth, + recovered, + interval_lower, + interval_upper, + truth_times, + recovered_times, + se_gate_k, + authored_by_llm, + }) + } + + /// Return the stamped run identity. + #[must_use] + pub fn run_id(&self) -> &str { + &self.run_id + } + + /// Return the stamped binding digest. + #[must_use] + pub fn binding_sha256(&self) -> &str { + &self.binding_sha256 + } + + /// Return the study label. + #[must_use] + pub fn study_label(&self) -> &str { + &self.study_label + } + + /// Return the truth vector. + #[must_use] + pub fn truth(&self) -> &[f64] { + &self.truth + } + + /// Return the recovered vector. + #[must_use] + pub fn recovered(&self) -> &[f64] { + &self.recovered + } + + /// Return interval lower bounds. + #[must_use] + pub fn interval_lower(&self) -> &[f64] { + &self.interval_lower + } + + /// Return interval upper bounds. + #[must_use] + pub fn interval_upper(&self) -> &[f64] { + &self.interval_upper + } + + /// Return truth event times. + #[must_use] + pub fn truth_times(&self) -> &[f64] { + &self.truth_times + } + + /// Return recovered event times. + #[must_use] + pub fn recovered_times(&self) -> &[f64] { + &self.recovered_times + } + + /// Return the SE-gate multiplier. + #[must_use] + pub const fn se_gate_k(&self) -> f64 { + self.se_gate_k + } + + /// Return whether an LLM authored the recovery. + #[must_use] + pub const fn authored_by_llm(&self) -> bool { + self.authored_by_llm + } + + fn digest_hex(&self) -> String { + let mut canonical = String::from("tepp.recovery_observation.v1\n"); + let _ = writeln!(canonical, "run_id={}", self.run_id); + let _ = writeln!(canonical, "binding={}", self.binding_sha256); + let _ = writeln!(canonical, "study={}", self.study_label); + let _ = writeln!(canonical, "se_gate_k={:016x}", self.se_gate_k.to_bits()); + let _ = writeln!(canonical, "authored_by_llm={}", self.authored_by_llm); + append_f64_vector(&mut canonical, "truth", &self.truth); + append_f64_vector(&mut canonical, "recovered", &self.recovered); + append_f64_vector(&mut canonical, "interval_lower", &self.interval_lower); + append_f64_vector(&mut canonical, "interval_upper", &self.interval_upper); + append_f64_vector(&mut canonical, "truth_times", &self.truth_times); + append_f64_vector(&mut canonical, "recovered_times", &self.recovered_times); + format_hex(Sha256::digest(canonical.into_bytes())) + } +} + +/// Operator-usable scientific acceptance evidence for one completed run. +#[derive(Clone, Debug, PartialEq, Serialize)] +pub struct ScientificAcceptanceEvidence { + schema_version: String, + run_id: String, + binding_sha256: String, + recovery_sha256: String, + tenant_workspace_id: String, + snapshot_id: String, + knowledge_cutoff: String, + model: String, + seed: u64, + backend: String, + precision: String, + output_profile: String, + eligible_evidence_count: u64, + se_gate_accepted: bool, + se_gate_k: f64, + report: ValidationReport, +} + +impl ScientificAcceptanceEvidence { + /// Serialize canonical evidence JSON after report validation. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::Validation`] when a report field is + /// non-finite, or [`AnalysisEngineError::SerializationFailure`] when JSON + /// encoding fails. + pub fn to_json(&self) -> Result { + self.report.validate()?; + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure) + } + + /// Return the lowercase SHA-256 digest of the canonical evidence JSON. + /// + /// # Errors + /// + /// Returns the same failures as [`Self::to_json`]. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_hex(Sha256::digest(json.into_bytes()))) + } + + /// Render the nested recovery summary line. + #[must_use] + pub fn to_human_summary(&self) -> String { + self.report.to_human_summary() + } + + /// Return the artifact schema version. + #[must_use] + pub fn schema_version(&self) -> &str { + &self.schema_version + } + + /// Return the durable run identity. + #[must_use] + pub fn run_id(&self) -> &str { + &self.run_id + } + + /// Return the scientific binding digest. + #[must_use] + pub fn binding_sha256(&self) -> &str { + &self.binding_sha256 + } + + /// Return the recovery-vector digest stamped into this artifact. + #[must_use] + pub fn recovery_sha256(&self) -> &str { + &self.recovery_sha256 + } + + /// Return the tenant workspace identity. + #[must_use] + pub fn tenant_workspace_id(&self) -> &str { + &self.tenant_workspace_id + } + + /// Return the snapshot identity. + #[must_use] + pub fn snapshot_id(&self) -> &str { + &self.snapshot_id + } + + /// Return the knowledge cutoff. + #[must_use] + pub fn knowledge_cutoff(&self) -> &str { + &self.knowledge_cutoff + } + + /// Return the bound model identity. + #[must_use] + pub fn model(&self) -> &str { + &self.model + } + + /// Return the bound seed. + #[must_use] + pub const fn seed(&self) -> u64 { + self.seed + } + + /// Return the bound backend. + #[must_use] + pub fn backend(&self) -> &str { + &self.backend + } + + /// Return the bound precision. + #[must_use] + pub fn precision(&self) -> &str { + &self.precision + } + + /// Return the output profile. + #[must_use] + pub fn output_profile(&self) -> &str { + &self.output_profile + } + + /// Return the eligible evidence count. + #[must_use] + pub const fn eligible_evidence_count(&self) -> u64 { + self.eligible_evidence_count + } + + /// Return whether the SE-aware gate accepted RMSE toward 0. + #[must_use] + pub const fn se_gate_accepted(&self) -> bool { + self.se_gate_accepted + } + + /// Return the SE-gate multiplier used for acceptance. + #[must_use] + pub const fn se_gate_k(&self) -> f64 { + self.se_gate_k + } +} + +struct CanonicalBinding { + tenant_workspace_id: String, + snapshot_id: String, + knowledge_cutoff: String, + model: String, + seed: u64, + backend: String, + precision: String, + output_profile: String, + se_gate_k: f64, + eligible_ids: Vec, +} + +impl CanonicalBinding { + fn digest_hex(&self) -> String { + format_hex(Sha256::digest(self.canonical_bytes())) + } + + fn run_id(digest_hex: &str) -> String { + let mut run_id = + String::with_capacity(VALIDATION_RUN_ID_PREFIX.len() + VALIDATION_RUN_ID_HEX_LEN); + run_id.push_str(VALIDATION_RUN_ID_PREFIX); + run_id.push_str(&digest_hex[..VALIDATION_RUN_ID_HEX_LEN]); + run_id + } + + fn eligible_count(&self) -> u64 { + self.eligible_ids.len() as u64 + } + + fn canonical_bytes(&self) -> String { + let mut canonical = String::from("tepp.validation_binding.v1\n"); + let _ = writeln!(canonical, "tenant={}", self.tenant_workspace_id); + let _ = writeln!(canonical, "snapshot={}", self.snapshot_id); + let _ = writeln!(canonical, "cutoff={}", self.knowledge_cutoff); + let _ = writeln!(canonical, "model={}", self.model); + let _ = writeln!(canonical, "seed={}", self.seed); + let _ = writeln!(canonical, "backend={}", self.backend); + let _ = writeln!(canonical, "precision={}", self.precision); + let _ = writeln!(canonical, "profile={}", self.output_profile); + let _ = writeln!(canonical, "se_gate_k={:016x}", self.se_gate_k.to_bits()); + for identity in &self.eligible_ids { + let _ = writeln!(canonical, "evidence={identity}"); + } + canonical + } + + fn identity_record(&self, digest_hex: &str, run_id: &str) -> String { + format!( + "{run_id}\n{digest_hex}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{:016x}", + self.tenant_workspace_id, + self.snapshot_id, + self.knowledge_cutoff, + self.model, + self.seed, + self.backend, + self.precision, + self.output_profile, + self.eligible_count(), + self.se_gate_k.to_bits() + ) + } +} + +/// Submit immutable evidence to a durable validation run. +/// +/// The receipt is hash-stable for one scientific binding and never includes +/// RMSE, bias, coverage, or gate results. +/// +/// # Errors +/// +/// Returns a fail-closed engine error for an invalid request, wrong output +/// profile or model, an out-of-policy SE-gate multiplier, snapshot mismatch, +/// duplicate or empty evidence, or a cutoff that admits no evidence. +pub fn submit_validation_run( + request: &AnalysisRunRequest, + corpus: &AnalysisCorpus, + seed: u64, + se_gate_k: f64, +) -> Result { + let binding = bind_validation_run(request, corpus, seed, se_gate_k)?; + let binding_sha256 = binding.digest_hex(); + let run_id = CanonicalBinding::run_id(&binding_sha256); + let eligible_evidence_count = binding.eligible_count(); + Ok(ValidationRunReceipt { + run_id, + binding_sha256, + tenant_workspace_id: binding.tenant_workspace_id, + snapshot_id: binding.snapshot_id, + knowledge_cutoff: binding.knowledge_cutoff, + model: binding.model, + seed: binding.seed, + backend: binding.backend, + precision: binding.precision, + output_profile: binding.output_profile, + eligible_evidence_count, + se_gate_k: binding.se_gate_k, + }) +} + +/// Complete a previously submitted validation run with known-truth recovery. +/// +/// # Errors +/// +/// Returns a fail-closed engine error when recovery was LLM-authored, the +/// observation is stamped to a different run identity, the receipt does not +/// match the rebound scientific identity, metric inputs are invalid, or +/// report validation fails. +pub fn complete_validation_run( + receipt: &ValidationRunReceipt, + request: &AnalysisRunRequest, + corpus: &AnalysisCorpus, + observation: &RecoveryObservation, +) -> Result { + if observation.authored_by_llm { + return Err(AnalysisEngineError::LlmAuthoredRecovery); + } + if observation.run_id != receipt.run_id + || observation.binding_sha256 != receipt.binding_sha256 + || observation.se_gate_k.to_bits() != receipt.se_gate_k.to_bits() + || receipt.output_profile != SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::BindingMismatch); + } + let binding = bind_validation_run(request, corpus, receipt.seed, receipt.se_gate_k)?; + let binding_sha256 = binding.digest_hex(); + let run_id = CanonicalBinding::run_id(&binding_sha256); + if receipt.identity_record() != binding.identity_record(&binding_sha256, &run_id) { + return Err(AnalysisEngineError::BindingMismatch); + } + let report = compute_report(observation)?; + let se_gate_accepted = accept_within_standard_errors( + report.rmse, + 0.0, + report.rmse_standard_error, + receipt.se_gate_k, + )?; + let eligible_evidence_count = binding.eligible_count(); + let evidence = ScientificAcceptanceEvidence { + schema_version: SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION.to_owned(), + run_id, + binding_sha256, + recovery_sha256: observation.digest_hex(), + tenant_workspace_id: binding.tenant_workspace_id, + snapshot_id: binding.snapshot_id, + knowledge_cutoff: binding.knowledge_cutoff, + model: binding.model, + seed: binding.seed, + backend: binding.backend, + precision: binding.precision, + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.to_owned(), + eligible_evidence_count, + se_gate_accepted, + se_gate_k: receipt.se_gate_k, + report, + }; + evidence.to_json()?; + Ok(evidence) +} + +fn require_se_gate_k(se_gate_k: f64) -> Result { + if !se_gate_k.is_finite() { + return Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput, + )); + } + if se_gate_k < 0.0 || se_gate_k > MAX_SE_GATE_K { + return Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration, + )); + } + Ok(if se_gate_k == 0.0 { 0.0 } else { se_gate_k }) +} + +fn bind_validation_run( + request: &AnalysisRunRequest, + corpus: &AnalysisCorpus, + seed: u64, + se_gate_k: f64, +) -> Result { + request.to_json()?; + let se_gate_k = require_se_gate_k(se_gate_k)?; + let requested = format!( + "{}\n{}", + request.output_profile, request.model_contract_version + ); + let expected = format!("{SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE}\n{VALIDATION_CPU_F64_MODEL}"); + if requested != expected { + return Err(AnalysisEngineError::InvalidValidationProfile); + } + if request.snapshot_id != corpus.snapshot_id() { + return Err(AnalysisEngineError::SnapshotMismatch); + } + let cutoff = KnowledgeCutoff::parse_rfc3339(&request.knowledge_cutoff) + .map_err(|_| AnalysisEngineError::Api(ApiError::InvalidWirePayload))?; + let mut identities = BTreeSet::new(); + let mut eligible = BTreeSet::new(); + for unit in corpus.evidence_units() { + if !identities.insert(unit.evidence_id()) { + return Err(AnalysisEngineError::DuplicateEvidence); + } + if unit.available_time().instant() <= cutoff.instant() { + eligible.insert(unit.evidence_id().to_owned()); + } + } + if identities.is_empty() { + return Err(AnalysisEngineError::InvalidEvidence); + } + if eligible.is_empty() { + return Err(AnalysisEngineError::NoEligibleEvidence); + } + Ok(CanonicalBinding { + tenant_workspace_id: request.tenant_workspace_id.clone(), + snapshot_id: request.snapshot_id.clone(), + knowledge_cutoff: cutoff.to_rfc3339(), + model: VALIDATION_CPU_F64_MODEL.to_owned(), + seed, + backend: VALIDATION_BACKEND.to_owned(), + precision: VALIDATION_PRECISION.to_owned(), + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.to_owned(), + se_gate_k, + eligible_ids: eligible.into_iter().collect(), + }) +} + +fn compute_report( + observation: &RecoveryObservation, +) -> Result { + let rmse = root_mean_square_error(&observation.truth, &observation.recovered)?; + let rmse_standard_error = rmse_standard_error(&observation.truth, &observation.recovered)?; + let mean_bias = mean_bias(&observation.truth, &observation.recovered)?; + let bias_standard_error = bias_standard_error(&observation.truth, &observation.recovered)?; + let interval_coverage = interval_coverage( + &observation.truth, + &observation.interval_lower, + &observation.interval_upper, + )?; + let (coverage_wilson_lower, coverage_wilson_upper) = wilson_coverage_interval( + &observation.truth, + &observation.interval_lower, + &observation.interval_upper, + WILSON_Z, + )?; + let temporal_order_accuracy = + temporal_order_accuracy(&observation.truth_times, &observation.recovered_times)?; + let report = ValidationReport { + study_label: observation.study_label.clone(), + rmse, + rmse_standard_error, + mean_bias, + bias_standard_error, + interval_coverage, + coverage_wilson_lower, + coverage_wilson_upper, + temporal_order_accuracy, + monte_carlo_rmse: None, + }; + report.validate()?; + Ok(report) +} + +fn append_f64_vector(canonical: &mut String, label: &str, values: &[f64]) { + let _ = writeln!(canonical, "{label}_len={}", values.len()); + for value in values { + let _ = writeln!(canonical, "{label}={:016x}", value.to_bits()); + } +} + +fn format_hex(digest: impl AsRef<[u8]>) -> String { + let bytes = digest.as_ref(); + let mut output = String::with_capacity(bytes.len() * 2); + for byte in bytes { + let _ = write!(output, "{byte:02x}"); + } + output +} + +#[cfg(test)] +mod tests { + use super::{ + MAX_RECOVERY_VECTOR_LEN, MAX_SE_GATE_K, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + VALIDATION_BACKEND, VALIDATION_CPU_F64_MODEL, VALIDATION_PRECISION, + VALIDATION_RUN_ID_PREFIX, WILSON_Z, complete_validation_run, submit_validation_run, + }; + use crate::{ + AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, MAX_ANALYSIS_IDENTIFIER_BYTES, + }; + use temporal_core::{AvailableTime, EventTime}; + use tepp_api::AnalysisRunRequest; + use validation_core::ValidationError; + + fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "idem-validation-1".into(), + tenant_workspace_id: "tenant-workspace-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: VALIDATION_CPU_F64_MODEL.into(), + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.into(), + } + } + + fn unit(id: &str, available: &str) -> AnalysisEvidenceUnit { + AnalysisEvidenceUnit::new( + id, + EventTime::parse_rfc3339("2026-07-01T00:00:00Z").expect("event"), + AvailableTime::parse_rfc3339(available).expect("available"), + 1, + ) + .expect("unit") + } + + fn corpus(units: Vec) -> AnalysisCorpus { + AnalysisCorpus::new("snapshot-1", units).expect("corpus") + } + + fn recovery( + receipt: &super::ValidationRunReceipt, + truth: Vec, + recovered: Vec, + k: f64, + ) -> RecoveryObservation { + let times = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]; + let times = times[..truth.len()].to_vec(); + let lower: Vec = truth.iter().map(|value| value - 0.5).collect(); + let upper: Vec = truth.iter().map(|value| value + 0.5).collect(); + RecoveryObservation::new( + receipt, + "foundation-recovery", + truth, + recovered, + lower, + upper, + times.clone(), + times, + k, + false, + ) + .expect("observation") + } + + fn two_point( + receipt: &super::ValidationRunReceipt, + label: impl Into, + k: f64, + authored_by_llm: bool, + ) -> Result { + RecoveryObservation::new( + receipt, + label, + vec![1.0, 2.0], + vec![1.0, 2.0], + vec![0.0, 1.0], + vec![2.0, 3.0], + vec![1.0, 2.0], + vec![1.0, 2.0], + k, + authored_by_llm, + ) + } + + #[test] + fn submit_is_hash_stable_cutoff_safe_and_metric_free() { + let first = corpus(vec![ + unit("evidence-b", "2026-07-15T00:00:00Z"), + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("late", "2026-08-02T00:00:00Z"), + ]); + let second = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt_a = submit_validation_run(&request(), &first, 7, 3.0).expect("submit a"); + let receipt_b = submit_validation_run(&request(), &second, 7, 3.0).expect("submit b"); + assert_eq!(receipt_a.run_id(), receipt_b.run_id()); + assert_eq!(receipt_a.binding_sha256(), receipt_b.binding_sha256()); + assert!(receipt_a.run_id().starts_with(VALIDATION_RUN_ID_PREFIX)); + assert_eq!( + receipt_a.run_id().len(), + VALIDATION_RUN_ID_PREFIX.len() + 32 + ); + assert_eq!(receipt_a.eligible_evidence_count(), 2); + assert_eq!(receipt_a.tenant_workspace_id(), "tenant-workspace-1"); + assert_eq!(receipt_a.snapshot_id(), "snapshot-1"); + assert_eq!(receipt_a.knowledge_cutoff(), "2026-08-01T00:00:00Z"); + assert_eq!(receipt_a.model(), VALIDATION_CPU_F64_MODEL); + assert_eq!(receipt_a.seed(), 7); + assert_eq!(receipt_a.backend(), VALIDATION_BACKEND); + assert_eq!(receipt_a.precision(), VALIDATION_PRECISION); + assert_eq!( + receipt_a.output_profile(), + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + ); + assert!((receipt_a.se_gate_k() - 3.0).abs() < f64::EPSILON); + let json = receipt_a.to_json().expect("json"); + assert!(!json.contains("rmse")); + assert!(!json.contains("bias")); + assert!(!json.contains("coverage")); + let other_seed = submit_validation_run(&request(), &first, 8, 3.0).expect("seed"); + assert_ne!(receipt_a.run_id(), other_seed.run_id()); + let mut other_tenant = request(); + other_tenant.tenant_workspace_id = "tenant-workspace-2".into(); + let other_tenant_receipt = + submit_validation_run(&other_tenant, &first, 7, 3.0).expect("tenant"); + assert_ne!(receipt_a.run_id(), other_tenant_receipt.run_id()); + let other_k = submit_validation_run(&request(), &first, 7, 4.0).expect("other k"); + assert_ne!(receipt_a.run_id(), other_k.run_id()); + assert!((other_k.se_gate_k() - 4.0).abs() < f64::EPSILON); + } + + #[test] + fn complete_emits_operator_usable_scientific_acceptance_evidence() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 11, 3.0).expect("submit"); + let observation = recovery( + &receipt, + vec![0.70, 0.55, 0.40, -0.20, 0.85], + vec![0.70, 0.55, 0.40, -0.20, 0.85], + 3.0, + ); + assert_eq!(observation.run_id(), receipt.run_id()); + assert_eq!(observation.binding_sha256(), receipt.binding_sha256()); + assert_eq!(observation.study_label(), "foundation-recovery"); + assert_eq!(observation.truth().len(), 5); + assert_eq!(observation.recovered().len(), 5); + assert_eq!(observation.interval_lower().len(), 5); + assert_eq!(observation.interval_upper().len(), 5); + assert_eq!(observation.truth_times().len(), 5); + assert_eq!(observation.recovered_times().len(), 5); + assert!((observation.se_gate_k() - 3.0).abs() < f64::EPSILON); + assert!(!observation.authored_by_llm()); + let evidence = complete_validation_run(&receipt, &request(), &offered, &observation) + .expect("complete"); + assert_eq!( + evidence.schema_version(), + SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION + ); + assert_eq!(evidence.run_id(), receipt.run_id()); + assert_eq!(evidence.binding_sha256(), receipt.binding_sha256()); + assert_eq!(evidence.recovery_sha256().len(), 64); + assert_eq!(evidence.tenant_workspace_id(), "tenant-workspace-1"); + assert!(evidence.se_gate_accepted()); + assert_eq!( + evidence.output_profile(), + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + ); + assert_eq!(evidence.eligible_evidence_count(), 2); + assert!((evidence.se_gate_k() - 3.0).abs() < f64::EPSILON); + assert_eq!(evidence.snapshot_id(), "snapshot-1"); + assert_eq!(evidence.knowledge_cutoff(), "2026-08-01T00:00:00Z"); + assert_eq!(evidence.model(), VALIDATION_CPU_F64_MODEL); + assert_eq!(evidence.seed(), 11); + assert_eq!(evidence.backend(), VALIDATION_BACKEND); + assert_eq!(evidence.precision(), VALIDATION_PRECISION); + assert!((WILSON_Z - 1.96).abs() < f64::EPSILON); + let json = evidence.to_json().expect("json"); + assert!(json.contains(SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION)); + assert!(json.contains("rmse")); + assert!(json.contains("recovery_sha256")); + assert_eq!(evidence.sha256().expect("digest").len(), 64); + assert!(evidence.to_human_summary().contains("foundation-recovery")); + let rejected = recovery( + &receipt, + vec![0.0, 1.0, 2.0, 3.0, 4.0], + vec![10.0, 11.0, 12.0, 13.0, 14.0], + 3.0, + ); + let refused = + complete_validation_run(&receipt, &request(), &offered, &rejected).expect("refused"); + assert!(!refused.se_gate_accepted()); + assert_ne!(refused.recovery_sha256(), evidence.recovery_sha256()); + } + + #[test] + fn submit_and_complete_fail_closed_on_trust_and_shape_errors() { + let offered = corpus(vec![unit("evidence-a", "2026-07-10T00:00:00Z")]); + let mut wrong_profile = request(); + wrong_profile.output_profile = "validation-report".into(); + assert_eq!( + submit_validation_run(&wrong_profile, &offered, 1, 3.0), + Err(AnalysisEngineError::InvalidValidationProfile) + ); + let mut wrong_model = request(); + wrong_model.model_contract_version = "temporal-evidence-v1".into(); + assert_eq!( + submit_validation_run(&wrong_model, &offered, 1, 3.0), + Err(AnalysisEngineError::InvalidValidationProfile) + ); + let mismatched = AnalysisCorpus::new( + "snapshot-other", + vec![unit("evidence-a", "2026-07-10T00:00:00Z")], + ) + .expect("other"); + assert_eq!( + submit_validation_run(&request(), &mismatched, 1, 3.0), + Err(AnalysisEngineError::SnapshotMismatch) + ); + let duplicate = corpus(vec![ + unit("same", "2026-07-10T00:00:00Z"), + unit("same", "2026-07-11T00:00:00Z"), + ]); + assert_eq!( + submit_validation_run(&request(), &duplicate, 1, 3.0), + Err(AnalysisEngineError::DuplicateEvidence) + ); + let empty = AnalysisCorpus::new("snapshot-1", Vec::new()).expect("empty"); + assert_eq!( + submit_validation_run(&request(), &empty, 1, 3.0), + Err(AnalysisEngineError::InvalidEvidence) + ); + let late = corpus(vec![unit("late", "2026-08-02T00:00:00Z")]); + assert_eq!( + submit_validation_run(&request(), &late, 1, 3.0), + Err(AnalysisEngineError::NoEligibleEvidence) + ); + let mut invalid_request = request(); + invalid_request.idempotency_key.clear(); + assert!(matches!( + submit_validation_run(&invalid_request, &offered, 1, 3.0), + Err(AnalysisEngineError::Api(_)) + )); + let mut invalid_cutoff = request(); + invalid_cutoff.knowledge_cutoff = "not-a-time".into(); + assert!(matches!( + submit_validation_run(&invalid_cutoff, &offered, 1, 3.0), + Err(AnalysisEngineError::Api(_)) + )); + } + + #[test] + fn complete_refuses_llm_authored_recovery_and_tampered_receipt() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 3, 3.0).expect("submit"); + let llm = RecoveryObservation::new( + &receipt, + "llm-study", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + true, + ) + .expect("llm"); + assert!(llm.authored_by_llm()); + assert_eq!( + complete_validation_run(&receipt, &request(), &offered, &llm), + Err(AnalysisEngineError::LlmAuthoredRecovery) + ); + let mut tampered = receipt.clone(); + tampered.run_id = "tepp-validation-deadbeefdeadbeefdeadbeefdeadbeef".into(); + let observation = recovery(&receipt, vec![1.0, 2.0, 3.0], vec![1.0, 2.0, 3.0], 3.0); + assert_eq!( + complete_validation_run(&tampered, &request(), &offered, &observation), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut profile_tampered = receipt.clone(); + profile_tampered.output_profile = "tampered-profile".into(); + assert_eq!( + complete_validation_run(&profile_tampered, &request(), &offered, &observation), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut k_tampered = observation.clone(); + k_tampered.se_gate_k = 8.0; + assert_eq!( + complete_validation_run(&receipt, &request(), &offered, &k_tampered), + Err(AnalysisEngineError::BindingMismatch) + ); + } + + #[test] + fn complete_refuses_recovery_stamped_to_a_different_run() { + let first = corpus(vec![unit("evidence-a", "2026-07-10T00:00:00Z")]); + let second = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt_a = submit_validation_run(&request(), &first, 3, 3.0).expect("a"); + let receipt_b = submit_validation_run(&request(), &second, 3, 3.0).expect("b"); + assert_ne!(receipt_a.run_id(), receipt_b.run_id()); + let foreign = recovery(&receipt_a, vec![1.0, 2.0, 3.0], vec![1.0, 2.0, 3.0], 3.0); + assert_eq!( + complete_validation_run(&receipt_b, &request(), &second, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let other_seed = submit_validation_run(&request(), &first, 9, 3.0).expect("seed"); + assert_eq!( + complete_validation_run(&other_seed, &request(), &first, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut other_tenant = request(); + other_tenant.tenant_workspace_id = "tenant-workspace-other".into(); + let other_tenant_receipt = + submit_validation_run(&other_tenant, &first, 3, 3.0).expect("tenant"); + assert_eq!( + complete_validation_run(&other_tenant_receipt, &other_tenant, &first, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + } + + #[test] + fn complete_refuses_nonfinite_recovery_and_invalid_observation() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 3, 3.0).expect("submit"); + let nan = RecoveryObservation::new( + &receipt, + "nan-study", + vec![1.0, 2.0, 3.0], + vec![f64::NAN, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + false, + ) + .expect("nan observation"); + assert_eq!( + complete_validation_run(&receipt, &request(), &offered, &nan), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput + )) + ); + assert_eq!( + two_point(&receipt, "", 3.0, false), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + two_point( + &receipt, + "x".repeat(MAX_ANALYSIS_IDENTIFIER_BYTES + 1), + 3.0, + false + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + two_point(&receipt, "nan-k", f64::NAN, false), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput + )) + ); + assert_eq!( + two_point(&receipt, "neg-k", -1.0, false), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration + )) + ); + let oversized = vec![0.0; MAX_RECOVERY_VECTOR_LEN + 1]; + assert_eq!( + RecoveryObservation::new( + &receipt, + "oversized", + oversized.clone(), + oversized.clone(), + oversized.clone(), + oversized.clone(), + oversized.clone(), + oversized, + 3.0, + false, + ), + Err(AnalysisEngineError::LimitExceeded) + ); + let converted: AnalysisEngineError = ValidationError::InvalidInput.into(); + assert_eq!(converted.to_string(), "invalid validation input"); + } + + #[test] + fn se_gate_k_is_pre_registered_and_empty_vectors_fail_closed() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 3, 3.0).expect("submit"); + assert_eq!( + two_point(&receipt, "huge-k", MAX_SE_GATE_K + 0.01, false), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration + )) + ); + assert_eq!( + two_point(&receipt, "post-hoc-k", 4.0, false), + Err(AnalysisEngineError::BindingMismatch) + ); + assert_eq!( + submit_validation_run(&request(), &offered, 3, MAX_SE_GATE_K + 0.01), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration + )) + ); + assert_eq!( + submit_validation_run(&request(), &offered, 3, f64::NAN), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput + )) + ); + assert!(submit_validation_run(&request(), &offered, 3, MAX_SE_GATE_K).is_ok()); + assert_eq!( + RecoveryObservation::new( + &receipt, + "empty", + Vec::new(), + Vec::new(), + Vec::new(), + Vec::new(), + Vec::new(), + Vec::new(), + 3.0, + false, + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + RecoveryObservation::new( + &receipt, + "mismatch", + vec![1.0, 2.0], + vec![1.0], + vec![0.0, 1.0], + vec![2.0, 3.0], + vec![1.0, 2.0], + vec![1.0, 2.0], + 3.0, + false, + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} diff --git a/crates/analysis_engine/tests/validation_run_contract.rs b/crates/analysis_engine/tests/validation_run_contract.rs new file mode 100644 index 000000000..f7a7bc7be --- /dev/null +++ b/crates/analysis_engine/tests/validation_run_contract.rs @@ -0,0 +1,168 @@ +//! GAP-003A scientific-acceptance validation-run contract. + +use analysis_engine::{ + AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + VALIDATION_CPU_F64_MODEL, complete_validation_run, submit_validation_run, +}; +use temporal_core::{AvailableTime, EventTime}; +use tepp_api::AnalysisRunRequest; + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "idem-validation-contract".into(), + tenant_workspace_id: "tenant-workspace-contract".into(), + snapshot_id: "snapshot-contract".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: VALIDATION_CPU_F64_MODEL.into(), + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.into(), + } +} + +fn unit(id: &str, available: &str) -> AnalysisEvidenceUnit { + AnalysisEvidenceUnit::new( + id, + EventTime::parse_rfc3339("2026-07-01T00:00:00Z").expect("event"), + AvailableTime::parse_rfc3339(available).expect("available"), + 1, + ) + .expect("unit") +} + +#[test] +fn durable_run_binds_cutoff_eligible_evidence_and_emits_acceptance_evidence() { + let corpus = AnalysisCorpus::new( + "snapshot-contract", + vec![ + unit("evidence-2", "2026-07-20T00:00:00Z"), + unit("evidence-1", "2026-07-10T00:00:00Z"), + unit("future", "2026-08-02T00:00:00Z"), + ], + ) + .expect("corpus"); + let receipt = submit_validation_run(&request(), &corpus, 42, 3.0).expect("submit"); + assert_eq!(receipt.eligible_evidence_count(), 2); + assert!((receipt.se_gate_k() - 3.0).abs() < f64::EPSILON); + assert!(!receipt.to_json().expect("json").contains("rmse")); + let observation = RecoveryObservation::new( + &receipt, + "contract-recovery", + vec![0.70, 0.55, 0.40, -0.20, 0.85], + vec![0.70, 0.55, 0.40, -0.20, 0.85], + vec![0.50, 0.35, 0.20, -0.40, 0.65], + vec![0.90, 0.75, 0.60, 0.00, 1.00], + vec![1.0, 2.0, 3.0, 4.0, 5.0], + vec![1.1, 1.9, 3.2, 3.8, 5.1], + 3.0, + false, + ) + .expect("observation"); + let evidence = + complete_validation_run(&receipt, &request(), &corpus, &observation).expect("complete"); + assert_eq!( + evidence.schema_version(), + SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION + ); + assert_eq!(evidence.run_id(), receipt.run_id()); + assert_eq!(evidence.recovery_sha256().len(), 64); + assert!(evidence.se_gate_accepted()); + assert!(evidence.to_json().expect("json").contains("rmse")); +} + +#[test] +fn llm_authored_recovery_and_cutoff_empty_corpora_fail_closed() { + let corpus = AnalysisCorpus::new( + "snapshot-contract", + vec![unit("only-future", "2026-08-02T00:00:00Z")], + ) + .expect("corpus"); + assert_eq!( + submit_validation_run(&request(), &corpus, 1, 3.0), + Err(AnalysisEngineError::NoEligibleEvidence) + ); + let eligible = AnalysisCorpus::new( + "snapshot-contract", + vec![unit("evidence-1", "2026-07-10T00:00:00Z")], + ) + .expect("eligible"); + let receipt = submit_validation_run(&request(), &eligible, 1, 3.0).expect("submit"); + let llm = RecoveryObservation::new( + &receipt, + "llm-contract", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + true, + ) + .expect("llm"); + assert_eq!( + complete_validation_run(&receipt, &request(), &eligible, &llm), + Err(AnalysisEngineError::LlmAuthoredRecovery) + ); +} + +#[test] +fn recovery_from_a_different_run_or_tenant_fails_closed() { + let first = AnalysisCorpus::new( + "snapshot-contract", + vec![unit("evidence-1", "2026-07-10T00:00:00Z")], + ) + .expect("first"); + let second = AnalysisCorpus::new( + "snapshot-contract", + vec![ + unit("evidence-1", "2026-07-10T00:00:00Z"), + unit("evidence-2", "2026-07-20T00:00:00Z"), + ], + ) + .expect("second"); + let receipt_a = submit_validation_run(&request(), &first, 1, 3.0).expect("a"); + let receipt_b = submit_validation_run(&request(), &second, 1, 3.0).expect("b"); + let foreign = RecoveryObservation::new( + &receipt_a, + "foreign-recovery", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + false, + ) + .expect("foreign"); + assert_eq!( + complete_validation_run(&receipt_b, &request(), &second, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut other_tenant = request(); + other_tenant.tenant_workspace_id = "tenant-workspace-other".into(); + let other_receipt = submit_validation_run(&other_tenant, &first, 1, 3.0).expect("tenant"); + assert_ne!(receipt_a.run_id(), other_receipt.run_id()); + assert_eq!( + complete_validation_run(&other_receipt, &other_tenant, &first, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let other_k = submit_validation_run(&request(), &first, 1, 4.0).expect("other k"); + assert_ne!(receipt_a.run_id(), other_k.run_id()); + assert_eq!( + RecoveryObservation::new( + &receipt_a, + "post-hoc-k", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 4.0, + false, + ), + Err(AnalysisEngineError::BindingMismatch) + ); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..2bd5f446b 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -50,6 +50,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | leakage-safe availability/cutoff snapshots | PRD; ADR 0002/0013 | `corpus_split` on protected main | implemented-main | | Unicode NFC/NFD canonical identity for split leakage | ADR 0004/0008/0013; UAX #15 | `corpus_split` canonical-equivalence links on active PR #59 | active-PR | | recovery metrics (RMSE, bias, coverage, graph, temporal order, Monte Carlo SE gates) | PRD; Test Strategy; ADR 0007/0014 | `validation_core` on protected main (PR #19); SE-aware Monte Carlo gates included | implemented-main | +| durable validation-run scientific acceptance evidence | ADR 0026; ADR 0002/0008/0014; GAP-003A; National Academies (2019); Wasserstein & Lazar (2016); Wilson (1927) | `analysis_engine` `submit_validation_run` / `complete_validation_run` on this PR: hash-stable cutoff-safe tenant/profile/`k` binding, metric-free private-field receipt, recovery stamped to that `run_id` with pre-registered `k` (`MAX_SE_GATE_K` = 8), `tepp.scientific_acceptance.v1` RMSE/bias/coverage/temporal-order/SE-aware evidence with recovery digest; LLM/NaN/empty/duplicate/snapshot/foreign-run/post-hoc-k/tampered-profile/oversized/cutoff-empty fail closed; not implemented-main; Postgres persistence remains GAP-003B | active-PR | | PostgreSQL bitemporal/lineage persistence | ADR 0013; Architecture/ERD | `persistence_postgres` migration contracts, in-memory adapters, live SQL session/document SQL port, tenant RLS (`0002` + session GUC/role helpers), `DATABASE_URL` SQLx gate, optional session-affine `live-sqlx` `PgPool` driver, exact-head live PostgreSQL CI with isolation proof, append-only immutability triggers (`0004`), temporal interval ordering CHECKs (`0005`), typed membership assignment (`0006` implemented-main), event-relation/mention/instance SQL (#37–#39 implemented-main), source-artifact SQL (#40 implemented-main), audit-event SQL (#41 implemented-main), concurrent document-write stress (#43 implemented-main), backup/restore integrity revalidation (#44 implemented-main), `revision_order` later-revision system-time ordering implemented-main, entity/project target SQL on PR #131; remaining physical ERD constraints | partial | | known-truth temporal/event simulation manifests | PRD; TRD; Test Strategy | `tepp_simulation` on protected main; recovery metrics in `validation_core` | implemented-main | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); request-bound terminal result active in PR #157; HTTP service remains accepted-target; the `orchestrator_live` loopback interpretation listener is on this PR | partial | diff --git a/docs/adr/0026-validation-run-scientific-acceptance.md b/docs/adr/0026-validation-run-scientific-acceptance.md new file mode 100644 index 000000000..8e9290a67 --- /dev/null +++ b/docs/adr/0026-validation-run-scientific-acceptance.md @@ -0,0 +1,99 @@ +# ADR 0026 — Durable validation-run scientific acceptance evidence + +**Decision status:** Accepted +**Implementation maturity:** active-PR — library-level `analysis_engine` binding on this PR; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0002, ADR 0008, ADR 0013, ADR 0014, and ADR 0022. +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +GAP-003A records that immutable evidence cannot yet be submitted to a durable +validation run that produces operator-usable scientific acceptance evidence. +`validation_core` already computes RMSE, bias, interval coverage, temporal-order +accuracy, and SE-aware gates, but those metrics remain library-level. An +accepted analysis run is a transport receipt; it must never carry scientific +results. Operators therefore have no hash-stable identity that binds cutoff-safe +evidence, model, seed, backend, and precision to one completion that emits +scientific acceptance evidence. + +Postgres persistence, restart/recovery, and Compose E2E remain GAP-003B. + +## Decision + +Add a bounded `analysis_engine` validation-run executor: + +- `submit_validation_run` binds sorted cutoff-eligible evidence identities, + tenant workspace, snapshot, knowledge cutoff, model `validation_cpu_f64_v1`, + seed, backend `cpu`, precision `f64`, output profile + `scientific_acceptance_v1`, and the SE-gate multiplier `k` into a canonical + SHA-256 digest. `k` must be finite, non-negative, and at most + `MAX_SE_GATE_K` (8). Changing `k` changes the run identity. The durable + `run_id` is `tepp-validation-{32 hex}`. The receipt carries no RMSE, bias, + coverage, or gate-result fields. Receipt fields are private. +- `complete_validation_run` rebinds the same scientific identity, requires + recovery vectors stamped to that `run_id` and binding digest with the same + pre-registered `k`, refuses LLM-authored recovery, computes + `validation_core` recovery metrics, applies the SE-aware gate + `|RMSE − 0| ≤ k · SE(RMSE)` using the submitted `k`, and emits + `tepp.scientific_acceptance.v1` under output profile + `scientific_acceptance_v1`. The artifact records a SHA-256 of the stamped + recovery vectors. Evidence fields are private after completion. +- Empty corpora, duplicate evidence identities, snapshot mismatch, invalid + profiles, non-finite inputs, oversized recovery vectors, empty or + length-mismatched recovery, a post-hoc or oversized `k`, a different run / + tenant / seed / eligible evidence set, a tampered output profile, and + cutoff-empty eligibility fail closed. +- A computed recovery that fails the SE-aware gate still emits evidence with + `se_gate_accepted = false` so operators can read the metrics. Invalid or + LLM-authored recovery never emits evidence. +- The engine does not persist rows, claim implemented-main, or replace ADR 0014 + exact-head claim promotion. + +## Alternatives considered + +1. Keep metrics library-only — rejected because GAP-003A is the operator-visible + product-completion gap and has no live implementation PR. +2. Return scientific metrics on `AnalysisRunAccepted` — rejected because + accepted/running receipts must never carry scientific results. +3. Persist the run in PostgreSQL in this slice — rejected because durable + storage, restart, and Compose recovery belong to GAP-003B / issue #287. +4. Bind evidence, cutoff, model, seed, backend, and precision in + `analysis_engine` and emit scientific acceptance evidence through + `validation_core` — accepted because it is independently testable and does + not weaken fail-closed gates. + +## Consequences + +Operators can submit one immutable evidence snapshot and later complete it with +known-truth recovery to obtain a digest-bound scientific acceptance artifact. +The run identity is hash-stable for the scientific binding, independent of +evidence input order and of ineligible post-cutoff units. Persistence, HTTP +ingress, and release promotion remain later slices. LLM output still cannot +become scientific authority. + +## Verification + +The stacked PR includes unit and integration tests for hash-stable identity, +pre-registered SE-gate `k`, cutoff exclusion, metric-free private receipts, +SE-aware accept and refuse, recovery stamped to a foreign run or tenant, a +post-hoc `k`, a tampered output profile, oversized or empty vectors, and +fail-closed LLM, NaN, empty, duplicate, snapshot, profile, and cutoff-empty +paths. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +``` + +Doctoring and APA 7th citations are in +`docs/research/validation-run-scientific-acceptance.md`. + +## Rollback and supersession + +Rollback removes `validation_run` from `analysis_engine` and stops exporting +the scientific-acceptance artifact. No persisted schema migration is +introduced. Supersession requires a new ADR if execution changes cutoff +semantics, binding identity, LLM-refusal, or scientific estimands. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..b068bad74 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0026](0026-validation-run-scientific-acceptance.md) | Durable validation-run scientific acceptance evidence | Accepted | active-PR | GAP-003A first slice binds cutoff-safe evidence to a hash-stable run that emits `tepp.scientific_acceptance.v1`; Postgres persistence remains GAP-003B. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 5ea0c95bf..c3e5a0780 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -248,7 +248,7 @@ the coordinated visual workspace, or a supported multi-tenant release. |---|---|---|---|---|---|---|---| | GAP-001 | Submission produces a durable accepted receipt, and the deterministic terminal-result lifecycle is now implemented-main. | `implemented-main` | closed on protected main | `340087494b0a` lineage merged through [PR #157](https://github.com/ContextualWisdomLab/TEPP/pull/157) (merged 2026-08-25T02:53Z); [#156](https://github.com/ContextualWisdomLab/TEPP/issues/156) CLOSED | — | — | Exact request/result/snapshot/cutoff/model/profile binding, typed terminal failures, deterministic retrieval, and cutoff-safe execution are protected-main behavior as of the #157 merge. | | GAP-002 | LineageWeave and other modular consumers can rely on the complete protected-main HTTP evidence/result boundary. | `partial` | consumer hardening remains | Terminal-result lifecycle implemented-main via #157; versioned API contract intact | [#156](https://github.com/ContextualWisdomLab/TEPP/issues/156) (closed) / [PR #155](https://github.com/ContextualWisdomLab/TEPP/pull/155) (merged) | — | Remaining work is consumer-side adoption evidence and any versioned-contract drift discovered during integration; core boundary is no longer the gap it was. | -| GAP-003A | Immutable evidence cannot yet be submitted to a durable validation run that produces operator-usable scientific acceptance evidence. | `accepted-target` | product-completion | `e65cd66` (validation metrics are library-level only) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | `—` (issue program; no current implementation PR) | Compose/CLI/API execution must bind immutable evidence, cutoffs, model configuration, validation metrics, and reproducibility manifests to one idempotent run. | +| GAP-003A | Immutable evidence cannot yet be submitted to a durable validation run that produces operator-usable scientific acceptance evidence. | `accepted-target` | product-completion | `e65cd66` (validation metrics are library-level only) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | [#356](https://github.com/ContextualWisdomLab/TEPP/pull/356) `7916603` (`feat/validation-run-scientific-acceptance`; pre-registered SE-gate `k`; not implemented-main; not Compose persistence) | Compose/CLI/API execution must bind immutable evidence, cutoffs, model configuration, validation metrics, and reproducibility manifests to one idempotent run. | | GAP-003B | Scientific result artifacts cannot yet be persisted, restarted, and recovered as one supported operator workflow. | `accepted-target` | product-completion | `e65cd66` (persistence contracts lack E2E recovery) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | `—` (issue program; no current implementation PR) | Durable storage, migration/rollback, restart/recovery, artifact digest verification, and terminal retrieval must pass against a real Compose deployment. | | GAP-003C | The persistence slice classifies concurrent-write SQLSTATEs, but has no measured hot-partition detection, routing, or mitigation for tenant/result workloads. | `accepted-target` | product-completion | `e65cd66` (conflict classification only; no measured partition control) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | `—` (issue program; no current implementation PR) | A real Compose/PostgreSQL workload identifies hot keys and partition skew, applies bounded tenant/time or result routing without weakening 3NF or temporal authority, and proves conflict rate, latency, recovery, and migration/rollback behavior under load. | | GAP-004 | The central shared-latent temporal/relational topic estimator is absent. | `partial` | product vertical | CPU `f64` TRSL-TM reference estimator with ALR/ILR coordinates and refusal gates is implemented-main (v0.2.0 `topic_measurement`); fitted candidate-`K` scoring present | [#167](https://github.com/ContextualWisdomLab/TEPP/issues/167) | — | GPU, method effects, full Bayesian sampling, and topic birth/split/merge remain. This is not full #167 closure. | diff --git a/docs/research/validation-run-scientific-acceptance.md b/docs/research/validation-run-scientific-acceptance.md new file mode 100644 index 000000000..2ce820517 --- /dev/null +++ b/docs/research/validation-run-scientific-acceptance.md @@ -0,0 +1,38 @@ +# Validation-run scientific acceptance evidence + +## Scope + +This note doctors the first GAP-003A slice in `analysis_engine`: + +1. immutable evidence, tenant workspace, knowledge cutoff, model, seed, backend, precision, output profile, and the SE-gate multiplier `k` bind to one hash-stable validation run; +2. the accepted receipt carries no scientific metrics and cannot be rewritten after submit; +3. completion emits `tepp.scientific_acceptance.v1` with RMSE, bias, interval coverage (Wilson bounds), temporal-order accuracy, an SE-aware gate using that pre-registered `k`, and a SHA-256 of the stamped recovery vectors; +4. recovery vectors must be constructed against that receipt with the same `k`; a different run, tenant, seed, snapshot, profile, eligible evidence set, or post-hoc `k` fails closed; +5. LLM-authored recovery, non-finite inputs, empty or duplicate evidence, snapshot mismatch, oversized recovery, oversized `k` (`MAX_SE_GATE_K` = 8), and cutoff-empty corpora fail closed. + +Postgres persistence, restart/recovery, and Compose execution remain GAP-003B. This slice is not implemented-main. + +## Authoritative sources + +National Academies of Sciences, Engineering, and Medicine. (2019). *Reproducibility and replicability in science*. The National Academies Press. https://doi.org/10.17226/25303 + +Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on *p*-values: Context, process, and purpose. *The American Statistician, 70*(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108 + +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. *Journal of the American Statistical Association, 22*(158), 209–212. https://doi.org/10.1080/01621459.1927.10502953 + +## Application + +The National Academies (2019) separate computational reproducibility (same binding, same digest) from a scientific claim that recovery is correct. Wasserstein and Lazar (2016) refuse to treat a passing threshold as automatic scientific authority and refuse post-hoc threshold shopping, so `k` is pre-registered in the run binding and a failed SE-aware gate still reports metrics with `se_gate_accepted = false` rather than inventing a passing claim. Wilson (1927) supplies the coverage interval bounds already implemented in `validation_core`. TEPP therefore binds cutoff-safe evidence, tenant/profile identity, and `k` before any metric is computed, stamps recovery vectors to that binding, refuses LLM-authored or post-hoc-`k` recovery, and reports RMSE, bias, coverage, temporal order, and the SE-aware gate as operator-usable evidence (National Academies of Sciences, Engineering, and Medicine, 2019; Wasserstein & Lazar, 2016; Wilson, 1927). Meredith (1993) remains unread (Unpaywall/OpenAlex 2026-08-31T07:45Z: `is_oa: false`, 0 locations). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). + +## Verification + +- identical eligible evidence in any order and with extra post-cutoff units yields the same `tepp-validation-{32 hex}` identity; +- distinct SE-gate multipliers yield distinct run identities; `k > 8` fails closed; +- receipts serialize without RMSE, bias, or coverage fields; +- known-truth recovery emits `tepp.scientific_acceptance.v1` with a digest-stable JSON body; +- a large residual vector remains operator-readable with `se_gate_accepted = false`; +- recovery stamped to a different run, tenant, seed, or post-hoc `k` returns `BindingMismatch`; +- a tampered receipt `output_profile` returns `BindingMismatch`; +- an oversized recovery vector returns `LimitExceeded`; +- empty or length-mismatched recovery vectors return `InvalidEvidence`; +- LLM authorship, NaN recovery, empty corpora, duplicates, snapshot mismatch, wrong profile/model, and cutoff-empty eligibility return dedicated fail-closed errors.