diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index e6dafbcc2..a027f4e3e 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -70,7 +70,7 @@ boundaries above remain the target modular MSA architecture. | `tepp_simulation` | known-truth temporal/event data generation | | `validation_core` | RMSE, bias, coverage, graph, Monte Carlo, and exact-head claim-promotion metrics | | `tepp_api` | versioned DTO, schema, terminal-result, and export contracts | -| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution and digest-bound terminal artifacts | +| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution, digest-bound terminal artifacts, GAP-003A scientific-acceptance validation runs (`tepp.scientific_acceptance.v1`; not implemented-main), and loopback `POST /v1/analysis-runs/{run_id}/execute` that produces that artifact without a caller-supplied payload | | `episode_membership` | event-time episode membership containment gate | | `prompt_source` | prompt boilerplate is not unique latent content and not stopword deletion | | `corpus_background` | corpus-background wording is not unique latent content and not stopword deletion | @@ -113,7 +113,7 @@ boundaries above remain the target modular MSA architecture. | `episode_membership` | episode membership cannot escape the episode event-time interval | | `membership_target` | language, episode, template, department, and opportunity-pool targets cannot collapse into entity or project | | `topic_measurement` | logistic-normal ALR/ILR coordinates and the CPU `f64` TRSL-TM reference estimator | -| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution and digest-bound terminal artifacts | +| `analysis_engine` | bounded cutoff-safe temporal evidence readiness execution, digest-bound terminal artifacts, GAP-003A scientific-acceptance validation runs (`tepp.scientific_acceptance.v1`; not implemented-main), and loopback `POST /v1/analysis-runs/{run_id}/execute` that produces that artifact without a caller-supplied payload | | `psychometric_core` | posterior-aware structural input gates, CWC within/between OLS plus the contextual effect, event-time log-rate, unequal-interval discrete-lag remapping, constant-predictor discrete effect, time-varying-predictor discrete effect (Eq. 14), exact scalar discrete process noise (Driver et al., 2017, Eq. 3), lagged latent covariance and unconditional latent variance (Driver et al., 2017, Eq. 3–4), stationary within-subject variance (Driver et al., 2017, Eq. 4 as `Δt → ∞`; `asymDIFFUSION`), trait-plus-state variance (Driver et al., 2017, §4.3 `TRAITVAR`; not process noise), observed-indicator variance and lagged observed covariance (Driver et al., 2017, Eq. 5; Table 2 `MANIFESTVAR` is `Θ`, not `Var(y)`; `MANIFESTTRAITVAR` is not `MANIFESTVAR`; `Θ` does not enter lagged observed covariance; observed-indicator mean is `τ + λ μ`; `MANIFESTMEANS` is not `E(y)`; `CINT` is not `MANIFESTMEANS`; discrete latent mean is `exp(a Δt) μ_0 + (exp(a Δt) − 1)/a κ`; `T0MEANS` is not `μ_t`; `CINT` is not the discrete increment; evolved observed mean is `τ + λ μ_t`; `τ + λ μ_0` is not `E(y_t)`; contemporaneous `TDPREDEFFECT` impulse is `m x`, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that contemporaneous impulse is `τ + λ(μ_t + m x)`, and `τ + λ μ_t` is not that observed mean; time-independent `TIPREDEFFECT` increment is `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, not `M x`, not Voelkle Eq. 14, and not the coefficient `B`; Eq. 5 of that increment is `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; `τ + λ(μ_t + e^{a(t−u)} m x)` is not that observed mean when `u ≠ t`; within-interval `TDPREDEFFECT` carry is `e^{A(t−u)} M x` for `t0 < u < t`, not the contemporaneous Dirac, not `CINT`, not `TIPREDEFFECT`, and not Voelkle Eq. 14; Eq. 5 of that carry is `τ + λ(μ_t + e^{a(t−u)} m x)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that carried observed mean when `u ≠ t`; first-occasion `T0TIPREDEFFECT` shift is `t0_b z` and Eq. 3 first-summand carry is `e^{A Δt} t0_b z` (`T0TIPREDEFFECT` is not `TIPREDEFFECT` `B`; `t0_b z` is not `A^{-1}[e^{A Δt} − I] B z`; `e^{A Δt} t0_b z` is not `t0_b z`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_b z)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean), first-occasion `T0TDPREDEFFECT` shift is `t0_m x0` and Eq. 3 first-summand carry is `e^{A Δt} t0_m x0` (`T0TDPREDEFFECT` is not `TDPREDEFFECT` `M`; `t0_m x0` is not `M x`; `e^{A Δt} t0_m x0` is not `t0_m x0`; `e^{A Δt} t0_m x0` is not `e^{A(t−u)} M x` for `t0 < u < t`; `t0_m x0` is not `t0_b z`; an impulse at `u ≤ t0` that used `M` is already in `η(t0)` as `TDPREDEFFECT`, not as `T0TDPREDEFFECT`; Eq. 5 of that carry is `τ + λ(μ_t + e^{a Δt} t0_m x0)`, and `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + A^{-1}[e^{A Δt} − I] B z)` is not that observed mean; `τ + λ(μ_t + e^{a Δt} t0_b z)` is not that observed mean; §7.2 level-change `CINT` is `κ = −a m x` with `a < 0` so `−κ / a = m x` (`−a m x` is not the dissipating Dirac, not a free `CINT`, not `TIPREDEFFECT`, and not the extra near-zero-drift latent process also named in §7.2; Eq. 3 of that setting is `(1 − e^{a Δt}) m x`, which is not `m x`, not `κ`, and not `TIPREDEFFECT`; §7.2 extra-process contribution is `a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)` (`ε = a` is `a_{ηξ} x Δt e^{a Δt}`; identification `TDPREDEFFECT` on the extra process is 1; printed extra `DRIFT` is `−0.000001`; not `κ = −a m x`, not `(1 − e^{a Δt}) m x`, and not the dissipating Dirac `m x`; `ε ≥ 0` fails closed; Eq. 5 of that contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε Δt} − e^{a Δt}) / (ε − a)`; the extra process has `LAMBDA` 0 and is not an observed indicator; `τ + λ μ_t` is not that observed mean; `τ + λ(μ_t + m x)` is not that observed mean; the contribution is not `E(y_t)`; the evolved-plus-contribution latent mean is not `E(y_t)`; after-t0 extra-process `TDPREDEFFECT` is `a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)` for `t0 < u < t` while `μ_t` uses `Δt`; Eq. 5 of that after-t0 contribution is `τ + λ(μ_t + a_{ηξ} x (e^{ε(t−u)} − e^{a(t−u)}) / (ε − a)`; the first-occasion extra-process observed mean is not that observed mean when `u ≠ t0`; `e^{a(t−u)} m x` is a Dirac on the original process, not this `DRIFT` drive; §7.2 `asymTIPREDEFFECT` is `-B z / a` for `a < 0` (`-B z / a` is not the coefficient `B`, not `A^{-1}[e^{A Δt} − I] B z`, not `CINT`, and not `M x`; §7.2 `addedTIPREDVAR` is `(B / a)² v`, not `TRAITVAR`, not `asymDIFFUSION`, and not `-B z / a`; Table 2 `asymCINT` is `-κ / a` for `a < 0` and is not `κ`, not `A^{-1}[e^{A Δt} − I] κ`, not `T0MEANS`, and not `-B z / a`; p. 16 stationary `T0MEANS` is `-κ / a + −B z / a` and is not free `T0MEANS`, not `asymCINT` alone, not `asymTIPREDEFFECT` alone, and not the finite-interval discrete latent mean; Eq. 5 of that constrained mean is `τ + λ(−κ / a + −B z / a)`; `τ + λ μ_0` is not that observed mean; `τ + λ(−κ / a)` is not that observed mean when `B z ≠ 0`; `τ + λ μ_t` is not that observed mean; `MANIFESTMEANS` is not `E(y_0)`; the constrained latent mean is not `E(y_0)`; stationary `T0VAR` is `trait + −q / (2 a) + (B / a)² v` (not free `T0VAR`, not `asymDIFFUSION` alone, not `TRAITVAR` alone, not `addedTIPREDVAR` alone, and not the finite-interval discrete latent variance. Eq. 5 of that constrained variance is `λ²(trait + −q / (2 a) + (B / a)² v) + θ + ψ` (JSS PDF re-opened 2026-08-22T03:20Z; form the stationary latent variance first, then `λ² p + θ + ψ`; `λ² p_0` is not that observed variance; `λ²(−q / (2 a)) + θ` is not that observed variance when `TRAITVAR` or `addedTIPREDVAR` is nonzero; `MANIFESTVAR` is not `Var(y_0)`; the constrained latent variance is not `Var(y_0)`); lagged stationary `T0VAR` is `trait + e^{a Δt}(−q / (2 a)) + (B / a)² v` (trait and `addedTIPREDVAR` do not decay; contemporaneous `T0VAR` is not that lagged map; decaying the constrained total as if it were all state is not that lagged map; Eq. 5 of that lagged covariance is `λ²(trait + e^{a Δt}(−q / (2 a)) + (B / a)² v) + ψ`; `Θ` does not enter; contemporaneous `Var(y_0)` is not that lagged observed covariance; the lagged latent covariance is not that observed covariance); later-occasion stationary `T0VAR` is `trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v` (trait and `addedTIPREDVAR` do not enter `Q_Δt`; under stationarity that composition equals contemporaneous `T0VAR`; evolving the constrained total as if it were all state is not that later map; the lagged covariance omits `Q_Δt`; `Q_Δt` is not that later map; Eq. 5 of that later-occasion variance is `λ²(trait + e^{2 a Δt}(−q / (2 a)) + Q_Δt + (B / a)² v) + θ + ψ`; lagged observed covariance omits `Q_Δt` and `θ`; `MANIFESTVAR` is not `Var(y_t)`; the later-occasion latent variance is not `Var(y_t)`))), irregular already-centered residual lag, Rubin `T` on OLS loadings, and strong-gated latent means (two-observation residual variance is identically `0` and caps at strong/scalar; Putnick & Bornstein, 2016) | | `validation_core` | RMSE, bias, coverage, graph, and Monte Carlo metrics | | `tepp_api` | versioned DTO, schema, and export contracts | diff --git a/CHANGELOG.d/scientific-acceptance-loopback-engine.md b/CHANGELOG.d/scientific-acceptance-loopback-engine.md new file mode 100644 index 000000000..1a70390a2 --- /dev/null +++ b/CHANGELOG.d/scientific-acceptance-loopback-engine.md @@ -0,0 +1,3 @@ +### Added + +- `analysis_engine` GAP-003A engine-on-loopback slice (ADR 0032, active-PR, not implemented-main): `ScientificAcceptanceLoopbackService` serves `POST /v1/analysis-runs/{run_id}/execute` so a `scientific_acceptance_v1` run produces `tepp.scientific_acceptance.v1` without a caller-supplied artifact. Persistence remains GAP-003B. diff --git a/CHANGELOG.md b/CHANGELOG.md index dcbe8b94a..1dff8bc6e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,10 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- `analysis_engine` GAP-003A engine-on-loopback slice (ADR 0032, active-PR, not implemented-main): `ScientificAcceptanceLoopbackService` serves `POST /v1/analysis-runs/{run_id}/execute` so an accepted `scientific_acceptance_v1` loopback run produces `tepp.scientific_acceptance.v1` without a caller-supplied artifact. The execute body carries corpus, recovery, seed, and pre-registered SE-gate `k` and refuses `scientific_acceptance_json` plus receipt metric keys. Wrong profile, LLM recovery, unknown run, consumer mismatch, already-terminal status, and digest mismatch fail closed. This does not duplicate the engine library (#356), terminal-result DTO (#358), GET (#359), lifecycle POST (#360), cancel HTTP (#361), loopback CLI (#362), or collection GET (#368); persistence remains GAP-003B. + +- `analysis_engine` GAP-003A first slice (ADR 0026, active-PR, not implemented-main): `submit_validation_run` binds cutoff-eligible evidence identities, tenant workspace, snapshot, knowledge cutoff, CPU `f64` model, seed, backend, precision, output profile, and a pre-registered SE-gate multiplier `k` (`0 ≤ k ≤ MAX_SE_GATE_K` = 8) to a hash-stable `tepp-validation-{32 hex}` receipt that carries no scientific metrics. Receipt fields are private. `complete_validation_run` requires recovery vectors stamped to that receipt with the same `k`, records a SHA-256 of those vectors, and emits `tepp.scientific_acceptance.v1` with RMSE, bias, Wilson coverage, temporal-order accuracy, and an SE-aware gate through `validation_core` using the submitted `k`. LLM-authored recovery, a different run/tenant/seed/evidence set, a post-hoc or oversized `k`, a tampered output profile, oversized/empty/mismatched vectors, non-finite inputs, empty or duplicate evidence, snapshot mismatch, and cutoff-empty corpora fail closed. Postgres persistence remains GAP-003B. + - `tepp_api` loopback `AnalysisRunLiveService` now serves production `POST /v1/analysis-runs/{run_id}/running` and `POST /v1/analysis-runs/{run_id}/terminal` so accepted/running stay metric-free and only a succeeded status with profile `scientific_acceptance_v1` may return `tepp.scientific_acceptance.v1` after a lifecycle POST. Canonical artifact bytes travel as `scientific_acceptance_json`. Reverse transitions, mutating a terminal run, failed-plus-artifact emission, receipt RMSE/bias/coverage/SE-gate keys, an unknown run, and consumer mismatch fail closed. This is the GAP-003A HTTP lifecycle slice for issue #166; it does not duplicate the `analysis_engine` library bind (#356), the terminal-result DTO wire (#358), or the GET status slice (#359); persistence remains GAP-003B. - `tepp_api` loopback `AnalysisRunLiveService` now serves `GET /v1/analysis-runs/{run_id}` so accepted/running statuses stay metric-free and only a succeeded status with profile `scientific_acceptance_v1` may return `tepp.scientific_acceptance.v1`. Receipt RMSE/bias/coverage/SE-gate keys, a GET body, failed-plus-artifact emission, an all-zero digest, and digest mismatch fail closed. This is the GAP-003A HTTP status slice for issue #166; it does not duplicate the `analysis_engine` library bind (#356) or the terminal-result DTO wire (#358); persistence remains GAP-003B. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..feb37e96e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -82,6 +82,7 @@ dependencies = [ "tepp_api", "topic_measurement", "uuid", + "validation_core", ] [[package]] diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 6fa4b9683..a39eb96b0 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -58,6 +58,8 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Episode-membership identity doctoring | [`docs/research/episode-membership-identity.md`](docs/research/episode-membership-identity.md) | | Entity/project target SQL doctoring | [`docs/research/entity-project-sql.md`](docs/research/entity-project-sql.md) | | Scientific claim-promotion gate doctoring | [`docs/research/scientific-claim-promotion-gates.md`](docs/research/scientific-claim-promotion-gates.md) | +| Validation-run scientific-acceptance doctoring | [`docs/research/validation-run-scientific-acceptance.md`](docs/research/validation-run-scientific-acceptance.md) | +| Scientific-acceptance loopback engine doctoring | [`docs/research/scientific-acceptance-loopback-engine.md`](docs/research/scientific-acceptance-loopback-engine.md) | | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) | | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) | | Relation absence is not negative evidence | [`docs/research/relation-absence-not-negative.md`](docs/research/relation-absence-not-negative.md) | @@ -147,6 +149,8 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Episode-membership identity doctoring | [`docs/research/episode-membership-identity.md`](docs/research/episode-membership-identity.md) | | Entity/project target SQL doctoring | [`docs/research/entity-project-sql.md`](docs/research/entity-project-sql.md) | | Scientific claim-promotion gate doctoring | [`docs/research/scientific-claim-promotion-gates.md`](docs/research/scientific-claim-promotion-gates.md) | +| Validation-run scientific-acceptance doctoring | [`docs/research/validation-run-scientific-acceptance.md`](docs/research/validation-run-scientific-acceptance.md) | +| Scientific-acceptance loopback engine doctoring | [`docs/research/scientific-acceptance-loopback-engine.md`](docs/research/scientific-acceptance-loopback-engine.md) | | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) | | Stopword-deletion doctoring | [`docs/research/stopword-deletion.md`](docs/research/stopword-deletion.md) | | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) | diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..0d384448b 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -22,6 +22,7 @@ tepp_api = { path = "../tepp_api", version = "0.2.0" } temporal_core = { path = "../temporal_core", version = "0.2.0" } topic_measurement = { path = "../topic_measurement", version = "0.2.0" } uuid.workspace = true +validation_core = { path = "../validation_core", version = "0.2.0" } [dev-dependencies] corpus_split = { path = "../corpus_split", version = "0.2.0" } diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..adebea503 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -8,12 +8,17 @@ //! through [`tepp_api`]. It deliberately does not claim latent-variable or topic //! estimation authority; those estimators remain separate scientific crates. //! estimation authority; it invokes estimators through their scientific crate -//! contracts and preserves their artifact meaning. +//! contracts and preserves their artifact meaning. Loopback +//! [`ScientificAcceptanceLoopbackService`] executes scientific acceptance on an +//! accepted analysis run so GET can return `tepp.scientific_acceptance.v1` +//! without a caller-supplied artifact. mod case_deletion_refit; mod lineage_criterion; +mod loopback_execute; mod topic_context_posterior; mod topic_lineage_artifact; +mod validation_run; use serde::Serialize; use sha2::{Digest, Sha256}; @@ -26,6 +31,7 @@ use tepp_api::{ ApiError, }; use topic_measurement::TopicMeasurementError; +use validation_core::ValidationError; /// One document admitted to exhaustive case-deletion fitting. pub use case_deletion_refit::CaseDeletionDocument; @@ -46,6 +52,11 @@ pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, fit_lineage_criterion_posteriors, }; +/// Loopback engine-execute contracts for scientific acceptance. +pub use loopback_execute::{ + ANALYSIS_RUN_EXECUTE_CONTRACT_VERSION, ANALYSIS_RUN_EXECUTE_PATH_SUFFIX, + ScientificAcceptanceLoopbackService, +}; /// Bounded posterior topic-context producer contract and record types. pub use topic_context_posterior::{ TOPIC_CONTEXT_POSTERIOR_BYTE_LIMIT, TOPIC_CONTEXT_POSTERIOR_SCHEMA_VERSION, @@ -58,6 +69,14 @@ pub use topic_lineage_artifact::{ TOPIC_LINEAGE_MODEL_CONTRACT_VERSION, TOPIC_LINEAGE_OUTPUT_PROFILE, TopicLineageArtifact, TopicLineageArtifactEdge, TopicLineageExecution, execute_topic_lineage_run, }; +/// Durable scientific-acceptance validation-run contracts. +pub use validation_run::{ + MAX_RECOVERY_VECTOR_LEN, MAX_SE_GATE_K, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + ScientificAcceptanceEvidence, VALIDATION_BACKEND, VALIDATION_CPU_F64_MODEL, + VALIDATION_PRECISION, VALIDATION_RUN_ID_HEX_LEN, VALIDATION_RUN_ID_PREFIX, + ValidationRunReceipt, WILSON_Z, complete_validation_run, submit_validation_run, +}; /// Versioned artifact schema emitted by this engine. pub const ANALYSIS_ARTIFACT_SCHEMA_VERSION: &str = "tepp.temporal_evidence_readiness.v1"; @@ -248,6 +267,16 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A `validation_core` metric or report rejected the recovery vectors. + Validation(ValidationError), + /// Recovery was authored by an LLM and cannot become scientific evidence. + LlmAuthoredRecovery, + /// The corpus contained evidence, but none was available at the cutoff. + NoEligibleEvidence, + /// A completion receipt did not match the rebound scientific binding. + BindingMismatch, + /// The request did not select the scientific-acceptance profile and model. + InvalidValidationProfile, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +291,13 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::Validation(error) => return error.fmt(formatter), + Self::LlmAuthoredRecovery => { + "llm-authored recovery is not scientific acceptance evidence" + } + Self::NoEligibleEvidence => "no eligible evidence for validation run", + Self::BindingMismatch => "validation run binding mismatch", + Self::InvalidValidationProfile => "invalid scientific acceptance profile", }; formatter.write_str(message) } @@ -281,6 +317,12 @@ impl From for AnalysisEngineError { } } +impl From for AnalysisEngineError { + fn from(error: ValidationError) -> Self { + Self::Validation(error) + } +} + /// Execute the cutoff-safe temporal evidence readiness analysis. /// /// Evidence whose `available_time` is later than the request cutoff is excluded @@ -402,7 +444,7 @@ fn format_digest(digest: impl AsRef<[u8]>) -> String { output } -fn valid_identifier(value: &str) -> bool { +pub(crate) fn valid_identifier(value: &str) -> bool { !value.trim().is_empty() && value.len() <= MAX_ANALYSIS_IDENTIFIER_BYTES && !value.chars().any(char::is_control) @@ -681,6 +723,26 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::Validation(validation_core::ValidationError::InvalidInput), + "invalid validation input", + ), + ( + AnalysisEngineError::LlmAuthoredRecovery, + "llm-authored recovery is not scientific acceptance evidence", + ), + ( + AnalysisEngineError::NoEligibleEvidence, + "no eligible evidence for validation run", + ), + ( + AnalysisEngineError::BindingMismatch, + "validation run binding mismatch", + ), + ( + AnalysisEngineError::InvalidValidationProfile, + "invalid scientific acceptance profile", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); @@ -689,6 +751,9 @@ mod tests { assert_eq!(converted.to_string(), "invalid API wire payload"); let from_topic: AnalysisEngineError = TopicMeasurementError::DidNotConverge.into(); assert_eq!(from_topic.to_string(), "topic estimator did not converge"); + let from_validation: AnalysisEngineError = + validation_core::ValidationError::InvalidInput.into(); + assert_eq!(from_validation.to_string(), "invalid validation input"); assert_eq!( add_membership_count(u64::MAX, 1), Err(AnalysisEngineError::ArithmeticOverflow) diff --git a/crates/analysis_engine/src/loopback_execute.rs b/crates/analysis_engine/src/loopback_execute.rs new file mode 100644 index 000000000..14e8a655a --- /dev/null +++ b/crates/analysis_engine/src/loopback_execute.rs @@ -0,0 +1,333 @@ +//! Execute scientific-acceptance analysis on the loopback lifecycle path. +//! +//! GAP-003A engine-on-loopback slice: `POST /v1/analysis-runs/{run_id}/execute` +//! runs [`submit_validation_run`] and [`complete_validation_run`] against an +//! already accepted loopback run and records running then terminal status with +//! the produced `tepp.scientific_acceptance.v1` bytes. The execute body carries +//! corpus, recovery vectors, seed, and the pre-registered SE-gate multiplier. +//! It must not carry `scientific_acceptance_json`. GET then returns the artifact +//! without a caller-supplied terminal payload. Persistence remains GAP-003B. + +use crate::{ + AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + VALIDATION_CPU_F64_MODEL, complete_validation_run, submit_validation_run, +}; +use serde::Deserialize; +use temporal_core::{AvailableTime, EventTime}; +use tepp_api::{ + ANALYSIS_RUN_STATUS_PATH, AnalysisResultSummary, AnalysisRunStatus, AnalysisRunStatusState, + AnalysisRunTerminalResult, ApiError, DEFAULT_ANALYSIS_RUN_BYTE_LIMIT, + DEFAULT_PROJECT_HISTORY_BYTE_LIMIT, ErrorEnvelope, NaruonLiveResponse, + SCIENTIFIC_ACCEPTANCE_HTTP_PROFILE, SCIENTIFIC_ACCEPTANCE_HTTP_SCHEMA, + analysis_run_execute_path_run_id, parse_loopback_http_parts, +}; + +/// Result-metric keys that must not appear on an execute request object. +const EXECUTE_FORBIDDEN_RESULT_KEYS: [&str; 11] = [ + "rmse", + "rmse_standard_error", + "mean_bias", + "bias_standard_error", + "interval_coverage", + "coverage_wilson_lower", + "coverage_wilson_upper", + "temporal_order_accuracy", + "se_gate_accepted", + "scientific_acceptance", + "report", +]; + +/// Contract version for the loopback execute body. +pub const ANALYSIS_RUN_EXECUTE_CONTRACT_VERSION: u16 = 1; +/// Loopback path suffix owned by this engine glue. +pub const ANALYSIS_RUN_EXECUTE_PATH_SUFFIX: &str = "execute"; + +/// Loopback analysis-run service that can execute scientific acceptance. +/// +/// Create, GET, running, and terminal stay on [`tepp_api::AnalysisRunLiveService`]. +/// This wrapper intercepts `POST /v1/analysis-runs/{run_id}/execute` so the +/// engine produces the artifact. `tepp_api` cannot depend on this crate. +#[derive(Debug)] +pub struct ScientificAcceptanceLoopbackService { + live: tepp_api::AnalysisRunLiveService, + next_request_serial: u64, +} + +impl Default for ScientificAcceptanceLoopbackService { + fn default() -> Self { + Self::new() + } +} + +impl ScientificAcceptanceLoopbackService { + /// Construct an in-memory handler with no bound socket. + #[must_use] + pub fn new() -> Self { + Self { + live: tepp_api::AnalysisRunLiveService::new(), + next_request_serial: 1, + } + } + + /// Wrap an existing loopback listener. + #[must_use] + pub fn from_live(live: tepp_api::AnalysisRunLiveService) -> Self { + Self { + live, + next_request_serial: 1, + } + } + + /// Return the inner loopback service. + #[must_use] + pub fn live(&self) -> &tepp_api::AnalysisRunLiveService { + &self.live + } + + /// Return the inner loopback service mutably. + pub fn live_mut(&mut self) -> &mut tepp_api::AnalysisRunLiveService { + &mut self.live + } + + /// Parse and handle one complete HTTP/1.1 request already in memory. + #[must_use] + pub fn handle_http_request(&mut self, request: &str) -> NaruonLiveResponse { + match self.dispatch_http_request(request) { + Ok(response) => response, + Err(error) => self.response_from_error(error), + } + } + + fn dispatch_http_request(&mut self, request: &str) -> Result { + let Ok(parts) = parse_loopback_http_parts(request, DEFAULT_PROJECT_HISTORY_BYTE_LIMIT) + else { + return Ok(self.live.handle_http_request(request)); + }; + if parts.method == "POST" && is_execute_attempt(parts.path) { + let run_id = analysis_run_execute_path_run_id(parts.path)?; + return self.execute_scientific_acceptance(&run_id, &parts.headers, parts.body); + } + Ok(self.live.handle_http_request(request)) + } + + fn execute_scientific_acceptance( + &mut self, + path_run_id: &str, + headers: &std::collections::HashMap, + body: &str, + ) -> Result { + refuse_result_metrics_on_execute(body)?; + let consumer = self.live.authorize_loopback_headers(headers)?; + let idempotency_key = header_required(headers, "idempotency-key")?; + let execute = parse_execute_body(body)?; + if execute.run_id != path_run_id || execute.idempotency_key != idempotency_key { + return Err(ApiError::InvalidWirePayload); + } + let stored = self.live.loopback_run(path_run_id)?; + if stored.consumer != consumer || stored.accepted.idempotency_key != idempotency_key { + return Err(ApiError::InvalidWirePayload); + } + if stored.status.run_state != AnalysisRunStatusState::Accepted { + return Err(ApiError::InvalidWirePayload); + } + if stored.request.output_profile != SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + || stored.request.output_profile != SCIENTIFIC_ACCEPTANCE_HTTP_PROFILE + || stored.request.model_contract_version != VALIDATION_CPU_F64_MODEL + { + return Err(ApiError::InvalidWirePayload); + } + let corpus = execute.corpus.to_corpus().map_err(map_engine_error)?; + let receipt = + submit_validation_run(&stored.request, &corpus, execute.seed, execute.se_gate_k) + .map_err(map_engine_error)?; + let observation = RecoveryObservation::new( + &receipt, + execute.study_label.clone(), + execute.truth.clone(), + execute.recovered.clone(), + execute.interval_lower.clone(), + execute.interval_upper.clone(), + execute.truth_times.clone(), + execute.recovered_times.clone(), + execute.se_gate_k, + execute.authored_by_llm, + ) + .map_err(map_engine_error)?; + let evidence = complete_validation_run(&receipt, &stored.request, &corpus, &observation) + .map_err(map_engine_error)?; + let artifact_json = evidence.to_json().map_err(map_engine_error)?; + let digest = evidence.sha256().map_err(map_engine_error)?; + if evidence.schema_version() != SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION + || evidence.schema_version() != SCIENTIFIC_ACCEPTANCE_HTTP_SCHEMA + || evidence.output_profile() != SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + { + return Err(ApiError::InvalidWirePayload); + } + let summary = AnalysisResultSummary::new( + "scientific_acceptance", + evidence.eligible_evidence_count(), + 4, + "validated", + )?; + let terminal = AnalysisRunTerminalResult::succeeded( + &stored.request, + &stored.accepted, + evidence.run_id(), + digest, + SCIENTIFIC_ACCEPTANCE_HTTP_SCHEMA, + execute.completed_at.clone(), + summary, + )?; + let running = AnalysisRunStatus::running(&stored.accepted)?; + let terminal_status = + AnalysisRunStatus::terminal(&stored.request, &stored.accepted, terminal)?; + self.live + .record_loopback_status(path_run_id, running, None)?; + self.live + .record_loopback_status(path_run_id, terminal_status, Some(artifact_json))?; + let response_body = self.live.loopback_status_json(path_run_id)?; + Ok(NaruonLiveResponse { + status_code: 200, + reason_phrase: "OK", + body: response_body, + }) + } + + fn response_from_error(&mut self, error: ApiError) -> NaruonLiveResponse { + let request_id = format!("scientific-acceptance-execute-{}", self.next_request_serial); + self.next_request_serial += 1; + let (status_code, reason_phrase) = match error { + ApiError::AuthorizationDenied => (403, "Forbidden"), + ApiError::LimitExceeded => (413, "Payload Too Large"), + ApiError::UnsupportedContractVersion => (422, "Unprocessable Entity"), + _ => (400, "Bad Request"), + }; + let body = ErrorEnvelope::from_api_error(error, request_id) + .and_then(|envelope| envelope.to_json()) + .unwrap_or_else(|_| { + "{\"error_code\":\"invalid_wire_payload\",\"message\":\"invalid API wire payload\",\"request_id\":\"scientific-acceptance-execute-fallback\",\"retryable\":false}".to_owned() + }); + NaruonLiveResponse { + status_code, + reason_phrase, + body, + } + } +} + +/// Supported execute-body contract version. +#[derive(Clone, Debug, Deserialize, PartialEq)] +#[serde(deny_unknown_fields)] +struct ScientificAcceptanceExecuteRequest { + contract_version: u16, + run_id: String, + idempotency_key: String, + seed: u64, + se_gate_k: f64, + completed_at: String, + study_label: String, + authored_by_llm: bool, + corpus: ExecuteCorpus, + truth: Vec, + recovered: Vec, + interval_lower: Vec, + interval_upper: Vec, + truth_times: Vec, + recovered_times: Vec, +} + +#[derive(Clone, Debug, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +struct ExecuteCorpus { + snapshot_id: String, + evidence_units: Vec, +} + +#[derive(Clone, Debug, Deserialize, PartialEq, Eq)] +#[serde(deny_unknown_fields)] +struct ExecuteEvidenceUnit { + evidence_id: String, + event_time: String, + available_time: String, + membership_count: u32, +} + +impl ExecuteCorpus { + fn to_corpus(&self) -> Result { + let mut units = Vec::with_capacity(self.evidence_units.len()); + for unit in &self.evidence_units { + let event_time = EventTime::parse_rfc3339(&unit.event_time) + .map_err(|_| AnalysisEngineError::InvalidEvidence)?; + let available_time = AvailableTime::parse_rfc3339(&unit.available_time) + .map_err(|_| AnalysisEngineError::InvalidEvidence)?; + units.push(AnalysisEvidenceUnit::new( + unit.evidence_id.clone(), + event_time, + available_time, + unit.membership_count, + )?); + } + AnalysisCorpus::new(self.snapshot_id.clone(), units) + } +} + +fn parse_execute_body(body: &str) -> Result { + if body.len() > DEFAULT_ANALYSIS_RUN_BYTE_LIMIT { + return Err(ApiError::LimitExceeded); + } + refuse_result_metrics_on_execute(body)?; + let execute: ScientificAcceptanceExecuteRequest = + serde_json::from_str(body).map_err(|_| ApiError::InvalidWirePayload)?; + if execute.contract_version != ANALYSIS_RUN_EXECUTE_CONTRACT_VERSION { + return Err(ApiError::UnsupportedContractVersion); + } + if execute.run_id.is_empty() + || execute.idempotency_key.is_empty() + || execute.completed_at.is_empty() + || execute.study_label.is_empty() + { + return Err(ApiError::InvalidWirePayload); + } + Ok(execute) +} + +fn refuse_result_metrics_on_execute(payload: &str) -> Result<(), ApiError> { + let Ok(value) = serde_json::from_str::(payload) else { + return Ok(()); + }; + let Some(object) = value.as_object() else { + return Err(ApiError::InvalidWirePayload); + }; + if EXECUTE_FORBIDDEN_RESULT_KEYS + .iter() + .any(|key| object.contains_key(*key)) + { + return Err(ApiError::InvalidWirePayload); + } + Ok(()) +} + +fn is_execute_attempt(path: &str) -> bool { + path.rsplit('/').next() == Some(ANALYSIS_RUN_EXECUTE_PATH_SUFFIX) + && path.starts_with(ANALYSIS_RUN_STATUS_PATH) +} + +fn header_required<'a>( + headers: &'a std::collections::HashMap, + name: &str, +) -> Result<&'a str, ApiError> { + let value = headers.get(name).ok_or(ApiError::InvalidWirePayload)?; + if value.is_empty() { + return Err(ApiError::InvalidWirePayload); + } + Ok(value.as_str()) +} + +fn map_engine_error(error: AnalysisEngineError) -> ApiError { + match error { + AnalysisEngineError::Api(api) => api, + AnalysisEngineError::LimitExceeded => ApiError::LimitExceeded, + _ => ApiError::InvalidWirePayload, + } +} diff --git a/crates/analysis_engine/src/validation_run.rs b/crates/analysis_engine/src/validation_run.rs new file mode 100644 index 000000000..2a31ae445 --- /dev/null +++ b/crates/analysis_engine/src/validation_run.rs @@ -0,0 +1,1223 @@ +//! Bind immutable evidence to a durable scientific-acceptance validation run. +//! +//! This is the first GAP-003A slice: cutoff-safe evidence, tenant workspace, +//! output profile, model, seed, backend, precision, and the SE-gate multiplier +//! hash to one durable run identity. The accepted receipt carries no scientific +//! metrics. Completion asks `validation_core` for RMSE, bias, coverage, +//! temporal-order accuracy, and an SE-aware gate, then emits +//! `tepp.scientific_acceptance.v1`. Recovery vectors must be stamped with that +//! same run identity and the pre-registered multiplier; a different run, model, +//! snapshot, seed, tenant, profile, eligible evidence set, or post-hoc `k` +//! fails closed. LLM-authored recovery, non-finite inputs, empty or duplicate +//! evidence, snapshot mismatch, oversized recovery, oversized `k`, and +//! cutoff-empty corpora fail closed. Postgres persistence remains GAP-003B. + +use crate::{AnalysisCorpus, AnalysisEngineError, valid_identifier}; +use serde::Serialize; +use sha2::{Digest, Sha256}; +use std::collections::BTreeSet; +use std::fmt::Write as _; +use temporal_core::KnowledgeCutoff; +use tepp_api::{AnalysisRunRequest, ApiError}; +use validation_core::{ + ValidationError, ValidationReport, accept_within_standard_errors, bias_standard_error, + interval_coverage, mean_bias, rmse_standard_error, root_mean_square_error, + temporal_order_accuracy, wilson_coverage_interval, +}; + +/// Versioned scientific-acceptance artifact schema. +pub const SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION: &str = "tepp.scientific_acceptance.v1"; +/// Output profile that selects this validation-run executor. +pub const SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE: &str = "scientific_acceptance_v1"; +/// CPU `f64` reference model identity for scientific acceptance. +pub const VALIDATION_CPU_F64_MODEL: &str = "validation_cpu_f64_v1"; +/// Backend identity bound into the durable run. +pub const VALIDATION_BACKEND: &str = "cpu"; +/// Numeric precision bound into the durable run. +pub const VALIDATION_PRECISION: &str = "f64"; +/// Prefix of the hash-stable durable run identity. +pub const VALIDATION_RUN_ID_PREFIX: &str = "tepp-validation-"; +/// Hex characters taken from the binding digest for `run_id`. +pub const VALIDATION_RUN_ID_HEX_LEN: usize = 32; +/// Wilson critical value for nominal 95% coverage bounds. +pub const WILSON_Z: f64 = 1.96; +/// Maximum length of one recovery, interval, or event-time vector. +pub const MAX_RECOVERY_VECTOR_LEN: usize = 10_000; +/// Largest finite SE-gate multiplier that may be pre-registered on a run. +/// +/// Conventional three-SE gates sit inside this bound. A larger `k` would make +/// `|RMSE| ≤ k · SE(RMSE)` an effectively unlimited post-hoc acceptance rule. +pub const MAX_SE_GATE_K: f64 = 8.0; + +/// Durable identity of one submitted validation run. Receipts never carry +/// scientific metrics. Fields are private so callers cannot rewrite the +/// binding or the pre-registered SE-gate multiplier after submit. +#[derive(Clone, Debug, PartialEq, Serialize)] +pub struct ValidationRunReceipt { + run_id: String, + binding_sha256: String, + tenant_workspace_id: String, + snapshot_id: String, + knowledge_cutoff: String, + model: String, + seed: u64, + backend: String, + precision: String, + output_profile: String, + eligible_evidence_count: u64, + se_gate_k: f64, +} + +impl ValidationRunReceipt { + /// Serialize the receipt after confirming it carries no metrics. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::SerializationFailure`] when JSON encoding + /// fails. + pub fn to_json(&self) -> Result { + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure) + } + + /// Return the durable run identity. + #[must_use] + pub fn run_id(&self) -> &str { + &self.run_id + } + + /// Return the canonical binding digest. + #[must_use] + pub fn binding_sha256(&self) -> &str { + &self.binding_sha256 + } + + /// Return the bound tenant workspace identity. + #[must_use] + pub fn tenant_workspace_id(&self) -> &str { + &self.tenant_workspace_id + } + + /// Return the bound snapshot identity. + #[must_use] + pub fn snapshot_id(&self) -> &str { + &self.snapshot_id + } + + /// Return the bound knowledge cutoff. + #[must_use] + pub fn knowledge_cutoff(&self) -> &str { + &self.knowledge_cutoff + } + + /// Return the bound model identity. + #[must_use] + pub fn model(&self) -> &str { + &self.model + } + + /// Return the bound seed. + #[must_use] + pub const fn seed(&self) -> u64 { + self.seed + } + + /// Return the bound backend. + #[must_use] + pub fn backend(&self) -> &str { + &self.backend + } + + /// Return the bound precision. + #[must_use] + pub fn precision(&self) -> &str { + &self.precision + } + + /// Return the bound output profile. + #[must_use] + pub fn output_profile(&self) -> &str { + &self.output_profile + } + + /// Return the eligible evidence count. + #[must_use] + pub const fn eligible_evidence_count(&self) -> u64 { + self.eligible_evidence_count + } + + /// Return the pre-registered SE-gate multiplier. + #[must_use] + pub const fn se_gate_k(&self) -> f64 { + self.se_gate_k + } + + fn identity_record(&self) -> String { + format!( + "{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{:016x}", + self.run_id, + self.binding_sha256, + self.tenant_workspace_id, + self.snapshot_id, + self.knowledge_cutoff, + self.model, + self.seed, + self.backend, + self.precision, + self.output_profile, + self.eligible_evidence_count, + self.se_gate_k.to_bits() + ) + } +} + +/// Known-truth recovery vectors offered to complete a validation run. +#[derive(Clone, Debug, PartialEq)] +pub struct RecoveryObservation { + run_id: String, + binding_sha256: String, + study_label: String, + truth: Vec, + recovered: Vec, + interval_lower: Vec, + interval_upper: Vec, + truth_times: Vec, + recovered_times: Vec, + se_gate_k: f64, + authored_by_llm: bool, +} + +impl RecoveryObservation { + /// Construct a recovery observation stamped to one submitted receipt. + /// + /// LLM authorship is recorded here and refused at completion. The SE-gate + /// multiplier must equal the pre-registered receipt value; a post-hoc `k` + /// is a binding mismatch. Empty, length-mismatched, or oversized vectors + /// fail closed immediately. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidEvidence`] for an empty, oversized, + /// or control-bearing study label or for empty/mismatched vectors, + /// [`AnalysisEngineError::LimitExceeded`] when any vector exceeds + /// [`MAX_RECOVERY_VECTOR_LEN`], [`AnalysisEngineError::BindingMismatch`] + /// when `se_gate_k` differs from the receipt, and + /// [`AnalysisEngineError::Validation`] for a non-finite or out-of-policy SE + /// multiplier. + #[allow(clippy::too_many_arguments)] + pub fn new( + receipt: &ValidationRunReceipt, + study_label: impl Into, + truth: Vec, + recovered: Vec, + interval_lower: Vec, + interval_upper: Vec, + truth_times: Vec, + recovered_times: Vec, + se_gate_k: f64, + authored_by_llm: bool, + ) -> Result { + let study_label = study_label.into(); + if !valid_identifier(&study_label) { + return Err(AnalysisEngineError::InvalidEvidence); + } + let lengths = [ + truth.len(), + recovered.len(), + interval_lower.len(), + interval_upper.len(), + truth_times.len(), + recovered_times.len(), + ]; + if lengths.iter().any(|len| *len > MAX_RECOVERY_VECTOR_LEN) { + return Err(AnalysisEngineError::LimitExceeded); + } + let n = truth.len(); + if n == 0 || lengths.iter().any(|len| *len != n) { + return Err(AnalysisEngineError::InvalidEvidence); + } + let se_gate_k = require_se_gate_k(se_gate_k)?; + if se_gate_k.to_bits() != receipt.se_gate_k.to_bits() { + return Err(AnalysisEngineError::BindingMismatch); + } + Ok(Self { + run_id: receipt.run_id.clone(), + binding_sha256: receipt.binding_sha256.clone(), + study_label, + truth, + recovered, + interval_lower, + interval_upper, + truth_times, + recovered_times, + se_gate_k, + authored_by_llm, + }) + } + + /// Return the stamped run identity. + #[must_use] + pub fn run_id(&self) -> &str { + &self.run_id + } + + /// Return the stamped binding digest. + #[must_use] + pub fn binding_sha256(&self) -> &str { + &self.binding_sha256 + } + + /// Return the study label. + #[must_use] + pub fn study_label(&self) -> &str { + &self.study_label + } + + /// Return the truth vector. + #[must_use] + pub fn truth(&self) -> &[f64] { + &self.truth + } + + /// Return the recovered vector. + #[must_use] + pub fn recovered(&self) -> &[f64] { + &self.recovered + } + + /// Return interval lower bounds. + #[must_use] + pub fn interval_lower(&self) -> &[f64] { + &self.interval_lower + } + + /// Return interval upper bounds. + #[must_use] + pub fn interval_upper(&self) -> &[f64] { + &self.interval_upper + } + + /// Return truth event times. + #[must_use] + pub fn truth_times(&self) -> &[f64] { + &self.truth_times + } + + /// Return recovered event times. + #[must_use] + pub fn recovered_times(&self) -> &[f64] { + &self.recovered_times + } + + /// Return the SE-gate multiplier. + #[must_use] + pub const fn se_gate_k(&self) -> f64 { + self.se_gate_k + } + + /// Return whether an LLM authored the recovery. + #[must_use] + pub const fn authored_by_llm(&self) -> bool { + self.authored_by_llm + } + + fn digest_hex(&self) -> String { + let mut canonical = String::from("tepp.recovery_observation.v1\n"); + let _ = writeln!(canonical, "run_id={}", self.run_id); + let _ = writeln!(canonical, "binding={}", self.binding_sha256); + let _ = writeln!(canonical, "study={}", self.study_label); + let _ = writeln!(canonical, "se_gate_k={:016x}", self.se_gate_k.to_bits()); + let _ = writeln!(canonical, "authored_by_llm={}", self.authored_by_llm); + append_f64_vector(&mut canonical, "truth", &self.truth); + append_f64_vector(&mut canonical, "recovered", &self.recovered); + append_f64_vector(&mut canonical, "interval_lower", &self.interval_lower); + append_f64_vector(&mut canonical, "interval_upper", &self.interval_upper); + append_f64_vector(&mut canonical, "truth_times", &self.truth_times); + append_f64_vector(&mut canonical, "recovered_times", &self.recovered_times); + format_hex(Sha256::digest(canonical.into_bytes())) + } +} + +/// Operator-usable scientific acceptance evidence for one completed run. +#[derive(Clone, Debug, PartialEq, Serialize)] +pub struct ScientificAcceptanceEvidence { + schema_version: String, + run_id: String, + binding_sha256: String, + recovery_sha256: String, + tenant_workspace_id: String, + snapshot_id: String, + knowledge_cutoff: String, + model: String, + seed: u64, + backend: String, + precision: String, + output_profile: String, + eligible_evidence_count: u64, + se_gate_accepted: bool, + se_gate_k: f64, + report: ValidationReport, +} + +impl ScientificAcceptanceEvidence { + /// Serialize canonical evidence JSON after report validation. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::Validation`] when a report field is + /// non-finite, or [`AnalysisEngineError::SerializationFailure`] when JSON + /// encoding fails. + pub fn to_json(&self) -> Result { + self.report.validate()?; + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure) + } + + /// Return the lowercase SHA-256 digest of the canonical evidence JSON. + /// + /// # Errors + /// + /// Returns the same failures as [`Self::to_json`]. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_hex(Sha256::digest(json.into_bytes()))) + } + + /// Render the nested recovery summary line. + #[must_use] + pub fn to_human_summary(&self) -> String { + self.report.to_human_summary() + } + + /// Return the artifact schema version. + #[must_use] + pub fn schema_version(&self) -> &str { + &self.schema_version + } + + /// Return the durable run identity. + #[must_use] + pub fn run_id(&self) -> &str { + &self.run_id + } + + /// Return the scientific binding digest. + #[must_use] + pub fn binding_sha256(&self) -> &str { + &self.binding_sha256 + } + + /// Return the recovery-vector digest stamped into this artifact. + #[must_use] + pub fn recovery_sha256(&self) -> &str { + &self.recovery_sha256 + } + + /// Return the tenant workspace identity. + #[must_use] + pub fn tenant_workspace_id(&self) -> &str { + &self.tenant_workspace_id + } + + /// Return the snapshot identity. + #[must_use] + pub fn snapshot_id(&self) -> &str { + &self.snapshot_id + } + + /// Return the knowledge cutoff. + #[must_use] + pub fn knowledge_cutoff(&self) -> &str { + &self.knowledge_cutoff + } + + /// Return the bound model identity. + #[must_use] + pub fn model(&self) -> &str { + &self.model + } + + /// Return the bound seed. + #[must_use] + pub const fn seed(&self) -> u64 { + self.seed + } + + /// Return the bound backend. + #[must_use] + pub fn backend(&self) -> &str { + &self.backend + } + + /// Return the bound precision. + #[must_use] + pub fn precision(&self) -> &str { + &self.precision + } + + /// Return the output profile. + #[must_use] + pub fn output_profile(&self) -> &str { + &self.output_profile + } + + /// Return the eligible evidence count. + #[must_use] + pub const fn eligible_evidence_count(&self) -> u64 { + self.eligible_evidence_count + } + + /// Return whether the SE-aware gate accepted RMSE toward 0. + #[must_use] + pub const fn se_gate_accepted(&self) -> bool { + self.se_gate_accepted + } + + /// Return the SE-gate multiplier used for acceptance. + #[must_use] + pub const fn se_gate_k(&self) -> f64 { + self.se_gate_k + } +} + +struct CanonicalBinding { + tenant_workspace_id: String, + snapshot_id: String, + knowledge_cutoff: String, + model: String, + seed: u64, + backend: String, + precision: String, + output_profile: String, + se_gate_k: f64, + eligible_ids: Vec, +} + +impl CanonicalBinding { + fn digest_hex(&self) -> String { + format_hex(Sha256::digest(self.canonical_bytes())) + } + + fn run_id(digest_hex: &str) -> String { + let mut run_id = + String::with_capacity(VALIDATION_RUN_ID_PREFIX.len() + VALIDATION_RUN_ID_HEX_LEN); + run_id.push_str(VALIDATION_RUN_ID_PREFIX); + run_id.push_str(&digest_hex[..VALIDATION_RUN_ID_HEX_LEN]); + run_id + } + + fn eligible_count(&self) -> u64 { + self.eligible_ids.len() as u64 + } + + fn canonical_bytes(&self) -> String { + let mut canonical = String::from("tepp.validation_binding.v1\n"); + let _ = writeln!(canonical, "tenant={}", self.tenant_workspace_id); + let _ = writeln!(canonical, "snapshot={}", self.snapshot_id); + let _ = writeln!(canonical, "cutoff={}", self.knowledge_cutoff); + let _ = writeln!(canonical, "model={}", self.model); + let _ = writeln!(canonical, "seed={}", self.seed); + let _ = writeln!(canonical, "backend={}", self.backend); + let _ = writeln!(canonical, "precision={}", self.precision); + let _ = writeln!(canonical, "profile={}", self.output_profile); + let _ = writeln!(canonical, "se_gate_k={:016x}", self.se_gate_k.to_bits()); + for identity in &self.eligible_ids { + let _ = writeln!(canonical, "evidence={identity}"); + } + canonical + } + + fn identity_record(&self, digest_hex: &str, run_id: &str) -> String { + format!( + "{run_id}\n{digest_hex}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{}\n{:016x}", + self.tenant_workspace_id, + self.snapshot_id, + self.knowledge_cutoff, + self.model, + self.seed, + self.backend, + self.precision, + self.output_profile, + self.eligible_count(), + self.se_gate_k.to_bits() + ) + } +} + +/// Submit immutable evidence to a durable validation run. +/// +/// The receipt is hash-stable for one scientific binding and never includes +/// RMSE, bias, coverage, or gate results. +/// +/// # Errors +/// +/// Returns a fail-closed engine error for an invalid request, wrong output +/// profile or model, an out-of-policy SE-gate multiplier, snapshot mismatch, +/// duplicate or empty evidence, or a cutoff that admits no evidence. +pub fn submit_validation_run( + request: &AnalysisRunRequest, + corpus: &AnalysisCorpus, + seed: u64, + se_gate_k: f64, +) -> Result { + let binding = bind_validation_run(request, corpus, seed, se_gate_k)?; + let binding_sha256 = binding.digest_hex(); + let run_id = CanonicalBinding::run_id(&binding_sha256); + let eligible_evidence_count = binding.eligible_count(); + Ok(ValidationRunReceipt { + run_id, + binding_sha256, + tenant_workspace_id: binding.tenant_workspace_id, + snapshot_id: binding.snapshot_id, + knowledge_cutoff: binding.knowledge_cutoff, + model: binding.model, + seed: binding.seed, + backend: binding.backend, + precision: binding.precision, + output_profile: binding.output_profile, + eligible_evidence_count, + se_gate_k: binding.se_gate_k, + }) +} + +/// Complete a previously submitted validation run with known-truth recovery. +/// +/// # Errors +/// +/// Returns a fail-closed engine error when recovery was LLM-authored, the +/// observation is stamped to a different run identity, the receipt does not +/// match the rebound scientific identity, metric inputs are invalid, or +/// report validation fails. +pub fn complete_validation_run( + receipt: &ValidationRunReceipt, + request: &AnalysisRunRequest, + corpus: &AnalysisCorpus, + observation: &RecoveryObservation, +) -> Result { + if observation.authored_by_llm { + return Err(AnalysisEngineError::LlmAuthoredRecovery); + } + if observation.run_id != receipt.run_id + || observation.binding_sha256 != receipt.binding_sha256 + || observation.se_gate_k.to_bits() != receipt.se_gate_k.to_bits() + || receipt.output_profile != SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::BindingMismatch); + } + let binding = bind_validation_run(request, corpus, receipt.seed, receipt.se_gate_k)?; + let binding_sha256 = binding.digest_hex(); + let run_id = CanonicalBinding::run_id(&binding_sha256); + if receipt.identity_record() != binding.identity_record(&binding_sha256, &run_id) { + return Err(AnalysisEngineError::BindingMismatch); + } + let report = compute_report(observation)?; + let se_gate_accepted = accept_within_standard_errors( + report.rmse, + 0.0, + report.rmse_standard_error, + receipt.se_gate_k, + )?; + let eligible_evidence_count = binding.eligible_count(); + let evidence = ScientificAcceptanceEvidence { + schema_version: SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION.to_owned(), + run_id, + binding_sha256, + recovery_sha256: observation.digest_hex(), + tenant_workspace_id: binding.tenant_workspace_id, + snapshot_id: binding.snapshot_id, + knowledge_cutoff: binding.knowledge_cutoff, + model: binding.model, + seed: binding.seed, + backend: binding.backend, + precision: binding.precision, + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.to_owned(), + eligible_evidence_count, + se_gate_accepted, + se_gate_k: receipt.se_gate_k, + report, + }; + evidence.to_json()?; + Ok(evidence) +} + +fn require_se_gate_k(se_gate_k: f64) -> Result { + if !se_gate_k.is_finite() { + return Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput, + )); + } + if se_gate_k < 0.0 || se_gate_k > MAX_SE_GATE_K { + return Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration, + )); + } + Ok(if se_gate_k == 0.0 { 0.0 } else { se_gate_k }) +} + +fn bind_validation_run( + request: &AnalysisRunRequest, + corpus: &AnalysisCorpus, + seed: u64, + se_gate_k: f64, +) -> Result { + request.to_json()?; + let se_gate_k = require_se_gate_k(se_gate_k)?; + let requested = format!( + "{}\n{}", + request.output_profile, request.model_contract_version + ); + let expected = format!("{SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE}\n{VALIDATION_CPU_F64_MODEL}"); + if requested != expected { + return Err(AnalysisEngineError::InvalidValidationProfile); + } + if request.snapshot_id != corpus.snapshot_id() { + return Err(AnalysisEngineError::SnapshotMismatch); + } + let cutoff = KnowledgeCutoff::parse_rfc3339(&request.knowledge_cutoff) + .map_err(|_| AnalysisEngineError::Api(ApiError::InvalidWirePayload))?; + let mut identities = BTreeSet::new(); + let mut eligible = BTreeSet::new(); + for unit in corpus.evidence_units() { + if !identities.insert(unit.evidence_id()) { + return Err(AnalysisEngineError::DuplicateEvidence); + } + if unit.available_time().instant() <= cutoff.instant() { + eligible.insert(unit.evidence_id().to_owned()); + } + } + if identities.is_empty() { + return Err(AnalysisEngineError::InvalidEvidence); + } + if eligible.is_empty() { + return Err(AnalysisEngineError::NoEligibleEvidence); + } + Ok(CanonicalBinding { + tenant_workspace_id: request.tenant_workspace_id.clone(), + snapshot_id: request.snapshot_id.clone(), + knowledge_cutoff: cutoff.to_rfc3339(), + model: VALIDATION_CPU_F64_MODEL.to_owned(), + seed, + backend: VALIDATION_BACKEND.to_owned(), + precision: VALIDATION_PRECISION.to_owned(), + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.to_owned(), + se_gate_k, + eligible_ids: eligible.into_iter().collect(), + }) +} + +fn compute_report( + observation: &RecoveryObservation, +) -> Result { + let rmse = root_mean_square_error(&observation.truth, &observation.recovered)?; + let rmse_standard_error = rmse_standard_error(&observation.truth, &observation.recovered)?; + let mean_bias = mean_bias(&observation.truth, &observation.recovered)?; + let bias_standard_error = bias_standard_error(&observation.truth, &observation.recovered)?; + let interval_coverage = interval_coverage( + &observation.truth, + &observation.interval_lower, + &observation.interval_upper, + )?; + let (coverage_wilson_lower, coverage_wilson_upper) = wilson_coverage_interval( + &observation.truth, + &observation.interval_lower, + &observation.interval_upper, + WILSON_Z, + )?; + let temporal_order_accuracy = + temporal_order_accuracy(&observation.truth_times, &observation.recovered_times)?; + let report = ValidationReport { + study_label: observation.study_label.clone(), + rmse, + rmse_standard_error, + mean_bias, + bias_standard_error, + interval_coverage, + coverage_wilson_lower, + coverage_wilson_upper, + temporal_order_accuracy, + monte_carlo_rmse: None, + }; + report.validate()?; + Ok(report) +} + +fn append_f64_vector(canonical: &mut String, label: &str, values: &[f64]) { + let _ = writeln!(canonical, "{label}_len={}", values.len()); + for value in values { + let _ = writeln!(canonical, "{label}={:016x}", value.to_bits()); + } +} + +fn format_hex(digest: impl AsRef<[u8]>) -> String { + let bytes = digest.as_ref(); + let mut output = String::with_capacity(bytes.len() * 2); + for byte in bytes { + let _ = write!(output, "{byte:02x}"); + } + output +} + +#[cfg(test)] +mod tests { + use super::{ + MAX_RECOVERY_VECTOR_LEN, MAX_SE_GATE_K, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + VALIDATION_BACKEND, VALIDATION_CPU_F64_MODEL, VALIDATION_PRECISION, + VALIDATION_RUN_ID_PREFIX, WILSON_Z, complete_validation_run, submit_validation_run, + }; + use crate::{ + AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, MAX_ANALYSIS_IDENTIFIER_BYTES, + }; + use temporal_core::{AvailableTime, EventTime}; + use tepp_api::AnalysisRunRequest; + use validation_core::ValidationError; + + fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "idem-validation-1".into(), + tenant_workspace_id: "tenant-workspace-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: VALIDATION_CPU_F64_MODEL.into(), + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.into(), + } + } + + fn unit(id: &str, available: &str) -> AnalysisEvidenceUnit { + AnalysisEvidenceUnit::new( + id, + EventTime::parse_rfc3339("2026-07-01T00:00:00Z").expect("event"), + AvailableTime::parse_rfc3339(available).expect("available"), + 1, + ) + .expect("unit") + } + + fn corpus(units: Vec) -> AnalysisCorpus { + AnalysisCorpus::new("snapshot-1", units).expect("corpus") + } + + fn recovery( + receipt: &super::ValidationRunReceipt, + truth: Vec, + recovered: Vec, + k: f64, + ) -> RecoveryObservation { + let times = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0]; + let times = times[..truth.len()].to_vec(); + let lower: Vec = truth.iter().map(|value| value - 0.5).collect(); + let upper: Vec = truth.iter().map(|value| value + 0.5).collect(); + RecoveryObservation::new( + receipt, + "foundation-recovery", + truth, + recovered, + lower, + upper, + times.clone(), + times, + k, + false, + ) + .expect("observation") + } + + fn two_point( + receipt: &super::ValidationRunReceipt, + label: impl Into, + k: f64, + authored_by_llm: bool, + ) -> Result { + RecoveryObservation::new( + receipt, + label, + vec![1.0, 2.0], + vec![1.0, 2.0], + vec![0.0, 1.0], + vec![2.0, 3.0], + vec![1.0, 2.0], + vec![1.0, 2.0], + k, + authored_by_llm, + ) + } + + #[test] + fn submit_is_hash_stable_cutoff_safe_and_metric_free() { + let first = corpus(vec![ + unit("evidence-b", "2026-07-15T00:00:00Z"), + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("late", "2026-08-02T00:00:00Z"), + ]); + let second = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt_a = submit_validation_run(&request(), &first, 7, 3.0).expect("submit a"); + let receipt_b = submit_validation_run(&request(), &second, 7, 3.0).expect("submit b"); + assert_eq!(receipt_a.run_id(), receipt_b.run_id()); + assert_eq!(receipt_a.binding_sha256(), receipt_b.binding_sha256()); + assert!(receipt_a.run_id().starts_with(VALIDATION_RUN_ID_PREFIX)); + assert_eq!( + receipt_a.run_id().len(), + VALIDATION_RUN_ID_PREFIX.len() + 32 + ); + assert_eq!(receipt_a.eligible_evidence_count(), 2); + assert_eq!(receipt_a.tenant_workspace_id(), "tenant-workspace-1"); + assert_eq!(receipt_a.snapshot_id(), "snapshot-1"); + assert_eq!(receipt_a.knowledge_cutoff(), "2026-08-01T00:00:00Z"); + assert_eq!(receipt_a.model(), VALIDATION_CPU_F64_MODEL); + assert_eq!(receipt_a.seed(), 7); + assert_eq!(receipt_a.backend(), VALIDATION_BACKEND); + assert_eq!(receipt_a.precision(), VALIDATION_PRECISION); + assert_eq!( + receipt_a.output_profile(), + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + ); + assert!((receipt_a.se_gate_k() - 3.0).abs() < f64::EPSILON); + let json = receipt_a.to_json().expect("json"); + assert!(!json.contains("rmse")); + assert!(!json.contains("bias")); + assert!(!json.contains("coverage")); + let other_seed = submit_validation_run(&request(), &first, 8, 3.0).expect("seed"); + assert_ne!(receipt_a.run_id(), other_seed.run_id()); + let mut other_tenant = request(); + other_tenant.tenant_workspace_id = "tenant-workspace-2".into(); + let other_tenant_receipt = + submit_validation_run(&other_tenant, &first, 7, 3.0).expect("tenant"); + assert_ne!(receipt_a.run_id(), other_tenant_receipt.run_id()); + let other_k = submit_validation_run(&request(), &first, 7, 4.0).expect("other k"); + assert_ne!(receipt_a.run_id(), other_k.run_id()); + assert!((other_k.se_gate_k() - 4.0).abs() < f64::EPSILON); + } + + #[test] + fn complete_emits_operator_usable_scientific_acceptance_evidence() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 11, 3.0).expect("submit"); + let observation = recovery( + &receipt, + vec![0.70, 0.55, 0.40, -0.20, 0.85], + vec![0.70, 0.55, 0.40, -0.20, 0.85], + 3.0, + ); + assert_eq!(observation.run_id(), receipt.run_id()); + assert_eq!(observation.binding_sha256(), receipt.binding_sha256()); + assert_eq!(observation.study_label(), "foundation-recovery"); + assert_eq!(observation.truth().len(), 5); + assert_eq!(observation.recovered().len(), 5); + assert_eq!(observation.interval_lower().len(), 5); + assert_eq!(observation.interval_upper().len(), 5); + assert_eq!(observation.truth_times().len(), 5); + assert_eq!(observation.recovered_times().len(), 5); + assert!((observation.se_gate_k() - 3.0).abs() < f64::EPSILON); + assert!(!observation.authored_by_llm()); + let evidence = complete_validation_run(&receipt, &request(), &offered, &observation) + .expect("complete"); + assert_eq!( + evidence.schema_version(), + SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION + ); + assert_eq!(evidence.run_id(), receipt.run_id()); + assert_eq!(evidence.binding_sha256(), receipt.binding_sha256()); + assert_eq!(evidence.recovery_sha256().len(), 64); + assert_eq!(evidence.tenant_workspace_id(), "tenant-workspace-1"); + assert!(evidence.se_gate_accepted()); + assert_eq!( + evidence.output_profile(), + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE + ); + assert_eq!(evidence.eligible_evidence_count(), 2); + assert!((evidence.se_gate_k() - 3.0).abs() < f64::EPSILON); + assert_eq!(evidence.snapshot_id(), "snapshot-1"); + assert_eq!(evidence.knowledge_cutoff(), "2026-08-01T00:00:00Z"); + assert_eq!(evidence.model(), VALIDATION_CPU_F64_MODEL); + assert_eq!(evidence.seed(), 11); + assert_eq!(evidence.backend(), VALIDATION_BACKEND); + assert_eq!(evidence.precision(), VALIDATION_PRECISION); + assert!((WILSON_Z - 1.96).abs() < f64::EPSILON); + let json = evidence.to_json().expect("json"); + assert!(json.contains(SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION)); + assert!(json.contains("rmse")); + assert!(json.contains("recovery_sha256")); + assert_eq!(evidence.sha256().expect("digest").len(), 64); + assert!(evidence.to_human_summary().contains("foundation-recovery")); + let rejected = recovery( + &receipt, + vec![0.0, 1.0, 2.0, 3.0, 4.0], + vec![10.0, 11.0, 12.0, 13.0, 14.0], + 3.0, + ); + let refused = + complete_validation_run(&receipt, &request(), &offered, &rejected).expect("refused"); + assert!(!refused.se_gate_accepted()); + assert_ne!(refused.recovery_sha256(), evidence.recovery_sha256()); + } + + #[test] + fn submit_and_complete_fail_closed_on_trust_and_shape_errors() { + let offered = corpus(vec![unit("evidence-a", "2026-07-10T00:00:00Z")]); + let mut wrong_profile = request(); + wrong_profile.output_profile = "validation-report".into(); + assert_eq!( + submit_validation_run(&wrong_profile, &offered, 1, 3.0), + Err(AnalysisEngineError::InvalidValidationProfile) + ); + let mut wrong_model = request(); + wrong_model.model_contract_version = "temporal-evidence-v1".into(); + assert_eq!( + submit_validation_run(&wrong_model, &offered, 1, 3.0), + Err(AnalysisEngineError::InvalidValidationProfile) + ); + let mismatched = AnalysisCorpus::new( + "snapshot-other", + vec![unit("evidence-a", "2026-07-10T00:00:00Z")], + ) + .expect("other"); + assert_eq!( + submit_validation_run(&request(), &mismatched, 1, 3.0), + Err(AnalysisEngineError::SnapshotMismatch) + ); + let duplicate = corpus(vec![ + unit("same", "2026-07-10T00:00:00Z"), + unit("same", "2026-07-11T00:00:00Z"), + ]); + assert_eq!( + submit_validation_run(&request(), &duplicate, 1, 3.0), + Err(AnalysisEngineError::DuplicateEvidence) + ); + let empty = AnalysisCorpus::new("snapshot-1", Vec::new()).expect("empty"); + assert_eq!( + submit_validation_run(&request(), &empty, 1, 3.0), + Err(AnalysisEngineError::InvalidEvidence) + ); + let late = corpus(vec![unit("late", "2026-08-02T00:00:00Z")]); + assert_eq!( + submit_validation_run(&request(), &late, 1, 3.0), + Err(AnalysisEngineError::NoEligibleEvidence) + ); + let mut invalid_request = request(); + invalid_request.idempotency_key.clear(); + assert!(matches!( + submit_validation_run(&invalid_request, &offered, 1, 3.0), + Err(AnalysisEngineError::Api(_)) + )); + let mut invalid_cutoff = request(); + invalid_cutoff.knowledge_cutoff = "not-a-time".into(); + assert!(matches!( + submit_validation_run(&invalid_cutoff, &offered, 1, 3.0), + Err(AnalysisEngineError::Api(_)) + )); + } + + #[test] + fn complete_refuses_llm_authored_recovery_and_tampered_receipt() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 3, 3.0).expect("submit"); + let llm = RecoveryObservation::new( + &receipt, + "llm-study", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + true, + ) + .expect("llm"); + assert!(llm.authored_by_llm()); + assert_eq!( + complete_validation_run(&receipt, &request(), &offered, &llm), + Err(AnalysisEngineError::LlmAuthoredRecovery) + ); + let mut tampered = receipt.clone(); + tampered.run_id = "tepp-validation-deadbeefdeadbeefdeadbeefdeadbeef".into(); + let observation = recovery(&receipt, vec![1.0, 2.0, 3.0], vec![1.0, 2.0, 3.0], 3.0); + assert_eq!( + complete_validation_run(&tampered, &request(), &offered, &observation), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut profile_tampered = receipt.clone(); + profile_tampered.output_profile = "tampered-profile".into(); + assert_eq!( + complete_validation_run(&profile_tampered, &request(), &offered, &observation), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut k_tampered = observation.clone(); + k_tampered.se_gate_k = 8.0; + assert_eq!( + complete_validation_run(&receipt, &request(), &offered, &k_tampered), + Err(AnalysisEngineError::BindingMismatch) + ); + } + + #[test] + fn complete_refuses_recovery_stamped_to_a_different_run() { + let first = corpus(vec![unit("evidence-a", "2026-07-10T00:00:00Z")]); + let second = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt_a = submit_validation_run(&request(), &first, 3, 3.0).expect("a"); + let receipt_b = submit_validation_run(&request(), &second, 3, 3.0).expect("b"); + assert_ne!(receipt_a.run_id(), receipt_b.run_id()); + let foreign = recovery(&receipt_a, vec![1.0, 2.0, 3.0], vec![1.0, 2.0, 3.0], 3.0); + assert_eq!( + complete_validation_run(&receipt_b, &request(), &second, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let other_seed = submit_validation_run(&request(), &first, 9, 3.0).expect("seed"); + assert_eq!( + complete_validation_run(&other_seed, &request(), &first, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut other_tenant = request(); + other_tenant.tenant_workspace_id = "tenant-workspace-other".into(); + let other_tenant_receipt = + submit_validation_run(&other_tenant, &first, 3, 3.0).expect("tenant"); + assert_eq!( + complete_validation_run(&other_tenant_receipt, &other_tenant, &first, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + } + + #[test] + fn complete_refuses_nonfinite_recovery_and_invalid_observation() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 3, 3.0).expect("submit"); + let nan = RecoveryObservation::new( + &receipt, + "nan-study", + vec![1.0, 2.0, 3.0], + vec![f64::NAN, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + false, + ) + .expect("nan observation"); + assert_eq!( + complete_validation_run(&receipt, &request(), &offered, &nan), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput + )) + ); + assert_eq!( + two_point(&receipt, "", 3.0, false), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + two_point( + &receipt, + "x".repeat(MAX_ANALYSIS_IDENTIFIER_BYTES + 1), + 3.0, + false + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + two_point(&receipt, "nan-k", f64::NAN, false), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput + )) + ); + assert_eq!( + two_point(&receipt, "neg-k", -1.0, false), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration + )) + ); + let oversized = vec![0.0; MAX_RECOVERY_VECTOR_LEN + 1]; + assert_eq!( + RecoveryObservation::new( + &receipt, + "oversized", + oversized.clone(), + oversized.clone(), + oversized.clone(), + oversized.clone(), + oversized.clone(), + oversized, + 3.0, + false, + ), + Err(AnalysisEngineError::LimitExceeded) + ); + let converted: AnalysisEngineError = ValidationError::InvalidInput.into(); + assert_eq!(converted.to_string(), "invalid validation input"); + } + + #[test] + fn se_gate_k_is_pre_registered_and_empty_vectors_fail_closed() { + let offered = corpus(vec![ + unit("evidence-a", "2026-07-10T00:00:00Z"), + unit("evidence-b", "2026-07-15T00:00:00Z"), + ]); + let receipt = submit_validation_run(&request(), &offered, 3, 3.0).expect("submit"); + assert_eq!( + two_point(&receipt, "huge-k", MAX_SE_GATE_K + 0.01, false), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration + )) + ); + assert_eq!( + two_point(&receipt, "post-hoc-k", 4.0, false), + Err(AnalysisEngineError::BindingMismatch) + ); + assert_eq!( + submit_validation_run(&request(), &offered, 3, MAX_SE_GATE_K + 0.01), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidConfiguration + )) + ); + assert_eq!( + submit_validation_run(&request(), &offered, 3, f64::NAN), + Err(AnalysisEngineError::Validation( + ValidationError::InvalidInput + )) + ); + assert!(submit_validation_run(&request(), &offered, 3, MAX_SE_GATE_K).is_ok()); + assert_eq!( + RecoveryObservation::new( + &receipt, + "empty", + Vec::new(), + Vec::new(), + Vec::new(), + Vec::new(), + Vec::new(), + Vec::new(), + 3.0, + false, + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + RecoveryObservation::new( + &receipt, + "mismatch", + vec![1.0, 2.0], + vec![1.0], + vec![0.0, 1.0], + vec![2.0, 3.0], + vec![1.0, 2.0], + vec![1.0, 2.0], + 3.0, + false, + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} diff --git a/crates/analysis_engine/tests/loopback_execute_contract.rs b/crates/analysis_engine/tests/loopback_execute_contract.rs new file mode 100644 index 000000000..4ea2333af --- /dev/null +++ b/crates/analysis_engine/tests/loopback_execute_contract.rs @@ -0,0 +1,268 @@ +//! GAP-003A engine-on-loopback scientific-acceptance contract. + +use analysis_engine::{ + ANALYSIS_RUN_EXECUTE_CONTRACT_VERSION, SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, + SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, ScientificAcceptanceLoopbackService, + VALIDATION_CPU_F64_MODEL, +}; +use tepp_api::{ + ANALYSIS_RUN_CONTRACT_VERSION, AnalysisRunAccepted, AnalysisRunRequest, + NARUON_ANALYSIS_RUN_PATH, NARUON_CONSUMER_CODE, SCIENTIFIC_ACCEPTANCE_HTTP_PROFILE, + SCIENTIFIC_ACCEPTANCE_HTTP_SCHEMA, +}; + +fn request(profile: &str, model: &str) -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: ANALYSIS_RUN_CONTRACT_VERSION, + idempotency_key: "idem-loopback-execute".into(), + tenant_workspace_id: "tenant-workspace-execute".into(), + snapshot_id: "snapshot-execute".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: model.into(), + output_profile: profile.into(), + } +} + +fn http_post(path: &str, body: &str, consumer: &str, idempotency_key: &str) -> String { + format!( + "POST {path} HTTP/1.1\r\nHost: 127.0.0.1\r\ncontent-type: application/json\r\ntepp-consumer: {consumer}\r\ntepp-contract-version: 1\r\nidempotency-key: {idempotency_key}\r\ncontent-length: {}\r\n\r\n{body}", + body.len() + ) +} + +fn http_get(path: &str, consumer: &str, idempotency_key: &str) -> String { + format!( + "GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\ncontent-type: application/json\r\ntepp-consumer: {consumer}\r\ntepp-contract-version: 1\r\nidempotency-key: {idempotency_key}\r\ncontent-length: 0\r\n\r\n" + ) +} + +fn execute_body(run_id: &str, authored_by_llm: bool) -> String { + serde_json::json!({ + "contract_version": ANALYSIS_RUN_EXECUTE_CONTRACT_VERSION, + "run_id": run_id, + "idempotency_key": "idem-loopback-execute", + "seed": 42, + "se_gate_k": 3.0, + "completed_at": "2026-08-31T10:00:00Z", + "study_label": "loopback-recovery", + "authored_by_llm": authored_by_llm, + "corpus": { + "snapshot_id": "snapshot-execute", + "evidence_units": [ + { + "evidence_id": "evidence-1", + "event_time": "2026-07-01T00:00:00Z", + "available_time": "2026-07-10T00:00:00Z", + "membership_count": 1 + }, + { + "evidence_id": "evidence-2", + "event_time": "2026-07-01T00:00:00Z", + "available_time": "2026-07-20T00:00:00Z", + "membership_count": 1 + }, + { + "evidence_id": "future", + "event_time": "2026-07-01T00:00:00Z", + "available_time": "2026-08-02T00:00:00Z", + "membership_count": 1 + } + ] + }, + "truth": [0.70, 0.55, 0.40, -0.20, 0.85], + "recovered": [0.70, 0.55, 0.40, -0.20, 0.85], + "interval_lower": [0.50, 0.35, 0.20, -0.40, 0.65], + "interval_upper": [0.90, 0.75, 0.60, 0.00, 1.00], + "truth_times": [1.0, 2.0, 3.0, 4.0, 5.0], + "recovered_times": [1.1, 1.9, 3.2, 3.8, 5.1] + }) + .to_string() +} + +fn accept_run( + service: &mut ScientificAcceptanceLoopbackService, + profile: &str, + model: &str, +) -> AnalysisRunAccepted { + let run = request(profile, model); + let accepted = service.handle_http_request(&http_post( + NARUON_ANALYSIS_RUN_PATH, + &run.to_json().expect("create json"), + NARUON_CONSUMER_CODE, + run.idempotency_key.as_str(), + )); + assert_eq!(accepted.status_code, 202, "{}", accepted.body); + assert!(!accepted.body.contains("rmse")); + assert!(!accepted.body.contains("scientific_acceptance")); + AnalysisRunAccepted::from_json(&accepted.body).expect("accepted") +} + +#[test] +fn execute_produces_scientific_acceptance_without_caller_artifact() { + let mut service = ScientificAcceptanceLoopbackService::new(); + let accepted = accept_run( + &mut service, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, + VALIDATION_CPU_F64_MODEL, + ); + let body = execute_body(&accepted.run_id, false); + assert!(!body.contains("scientific_acceptance_json")); + assert!(!body.contains("rmse")); + let execute = service.handle_http_request(&http_post( + &format!("{NARUON_ANALYSIS_RUN_PATH}/{}/execute", accepted.run_id), + &body, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )); + assert_eq!(execute.status_code, 200, "{}", execute.body); + assert!(execute.body.contains(SCIENTIFIC_ACCEPTANCE_HTTP_SCHEMA)); + assert!(execute.body.contains("scientific_acceptance")); + assert!(execute.body.contains("\"succeeded\"")); + + let get = service.handle_http_request(&http_get( + &format!("{NARUON_ANALYSIS_RUN_PATH}/{}", accepted.run_id), + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )); + assert_eq!(get.status_code, 200, "{}", get.body); + assert!(get.body.contains(SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION)); + assert!(get.body.contains(SCIENTIFIC_ACCEPTANCE_HTTP_PROFILE)); + assert!(get.body.contains("scientific_acceptance")); + assert!(get.body.contains("rmse")); +} + +#[test] +fn execute_fail_closed_for_hostile_bodies_and_bindings() { + let mut service = ScientificAcceptanceLoopbackService::new(); + let accepted = accept_run( + &mut service, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, + VALIDATION_CPU_F64_MODEL, + ); + let body = execute_body(&accepted.run_id, false); + let path = format!("{NARUON_ANALYSIS_RUN_PATH}/{}/execute", accepted.run_id); + + let with_rmse = body.replacen('{', r#"{"rmse":0.1,"#, 1); + assert_eq!( + service + .handle_http_request(&http_post( + &path, + &with_rmse, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )) + .status_code, + 400 + ); + + let with_artifact = body.replacen('{', r#"{"scientific_acceptance_json":"{}","#, 1); + assert_eq!( + service + .handle_http_request(&http_post( + &path, + &with_artifact, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )) + .status_code, + 400 + ); + + let llm = execute_body(&accepted.run_id, true); + assert_eq!( + service + .handle_http_request(&http_post( + &path, + &llm, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )) + .status_code, + 400 + ); + + assert_eq!( + service + .handle_http_request(&http_post( + &path, + &body, + "lineageweave", + "idem-loopback-execute", + )) + .status_code, + 400 + ); + + assert_eq!( + service + .handle_http_request(&http_post( + &format!("{NARUON_ANALYSIS_RUN_PATH}/tepp-run-999/execute"), + &execute_body("tepp-run-999", false), + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )) + .status_code, + 400 + ); + + let wrong_version = body.replace( + &format!("\"contract_version\":{ANALYSIS_RUN_EXECUTE_CONTRACT_VERSION}"), + "\"contract_version\":9", + ); + assert_eq!( + service + .handle_http_request(&http_post( + &path, + &wrong_version, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )) + .status_code, + 422 + ); + + let ok = service.handle_http_request(&http_post( + &path, + &body, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )); + assert_eq!(ok.status_code, 200, "{}", ok.body); + assert_eq!( + service + .handle_http_request(&http_post( + &path, + &body, + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )) + .status_code, + 400 + ); +} + +#[test] +fn execute_refuses_wrong_profile_and_leaves_accepted_metric_free() { + let mut service = ScientificAcceptanceLoopbackService::new(); + let accepted = accept_run( + &mut service, + "calibrated_event_measurement", + VALIDATION_CPU_F64_MODEL, + ); + let execute = service.handle_http_request(&http_post( + &format!("{NARUON_ANALYSIS_RUN_PATH}/{}/execute", accepted.run_id), + &execute_body(&accepted.run_id, false), + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )); + assert_eq!(execute.status_code, 400, "{}", execute.body); + let get = service.handle_http_request(&http_get( + &format!("{NARUON_ANALYSIS_RUN_PATH}/{}", accepted.run_id), + NARUON_CONSUMER_CODE, + "idem-loopback-execute", + )); + assert_eq!(get.status_code, 200, "{}", get.body); + assert!(get.body.contains("\"accepted\"")); + assert!(!get.body.contains("rmse")); + assert!(!get.body.contains("scientific_acceptance")); +} diff --git a/crates/analysis_engine/tests/validation_run_contract.rs b/crates/analysis_engine/tests/validation_run_contract.rs new file mode 100644 index 000000000..f7a7bc7be --- /dev/null +++ b/crates/analysis_engine/tests/validation_run_contract.rs @@ -0,0 +1,168 @@ +//! GAP-003A scientific-acceptance validation-run contract. + +use analysis_engine::{ + AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, RecoveryObservation, + SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE, SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION, + VALIDATION_CPU_F64_MODEL, complete_validation_run, submit_validation_run, +}; +use temporal_core::{AvailableTime, EventTime}; +use tepp_api::AnalysisRunRequest; + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "idem-validation-contract".into(), + tenant_workspace_id: "tenant-workspace-contract".into(), + snapshot_id: "snapshot-contract".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: VALIDATION_CPU_F64_MODEL.into(), + output_profile: SCIENTIFIC_ACCEPTANCE_OUTPUT_PROFILE.into(), + } +} + +fn unit(id: &str, available: &str) -> AnalysisEvidenceUnit { + AnalysisEvidenceUnit::new( + id, + EventTime::parse_rfc3339("2026-07-01T00:00:00Z").expect("event"), + AvailableTime::parse_rfc3339(available).expect("available"), + 1, + ) + .expect("unit") +} + +#[test] +fn durable_run_binds_cutoff_eligible_evidence_and_emits_acceptance_evidence() { + let corpus = AnalysisCorpus::new( + "snapshot-contract", + vec![ + unit("evidence-2", "2026-07-20T00:00:00Z"), + unit("evidence-1", "2026-07-10T00:00:00Z"), + unit("future", "2026-08-02T00:00:00Z"), + ], + ) + .expect("corpus"); + let receipt = submit_validation_run(&request(), &corpus, 42, 3.0).expect("submit"); + assert_eq!(receipt.eligible_evidence_count(), 2); + assert!((receipt.se_gate_k() - 3.0).abs() < f64::EPSILON); + assert!(!receipt.to_json().expect("json").contains("rmse")); + let observation = RecoveryObservation::new( + &receipt, + "contract-recovery", + vec![0.70, 0.55, 0.40, -0.20, 0.85], + vec![0.70, 0.55, 0.40, -0.20, 0.85], + vec![0.50, 0.35, 0.20, -0.40, 0.65], + vec![0.90, 0.75, 0.60, 0.00, 1.00], + vec![1.0, 2.0, 3.0, 4.0, 5.0], + vec![1.1, 1.9, 3.2, 3.8, 5.1], + 3.0, + false, + ) + .expect("observation"); + let evidence = + complete_validation_run(&receipt, &request(), &corpus, &observation).expect("complete"); + assert_eq!( + evidence.schema_version(), + SCIENTIFIC_ACCEPTANCE_SCHEMA_VERSION + ); + assert_eq!(evidence.run_id(), receipt.run_id()); + assert_eq!(evidence.recovery_sha256().len(), 64); + assert!(evidence.se_gate_accepted()); + assert!(evidence.to_json().expect("json").contains("rmse")); +} + +#[test] +fn llm_authored_recovery_and_cutoff_empty_corpora_fail_closed() { + let corpus = AnalysisCorpus::new( + "snapshot-contract", + vec![unit("only-future", "2026-08-02T00:00:00Z")], + ) + .expect("corpus"); + assert_eq!( + submit_validation_run(&request(), &corpus, 1, 3.0), + Err(AnalysisEngineError::NoEligibleEvidence) + ); + let eligible = AnalysisCorpus::new( + "snapshot-contract", + vec![unit("evidence-1", "2026-07-10T00:00:00Z")], + ) + .expect("eligible"); + let receipt = submit_validation_run(&request(), &eligible, 1, 3.0).expect("submit"); + let llm = RecoveryObservation::new( + &receipt, + "llm-contract", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + true, + ) + .expect("llm"); + assert_eq!( + complete_validation_run(&receipt, &request(), &eligible, &llm), + Err(AnalysisEngineError::LlmAuthoredRecovery) + ); +} + +#[test] +fn recovery_from_a_different_run_or_tenant_fails_closed() { + let first = AnalysisCorpus::new( + "snapshot-contract", + vec![unit("evidence-1", "2026-07-10T00:00:00Z")], + ) + .expect("first"); + let second = AnalysisCorpus::new( + "snapshot-contract", + vec![ + unit("evidence-1", "2026-07-10T00:00:00Z"), + unit("evidence-2", "2026-07-20T00:00:00Z"), + ], + ) + .expect("second"); + let receipt_a = submit_validation_run(&request(), &first, 1, 3.0).expect("a"); + let receipt_b = submit_validation_run(&request(), &second, 1, 3.0).expect("b"); + let foreign = RecoveryObservation::new( + &receipt_a, + "foreign-recovery", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 3.0, + false, + ) + .expect("foreign"); + assert_eq!( + complete_validation_run(&receipt_b, &request(), &second, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let mut other_tenant = request(); + other_tenant.tenant_workspace_id = "tenant-workspace-other".into(); + let other_receipt = submit_validation_run(&other_tenant, &first, 1, 3.0).expect("tenant"); + assert_ne!(receipt_a.run_id(), other_receipt.run_id()); + assert_eq!( + complete_validation_run(&other_receipt, &other_tenant, &first, &foreign), + Err(AnalysisEngineError::BindingMismatch) + ); + let other_k = submit_validation_run(&request(), &first, 1, 4.0).expect("other k"); + assert_ne!(receipt_a.run_id(), other_k.run_id()); + assert_eq!( + RecoveryObservation::new( + &receipt_a, + "post-hoc-k", + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + vec![0.5, 1.5, 2.5], + vec![1.5, 2.5, 3.5], + vec![1.0, 2.0, 3.0], + vec![1.0, 2.0, 3.0], + 4.0, + false, + ), + Err(AnalysisEngineError::BindingMismatch) + ); +} diff --git a/crates/tepp_api/src/analysis_run_live.rs b/crates/tepp_api/src/analysis_run_live.rs index 624d5ef46..1112ac7ce 100644 --- a/crates/tepp_api/src/analysis_run_live.rs +++ b/crates/tepp_api/src/analysis_run_live.rs @@ -8,8 +8,9 @@ //! `tepp.scientific_acceptance.v1` only on a succeeded status whose request //! profile is `scientific_acceptance_v1`. `POST /v1/analysis-runs/{run_id}/running` //! and `POST /v1/analysis-runs/{run_id}/terminal` are the production -//! status-update path. Completed psychometric estimation remains outside this -//! crate. +//! status-update path. `POST /v1/analysis-runs/{run_id}/execute` is recognized +//! and refused here; `analysis_engine` owns engine execution on this listener. +//! Completed psychometric estimation remains outside this crate. use std::collections::HashMap; use std::io::Write; @@ -49,6 +50,22 @@ struct LiveAnalysisRun { scientific_acceptance_json: Option, } +/// Public copy of one accepted loopback run for engine glue. +/// +/// Scientific-acceptance bytes stay private on the live service. Engine +/// execution records them through [`AnalysisRunLiveService::record_loopback_status`]. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct LoopbackAnalysisRun { + /// Consumer identity that accepted the run. + pub consumer: String, + /// Original create request. + pub request: AnalysisRunRequest, + /// Accepted receipt. + pub accepted: AnalysisRunAccepted, + /// Current lifecycle status. + pub status: AnalysisRunStatus, +} + /// Loopback HTTP/1.1 analysis-run service shared by published CWL consumers. /// /// The service accepts only Naruon and `LineageWeave` consumer identities. Its @@ -197,7 +214,9 @@ impl AnalysisRunLiveService { AnalysisRunLiveRoute::Terminal { run_id } => { self.post_terminal_status(&run_id, &headers, body) } - AnalysisRunLiveRoute::Status { .. } => Err(ApiError::InvalidWirePayload), + AnalysisRunLiveRoute::Execute { .. } | AnalysisRunLiveRoute::Status { .. } => { + Err(ApiError::InvalidWirePayload) + } } } @@ -285,9 +304,17 @@ impl AnalysisRunLiveService { /// /// HTTP `POST /v1/analysis-runs/{run_id}/running` and /// `POST /v1/analysis-runs/{run_id}/terminal` call this after the request - /// is authenticated and parsed. Psychometric execution remains outside this - /// crate; this records the supplied status only. - pub(crate) fn record_loopback_status( + /// is authenticated and parsed. `analysis_engine` records a produced + /// scientific-acceptance artifact through the same method so GET can return + /// `tepp.scientific_acceptance.v1` without a caller-supplied artifact. + /// Psychometric execution remains outside this crate; this records the + /// supplied status only. + /// + /// # Errors + /// + /// Returns [`ApiError::InvalidWirePayload`] for an unknown run, a binding + /// mismatch, or an HTTP-layer scientific-acceptance gate failure. + pub fn record_loopback_status( &mut self, run_id: &str, status: AnalysisRunStatus, @@ -316,6 +343,66 @@ impl AnalysisRunLiveService { Ok(()) } + /// Return a copy of one accepted loopback run. + /// + /// # Errors + /// + /// Returns [`ApiError::InvalidWirePayload`] when the run identity is unknown + /// or internally inconsistent. + pub fn loopback_run(&self, run_id: &str) -> Result { + let replay_key = self + .runs_by_id + .get(run_id) + .ok_or(ApiError::InvalidWirePayload)?; + let stored = self + .accepted_runs + .get(replay_key) + .ok_or(ApiError::InvalidWirePayload)?; + if stored.accepted.run_id != run_id { + return Err(ApiError::InvalidWirePayload); + } + Ok(LoopbackAnalysisRun { + consumer: stored.consumer.clone(), + request: stored.request.clone(), + accepted: stored.accepted.clone(), + status: stored.status.clone(), + }) + } + + /// Serialize the current loopback status GET body for one run. + /// + /// # Errors + /// + /// Returns the same fail-closed errors as the GET status path. + pub fn loopback_status_json(&self, run_id: &str) -> Result { + let replay_key = self + .runs_by_id + .get(run_id) + .ok_or(ApiError::InvalidWirePayload)?; + let stored = self + .accepted_runs + .get(replay_key) + .ok_or(ApiError::InvalidWirePayload)?; + status_http_json( + &stored.status, + &stored.request, + stored.scientific_acceptance_json.as_deref(), + ) + } + + /// Authorize consumer-scoped loopback headers against this listener. + /// + /// # Errors + /// + /// Returns [`ApiError::InvalidWirePayload`] for missing contract headers + /// and [`ApiError::AuthorizationDenied`] for credential headers. + pub fn authorize_loopback_headers<'a>( + &self, + headers: &'a HashMap, + ) -> Result<&'a str, ApiError> { + require_headers(headers, self.bound_addr, true) + } + fn post_running_status( &mut self, run_id: &str, @@ -1484,6 +1571,17 @@ mod tests { accepted_dto.run_id, run.idempotency_key ); assert_eq!(service.handle_http_request(&put).status_code, 400); + assert_eq!( + service + .handle_http_request(&lifecycle_post( + &format!("{NARUON_ANALYSIS_RUN_PATH}/{}/execute", accepted_dto.run_id), + "{}", + NARUON_CONSUMER_CODE, + run.idempotency_key.as_str(), + )) + .status_code, + 400 + ); assert_eq!( service .handle_http_request(&lifecycle_post( diff --git a/crates/tepp_api/src/analysis_run_status_http.rs b/crates/tepp_api/src/analysis_run_status_http.rs index 08ea5d114..fc9f7e464 100644 --- a/crates/tepp_api/src/analysis_run_status_http.rs +++ b/crates/tepp_api/src/analysis_run_status_http.rs @@ -22,6 +22,8 @@ pub(crate) enum AnalysisRunLiveRoute { Running { run_id: String }, /// `POST /v1/analysis-runs/{run_id}/terminal` Terminal { run_id: String }, + /// `POST /v1/analysis-runs/{run_id}/execute` + Execute { run_id: String }, } /// Build a provider-owned `GET` analysis-run status exchange. @@ -137,7 +139,7 @@ fn require_run_id_length(run_id: &str) -> Result<(), ApiError> { } } -/// Parse `GET` status and `POST` running/terminal loopback routes. +/// Parse `GET` status and `POST` running/terminal/execute loopback routes. /// /// # Errors /// @@ -171,10 +173,35 @@ pub(crate) fn parse_analysis_run_live_route(path: &str) -> Result { + let run_id = decode_path_segment(encoded_id)?; + require_run_id_length(&run_id)?; + Ok(AnalysisRunLiveRoute::Execute { run_id }) + } Some(_) => Err(ApiError::InvalidWirePayload), } } +/// Extract the opaque run identity from `POST /v1/analysis-runs/{run_id}/execute`. +/// +/// The loopback listener in this crate refuses the execute suffix. Engine glue +/// in `analysis_engine` owns the execution path and uses this parser so hostile +/// encodings fail closed before any scientific work. +/// +/// # Errors +/// +/// Returns [`ApiError::InvalidWirePayload`] for a non-execute path or a hostile +/// encoding, and [`ApiError::LimitExceeded`] when the decoded identity exceeds +/// [`ANALYSIS_RUN_ID_MAX_LEN`]. +pub fn analysis_run_execute_path_run_id(path: &str) -> Result { + match parse_analysis_run_live_route(path)? { + AnalysisRunLiveRoute::Execute { run_id } => Ok(run_id), + AnalysisRunLiveRoute::Status { .. } + | AnalysisRunLiveRoute::Running { .. } + | AnalysisRunLiveRoute::Terminal { .. } => Err(ApiError::InvalidWirePayload), + } +} + /// Extract the opaque run identity from `GET /v1/analysis-runs/{run_id}`. /// /// # Errors @@ -185,9 +212,9 @@ pub(crate) fn parse_analysis_run_live_route(path: &str) -> Result Result { match parse_analysis_run_live_route(path)? { AnalysisRunLiveRoute::Status { run_id } => Ok(run_id), - AnalysisRunLiveRoute::Running { .. } | AnalysisRunLiveRoute::Terminal { .. } => { - Err(ApiError::InvalidWirePayload) - } + AnalysisRunLiveRoute::Running { .. } + | AnalysisRunLiveRoute::Terminal { .. } + | AnalysisRunLiveRoute::Execute { .. } => Err(ApiError::InvalidWirePayload), } } @@ -240,6 +267,7 @@ mod tests { } #[test] + #[allow(clippy::too_many_lines)] fn decodes_status_path_identities_and_refuses_hostile_segments() { assert_eq!( analysis_run_status_path_run_id("/v1/analysis-runs/tepp-run-1").expect("plain"), @@ -314,6 +342,29 @@ mod tests { run_id: "tepp-run-1".into() } ); + assert_eq!( + parse_analysis_run_live_route("/v1/analysis-runs/tepp-run-1/execute").expect("execute"), + AnalysisRunLiveRoute::Execute { + run_id: "tepp-run-1".into() + } + ); + assert_eq!( + analysis_run_execute_path_run_id("/v1/analysis-runs/tepp-run-1/execute") + .expect("execute id"), + "tepp-run-1" + ); + assert_eq!( + analysis_run_execute_path_run_id("/v1/analysis-runs/tepp-run-1"), + Err(ApiError::InvalidWirePayload) + ); + assert_eq!( + analysis_run_status_path_run_id("/v1/analysis-runs/tepp-run-1/execute"), + Err(ApiError::InvalidWirePayload) + ); + assert_eq!( + parse_analysis_run_live_route("/v1/analysis-runs/tepp-run-1/execute/extra"), + Err(ApiError::InvalidWirePayload) + ); assert_eq!( parse_analysis_run_live_route("/v1/analysis-runs/tepp-run-1/running/extra"), Err(ApiError::InvalidWirePayload) diff --git a/crates/tepp_api/src/lib.rs b/crates/tepp_api/src/lib.rs index facecf471..19774f10c 100644 --- a/crates/tepp_api/src/lib.rs +++ b/crates/tepp_api/src/lib.rs @@ -14,7 +14,9 @@ //! only a succeeded status with profile `scientific_acceptance_v1` may return //! `tepp.scientific_acceptance.v1`. `POST /v1/analysis-runs/{run_id}/running` //! and `POST /v1/analysis-runs/{run_id}/terminal` record those statuses on the -//! same loopback listener. +//! same loopback listener. `POST /v1/analysis-runs/{run_id}/execute` is refused +//! here; `analysis_engine` owns engine execution so a scientific-acceptance run +//! can produce `tepp.scientific_acceptance.v1` without a caller-supplied artifact. mod analysis_result; mod analysis_run; @@ -85,9 +87,11 @@ pub use analysis_run_lifecycle_http::naruon_analysis_run_running_exchange; /// Provider-owned terminal-status HTTP exchange builder. pub use analysis_run_lifecycle_http::naruon_analysis_run_terminal_exchange; /// Consumer-neutral loopback analysis-run service. -pub use analysis_run_live::AnalysisRunLiveService; +pub use analysis_run_live::{AnalysisRunLiveService, LoopbackAnalysisRun}; /// Analysis-run status HTTP exchange re-exports. -pub use analysis_run_status_http::{ANALYSIS_RUN_ID_MAX_LEN, naruon_analysis_run_status_exchange}; +pub use analysis_run_status_http::{ + ANALYSIS_RUN_ID_MAX_LEN, analysis_run_execute_path_run_id, naruon_analysis_run_status_exchange, +}; /// Corpus-split leakage-audit contract version. pub use corpus_split_manifest::CORPUS_SPLIT_MANIFEST_CONTRACT_VERSION; /// Versioned corpus-split leakage-audit manifest. @@ -110,6 +114,8 @@ pub use export::GraphMlExport; pub use export::JsonLdExport; /// Reproducibility manifest. pub use export::ReproducibilityManifest; +/// Parsed loopback HTTP/1.1 request parts for engine glue. +pub use live_http::{LoopbackHttpParts, parse_loopback_http_parts}; /// Output profile that authorizes scientific-acceptance on a loopback GET. pub use scientific_acceptance_http::SCIENTIFIC_ACCEPTANCE_HTTP_PROFILE; /// Schema identity returned on a succeeded scientific-acceptance GET. diff --git a/crates/tepp_api/src/live_http.rs b/crates/tepp_api/src/live_http.rs index ed8d7e564..274e650ba 100644 --- a/crates/tepp_api/src/live_http.rs +++ b/crates/tepp_api/src/live_http.rs @@ -53,7 +53,41 @@ pub(crate) fn read_http_request_with_limit( Ok(format!("{header_text}{body_text}")) } -/// Split one complete request into its header block and UTF-8 body. +/// Parsed loopback HTTP/1.1 request already in memory. +#[derive(Debug)] +pub struct LoopbackHttpParts<'a> { + /// HTTP method token from the request line. + pub method: &'a str, + /// Request-target path. Query strings are refused. + pub path: &'a str, + /// Lowercased unique headers. + pub headers: HashMap, + /// UTF-8 body whose length matches `Content-Length`. + pub body: &'a str, +} + +/// Parse one complete loopback HTTP/1.1 request already in memory. +/// +/// # Errors +/// +/// Returns [`ApiError::InvalidWirePayload`] for framing, version, or header +/// violations and [`ApiError::LimitExceeded`] when the header block or body +/// exceeds the configured bound. +pub fn parse_loopback_http_parts( + request: &str, + maximum_body_bytes: usize, +) -> Result, ApiError> { + let (header_block, body) = split_request_with_limit(request, maximum_body_bytes)?; + let mut lines = header_block.split("\r\n"); + let (method, path) = parse_request_line(lines.next().unwrap_or(""))?; + let headers = parse_headers(&mut lines)?; + Ok(LoopbackHttpParts { + method, + path, + headers, + body, + }) +} pub(crate) fn split_request(request: &str) -> Result<(&str, &str), ApiError> { split_request_with_limit(request, DEFAULT_ANALYSIS_RUN_BYTE_LIMIT) } diff --git a/docs/API_CONTRACT.md b/docs/API_CONTRACT.md index 87a7a4a87..81762d265 100644 --- a/docs/API_CONTRACT.md +++ b/docs/API_CONTRACT.md @@ -105,7 +105,12 @@ bodies stay metric-free, and only a succeeded status with profile `POST /v1/analysis-runs/{run_id}/running` and `POST /v1/analysis-runs/{run_id}/terminal` are the production loopback status-update path that records those statuses; they do not persist and do not -execute psychometric estimation. Production TLS remains a later adapter. +execute psychometric estimation. `POST /v1/analysis-runs/{run_id}/execute` on +`analysis_engine::ScientificAcceptanceLoopbackService` is the engine-owned +loopback path that produces `tepp.scientific_acceptance.v1` for a +`scientific_acceptance_v1` run without a caller-supplied artifact. The execute +body carries corpus, recovery, seed, and the pre-registered SE-gate multiplier +and refuses `scientific_acceptance_json`. Production TLS remains a later adapter. The stacked `analysis_engine` slice provides the first executable service-side path behind these DTOs. It consumes a bounded identity-free snapshot, excludes diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index b90843a85..f92f42e2d 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -50,11 +50,13 @@ The full APA 7th standards/literature register remains `docs/research/standards- | leakage-safe availability/cutoff snapshots | PRD; ADR 0002/0013 | `corpus_split` on protected main | implemented-main | | Unicode NFC/NFD canonical identity for split leakage | ADR 0004/0008/0013; UAX #15 | `corpus_split` canonical-equivalence links on active PR #59 | active-PR | | recovery metrics (RMSE, bias, coverage, graph, temporal order, Monte Carlo SE gates) | PRD; Test Strategy; ADR 0007/0014 | `validation_core` on protected main (PR #19); SE-aware Monte Carlo gates included | implemented-main | +| durable validation-run scientific acceptance evidence | ADR 0026; ADR 0002/0008/0014; GAP-003A; National Academies (2019); Wasserstein & Lazar (2016); Wilson (1927) | `analysis_engine` `submit_validation_run` / `complete_validation_run` on this PR: hash-stable cutoff-safe tenant/profile/`k` binding, metric-free private-field receipt, recovery stamped to that `run_id` with pre-registered `k` (`MAX_SE_GATE_K` = 8), `tepp.scientific_acceptance.v1` RMSE/bias/coverage/temporal-order/SE-aware evidence with recovery digest; LLM/NaN/empty/duplicate/snapshot/foreign-run/post-hoc-k/tampered-profile/oversized/cutoff-empty fail closed; not implemented-main; Postgres persistence remains GAP-003B | active-PR | | PostgreSQL bitemporal/lineage persistence | ADR 0013; Architecture/ERD | `persistence_postgres` migration contracts, in-memory adapters, live SQL session/document SQL port, tenant RLS (`0002` + session GUC/role helpers), `DATABASE_URL` SQLx gate, optional session-affine `live-sqlx` `PgPool` driver, exact-head live PostgreSQL CI with isolation proof, append-only immutability triggers (`0004`), temporal interval ordering CHECKs (`0005`), typed membership assignment (`0006` implemented-main), event-relation/mention/instance SQL (#37–#39 implemented-main), source-artifact SQL (#40 implemented-main), audit-event SQL (#41 implemented-main), concurrent document-write stress (#43 implemented-main), backup/restore integrity revalidation (#44 implemented-main), `revision_order` later-revision system-time ordering implemented-main, entity/project target SQL on PR #131; remaining physical ERD constraints | partial | | known-truth temporal/event simulation manifests | PRD; TRD; Test Strategy | `tepp_simulation` on protected main; recovery metrics in `validation_core` | implemented-main | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); request-bound terminal result active in PR #157; HTTP service remains accepted-target; the `orchestrator_live` loopback interpretation listener is on this PR | partial | | loopback analysis-run scientific-acceptance GET | ADR 0027; API contract; RFC 9110; FIPS 180-4 | `tepp_api` `GET /v1/analysis-runs/{run_id}` on `AnalysisRunLiveService` (#359): accepted/running stay metric-free; `tepp.scientific_acceptance.v1` only on succeeded `scientific_acceptance_v1`; not implemented-main | active-PR | | loopback analysis-run scientific-acceptance lifecycle POST | ADR 0028; API contract; RFC 9110; FIPS 180-4 | `tepp_api` `POST /v1/analysis-runs/{run_id}/running` and `/terminal` on `AnalysisRunLiveService` (this PR): production status-update path; accepted/running stay metric-free; `tepp.scientific_acceptance.v1` only after succeeded `scientific_acceptance_v1`; not implemented-main | active-PR | +| loopback scientific-acceptance engine execute | ADR 0032; ADR 0026/0027/0028; API contract; National Academies (2019); Wasserstein & Lazar (2016); Wilson (1927) | `analysis_engine` `POST /v1/analysis-runs/{run_id}/execute` on `ScientificAcceptanceLoopbackService` (this PR): engine produces `tepp.scientific_acceptance.v1` without caller-supplied artifact; metric keys/LLM/wrong-profile/unknown-run/consumer-mismatch/already-terminal fail closed; not implemented-main; Postgres persistence remains GAP-003B | active-PR | | executable cutoff-safe analysis-run readiness | ADR 0021; temporal research; API terminal-result contract | stacked `analysis_engine` PR on #157: availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound artifact, realistic end-to-end tests | active-PR | | delayed-reporting cutoff eligibility in truth corpora | ADR 0002; research | `tepp_simulation` eligible-at-cutoff filter on the active PR | active-PR | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | diff --git a/docs/adr/0026-validation-run-scientific-acceptance.md b/docs/adr/0026-validation-run-scientific-acceptance.md new file mode 100644 index 000000000..8e9290a67 --- /dev/null +++ b/docs/adr/0026-validation-run-scientific-acceptance.md @@ -0,0 +1,99 @@ +# ADR 0026 — Durable validation-run scientific acceptance evidence + +**Decision status:** Accepted +**Implementation maturity:** active-PR — library-level `analysis_engine` binding on this PR; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0002, ADR 0008, ADR 0013, ADR 0014, and ADR 0022. +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +GAP-003A records that immutable evidence cannot yet be submitted to a durable +validation run that produces operator-usable scientific acceptance evidence. +`validation_core` already computes RMSE, bias, interval coverage, temporal-order +accuracy, and SE-aware gates, but those metrics remain library-level. An +accepted analysis run is a transport receipt; it must never carry scientific +results. Operators therefore have no hash-stable identity that binds cutoff-safe +evidence, model, seed, backend, and precision to one completion that emits +scientific acceptance evidence. + +Postgres persistence, restart/recovery, and Compose E2E remain GAP-003B. + +## Decision + +Add a bounded `analysis_engine` validation-run executor: + +- `submit_validation_run` binds sorted cutoff-eligible evidence identities, + tenant workspace, snapshot, knowledge cutoff, model `validation_cpu_f64_v1`, + seed, backend `cpu`, precision `f64`, output profile + `scientific_acceptance_v1`, and the SE-gate multiplier `k` into a canonical + SHA-256 digest. `k` must be finite, non-negative, and at most + `MAX_SE_GATE_K` (8). Changing `k` changes the run identity. The durable + `run_id` is `tepp-validation-{32 hex}`. The receipt carries no RMSE, bias, + coverage, or gate-result fields. Receipt fields are private. +- `complete_validation_run` rebinds the same scientific identity, requires + recovery vectors stamped to that `run_id` and binding digest with the same + pre-registered `k`, refuses LLM-authored recovery, computes + `validation_core` recovery metrics, applies the SE-aware gate + `|RMSE − 0| ≤ k · SE(RMSE)` using the submitted `k`, and emits + `tepp.scientific_acceptance.v1` under output profile + `scientific_acceptance_v1`. The artifact records a SHA-256 of the stamped + recovery vectors. Evidence fields are private after completion. +- Empty corpora, duplicate evidence identities, snapshot mismatch, invalid + profiles, non-finite inputs, oversized recovery vectors, empty or + length-mismatched recovery, a post-hoc or oversized `k`, a different run / + tenant / seed / eligible evidence set, a tampered output profile, and + cutoff-empty eligibility fail closed. +- A computed recovery that fails the SE-aware gate still emits evidence with + `se_gate_accepted = false` so operators can read the metrics. Invalid or + LLM-authored recovery never emits evidence. +- The engine does not persist rows, claim implemented-main, or replace ADR 0014 + exact-head claim promotion. + +## Alternatives considered + +1. Keep metrics library-only — rejected because GAP-003A is the operator-visible + product-completion gap and has no live implementation PR. +2. Return scientific metrics on `AnalysisRunAccepted` — rejected because + accepted/running receipts must never carry scientific results. +3. Persist the run in PostgreSQL in this slice — rejected because durable + storage, restart, and Compose recovery belong to GAP-003B / issue #287. +4. Bind evidence, cutoff, model, seed, backend, and precision in + `analysis_engine` and emit scientific acceptance evidence through + `validation_core` — accepted because it is independently testable and does + not weaken fail-closed gates. + +## Consequences + +Operators can submit one immutable evidence snapshot and later complete it with +known-truth recovery to obtain a digest-bound scientific acceptance artifact. +The run identity is hash-stable for the scientific binding, independent of +evidence input order and of ineligible post-cutoff units. Persistence, HTTP +ingress, and release promotion remain later slices. LLM output still cannot +become scientific authority. + +## Verification + +The stacked PR includes unit and integration tests for hash-stable identity, +pre-registered SE-gate `k`, cutoff exclusion, metric-free private receipts, +SE-aware accept and refuse, recovery stamped to a foreign run or tenant, a +post-hoc `k`, a tampered output profile, oversized or empty vectors, and +fail-closed LLM, NaN, empty, duplicate, snapshot, profile, and cutoff-empty +paths. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +``` + +Doctoring and APA 7th citations are in +`docs/research/validation-run-scientific-acceptance.md`. + +## Rollback and supersession + +Rollback removes `validation_run` from `analysis_engine` and stops exporting +the scientific-acceptance artifact. No persisted schema migration is +introduced. Supersession requires a new ADR if execution changes cutoff +semantics, binding identity, LLM-refusal, or scientific estimands. diff --git a/docs/adr/0032-scientific-acceptance-loopback-engine.md b/docs/adr/0032-scientific-acceptance-loopback-engine.md new file mode 100644 index 000000000..fea1e086e --- /dev/null +++ b/docs/adr/0032-scientific-acceptance-loopback-engine.md @@ -0,0 +1,115 @@ +# ADR 0032 — Scientific-acceptance loopback engine execute + +**Decision status:** Accepted +**Implementation maturity:** active-PR +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0026 (engine library), ADR 0027 (GET), and ADR 0028 (lifecycle POST). Does not reuse ADR 0030 or ADR 0031. Does not supersede ADR 0014 claim-promotion authority. + +## Context + +ADR 0026 binds cutoff-safe evidence to a hash-stable validation run that emits +`tepp.scientific_acceptance.v1`. ADR 0027 serves GET status. ADR 0028 records +running and terminal status, but the terminal POST still requires +caller-supplied `scientific_acceptance_json`. Operators therefore cannot obtain +scientific-acceptance evidence from a `scientific_acceptance_v1` loopback run +without already possessing the artifact. Duplicating GET, lifecycle POST, the +engine library, collection GET, cancel HTTP, or the loopback CLI would collide +with live PRs. `analysis_engine` already depends on `tepp_api`; the reverse +dependency would cycle. + +## Decision + +`analysis_engine` owns `POST /v1/analysis-runs/{run_id}/execute` on a wrapper +around `AnalysisRunLiveService`: + +- The execute body carries corpus, recovery vectors, seed, pre-registered + SE-gate `k`, study label, completion time, and an explicit LLM-authorship + flag. It must not carry `scientific_acceptance_json` or receipt metric keys. +- The wrapper calls `submit_validation_run` then `complete_validation_run`, + records running then terminal through the public loopback recorder, and + returns the same status body GET would return. +- Only an accepted run whose request profile is `scientific_acceptance_v1` and + whose model is `validation_cpu_f64_v1` may execute. GET then returns + `tepp.scientific_acceptance.v1` without a caller-supplied artifact. +- Wrong profile, LLM recovery, metric keys, unknown run, consumer mismatch, + already-terminal status, digest mismatch, and unknown execute fields fail + closed. +- `tepp_api` recognizes the `/execute` suffix and refuses it so the raw + listener cannot pretend to execute. Persistence remains GAP-003B. + +## Non-goals + +- Production TLS, public bind, or durable status storage. +- Leiden community detection, Driver p.16 std-family restoration, or Figma/export work. +- Promoting an ADR 0014 scientific claim from HTTP success. +- Collection GET, cancel HTTP, or loopback CLI. + +## Alternatives considered + +1. **Keep caller-supplied terminal artifacts** — rejected because GAP-003A is + the operator-visible product-completion gap and the engine already emits the + schema. +2. **Add `analysis_engine` as a `tepp_api` dependency** — rejected as a crate + cycle. +3. **Persist execute rows in PostgreSQL** — rejected as GAP-003B / live draft + #287. +4. **Engine wrapper that records produced bytes on the existing lifecycle + path** — accepted. + +## Consequences + +- Operators can POST create, POST execute, and GET `tepp.scientific_acceptance.v1` + without supplying the artifact. +- Lifecycle POST remains the generic status-update path (ADR 0028). Execute is + the engine-owned production of those bytes. +- HTTP 200 on execute is not release evidence. + +## Failure and recovery + +Unknown run identities, extra path segments, metric keys, LLM recovery, wrong +profile, already-terminal runs, and consumer mismatch return a redacted `400` +envelope. Unsupported execute contract versions return `422`. Credential +headers remain `403`. The in-memory registry is not durable. + +## Security, privacy, scientific-integrity, and governance impact + +- No credential headers cross the consumer boundary. +- Execute remains loopback-only, size-bounded, and content-redacting. +- SHA-256 digest agreement is a byte-identity check, not a validity claim. +- LLM-authored recovery cannot become scientific authority. + +## Compatibility and migration + +GET status, POST create, POST running/terminal, temporal-context, and +project-history paths are unchanged. Production adapters may replace loopback +while preserving metric-free receipts and engine-produced scientific +acceptance. + +## Verification + +Falsifiable evidence: + +- POST create stays metric-free; +- POST execute without `scientific_acceptance_json` then GET returns + `tepp.scientific_acceptance.v1`; +- wrong profile, LLM recovery, metric keys, unknown run, consumer mismatch, + and a second execute on a terminal run fail closed; +- Clippy `-D warnings`, `analysis_engine` and `tepp_api` tests, rustdoc, and + exact-head review remain required. + +## Rollback and supersession + +Rollback removes the execute wrapper; GET, lifecycle POST, and the engine +library remain valid. A superseding ADR is required to persist status, bind a +public address, or treat HTTP success as an ADR 0014 claim. + +## Related authority + +- ADR 0026 owns the validation-run library bind. +- ADR 0027 owns the GET status read. +- ADR 0028 owns POST running/terminal. +- ADR 0018 owns consumer-scoped ingress and metric-free `202 Accepted`. +- ADR 0022 owns deterministic execution to a digest-bound terminal result. +- ADR 0014 owns scientific claim promotion. +- ADR 0008 owns SHA-256 identity. +- ADR 0011 owns standalone/modular HTTP boundaries. diff --git a/docs/adr/README.md b/docs/adr/README.md index 421caeadd..4bff0761f 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,8 +30,10 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0026](0026-validation-run-scientific-acceptance.md) | Durable validation-run scientific acceptance evidence | Accepted | active-PR | GAP-003A first slice binds cutoff-safe evidence to a hash-stable run that emits `tepp.scientific_acceptance.v1`; Postgres persistence remains GAP-003B. | | [0027](0027-scientific-acceptance-http-status.md) | Scientific-acceptance loopback HTTP status path | Accepted | active-PR | GET `/v1/analysis-runs/{run_id}` stays metric-free on accepted/running; `tepp.scientific_acceptance.v1` only on succeeded `scientific_acceptance_v1`. | | [0028](0028-scientific-acceptance-http-lifecycle.md) | Scientific-acceptance loopback HTTP lifecycle POST | Accepted | active-PR | POST `/running` and `/terminal` are the production status-update path; GET remains ADR 0027. Persistence remains GAP-003B. | +| [0032](0032-scientific-acceptance-loopback-engine.md) | Scientific-acceptance loopback engine execute | Accepted | active-PR | POST `/execute` runs `analysis_engine` on the loopback lifecycle so GET returns `tepp.scientific_acceptance.v1` without a caller-supplied artifact. Persistence remains GAP-003B. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | @@ -140,6 +142,7 @@ Use the narrowest owning ADR when decisions overlap: - **project-history wire-size symmetry:** ADR 0019. - **LineageWeave project-history service boundary:** ADR 0021. - **accepted-run execution and terminal artifact production:** ADR 0022. +- **scientific-acceptance loopback engine execute:** ADR 0032. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. diff --git a/docs/connectors/naruon-artifact-consumer.md b/docs/connectors/naruon-artifact-consumer.md index 9c6f6d185..b28eec968 100644 --- a/docs/connectors/naruon-artifact-consumer.md +++ b/docs/connectors/naruon-artifact-consumer.md @@ -28,6 +28,7 @@ TEPP remains the scientific authority for estimation, recovery metrics, temporal | HTTP analysis-run create | `tepp_api` `naruon_analysis_run_exchange` → `POST /v1/analysis-runs` | naruon → TEPP | | HTTP export authorize | `tepp_api` `naruon_export_exchange` → `POST /v1/exports` | naruon → TEPP | | Live loopback POST | `tepp_api` `NaruonLiveService` → `POST /v1/analysis-runs` and `/v1/exports` | naruon → TEPP | +| Live loopback scientific-acceptance execute | `analysis_engine` `ScientificAcceptanceLoopbackService` → `POST /v1/analysis-runs/{run_id}/execute` | naruon → TEPP | Committed examples live under `examples/`. Schemas for analysis-run requests and corpus-split manifests live under `schemas/`. diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 5ea0c95bf..c3e5a0780 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -248,7 +248,7 @@ the coordinated visual workspace, or a supported multi-tenant release. |---|---|---|---|---|---|---|---| | GAP-001 | Submission produces a durable accepted receipt, and the deterministic terminal-result lifecycle is now implemented-main. | `implemented-main` | closed on protected main | `340087494b0a` lineage merged through [PR #157](https://github.com/ContextualWisdomLab/TEPP/pull/157) (merged 2026-08-25T02:53Z); [#156](https://github.com/ContextualWisdomLab/TEPP/issues/156) CLOSED | — | — | Exact request/result/snapshot/cutoff/model/profile binding, typed terminal failures, deterministic retrieval, and cutoff-safe execution are protected-main behavior as of the #157 merge. | | GAP-002 | LineageWeave and other modular consumers can rely on the complete protected-main HTTP evidence/result boundary. | `partial` | consumer hardening remains | Terminal-result lifecycle implemented-main via #157; versioned API contract intact | [#156](https://github.com/ContextualWisdomLab/TEPP/issues/156) (closed) / [PR #155](https://github.com/ContextualWisdomLab/TEPP/pull/155) (merged) | — | Remaining work is consumer-side adoption evidence and any versioned-contract drift discovered during integration; core boundary is no longer the gap it was. | -| GAP-003A | Immutable evidence cannot yet be submitted to a durable validation run that produces operator-usable scientific acceptance evidence. | `accepted-target` | product-completion | `e65cd66` (validation metrics are library-level only) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | `—` (issue program; no current implementation PR) | Compose/CLI/API execution must bind immutable evidence, cutoffs, model configuration, validation metrics, and reproducibility manifests to one idempotent run. | +| GAP-003A | Immutable evidence cannot yet be submitted to a durable validation run that produces operator-usable scientific acceptance evidence. | `accepted-target` | product-completion | `e65cd66` (validation metrics are library-level only) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | [#356](https://github.com/ContextualWisdomLab/TEPP/pull/356) `7916603` (`feat/validation-run-scientific-acceptance`; pre-registered SE-gate `k`; not implemented-main; not Compose persistence) | Compose/CLI/API execution must bind immutable evidence, cutoffs, model configuration, validation metrics, and reproducibility manifests to one idempotent run. | | GAP-003B | Scientific result artifacts cannot yet be persisted, restarted, and recovered as one supported operator workflow. | `accepted-target` | product-completion | `e65cd66` (persistence contracts lack E2E recovery) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | `—` (issue program; no current implementation PR) | Durable storage, migration/rollback, restart/recovery, artifact digest verification, and terminal retrieval must pass against a real Compose deployment. | | GAP-003C | The persistence slice classifies concurrent-write SQLSTATEs, but has no measured hot-partition detection, routing, or mitigation for tenant/result workloads. | `accepted-target` | product-completion | `e65cd66` (conflict classification only; no measured partition control) | [#166](https://github.com/ContextualWisdomLab/TEPP/issues/166) | `—` (issue program; no current implementation PR) | A real Compose/PostgreSQL workload identifies hot keys and partition skew, applies bounded tenant/time or result routing without weakening 3NF or temporal authority, and proves conflict rate, latency, recovery, and migration/rollback behavior under load. | | GAP-004 | The central shared-latent temporal/relational topic estimator is absent. | `partial` | product vertical | CPU `f64` TRSL-TM reference estimator with ALR/ILR coordinates and refusal gates is implemented-main (v0.2.0 `topic_measurement`); fitted candidate-`K` scoring present | [#167](https://github.com/ContextualWisdomLab/TEPP/issues/167) | — | GPU, method effects, full Bayesian sampling, and topic birth/split/merge remain. This is not full #167 closure. | diff --git a/docs/research/scientific-acceptance-loopback-engine.md b/docs/research/scientific-acceptance-loopback-engine.md new file mode 100644 index 000000000..eadf07e90 --- /dev/null +++ b/docs/research/scientific-acceptance-loopback-engine.md @@ -0,0 +1,31 @@ +# Scientific-acceptance loopback engine execute + +## Scope + +This note doctors the GAP-003A engine-on-loopback slice: + +1. `POST /v1/analysis-runs/{run_id}/execute` runs `submit_validation_run` and `complete_validation_run` against an accepted loopback run; +2. the execute body carries corpus, recovery, seed, and pre-registered SE-gate `k` and must not carry `scientific_acceptance_json`; +3. the engine records running then terminal so GET returns `tepp.scientific_acceptance.v1` without a caller-supplied artifact; +4. wrong profile, LLM-authored recovery, receipt metric keys, unknown run, consumer mismatch, already-terminal status, and digest mismatch fail closed. + +Postgres persistence, restart/recovery, and Compose execution remain GAP-003B. This slice is not implemented-main. It does not duplicate the engine library (#356), terminal-result DTO (#358), GET (#359), lifecycle POST (#360), cancel HTTP (#361), loopback CLI (#362), or collection GET (#368). + +## Authoritative sources + +National Academies of Sciences, Engineering, and Medicine. (2019). *Reproducibility and replicability in science*. The National Academies Press. https://doi.org/10.17226/25303 + +Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on *p*-values: Context, process, and purpose. *The American Statistician, 70*(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108 + +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. *Journal of the American Statistical Association, 22*(158), 209–212. https://doi.org/10.1080/01621459.1927.10502953 + +## Application + +The National Academies (2019) separate computational reproducibility (same binding, same digest) from a scientific claim that recovery is correct. Wasserstein and Lazar (2016) refuse to treat a passing threshold as automatic scientific authority, so execute produces the same `tepp.scientific_acceptance.v1` evidence as the library bind, including `se_gate_accepted = false` when the pre-registered gate fails, and never treats HTTP `200` as ADR 0014 promotion. Wilson (1927) supplies the coverage interval already implemented in `validation_core`. TEPP therefore executes the engine on the loopback lifecycle path, refuses caller-supplied artifacts and LLM-authored recovery, and reports RMSE, bias, coverage, temporal order, and the SE-aware gate only after engine completion (National Academies of Sciences, Engineering, and Medicine, 2019; Wasserstein & Lazar, 2016; Wilson, 1927). Meredith (1993) remains unread (Unpaywall/OpenAlex 2026-08-31T10:00Z: `is_oa: false`, 0 locations). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). + +## Verification + +- POST create then POST execute without `scientific_acceptance_json` then GET returns `tepp.scientific_acceptance.v1`; +- a `calibrated_event_measurement` run stays accepted and metric-free after execute is refused; +- LLM authorship, receipt metric keys, unknown-field `scientific_acceptance_json`, unknown run, consumer mismatch, and a second execute on a terminal run fail closed; +- the raw `AnalysisRunLiveService` still returns `400` for `/execute`. diff --git a/docs/research/validation-run-scientific-acceptance.md b/docs/research/validation-run-scientific-acceptance.md new file mode 100644 index 000000000..2ce820517 --- /dev/null +++ b/docs/research/validation-run-scientific-acceptance.md @@ -0,0 +1,38 @@ +# Validation-run scientific acceptance evidence + +## Scope + +This note doctors the first GAP-003A slice in `analysis_engine`: + +1. immutable evidence, tenant workspace, knowledge cutoff, model, seed, backend, precision, output profile, and the SE-gate multiplier `k` bind to one hash-stable validation run; +2. the accepted receipt carries no scientific metrics and cannot be rewritten after submit; +3. completion emits `tepp.scientific_acceptance.v1` with RMSE, bias, interval coverage (Wilson bounds), temporal-order accuracy, an SE-aware gate using that pre-registered `k`, and a SHA-256 of the stamped recovery vectors; +4. recovery vectors must be constructed against that receipt with the same `k`; a different run, tenant, seed, snapshot, profile, eligible evidence set, or post-hoc `k` fails closed; +5. LLM-authored recovery, non-finite inputs, empty or duplicate evidence, snapshot mismatch, oversized recovery, oversized `k` (`MAX_SE_GATE_K` = 8), and cutoff-empty corpora fail closed. + +Postgres persistence, restart/recovery, and Compose execution remain GAP-003B. This slice is not implemented-main. + +## Authoritative sources + +National Academies of Sciences, Engineering, and Medicine. (2019). *Reproducibility and replicability in science*. The National Academies Press. https://doi.org/10.17226/25303 + +Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on *p*-values: Context, process, and purpose. *The American Statistician, 70*(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108 + +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. *Journal of the American Statistical Association, 22*(158), 209–212. https://doi.org/10.1080/01621459.1927.10502953 + +## Application + +The National Academies (2019) separate computational reproducibility (same binding, same digest) from a scientific claim that recovery is correct. Wasserstein and Lazar (2016) refuse to treat a passing threshold as automatic scientific authority and refuse post-hoc threshold shopping, so `k` is pre-registered in the run binding and a failed SE-aware gate still reports metrics with `se_gate_accepted = false` rather than inventing a passing claim. Wilson (1927) supplies the coverage interval bounds already implemented in `validation_core`. TEPP therefore binds cutoff-safe evidence, tenant/profile identity, and `k` before any metric is computed, stamps recovery vectors to that binding, refuses LLM-authored or post-hoc-`k` recovery, and reports RMSE, bias, coverage, temporal order, and the SE-aware gate as operator-usable evidence (National Academies of Sciences, Engineering, and Medicine, 2019; Wasserstein & Lazar, 2016; Wilson, 1927). Meredith (1993) remains unread (Unpaywall/OpenAlex 2026-08-31T07:45Z: `is_oa: false`, 0 locations). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). + +## Verification + +- identical eligible evidence in any order and with extra post-cutoff units yields the same `tepp-validation-{32 hex}` identity; +- distinct SE-gate multipliers yield distinct run identities; `k > 8` fails closed; +- receipts serialize without RMSE, bias, or coverage fields; +- known-truth recovery emits `tepp.scientific_acceptance.v1` with a digest-stable JSON body; +- a large residual vector remains operator-readable with `se_gate_accepted = false`; +- recovery stamped to a different run, tenant, seed, or post-hoc `k` returns `BindingMismatch`; +- a tampered receipt `output_profile` returns `BindingMismatch`; +- an oversized recovery vector returns `LimitExceeded`; +- empty or length-mismatched recovery vectors return `InvalidEvidence`; +- LLM authorship, NaN recovery, empty corpora, duplicates, snapshot mismatch, wrong profile/model, and cutoff-empty eligibility return dedicated fail-closed errors.