diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..e9915aac3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- `analysis_engine` binds Enders and Tofighi (2007) CWC within/between/contextual OLS (`recover_cluster_mean_within_between_slopes`) to the `longitudinal_cwc_v1` analysis-run output profile. Rows unavailable at the request cutoff are excluded; the digest-bound `tepp.longitudinal_cwc.v1` artifact records the three slopes and refuses causal promotion. This is not a new ESEM/DSEM estimator, not a Driver p.16 `std` restore, and not persistence. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..28a0f0cb8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -74,6 +74,7 @@ dependencies = [ "corpus_split", "event_core", "membership_core", + "psychometric_core", "relation_graph", "serde", "serde_json", diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..d1f55851c 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -15,6 +15,7 @@ publish = false [dependencies] event_core = { path = "../event_core", version = "0.2.0" } +psychometric_core = { path = "../psychometric_core", version = "0.2.0" } serde = { workspace = true } serde_json = { workspace = true } sha2 = { workspace = true } @@ -26,6 +27,7 @@ uuid.workspace = true [dev-dependencies] corpus_split = { path = "../corpus_split", version = "0.2.0" } membership_core = { path = "../membership_core", version = "0.2.0" } +psychometric_core = { path = "../psychometric_core", version = "0.2.0" } relation_graph = { path = "../relation_graph", version = "0.2.0" } [lints] diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..a0930f704 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -8,13 +8,16 @@ //! through [`tepp_api`]. It deliberately does not claim latent-variable or topic //! estimation authority; those estimators remain separate scientific crates. //! estimation authority; it invokes estimators through their scientific crate -//! contracts and preserves their artifact meaning. +//! contracts and preserves their artifact meaning. Longitudinal CWC composition +//! is invoked through [`psychometric_core`] and is not a causal estimand. mod case_deletion_refit; mod lineage_criterion; +mod longitudinal_cwc_artifact; mod topic_context_posterior; mod topic_lineage_artifact; +use psychometric_core::PsychometricError; use serde::Serialize; use sha2::{Digest, Sha256}; use std::collections::BTreeSet; @@ -46,6 +49,13 @@ pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, fit_lineage_criterion_posteriors, }; +/// Longitudinal CWC composition artifact and execution contracts. +pub use longitudinal_cwc_artifact::{ + LONGITUDINAL_CWC_ARTIFACT_BYTE_LIMIT, LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION, + LONGITUDINAL_CWC_MODEL_CONTRACT_VERSION, LONGITUDINAL_CWC_OUTPUT_PROFILE, + LongitudinalClusterScore, LongitudinalCwcArtifact, LongitudinalCwcExecution, + execute_longitudinal_cwc_run, +}; /// Bounded posterior topic-context producer contract and record types. pub use topic_context_posterior::{ TOPIC_CONTEXT_POSTERIOR_BYTE_LIMIT, TOPIC_CONTEXT_POSTERIOR_SCHEMA_VERSION, @@ -248,6 +258,10 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A psychometric recovery rejected the offered coordinates. + Psychometric(PsychometricError), + /// A longitudinal CWC artifact violated its bounded schema or count invariants. + InvalidLongitudinalCwcArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +276,8 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::Psychometric(error) => return error.fmt(formatter), + Self::InvalidLongitudinalCwcArtifact => "invalid longitudinal CWC artifact", }; formatter.write_str(message) } @@ -281,6 +297,12 @@ impl From for AnalysisEngineError { } } +impl From for AnalysisEngineError { + fn from(error: PsychometricError) -> Self { + Self::Psychometric(error) + } +} + /// Execute the cutoff-safe temporal evidence readiness analysis. /// /// Evidence whose `available_time` is later than the request cutoff is excluded @@ -413,7 +435,8 @@ mod tests { use super::{ ANALYSIS_ARTIFACT_SCHEMA_VERSION, ANALYSIS_STATISTIC_COUNT, AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, MAX_ANALYSIS_IDENTIFIER_BYTES, - MAX_EVIDENCE_UNITS, TopicMeasurementError, add_membership_count, execute_analysis_run, + MAX_EVIDENCE_UNITS, PsychometricError, TopicMeasurementError, add_membership_count, + execute_analysis_run, }; use temporal_core::{AvailableTime, EventTime}; use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState, ApiError}; @@ -681,6 +704,14 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::Psychometric(PsychometricError::CausalUnderidentified), + "temporal precedence is not causal identification", + ), + ( + AnalysisEngineError::InvalidLongitudinalCwcArtifact, + "invalid longitudinal CWC artifact", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); @@ -689,6 +720,11 @@ mod tests { assert_eq!(converted.to_string(), "invalid API wire payload"); let from_topic: AnalysisEngineError = TopicMeasurementError::DidNotConverge.into(); assert_eq!(from_topic.to_string(), "topic estimator did not converge"); + let from_psych: AnalysisEngineError = PsychometricError::CausalUnderidentified.into(); + assert_eq!( + from_psych.to_string(), + "temporal precedence is not causal identification" + ); assert_eq!( add_membership_count(u64::MAX, 1), Err(AnalysisEngineError::ArithmeticOverflow) diff --git a/crates/analysis_engine/src/longitudinal_cwc_artifact.rs b/crates/analysis_engine/src/longitudinal_cwc_artifact.rs new file mode 100644 index 000000000..97927ac32 --- /dev/null +++ b/crates/analysis_engine/src/longitudinal_cwc_artifact.rs @@ -0,0 +1,417 @@ +//! Digest-bound CWC within/between composition as an analysis-run profile. + +use psychometric_core::{ + CausalHeuristic, ClusteredScore, PsychometricError, claim_causal_effect, + recover_cluster_mean_within_between_slopes, +}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; + +use crate::{ + AnalysisEngineError, MAX_EVIDENCE_UNITS, format_digest, require_receipt_identity, + valid_identifier, +}; + +/// Versioned schema for a completed longitudinal CWC artifact. +pub const LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION: &str = "tepp.longitudinal_cwc.v1"; +/// Model contract required by the CWC composition execution path. +pub const LONGITUDINAL_CWC_MODEL_CONTRACT_VERSION: &str = "longitudinal_cwc_v1"; +/// Analysis-run output profile required for a longitudinal CWC artifact. +pub const LONGITUDINAL_CWC_OUTPUT_PROFILE: &str = "longitudinal_cwc_v1"; +/// Maximum canonical artifact JSON size. +pub const LONGITUDINAL_CWC_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const LONGITUDINAL_CWC_INFERENCE_STATUS: &str = "composed_cwc_slopes_not_causal"; + +/// One already-mapped clustered score offered to a cutoff-safe CWC run. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct LongitudinalClusterScore { + cluster_key: u64, + predictor: f64, + outcome: f64, + available_time: AvailableTime, +} + +impl LongitudinalClusterScore { + /// Bind one clustered predictor–outcome pair to an availability clock. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidEvidence`] when either coordinate + /// is non-finite. + pub fn new( + cluster_key: u64, + predictor: f64, + outcome: f64, + available_time: AvailableTime, + ) -> Result { + if !predictor.is_finite() || !outcome.is_finite() { + return Err(AnalysisEngineError::InvalidEvidence); + } + Ok(Self { + cluster_key, + predictor, + outcome, + available_time, + }) + } + + /// Return the cluster identity. + #[must_use] + pub const fn cluster_key(self) -> u64 { + self.cluster_key + } + + /// Return the already-mapped predictor. + #[must_use] + pub const fn predictor(self) -> f64 { + self.predictor + } + + /// Return the already-mapped outcome. + #[must_use] + pub const fn outcome(self) -> f64 { + self.outcome + } + + /// Return the availability clock used for cutoff eligibility. + #[must_use] + pub const fn available_time(self) -> AvailableTime { + self.available_time + } +} + +/// Completed, bounded CWC composition consumed by analysis-run clients. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct LongitudinalCwcArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used by the composition. + pub knowledge_cutoff: String, + /// Eligible clustered rows after cutoff. + pub row_count: u64, + /// Distinct clusters among eligible rows. + pub cluster_count: u64, + /// Rows excluded because availability was after the cutoff. + pub excluded_after_cutoff_count: u64, + /// Within-cluster OLS slope after CWC. + pub within_slope: f64, + /// Between-cluster OLS slope of cluster means. + pub between_slope: f64, + /// CWC contextual effect `between − within`. + pub contextual_effect: f64, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl LongitudinalCwcArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidLongitudinalCwcArtifact`] when the + /// schema, identifiers, counts, slopes, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > LONGITUDINAL_CWC_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidLongitudinalCwcArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn to_json(&self) -> Result { + self.validate()?; + let payload = + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; + Ok(payload) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + if self.schema_version != LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || self.row_count < 2 + || self.cluster_count < 2 + || self.cluster_count > self.row_count + || !self.within_slope.is_finite() + || !self.between_slope.is_finite() + || !self.contextual_effect.is_finite() + || self.inference_status != LONGITUDINAL_CWC_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidLongitudinalCwcArtifact); + } + Ok(()) + } +} + +/// One completed CWC artifact and its request-bound terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct LongitudinalCwcExecution { + /// Digest-bound completed composition artifact. + pub artifact: LongitudinalCwcArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +struct EligibleCwcRows { + scores: Vec, + excluded_after_cutoff_count: u64, +} + +fn admit_scores_at_cutoff( + scores: &[LongitudinalClusterScore], + knowledge_cutoff: KnowledgeCutoff, +) -> Result { + if scores.len() > MAX_EVIDENCE_UNITS { + return Err(AnalysisEngineError::LimitExceeded); + } + let mut eligible = Vec::new(); + let mut excluded_after_cutoff_count = 0_u64; + for score in scores { + if score.available_time.instant() <= knowledge_cutoff.instant() { + eligible.push(ClusteredScore { + cluster_key: score.cluster_key, + predictor: score.predictor, + outcome: score.outcome, + }); + } else { + excluded_after_cutoff_count += 1; + } + } + if eligible.is_empty() { + return Err(AnalysisEngineError::Psychometric( + PsychometricError::InvalidNumericInput, + )); + } + Ok(EligibleCwcRows { + scores: eligible, + excluded_after_cutoff_count, + }) +} + +/// Execute cutoff-safe CWC within/between composition as one analysis-run profile. +/// +/// The caller supplies already-mapped clustered coordinates. This executor does +/// not invent an ESEM/DSEM estimator, persist rows, or treat the recovered +/// slopes as a causal effect. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, psychometric +/// recovery failure, or invalid artifact error. +#[expect( + clippy::missing_panics_doc, + reason = "bounded summary constants cannot fail" +)] +pub fn execute_longitudinal_cwc_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + scores: &[LongitudinalClusterScore], + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + || request.model_contract_version != LONGITUDINAL_CWC_MODEL_CONTRACT_VERSION + || request.output_profile != LONGITUDINAL_CWC_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let eligible = admit_scores_at_cutoff(scores, knowledge_cutoff)?; + let slopes = recover_cluster_mean_within_between_slopes(&eligible.scores)?; + let _ = claim_causal_effect(CausalHeuristic::TemporalPrecedence); + + let mut clusters = std::collections::BTreeSet::new(); + for score in &eligible.scores { + clusters.insert(score.cluster_key); + } + let row_count = u64::try_from(eligible.scores.len()) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let cluster_count = + u64::try_from(clusters.len()).map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let artifact = LongitudinalCwcArtifact { + schema_version: LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + row_count, + cluster_count, + excluded_after_cutoff_count: eligible.excluded_after_cutoff_count, + within_slope: slopes.within_slope, + between_slope: slopes.between_slope, + contextual_effect: slopes.contextual_effect, + inference_status: LONGITUDINAL_CWC_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "longitudinal_cwc", + row_count, + 3, + LONGITUDINAL_CWC_INFERENCE_STATUS, + ) + .expect("bounded longitudinal CWC summary constants are valid"); + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("longitudinal_cwc_artifact_{}", &digest[..16]), + digest, + LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(LongitudinalCwcExecution { + artifact, + terminal_result, + }) +} + +#[cfg(test)] +mod tests { + use super::{ + LONGITUDINAL_CWC_ARTIFACT_BYTE_LIMIT, LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION, + LONGITUDINAL_CWC_INFERENCE_STATUS, LongitudinalCwcArtifact, + }; + use crate::AnalysisEngineError; + + fn artifact() -> LongitudinalCwcArtifact { + LongitudinalCwcArtifact { + schema_version: LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + row_count: 4, + cluster_count: 2, + excluded_after_cutoff_count: 0, + within_slope: 0.5, + between_slope: 2.0, + contextual_effect: 1.5, + inference_status: LONGITUDINAL_CWC_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &LongitudinalCwcArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidLongitudinalCwcArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + LongitudinalCwcArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + LongitudinalCwcArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidLongitudinalCwcArtifact) + ); + assert_eq!( + LongitudinalCwcArtifact::from_json( + &"x".repeat(LONGITUDINAL_CWC_ARTIFACT_BYTE_LIMIT + 1) + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.row_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.cluster_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.cluster_count = 5; + value + }, + { + let mut value = artifact.clone(); + value.within_slope = f64::NAN; + value + }, + { + let mut value = artifact.clone(); + value.between_slope = f64::INFINITY; + value + }, + { + let mut value = artifact.clone(); + value.contextual_effect = f64::NEG_INFINITY; + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } +} diff --git a/crates/analysis_engine/tests/longitudinal_cwc_execution_contract.rs b/crates/analysis_engine/tests/longitudinal_cwc_execution_contract.rs new file mode 100644 index 000000000..96f7eb75d --- /dev/null +++ b/crates/analysis_engine/tests/longitudinal_cwc_execution_contract.rs @@ -0,0 +1,266 @@ +//! End-to-end contract for cutoff-safe longitudinal CWC composition. + +use analysis_engine::{ + AnalysisEngineError, LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION, + LONGITUDINAL_CWC_MODEL_CONTRACT_VERSION, LONGITUDINAL_CWC_OUTPUT_PROFILE, + LongitudinalClusterScore, MAX_EVIDENCE_UNITS, execute_longitudinal_cwc_run, +}; +use psychometric_core::PsychometricError; +use temporal_core::{AvailableTime, KnowledgeCutoff}; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; + +fn available(stamp: &str) -> AvailableTime { + AvailableTime::parse_rfc3339(stamp).expect("available") +} + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") +} + +fn noiseless_rows() -> Vec { + vec![ + LongitudinalClusterScore::new(1, 0.0, 2.0, available("2026-07-01T00:00:00Z")).expect("r1"), + LongitudinalClusterScore::new(1, 2.0, 3.0, available("2026-07-01T00:00:00Z")).expect("r2"), + LongitudinalClusterScore::new(2, 4.0, 10.0, available("2026-07-01T00:00:00Z")).expect("r3"), + LongitudinalClusterScore::new(2, 6.0, 11.0, available("2026-07-01T00:00:00Z")).expect("r4"), + ] +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "longitudinal-cwc-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-longitudinal-cwc".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: LONGITUDINAL_CWC_MODEL_CONTRACT_VERSION.into(), + output_profile: LONGITUDINAL_CWC_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new("run-longitudinal-cwc", "accepted", &request.idempotency_key) + .expect("accepted") +} + +#[test] +fn noiseless_cwc_emits_digest_bound_within_between_and_contextual() { + let request = request(); + let accepted = accepted(&request); + let rows = noiseless_rows(); + let execution = execute_longitudinal_cwc_run( + &request, + &accepted, + "snapshot-longitudinal-cwc", + cutoff(), + &rows, + "2026-08-02T00:00:00Z", + ) + .expect("execution"); + + assert_eq!( + execution.artifact.schema_version, + LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!(execution.artifact.row_count, 4); + assert_eq!(execution.artifact.cluster_count, 2); + assert_eq!(execution.artifact.excluded_after_cutoff_count, 0); + assert!((execution.artifact.within_slope - 0.5).abs() < 1e-12); + assert!((execution.artifact.between_slope - 2.0).abs() < 1e-12); + assert!((execution.artifact.contextual_effect - 1.5).abs() < 1e-12); + assert_eq!( + execution.artifact.inference_status, + "composed_cwc_slopes_not_causal" + ); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(LONGITUDINAL_CWC_ARTIFACT_SCHEMA_VERSION) + ); + assert_eq!(rows[0].cluster_key(), 1); + assert!((rows[0].predictor() - 0.0).abs() < f64::EPSILON); + assert!((rows[0].outcome() - 2.0).abs() < f64::EPSILON); + assert_eq!(rows[0].available_time(), available("2026-07-01T00:00:00Z")); +} + +#[test] +fn execution_excludes_rows_unavailable_at_the_request_cutoff() { + let request = request(); + let accepted = accepted(&request); + let mut rows = noiseless_rows(); + rows.push( + LongitudinalClusterScore::new(3, 8.0, 20.0, available("2026-08-15T00:00:00Z")) + .expect("late"), + ); + let execution = execute_longitudinal_cwc_run( + &request, + &accepted, + "snapshot-longitudinal-cwc", + cutoff(), + &rows, + "2026-08-02T00:00:00Z", + ) + .expect("execution"); + assert_eq!(execution.artifact.row_count, 4); + assert_eq!(execution.artifact.cluster_count, 2); + assert_eq!(execution.artifact.excluded_after_cutoff_count, 1); + assert!((execution.artifact.within_slope - 0.5).abs() < 1e-12); + assert!((execution.artifact.between_slope - 2.0).abs() < 1e-12); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + let accepted = accepted(&request); + let rows = noiseless_rows(); + assert_eq!( + execute_longitudinal_cwc_run( + &request, + &accepted, + "other-snapshot", + cutoff(), + &rows, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + for invalid_request in [ + { + let mut value = request.clone(); + value.knowledge_cutoff = "2026-08-02T00:00:00Z".into(); + value + }, + { + let mut value = request.clone(); + value.model_contract_version = "other-model".into(); + value + }, + { + let mut value = request.clone(); + value.output_profile = "other-profile".into(); + value + }, + ] { + assert_eq!( + execute_longitudinal_cwc_run( + &invalid_request, + &accepted, + "snapshot-longitudinal-cwc", + cutoff(), + &rows, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} + +#[test] +fn execution_refuses_empty_cutoff_one_cluster_and_receipt_mismatch() { + let request = request(); + let accepted = accepted(&request); + assert_eq!( + LongitudinalClusterScore::new(1, f64::NAN, 1.0, available("2026-07-01T00:00:00Z")), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + LongitudinalClusterScore::new(1, 1.0, f64::NAN, available("2026-07-01T00:00:00Z")), + Err(AnalysisEngineError::InvalidEvidence) + ); + + let mut early_request = request.clone(); + early_request.knowledge_cutoff = "2026-06-01T00:00:00Z".into(); + let too_early = KnowledgeCutoff::parse_rfc3339("2026-06-01T00:00:00Z").expect("cutoff"); + assert_eq!( + execute_longitudinal_cwc_run( + &early_request, + &accepted, + "snapshot-longitudinal-cwc", + too_early, + &noiseless_rows(), + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::Psychometric( + PsychometricError::InvalidNumericInput + )) + ); + + let late_cluster_two = vec![ + LongitudinalClusterScore::new(1, 0.0, 2.0, available("2026-07-01T00:00:00Z")).expect("r1"), + LongitudinalClusterScore::new(1, 2.0, 3.0, available("2026-07-01T00:00:00Z")).expect("r2"), + LongitudinalClusterScore::new(2, 4.0, 10.0, available("2026-08-15T00:00:00Z")).expect("r3"), + LongitudinalClusterScore::new(2, 6.0, 11.0, available("2026-08-15T00:00:00Z")).expect("r4"), + ]; + assert_eq!( + execute_longitudinal_cwc_run( + &request, + &accepted, + "snapshot-longitudinal-cwc", + cutoff(), + &late_cluster_two, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::Psychometric( + PsychometricError::InsufficientClusters + )) + ); + + let wrong_receipt = AnalysisRunAccepted::new("run-longitudinal-cwc", "accepted", "other-key") + .expect("accepted"); + assert_eq!( + execute_longitudinal_cwc_run( + &request, + &wrong_receipt, + "snapshot-longitudinal-cwc", + cutoff(), + &noiseless_rows(), + "2026-08-02T00:00:00Z", + ) + .expect_err("receipt"), + AnalysisEngineError::Api(tepp_api::ApiError::InvalidWirePayload) + ); + + let oversized = + vec![ + LongitudinalClusterScore::new(1, 0.0, 1.0, available("2026-07-01T00:00:00Z")) + .expect("row"); + MAX_EVIDENCE_UNITS + 1 + ]; + assert_eq!( + execute_longitudinal_cwc_run( + &request, + &accepted, + "snapshot-longitudinal-cwc", + cutoff(), + &oversized, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::LimitExceeded) + ); +} + +#[test] +fn execution_refuses_invalid_completion_time() { + let request = request(); + let accepted = accepted(&request); + assert_eq!( + execute_longitudinal_cwc_run( + &request, + &accepted, + "snapshot-longitudinal-cwc", + cutoff(), + &noiseless_rows(), + "invalid", + ), + Err(AnalysisEngineError::Api( + tepp_api::ApiError::InvalidWirePayload + )) + ); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..249dad53b 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| cutoff-safe longitudinal CWC composition | ADR 0005/0033; Enders & Tofighi (2007) | `analysis_engine` `longitudinal_cwc_v1` binds `psychometric_core` CWC within/between/contextual slopes to a digest-bound `tepp.longitudinal_cwc.v1` artifact; causal promotion refused; not ESEM/DSEM estimation | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0033-longitudinal-cwc-analysis-run.md b/docs/adr/0033-longitudinal-cwc-analysis-run.md new file mode 100644 index 000000000..f515e58aa --- /dev/null +++ b/docs/adr/0033-longitudinal-cwc-analysis-run.md @@ -0,0 +1,73 @@ +# ADR 0033 — Longitudinal CWC composition as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0005 (ESEM/DSEM interpretation) and ADR 0022 (cutoff-safe analysis-run execution). +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already recovers Enders and Tofighi (2007) cluster-mean-centered +within/between OLS and the CWC contextual effect inside `psychometric_core`. +Operators still cannot request that composition as a digest-bound analysis-run +output. Recovery primitives alone are not the ESEM/DSEM engine (GAP-006 / #169). +A second Driver p.16 `std`-family restore would not close this operator-visible +gap. + +## Decision + +Add the `longitudinal_cwc_v1` analysis-run output profile to `analysis_engine`. +The executor: + +- consumes already-mapped clustered predictor/outcome coordinates plus + `available_time`; +- excludes rows whose availability is later than the request `knowledge_cutoff`; +- invokes `recover_cluster_mean_within_between_slopes` without reimplementing + CWC; +- invokes `claim_causal_effect` so temporal precedence cannot promote the + slopes to a causal estimand; +- emits a canonical SHA-256-digested `tepp.longitudinal_cwc.v1` artifact with + row/cluster counts, excluded-after-cutoff count, within/between/contextual + slopes, and inference status `composed_cwc_slopes_not_causal`; +- does not invent an ESEM/DSEM sampler, persist rows, or claim strong + invariance or Rubin pooling. + +This is two-level OLS composition, not DSEM, not RI-CLPM, and not a +random-effects sampler. + +## Alternatives considered + +1. Restore another Driver p.16 standardised matrix — rejected because those + recoveries are already a live micro-PR family and do not bind composition to + an analysis run. +2. Put CWC execution into `tepp_api` — rejected because transport contracts and + scientific composition would become one service boundary. +3. Bind the existing `psychometric_core` CWC recovery to ADR 0022's analysis-run + profile — accepted. + +## Consequences + +Operators can request cutoff-safe within/between/contextual slopes as a +digest-bound terminal result. The artifact is not a causal effect, not an ESEM +fit, and not implemented-main until exact-head Checks and two independent +approvals land. + +## Verification + +```text +cargo fmt -p analysis_engine -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +``` + +Known-truth noiseless CWC recovers within `0.5`, between `2.0`, contextual +`1.5`. Cutoff exclusion, snapshot/profile mismatch, empty eligibility, and +single-cluster remainder fail closed. + +## Rollback and supersession + +Rollback removes the `longitudinal_cwc_v1` profile. No persisted schema +migration is introduced. Supersede only with an ADR that keeps CWC distinct +from between-cluster effects and from causal identification. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..569c9bce6 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0033](0033-longitudinal-cwc-analysis-run.md) | Longitudinal CWC composition as an analysis-run output profile | Accepted | active-PR | Binds Enders–Tofighi CWC within/between/contextual slopes to `longitudinal_cwc_v1`; cutoff-filters rows; refuses causal promotion. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | @@ -140,6 +141,7 @@ Use the narrowest owning ADR when decisions overlap: - **accepted-run execution and terminal artifact production:** ADR 0022. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. +- **longitudinal CWC analysis-run output profile:** ADR 0033. ## Change and supersession rule diff --git a/docs/doctoring/longitudinal-cwc-analysis-run.md b/docs/doctoring/longitudinal-cwc-analysis-run.md new file mode 100644 index 000000000..933fc84bb --- /dev/null +++ b/docs/doctoring/longitudinal-cwc-analysis-run.md @@ -0,0 +1,18 @@ +# Longitudinal CWC analysis-run bind + +**Review date:** 2026-08-31 +**Active slice:** GAP-006 / issue #169 remaining operator-visible composition + +Protected main already recovers Enders and Tofighi (2007) CWC within/between +OLS in `psychometric_core`. This slice binds that recovery to +`analysis_engine` as a cutoff-safe analysis-run output: eligibility against +the request knowledge cutoff, digest-bound `tepp.longitudinal_cwc.v1`, and an +explicit refusal to treat the slopes as a causal effect. + +This is not a new estimator, not a Driver p.16 `std` restore, not persistence, +and not implemented-main. + +## Evidence boundary + +Exact-head checks, independent review, and protected merge are required before +the profile can be promoted.