From fd3e85a1f507cbcb28aa270b083304a2293c24a0 Mon Sep 17 00:00:00 2001 From: seonghobae Date: Mon, 31 Aug 2026 17:45:39 +0000 Subject: [PATCH] feat(analysis): bind joint posterior Laplace draws to an analysis-run profile GAP-004 remaining slice (ADR 0051): operators request cutoff-safe joint_posterior_draws_v1 which fits the CPU f64 TRSL-TM reference, builds the identified Gauss-Newton Laplace precision, and materializes Philox/ Box-Muller/Cholesky plausible values. Not MCMC, not Schwarz candidate-K (#404), not GPU, not topic birth/split/merge. Persistence remains later. --- CHANGELOG.md | 2 + DOCUMENTATION.md | 1 + .../src/joint_posterior_draws_artifact.rs | 330 ++++++++++++++++++ crates/analysis_engine/src/lib.rs | 18 +- ...oint_posterior_draws_execution_contract.rs | 283 +++++++++++++++ docs/TRACEABILITY.md | 1 + ...0052-joint-posterior-draws-analysis-run.md | 83 +++++ docs/adr/README.md | 2 + .../joint-posterior-draws-analysis-run.md | 18 + 9 files changed, 737 insertions(+), 1 deletion(-) create mode 100644 crates/analysis_engine/src/joint_posterior_draws_artifact.rs create mode 100644 crates/analysis_engine/tests/joint_posterior_draws_execution_contract.rs create mode 100644 docs/adr/0052-joint-posterior-draws-analysis-run.md create mode 100644 docs/doctoring/joint-posterior-draws-analysis-run.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..3fe98870e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- **Joint posterior Laplace draws analysis-run profile**: cutoff-safe `joint_posterior_draws_v1` binds `draw_joint_gaussian` (Philox/Box-Muller/Cholesky) and refuses to claim MCMC, GPU, or topic birth/split/merge (`analysis_engine`). Not implemented-main. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 6fa4b9683..e2b8e664c 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -71,6 +71,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) | | Analysis engine v1 doctoring | [`docs/doctoring/analysis-engine-v1.md`](docs/doctoring/analysis-engine-v1.md) | | Analysis engine gap-closure doctoring | [`docs/doctoring/analysis-engine-gap-closure.md`](docs/doctoring/analysis-engine-gap-closure.md) | +| Joint posterior Laplace draws analysis-run doctoring | [`docs/doctoring/joint-posterior-draws-analysis-run.md`](docs/doctoring/joint-posterior-draws-analysis-run.md) | | Corpus-split leakage-audit wire doctoring | [`docs/research/corpus-split-manifest-wire.md`](docs/research/corpus-split-manifest-wire.md) | | Unicode canonical-identity doctoring | [`docs/research/unicode-canonical-identity.md`](docs/research/unicode-canonical-identity.md) | | Change history | [`CHANGELOG.md`](CHANGELOG.md) | diff --git a/crates/analysis_engine/src/joint_posterior_draws_artifact.rs b/crates/analysis_engine/src/joint_posterior_draws_artifact.rs new file mode 100644 index 000000000..3d156c9d5 --- /dev/null +++ b/crates/analysis_engine/src/joint_posterior_draws_artifact.rs @@ -0,0 +1,330 @@ +//! Digest-bound joint Gaussian Laplace plausible-value draws as an analysis-run profile. + +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::KnowledgeCutoff; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; +use topic_measurement::{ + JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION, ReferenceTopicInput, ReferenceTopicModelConfig, + fit_reference_topic_model, +}; +use uuid::Uuid; + +use crate::{AnalysisEngineError, format_digest, require_receipt_identity, valid_identifier}; + +/// Versioned schema for a completed joint-posterior-draw artifact. +pub const JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION: &str = "tepp.joint_posterior_draws.v1"; +/// Model contract required by the joint-posterior-draw execution path. +pub const JOINT_POSTERIOR_DRAWS_MODEL_CONTRACT_VERSION: &str = "joint_posterior_draws_v1"; +/// Analysis-run output profile required for a joint-posterior-draw artifact. +pub const JOINT_POSTERIOR_DRAWS_OUTPUT_PROFILE: &str = "joint_posterior_draws_v1"; +/// Maximum canonical artifact JSON size. +pub const JOINT_POSTERIOR_DRAWS_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const JOINT_POSTERIOR_DRAWS_INFERENCE_STATUS: &str = + "joint_gaussian_laplace_plausible_values_not_mcmc"; +const JOINT_GAUSS_NEWTON_LAPLACE: &str = "joint_gauss_newton_laplace"; + +/// Completed, bounded joint posterior Laplace draws for analysis-run clients. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct JointPosteriorDrawsArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used by the fit and draws. + pub knowledge_cutoff: String, + /// SHA-256 identity binding algorithm, seed, basis, fit, and draws. + pub draw_set_id: String, + /// Stable counter-based draw algorithm identity. + pub algorithm_version: String, + /// Explicit Philox seed used for the draw set. + pub seed: u64, + /// Number of joint Gaussian draws materialized. + pub draw_count: u64, + /// Number of modeled evidence documents. + pub document_count: u64, + /// Number of global topics in the fitted model. + pub topic_count: u64, + /// Laplace approximation identity (not MCMC). + pub approximation: String, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl JointPosteriorDrawsArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidJointPosteriorDrawsArtifact`] when + /// the schema, identifiers, counts, algorithm, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > JOINT_POSTERIOR_DRAWS_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidJointPosteriorDrawsArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation, serialization, or size failure. + pub fn to_json(&self) -> Result { + self.validate()?; + let payload = + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; + if payload.len() > JOINT_POSTERIOR_DRAWS_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + Ok(payload) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + if self.schema_version != JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || !valid_identifier(&self.draw_set_id) + || self.draw_set_id.len() != 64 + || self.algorithm_version != JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION + || self.draw_count == 0 + || self.document_count < 2 + || self.topic_count < 2 + || self.approximation != JOINT_GAUSS_NEWTON_LAPLACE + || self.inference_status != JOINT_POSTERIOR_DRAWS_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidJointPosteriorDrawsArtifact); + } + Ok(()) + } +} + +/// One completed joint-posterior-draw artifact and its terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct JointPosteriorDrawsExecution { + /// Digest-bound completed draw-set artifact. + pub artifact: JointPosteriorDrawsArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +/// Execute cutoff-safe joint Gaussian Laplace draws as one analysis-run profile. +/// +/// The executor fits the CPU `f64` TRSL-TM reference, builds the identified +/// joint Gauss-Newton Laplace precision, and invokes +/// [`topic_measurement::JointCoordinatePrecision::draw_joint_gaussian`]. It +/// does not invent MCMC, select GPU backends, score candidate `K`, or emit +/// topic birth/split/merge events. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, estimator failure, +/// invalid draw request, or invalid artifact error. +#[allow(clippy::too_many_arguments)] +pub fn execute_joint_posterior_draws_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + input: &ReferenceTopicInput, + config: &ReferenceTopicModelConfig, + topic_ids: Vec, + draw_count: usize, + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + || request.model_contract_version != JOINT_POSTERIOR_DRAWS_MODEL_CONTRACT_VERSION + || request.output_profile != JOINT_POSTERIOR_DRAWS_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let model = fit_reference_topic_model(input, config)?; + let precision = input.build_joint_coordinate_precision(&model, config, topic_ids)?; + let draws = precision.draw_joint_gaussian(model.seed, draw_count)?; + let document_count = u64::try_from(draws.document_ids().len()) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let topic_count = u64::try_from(draws.topic_ids().len()) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let draw_count = + u64::try_from(draws.draws().len()).map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let artifact = JointPosteriorDrawsArtifact { + schema_version: JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + draw_set_id: draws.draw_set_id().to_owned(), + algorithm_version: JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION.into(), + seed: draws.seed(), + draw_count, + document_count, + topic_count, + approximation: JOINT_GAUSS_NEWTON_LAPLACE.into(), + inference_status: JOINT_POSTERIOR_DRAWS_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "joint_posterior_draws", + document_count, + 4, + JOINT_POSTERIOR_DRAWS_INFERENCE_STATUS, + )?; + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("joint_posterior_draws_artifact_{}", &digest[..16]), + digest, + JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(JointPosteriorDrawsExecution { + artifact, + terminal_result, + }) +} + +#[cfg(test)] +mod tests { + use super::{ + JOINT_POSTERIOR_DRAWS_ARTIFACT_BYTE_LIMIT, JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION, + JOINT_POSTERIOR_DRAWS_INFERENCE_STATUS, JointPosteriorDrawsArtifact, + }; + use crate::AnalysisEngineError; + use topic_measurement::JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION; + + fn artifact() -> JointPosteriorDrawsArtifact { + JointPosteriorDrawsArtifact { + schema_version: JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + draw_set_id: "a".repeat(64), + algorithm_version: JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION.into(), + seed: 7, + draw_count: 4, + document_count: 4, + topic_count: 2, + approximation: "joint_gauss_newton_laplace".into(), + inference_status: JOINT_POSTERIOR_DRAWS_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &JointPosteriorDrawsArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidJointPosteriorDrawsArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + JointPosteriorDrawsArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + JointPosteriorDrawsArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidJointPosteriorDrawsArtifact) + ); + assert_eq!( + JointPosteriorDrawsArtifact::from_json( + &"x".repeat(JOINT_POSTERIOR_DRAWS_ARTIFACT_BYTE_LIMIT + 1) + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.draw_set_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.algorithm_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.draw_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.document_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.topic_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.approximation.clear(); + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } +} diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..595cf421d 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -8,9 +8,12 @@ //! through [`tepp_api`]. It deliberately does not claim latent-variable or topic //! estimation authority; those estimators remain separate scientific crates. //! estimation authority; it invokes estimators through their scientific crate -//! contracts and preserves their artifact meaning. +//! contracts and preserves their artifact meaning. Joint Gaussian Laplace +//! plausible-value draws are invoked through `topic_measurement` and are not +//! MCMC. mod case_deletion_refit; +mod joint_posterior_draws_artifact; mod lineage_criterion; mod topic_context_posterior; mod topic_lineage_artifact; @@ -41,6 +44,12 @@ pub use case_deletion_refit::ExhaustiveCaseDeletionError; pub use case_deletion_refit::ExhaustiveCaseDeletionFits; /// Fit the full corpus and every actual one-document deletion. pub use case_deletion_refit::fit_exhaustive_case_deletion; +/// Joint posterior Laplace-draw artifact and execution contracts from this engine. +pub use joint_posterior_draws_artifact::{ + JOINT_POSTERIOR_DRAWS_ARTIFACT_BYTE_LIMIT, JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION, + JOINT_POSTERIOR_DRAWS_MODEL_CONTRACT_VERSION, JOINT_POSTERIOR_DRAWS_OUTPUT_PROFILE, + JointPosteriorDrawsArtifact, JointPosteriorDrawsExecution, execute_joint_posterior_draws_run, +}; /// Rust-owned independent TDT link-criterion posterior fitting contracts. pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, @@ -248,6 +257,8 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A joint-posterior-draw artifact violated its bounded schema or counts. + InvalidJointPosteriorDrawsArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +273,7 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::InvalidJointPosteriorDrawsArtifact => "invalid joint posterior draws artifact", }; formatter.write_str(message) } @@ -681,6 +693,10 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::InvalidJointPosteriorDrawsArtifact, + "invalid joint posterior draws artifact", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); diff --git a/crates/analysis_engine/tests/joint_posterior_draws_execution_contract.rs b/crates/analysis_engine/tests/joint_posterior_draws_execution_contract.rs new file mode 100644 index 000000000..9dcfff38d --- /dev/null +++ b/crates/analysis_engine/tests/joint_posterior_draws_execution_contract.rs @@ -0,0 +1,283 @@ +//! End-to-end contract for cutoff-safe joint posterior Laplace draws. + +use analysis_engine::{ + AnalysisEngineError, JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION, + JOINT_POSTERIOR_DRAWS_MODEL_CONTRACT_VERSION, JOINT_POSTERIOR_DRAWS_OUTPUT_PROFILE, + execute_joint_posterior_draws_run, +}; +use corpus_split::{CorpusDocument, CorpusSnapshot}; +use membership_core::{ + GroupId, MemberId, MembershipAssignment, MembershipNetwork, MembershipRole, MembershipWeight, +}; +use relation_graph::{ + RelationEdge, RelationEndpointId, RelationEvidenceStatus, RelationGraph, RelationKind, +}; +use temporal_core::{ + AvailableTime, EventTime, KnowledgeCutoff, TemporalBoundary, TemporalInterval, + TemporalPrecision, +}; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; +use topic_measurement::{ + JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION, ReferenceTopicInput, ReferenceTopicModelConfig, + SparseMatrix, TopicMeasurementError, +}; +use uuid::Uuid; + +fn event_time(day: u8) -> EventTime { + EventTime::parse_rfc3339(&format!("2026-07-{day:02}T00:00:00Z")).expect("event time") +} + +fn fixture() -> ( + CorpusSnapshot, + Vec, + Vec, + MembershipNetwork, + RelationGraph, +) { + let ids: Vec<_> = (1_u128..=4).map(Uuid::from_u128).collect(); + let times: Vec<_> = (1_u8..=4).map(event_time).collect(); + let cutoff = KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff"); + let available = AvailableTime::parse_rfc3339("2026-07-01T00:00:00Z").expect("available"); + let mut snapshot = CorpusSnapshot::new(); + let mut memberships = MembershipNetwork::new(); + for id in &ids { + snapshot + .insert_if_eligible(CorpusDocument::new(*id, available), &cutoff) + .expect("eligible"); + memberships + .insert( + MembershipAssignment::new( + MemberId::from_uuid(*id), + GroupId::from_uuid(Uuid::from_u128(100)), + MembershipRole::Project, + MembershipWeight::full().expect("weight"), + event_time(1), + event_time(9), + ) + .expect("membership"), + ) + .expect("insert"); + } + let mut relations = RelationGraph::new(); + for (source, target, source_day, target_day) in [(0, 1, 1, 2), (1, 2, 2, 3), (2, 3, 3, 4)] { + let interval = |day| { + TemporalInterval::bounded( + TemporalBoundary::Included(event_time(day)), + TemporalBoundary::Included( + EventTime::parse_rfc3339(&format!("2026-07-{day:02}T12:00:00Z")).expect("end"), + ), + TemporalPrecision::Second, + ) + .expect("interval") + }; + relations + .insert( + RelationEdge::new( + RelationKind::TransitionsTo, + RelationEndpointId::from_uuid(ids[source]), + RelationEndpointId::from_uuid(ids[target]), + RelationEvidenceStatus::Observed, + interval(source_day), + interval(target_day), + ) + .expect("relation"), + ) + .expect("insert relation"); + } + (snapshot, ids, times, memberships, relations) +} + +fn separated_input() -> ReferenceTopicInput { + let (snapshot, ids, times, memberships, relations) = fixture(); + let counts = SparseMatrix::from_csr( + 4, + 4, + vec![0, 2, 4, 6, 8], + vec![0, 1, 0, 1, 2, 3, 2, 3], + vec![90.0, 10.0, 85.0, 15.0, 10.0, 90.0, 15.0, 85.0], + ) + .expect("counts"); + ReferenceTopicInput::new( + &snapshot, + ids, + &counts, + ×, + None, + &memberships, + &relations, + ) + .expect("input") +} + +fn recovery_config() -> ReferenceTopicModelConfig { + ReferenceTopicModelConfig::new(2, vec![7, 11], 2_000, 1e-5) + .expect("config") + .with_hyperparameters(1.0, 0.5, 0.01, 0.05, 0.2) + .expect("hyperparameters") +} + +fn topic_ids() -> Vec { + (1001_u128..=1002).map(Uuid::from_u128).collect() +} + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "joint-posterior-draws-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-joint-posterior-draws".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: JOINT_POSTERIOR_DRAWS_MODEL_CONTRACT_VERSION.into(), + output_profile: JOINT_POSTERIOR_DRAWS_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new( + "run-joint-posterior-draws", + "accepted", + &request.idempotency_key, + ) + .expect("accepted") +} + +fn execute( + request: &AnalysisRunRequest, + draw_count: usize, +) -> Result { + execute_joint_posterior_draws_run( + request, + &accepted(request), + "snapshot-joint-posterior-draws", + cutoff(), + &separated_input(), + &recovery_config(), + topic_ids(), + draw_count, + "2026-08-02T00:00:00Z", + ) +} + +#[test] +fn fitted_precision_emits_digest_bound_laplace_draws() { + let request = request(); + let execution = execute(&request, 4).expect("execution"); + assert_eq!( + execution.artifact.schema_version, + JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!( + execution.artifact.algorithm_version, + JOINT_POSTERIOR_DRAW_ALGORITHM_VERSION + ); + assert_eq!(execution.artifact.draw_count, 4); + assert_eq!(execution.artifact.document_count, 4); + assert_eq!(execution.artifact.topic_count, 2); + assert_eq!(execution.artifact.draw_set_id.len(), 64); + assert_eq!( + execution.artifact.approximation, + "joint_gauss_newton_laplace" + ); + assert_eq!( + execution.artifact.inference_status, + "joint_gaussian_laplace_plausible_values_not_mcmc" + ); + assert!(!execution.artifact.to_json().expect("json").contains("rmse")); + assert!( + !execution + .artifact + .to_json() + .expect("json") + .contains("scientific_acceptance") + ); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(JOINT_POSTERIOR_DRAWS_ARTIFACT_SCHEMA_VERSION) + ); +} + +#[test] +fn execution_refuses_zero_draws_and_nonconvergence() { + let request = request(); + assert_eq!( + execute(&request, 0), + Err(AnalysisEngineError::TopicMeasurement( + TopicMeasurementError::InvalidModelInput + )) + ); + let exhausted = ReferenceTopicModelConfig::new(2, vec![1], 2, 1e-12).expect("exhausted"); + assert_eq!( + execute_joint_posterior_draws_run( + &request, + &accepted(&request), + "snapshot-joint-posterior-draws", + cutoff(), + &separated_input(), + &exhausted, + topic_ids(), + 4, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::TopicMeasurement( + TopicMeasurementError::DidNotConverge + )) + ); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + assert_eq!( + execute_joint_posterior_draws_run( + &request, + &accepted(&request), + "other-snapshot", + cutoff(), + &separated_input(), + &recovery_config(), + topic_ids(), + 4, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + for invalid_request in [ + { + let mut value = request.clone(); + value.knowledge_cutoff = "2026-08-02T00:00:00Z".into(); + value + }, + { + let mut value = request.clone(); + value.model_contract_version = "trsl_tm_cpu_f64_v1".into(); + value + }, + { + let mut value = request.clone(); + value.output_profile = "trsl_topic_lineage_v1".into(); + value + }, + { + let mut value = request.clone(); + value.output_profile = "fitted_candidate_k_v1".into(); + value + }, + ] { + assert_eq!( + execute(&invalid_request, 4), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..162259fa0 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| joint posterior Laplace draws analysis-run profile | ADR 0012/0022/0052; Philox/Box-Muller/Cholesky | `analysis_engine` `joint_posterior_draws_v1` binds `draw_joint_gaussian`; digest-bound Laplace plausible values, not MCMC, not Schwarz candidate-`K`, not GPU, not topic birth/split/merge; not implemented-main | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0052-joint-posterior-draws-analysis-run.md b/docs/adr/0052-joint-posterior-draws-analysis-run.md new file mode 100644 index 000000000..e08929dd3 --- /dev/null +++ b/docs/adr/0052-joint-posterior-draws-analysis-run.md @@ -0,0 +1,83 @@ +# ADR 0052 — Joint posterior Laplace draws as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0012 (TRSL-TM joint precision / plausible values) and ADR 0022 (cutoff-safe analysis-run execution). Does not reuse ADR 0049 (fitted candidate-`K`), ADR 0050 (interpreter/verifier), or ADR 0051 (topic activity/dormancy). +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already materializes deterministic joint Gaussian Laplace +plausible values from an identified Gauss-Newton precision via +`JointCoordinatePrecision::draw_joint_gaussian` (Philox4x32-10, Box-Muller, +Cholesky). Operators still cannot request that draw set as a digest-bound +analysis-run output. Fitted candidate-`K` (#404 / ADR 0049) is Schwarz +selection and explicitly not a sampler. Fixed-`K` `trsl_topic_lineage_v1` +emits predecessor/successor edges, not draws. The +`tepp.topic_context_posterior.v1` producer contract remains DTO-only. + +GPU kernels, method effects, MCMC, and topic birth/split/merge remain later +GAP-004 work and are not this slice. + +## Decision + +Add the `joint_posterior_draws_v1` analysis-run output profile to +`analysis_engine`. The executor: + +- consumes an already-validated `ReferenceTopicInput` plus + `ReferenceTopicModelConfig` and caller-owned topic identities; +- requires the request snapshot and knowledge cutoff to match the offered + input construction; +- fits the CPU `f64` TRSL-TM reference, builds the joint Laplace precision, + and draws through `draw_joint_gaussian` without reimplementing Philox, + Box-Muller, or Cholesky; +- emits a canonical SHA-256-digested `tepp.joint_posterior_draws.v1` artifact + with `draw_set_id`, algorithm version, seed, draw/document/topic counts, and + inference status `joint_gaussian_laplace_plausible_values_not_mcmc`; +- does not persist draw coordinates on the operator artifact (the draw-set + digest already binds them), invent MCMC, select GPU backends, score + candidate `K`, or emit topic birth/split/merge events. + +This is Laplace plausible-value materialization, not MCMC and not Schwarz +candidate-`K` selection. + +## Alternatives considered + +1. Restore another Driver p.16 standardised matrix — rejected because those + recoveries are already a live micro-PR family and do not bind posterior + draws to an analysis run. +2. Duplicate fitted candidate-`K` (#404) — rejected because that profile + explicitly refuses to be a sampler. +3. Emit the full `tepp.topic_context_posterior.v1` producer contract — + rejected because that DTO also requires membership, activity, and lineage + events this slice does not fit. +4. Bind the existing joint Laplace draw generator to ADR 0022's analysis-run + profile — accepted. + +## Consequences + +Operators can request cutoff-safe joint Laplace draws as a digest-bound +terminal result. The artifact does not claim MCMC, GPU parity, method +effects, or topic birth/split/merge. Snapshot/profile/cutoff mismatch, failed +fits, and zero/oversized draw counts fail closed. + +## Verification + +The PR includes Rust unit and integration tests for digest-bound draws on a +separated two-topic corpus, zero-draw refusal, non-convergence, snapshot / +profile / cutoff mismatch, and artifact tampering. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +python3 scripts/validate_documentation.py +``` + +## Rollback and supersession + +Rollback removes the `joint_posterior_draws_v1` profile. No persisted schema +migration is introduced. Supersede only with an ADR that keeps Laplace +plausible values distinct from MCMC, Schwarz candidate-`K`, and GPU kernels. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..c4b7cf6f1 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0052](0052-joint-posterior-draws-analysis-run.md) | Joint posterior Laplace draws as an analysis-run profile | Accepted | active-PR | Complements ADR 0012/0022; Philox/Box-Muller/Cholesky Laplace draws, not MCMC, not Schwarz candidate-`K`. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | @@ -138,6 +139,7 @@ Use the narrowest owning ADR when decisions overlap: - **project-history wire-size symmetry:** ADR 0019. - **LineageWeave project-history service boundary:** ADR 0021. - **accepted-run execution and terminal artifact production:** ADR 0022. +- **joint posterior Laplace draws analysis-run profile:** ADR 0052. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. diff --git a/docs/doctoring/joint-posterior-draws-analysis-run.md b/docs/doctoring/joint-posterior-draws-analysis-run.md new file mode 100644 index 000000000..00d6b4d29 --- /dev/null +++ b/docs/doctoring/joint-posterior-draws-analysis-run.md @@ -0,0 +1,18 @@ +# Joint posterior Laplace draws analysis-run composition + +**Active slice:** ADR 0052 / `joint_posterior_draws_v1` +**Protected-main status:** not implemented-main + +`topic_measurement` already draws deterministic joint Gaussian Laplace +plausible values from an identified Gauss-Newton precision (Philox4x32-10, +Box-Muller, Cholesky). This slice binds that generator to a cutoff-safe +analysis-run profile so an operator can request a digest-bound terminal +result. + +The executor does not invent MCMC, select GPU backends, score candidate `K`, +or emit topic birth/split/merge events. It is not the +`fitted_candidate_k_v1` Schwarz selector and not the fixed-`K` +`trsl_topic_lineage_v1` lineage profile. + +Exact-head Checks and two independent approvals are required before any +implemented-main claim.