diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..a72f8b457 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- **Fitted candidate-`K` analysis-run profile**: cutoff-safe `fitted_candidate_k_v1` binds `select_fitted_candidate_k` and refuses lexical methods and LLM-vote authority (`analysis_engine`). Not a Bayesian sampler and not implemented-main. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..0949729f1 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -74,6 +74,7 @@ dependencies = [ "corpus_split", "event_core", "membership_core", + "model_selection", "relation_graph", "serde", "serde_json", diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..c45e2eed7 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -15,6 +15,7 @@ publish = false [dependencies] event_core = { path = "../event_core", version = "0.2.0" } +model_selection = { path = "../model_selection", version = "0.2.0" } serde = { workspace = true } serde_json = { workspace = true } sha2 = { workspace = true } diff --git a/crates/analysis_engine/src/fitted_candidate_k_artifact.rs b/crates/analysis_engine/src/fitted_candidate_k_artifact.rs new file mode 100644 index 000000000..16080bae3 --- /dev/null +++ b/crates/analysis_engine/src/fitted_candidate_k_artifact.rs @@ -0,0 +1,302 @@ +//! Digest-bound fitted candidate-`K` selection as an analysis-run profile. + +use model_selection::{FittedCandidateKConfig, select_fitted_candidate_k}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::KnowledgeCutoff; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; +use topic_measurement::ReferenceTopicInput; + +use crate::{AnalysisEngineError, format_digest, require_receipt_identity, valid_identifier}; + +/// Versioned schema for a completed fitted candidate-`K` artifact. +pub const FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION: &str = "tepp.fitted_candidate_k.v1"; +/// Model contract required by the fitted candidate-`K` execution path. +pub const FITTED_CANDIDATE_K_MODEL_CONTRACT_VERSION: &str = "fitted_candidate_k_v1"; +/// Analysis-run output profile required for a fitted candidate-`K` artifact. +pub const FITTED_CANDIDATE_K_OUTPUT_PROFILE: &str = "fitted_candidate_k_v1"; +/// Maximum canonical artifact JSON size. +pub const FITTED_CANDIDATE_K_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const FITTED_CANDIDATE_K_INFERENCE_STATUS: &str = "fitted_schwarz_candidate_k_not_bayesian_sampler"; + +/// Completed, bounded fitted candidate-`K` selection for analysis-run clients. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct FittedCandidateKArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used by the selection. + pub knowledge_cutoff: String, + /// Statistically selected topic count `K`. + pub selected_k: u64, + /// Number of candidate topic counts offered to the selector. + pub candidate_count: u64, + /// Number of modeled evidence documents. + pub evidence_count: u64, + /// Declared statistical method identity (not an LLM label). + pub method_name: String, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl FittedCandidateKArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidFittedCandidateKArtifact`] when the + /// schema, identifiers, counts, method, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > FITTED_CANDIDATE_K_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidFittedCandidateKArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn to_json(&self) -> Result { + self.validate()?; + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + if self.schema_version != FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || self.selected_k < 2 + || self.candidate_count == 0 + || self.evidence_count < 2 + || !valid_identifier(&self.method_name) + || self.inference_status != FITTED_CANDIDATE_K_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidFittedCandidateKArtifact); + } + Ok(()) + } +} + +/// One completed fitted candidate-`K` artifact and its terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct FittedCandidateKExecution { + /// Digest-bound completed selection artifact. + pub artifact: FittedCandidateKArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +#[allow( + clippy::too_many_arguments, + clippy::missing_panics_doc, + reason = "audited cutoff, method, vote, and selection-config gates" +)] +/// Execute cutoff-safe fitted candidate-`K` selection as one analysis-run profile. +/// +/// The executor invokes [`select_fitted_candidate_k`] and does not reimplement +/// Schwarz scoring, Pareto admission, or the CPU `f64` reference fit. LLM votes +/// cannot define the numerical optimum. This is not a Bayesian sampler, not GPU +/// execution, and not topic birth/split/merge. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, model-selection +/// failure, or invalid artifact error. +pub fn execute_fitted_candidate_k_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + input: &ReferenceTopicInput, + config: &FittedCandidateKConfig, + method_name: &str, + llm_votes: &[u32], + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + || request.model_contract_version != FITTED_CANDIDATE_K_MODEL_CONTRACT_VERSION + || request.output_profile != FITTED_CANDIDATE_K_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + if !valid_identifier(method_name) { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let selected_k = u64::from(select_fitted_candidate_k( + input, + config, + method_name, + llm_votes, + )?); + let candidate_count = u64::try_from(config.candidate_topic_counts().len()) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let evidence_count = u64::try_from(input.document_count()) + .map_err(|_| AnalysisEngineError::ArithmeticOverflow)?; + let artifact = FittedCandidateKArtifact { + schema_version: FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + selected_k, + candidate_count, + evidence_count, + method_name: method_name.to_owned(), + inference_status: FITTED_CANDIDATE_K_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "fitted_candidate_k", + evidence_count, + 2, + FITTED_CANDIDATE_K_INFERENCE_STATUS, + ) + .expect("fixed summary fields and bounded evidence count are valid"); + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("fitted_candidate_k_artifact_{}", &digest[..16]), + digest, + FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(FittedCandidateKExecution { + artifact, + terminal_result, + }) +} + +#[cfg(test)] +mod tests { + use super::{ + FITTED_CANDIDATE_K_ARTIFACT_BYTE_LIMIT, FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION, + FITTED_CANDIDATE_K_INFERENCE_STATUS, FittedCandidateKArtifact, + }; + use crate::AnalysisEngineError; + + fn artifact() -> FittedCandidateKArtifact { + FittedCandidateKArtifact { + schema_version: FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + selected_k: 2, + candidate_count: 2, + evidence_count: 6, + method_name: "trsl_tm_reference".into(), + inference_status: FITTED_CANDIDATE_K_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &FittedCandidateKArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidFittedCandidateKArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + FittedCandidateKArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + FittedCandidateKArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidFittedCandidateKArtifact) + ); + assert_eq!( + FittedCandidateKArtifact::from_json( + &"x".repeat(FITTED_CANDIDATE_K_ARTIFACT_BYTE_LIMIT + 1) + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.selected_k = 1; + value + }, + { + let mut value = artifact.clone(); + value.candidate_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.evidence_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.method_name.clear(); + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } +} diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..e3303b66b 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -8,13 +8,17 @@ //! through [`tepp_api`]. It deliberately does not claim latent-variable or topic //! estimation authority; those estimators remain separate scientific crates. //! estimation authority; it invokes estimators through their scientific crate -//! contracts and preserves their artifact meaning. +//! contracts and preserves their artifact meaning. Fitted candidate-`K` +//! selection is invoked through [`model_selection`] and is not a Bayesian +//! sampler. mod case_deletion_refit; +mod fitted_candidate_k_artifact; mod lineage_criterion; mod topic_context_posterior; mod topic_lineage_artifact; +use model_selection::ModelSelectionError; use serde::Serialize; use sha2::{Digest, Sha256}; use std::collections::BTreeSet; @@ -41,6 +45,12 @@ pub use case_deletion_refit::ExhaustiveCaseDeletionError; pub use case_deletion_refit::ExhaustiveCaseDeletionFits; /// Fit the full corpus and every actual one-document deletion. pub use case_deletion_refit::fit_exhaustive_case_deletion; +/// Fitted candidate-`K` artifact and execution contracts from this engine. +pub use fitted_candidate_k_artifact::{ + FITTED_CANDIDATE_K_ARTIFACT_BYTE_LIMIT, FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION, + FITTED_CANDIDATE_K_MODEL_CONTRACT_VERSION, FITTED_CANDIDATE_K_OUTPUT_PROFILE, + FittedCandidateKArtifact, FittedCandidateKExecution, execute_fitted_candidate_k_run, +}; /// Rust-owned independent TDT link-criterion posterior fitting contracts. pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, @@ -248,6 +258,10 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A model-selection gate rejected the offered candidates or method. + ModelSelection(ModelSelectionError), + /// A fitted candidate-`K` artifact violated its bounded schema or counts. + InvalidFittedCandidateKArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +276,8 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::ModelSelection(error) => return error.fmt(formatter), + Self::InvalidFittedCandidateKArtifact => "invalid fitted candidate-k artifact", }; formatter.write_str(message) } @@ -281,6 +297,12 @@ impl From for AnalysisEngineError { } } +impl From for AnalysisEngineError { + fn from(error: ModelSelectionError) -> Self { + Self::ModelSelection(error) + } +} + /// Execute the cutoff-safe temporal evidence readiness analysis. /// /// Evidence whose `available_time` is later than the request cutoff is excluded @@ -413,7 +435,8 @@ mod tests { use super::{ ANALYSIS_ARTIFACT_SCHEMA_VERSION, ANALYSIS_STATISTIC_COUNT, AnalysisCorpus, AnalysisEngineError, AnalysisEvidenceUnit, MAX_ANALYSIS_IDENTIFIER_BYTES, - MAX_EVIDENCE_UNITS, TopicMeasurementError, add_membership_count, execute_analysis_run, + MAX_EVIDENCE_UNITS, ModelSelectionError, TopicMeasurementError, add_membership_count, + execute_analysis_run, }; use temporal_core::{AvailableTime, EventTime}; use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState, ApiError}; @@ -681,6 +704,14 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::ModelSelection(ModelSelectionError::NoSuccessfulFit), + "no fitted candidate produced a finite diagnostic", + ), + ( + AnalysisEngineError::InvalidFittedCandidateKArtifact, + "invalid fitted candidate-k artifact", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); diff --git a/crates/analysis_engine/tests/fitted_candidate_k_execution_contract.rs b/crates/analysis_engine/tests/fitted_candidate_k_execution_contract.rs new file mode 100644 index 000000000..276ca6942 --- /dev/null +++ b/crates/analysis_engine/tests/fitted_candidate_k_execution_contract.rs @@ -0,0 +1,309 @@ +//! End-to-end contract for cutoff-safe fitted candidate-`K` selection. + +use analysis_engine::{ + AnalysisEngineError, FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION, + FITTED_CANDIDATE_K_MODEL_CONTRACT_VERSION, FITTED_CANDIDATE_K_OUTPUT_PROFILE, + execute_fitted_candidate_k_run, +}; +use corpus_split::{CorpusDocument, CorpusSnapshot}; +use membership_core::{ + GroupId, MemberId, MembershipAssignment, MembershipNetwork, MembershipRole, MembershipWeight, +}; +use model_selection::{FittedCandidateKConfig, ModelSelectionError}; +use relation_graph::{ + RelationEdge, RelationEndpointId, RelationEvidenceStatus, RelationGraph, RelationKind, +}; +use temporal_core::{ + AvailableTime, EventTime, KnowledgeCutoff, TemporalBoundary, TemporalInterval, + TemporalPrecision, +}; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; +use topic_measurement::{ReferenceTopicInput, SparseMatrix}; +use uuid::Uuid; + +fn event_time(day: u8) -> EventTime { + EventTime::parse_rfc3339(&format!("2026-01-{day:02}T00:00:00Z")).expect("event time") +} + +fn relation(source: Uuid, target: Uuid, source_day: u8, target_day: u8) -> RelationEdge { + let interval = |day| { + TemporalInterval::bounded( + TemporalBoundary::Included(event_time(day)), + TemporalBoundary::Included( + EventTime::parse_rfc3339(&format!("2026-01-{day:02}T12:00:00Z")) + .expect("interval end"), + ), + TemporalPrecision::Second, + ) + .expect("bounded interval") + }; + RelationEdge::new( + RelationKind::TransitionsTo, + RelationEndpointId::from_uuid(source), + RelationEndpointId::from_uuid(target), + RelationEvidenceStatus::Observed, + interval(source_day), + interval(target_day), + ) + .expect("forward relation") +} + +fn separated_topic_input() -> ReferenceTopicInput { + let document_ids: Vec<_> = (1_u128..=6).map(Uuid::from_u128).collect(); + let times: Vec<_> = (1_u8..=6).map(event_time).collect(); + let available = AvailableTime::parse_rfc3339("2026-01-10T00:00:00Z").expect("available"); + let cutoff = KnowledgeCutoff::parse_rfc3339("2026-02-01T00:00:00Z").expect("cutoff"); + let mut snapshot = CorpusSnapshot::new(); + for id in &document_ids { + snapshot + .insert_if_eligible(CorpusDocument::new(*id, available), &cutoff) + .expect("eligible"); + } + + let organization = GroupId::from_uuid(Uuid::from_u128(100)); + let projects = [ + GroupId::from_uuid(Uuid::from_u128(101)), + GroupId::from_uuid(Uuid::from_u128(102)), + ]; + let validity_start = event_time(1); + let validity_end = event_time(9); + let mut memberships = MembershipNetwork::new(); + for (index, id) in document_ids.iter().enumerate() { + let member = MemberId::from_uuid(*id); + memberships + .insert( + MembershipAssignment::new( + member, + organization, + MembershipRole::Organization, + MembershipWeight::full().expect("full"), + validity_start, + validity_end, + ) + .expect("organization membership"), + ) + .expect("insert organization"); + memberships + .insert( + MembershipAssignment::new( + member, + projects[usize::from(index >= 3)], + MembershipRole::Project, + MembershipWeight::new(0.75).expect("partial"), + validity_start, + validity_end, + ) + .expect("project membership"), + ) + .expect("insert project"); + } + + let mut relations = RelationGraph::new(); + for (source, target, source_day, target_day) in [ + (0, 1, 1, 2), + (1, 2, 2, 3), + (2, 3, 3, 4), + (3, 4, 4, 5), + (4, 5, 5, 6), + ] { + relations + .insert(relation( + document_ids[source], + document_ids[target], + source_day, + target_day, + )) + .expect("insert relation"); + } + + let counts = SparseMatrix::from_csr( + 6, + 4, + vec![0, 2, 4, 6, 8, 10, 12], + vec![0, 1, 0, 1, 0, 1, 2, 3, 2, 3, 2, 3], + vec![ + 90.0, 10.0, 85.0, 15.0, 80.0, 20.0, 10.0, 90.0, 15.0, 85.0, 20.0, 80.0, + ], + ) + .expect("counts"); + ReferenceTopicInput::new( + &snapshot, + document_ids, + &counts, + ×, + None, + &memberships, + &relations, + ) + .expect("validated input") +} + +fn recovery_config() -> FittedCandidateKConfig { + FittedCandidateKConfig::new(vec![2, 3], vec![7, 11, 19], 2_000, 1e-5) + .expect("candidate configuration") + .with_hyperparameters(1.0, 0.5, 0.01, 0.05, 0.2) + .expect("hyperparameters") +} + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-02-01T00:00:00Z").expect("cutoff") +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "fitted-candidate-k-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-fitted-candidate-k".into(), + knowledge_cutoff: "2026-02-01T00:00:00Z".into(), + model_contract_version: FITTED_CANDIDATE_K_MODEL_CONTRACT_VERSION.into(), + output_profile: FITTED_CANDIDATE_K_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new( + "run-fitted-candidate-k", + "accepted", + &request.idempotency_key, + ) + .expect("accepted") +} + +fn execute( + request: &AnalysisRunRequest, + method_name: &str, + llm_votes: &[u32], +) -> Result { + execute_fitted_candidate_k_run( + request, + &accepted(request), + "snapshot-fitted-candidate-k", + cutoff(), + &separated_topic_input(), + &recovery_config(), + method_name, + llm_votes, + "2026-02-02T00:00:00Z", + ) +} + +#[test] +fn separated_topics_select_true_k_and_refuse_llm_vote_as_authority() { + let request = request(); + let execution = execute(&request, "trsl_tm_reference", &[3]).expect("execution"); + assert_eq!( + execution.artifact.schema_version, + FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!(execution.artifact.selected_k, 2); + assert_eq!(execution.artifact.candidate_count, 2); + assert_eq!(execution.artifact.evidence_count, 6); + assert_eq!(execution.artifact.method_name, "trsl_tm_reference"); + assert_eq!( + execution.artifact.inference_status, + "fitted_schwarz_candidate_k_not_bayesian_sampler" + ); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(FITTED_CANDIDATE_K_ARTIFACT_SCHEMA_VERSION) + ); +} + +#[test] +fn execution_refuses_lexical_methods_and_failed_fits() { + let request = request(); + assert_eq!( + execute(&request, "", &[]), + Err(AnalysisEngineError::InvalidEvidence) + ); + assert_eq!( + execute(&request, "tf-idf", &[]), + Err(AnalysisEngineError::ModelSelection( + ModelSelectionError::LexicalWeightForbidden + )) + ); + let exhausted = FittedCandidateKConfig::new(vec![2], vec![1], 2, 1e-12).expect("exhausted"); + assert_eq!( + execute_fitted_candidate_k_run( + &request, + &accepted(&request), + "snapshot-fitted-candidate-k", + cutoff(), + &separated_topic_input(), + &exhausted, + "trsl_tm_reference", + &[], + "2026-02-02T00:00:00Z", + ), + Err(AnalysisEngineError::ModelSelection( + ModelSelectionError::NoSuccessfulFit + )) + ); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + assert_eq!( + execute_fitted_candidate_k_run( + &request, + &accepted(&request), + "other-snapshot", + cutoff(), + &separated_topic_input(), + &recovery_config(), + "trsl_tm_reference", + &[], + "2026-02-02T00:00:00Z", + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + for invalid_request in [ + { + let mut value = request.clone(); + value.knowledge_cutoff = "2026-08-02T00:00:00Z".into(); + value + }, + { + let mut value = request.clone(); + value.model_contract_version = "other-model".into(); + value + }, + { + let mut value = request.clone(); + value.output_profile = "trsl_topic_lineage_v1".into(); + value + }, + ] { + assert_eq!( + execute(&invalid_request, "trsl_tm_reference", &[]), + Err(AnalysisEngineError::InvalidEvidence) + ); + } + + assert_eq!( + execute_fitted_candidate_k_run( + &request, + &accepted(&request), + "snapshot-fitted-candidate-k", + cutoff(), + &separated_topic_input(), + &recovery_config(), + "trsl_tm_reference", + &[], + "invalid", + ), + Err(AnalysisEngineError::Api( + tepp_api::ApiError::InvalidWirePayload + )) + ); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..e88e35dec 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| fitted candidate-K analysis-run selection | ADR 0012/0022/0049; Schwarz 1978 | `analysis_engine` `fitted_candidate_k_v1` binds `select_fitted_candidate_k`; refuses lexical methods and LLM-vote authority; not a Bayesian sampler and not implemented-main | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0049-fitted-candidate-k-analysis-run.md b/docs/adr/0049-fitted-candidate-k-analysis-run.md new file mode 100644 index 000000000..219c59c7e --- /dev/null +++ b/docs/adr/0049-fitted-candidate-k-analysis-run.md @@ -0,0 +1,81 @@ +# ADR 0049 — Fitted candidate-`K` selection as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0012 (TRSL-TM / candidate-`K` gates) and ADR 0022 (cutoff-safe analysis-run execution). +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already fits each candidate `K` with the CPU `f64` TRSL-TM +reference and scores Schwarz's (1978) `ℓ − (p ln N)/2` inside +`model_selection::select_fitted_candidate_k`. Operators still cannot request +that selection as a digest-bound analysis-run output. Recovery of a single +fixed-`K` topic-lineage artifact is a different profile. Full Bayesian +sampling, GPU, method effects, and topic birth/split/merge remain later +GAP-004 work and are not this slice. + +An LLM vote must not define the numerical optimum. Lexical TF-IDF/BM25, +stopword deletion, and LLM labels remain forbidden inferential coordinates. + +## Decision + +Add the `fitted_candidate_k_v1` analysis-run output profile to +`analysis_engine`. The executor: + +- consumes an already-validated `ReferenceTopicInput` plus + `FittedCandidateKConfig`; +- requires the request snapshot and knowledge cutoff to match the offered + input construction; +- invokes `select_fitted_candidate_k` without reimplementing Schwarz scoring + or the reference fit; +- refuses lexical methods and refuses LLM-vote-only authority; +- emits a canonical SHA-256-digested `tepp.fitted_candidate_k.v1` artifact + with selected `K`, candidate count, evidence count, and inference status + `fitted_schwarz_candidate_k_not_bayesian_sampler`; +- does not invent a Bayesian sampler, persist rows, select GPU backends, or + emit topic-lineage edges. + +This is statistical candidate-`K` selection, not a posterior sampler and not +the `trsl_topic_lineage_v1` fixed-`K` lineage profile. + +## Alternatives considered + +1. Restore another Driver p.16 standardised matrix — rejected because those + recoveries are already a live micro-PR family and do not bind candidate-`K` + selection to an analysis run. +2. Bind GAP-013 interpreter/verifier — rejected because that is a different + orchestrator/LLM port, not model-selection authority. +3. Put candidate-`K` selection into `tepp_api` — rejected because transport + contracts and scientific composition would become one service boundary. +4. Bind the existing `model_selection` fitted candidate-`K` selector to + ADR 0022's analysis-run profile — accepted. + +## Consequences + +Operators can request cutoff-safe fitted candidate-`K` as a digest-bound +terminal result. The artifact does not claim Bayesian sampling, GPU parity, +or topic birth/split/merge. Snapshot/profile/cutoff mismatch, lexical +methods, failed fits, and LLM-only authority fail closed. + +## Verification + +The PR includes Rust unit and integration tests for true-`K` recovery on a +separated two-topic corpus, LLM-vote non-authority, lexical refusal, failed +fits, snapshot/profile/cutoff mismatch, and artifact tampering. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +python3 scripts/validate_documentation.py +``` + +## Rollback and supersession + +Rollback removes the `fitted_candidate_k_v1` profile. No persisted schema +migration is introduced. Supersede only with an ADR that keeps Schwarz +fitted selection distinct from LLM votes, lexical weights, and Bayesian +sampling. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..77be25466 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0049](0049-fitted-candidate-k-analysis-run.md) | Fitted candidate-`K` as an analysis-run profile | Accepted | active-PR | Complements ADR 0012/0022; Schwarz fitted selection, not a Bayesian sampler and not fixed-`K` topic lineage. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | diff --git a/docs/doctoring/fitted-candidate-k-analysis-run.md b/docs/doctoring/fitted-candidate-k-analysis-run.md new file mode 100644 index 000000000..58c2cfd5b --- /dev/null +++ b/docs/doctoring/fitted-candidate-k-analysis-run.md @@ -0,0 +1,16 @@ +# Fitted candidate-`K` analysis-run composition + +**Active slice:** ADR 0049 / `fitted_candidate_k_v1` +**Protected-main status:** not implemented-main + +`model_selection` already scores fitted candidate `K` with Schwarz's (1978) +penalty on the CPU `f64` TRSL-TM reference. This slice binds that selector to +a cutoff-safe analysis-run profile so an operator can request a digest-bound +terminal result. + +The executor refuses lexical methods and refuses LLM-vote-only authority. It +is not a Bayesian sampler, not GPU execution, not topic birth/split/merge, +and not the fixed-`K` topic-lineage profile. + +Exact-head Checks and two independent approvals are required before any +implemented-main claim.