feat: Haberman subscore added-value analysis (PRMSE)

[seonghobae]

Haberman subscore added-value analysis (PRMSE)

Iteration 4 of the autonomous paper-implementation loop (stacked on #220).

Implements the Haberman (2008) subscore added-value methodology as
mlsirm_core::subscores + fast_mlsirm.subscore_analysis: for each subscale
of a disjoint, exhaustive item partition, the PRMSEs of the three CTT
true-subscore estimators (observed subscore / observed total / augmented),
the augmented-regression weights, per-person estimator matrices,
disattenuated subscore correlations, and added-value decisions.

Sources (paper-first discipline)

Source	Status
Sinharay, S. (2010). When can subscores be expected to have added value? ETS RR-10-16 (Appendix reproduces the Haberman methodology)	READ (OA PDF from ERIC)
CRAN subscore package R source (subscore.s.r, subscore.x.R, subscore.sx.R, data.prep.R, CTTsub.R)	READ line-by-line (formula/disambiguation oracle)
Haberman, S. J. (2008). JEBS 33(2), 204-229	NOT read (paywalled) — cited only as cited in Sinharay (2010)
Wainer et al. (2001), Test scoring ch. 9	NOT read — cited only as cited in Sinharay (2010)

Formulas (all source-verified)

• Unbiased (n−1) moments throughout — R parity.
• C_T: observed covariances off-diagonal, alpha_k * V(s_k) on the diagonal;
  cov_k = row sum over the K subscore columns only (total column excluded).
• rho^2(s_t, x_t) = cov_k^2 / (V(s_kt) V(x_t)); PRMSE_x = rho^2 * alpha_x;
  tau = (sqrt(alpha_x) sqrt(rho^2) − r sqrt(alpha_k)) / (1 − r^2);
  PRMSE_sx = alpha_k + tau^2 (1 − r^2); beta = sqrt(alpha_k)(sqrt(alpha_k) − r tau);
  gamma = sqrt(alpha_k) tau sigma_s / sigma_x.
• Added value: PRMSE_s > PRMSE_x (Haberman); PRMSE_sx > max + 0.01
  (Sinharay 2010 convention, labeled; CRAN CTTsub's 0.1(1−max) rule
  documented but not implemented).

Documented divergences from CRAN

• Partition validated (CRAN data.prep silently allows non-union totals).
• Degenerate samples rejected (alpha ∉ (0,1], zero variance, subscore
  collinear with total, non-finite moments, PRMSE out of [0,1]) instead of
  propagating NaN.
• s_hat_x uses CRAN's nonnegative-root convention even when the signed
  appendix correlation would be negative (doc-noted; guards do not force
  cov(s_t, x_t) > 0).
• Complete data required in v1.

Adversarial process evidence

• Spec-verify (pre-implementation): REDUCED-SCOPE verdict with 14 required
  fixes, all applied before any code was written (e.g. duplicate-subscale
  "parallel forms anchor" was proven invalid — corr(s,x)=1 makes tau divide by
  zero — and became a rejection test; dominance downgraded to conditional).
• Impl-review (post-implementation): independent NumPy re-derivation from
  the R oracle matched the crate to all pinned literals; four confirmed
  findings, all fixed:
  1. PyO3 capacity-overflow panic via 2^63 * 2 wrap → checked_mul +
     dimension validation before any allocation.
  2. Zero-column arrays crossed into Rust → Python wrapper rejects
     n_persons < 3 or n_items < 4 pre-boundary.
  3. Module doc overclaimed that guards enforce the positive-root assumption →
     doc corrected.
  4. CHANGELOG overclaimed "rejection tests for all guards" → narrowed (the
     defensive PRMSE-range guard is not separately exercised).

Test evidence (mutation-resistant discipline)

Every assert reads crate output; fixture literals come from an independent
NumPy transcription of the R semantics (never calling the crate). The 10×4
fixture is deliberately asymmetric: added_value_sx = [false, true], and
subscale 0's margin (0.00754 < 0.01) kills any mutant dropping the +0.01 rule.

Check	Result
cargo test -p mlsirm-core subscores	11 passed (1 #[ignore])
#[ignore] 500-rep Monte Carlo (augmented MSE ≤ observed MSE vs true subscore)	passed
Full crate suite	458 passed
Python pytest -k subscore	2 passed
Mutation 1: drop m/(m−1) in alpha	7 tests FAIL (killed)
Mutation 2: rowsum includes total column	8 tests FAIL (killed)
Mutation 3: tau numerator sign flip	8 tests FAIL (killed)
Hostile-input probes (2^63 persons, sparse 10^9 group index, zero-column array)	clean ValueErrors, no panic/alloc

Known unkillable identity, disclosed in-test: PRMSE_sx ≥ PRMSE_s is
alpha + tau²(1−r²) ≥ alpha, true for any tau — the discriminating anchor is
the PRMSE_sx ≥ PRMSE_x half plus the pinned PRMSE_x literals.

LLM-as-a-Judge relevance

Decides whether per-domain judge subscores carry diagnostic information beyond
the overall score, or whether reporting them would be statistically misleading
— directly applicable to evaluation-item quality management.

[Copilot]

Pull request overview

Adds Haberman (2008, as cited in Sinharay 2010) subscore added-value analysis to the Rust core and exposes it through a Python API, with pinned reference-based tests to ensure numerical and guard-behavior parity with the documented CRAN subscore semantics.

Changes:

• Implemented mlsirm_core::subscores to compute PRMSE-based subscore estimators, augmented-regression weights, correlations, and added-value decisions with explicit degeneracy/partition guards.
• Added Python wrapper fast_mlsirm.subscore_analysis returning a typed SubscoreResult, including validation to keep hostile/degenerate shapes from crossing the Rust boundary.
• Added Rust + Python tests that pin outputs to independently-derived NumPy literals and exercise mandated rejection behavior; documented the feature in the changelog.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/unit/subscores_tests.rs	New Rust unit tests pinning subscores outputs to independent reference literals and exercising guard behavior (plus an ignored Monte Carlo check).
tests/test_paper_features.py	New Python integration tests validating wrapper output against reference literals and verifying degenerate-input rejection.
python/fast_mlsirm/subscores.py	New Python API (SubscoreResult, subscore_analysis) with input validation and marshaling into the Rust core.
python/fast_mlsirm/init.py	Exports subscore_analysis and SubscoreResult at the top-level package API.
crates/mlsirm-core/src/subscores.rs	New Rust implementation of Haberman/Sinharay subscore added-value analysis with documented sources/guards and per-person estimators.
crates/mlsirm-core/src/lib.rs	Wires the new subscores module into the core crate’s public module tree.
crates/fast-mlsirm-py/src/lib.rs	Adds the PyO3 binding function subscore_analysis and exposes it on the compiled core module.
CHANGELOG.md	Documents the new subscore added-value feature, outputs, and the testing/guarding approach.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.