Skip to content

Add generalizability theory G/D studies (crossed p x i, p x i x o) - #229

Merged
seonghobae merged 1 commit into
mainfrom
seonghobae-gtheory
Jul 25, 2026
Merged

seonghobae merged 1 commit into
mainfrom
seonghobae-gtheory

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Summary

Iteration 12 of the paper-implementation loop: Generalizability Theory G/D studies for the one-facet crossed p x i and two-facet crossed p x i x o designs (Huebner & Lucht, 2019, PARE 24(5), doi:10.7275/5065-gc10 — read in full).

  • mlsirm_core::gtheory: random-effects ANOVA variance components (EMS inversion), D-study relative/absolute error variances, generalizability coefficient E-rho^2, dependability Phi over proposed facet sizes.
  • PyO3 bindings gtheory_pi / gtheory_pio; thin Python wrappers in fast_mlsirm.gtheory with dataclass results.
  • Negative raw components: reported raw AND clamped to zero for the D study (documented clamped-ANOVA implementation policy, not paper-prescribed). Coefficients NaN when denominator <= 1e-12.

Source discipline

Source Status
Huebner & Lucht (2019) READ in full (ERIC EJ1222666), incl. worked examples + data appendix
Brennan (2001), Shavelson & Webb (1991) Cited by the paper for EMS derivations; NOT read — inversions hand-derived and numerically verified against the paper's Tables 3-6

Evidence

Check Result
Adversarial spec-verify (pre-impl) GO-WITH-FIXES; all fixes applied (clamp disclosure, NaN rule, layout spec)
Paper Tables 3-6 reproduction Full precision in Rust tests (gt_pi_paper_fixture, gt_pio_paper_fixture)
Independent fixtures RNG-pinned 7x5 p x i, 4x3 negative-variance, 5x4x3 p x i x o (natural negative sigma^2_pi anchor)
Executed mutation kills M1 divisor swap, M3 EMS sign flip, M5 clamp drop, M6 layout transpose — each applied, observed FAIL, reverted
500-rep Monte-Carlo recovery PASS (gt_pi_mc_recovery_500, #[ignore])
Adversarial impl-review (post-impl) 1 confirmed defect: ss_pio = ss_total - ... catastrophic cancellation (negative SS on 1e12-offset data) — fixed with direct three-way residual sum + regression test
Full crate suite 505 passed / 0 failed
pytest gtheory tests pass against rebuilt pyd

Stacked on #228 (seonghobae-livingston-lewis).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Generalizability Theory (G/D-study) support for fully crossed, balanced designs to fast-mlsirm, implementing the random-effects ANOVA variance components and D-study coefficients in Rust with PyO3 + Python wrappers and paper/fixture-backed tests.

Changes:

  • Implement mlsirm_core::gtheory for crossed p x i and p x i x o designs, including clamped-ANOVA D-study outputs and NaN coefficient rules.
  • Expose gtheory_pi / gtheory_pio through the PyO3 core module and add thin Python API wrappers + exports.
  • Add Rust unit tests reproducing Huebner & Lucht (2019) tables and Python paper-feature tests validating the public API.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
crates/mlsirm-core/src/lib.rs Exposes the new gtheory module from the Rust core crate.
crates/mlsirm-core/src/gtheory.rs Implements G-study ANOVA + D-study computations for p x i and p x i x o crossed designs.
crates/fast-mlsirm-py/src/lib.rs Adds PyO3 bindings for gtheory_pi / gtheory_pio and marshals results into Python dicts/lists.
python/fast_mlsirm/gtheory.py Adds Python-facing validation/marshaling wrappers returning dataclass-based results.
python/fast_mlsirm/__init__.py Re-exports the new GTheory functions/types from the package top-level.
tests/unit/gtheory_tests.rs Adds Rust fixtures reproducing paper tables + independent fixtures + regression for large-offset cancellation.
tests/test_paper_features.py Adds Python-level paper-table checks and input-rejection coverage for the new API.
CHANGELOG.md Documents the new generalizability-theory feature and its policies (clamp + NaN rules).
Comments suppressed due to low confidence (1)

crates/mlsirm-core/src/gtheory.rs:271

  • n_p * n_i * n_o and the derived intermediate products used for validation/allocations can overflow usize if a caller passes very large facet sizes (even before any meaningful computation). Use checked_mul_usize (already in this crate) to compute the expected buffer length and the interaction-buffer sizes so overflow is reported as a clean error instead of producing incorrect length checks or panics.
    validate(x, n_p * n_i * n_o, "gtheory_pio")?;

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/mlsirm-core/src/gtheory.rs Outdated
@seonghobae

Copy link
Copy Markdown
Contributor Author

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Copilot AI commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested ch...

Fixed in commits fix: use checked_mul_usize... and the follow-up message-clarification commit.

  • Line 188 (gtheory_pi): replaced n_p * n_i with crate::checked_mul_usize(n_p, n_i, "n_p * n_i overflows usize")?
  • Line 271 (gtheory_pio): replaced n_p * n_i * n_o with two chained checked_mul_usize calls — first n_p * n_i (message: "n_p * n_i overflows usize"), then the intermediate result * n_o (message: "n_p * n_i * n_o overflows usize") — matching the pattern used elsewhere in the crate.

All 7 gtheory tests continue to pass.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Comment thread crates/mlsirm-core/src/gtheory.rs
Comment thread python/fast_mlsirm/gtheory.py
Comment thread python/fast_mlsirm/gtheory.py
@seonghobae
seonghobae force-pushed the seonghobae-livingston-lewis branch from 9ff6ca2 to c0ec7b2 Compare July 25, 2026 09:30
@seonghobae
seonghobae force-pushed the seonghobae-gtheory branch from d206a8f to 6f9d42f Compare July 25, 2026 09:30
@seonghobae
seonghobae force-pushed the seonghobae-livingston-lewis branch from c0ec7b2 to 51fb72f Compare July 25, 2026 10:05
@seonghobae
seonghobae force-pushed the seonghobae-gtheory branch from 6f9d42f to f6160e9 Compare July 25, 2026 10:05
@seonghobae
seonghobae force-pushed the seonghobae-livingston-lewis branch from 51fb72f to 940a26c Compare July 25, 2026 10:14
@seonghobae
seonghobae force-pushed the seonghobae-gtheory branch from f6160e9 to d0152fc Compare July 25, 2026 10:15
@seonghobae
seonghobae force-pushed the seonghobae-livingston-lewis branch from 940a26c to 0abfd60 Compare July 25, 2026 11:59
Base automatically changed from seonghobae-livingston-lewis to main July 25, 2026 12:08
Rebased onto main after #228 squash merge.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@seonghobae
seonghobae force-pushed the seonghobae-gtheory branch from d0152fc to 7185242 Compare July 25, 2026 12:09
@seonghobae
seonghobae merged commit cd6c3a2 into main Jul 25, 2026
32 checks passed
@seonghobae
seonghobae deleted the seonghobae-gtheory branch July 25, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants