Skip to content

Add Cronbach alpha and Feldt (1965) exact-F confidence interval - #227

Merged
seonghobae merged 2 commits into
mainfrom
seonghobae-feldt-alpha-ci
Jul 25, 2026
Merged

seonghobae merged 2 commits into
mainfrom
seonghobae-feldt-alpha-ci

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Iteration 10 of the paper-implementation loop. Stacked on #226 (base seonghobae-tenberge-mu).

What

  • mlsirm_core::reliability::cronbach_alpha — raw-covariance Cronbach's alpha: p/(p-1) * (1 - tr(C)/sum(C)) (Cronbach, 1951).
  • mlsirm_core::reliability::feldt_alpha_ci — Feldt (1965) exact-F CI: pivot (1-alpha)/(1-alpha_hat) ~ F(n-1, (n-1)(p-1)); lower = 1-(1-a)*qF(1-d/2), upper = 1-(1-a)*qF(d/2); plus implied r_bar. In-crate F quantile (Lentz continued-fraction incomplete beta + bisection; endpoints guarded to 0/+inf).
  • PyO3 bindings + Python wrappers cronbach_alpha / feldt_alpha_ci / AlphaCiResult.

Sources actually verified

Claim Source Status
CI bound mapping, dfs, r_bar psych 2.6.5 alpha.ci (R/alpha.R) read line by line
Numeric bounds scipy.stats.f.ppf independent fixture replication
Feldt (1965) paper doi:10.1007/BF02289499 unobtainable — as cited in Revelle (2025)
Cronbach covariance form psych raw-alpha computation verified; 1951 paper not re-read

Evidence

  • Spec-verify (adversarial, before impl): GO-WITH-FIXES, all applied — pivot wording inverted; negative alpha admitted to CI; zero-variance guard documented as divergence; identity anchor demoted from EXACT to 1e-10; reviewer numeric cross-check (alpha=.8, n=30, p=6) matched R oracle to full precision.
  • Impl-review (adversarial, after impl): 2 confirmed defects, both fixed — (1) f_quantile returned a finite bisection cap at prob>=1 where qf/scipy give inf → endpoint guards + regression test; (2) wrapper docstrings were "..." + CONST expressions (__doc__ None) → literal docstring + __doc__ +=.
  • Fixtures pinned at 1e-9 from independent NumPy+scipy replication: A (n=30, p=6, unequal variances), B (n=24, p=5), negative-alpha p=2 (CI not clamped). Identity anchor: alpha on z-scored data == guttman_lambdas lambda3 at 1e-10 (crate-vs-crate).
  • Mutation kills (4/4): tail swap d/2<->1-d/2 → 3 fixtures FAIL; f_cdf z-orientation flip → 3 FAIL; df2=n-1 → A/B FAIL (p=2 passes by identity, documented); tr/sum inversion → 3 FAIL. Disclosed equivalent mutant: dropping the inc_beta symmetry branch survives (convergence acceleration only at these dfs; scipy literals anchor everything coarser).
  • MC (#[ignore], 500 reps): empirical 95% CI coverage of true tau-equivalent reliability within [0.925, 0.975] — passed.
  • Full crate suite: 492 passed. Python: 6 reliability tests passed.

Divergences from psych (documented in module docs)

Raw-data input only (no reverse-keying/check.keys); zero-variance items rejected; confidence level instead of p.val; hard errors instead of NA; bounds not clamped (negative alpha allowed, matching psych).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds coefficient alpha reliability support (Cronbach’s alpha) and Feldt (1965) exact-F confidence intervals to the mlsirm_core::reliability Rust crate, exposes them through the PyO3 core module, and provides Python-facing wrappers + tests to pin the numeric contract against independent NumPy/SciPy fixtures.

Changes:

  • Implement cronbach_alpha (raw-covariance form) and feldt_alpha_ci (Feldt exact-F CI) in crates/mlsirm-core.
  • Add PyO3 bindings and Python wrapper API (cronbach_alpha, feldt_alpha_ci, AlphaCiResult) and export them from fast_mlsirm.
  • Add Rust + Python fixtures/rejection tests and document the feature in CHANGELOG.md.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/reliability_tests.rs Adds Rust-side fixture tests, regression for extreme CI endpoint behavior, and rejection coverage.
tests/test_paper_features.py Adds Python-level fixture test and boundary rejection tests for the new reliability APIs.
python/fast_mlsirm/reliability.py Introduces Python wrappers + AlphaCiResult dataclass and appends reference text to docstrings.
python/fast_mlsirm/init.py Re-exports the new reliability functions/types at the package top-level.
crates/mlsirm-core/src/reliability.rs Implements Cronbach alpha, Feldt CI, and internal F CDF/quantile utilities (incomplete beta + bisection).
crates/mlsirm-core/src/fitstats.rs Makes ln_gamma pub(crate) for reuse in the new incomplete-beta implementation.
crates/fast-mlsirm-py/src/lib.rs Adds PyO3 functions cronbach_alpha / feldt_alpha_ci and wires them into the module init.
CHANGELOG.md Documents the newly added reliability metrics and key behavioral divergences from psych.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/mlsirm-core/src/reliability.rs
Base automatically changed from seonghobae-tenberge-mu to seonghobae-guttman-lambdas July 24, 2026 11:27
@seonghobae

Copy link
Copy Markdown
Contributor Author

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Copilot AI commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested ch...

Implemented the requested review-thread fix in commit 66048a2: f_quantile now bisects directly in z-space (inc_beta(...) in-loop) and only converts z -> x once at return.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

crates/mlsirm-core/src/reliability.rs:632

  • df2 is computed as ((n - 1) * (p - 1)) as f64; the (n-1)*(p-1) multiplication can overflow usize in release builds before the cast. Please use checked_mul (as elsewhere in the crate) so extreme inputs fail fast with a clear error instead of wrapping to an incorrect df2.
    let df1 = (n - 1) as f64;
    let df2 = ((n - 1) * (p - 1)) as f64;
    let delta = 1.0 - level;

Comment thread crates/mlsirm-core/src/reliability.rs
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from d324467 to 1d8bf72 Compare July 25, 2026 09:29
@seonghobae
seonghobae force-pushed the seonghobae-feldt-alpha-ci branch from 40a260a to afb84cf Compare July 25, 2026 09:29
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from 1d8bf72 to 05e23ce Compare July 25, 2026 10:04
@seonghobae
seonghobae force-pushed the seonghobae-feldt-alpha-ci branch from afb84cf to 8d59b0f Compare July 25, 2026 10:04
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from 05e23ce to 0fc4ced Compare July 25, 2026 10:14
@seonghobae
seonghobae force-pushed the seonghobae-feldt-alpha-ci branch from 8d59b0f to 64418ec Compare July 25, 2026 10:14
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch 2 times, most recently from 3c00995 to 7463911 Compare July 25, 2026 11:24
Base automatically changed from seonghobae-guttman-lambdas to main July 25, 2026 11:33
@seonghobae
seonghobae force-pushed the seonghobae-feldt-alpha-ci branch from 64418ec to 67b034e Compare July 25, 2026 11:36
Rebased onto main after #225 squash merge; preserve audit-safe fitstats checks and
only expose ln_gamma as pub(crate) for reliability CI routines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@seonghobae
seonghobae force-pushed the seonghobae-feldt-alpha-ci branch from 67b034e to 863f33e Compare July 25, 2026 11:44
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@seonghobae
seonghobae merged commit ede77b5 into main Jul 25, 2026
32 checks passed
@seonghobae
seonghobae deleted the seonghobae-feldt-alpha-ci branch July 25, 2026 11:54
seonghobae added a commit that referenced this pull request Jul 25, 2026
Rebased onto main after #227 squash merge.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
seonghobae added a commit that referenced this pull request Jul 25, 2026
…#228)

Rebased onto main after #227 squash merge.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants