Skip to content

Add Guttman lambda reliability coefficients (lambda1-6, split-half) - #225

Merged
seonghobae merged 1 commit into
mainfrom
seonghobae-guttman-lambdas
Jul 25, 2026
Merged

seonghobae merged 1 commit into
mainfrom
seonghobae-guttman-lambdas

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Implements Guttman (1945) lambda reliability coefficients (lambda1-lambda6 + split-half lambda4/beta/mean) as a new mlsirm_core::reliability module with PyO3 binding and thin Python wrapper (fast_mlsirm.guttman_lambdas -> GuttmanResult). Stacked on #224 (parallel analysis).

Sources actually read

  • CRAN psych 2.6.5 sources guttman.R, splitHalf.R, smc.R — read line by line (oracle).
  • Guttman (1945) itself is paywalled and was NOT read; attribution is "as cited in / as implemented by Revelle (2025)" throughout.

Contract (on the Pearson correlation matrix R, p items)

  • Vt = sum(R); lambda1 = 1 - p/Vt; lambda2 = (sum_off + sqrt(sumsq_off * p/(p-1)))/Vt; lambda3 = p/(p-1) * lambda1 (= alpha); lambda5 = lambda1 + 2*sqrt(max_j C_j)/Vt; lambda6 = (sum_off + sum smc_j)/Vt with smc_j = 1 - 1/[R^-1]_jj clamped to [0,1].
  • Split halves: A of size floor(p/2), rb = |4*S_AB/Vt|; exhaustive lexicographic enumeration when C(p, floor(p/2)) <= n_sample_splits (psych brute cutoff 15000), else LCG-sampled partial Fisher-Yates.

Declared divergences from psych (documented in module docs)

  1. No check.keys auto-reversal. 2. abs() split correlations in BOTH branches (psych's sampled branch is signed — its own inconsistency). 3. Plain symmetric inverse with hard error on singular R (psych silently uses Pinv). 4. Crate LCG, not R sample() (sampled results psych-inspired, not bit-identical). 5. Duplicate sampled subsets allowed. 6. No lambda5p/alpha.pc/glb/tenberge.

Evidence

Check Result
Fixture A (n=30, p=6, exhaustive C(6,3)=20) vs independent NumPy replication all 8 statistics pinned at 1e-9 PASS
Fixture B (n=24, p=5, odd p, C(5,2)=10) PASS
Fixture C (n=60, p=20, sampled budget=200) PASS
Rejections (n<3, p<3, NaN, zero variance, budget 0, singular R) PASS
500-rep tau-equivalent MC (#[ignore]): mean lambda3 vs analytic 0.7967 within 0.01 PASS
Mutation kills (hand-verified FAIL then restored): abs drop; lambda2 p/(p-1) drop; lambda5 sum-for-max; split stepper truncation 4/4 killed
cargo test -p mlsirm-core 482 passed
Python tests (fixture parity + rejections) 2 passed
Adversarial impl-review (stepper, Gauss-Jordan, overflow, wrapper scalars, MC target) NO CONFIRMED DEFECTS

Known identity limits are disclosed in the test header (lambda3 = p/(p-1)*lambda1 exact algebra; per-split Vt partition identity) with the discriminating anchors named.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Guttman (1945) lambda reliability coefficients (λ1–λ6) plus split-half summaries (λ4/beta/mean) to the Rust core (mlsirm_core::reliability), exposes them through the PyO3 extension, and provides a thin Python wrapper (fast_mlsirm.guttman_lambdas → GuttmanResult) with fixture-based tests.

Changes:

  • Implement mlsirm_core::reliability::guttman_lambdas (λ1–λ6 + split-half enumeration/sampling) and wire it into the Rust crate public API.
  • Add PyO3 binding and Python wrapper/dataclass, then export via fast_mlsirm.__init__.
  • Add Rust + Python tests that pin outputs against independent NumPy fixture literals; update CHANGELOG.md.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/reliability_tests.rs New Rust unit tests pinning λ1–λ6 and split-half summaries for exhaustive + sampled branches and rejection cases.
tests/test_paper_features.py Adds Python-level wrapper tests (fixture parity + degenerate-input rejections).
python/fast_mlsirm/reliability.py New Python wrapper + GuttmanResult dataclass; marshals to/from the compiled core.
python/fast_mlsirm/init.py Re-exports guttman_lambdas and GuttmanResult at package top-level.
crates/mlsirm-core/src/reliability.rs New Rust implementation of Guttman lambdas + split-half machinery, including SMC via explicit inverse.
crates/mlsirm-core/src/parallel.rs Makes correlation_matrix/LCG helpers pub(crate) for reuse; currently introduces a BOM at the start of the file (needs removal).
crates/mlsirm-core/src/lib.rs Exposes the new reliability module.
crates/fast-mlsirm-py/src/lib.rs Adds #[pyfunction] guttman_lambdas binding returning a Python dict of results.
CHANGELOG.md Documents the new reliability feature and its declared divergences from psych.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/mlsirm-core/src/parallel.rs Outdated
@seonghobae

Copy link
Copy Markdown
Contributor Author

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Copilot AI commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested ch...

Removed the UTF-8 BOM (EF BB BF) from the start of crates/mlsirm-core/src/parallel.rs. The file now begins directly with //!. All 486 Rust tests still pass.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread python/fast_mlsirm/reliability.py
Comment thread crates/mlsirm-core/src/reliability.rs
@seonghobae
seonghobae force-pushed the seonghobae-parallel-analysis branch from 603c583 to 1dbe283 Compare July 25, 2026 09:29
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from d324467 to 1d8bf72 Compare July 25, 2026 09:29
@seonghobae
seonghobae force-pushed the seonghobae-parallel-analysis branch from 1dbe283 to d4c5c73 Compare July 25, 2026 10:04
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from 1d8bf72 to 05e23ce Compare July 25, 2026 10:04
@seonghobae
seonghobae force-pushed the seonghobae-parallel-analysis branch from d4c5c73 to 636027c Compare July 25, 2026 10:14
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from 05e23ce to 0fc4ced Compare July 25, 2026 10:14
@seonghobae
seonghobae force-pushed the seonghobae-parallel-analysis branch from 636027c to f70f44d Compare July 25, 2026 10:45

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 0fc4ced020d425da18b752ba43f42d99d2cde38b.

  • Head SHA: 0fc4ced020d425da18b752ba43f42d99d2cde38b

  • Workflow run: 30155424351

  • Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (16 files)"]
  S1 --> I1["repository behavior"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["required checks"]
  Evidence --> S2["Test (5 files)"]
  S2 --> I2["regression suite"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["targeted test run"]
Loading

@opencode-agent

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: 0fc4ced020d425da18b752ba43f42d99d2cde38b
  • Workflow run: 30155424351
  • Workflow attempt: 1
  • Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 0fc4ced020d425da18b752ba43f42d99d2cde38b.

  • Head SHA: 0fc4ced020d425da18b752ba43f42d99d2cde38b

  • Workflow run: 30155424351

  • Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Changed file (16 files)"]
  S1 --> I1["repository behavior"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["required checks"]
  Evidence --> S2["Test (5 files)"]
  S2 --> I2["regression suite"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["targeted test run"]
Loading

Merge Conflict Guidance

  • Current merge state: DIRTY
  • Base branch: seonghobae-parallel-analysis
  • Head branch: seonghobae-guttman-lambdas
  • Fix direction: merge or rebase origin/seonghobae-parallel-analysis into seonghobae-guttman-lambdas, resolve conflict markers in the changed files, rerun the focused checks, then push the same branch.
  • Repair commands:
gh pr checkout 225 --repo ContextualWisdomLab/fast-mlsirm
git fetch origin seonghobae-parallel-analysis
git merge --no-ff origin/seonghobae-parallel-analysis  # or: git rebase origin/seonghobae-parallel-analysis
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:seonghobae-guttman-lambdas
# rebase path only: git push --force-with-lease origin HEAD:seonghobae-guttman-lambdas

Base automatically changed from seonghobae-parallel-analysis to main July 25, 2026 11:11
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from 0fc4ced to 3c00995 Compare July 25, 2026 11:17
Rebased onto main after #224 squash merge; include shared parallel helpers required by reliability.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@seonghobae
seonghobae force-pushed the seonghobae-guttman-lambdas branch from 3c00995 to 7463911 Compare July 25, 2026 11:24
@seonghobae
seonghobae dismissed opencode-agent[bot]’s stale review July 25, 2026 11:32

Superseded by latest rebased commit and passing required checks.

@seonghobae
seonghobae merged commit 8be7ccb into main Jul 25, 2026
32 checks passed
@seonghobae
seonghobae deleted the seonghobae-guttman-lambdas branch July 25, 2026 11:33
seonghobae added a commit that referenced this pull request Jul 25, 2026
Rebased onto main after #225 squash merge.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
seonghobae added a commit that referenced this pull request Jul 25, 2026
Rebased onto main after #225 squash merge; preserve audit-safe fitstats checks and
only expose ln_gamma as pub(crate) for reliability CI routines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
seonghobae added a commit that referenced this pull request Jul 25, 2026
* feat: Cronbach alpha and Feldt (1965) exact-F confidence interval

Rebased onto main after #225 squash merge; preserve audit-safe fitstats checks and
only expose ln_gamma as pub(crate) for reliability CI routines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(ksirt): guard ksirt_occ against zero-item panic

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants