Repository navigation
Add Livingston-Lewis (1995) classification accuracy and consistency - #228
Conversation
There was a problem hiding this comment.
Pull request overview
Adds Livingston–Lewis (1995) single-administration classification accuracy/consistency to the Rust core and exposes it through the PyO3 extension and Python API, with pinned-literal tests validating the numerical pipeline against an independent SciPy quadrature replication.
Changes:
- Implement
mlsirm_core::classification::livingston_lewis, including effective test length → beta moment fit → binomial observed-score integrals (accuracy/consistency/kappa). - Expose the new functionality via PyO3 (
fast_mlsirm._core) and a Python wrapper returning aLivingstonLewisResultdataclass. - Add Rust and Python test fixtures covering 2P fail-safe, 4P path, threshold rounding behavior, and trust-boundary input validation; update the changelog.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
crates/mlsirm-core/src/classification.rs |
Core Livingston–Lewis implementation, quadrature/integral machinery, and result struct. |
crates/mlsirm-core/src/reliability.rs |
Makes inc_beta pub(crate) so classification can reuse it for binomial tail computation. |
crates/fast-mlsirm-py/src/lib.rs |
Adds PyO3 binding that returns a dict of Livingston–Lewis outputs. |
python/fast_mlsirm/classification.py |
Adds LivingstonLewisResult dataclass + livingston_lewis() wrapper calling the compiled core. |
python/fast_mlsirm/__init__.py |
Re-exports livingston_lewis at the package top level. |
tests/unit/classification_tests.rs |
Adds Rust-side fixtures and invariants for Livingston–Lewis outputs. |
tests/test_paper_features.py |
Adds Python-side pinned-literal and rejection tests for the wrapper/core boundary. |
CHANGELOG.md |
Documents the newly added Livingston–Lewis feature and its documented divergences from the oracle. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@copilot Fix the code for all comments in this review thread. When a review comment includes a suggested change, apply the suggestion exactly. Do not make changes beyond what is described in the linked review thread. |
All four review items addressed in commit
|
40a260a to
afb84cf
Compare
9ff6ca2 to
c0ec7b2
Compare
afb84cf to
8d59b0f
Compare
c0ec7b2 to
51fb72f
Compare
8d59b0f to
64418ec
Compare
51fb72f to
940a26c
Compare
67b034e to
863f33e
Compare
Rebased onto main after #227 squash merge. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
940a26c to
0abfd60
Compare
Rebased onto main after #228 squash merge. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Implements Livingston & Lewis (1995) single-administration classification accuracy and consistency, as implemented in CRAN betafunctions 1.9.0
LL.CA(read line by line; the original article was not consulted directly). Stacked on #227.What
mlsirm_core::classification::livingston_lewis: effective test length -> true-score raw moments (factorial-moment identity, unrounded ETL scale) -> Hanson (1991, as cited in Haakstad, 2022) four-parameter beta moment fit with two-parameter [0,1] fail-safe -> binomial observed-score model withN = round_ties_even(ETL)-> accuracy cells (tp/fp/tf/ff), sensitivity/specificity, consistency cells, Cohen's kappa.fast_mlsirm.livingston_lewisreturning a dataclass.Documented divergences from the oracle
k = round_ties_even(N*c)threshold in BOTH accuracy and consistency blocks. betafunctions mixesround(accuracy) withfloor(consistency), making its consistency cells asymmetric (measured p.ij-p.ji = 0.0345 on a test fixture); herep_ij == p_jiby construction.Evidence
cargo test -p mlsirm-core#[ignore])ratioguard + documented semantics + regression testKnown limitation (disclosed in-test): at exactly-zero denominators IEEE 0/0 already yields NaN, so a dropped
ratioguard is not killed by the NaN test; the guard's value is for tiny nonzero denominators.