diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ae818e94..ccefb5a1d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -428,6 +428,11 @@ tau update, and the covariate update while preserving the Rust production backend and public model contracts. +#### S-X2 and person-fit Rust ownership fail-closed + +- Public `s_x2()` and `person_fit()` require the compiled Rust core entrypoints and no longer fall back to Python/NumPy numerical implementations when the core or symbols are missing. +- `s_x2()` always dispatches trait `prior_mean` through the native S-X² entrypoint instead of selecting the Python reference path whenever a prior is supplied. + #### Parallel-analysis input and workspace bounds - `parallel_analysis()` now rejects booleans, floats, strings, and caller-defined integer-conversion hooks for integer controls instead of silently coercing them before Rust dispatch. @@ -535,6 +540,24 @@ - Cap raw LLM-judge response JSON nesting at 32 levels before `json.loads`, failing closed with `JudgeFormatError` so hostile recursive objects cannot expand into parser resource exhaustion. - Keep valid shallow judge payloads accepted with the existing criterion/score contracts. +#### Fit-statistics infit/outfit and M2 fail closed + +- Public `infit_outfit()` and ordinary `m2()` fail closed when the compiled Rust + core or required entrypoints are missing, completing the residual ownership + gaps from issue #627 after S-X² and person-fit hardening. + +#### Documentation coverage vocabulary and shipped-capability matrix + +- Align architecture documentation contracts with the protected-main maturity + vocabulary and mark parallel-analysis control bounds and essay-report native + dark-mode accents as ancestral after their integration. + +#### Observed-information work-budget preflight + +- Public observed-information finite-difference assembly preflights dense + workspace ceilings before stencil evaluation so hostile dimensions cannot + request unbounded Hessian work. + #### Diagnostics-report focus and contrast preservation - Revealed the visually hidden diagnostics-report skip link for every actual `:focus` state while retaining the explicit `:focus-visible` treatment and strong outline. diff --git a/docs/changelog.d/776-fitstats-infit-m2-failclosed.md b/docs/changelog.d/776-fitstats-infit-m2-failclosed.md new file mode 100644 index 000000000..70efe38ab --- /dev/null +++ b/docs/changelog.d/776-fitstats-infit-m2-failclosed.md @@ -0,0 +1,7 @@ +# Fit-statistics infit/outfit and M2 fail closed + +## Fixed + +- Public `infit_outfit()` and ordinary `m2()` fail closed when the compiled Rust + core or required entrypoints are missing, completing the residual ownership + gaps from issue #627 after S-X² and person-fit hardening. diff --git a/docs/changelog.d/777-docs-coverage-vocabulary-sync.md b/docs/changelog.d/777-docs-coverage-vocabulary-sync.md new file mode 100644 index 000000000..c2aa1fa61 --- /dev/null +++ b/docs/changelog.d/777-docs-coverage-vocabulary-sync.md @@ -0,0 +1,7 @@ +# Documentation coverage vocabulary and shipped-capability matrix + +## Fixed + +- Align architecture documentation contracts with the protected-main maturity + vocabulary and mark parallel-analysis control bounds and essay-report native + dark-mode accents as ancestral after their integration. diff --git a/docs/changelog.d/781-observed-information-work-budget.md b/docs/changelog.d/781-observed-information-work-budget.md new file mode 100644 index 000000000..921c7b090 --- /dev/null +++ b/docs/changelog.d/781-observed-information-work-budget.md @@ -0,0 +1,7 @@ +# Observed-information work-budget preflight + +## Fixed + +- Public observed-information finite-difference assembly preflights dense + workspace ceilings before stencil evaluation so hostile dimensions cannot + request unbounded Hessian work. diff --git a/docs/documentation_coverage.md b/docs/documentation_coverage.md index d99dbeab1..be0f094c4 100644 --- a/docs/documentation_coverage.md +++ b/docs/documentation_coverage.md @@ -82,10 +82,10 @@ The table below records product truth, not documentation-file presence. “Imple | Fixed-anchor parameter linking arithmetic | IMPLEMENTED_ON_PROTECTED_MAIN | protected main owns scale/shift estimation and theta/alpha/b transformation in Rust/PyO3 | | Observed-information Hessian and second-order diagnostics | IMPLEMENTED_ON_PROTECTED_MAIN | protected main owns finite-difference coefficients/symmetric Hessian assembly and eigenvalue/positive-definiteness diagnostics in Rust/PyO3; Python only evaluates objective samples and transports results | | JMLE Adam/L-BFGS optimizer arithmetic | IMPLEMENTED_ON_PROTECTED_MAIN | PR #760 is ancestral to current protected main; `backend="rust"` delegates Adam/L-BFGS/combined optimizer control to compiled Rust while recovery evidence remains governed separately by issue #626 | -| Parallel-analysis public control/resource hardening | IMPLEMENTED_ON_ACTIVE_PR | current fail-first/implementation PR owns strict integer/control and bounded-workspace hardening; it remains non-shipped until exact-head integration | +| Parallel-analysis public control/resource hardening | IMPLEMENTED_ON_PROTECTED_MAIN | strict integer/control validation and bounded random-benchmark workspace ceilings are ancestral to protected main | | Hourly review-repair caller | IMPLEMENTED_ON_PROTECTED_MAIN / PARTIAL | PR #763 integrated the product-side bounded caller; operational scheduler/control-plane acceptance remains external evidence rather than a library capability | | LLM-judge raw JSON depth hardening | IMPLEMENTED_ON_PROTECTED_MAIN | PR #764 is ancestral to current protected main and bounds recursive JSON nesting before parser materialization | -| Essay-report native dark-mode status accents | IMPLEMENTED_ON_ACTIVE_PR | current accessibility PR owns the CSS-variable/media-query change; do not treat it as protected-main until integration | +| Essay-report native dark-mode status accents | IMPLEMENTED_ON_PROTECTED_MAIN | CSS-variable and prefers-color-scheme dark-mode status accents are ancestral to protected main | | Canonical PyO3/public-export governance | ACCEPTED_ARCHITECTURE / PARTIAL | ADR-0011 governs convergence; feature-by-feature hardening continues | | Purpose-limited sensitive-data handling | IMPLEMENTED_ON_PROTECTED_MAIN / DOWNSTREAM | reusable contracts prefer purpose limitation/minimization/separated identities; hosted authorization/retention execution remains downstream | | LLM orchestration/model credentials | IMPLEMENTED_ON_PROTECTED_MAIN | provider execution and independent reviewer identity/credential boundaries are governed; provider calls remain outside psychometric numerical core | @@ -95,9 +95,8 @@ The table below records product truth, not documentation-file presence. “Imple At this review, material open work includes: -- strict/bounded public controls and Rust allocation preflight for parallel analysis; -- the documentation-fitness refresh itself, which may describe current protected truth but is not authoritative until merged; and -- native dark-mode report status accents. +- Rust allocation preflight for parallel analysis, if it remains separate; +- the documentation-fitness refresh itself, which may describe current protected truth but is not authoritative until merged. These remain active-PR evidence, not protected-main capability. Their source heads, checks, reviews, writer leases, and mergeability are operational evidence and must be re-fetched rather than copied into timeless architecture prose. diff --git a/python/fast_mlsirm/fitstats.py b/python/fast_mlsirm/fitstats.py index ffc05c88b..ae07eb9f9 100644 --- a/python/fast_mlsirm/fitstats.py +++ b/python/fast_mlsirm/fitstats.py @@ -635,61 +635,39 @@ def infit_outfit( mask: np.ndarray | None = None, eps_distance: float = 1e-8, ) -> dict[str, np.ndarray]: - """Per-item infit/outfit mean squares at the EAP estimates.""" + """Per-item infit/outfit mean squares at the EAP estimates. + + Production numerical ownership is the compiled Rust core + (``infit_outfit_stat``). Missing or incomplete cores fail closed. + """ model = model.upper() - free_alpha = model not in {"MLSRM", "ULSRM"} - uses_space = model != "MIRT" y, observed, d_of_i = _prepare_dichotomous_diagnostic_inputs( responses, factor_id, mask ) core = _core_module() - if core is not None and hasattr(core, "infit_outfit_stat"): - n_persons = y.shape[0] - n_dims = int(d_of_i.max()) + 1 - bank = _bank_args(params, d_of_i, model, n_dims, eps_distance) - res = core.infit_outfit_stat( - y.ravel(), - observed.ravel(), - int(n_persons), - bank["alpha"], - bank["b"], - bank["zeta"], - bank["tau"], - bank["factor_id"], - bank["model"], - bank["n_dims"], - bank["latent_dim"], - bank["eps_distance"], - np.asarray(params.theta, dtype=np.float64).ravel(), - np.asarray(params.xi, dtype=np.float64).ravel(), - ) - return {"infit": np.asarray(res["infit"]), "outfit": np.asarray(res["outfit"])} - a = np.exp(params.alpha) if free_alpha else np.ones(len(params.b)) - eta = a[None, :] * np.asarray(params.theta)[:, d_of_i] + params.b[None, :] - if uses_space: - # Optimized distance computation: replace O(N*J*D) 3D broadcast with O(N*J) 2D dot product - xi = np.asarray(params.xi) - zeta = np.asarray(params.zeta) - x_sq = np.einsum("ij,ij->i", xi, xi) - z_sq = np.einsum("ij,ij->i", zeta, zeta) - dist_sq = x_sq[:, None] + z_sq[None, :] - 2 * np.dot(xi, zeta.T) - dist = np.sqrt(eps_distance + np.maximum(dist_sq, 0.0)) - eta = eta - math.exp(params.tau) * dist - p = np.clip(1.0 / (1.0 + np.exp(-np.clip(eta, -700, 700))), 1e-12, 1 - 1e-12) - v = p * (1.0 - p) - resid2 = np.subtract(y, p) - np.square(resid2, out=resid2) - np.multiply(resid2, observed, out=resid2) - n_obs = np.maximum(observed.sum(axis=0), 1) - - # Preserve the masked squared-residual numerator, then reuse its owned - # float64 buffer for the outfit division without a numeric mask copy. - resid2_sum = resid2.sum(axis=0) - infit_denominator = np.sum(v, axis=0, where=observed) - np.divide(resid2, v, out=resid2) - outfit = resid2.sum(axis=0) / n_obs - infit = resid2_sum / np.maximum(infit_denominator, 1e-12) - return {"infit": infit, "outfit": outfit} + if core is None or not hasattr(core, "infit_outfit_stat"): + raise RuntimeError("fit statistics require the compiled Rust core") + n_persons = y.shape[0] + n_dims = int(d_of_i.max()) + 1 + bank = _bank_args(params, d_of_i, model, n_dims, eps_distance) + res = core.infit_outfit_stat( + y.ravel(), + observed.ravel(), + int(n_persons), + bank["alpha"], + bank["b"], + bank["zeta"], + bank["tau"], + bank["factor_id"], + bank["model"], + bank["n_dims"], + bank["latent_dim"], + bank["eps_distance"], + np.asarray(params.theta, dtype=np.float64).ravel(), + np.asarray(params.xi, dtype=np.float64).ravel(), + ) + return {"infit": np.asarray(res["infit"]), "outfit": np.asarray(res["outfit"])} + # -------------------------------------------------------------------------- @@ -1851,56 +1829,44 @@ def m2( ) core = _core_module() - if core is not None and hasattr(core, "m2_stat"): - bank = _bank_args(params, d_of_i, model, n_dims, eps_distance) - res = core.m2_stat( - np.where(observed0, y0, 0.0).ravel(), - observed0.ravel(), - int(y0.shape[0]), - bank["alpha"], - bank["b"], - bank["zeta"], - bank["tau"], - bank["factor_id"], - bank["model"], - bank["n_dims"], - bank["latent_dim"], - bank["eps_distance"], - prior_mean, - prior_sd, - q_theta=int(q_theta), - xi_rule="gh", - q_xi=int(q_xi), - ) - result = M2Result( - m2=float(res["m2"]), - df=float(res["df"]), - p_value=float(res["p_value"]), - rmsea2=float(res["rmsea2"]), - rmsea2_ci_lower=float(res["rmsea2_ci_lower"]), - rmsea2_ci_upper=float(res["rmsea2_ci_upper"]), - srmsr=float(res["srmsr"]), - null_m2=float(res["null_m2"]), - null_df=float(res["null_df"]), - cfi=float(res["cfi"]), - tli=float(res["tli"]), - n_moments=int(res["n_moments"]), - n_parameters=int(res["n_parameters"]), - n_complete=int(res["n_complete"]), - ) - else: - result = _m2_numpy( - y0, - observed0, - d_of_i, - params, - model, - q_theta, - q_xi, - eps_distance, - prior_mean, - prior_sd, - ) + if core is None or not hasattr(core, "m2_stat"): + raise RuntimeError("fit statistics require the compiled Rust core") + bank = _bank_args(params, d_of_i, model, n_dims, eps_distance) + res = core.m2_stat( + np.where(observed0, y0, 0.0).ravel(), + observed0.ravel(), + int(y0.shape[0]), + bank["alpha"], + bank["b"], + bank["zeta"], + bank["tau"], + bank["factor_id"], + bank["model"], + bank["n_dims"], + bank["latent_dim"], + bank["eps_distance"], + prior_mean, + prior_sd, + q_theta=int(q_theta), + xi_rule="gh", + q_xi=int(q_xi), + ) + result = M2Result( + m2=float(res["m2"]), + df=float(res["df"]), + p_value=float(res["p_value"]), + rmsea2=float(res["rmsea2"]), + rmsea2_ci_lower=float(res["rmsea2_ci_lower"]), + rmsea2_ci_upper=float(res["rmsea2_ci_upper"]), + srmsr=float(res["srmsr"]), + null_m2=float(res["null_m2"]), + null_df=float(res["null_df"]), + cfi=float(res["cfi"]), + tli=float(res["tli"]), + n_moments=int(res["n_moments"]), + n_parameters=int(res["n_parameters"]), + n_complete=int(res["n_complete"]), + ) if estimator == "mmle": return result result.estimator = estimator diff --git a/python/fast_mlsirm/inference.py b/python/fast_mlsirm/inference.py index a3a859f2d..eb182fc6e 100644 --- a/python/fast_mlsirm/inference.py +++ b/python/fast_mlsirm/inference.py @@ -7,6 +7,48 @@ from .objective import neg_loglik_and_grad from .types import MLSIRMParams +# Package-owned support ceilings for the O(n^2) finite-difference observed +# information path. These are implementation safety limits, not psychometric +# recommendations or universal hardware-capacity claims. +_MAX_OBSERVED_INFORMATION_OBJECTIVE_CALLS = 250_001 +_MAX_OBSERVED_INFORMATION_WORKSPACE_BYTES = 128 * 1024 * 1024 + + +def _observed_information_work(n: int) -> tuple[int, int]: + """Return exact objective-call and fixed-width workspace requirements. + + The central-difference stencil evaluates the objective once at the base, + twice per diagonal, and four times per off-diagonal pair, for exactly + ``1 + 2*n**2`` calls. Fixed-width workspace accounts conservatively for the + finite-difference value arrays, one reusable perturbation vector, and the + dense Rust-owned result while those inputs remain live. + """ + n = int(n) + if n < 0: + raise ValueError("observed-information parameter count must be non-negative") + objective_calls = 1 + 2 * n * n + float64_bytes = np.dtype(np.float64).itemsize + workspace_values = 3 * n * n + n + workspace_bytes = workspace_values * float64_bytes + return objective_calls, workspace_bytes + + +def _preflight_observed_information(n: int) -> None: + """Reject unsupported finite-difference work before objective evaluation.""" + objective_calls, workspace_bytes = _observed_information_work(n) + if objective_calls > _MAX_OBSERVED_INFORMATION_OBJECTIVE_CALLS: + raise ValueError( + "observed_information objective-call budget exceeded: " + f"requires {objective_calls} calls, at most " + f"{_MAX_OBSERVED_INFORMATION_OBJECTIVE_CALLS} are supported" + ) + if workspace_bytes > _MAX_OBSERVED_INFORMATION_WORKSPACE_BYTES: + raise ValueError( + "observed_information workspace budget exceeded: " + f"requires {workspace_bytes} bytes, limit is " + f"{_MAX_OBSERVED_INFORMATION_WORKSPACE_BYTES}" + ) + def observed_information( responses: np.ndarray, @@ -23,6 +65,10 @@ def observed_information( The default Rust device is CPU so finite-difference curvature uses the f64 path even when model fitting defaults to ``rust_device="auto"`` on GPU hosts. Pass ``device=None`` to honor ``config.rust_device`` instead. + + The dense finite-difference path is preflighted against package-owned + objective-call and fixed-width workspace budgets before the first objective + evaluation. Exceeding either support ceiling fails closed with ``ValueError``. """ config = config or FitConfig() model = config.normalized_model() @@ -31,6 +77,9 @@ def observed_information( if not np.isfinite(step) or step <= 0: raise ValueError("step must be > 0 and finite") + n = x0.size + _preflight_observed_information(n) + def objective(x: np.ndarray) -> float: """Return the penalized negative log-likelihood at packed parameter vector ``x``.""" value, _, _ = neg_loglik_and_grad( @@ -46,19 +95,12 @@ def objective(x: np.ndarray) -> float: raise ValueError("objective must be finite for Hessian calculation") return float(value) - n = x0.size - MAX_HESSIAN_DIM = 5_000 - if n > MAX_HESSIAN_DIM: - raise ValueError( - f"observed_information supports at most {MAX_HESSIAN_DIM} parameters (got {n}); " - "the dense finite-difference Hessian is O(n^2) memory and O(n^2) objective calls" - ) base = objective(x0) - eye = np.eye(n, dtype=np.float64) h = float(step) # Python evaluates the scalar objective at FD offsets; Rust owns the - # finite-difference coefficients and symmetrised matrix assembly. + # finite-difference coefficients and symmetrised matrix assembly. One + # reusable trial vector replaces the former dense n x n identity matrix. diag_plus = np.empty(n, dtype=np.float64) diag_minus = np.empty(n, dtype=np.float64) off_n = n * (n - 1) // 2 @@ -66,15 +108,29 @@ def objective(x: np.ndarray) -> float: off_pm = np.empty(off_n, dtype=np.float64) off_mp = np.empty(off_n, dtype=np.float64) off_mm = np.empty(off_n, dtype=np.float64) + trial = np.array(x0, dtype=np.float64, copy=True) k = 0 for i in range(n): - diag_plus[i] = objective(x0 + h * eye[i]) - diag_minus[i] = objective(x0 - h * eye[i]) + base_i = float(x0[i]) + trial[i] = base_i + h + diag_plus[i] = objective(trial) + trial[i] = base_i - h + diag_minus[i] = objective(trial) + trial[i] = base_i for j in range(i + 1, n): - off_pp[k] = objective(x0 + h * eye[i] + h * eye[j]) - off_pm[k] = objective(x0 + h * eye[i] - h * eye[j]) - off_mp[k] = objective(x0 - h * eye[i] + h * eye[j]) - off_mm[k] = objective(x0 - h * eye[i] - h * eye[j]) + base_j = float(x0[j]) + trial[i] = base_i + h + trial[j] = base_j + h + off_pp[k] = objective(trial) + trial[j] = base_j - h + off_pm[k] = objective(trial) + trial[i] = base_i - h + trial[j] = base_j + h + off_mp[k] = objective(trial) + trial[j] = base_j - h + off_mm[k] = objective(trial) + trial[i] = base_i + trial[j] = base_j k += 1 from . import _core as core diff --git a/tests/test_architecture_documentation_contract.py b/tests/test_architecture_documentation_contract.py index dbc281d3d..0e9b68320 100644 --- a/tests/test_architecture_documentation_contract.py +++ b/tests/test_architecture_documentation_contract.py @@ -155,10 +155,17 @@ def test_documentation_index_and_completeness_matrix_cover_security_and_gaps() - "traceability/requirements-matrix.md", ): assert target in index - for state in ("IMPLEMENTED", "ACTIVE PR", "PLANNED", "DOWNSTREAM"): + for state in ( + "IMPLEMENTED_ON_PROTECTED_MAIN", + "IMPLEMENTED_ON_ACTIVE_PR", + "PLANNED", + "DOWNSTREAM", + "PRESENT_CURRENT", + "OWNED_BY_ACTIVE_PR", + ): assert state in coverage assert "P0 documentation gaps" in coverage - assert "Canonical PyO3/public-export registry" in coverage + assert "Canonical PyO3/public-export governance" in coverage def test_standards_watch_separates_published_sources_from_watch_items() -> None: @@ -268,8 +275,14 @@ def test_documentation_contract_distinguishes_implemented_rotation_from_active_w assert "Accepted CPU baseline / planned GPU and broader recovery extensions" in trace assert "Proposed/partial / active PR" in trace - assert "IMPLEMENTED / PLANNED extensions" in coverage - assert "ACTIVE PR" in coverage - assert "PLANNED/partial" in coverage + rotation_row = next( + line + for line in coverage.splitlines() + if line.startswith("| Adaptive rotation criterion selection |") + ) + assert "IMPLEMENTED_ON_PROTECTED_MAIN" in rotation_row + assert "PARTIAL" in rotation_row + assert "IMPLEMENTED_ON_ACTIVE_PR" not in rotation_row + assert "PLANNED" not in rotation_row assert "Status: **Accepted**" in rotation_adr assert "GPU/additional-criterion/recovery expansion remains planned" in rotation_adr diff --git a/tests/test_cov_c_fitstats.py b/tests/test_cov_c_fitstats.py index 8423c2b3a..b9720879c 100644 --- a/tests/test_cov_c_fitstats.py +++ b/tests/test_cov_c_fitstats.py @@ -189,50 +189,28 @@ def test_icc_grid_and_factorized_guards(): def test_sx2_numpy_fallback_realistic(monkeypatch): + """Incomplete core must fail closed; public S-X² no longer uses NumPy fallback.""" rng = np.random.default_rng(7) - n_items, n_persons = 6, 400 - alpha = np.zeros(n_items) - b = np.array([-1.0, -0.5, 0.0, 0.5, 1.0, 800.0]) # last item is always correct - params = SimpleNamespace(alpha=alpha, b=b, zeta=np.zeros((n_items, 1)), tau=-30.0) + n_items, n_persons = 6, 40 + params = SimpleNamespace( + alpha=np.zeros(n_items), + b=np.linspace(-1.0, 1.0, n_items), + zeta=np.zeros((n_items, 1)), + tau=-30.0, + ) fid = np.zeros(n_items, dtype=np.int64) - theta = rng.standard_normal(n_persons) - eta = theta[:, None] + b[None, :] - y = (rng.random((n_persons, n_items)) < 1.0 / (1.0 + np.exp(-eta))).astype(float) - + y = (rng.random((n_persons, n_items)) < 0.5).astype(float) monkeypatch.setattr(fm, "_core_module", lambda: _CoreWithFitStatsOnly()) - out = fm.s_x2(y, fid, params, "MIRT", min_expected=1.0) - # ordinary items get finite chi-square with >=1 df; the always-correct item - # forms no usable score group. - assert np.isfinite(out.statistic[:5]).all() - assert (out.df[:5] >= 1).all() - assert out.n_score_groups[5] == 0 - assert math.isnan(out.df[5]) - - # discriminating items with a large minimum-expected count force the top - # scores to be merged into the last closed score group (trailing-merge path). - a = np.full(n_items, 1.6) - disc = SimpleNamespace( - alpha=np.log(a), b=np.linspace(-1.2, 1.2, n_items), - zeta=np.zeros((n_items, 1)), tau=-30.0, - ) - eta2 = a[None, :] * theta[:, None] + disc.b[None, :] - y2 = (rng.random((n_persons, n_items)) < 1.0 / (1.0 + np.exp(-eta2))).astype(float) - merged = fm.s_x2(y2, fid, disc, "MIRT", min_expected=6.0) - assert np.isfinite(merged.statistic).all() + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + fm.s_x2(y, fid, params, "MIRT", min_expected=1.0) def test_sx2_numpy_fallback_spatial_and_dim_floors(monkeypatch): - # dim 0 has 3 items, dim 1 a single item (skipped), dim 2 two items but one - # is entirely missing so it has no complete cases. - y, fid, params = _spatial_params([0, 0, 0, 1, 2, 2], 300, seed=4) - mask = np.ones_like(y, dtype=bool) - mask[:, 4] = False + """Spatial S-X² with incomplete core fails closed before Python numerics.""" + y, fid, params = _spatial_params([0, 0, 0, 1, 2, 2], 30, seed=4) monkeypatch.setattr(fm, "_core_module", lambda: _CoreWithFitStatsOnly()) - out = fm.s_x2(y, fid, params, "MLS2PLM", mask=mask, min_expected=1.0) - assert out.statistic.shape == (6,) - # single-item and no-complete-case dimensions yield no score groups. - assert out.n_score_groups[3] == 0 - assert out.n_score_groups[4] == 0 and out.n_score_groups[5] == 0 + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + fm.s_x2(y, fid, params, "MLS2PLM", min_expected=1.0) # --------------------------------------------------------------------------- @@ -241,30 +219,23 @@ def test_sx2_numpy_fallback_spatial_and_dim_floors(monkeypatch): def test_person_fit_numpy_fallback(monkeypatch): - y, fid, params = _mirt_params(6, 250, seed=8) + """Incomplete core must fail closed; public person-fit no longer uses NumPy fallback.""" + y, fid, params = _mirt_params(6, 25, seed=8) monkeypatch.setattr(fm, "_core_module", lambda: _CoreWithFitStatsOnly()) - pf = fm.person_fit( - y, fid, params, "MIRT", prior_mean=np.full((y.shape[0], 1), 0.2) - ) - assert pf.lz.shape == (250, 1) - assert np.isfinite(pf.lz).any() - assert pf.flagged.shape == (250,) - - ys, fids, sparams = _spatial_params([0, 0, 1, 1], 200, seed=9) - pf2 = fm.person_fit(ys, fids, sparams, "MLS2PLM") - assert pf2.lz_star.shape == (200, 2) + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + fm.person_fit( + y, fid, params, "MIRT", prior_mean=np.full((y.shape[0], 1), 0.2) + ) def test_infit_outfit_numpy_fallback(monkeypatch): - y, fid, params = _mirt_params(6, 250, seed=10) + """Incomplete core must fail closed for public infit/outfit.""" + y, fid, params = _mirt_params(6, 25, seed=10) monkeypatch.setattr(fm, "_core_module", lambda: _CoreWithFitStatsOnly()) - io = fm.infit_outfit(y, fid, params, "MIRT") - assert io["infit"].shape == (6,) and io["outfit"].shape == (6,) - assert np.all(np.isfinite(io["infit"])) + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + fm.infit_outfit(y, fid, params, "MIRT") + - ys, fids, sparams = _spatial_params([0, 0, 1, 1], 200, seed=11) - io2 = fm.infit_outfit(ys, fids, sparams, "MLS2PLM") - assert io2["outfit"].shape == (4,) # --------------------------------------------------------------------------- @@ -625,9 +596,8 @@ def test_m2_public_guards_and_numpy_dispatch(monkeypatch): fm.m2(y, fid, params, "MIRT", prior_sd=np.array([0.0])) monkeypatch.setattr(fm, "_core_module", lambda: _CoreWithFitStatsOnly()) - result = fm.m2(y, fid, params, "MIRT", q_theta=7, q_xi=7) - assert np.isfinite(result.m2) - assert result.estimator == "mmle" + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + fm.m2(y, fid, params, "MIRT", q_theta=7, q_xi=7) # --------------------------------------------------------------------------- diff --git a/tests/test_documentation_coverage_fitness.py b/tests/test_documentation_coverage_fitness.py index 5b943a2a7..545d1a499 100644 --- a/tests/test_documentation_coverage_fitness.py +++ b/tests/test_documentation_coverage_fitness.py @@ -68,17 +68,15 @@ def test_recently_integrated_capabilities_are_not_left_as_active_pr_only() -> No assert "IMPLEMENTED_ON_PROTECTED_MAIN" in _row(source, capability) -def test_parallel_hardening_is_not_promoted_to_protected_main() -> None: - """The active parallel-analysis hardening lane cannot be called shipped before merge.""" +def test_parallel_hardening_is_promoted_on_protected_main() -> None: + """Parallel-analysis control bounds are ancestral after #767.""" source = _matrix() row = _row(source, "Parallel-analysis public control/resource hardening") - assert "IMPLEMENTED_ON_ACTIVE_PR" in row - assert "IMPLEMENTED_ON_PROTECTED_MAIN" not in row + assert "IMPLEMENTED_ON_PROTECTED_MAIN" in row -def test_dark_mode_report_change_is_not_promoted_before_merge() -> None: - """The current accessibility PR remains active-PR evidence until integrated.""" +def test_dark_mode_report_change_is_promoted_on_protected_main() -> None: + """Native dark-mode status accents are ancestral after #765.""" source = _matrix() row = _row(source, "Essay-report native dark-mode status accents") - assert "IMPLEMENTED_ON_ACTIVE_PR" in row - assert "IMPLEMENTED_ON_PROTECTED_MAIN" not in row + assert "IMPLEMENTED_ON_PROTECTED_MAIN" in row diff --git a/tests/test_fitstats_infit_outfit_allocations.py b/tests/test_fitstats_infit_outfit_allocations.py index 5187bfdf7..1f59554e7 100644 --- a/tests/test_fitstats_infit_outfit_allocations.py +++ b/tests/test_fitstats_infit_outfit_allocations.py @@ -81,106 +81,39 @@ def _former_equations( } -def test_fallback_source_reuses_residual_buffer_without_numeric_mask_copy() -> None: - """The source pins the bounded operations rather than a flaky peak heuristic.""" - - source = inspect.getsource(fitstats.infit_outfit) - - assert "observed.astype" not in source - assert "resid2 / v" not in source - assert "v * observed" not in source - assert "resid2 = np.subtract(y, p)" in source - assert "np.square(resid2, out=resid2)" in source - assert "np.multiply(resid2, observed, out=resid2)" in source - assert "np.divide(resid2, v, out=resid2)" in source - assert "np.sum(v, axis=0, where=observed)" in source - - -def test_fallback_matches_former_equations_with_sparse_and_boundary_inputs( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """Buffer reuse preserves missingness and probability-clipping semantics.""" +def test_public_infit_outfit_fails_closed_without_rust_core(monkeypatch: pytest.MonkeyPatch) -> None: + """Missing cores raise before NumPy residual arithmetic.""" responses, observed, factor_id, params = _fixture() - epsilon = 1e-8 - expected = _former_equations( - responses, - observed, - factor_id, - params, - eps_distance=epsilon, - ) monkeypatch.setattr(fitstats, "_core_module", lambda: None) + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + fitstats.infit_outfit(responses, factor_id, params, "mlsirm", mask=observed) - result = fitstats.infit_outfit( - responses, - factor_id, - params, - "mlsirm", - mask=observed, - eps_distance=epsilon, - ) - - np.testing.assert_allclose( - result["infit"], expected["infit"], rtol=1e-13, atol=1e-13 - ) - np.testing.assert_allclose( - result["outfit"], expected["outfit"], rtol=1e-13, atol=1e-13 - ) - assert result["infit"][0] == 0.0 - assert result["outfit"][0] == 0.0 +def test_public_infit_outfit_source_is_rust_owned() -> None: + """Production path must call the compiled entrypoint.""" + source = inspect.getsource(fitstats.infit_outfit) + assert "infit_outfit_stat" in source + assert "fit statistics require the compiled Rust core" in source + assert "resid2 = np.subtract(y, p)" not in source -def test_fallback_uses_in_place_division_and_boolean_where_reduction( - monkeypatch: pytest.MonkeyPatch, -) -> None: - """The fallback creates neither a numeric mask copy nor quotient buffer.""" +def test_public_infit_outfit_dispatches_to_core(monkeypatch: pytest.MonkeyPatch) -> None: + """A compatible core owns the public numerical result.""" responses, observed, factor_id, params = _fixture() - monkeypatch.setattr(fitstats, "_core_module", lambda: None) - original_divide = np.divide - original_sum = np.sum - divide_calls: list[tuple[np.ndarray, np.ndarray, object]] = [] - sum_calls: list[dict[str, Any]] = [] - - def recording_divide( - numerator: np.ndarray, - denominator: np.ndarray, - *args: Any, - **kwargs: Any, - ) -> np.ndarray: - """Record the quotient output identity before delegating to NumPy.""" - - divide_calls.append((numerator, denominator, kwargs.get("out"))) - return original_divide(numerator, denominator, *args, **kwargs) - - def recording_sum(array: np.ndarray, *args: Any, **kwargs: Any) -> np.ndarray: - """Record the Boolean where reduction before delegating to NumPy.""" - - sum_calls.append( - { - "array": array, - "axis": kwargs.get("axis"), - "where": kwargs.get("where"), + n_items = responses.shape[1] + class RecordingCore: + def __init__(self): + self.calls = 0 + def infit_outfit_stat(self, *args, **kwargs): + self.calls += 1 + return { + "infit": np.full(n_items, 0.9, dtype=np.float64), + "outfit": np.full(n_items, 1.1, dtype=np.float64), } - ) - return original_sum(array, *args, **kwargs) - - monkeypatch.setattr(fitstats.np, "divide", recording_divide) - monkeypatch.setattr(fitstats.np, "sum", recording_sum) - - fitstats.infit_outfit( - responses, - factor_id, - params, - "mlsirm", - mask=observed, - ) + core = RecordingCore() + monkeypatch.setattr(fitstats, "_core_module", lambda: core) + result = fitstats.infit_outfit(responses, factor_id, params, "mlsirm", mask=observed) + assert core.calls == 1 + np.testing.assert_array_equal(result["infit"], np.full(n_items, 0.9)) - assert len(divide_calls) == 1 - numerator, denominator, output = divide_calls[0] - assert output is numerator - assert numerator.shape == denominator.shape == observed.shape - assert len(sum_calls) == 1 - assert sum_calls[0]["axis"] == 0 - assert sum_calls[0]["where"] is observed diff --git a/tests/test_fitstats_infit_outfit_rust_ownership_failclosed.py b/tests/test_fitstats_infit_outfit_rust_ownership_failclosed.py new file mode 100644 index 000000000..cbfea4e45 --- /dev/null +++ b/tests/test_fitstats_infit_outfit_rust_ownership_failclosed.py @@ -0,0 +1,26 @@ +"""Fail-first ownership contracts for public infit/outfit statistics.""" +from __future__ import annotations +from types import SimpleNamespace +import numpy as np +import pytest +import fast_mlsirm.fitstats as fitstats_module +from fast_mlsirm.fitstats import infit_outfit + +def _fixture(): + responses = np.array([[0.,0.,1.,1.],[0.,1.,0.,1.],[1.,0.,1.,0.],[1.,1.,0.,0.]], dtype=np.float64) + factor_id = np.zeros(4, dtype=np.int64) + params = SimpleNamespace(alpha=np.zeros(4), b=np.linspace(-0.5,0.5,4), zeta=np.zeros((4,1)), tau=-30., theta=np.linspace(-1,1,4)[:,None], xi=np.zeros((4,1))) + return responses, factor_id, params + +def test_infit_outfit_missing_core_fails_before_python_numerics(monkeypatch): + responses, factor_id, params = _fixture() + monkeypatch.setattr(fitstats_module, "_core_module", lambda: None) + monkeypatch.setattr(fitstats_module.np, "exp", lambda *a, **k: (_ for _ in ()).throw(AssertionError("python"))) + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + infit_outfit(responses, factor_id, params, "MIRT") + +def test_infit_outfit_incomplete_core_fails_before_python_numerics(monkeypatch): + responses, factor_id, params = _fixture() + monkeypatch.setattr(fitstats_module, "_core_module", lambda: SimpleNamespace()) + with pytest.raises(RuntimeError, match="fit statistics require the compiled Rust core"): + infit_outfit(responses, factor_id, params, "MIRT") diff --git a/tests/test_fitstats_m2_rust_ownership.py b/tests/test_fitstats_m2_rust_ownership.py new file mode 100644 index 000000000..05bb47f67 --- /dev/null +++ b/tests/test_fitstats_m2_rust_ownership.py @@ -0,0 +1,57 @@ +"""Fail-first ownership contract for public limited-information M2.""" + +from __future__ import annotations + +from types import SimpleNamespace + +import numpy as np +import pytest + +import fast_mlsirm.fitstats as fitstats + + +def _mirt_case() -> tuple[np.ndarray, np.ndarray, SimpleNamespace]: + """Return one ordinary identified MIRT fixture that reaches public M2.""" + rng = np.random.default_rng(62701) + n_persons = 300 + n_items = 5 + alpha = np.log(np.linspace(0.8, 1.4, n_items)) + b = np.linspace(-1.0, 1.0, n_items) + theta = rng.standard_normal((n_persons, 1)) + eta = np.exp(alpha)[None, :] * theta + b[None, :] + responses = ( + rng.random((n_persons, n_items)) < 1.0 / (1.0 + np.exp(-eta)) + ).astype(float) + params = SimpleNamespace( + alpha=alpha, + b=b, + zeta=np.zeros((n_items, 1)), + tau=-30.0, + theta=theta, + xi=np.zeros((n_persons, 1)), + ) + return responses, np.zeros(n_items, dtype=np.int64), params + + +def test_public_m2_requires_compiled_rust_core(monkeypatch: pytest.MonkeyPatch) -> None: + """Ordinary public M2 must reject missing Rust before NumPy reference dispatch.""" + responses, factor_id, params = _mirt_case() + monkeypatch.setattr(fitstats, "_core_module", lambda: None) + + def forbidden_numpy_reference(*_args, **_kwargs): + raise AssertionError("public M2 dispatched to the private NumPy reference") + + monkeypatch.setattr(fitstats, "_m2_numpy", forbidden_numpy_reference) + + with pytest.raises( + RuntimeError, + match="fit statistics require the compiled Rust core", + ): + fitstats.m2( + responses, + factor_id, + params, + "MIRT", + q_theta=7, + q_xi=7, + ) diff --git a/tests/test_information_matrix_robustness.py b/tests/test_information_matrix_robustness.py index c031a2dc2..33d401eb3 100644 --- a/tests/test_information_matrix_robustness.py +++ b/tests/test_information_matrix_robustness.py @@ -144,17 +144,17 @@ def test_observed_information_rejects_nonpositive_or_nonfinite_step(step): def test_observed_information_bounds_dense_hessian_dimension(): - """The dense finite-difference Hessian is capped to a safe parameter count. + """The dense finite-difference Hessian is capped by a pre-objective work budget. The observed information is an O(n^2)-memory, O(n^2)-objective-call dense matrix, so an unbounded parameter vector would exhaust memory/time. The routine must reject an over-large model before doing any work rather than attempt the allocation. """ - params = _mirt_params(n_persons=5200) # packs to > 5000 free parameters + params = _mirt_params(n_persons=5200) responses = np.zeros((5200, 2)) factors = np.zeros(2, dtype=int) - with pytest.raises(ValueError, match="at most 5000 parameters"): + with pytest.raises(ValueError, match="objective-call budget exceeded"): observed_information( responses, factors, params, config=FitConfig(model="MIRT"), step=1e-4 ) diff --git a/tests/test_observed_information_work_budget.py b/tests/test_observed_information_work_budget.py new file mode 100644 index 000000000..0d7646b3d --- /dev/null +++ b/tests/test_observed_information_work_budget.py @@ -0,0 +1,138 @@ +"""Resource budgets for public observed-information evaluation.""" + +from __future__ import annotations + +import numpy as np +import pytest + +import fast_mlsirm._core as core +import fast_mlsirm.inference as inference +from fast_mlsirm.config import FitConfig +from fast_mlsirm.types import MLSIRMParams + + +def _small_mirt_problem() -> tuple[np.ndarray, np.ndarray, MLSIRMParams]: + """Return a tiny MIRT problem with a non-empty packed parameter vector.""" + responses = np.array([[1.0], [0.0]], dtype=np.float64) + factor_id = np.array([0], dtype=np.int64) + params = MLSIRMParams( + theta=np.array([[0.25], [-0.25]], dtype=np.float64), + alpha=np.array([0.1], dtype=np.float64), + b=np.array([0.0], dtype=np.float64), + xi=np.zeros((2, 1), dtype=np.float64), + zeta=np.zeros((1, 1), dtype=np.float64), + tau=-2.0, + ) + return responses, factor_id, params + + +def _objective_must_not_run(*_args, **_kwargs): + """Fail if resource preflight lets the expensive objective start.""" + raise AssertionError("objective evaluated before observed-information preflight") + + +def _constant_objective(*_args, **_kwargs): + """Return a cheap finite objective so allocation behavior can be isolated.""" + return 0.0, None, None + + +def test_observed_information_work_estimate_matches_stencil_and_workspace() -> None: + """Resource accounting must match the governed finite-difference layout.""" + calls, workspace = inference._observed_information_work(4) + assert calls == 1 + 2 * 4**2 + assert workspace == (3 * 4**2 + 4) * np.dtype(np.float64).itemsize + assert inference._observed_information_work(0) == (1, 0) + + +def test_observed_information_work_rejects_negative_internal_dimension() -> None: + """Internal resource accounting rejects impossible negative dimensions.""" + with pytest.raises(ValueError, match="parameter count"): + inference._observed_information_work(-1) + + +def test_observed_information_rejects_objective_call_budget_before_evaluation( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The exact ``1 + 2*n**2`` call budget must fail closed before objective work.""" + responses, factor_id, params = _small_mirt_problem() + monkeypatch.setattr(inference, "_MAX_OBSERVED_INFORMATION_OBJECTIVE_CALLS", 1) + monkeypatch.setattr( + inference, + "_MAX_OBSERVED_INFORMATION_WORKSPACE_BYTES", + 2**63 - 1, + ) + monkeypatch.setattr(inference, "neg_loglik_and_grad", _objective_must_not_run) + + with pytest.raises(ValueError, match="objective-call budget"): + inference.observed_information( + responses, + factor_id, + params, + config=FitConfig(model="MIRT", backend="rust"), + backend="rust", + device="cpu", + ) + + +def test_observed_information_rejects_workspace_budget_before_evaluation( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Dense finite-difference workspace must be budgeted before objective work.""" + responses, factor_id, params = _small_mirt_problem() + monkeypatch.setattr( + inference, + "_MAX_OBSERVED_INFORMATION_OBJECTIVE_CALLS", + 2**63 - 1, + ) + monkeypatch.setattr(inference, "_MAX_OBSERVED_INFORMATION_WORKSPACE_BYTES", 1) + monkeypatch.setattr(inference, "neg_loglik_and_grad", _objective_must_not_run) + + with pytest.raises(ValueError, match="workspace budget"): + inference.observed_information( + responses, + factor_id, + params, + config=FitConfig(model="MIRT", backend="rust"), + backend="rust", + device="cpu", + ) + + +def test_observed_information_avoids_dense_identity_workspace( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Coordinate perturbations must not require a second dense ``n x n`` identity.""" + responses, factor_id, params = _small_mirt_problem() + monkeypatch.setattr( + inference, + "_MAX_OBSERVED_INFORMATION_OBJECTIVE_CALLS", + 2**63 - 1, + ) + monkeypatch.setattr( + inference, + "_MAX_OBSERVED_INFORMATION_WORKSPACE_BYTES", + 2**63 - 1, + ) + monkeypatch.setattr(inference, "neg_loglik_and_grad", _constant_objective) + + def forbidden_eye(*_args, **_kwargs): + raise AssertionError("observed_information materialized a dense identity matrix") + + monkeypatch.setattr(inference.np, "eye", forbidden_eye) + monkeypatch.setattr( + core, + "observed_information", + lambda n, *_args, **_kwargs: [0.0] * (int(n) * int(n)), + ) + + result = inference.observed_information( + responses, + factor_id, + params, + config=FitConfig(model="MIRT", backend="rust"), + backend="rust", + device="cpu", + ) + + assert result.shape[0] == result.shape[1] + assert np.all(result == 0.0)