From f1076000df5e845cf76cf1e60680c4226d25202b Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 15:09:34 +0900 Subject: [PATCH 01/14] docs: make README product-first and license-aware --- README.md | 96 ++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 81 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index b2403d0..06390bb 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,97 @@ # nonnest2 -nonnest2 provides functionality for comparing non-nested models' fit and distinguishability, relying on theory from Vuong (1989). The authors acknowledge support from NSF grant SES-1061334. The contents of this package are those of the authors and do not reflect the views of the National Science Foundation. +**Evidence-based comparison of nested and non-nested statistical models in R.** -The package is intended to work automatically for models of many classes, including `lavaan`, `mirt`, and `glm`. It can also be applied to models of new, unseen classes, so long as the user provides functions to compute the model's casewise log-likelihoods and casewise first derivatives. A notable example is the [*merDeriv* package](https://github.com/nctingwang/merDeriv), which provides those functions for many models estimated via *lme4*. +`nonnest2` helps analysts answer two questions that ordinary fit indices do not answer by themselves: whether two fitted models are empirically distinguishable, and which model is better supported when they are distinguishable. The package implements tests based on Vuong (1989) and also provides confidence intervals for differences in AIC and BIC. +The package works with several established R model classes and exposes a small adapter boundary for additional classes. It is a statistical comparison library; it does not choose a substantive model for you, establish causal validity, or turn information criteria into proof of scientific truth. -# Example +## When to use it -Consider the two factor analysis models below, estimated in `lavaan`. The models are not nested due to `x4` and `x7`. +Use `nonnest2` when you have two fitted models for the same observations and need a casewise-likelihood comparison rather than a simple comparison of aggregate fit statistics. -```r -library("lavaan") +Typical jobs include: + +- testing whether two models are distinguishable from the observed data; +- comparing relative model fit using Vuong-style tests; +- obtaining interval estimates for differences in AIC and BIC; and +- extending the comparison machinery to another model class that can provide the required casewise contributions. + +Built-in `llcont()` methods currently cover model classes from ecosystems including `lavaan`, `mirt`, `OpenMx`, base/generalized linear models, and several other regression-model families. The exact exported API is `vuongtest()`, `icci()`, and `llcont()`. + +## Install from this source tree -m1 <- ' visual =~ x1 + x2 + x3 + x4 - textual =~ x4 + x5 + x6 - speed =~ x7 + x8 + x9 ' -fit1 <- cfa(m1, data=HolzingerSwineford1939) +The package metadata requires R 3.0.0 or later. From a checked-out revision: -m2 <- ' visual =~ x1 + x2 + x3 - textual =~ x4 + x5 + x6 + x7 - speed =~ x7 + x8 + x9 ' -fit2 <- cfa(m2, data=HolzingerSwineford1939) +```bash +R CMD INSTALL . ``` -We can use *nonnest2* to compare the models via Vuong tests and to obtain interval estimates of differences in the models' AIC and BIC values. +For development or release verification, use the repository's R package workflow rather than inferring quality from source presence alone: + +```bash +R CMD build . +R CMD check nonnest2_*.tar.gz +``` + +The repository also contains an `R-CMD-check` GitHub Actions workflow. Its current result should be verified on the exact revision you intend to use. + +## Quick start + +The example below compares two factor models that are not nested because the indicator assignments differ. ```r +library(lavaan) +library(nonnest2) + +m1 <- ' + visual =~ x1 + x2 + x3 + x4 + textual =~ x4 + x5 + x6 + speed =~ x7 + x8 + x9 +' +fit1 <- cfa(m1, data = HolzingerSwineford1939) + +m2 <- ' + visual =~ x1 + x2 + x3 + textual =~ x4 + x5 + x6 + x7 + speed =~ x7 + x8 + x9 +' +fit2 <- cfa(m2, data = HolzingerSwineford1939) + vuongtest(fit1, fit2) icci(fit1, fit2) ``` + +Interpret the distinguishability result before treating a relative-fit result as meaningful, and interpret both in the context of the models' assumptions and the scientific question. + +## Integration model + +`nonnest2` keeps the public comparison layer small. Model-specific integration happens through casewise likelihood/derivative contributions exposed by `llcont()` methods. New model classes should provide the statistical quantities expected by that adapter contract rather than embedding package-specific branching into `vuongtest()` itself. + +For the exact supported S3 methods, see [`NAMESPACE`](NAMESPACE). Package history is recorded in [`NEWS`](NEWS), and longer worked material lives under [`vignettes/`](vignettes/). + +## Statistical basis + +The package metadata cites: + +> Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses. *Econometrica, 57*(2), 307–333. https://doi.org/10.2307/1912557 + +The software implements statistical procedures based on that theory; using the software does not remove the need to check identification, estimator assumptions, data quality, model misspecification, and the substantive meaning of the compared models. + +## Project status and verification + +Current package metadata declares version `0.5-9` dated 2026-03-31. Treat that as source/package metadata, not by itself as evidence of a particular published artifact, deployment, benchmark, or certification. For any consequential analysis, bind results to the exact package revision and preserve the fitted-model inputs and software environment needed to reproduce the comparison. + +## Contributing + +Keep changes focused on the statistical contract and supported model adapters. New adapters should include realistic tests for casewise contributions and comparison behavior. Changes to numerical formulas, supported model semantics, or public return values should be documented and verified through the repository's ordinary R package checks. + +## License and commercial-use boundary + +The authoritative package metadata declares **`GPL-2 | GPL-3`** and identifies upstream/external authors and contributors. This repository therefore does **not** claim a new MIT or Apache-2.0 grant for the inherited package source. + +GPL licenses permit commercial use under their terms, but GPL-family source is not an approved default inbound component under ContextualWisdomLab's current commercial ecosystem policy. Accordingly, this repository should not be presented as approved for incorporation into a ContextualWisdomLab commercial product unless the exact provenance, copyright-holder rights, distribution model, and copyleft obligations are resolved through an explicitly approved repository-level path. + +See [`docs/commercial-license-boundary.md`](docs/commercial-license-boundary.md) for the evidence and closure conditions. Dependency licenses are separate from the license of `nonnest2` itself and do not relicense this package. + +The original NSF acknowledgement remains part of the package history: material in this package is partially based on work supported by NSF grant SES-1061334, and the package contents do not represent NSF views. From 61a98c4922491ad6c820f9f5f3eb366440626356 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 15:09:59 +0900 Subject: [PATCH 02/14] docs: record inherited GPL commercial boundary --- docs/commercial-license-boundary.md | 55 +++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 docs/commercial-license-boundary.md diff --git a/docs/commercial-license-boundary.md b/docs/commercial-license-boundary.md new file mode 100644 index 0000000..4a12b29 --- /dev/null +++ b/docs/commercial-license-boundary.md @@ -0,0 +1,55 @@ +# nonnest2 commercial-license boundary + +## Purpose + +This note records why the current `nonnest2` source cannot be silently converted to the normal ContextualWisdomLab Apache-2.0/MIT repository baseline, and what evidence would be required before the package could be treated as an approved commercial-ecosystem component. + +It is a provenance and engineering-control record, not legal advice and not a change to the package's existing license. + +## Current evidence + +Protected `master` package metadata currently states: + +- `Package: nonnest2`; +- `Version: 0.5-9`; +- `License: GPL-2 | GPL-3`; +- Edgar Merkle and Dongjun You are package authors, with additional contributors including Lennart Schneider, Mauricio Garnier-Villarreal, Seongho Bae, and Phil Chalmers; and +- the package URL is `https://github.com/qpsy/nonnest2`. + +The repository also contains historical and current ContextualWisdomLab maintenance commits. Those maintenance contributions do not establish ownership of every inherited copyright interest and therefore do not establish unilateral relicensing authority over the complete package. + +No root MIT/Apache license is added by this documentation lane. The R package `DESCRIPTION` license remains the authoritative source-license declaration unless and until an evidence-backed licensing change is approved by the relevant rights holders. + +## ContextualWisdomLab policy boundary + +GPL licenses permit commercial use under their terms. The blocker here is not a claim that GPL is noncommercial; it is that ContextualWisdomLab's current inbound policy does not accept GPL/LGPL/AGPL-family source as the normal incorporation baseline for its intended commercial distribution model. + +Therefore: + +- do not describe this repository as MIT-, Apache-2.0-, or otherwise permissively licensed; +- do not treat hosting or maintaining this repository as proof of relicensing authority; +- do not present the package as an approved dependency for incorporation into another ContextualWisdomLab commercial product under the normal permissive baseline; +- do not use dependency licenses to infer a different license for `nonnest2` source; and +- do not remove copyright, attribution, or source-license obligations in order to make the repository appear policy-clean. + +## Closure paths + +The commercial-policy blocker is resolved only by an evidence-backed repository-level outcome such as one of the following: + +1. **Rights-backed permissive relicensing.** Obtain authoritative provenance and copyright-holder/contributor rights sufficient to relicense all relevant source under a commercially compatible permissive license, then update package metadata, root licensing/NOTICE material, source headers where applicable, and release evidence together. +2. **Independent compatible replacement.** Replace GPL-covered implementation with independently authored, commercially compatible source without copying or deriving from GPL implementation details that would carry derivative obligations. Preserve the validated public statistical contract through black-box/reference testing and independent implementation evidence. +3. **Explicit approved GPL distribution model.** Adopt a repository-specific commercial distribution model that is demonstrably compatible with the exact GPL obligations and explicitly approved as an exception to the organization baseline. + +A README disclaimer, a new dependency, a process boundary, or the absence of a root `LICENSE` file is not sufficient closure. + +## Scientific continuity required by any replacement + +A licensing repair must not silently change the statistical product. At minimum, the surviving implementation must re-establish evidence for: + +- `vuongtest()` distinguishability and relative-fit behavior; +- `icci()` AIC/BIC difference interval behavior; +- `llcont()` casewise contribution semantics; +- supported adapter behavior for the model classes the repository continues to advertise; and +- edge cases, numerical tolerances, and failure behavior required by existing tests and documented examples. + +Release or incorporation claims should be bound to one exact source revision and ordinary package/security verification rather than inferred from this documentation record. From bc878190f5d6f8955187eb608d3f67b7791b1588 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 15:37:46 +0900 Subject: [PATCH 03/14] docs: add Ask DeepWiki badge --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 06390bb..82227d3 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # nonnest2 +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/nonnest2) + **Evidence-based comparison of nested and non-nested statistical models in R.** `nonnest2` helps analysts answer two questions that ordinary fit indices do not answer by themselves: whether two fitted models are empirically distinguishable, and which model is better supported when they are distinguishable. The package implements tests based on Vuong (1989) and also provides confidence intervals for differences in AIC and BIC. From 243eaa3755e8d59d5e20c632d768f86055f72bad Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 15:38:09 +0900 Subject: [PATCH 04/14] docs: add Pages-ready project landing --- docs/index.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..6b7da48 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,43 @@ +# nonnest2 + +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/nonnest2) + +`nonnest2` is an R package for evidence-based comparison of nested and non-nested statistical models. It implements Vuong-style distinguishability and relative-fit procedures and provides confidence intervals for differences in AIC and BIC. + +## Start here + +The task-first [README](../README.md) covers installation, a worked example, supported model adapters, verification, statistical interpretation, and contribution guidance. + +The public API is intentionally small: + +- `vuongtest()` — distinguishability and relative-fit comparison; +- `icci()` — interval estimates for information-criterion differences; +- `llcont()` — model-specific casewise contribution adapter boundary. + +## Statistical boundary + +These procedures compare fitted models under their assumptions. They do not establish causal validity, substitute for identification or data-quality checks, or turn information criteria into proof of scientific truth. Consequential analyses should preserve the exact package revision, fitted-model inputs, and software environment needed for reproduction. + +## Verification + +For a checked-out revision: + +```bash +R CMD build . +R CMD check nonnest2_*.tar.gz +``` + +Use current exact-head CI evidence when deciding whether a revision is fit for use; documentation presence is not a release or quality receipt. + +## License and commercial-use boundary + +The inherited package metadata declares `GPL-2 | GPL-3`. ContextualWisdomLab maintenance of this repository does not create a new permissive grant over upstream copyright. GPL permits commercial use under its terms, but GPL-family source is outside ContextualWisdomLab's normal inbound commercial baseline. + +See [commercial-license-boundary.md](commercial-license-boundary.md) for the recorded provenance, constraints, and closure paths. This site must not be read as approval for incorporation into a ContextualWisdomLab commercial product while that boundary remains unresolved. + +## More documentation + +- [README](../README.md) — product overview, usage, adapters, and interpretation +- [Commercial license boundary](commercial-license-boundary.md) — provenance and policy status +- [Vignettes](../vignettes/) — longer worked material in the source tree +- [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/nonnest2) — repository-aware navigation and questions From c3e74a2b0853f4ea6f36da413c3bc3bb0ff685c5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 20:11:34 +0900 Subject: [PATCH 05/14] docs: clarify nonnest2 comparison contracts --- README.md | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 82227d3..49d08b2 100644 --- a/README.md +++ b/README.md @@ -10,16 +10,16 @@ The package works with several established R model classes and exposes a small a ## When to use it -Use `nonnest2` when you have two fitted models for the same observations and need a casewise-likelihood comparison rather than a simple comparison of aggregate fit statistics. +Use `nonnest2` when you have two fitted models whose casewise quantities refer to the **same dependent variable(s), the same observations, and the same observation row order**. For `lavaan` objects, the modeled variables must likewise correspond. The package does not currently verify row alignment for you, so a reordered or mismatched analysis sample can produce a numerically valid-looking but scientifically invalid comparison. Typical jobs include: - testing whether two models are distinguishable from the observed data; - comparing relative model fit using Vuong-style tests; - obtaining interval estimates for differences in AIC and BIC; and -- extending the comparison machinery to another model class that can provide the required casewise contributions. +- extending the comparison machinery to another model class that can provide the required casewise quantities. -Built-in `llcont()` methods currently cover model classes from ecosystems including `lavaan`, `mirt`, `OpenMx`, base/generalized linear models, and several other regression-model families. The exact exported API is `vuongtest()`, `icci()`, and `llcont()`. +Built-in `llcont()` methods currently cover model classes from ecosystems including `lavaan`, `mirt`, `OpenMx`, base/generalized linear models, and several other regression-model families. The exact exported public API is `vuongtest()`, `icci()`, and `llcont()`. ## Install from this source tree @@ -40,7 +40,7 @@ The repository also contains an `R-CMD-check` GitHub Actions workflow. Its curre ## Quick start -The example below compares two factor models that are not nested because the indicator assignments differ. +The example below compares two factor models that are non-nested because the indicator assignments differ. Fit both models to the same rows in the same order. ```r library(lavaan) @@ -60,15 +60,27 @@ m2 <- ' ' fit2 <- cfa(m2, data = HolzingerSwineford1939) -vuongtest(fit1, fit2) +comparison <- vuongtest(fit1, fit2) +comparison +``` + +Interpret the distinguishability result before treating relative-fit evidence as meaningful. `icci()` is intended for **non-nested models that are distinguishable according to the `vuongtest()` variance/distinguishability test**. If the models are nested or indistinguishable, the intervals returned by `icci()` are not valid for their intended interpretation. Only after that condition is satisfied should you compute, for example: + +```r icci(fit1, fit2) ``` -Interpret the distinguishability result before treating a relative-fit result as meaningful, and interpret both in the context of the models' assumptions and the scientific question. +Use the inferential threshold and model assumptions appropriate to the analysis rather than treating a package default as a scientific decision rule. ## Integration model -`nonnest2` keeps the public comparison layer small. Model-specific integration happens through casewise likelihood/derivative contributions exposed by `llcont()` methods. New model classes should provide the statistical quantities expected by that adapter contract rather than embedding package-specific branching into `vuongtest()` itself. +`nonnest2` keeps the public comparison layer small, but its adapter responsibilities are distinct: + +- `llcont(model)` returns a numeric vector of **casewise log-likelihood contributions** in observation order; +- `vuongtest()` separately obtains casewise score contributions through `score1` / `score2` when supplied, or through the package's class-specific/default score path such as `sandwich::estfun()` or the supported `mirt` score path; and +- `vc1` / `vc2` supply the parameter covariance matrices used with those scores to construct the Vuong comparison matrices. + +A new model adapter must therefore preserve row alignment across its likelihood and score contributions and provide each statistical quantity through the correct interface; `llcont()` is not a combined likelihood-and-derivative return contract. For the exact supported S3 methods, see [`NAMESPACE`](NAMESPACE). Package history is recorded in [`NEWS`](NEWS), and longer worked material lives under [`vignettes/`](vignettes/). @@ -78,7 +90,7 @@ The package metadata cites: > Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses. *Econometrica, 57*(2), 307–333. https://doi.org/10.2307/1912557 -The software implements statistical procedures based on that theory; using the software does not remove the need to check identification, estimator assumptions, data quality, model misspecification, and the substantive meaning of the compared models. +The software implements statistical procedures based on that theory; using the software does not remove the need to check identification, estimator assumptions, data quality, model misspecification, observation alignment, and the substantive meaning of the compared models. ## Project status and verification @@ -86,7 +98,7 @@ Current package metadata declares version `0.5-9` dated 2026-03-31. Treat that a ## Contributing -Keep changes focused on the statistical contract and supported model adapters. New adapters should include realistic tests for casewise contributions and comparison behavior. Changes to numerical formulas, supported model semantics, or public return values should be documented and verified through the repository's ordinary R package checks. +Keep changes focused on the statistical contract and supported model adapters. New adapters should include realistic tests for casewise likelihood contributions, score/covariance integration, row alignment assumptions, and comparison behavior. Changes to numerical formulas, supported model semantics, or public return values should be documented and verified through the repository's ordinary R package checks. ## License and commercial-use boundary From 6cb90918b34f0ac2693f3383628f2ba1773874d5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 20:11:59 +0900 Subject: [PATCH 06/14] docs: bind license provenance to immutable revisions --- docs/commercial-license-boundary.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/commercial-license-boundary.md b/docs/commercial-license-boundary.md index 4a12b29..18c7d53 100644 --- a/docs/commercial-license-boundary.md +++ b/docs/commercial-license-boundary.md @@ -8,7 +8,7 @@ It is a provenance and engineering-control record, not legal advice and not a ch ## Current evidence -Protected `master` package metadata currently states: +This record was verified on **2026-09-02** against the repository default branch at immutable revision `master@807e9405f8c32faafdf186f977a24d0b23358b43` and re-read on this documentation lineage. The `DESCRIPTION` bytes on that source state: - `Package: nonnest2`; - `Version: 0.5-9`; @@ -16,7 +16,9 @@ Protected `master` package metadata currently states: - Edgar Merkle and Dongjun You are package authors, with additional contributors including Lennart Schneider, Mauricio Garnier-Villarreal, Seongho Bae, and Phil Chalmers; and - the package URL is `https://github.com/qpsy/nonnest2`. -The repository also contains historical and current ContextualWisdomLab maintenance commits. Those maintenance contributions do not establish ownership of every inherited copyright interest and therefore do not establish unilateral relicensing authority over the complete package. +The commit history reachable from `master@807e9405f8c32faafdf186f977a24d0b23358b43` was also inspected on 2026-09-02 and contains ContextualWisdomLab maintenance contributions. That bounded maintenance evidence does not establish ownership of every inherited copyright interest and therefore does not establish unilateral relicensing authority over the complete package. + +This document intentionally does not claim a branch-protection state that has not been established as licensing provenance. The immutable source revision and package metadata—not a mutable branch label—are the evidence used here. No root MIT/Apache license is added by this documentation lane. The R package `DESCRIPTION` license remains the authoritative source-license declaration unless and until an evidence-backed licensing change is approved by the relevant rights holders. @@ -47,8 +49,10 @@ A README disclaimer, a new dependency, a process boundary, or the absence of a r A licensing repair must not silently change the statistical product. At minimum, the surviving implementation must re-establish evidence for: - `vuongtest()` distinguishability and relative-fit behavior; -- `icci()` AIC/BIC difference interval behavior; -- `llcont()` casewise contribution semantics; +- `icci()` AIC/BIC difference interval behavior for non-nested, distinguishable models; +- `llcont()` casewise log-likelihood contribution semantics; +- the separate score-function and covariance-matrix contracts used by `vuongtest()`; +- exact observation/sample ordering assumptions across compared models and adapter outputs; - supported adapter behavior for the model classes the repository continues to advertise; and - edge cases, numerical tolerances, and failure behavior required by existing tests and documented examples. From c76d8b91f56f74e238da82257066596b420a92c2 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 20:12:13 +0900 Subject: [PATCH 07/14] docs: make nonnest2 Pages navigation durable --- docs/index.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/index.md b/docs/index.md index 6b7da48..94b4adb 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,16 +6,18 @@ ## Start here -The task-first [README](../README.md) covers installation, a worked example, supported model adapters, verification, statistical interpretation, and contribution guidance. +The task-first [README](https://github.com/ContextualWisdomLab/nonnest2/blob/master/README.md) covers installation, a worked example, model-alignment requirements, supported adapter contracts, verification, statistical interpretation, and contribution guidance. The public API is intentionally small: - `vuongtest()` — distinguishability and relative-fit comparison; -- `icci()` — interval estimates for information-criterion differences; -- `llcont()` — model-specific casewise contribution adapter boundary. +- `icci()` — information-criterion difference intervals for non-nested models that satisfy the distinguishability condition; and +- `llcont()` — model-specific casewise log-likelihood contributions. ## Statistical boundary +Compared models must refer to the same dependent variable(s), the same observations, and the same observation order. `nonnest2` does not currently verify row alignment. `llcont()` likelihood contributions, score contributions, and covariance inputs are separate statistical contracts and must remain aligned by observation and parameterization. + These procedures compare fitted models under their assumptions. They do not establish causal validity, substitute for identification or data-quality checks, or turn information criteria into proof of scientific truth. Consequential analyses should preserve the exact package revision, fitted-model inputs, and software environment needed for reproduction. ## Verification @@ -33,11 +35,13 @@ Use current exact-head CI evidence when deciding whether a revision is fit for u The inherited package metadata declares `GPL-2 | GPL-3`. ContextualWisdomLab maintenance of this repository does not create a new permissive grant over upstream copyright. GPL permits commercial use under its terms, but GPL-family source is outside ContextualWisdomLab's normal inbound commercial baseline. -See [commercial-license-boundary.md](commercial-license-boundary.md) for the recorded provenance, constraints, and closure paths. This site must not be read as approval for incorporation into a ContextualWisdomLab commercial product while that boundary remains unresolved. +See [commercial-license-boundary.md](commercial-license-boundary.md) for the recorded immutable provenance, constraints, and closure paths. This documentation source must not be read as approval for incorporation into a ContextualWisdomLab commercial product while that boundary remains unresolved. ## More documentation -- [README](../README.md) — product overview, usage, adapters, and interpretation +- [README](https://github.com/ContextualWisdomLab/nonnest2/blob/master/README.md) — product overview, usage, adapters, and interpretation - [Commercial license boundary](commercial-license-boundary.md) — provenance and policy status -- [Vignettes](../vignettes/) — longer worked material in the source tree +- [Vignettes](https://github.com/ContextualWisdomLab/nonnest2/tree/master/vignettes) — longer worked material in the source tree - [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/nonnest2) — repository-aware navigation and questions + +This file is only a publication-ready documentation source. It does not itself establish GitHub Pages publication or a released package artifact. From 7ce977da6c51d2a192ce856c08ff6a9821a688bd Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 20:16:31 +0900 Subject: [PATCH 08/14] docs: add nonnest2 product technical gap baseline --- docs/product-technical-gap-baseline.md | 103 +++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 docs/product-technical-gap-baseline.md diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md new file mode 100644 index 0000000..d3f2029 --- /dev/null +++ b/docs/product-technical-gap-baseline.md @@ -0,0 +1,103 @@ +# nonnest2 product and technical gap baseline + +**Snapshot:** 2026-09-02 +**Default-branch evidence:** `master@807e9405f8c32faafdf186f977a24d0b23358b43` +**Audience:** maintainers, scientific reviewers, integration reviewers, and licensing/provenance reviewers + +This document records current product responsibility, scientific contracts, provenance constraints, and buyer-visible gaps. It is a dated evidence ledger, not a substitute for re-reading live GitHub PR/check/release state before integration. + +## Product responsibility + +`nonnest2` is an R statistical library for comparison of fitted nested and non-nested models using Vuong-based distinguishability and relative-fit procedures, plus confidence intervals for differences in AIC and BIC. + +The repository owns the comparison procedures and adapter contracts exposed by `vuongtest()`, `icci()`, and `llcont()`. It does not own the fitted-model implementations supplied by `lavaan`, `mirt`, `OpenMx`, base R model classes, or other supported model ecosystems, and it does not turn statistical comparison into causal or substantive model validity. + +## Ubiquitous language and invariants + +- **Casewise log-likelihood contribution:** one numeric contribution per observation, returned by `llcont()` or a caller-supplied `ll1` / `ll2` function. +- **Score contribution:** casewise parameter-score information supplied separately through `score1` / `score2` or the supported class-specific/default score path. +- **Parameter covariance:** covariance evidence supplied through `vc1` / `vc2` and used separately from the likelihood-contribution contract. +- **Distinguishability:** whether the competing models can be statistically distinguished on the observed data under the Vuong procedure. +- **Relative fit:** direction/evidence of model-fit difference after the relevant distinguishability/nesting conditions are respected. +- **Observation alignment invariant:** compared casewise vectors must refer to the same dependent variable(s), same observations, and identical observation row order. Current source documentation explicitly states that the package does not verify this alignment. +- **ICCI applicability invariant:** `icci()` is intended for non-nested models that are distinguishable under the `vuongtest()` variance/distinguishability test; nested or indistinguishable inputs can yield intervals that are invalid for the intended interpretation. + +## Context Map + +```text +fitted-model packages / user models + | + | llcont + score + covariance adapters + v ++-------------------------------+ +| nonnest2 model comparison | +| - distinguishability | +| - relative fit | +| - AIC/BIC difference intervals| ++-------------------------------+ + | + v +analysis/reporting code owned by the caller +``` + +External model packages remain separate authorities for their estimands, fitting behavior, parameterization, likelihoods, scores, and covariance estimators. `nonnest2` consumes those quantities through adapter boundaries; it does not copy their domain/runtime authority. + +## Product flow UML + +```mermaid +sequenceDiagram + participant A as Analyst + participant M1 as Fitted model 1 + participant M2 as Fitted model 2 + participant N as nonnest2 + + A->>N: vuongtest(model1, model2) + N->>M1: casewise likelihood / score / covariance + N->>M2: casewise likelihood / score / covariance + N-->>A: distinguishability + relative-fit evidence + alt non-nested and distinguishable + A->>N: icci(model1, model2) + N-->>A: AIC/BIC difference intervals + else nested or indistinguishable + A-->>A: do not interpret icci intervals as valid + end +``` + +## Data / ERD boundary + +The current package has no repository-owned database or persistent domain schema. A database ERD would therefore invent authority that does not exist and is intentionally **not applicable** to this baseline. Fitted model objects and analysis data remain caller/model-package state. + +## Current implementation evidence + +Default-branch `DESCRIPTION` at `807e9405f8c32faafdf186f977a24d0b23358b43` declares package version `0.5-9`, R `>= 3.0.0`, imports `CompQuadForm`, `mvtnorm`, `lavaan`, `sandwich`, and `methods`, and declares source license `GPL-2 | GPL-3`. It also identifies Edgar Merkle and Dongjun You as authors plus additional contributors and points to the upstream `qpsy/nonnest2` repository. + +`R/vuongtest.R` and `R/icci.R` document the same dependent-variable / modeled-variable / observation-order requirement and explicitly state that current code does not check it. `R/icci.R` also states that nested or variance-test-indistinguishable models make its returned intervals incorrect for the intended interpretation. + +The ContextualWisdomLab GitHub repository currently exposes no GitHub Release. Package/source metadata must therefore not be presented as an immutable organization release artifact. + +## Gap register + +| Priority | Gap | Evidence / risk | Required action | Status | +| --- | --- | --- | --- | --- | +| P0 | Commercial source-license incompatibility with organization default | `DESCRIPTION` declares `GPL-2 | GPL-3`; inherited authors/contributors mean maintenance does not prove unilateral relicensing authority | Establish rights-backed permissive relicensing, independently authored compatible replacement with scientific regression evidence, or an explicitly approved repository-specific GPL distribution model | **Blocked / unresolved** | +| P0 | Observation alignment is not executable | Source docs say models must use the same dependent/modeled variables and identical observation order, but current code does not validate that invariant | Add a fail-closed comparison/adaptor contract that proves compatible casewise population/order where technically observable; otherwise require an explicit caller-supplied immutable alignment identity and test mismatch cases | **Open scientific-integrity gap** | +| P1 | Adapter conformance is implicit across multiple quantities | Likelihood contributions, score contributions, and covariance matrices are separate interfaces and can silently disagree in row/parameter semantics | Add executable adapter conformance fixtures for supported classes, including length/order, finite-value, parameter-dimension, missing-data and mismatch failure cases | **Open evidence gap** | +| P1 | No immutable ContextualWisdomLab release | GitHub release inventory is empty; `DESCRIPTION` version is source metadata only | After licensing and scientific gates are resolved, publish one protected-source package/release with reproducible checks, source provenance, checksum/SBOM/license evidence as applicable | **Not release-ready** | +| P1 | Open PR queue contains overlapping security/performance variants | Multiple current PRs target the same exported-input-validation and `llcont.glm` optimization surfaces | Reconcile by verified source-delta carryover into one canonical writer per concern; retire predecessors only after every valid test/fix/documentation delta is proven present | **Active reconciliation required** | +| P2 | Product documentation was legacy/terse | Canonical README PR #118 now owns product-first usage, statistical conditions, provenance, Pages-safe navigation, and this gap ledger | Keep README claims synchronized with protected source and exact release state; do not promote branch evidence to released truth | **In progress on #118** | + +## Current README / documentation integration lane + +PR #118 is the canonical public-surface writer for `README.md`, `docs/index.md`, `docs/commercial-license-boundary.md`, and this gap baseline. Badge-only PR #115 was closed only after its complete two-line Ask DeepWiki delta was verified present in #118. + +At snapshot time, #118 exact current head is `c76d8b91f56f74e238da82257066596b420a92c2` before this baseline commit; every source mutation after that head invalidates those predecessor checks. Live checks/reviews must always be re-read from the final unchanged head. + +## Licensing / provenance action boundary + +See [`commercial-license-boundary.md`](commercial-license-boundary.md). The organization must not add a root Apache-2.0/MIT file or rewrite `DESCRIPTION` merely because the repository is hosted under ContextualWisdomLab. Dependency licenses also do not change the inherited package source license. + +Any independent replacement must preserve the public statistical behavior without copying GPL implementation expression. Scientific acceptance should include reference/black-box parity for distinguishability, relative fit, AIC/BIC interval behavior, supported adapter classes, missingness/alignment edge cases, and numerical tolerances. + +## Verification and update rule + +Update this ledger when a protected-source statistical contract changes, a licensing/provenance fact changes, an immutable release appears, or a gap changes state. PR heads/check run IDs are snapshot evidence only; merge authority always comes from a fresh read of the unchanged candidate head, current base, reviews/threads, checks, and repository governance. From 4847a7ab397d8a9278aa1e2e27c8270778860586 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 20:16:54 +0900 Subject: [PATCH 09/14] docs: link nonnest2 gap baseline --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 49d08b2..440f04f 100644 --- a/README.md +++ b/README.md @@ -96,6 +96,8 @@ The software implements statistical procedures based on that theory; using the s Current package metadata declares version `0.5-9` dated 2026-03-31. Treat that as source/package metadata, not by itself as evidence of a particular published artifact, deployment, benchmark, or certification. For any consequential analysis, bind results to the exact package revision and preserve the fitted-model inputs and software environment needed to reproduce the comparison. +Maintainers can track current scientific, integration, release, and licensing gaps in [`docs/product-technical-gap-baseline.md`](docs/product-technical-gap-baseline.md). That dated ledger is not a substitute for fresh exact-head CI/review evidence. + ## Contributing Keep changes focused on the statistical contract and supported model adapters. New adapters should include realistic tests for casewise likelihood contributions, score/covariance integration, row alignment assumptions, and comparison behavior. Changes to numerical formulas, supported model semantics, or public return values should be documented and verified through the repository's ordinary R package checks. From 0027b6881d5678d4dbb4febe12691bd906f88ada Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 20:17:07 +0900 Subject: [PATCH 10/14] docs: link nonnest2 gap baseline from landing --- docs/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.md b/docs/index.md index 94b4adb..5655b17 100644 --- a/docs/index.md +++ b/docs/index.md @@ -40,6 +40,7 @@ See [commercial-license-boundary.md](commercial-license-boundary.md) for the rec ## More documentation - [README](https://github.com/ContextualWisdomLab/nonnest2/blob/master/README.md) — product overview, usage, adapters, and interpretation +- [Product and technical gap baseline](product-technical-gap-baseline.md) — current scientific, integration, release, and licensing gaps - [Commercial license boundary](commercial-license-boundary.md) — provenance and policy status - [Vignettes](https://github.com/ContextualWisdomLab/nonnest2/tree/master/vignettes) — longer worked material in the source tree - [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/nonnest2) — repository-aware navigation and questions From 797bb4d282004084e064eafe7effb261b7c48732 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 23:13:07 +0900 Subject: [PATCH 11/14] docs: correct icci quickstart and adapter alignment --- README.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 440f04f..dbe465f 100644 --- a/README.md +++ b/README.md @@ -64,11 +64,9 @@ comparison <- vuongtest(fit1, fit2) comparison ``` -Interpret the distinguishability result before treating relative-fit evidence as meaningful. `icci()` is intended for **non-nested models that are distinguishable according to the `vuongtest()` variance/distinguishability test**. If the models are nested or indistinguishable, the intervals returned by `icci()` are not valid for their intended interpretation. Only after that condition is satisfied should you compute, for example: +This particular Holzinger–Swineford pair is the package vignette's **indistinguishable-model** example, so it is useful for learning the first-stage variance/distinguishability test but it is **not** a valid `icci()` example. Interpret the distinguishability result before treating relative-fit evidence as meaningful. -```r -icci(fit1, fit2) -``` +`icci()` is intended for **non-nested models that are distinguishable according to the `vuongtest()` variance/distinguishability test**. If models are nested or indistinguishable, the intervals returned by `icci()` are incorrect for their intended interpretation. Use `icci()` only after a comparison satisfies that gate; the vignette's Political Democracy example demonstrates a distinguishable non-nested pair. Use the inferential threshold and model assumptions appropriate to the analysis rather than treating a package default as a scientific decision rule. @@ -77,10 +75,10 @@ Use the inferential threshold and model assumptions appropriate to the analysis `nonnest2` keeps the public comparison layer small, but its adapter responsibilities are distinct: - `llcont(model)` returns a numeric vector of **casewise log-likelihood contributions** in observation order; -- `vuongtest()` separately obtains casewise score contributions through `score1` / `score2` when supplied, or through the package's class-specific/default score path such as `sandwich::estfun()` or the supported `mirt` score path; and -- `vc1` / `vc2` supply the parameter covariance matrices used with those scores to construct the Vuong comparison matrices. +- `vuongtest()` separately obtains **casewise score contributions in the same observation order** through `score1` / `score2` when supplied, or through the package's class-specific/default score path such as `sandwich::estfun()` or the supported `mirt` score path; and +- `vc1` / `vc2` supply **parameter covariance matrices whose row/column ordering must match the score parameter ordering used by `calcAB()`**; they are not observation-level arrays. -A new model adapter must therefore preserve row alignment across its likelihood and score contributions and provide each statistical quantity through the correct interface; `llcont()` is not a combined likelihood-and-derivative return contract. +A new model adapter must therefore preserve observation-row alignment between likelihood and score contributions, preserve parameter-order alignment between scores and covariance matrices, and provide each statistical quantity through the correct interface. `llcont()` is not a combined likelihood-and-derivative return contract. For the exact supported S3 methods, see [`NAMESPACE`](NAMESPACE). Package history is recorded in [`NEWS`](NEWS), and longer worked material lives under [`vignettes/`](vignettes/). From 67d469c3c458089f2bdd544226f8edfbca2a7633 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 23:13:34 +0900 Subject: [PATCH 12/14] docs: separate observation and parameter alignment --- docs/index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index 5655b17..573cfdb 100644 --- a/docs/index.md +++ b/docs/index.md @@ -16,7 +16,9 @@ The public API is intentionally small: ## Statistical boundary -Compared models must refer to the same dependent variable(s), the same observations, and the same observation order. `nonnest2` does not currently verify row alignment. `llcont()` likelihood contributions, score contributions, and covariance inputs are separate statistical contracts and must remain aligned by observation and parameterization. +Compared models must refer to the same dependent variable(s), the same observations, and the same observation order. `nonnest2` does not currently verify row alignment. `llcont()` likelihood contributions and score contributions therefore share the observation ordering, while `vc1` / `vc2` are parameter covariance matrices whose rows and columns must match the score parameter ordering consumed by `calcAB()`; they are not observation-level inputs. + +`icci()` is valid only for non-nested models that are distinguishable according to the `vuongtest()` variance/distinguishability test. Nested or indistinguishable pairs must not be interpreted through its intervals. These procedures compare fitted models under their assumptions. They do not establish causal validity, substitute for identification or data-quality checks, or turn information criteria into proof of scientific truth. Consequential analyses should preserve the exact package revision, fitted-model inputs, and software environment needed for reproduction. From 5acc85e0184516228371ba48bf7c5b05918f1dec Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 23:15:14 +0900 Subject: [PATCH 13/14] docs: align vignette with icci validity contract --- vignettes/nonnest2.Rmd | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/vignettes/nonnest2.Rmd b/vignettes/nonnest2.Rmd index 3f72b95..48117e7 100644 --- a/vignettes/nonnest2.Rmd +++ b/vignettes/nonnest2.Rmd @@ -65,13 +65,9 @@ vuongtest(fit1, fit2) ``` We see that we obtain two tests, the *variance test* and the *non-nested likelihood ratio test*. The former test informs us of the models' distinguishability, while the latter test compares the fits of two distinguishable models. Focusing on the variance test here, we obtain a small test statistic and a large *p*-value. These results imply that, as we suspected, the two models are indistinguishable in the focal population. -We can also use Vuong's theory to obtain confidence intervals for the difference in the models' AIC or BIC statistics: -```{r} -icci(fit1, fit2) -``` -Based on this output, we see that the AIC and BIC statistics are close but lower for `fit1` than for `fit2`. The 95% confidence intervals overlap with 0, implying that the model fits are sufficiently close that neither can be preferred over the other. The confidence intervals for AIC and BIC differences are exactly the same because both models have the same numbers of parameters; this would not typically occur for other models. +Do **not** use `icci()` for this pair. The `icci()` function documentation states that its intervals are incorrect when models are nested or when the `vuongtest()` variance test indicates that the models are indistinguishable. The next example shows a non-nested pair that passes the distinguishability gate before information-criterion intervals are computed. -## Distiguishable and Non-Nested models +## Distinguishable and Non-Nested models Now consider the following structural equation models, using Bollen's political democracy data. The first model is the original (classic) model, while the second model estimates different residual covariance parameters. The models are non-nested due to the differing residual covariances. ```{r} @@ -119,7 +115,13 @@ We now apply the Vuong tests to the two models. ```{r} vuongtest(fit1, fit2) ``` -We now see that the variance tests indicates that the models are distinguishable (at least, at $\alpha=.05$). This allows us to move on to the second test, which compares the fits of the two models. Based on this second test, we conclude that the two models have equal fit in the population of interest. Note that *fit* is defined here in terms of Kullback-Leibler distance (between each candidate model and the true model), and we make no assumption about either of the two candidate models being the true model. +We now see that the variance test indicates that the models are distinguishable (at least, at $\alpha=.05$). This allows us to move on to the second test, which compares the fits of the two models. Based on this second test, we conclude that the two models have equal fit in the population of interest. Note that *fit* is defined here in terms of Kullback-Leibler distance (between each candidate model and the true model), and we make no assumption about either of the two candidate models being the true model. + +Because this pair is non-nested and passes the distinguishability gate, `icci()` can now be used for confidence intervals on the AIC and BIC differences: +```{r} +icci(fit1, fit2) +``` +Interpret those intervals together with the distinguishability and relative-fit evidence; they do not make an indistinguishable or nested comparison valid. ## Nested models Finally, `vuongtest()` includes a `nested` argument, using Vuong's theory to compare nested models. The resulting test statistics, which are very similar to the statistics described above, can be viewed as robust versions of the traditional likelihood ratio test (LRT) for nested models. This is because Vuong's theory makes no assumption about either candidate model being the true model, whereas the traditional LRT assumes that the full model is the truth. @@ -155,4 +157,4 @@ Both tests can be viewed as alternatives to the traditional likelihood ratio tes ```{r} anova(fit1, fit3) ``` -which provides similar results. In particular, the Vuong likelihood ratio test statistic is equal to the traditional likelihood ratio test statistic, though the *p*-values differ. This is because the null distributions differ: the traditional LRT uses a chi-square null distribution, whereas the Vuong LRT uses a weighted sum of chi-square distributions. The latter distribution converges to the traditional chi-square distribution when the full model is the true model. \ No newline at end of file +which provides similar results. In particular, the Vuong likelihood ratio test statistic is equal to the traditional likelihood ratio test statistic, though the *p*-values differ. This is because the null distributions differ: the traditional LRT uses a chi-square null distribution, whereas the Vuong LRT uses a weighted sum of chi-square distributions. The latter distribution converges to the traditional chi-square distribution when the full model is the true model. From f7d16bc87efa39224bbd132a26bac0ed214390cc Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 7 Sep 2026 09:11:50 +0900 Subject: [PATCH 14/14] docs: preserve GPL expression inside gap table --- docs/product-technical-gap-baseline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index d3f2029..ea4f8b5 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -79,7 +79,7 @@ The ContextualWisdomLab GitHub repository currently exposes no GitHub Release. P | Priority | Gap | Evidence / risk | Required action | Status | | --- | --- | --- | --- | --- | -| P0 | Commercial source-license incompatibility with organization default | `DESCRIPTION` declares `GPL-2 | GPL-3`; inherited authors/contributors mean maintenance does not prove unilateral relicensing authority | Establish rights-backed permissive relicensing, independently authored compatible replacement with scientific regression evidence, or an explicitly approved repository-specific GPL distribution model | **Blocked / unresolved** | +| P0 | Commercial source-license incompatibility with organization default | `DESCRIPTION` declares `GPL-2` \| `GPL-3`; inherited authors/contributors mean maintenance does not prove unilateral relicensing authority | Establish rights-backed permissive relicensing, independently authored compatible replacement with scientific regression evidence, or an explicitly approved repository-specific GPL distribution model | **Blocked / unresolved** | | P0 | Observation alignment is not executable | Source docs say models must use the same dependent/modeled variables and identical observation order, but current code does not validate that invariant | Add a fail-closed comparison/adaptor contract that proves compatible casewise population/order where technically observable; otherwise require an explicit caller-supplied immutable alignment identity and test mismatch cases | **Open scientific-integrity gap** | | P1 | Adapter conformance is implicit across multiple quantities | Likelihood contributions, score contributions, and covariance matrices are separate interfaces and can silently disagree in row/parameter semantics | Add executable adapter conformance fixtures for supported classes, including length/order, finite-value, parameter-dimension, missing-data and mismatch failure cases | **Open evidence gap** | | P1 | No immutable ContextualWisdomLab release | GitHub release inventory is empty; `DESCRIPTION` version is source metadata only | After licensing and scientific gates are resolved, publish one protected-source package/release with reproducible checks, source provenance, checksum/SBOM/license evidence as applicable | **Not release-ready** |