From e397aa2c48ad596b85305aa0dec7b347c686ec35 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sun, 16 Aug 2026 16:51:13 +0000 Subject: [PATCH 01/20] docs: add FIPC ADRs and correct bibliographic citations Record Kim (2006) as the implemented FIPC contract, contrast Stocking-Lord and Haebara as unimplemented alternatives, and replace the misattributed Kim & Kolen 2010 JEM record with Kang & Petersen (2012) and Kim & Kolen (2019). Co-authored-by: Seongho Bae --- ARCHITECTURE.md | 74 ++++++++++- CONTRIBUTING.md | 5 +- README.md | 48 +++++++ docs/adr/0000-template.md | 33 +++++ docs/adr/0001-fipc-linking-contract.md | 117 +++++++++++++++++ docs/adr/0002-mirt-mml-em-engine.md | 81 ++++++++++++ docs/adr/0003-ipd-dif-screening-delegation.md | 74 +++++++++++ docs/adr/README.md | 28 ++++ docs/fixed-parameter-item-calibration.md | 87 +++++++++++-- docs/papers/README.md | 120 +++++++++++++----- 10 files changed, 623 insertions(+), 44 deletions(-) create mode 100644 docs/adr/0000-template.md create mode 100644 docs/adr/0001-fipc-linking-contract.md create mode 100644 docs/adr/0002-mirt-mml-em-engine.md create mode 100644 docs/adr/0003-ipd-dif-screening-delegation.md create mode 100644 docs/adr/README.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3880f15c..cf858e15 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -19,6 +19,9 @@ metadata and CI are wired, and which parts are safe to evolve. - `.github/ISSUE_TEMPLATE/` - structured issue intake templates - `.github/CODEOWNERS` - code ownership map for reviews - `.github/dependabot.yml` - Automated Actions dependency updates +- `docs/adr/` - architecture decision records (FIPC contract, mirt engine) +- `docs/fixed-parameter-item-calibration.md` - Kim (2006) linking contract +- `docs/papers/README.md` - verified bibliographic sources and DOIs - `docs/coderabbit/review-commands.md` - CodeRabbit command quick reference - `docs/operations/maintenance-runbook.md` - recurring maintainer operations checklist - `README.md` - User/developer entrypoint @@ -44,6 +47,8 @@ R objects (data frame/matrix/model), and outputs are returned as an R list. - Apply fixed common-item constraints for test linking - Produce linked model and score/theta artifacts - Key dependency: `mirt` +- Method decision: FIPC (Kim, 2006), not Stocking–Lord (1983) or + Haebara (1980) transformation estimation; see `docs/adr/0001-fipc-linking-contract.md` ### 3.2 Package Metadata and API Surface @@ -100,15 +105,78 @@ package metadata, and CI workflow definitions in Git. - Evaluate migration path from historical `packrat/` to a modern lock workflow. -## 10. Project Identification +## 10. Bibliographic grounding + +`autoFIPC()` orchestrates FIPC (Kim, 2006): anchors keep old-form +values and the new form is calibrated onto that scale. That contract +is an alternative to separate calibration plus Stocking and Lord +(1983) or Haebara (1980) characteristic-curve linking, and to +concurrent calibration (Kolen & Brennan, 2014). This repository does +not implement those transformation estimators. + +Estimation lives in `mirt` MML-EM (Chalmers, 2012; Bock & Aitkin, +1981). Optional IPD screening calls `mirt::multipleGroup` and +`mirt::DIF`; it is not a published invariance claim (see +`docs/adr/0003-ipd-dif-screening-delegation.md`). Score-scale +interpretation is bounded by AERA, APA, and NCME (2014). + +A withdrawn cite attributed "Linking item parameters to a base scale" +to Kim and Kolen (2010) in JEM. That record is not kept. The title is +Kang and Petersen (2012). Kim and Kolen (2019) is a real later FIPC +application paper. + +Full APA 7th records and DOIs: `docs/papers/README.md`. Accepted +method ADRs: `docs/adr/`. + +Kim, S. (2006). A comparative study of IRT fixed parameter calibration +methods. *Journal of Educational Measurement, 43*(4), 355–381. + + +Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in +item response theory. *Applied Psychological Measurement, 7*(2), +201–210. + +Haebara, T. (1980). Equating logistic ability scales by a weighted +least squares method. *Japanese Psychological Research, 22*(3), +144–149. + +Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and +linking: Methods and practices* (3rd ed.). Springer. + + +Kim, S., & Kolen, M. J. (2019). Application of IRT fixed parameter +calibration to multiple-group test data. *Applied Measurement in +Education, 32*(4), 310–324. + + +Kang, T., & Petersen, N. S. (2012). Linking item parameters to a base +scale. *Asia Pacific Education Review, 13*(2), 311–321. + + +Chalmers, R. P. (2012). mirt: A multidimensional item response theory +package for the R environment. *Journal of Statistical Software, +48*(6), 1–29. + +Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood +estimation of item parameters: Application of an EM algorithm. +*Psychometrika, 46*(4), 443–459. + + +American Educational Research Association, American Psychological +Association, & National Council on Measurement in Education. (2014). +*Standards for educational and psychological testing*. American +Educational Research Association. + +## 11. Project Identification - Project Name: aFIPC - Repository URL: `https://github.com/seonghobae/aFIPC` - Primary Contact: Seongho Bae -- Date of Last Update: 2026-02-15 +- Date of Last Update: 2026-08-16 -## 11. Glossary / Acronyms +## 12. Glossary / Acronyms - FIPC: Fixed Item Parameter Calibration - IPD: Item Parameter Drift - IRT: Item Response Theory +- MML-EM: Marginal Maximum Likelihood via the EM algorithm diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cfc9dbbe..da361caf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,10 +8,13 @@ over feature velocity. ## Ground Rules 1. Avoid behavior changes in `R/aFIPC.R` unless backed by explicit regression - evidence and maintainer approval. + evidence and maintainer approval. 2. Prefer small, auditable pull requests. 3. Keep CI/security/docs healthy (`.github/workflows/`, `README.md`, `ARCHITECTURE.md`, `AGENTS.md`). +4. Methodological decisions (linking contract, estimation engine) are + recorded in `docs/adr/`. Cite verified sources from + `docs/papers/README.md`; do not invent bibliographic records. ## Development Setup diff --git a/README.md b/README.md index 7e5816f2..ae25ed00 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,58 @@ produce accurate fixed-item linking results. The current maintenance goal is to preserve numerical behavior while modernizing repository operations (documentation, CI, and dependency hygiene). +## Methodological sources + +`autoFIPC()` implements fixed item parameter calibration (FIPC): common +items keep their old-form parameter values while the new form is +calibrated onto that scale (Kim, 2006). That is a different design from +separate calibration plus a Stocking and Lord (1983) or Haebara (1980) +characteristic-curve transformation, and from concurrent calibration +(Kolen & Brennan, 2014). This package does not estimate Stocking–Lord +or Haebara linking constants. + +Estimation is delegated to `mirt` MML-EM (Chalmers, 2012; Bock & +Aitkin, 1981). Linked-score interpretation is bounded by AERA, APA, and +NCME (2014). Full APA records and DOIs are in +`docs/papers/README.md`; method decisions are in `docs/adr/`. + +Kim, S. (2006). A comparative study of IRT fixed parameter calibration +methods. *Journal of Educational Measurement, 43*(4), 355–381. + + +Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in +item response theory. *Applied Psychological Measurement, 7*(2), +201–210. + +Haebara, T. (1980). Equating logistic ability scales by a weighted +least squares method. *Japanese Psychological Research, 22*(3), +144–149. + +Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and +linking: Methods and practices* (3rd ed.). Springer. + + +Chalmers, R. P. (2012). mirt: A multidimensional item response theory +package for the R environment. *Journal of Statistical Software, +48*(6), 1–29. + +Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood +estimation of item parameters: Application of an EM algorithm. +*Psychometrika, 46*(4), 443–459. + + +American Educational Research Association, American Psychological +Association, & National Council on Measurement in Education. (2014). +*Standards for educational and psychological testing*. American +Educational Research Association. + ## What this repository contains - `R/aFIPC.R`: core `autoFIPC()` implementation - `DESCRIPTION`, `NAMESPACE`, `man/`: package metadata and generated docs +- `docs/adr/`: architecture decision records for FIPC and estimation +- `docs/fixed-parameter-item-calibration.md`: linking-contract restatement +- `docs/papers/README.md`: verified source papers and DOIs - `packrat/`: historical dependency lock/vendor directory - `.github/workflows/`: CI/security automation diff --git a/docs/adr/0000-template.md b/docs/adr/0000-template.md new file mode 100644 index 00000000..0a5bbe63 --- /dev/null +++ b/docs/adr/0000-template.md @@ -0,0 +1,33 @@ +# ADR-NNNN: Short decision title + +- Status: Proposed | Accepted | Deprecated | Superseded by ADR-NNNN +- Date: YYYY-MM-DD +- Deciders: maintainers + +## Context + +What question is being decided, and which constraints apply? Cite verified +sources. Do not invent bibliographic records. + +## Decision + +State the choice as a contract the package will keep. + +## Alternatives considered + +What was rejected and why, with sources when the alternative is a published +method. + +## Consequences + +Positive and negative follow-through for maintainers and callers. + +## Claim boundary + +What this decision does and does not claim. Separate orchestration in this +package from estimation or methodology that lives elsewhere. + +## References + +Use APA 7th. Include a DOI or publisher URL when one exists. Leave no empty +`DOI:` placeholders. diff --git a/docs/adr/0001-fipc-linking-contract.md b/docs/adr/0001-fipc-linking-contract.md new file mode 100644 index 00000000..eaf8e75a --- /dev/null +++ b/docs/adr/0001-fipc-linking-contract.md @@ -0,0 +1,117 @@ +# ADR-0001: FIPC as the linking contract + +- Status: Accepted +- Date: 2026-08-16 +- Deciders: maintainers + +## Context + +`autoFIPC()` places a new IRT form onto an established old-form scale by +declaring common items and calibrating the new form while those anchors +keep their old-form parameter values. The scientific question is which +published linking family that workflow implements. + +Kim (2006) frames fixed item parameter calibration (FIPC): old +operational or anchor parameters are treated as known during new-form +calibration so the new form is estimated directly on the base scale. +Kolen and Brennan (2014) survey the broader equating and linking +toolkit, including separate calibration plus a characteristic-curve +transformation, concurrent calibration, and fixed-parameter approaches. +The *Standards for Educational and Psychological Testing* +(AERA, APA, & NCME, 2014) limit how linked scores may be interpreted. + +A withdrawn repository cite attributed "Linking item parameters to a +base scale" to Kim and Kolen (2010) in the *Journal of Educational +Measurement*. That record is not a real JEM article. The title belongs +to Kang and Petersen (2012). A real Kim and Kolen FIPC paper is their +2019 *Applied Measurement in Education* application to multiple-group +data. + +## Decision + +This package implements FIPC as specified by Kim (2006) and restated in +`docs/fixed-parameter-item-calibration.md`: + +1. For each eligible anchor pair, copy the old-form item parameter + vector onto the matching new-form item. +2. Hold those copied parameters fixed (`est := FALSE`) during the + linked calibration. +3. Estimate only non-anchor new-form parameters on the scale defined by + the fixed anchors. + +`autoFIPC()` orchestrates that contract. It does not estimate a +Stocking–Lord or Haebara transformation. Inspection of `R/aFIPC.R` +shows no characteristic-curve linking objective; the linked call is +`mirt::mirt(..., pars = NewScaleParms)` after the copy-and-fix step. + +## Alternatives considered + +- **Separate calibration + Stocking and Lord (1983).** Calibrate each + form freely, then find a linear transformation that matches test + characteristic curves. Canonical characteristic-curve equating; not + what `autoFIPC()` computes. +- **Separate calibration + Haebara (1980).** Calibrate each form + freely, then match item characteristic curves by weighted least + squares. Also a post-calibration transformation; not implemented + here. +- **Concurrent calibration.** Estimate both forms in one run with + shared parameters for common items. `autoFIPC()` instead calibrates + forms separately and then fixes anchors (Kim, 2006). +- **Kang and Petersen (2012).** Correct source for the title "Linking + item parameters to a base scale." Useful background on placing + parameters onto a base scale; not the FIPC contract this package + implements. +- **Kim and Kolen (2019).** Later FIPC application to multiple-group + test data. Supports FIPC as a published method family; does not + replace Kim (2006) as the contract implemented here. + +## Consequences + +- Maintainers must preserve the copy-and-fix invariant unless a + regression fixture and explicit maintainer intent say otherwise. +- Docs must contrast FIPC with Stocking–Lord and Haebara so readers do + not infer that `autoFIPC()` returns those transformation constants. +- Score-scale claims stay inside AERA/APA/NCME (2014) limits: linking + does not by itself justify interchangeable high-stakes + interpretations. + +## Claim boundary + +This package orchestrates FIPC. Estimation of item-response +probabilities, the MML-EM cycles, and scores lives in `mirt` (see +ADR-0002). Accepting FIPC here is not a claim that Stocking–Lord, +Haebara, or concurrent calibration are inferior; they are different +published designs. It is also not a claim that linked scores meet a +particular testing-program validity argument. + +## References + +Kim, S. (2006). A comparative study of IRT fixed parameter calibration +methods. *Journal of Educational Measurement, 43*(4), 355–381. + + +Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in +item response theory. *Applied Psychological Measurement, 7*(2), +201–210. + +Haebara, T. (1980). Equating logistic ability scales by a weighted +least squares method. *Japanese Psychological Research, 22*(3), +144–149. + +Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and +linking: Methods and practices* (3rd ed.). Springer. + + +Kim, S., & Kolen, M. J. (2019). Application of IRT fixed parameter +calibration to multiple-group test data. *Applied Measurement in +Education, 32*(4), 310–324. + + +Kang, T., & Petersen, N. S. (2012). Linking item parameters to a base +scale. *Asia Pacific Education Review, 13*(2), 311–321. + + +American Educational Research Association, American Psychological +Association, & National Council on Measurement in Education. (2014). +*Standards for educational and psychological testing*. American +Educational Research Association. diff --git a/docs/adr/0002-mirt-mml-em-engine.md b/docs/adr/0002-mirt-mml-em-engine.md new file mode 100644 index 00000000..271cd5c0 --- /dev/null +++ b/docs/adr/0002-mirt-mml-em-engine.md @@ -0,0 +1,81 @@ +# ADR-0002: mirt MML-EM as the estimation engine + +- Status: Accepted +- Date: 2026-08-16 +- Deciders: maintainers + +## Context + +FIPC (ADR-0001) is a linking contract: which parameters are copied from +the old form and held fixed. Someone still has to estimate the free +item parameters and the ability distribution. This package is an R +orchestrator, not a new IRT estimator. + +Chalmers (2012) describes `mirt`, the package `DESCRIPTION` imports. +Bock and Aitkin (1981) give the marginal maximum likelihood EM +(MML-EM) algorithm that `mirt` implements when `method = "EM"`. +`autoFIPC()` uses that path when `tryEM` is true (the default) or when +the item type is nominal. + +## Decision + +Use `mirt` as the estimation engine and MML-EM as the default +calibration method: + +- Separate old-form and new-form fits call `mirt::mirt`. +- The linked fit calls `mirt::mirt` with `pars` after the FIPC + copy-and-fix step. +- Default linked estimation uses `method = "EM"` (Bock & Aitkin, + 1981, as implemented by Chalmers, 2012). +- Scores and expected-score artifacts use `mirt` helpers such as + `fscores` and `expected.test`. + +This package does not reimplement the likelihood, quadrature, or EM +cycles. + +## Alternatives considered + +- **A custom MML-EM implementation in `R/aFIPC.R`.** Would duplicate + a maintained estimator and risk silent numerical drift. Rejected. +- **A different IRT package.** Historical outputs were produced with + `mirt`. Changing engines would be a scientific behavior change, not + a docs fix. +- **`mirt` MHRM (`method = "MHRM"`).** `autoFIPC()` can take this + path when `tryEM` is false and the item type is not nominal. It is + an optional `mirt` method, not the default FIPC engine documented + here. + +## Consequences + +- Numerical changes in `mirt` can change `autoFIPC()` output even when + this repository's R sources are untouched. +- Formula-integrity reviews (see + `docs/fixed-parameter-item-calibration.md`) apply to orchestration + only. Estimation mathematics stay in `mirt`. +- `man/autoFIPC.Rd` remains roxygen-generated from `R/aFIPC.R`. + Method citations belong in these markdown ADRs and + `docs/papers/README.md` unless the roxygen `@references` block is + updated in the same change. + +## Claim boundary + +Choosing `mirt` and MML-EM is an engineering dependency decision. It +is not a claim that this package contributes a new estimation +algorithm, and it is not a claim about the relative accuracy of EM +versus other `mirt` methods. Linking-scale interpretation remains +bounded by ADR-0001 and AERA/APA/NCME (2014). + +## References + +Chalmers, R. P. (2012). mirt: A multidimensional item response theory +package for the R environment. *Journal of Statistical Software, +48*(6), 1–29. + +Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood +estimation of item parameters: Application of an EM algorithm. +*Psychometrika, 46*(4), 443–459. + + +Kim, S. (2006). A comparative study of IRT fixed parameter calibration +methods. *Journal of Educational Measurement, 43*(4), 355–381. + diff --git a/docs/adr/0003-ipd-dif-screening-delegation.md b/docs/adr/0003-ipd-dif-screening-delegation.md new file mode 100644 index 00000000..2d905944 --- /dev/null +++ b/docs/adr/0003-ipd-dif-screening-delegation.md @@ -0,0 +1,74 @@ +# ADR-0003: IPD/DIF screening is delegated to mirt + +- Status: Accepted +- Date: 2026-08-16 +- Deciders: maintainers + +## Context + +`autoFIPC()` can optionally screen common items for item parameter +drift (IPD) before the FIPC copy-and-fix step (`checkIPD`, default +true). The implementation builds a two-group response matrix and calls +`mirt::multipleGroup` plus `mirt::DIF`. Items flagged by that screen +may be dropped from the anchor set. + +That workflow is an operational convenience around `mirt`. It is not +itself a published invariance, DIF, or IPD methodology paper, and it +must not be documented as one. + +## Decision + +Treat IPD/DIF screening as delegated `mirt` machinery: + +- When `checkIPD` is true, screening uses `mirt::multipleGroup` and + `mirt::DIF` on the declared common items. +- Items retained after the screen become the anchors for the Kim + (2006) FIPC contract (ADR-0001). +- Documentation may describe the calls and the effect on the anchor + list. It must not present `autoFIPC()` as a new DIF/IPD statistic + or as evidence that anchors are invariant in a testing-program + sense. + +## Alternatives considered + +- **No IPD screen.** Callers can set `checkIPD = FALSE` and supply + anchors they have already reviewed. +- **A package-local DIF/IPD statistic.** Would be a new methodological + claim and a behavior change. Out of scope for documentation work + and not present in `R/aFIPC.R`. +- **Citing a security standard (NIST, OWASP) for this control.** + Those sources apply to security ADRs. IPD screening is a + psychometric operations step, not a security control. + +## Consequences + +- IPD results inherit `mirt` defaults, version behavior, and the + arguments `autoFIPC()` passes through. Changes in `mirt` can change + which anchors survive. +- Reviewers should not treat a clean IPD screen as a published + invariance argument (AERA, APA, & NCME, 2014). +- Tests that pin FIPC (anchors fixed to old-form values) are separate + from any claim about the DIF screen's Type I error or power. + +## Claim boundary + +This ADR records delegation. It does not claim that the `mirt` DIF +screen equals a named published IPD procedure, that surviving anchors +are drift-free, or that linked scores are interchangeable. The linking +contract remains FIPC (ADR-0001); estimation remains `mirt` MML-EM +(ADR-0002). + +## References + +Chalmers, R. P. (2012). mirt: A multidimensional item response theory +package for the R environment. *Journal of Statistical Software, +48*(6), 1–29. + +Kim, S. (2006). A comparative study of IRT fixed parameter calibration +methods. *Journal of Educational Measurement, 43*(4), 355–381. + + +American Educational Research Association, American Psychological +Association, & National Council on Measurement in Education. (2014). +*Standards for educational and psychological testing*. American +Educational Research Association. diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..be452207 --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,28 @@ +# Architecture decision records + +This folder records methodological and estimation decisions for `aFIPC`. +Each ADR states a contract, the alternatives considered, and a claim +boundary so maintainers do not treat orchestration in this package as a +new psychometric method. + +Use `0000-template.md` for new records. Status values are Proposed, +Accepted, Deprecated, or Superseded. Do not invent bibliographic records; +cite verified sources only. + +These ADRs are documentation. They do not change `autoFIPC()` numerical +behavior. + +## Index + +| ID | Title | Status | +| --- | --- | --- | +| [ADR-0001](0001-fipc-linking-contract.md) | FIPC as the linking contract | Accepted | +| [ADR-0002](0002-mirt-mml-em-engine.md) | mirt MML-EM as the estimation engine | Accepted | +| [ADR-0003](0003-ipd-dif-screening-delegation.md) | IPD/DIF screening is delegated to mirt | Accepted | + +## Related documents + +- Linking contract restatement: + [`docs/fixed-parameter-item-calibration.md`](../fixed-parameter-item-calibration.md) +- Source-paper list with DOIs: [`docs/papers/README.md`](../papers/README.md) +- Repository map: [`ARCHITECTURE.md`](../../ARCHITECTURE.md) diff --git a/docs/fixed-parameter-item-calibration.md b/docs/fixed-parameter-item-calibration.md index efad6448..500488c5 100644 --- a/docs/fixed-parameter-item-calibration.md +++ b/docs/fixed-parameter-item-calibration.md @@ -9,7 +9,15 @@ should move onto that base scale. This follows the fixed parameter calibration framing in Kim (2006): old operational or anchor item parameters are treated as known values during the new-form calibration so the new form is calibrated directly on the established -scale. The package test `test-fixed-parameter-calibration.R` reproduces this +scale. FIPC is one published linking design among others. Separate +calibration plus a Stocking and Lord (1983) or Haebara (1980) +characteristic-curve transformation, and concurrent calibration of both +forms, are alternatives surveyed by Kolen and Brennan (2014). +`autoFIPC()` implements FIPC only: it copies old-form anchor values, +holds them fixed, and re-estimates free new-form parameters in `mirt`. +It does not estimate a Stocking–Lord or Haebara linking transformation. + +The package test `test-fixed-parameter-calibration.R` reproduces this contract with generated 2PL data: 1. Generate old-form and new-form responses from known true item parameters. @@ -70,12 +78,75 @@ pinned to hand-computed reference values in contract (anchors fixed to old-form values, non-anchors left free) is pinned in `tests/testthat/test-fixed-parameter-calibration.R`. +## Relation to other linking methods + +Kolen and Brennan (2014) organize common IRT linking designs as: + +- **Separate calibration + characteristic-curve transformation.** + Each form is calibrated freely. A linear transformation is then + chosen to match test characteristic curves (Stocking & Lord, 1983) + or item characteristic curves (Haebara, 1980). +- **Concurrent calibration.** Both forms are estimated in one run with + shared parameters for common items. +- **Fixed item parameter calibration (FIPC).** Anchor parameters from + the old form are treated as known and held fixed while the new form + is calibrated onto that scale (Kim, 2006; see also Kim & Kolen, + 2019, for a later multiple-group FIPC application). + +`R/aFIPC.R` implements the third design. There is no Stocking–Lord or +Haebara objective, and no post-calibration slope/intercept estimator. +A previous draft cited a non-existent Kim and Kolen (2010) *Journal of +Educational Measurement* article titled "Linking item parameters to a +base scale." That record is withdrawn. The title belongs to Kang and +Petersen (2012). + +Linked scores still fall under the interpretation limits in the +*Standards for Educational and Psychological Testing* (AERA, APA, & +NCME, 2014). Estimation of free parameters uses `mirt` MML-EM +(Chalmers, 2012; Bock & Aitkin, 1981). See `docs/adr/` for the +accepted method decisions. + ## References -- Kim, S. (2006). A comparative study of IRT fixed parameter calibration - methods. Journal of Educational Measurement, 43(4), 355-381. -- Chalmers, R. P. `mirt::fixedCalib` documentation. The implementation note - describes fixed-item calibration methods based on Kim (2006) and points to - `multipleGroup` for more flexible anchor-item calibration. -- Kim, S., & Kolen, M. J. (2010). Linking item parameters to a base scale. - Journal of Educational Measurement, 47(2), 164-181. +Kim, S. (2006). A comparative study of IRT fixed parameter calibration +methods. *Journal of Educational Measurement, 43*(4), 355–381. + + +Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in +item response theory. *Applied Psychological Measurement, 7*(2), +201–210. + +Haebara, T. (1980). Equating logistic ability scales by a weighted +least squares method. *Japanese Psychological Research, 22*(3), +144–149. + +Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and +linking: Methods and practices* (3rd ed.). Springer. + + +Kim, S., & Kolen, M. J. (2019). Application of IRT fixed parameter +calibration to multiple-group test data. *Applied Measurement in +Education, 32*(4), 310–324. + + +Kang, T., & Petersen, N. S. (2012). Linking item parameters to a base +scale. *Asia Pacific Education Review, 13*(2), 311–321. + + +Chalmers, R. P. (2012). mirt: A multidimensional item response theory +package for the R environment. *Journal of Statistical Software, +48*(6), 1–29. + +Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood +estimation of item parameters: Application of an EM algorithm. +*Psychometrika, 46*(4), 443–459. + + +American Educational Research Association, American Psychological +Association, & National Council on Measurement in Education. (2014). +*Standards for educational and psychological testing*. American +Educational Research Association. + +Chalmers, R. P. `mirt::fixedCalib` documentation. The implementation +note describes fixed-item calibration methods based on Kim (2006) and +points to `multipleGroup` for more flexible anchor-item calibration. diff --git a/docs/papers/README.md b/docs/papers/README.md index 75b62728..5f2553ad 100644 --- a/docs/papers/README.md +++ b/docs/papers/README.md @@ -1,46 +1,102 @@ # Source papers for the calibration and linking mathematics -`autoFIPC()` implements the fixed item parameter calibration (FIPC) linking -contract; the underlying IRT estimation is delegated to `mirt`. The canonical -equations this package must match are documented in the sources below. +`autoFIPC()` implements the fixed item parameter calibration (FIPC) +linking contract; the underlying IRT estimation is delegated to `mirt`. +The canonical sources this package must match are listed below. -No PDFs are committed here: the two primary FIPC references (Kim, 2006; Kim & -Kolen, 2010) are published in the *Journal of Educational Measurement* and are -not open access, so they are cited by DOI rather than redistributed, to respect -copyright. The `mirt` reference is open access. +No PDFs are committed here. Copyrighted articles are cited by DOI +rather than redistributed. The `mirt` reference is open access. + +A previous draft listed Kim and Kolen (2010), "Linking item parameters +to a base scale," *Journal of Educational Measurement, 47*(2), +164–181. That bibliographic record is not a real JEM article and is +not kept. The title belongs to Kang and Petersen (2012). A real Kim +and Kolen FIPC paper is Kim and Kolen (2019). ## Primary source — the FIPC linking contract -- **Kim, S. (2006). A comparative study of IRT fixed parameter calibration - methods.** *Journal of Educational Measurement, 43*(4), 355-381. +- **Kim, S. (2006). A comparative study of IRT fixed parameter + calibration methods.** *Journal of Educational Measurement, 43*(4), + 355–381. DOI: - - Canonical rule implemented: old-form (anchor) item parameters are treated as - known and held fixed while the new form is calibrated directly onto the - established base scale. + - Canonical rule implemented: old-form (anchor) item parameters are + treated as known and held fixed while the new form is calibrated + directly onto the established base scale. + +- **Kim, S., & Kolen, M. J. (2019). Application of IRT fixed parameter + calibration to multiple-group test data.** *Applied Measurement in + Education, 32*(4), 310–324. + DOI: + - Later FIPC application to multiple-group data. Supports FIPC as a + published method family; does not replace Kim (2006) as the + contract `autoFIPC()` implements. + +## Characteristic-curve equating (not implemented here) + +`autoFIPC()` implements FIPC, not a Stocking–Lord or Haebara +transformation estimator. These papers are the canonical +characteristic-curve methods that FIPC is an alternative to (separate +calibration plus a linking transformation, versus concurrent +calibration, versus FIPC). + +- **Stocking, M. L., & Lord, F. M. (1983). Developing a common metric + in item response theory.** *Applied Psychological Measurement, + 7*(2), 201–210. + DOI: + - Test characteristic-curve linking after separate calibration. -- **Kim, S., & Kolen, M. J. (2010). Linking item parameters to a base scale.** - *Journal of Educational Measurement, 47*(2), 164-181. - DOI: - - Basis for treating the old-form scale as the fixed base onto which new-form - parameters are placed. +- **Haebara, T. (1980). Equating logistic ability scales by a weighted + least squares method.** *Japanese Psychological Research, 22*(3), + 144–149. + DOI: + - Item characteristic-curve linking by weighted least squares after + separate calibration. + +## Equating and linking handbook + +- **Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, + and linking: Methods and practices* (3rd ed.).** Springer. + DOI: + - Handbook survey of equating, scaling, and linking designs, + including the families contrasted above. + +## Correct source for the withdrawn title + +- **Kang, T., & Petersen, N. S. (2012). Linking item parameters to a + base scale.** *Asia Pacific Education Review, 13*(2), 311–321. + DOI: + - Real paper with this title (also circulated as ACT Research Report + 2009-2). Not the FIPC contract implemented here. ## Estimation engine (open access) -- **Chalmers, R. P. (2012). mirt: A Multidimensional Item Response Theory - Package for the R Environment.** *Journal of Statistical Software, 48*(6), - 1-29. DOI: (open access) - - Provides the MML-EM estimation, `fscores` (MAP), `expected.test`, and the - `multipleGroup`/`DIF` machinery used for item parameter drift screening. - - `?mirt::fixedCalib` documents fixed-item calibration methods based on - Kim (2006). +- **Chalmers, R. P. (2012). mirt: A multidimensional item response + theory package for the R environment.** *Journal of Statistical + Software, 48*(6), 1–29. + DOI: (open access) + - Provides the MML-EM estimation, `fscores` (MAP), `expected.test`, + and the `multipleGroup`/`DIF` machinery used for item parameter + drift screening. + - `?mirt::fixedCalib` documents fixed-item calibration methods based + on Kim (2006). + +- **Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood + estimation of item parameters: Application of an EM algorithm.** + *Psychometrika, 46*(4), 443–459. + DOI: + - The MML-EM algorithm underlying `mirt`'s `method = "EM"` + estimation. -## Supporting reference +## Score-scale interpretation limits -- **Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of - item parameters: Application of an EM algorithm.** *Psychometrika, 46*(4), - 443-459. DOI: - - The MML-EM algorithm underlying `mirt`'s `method = "EM"` estimation. +- **American Educational Research Association, American Psychological + Association, & National Council on Measurement in Education. + (2014). *Standards for educational and psychological testing*.** + American Educational Research Association. + - Limits on how linked scores and scales may be interpreted. No DOI + is used here; none was verified for this edition. -See `../fixed-parameter-item-calibration.md` for the equation restatement, the -formula-integrity audit of the performance refactors, and the tests that pin -these formulas to reference values. +See `../fixed-parameter-item-calibration.md` for the equation +restatement, the formula-integrity audit of the performance refactors, +and the tests that pin these formulas to reference values. Method +decisions are recorded in `../adr/`. From 7451510b79801a25dd2b747f22ed49f77a813bc3 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 17 Aug 2026 15:54:28 +0000 Subject: [PATCH 02/20] docs: point agent guides at ADRs and verified citations Link AGENTS.md and CLAUDE.md to docs/adr/ and docs/papers/README.md so the FIPC contract, mirt engine boundary, and withdrawn 2010 JEM record stay discoverable to agents. Co-authored-by: Seongho Bae --- AGENTS.md | 12 ++++++++++++ CLAUDE.md | 7 +++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b60b03df..ff8bd391 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,4 +106,16 @@ Applies to every agent (Claude, Codex, Cursor, opencode, ...) working in this re - For substantive calibration/linking changes, cite the relevant IRT and psychometrics literature. Commit paper PDFs only when redistribution is permitted; otherwise cite, link, and summarize. +- Method decisions are recorded in `docs/adr/`. Verified APA 7th records + and DOIs are in `docs/papers/README.md`. Do not invent bibliographic + records or leave empty `DOI:` placeholders. +- The implemented linking contract is FIPC (Kim, 2006): anchors keep + old-form values. `autoFIPC()` does not estimate a Stocking–Lord (1983) + or Haebara (1980) transformation (ADR-0001). Estimation is `mirt` + MML-EM (ADR-0002). IPD/DIF screening is delegated to `mirt` and is + not a published invariance claim (ADR-0003). +- Do not restore Kim and Kolen (2010), "Linking item parameters to a + base scale," *Journal of Educational Measurement*. That record is not + a real JEM article. The title is Kang and Petersen (2012). A real + Kim and Kolen FIPC paper is Kim and Kolen (2019). diff --git a/CLAUDE.md b/CLAUDE.md index 264efb15..23119f0e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -86,8 +86,9 @@ score/theta outputs, returned as an R list. The linking contract (Kim, 2006): anchor items keep their old-form parameter values fixed during new-form calibration so the new form is calibrated -directly onto the established scale. It is documented in -`docs/fixed-parameter-item-calibration.md` and enforced by +directly onto the established scale. That is FIPC, not a Stocking–Lord or +Haebara transformation. It is documented in +`docs/fixed-parameter-item-calibration.md` and `docs/adr/`, and enforced by `tests/testthat/test-fixed-parameter-calibration.R`. ## Key conventions @@ -123,3 +124,5 @@ Summarized from `AGENTS.md` and `CONTRIBUTING.md`; read those for detail. - `CONTRIBUTING.md` — contribution process and verification baseline - `docs/operations/maintenance-runbook.md` — recurring maintainer operations - `docs/fixed-parameter-item-calibration.md` — calibration/linking basis +- `docs/adr/` — FIPC, mirt MML-EM, and IPD/DIF decision records +- `docs/papers/README.md` — verified source papers and DOIs From 2ea4d91c182ad6f90e4ac6d04639d1e3914bd92b Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 17 Aug 2026 17:23:46 +0000 Subject: [PATCH 03/20] ci: retrigger Strix and Noema after GitHub API flake Org required workflows failed before scanning this PR: Strix could not resolve repo visibility because installation 141441800 hit the GitHub API rate limit (HTTP 403), and Noema died on a GraphQL 503. No package or citation change. Co-authored-by: Seongho Bae From 7b711e73aad8de5aa492a6d6866b10b8849d1c8d Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 18 Aug 2026 00:23:28 +0000 Subject: [PATCH 04/20] ci: retrigger Noema now that GitHub.com is operational Required noema-review still shows the HTTP 503 from the earlier GitHub.com outage (gh api graphql). Package checks already passed. Empty commit retriggers the org required review after status returned to All Systems Operational. Co-authored-by: Seongho Bae From 484f66325eff1b53cf506dd7bb21bdf33a42de59 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 31 Aug 2026 19:34:35 +0900 Subject: [PATCH 05/20] docs: restore Ground Rules list indentation --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index da361caf..99c6e5a3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,7 +8,7 @@ over feature velocity. ## Ground Rules 1. Avoid behavior changes in `R/aFIPC.R` unless backed by explicit regression - evidence and maintainer approval. + evidence and maintainer approval. 2. Prefer small, auditable pull requests. 3. Keep CI/security/docs healthy (`.github/workflows/`, `README.md`, `ARCHITECTURE.md`, `AGENTS.md`). From 53b7d39dbd16a6a1f47c0814a7dbd6783c567b3f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 07:30:35 +0900 Subject: [PATCH 06/20] docs: add organization DeepWiki badge --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index ae25ed00..5a1edf6f 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ Automated Fixed Item Parameter Calibration (FIPC) for IRT test linking. +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/aFIPC) + This package contains the original graduate-school implementation used to produce accurate fixed-item linking results. The current maintenance goal is to preserve numerical behavior while modernizing repository operations From c09c220d36f010cf523ea50e846d02b5cc035457 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 07:30:47 +0900 Subject: [PATCH 07/20] docs(pages): add public aFIPC landing source --- docs/index.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..6dae9edb --- /dev/null +++ b/docs/index.md @@ -0,0 +1,41 @@ +# aFIPC + +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/aFIPC) + +**aFIPC** provides automated Fixed Item Parameter Calibration (FIPC) for item-response-theory test linking. It preserves common-item parameters from an old form while calibrating new-form items onto the established scale, with estimation delegated to `mirt`. + +## Product scope + +- `autoFIPC()` implements the package's FIPC linking workflow. +- Common items retain old-form parameter values during linked calibration. +- The package uses `mirt` marginal maximum-likelihood / EM estimation rather than implementing a separate numerical IRT engine. +- FIPC is intentionally distinct from separate calibration followed by Stocking–Lord or Haebara characteristic-curve transformations and from concurrent calibration. +- Item-parameter-drift / DIF screening is delegated to the supported `mirt` machinery documented by the repository. + +## Start here + +Install or inspect the package from the organization-owned repository: + +```r +# install.packages("remotes") +remotes::install_github("ContextualWisdomLab/aFIPC") +``` + +For local development and verification, follow the repository README and contribution guidance rather than relying on the historical vendored Packrat installation tree. + +## Documentation + +- [Repository README](https://github.com/ContextualWisdomLab/aFIPC/blob/master/README.md) — package purpose, development status, and local package checks. +- [Architecture](https://github.com/ContextualWisdomLab/aFIPC/blob/master/ARCHITECTURE.md) — runtime and maintenance boundaries. +- [FIPC linking contract](fixed-parameter-item-calibration.md) — what is fixed, what is estimated, and how the linked scale is defined. +- [Architecture decisions](adr/) — reviewed decisions for FIPC-only linking, `mirt` MML-EM estimation, and IPD/DIF delegation. +- [Research sources](papers/) — verified methodological references and DOIs. +- [Contributing](https://github.com/ContextualWisdomLab/aFIPC/blob/master/CONTRIBUTING.md) — development and verification expectations. +- [GitHub Releases](https://github.com/ContextualWisdomLab/aFIPC/releases) — release history when available. +- [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/aFIPC) — repository-aware questions about the code and documentation. + +## Evidence boundary + +aFIPC is psychometric research software. Linking quality depends on anchor quality, model fit, calibration assumptions, and the comparability of the forms being linked. Repository tests and cited methodology provide implementation evidence; they do not make every pair of operational test forms automatically comparable. + +This page is suitable as the source for a minimal GitHub Pages site once the organization-owned metadata/Pages reconciler can publish the repository safely. A source commit alone is not evidence that Pages is live. From c7f31fe409af75b09a1bf88335227d7a65e43382 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:10:12 +0900 Subject: [PATCH 08/20] docs: consolidate product landing and license boundary --- README.md | 245 +++++++++++++++++++++++++++++++++--------------------- 1 file changed, 151 insertions(+), 94 deletions(-) diff --git a/README.md b/README.md index 5a1edf6f..220500f9 100644 --- a/README.md +++ b/README.md @@ -1,105 +1,162 @@ # aFIPC -Automated Fixed Item Parameter Calibration (FIPC) for IRT test linking. - [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/aFIPC) -This package contains the original graduate-school implementation used to -produce accurate fixed-item linking results. The current maintenance goal is to -preserve numerical behavior while modernizing repository operations -(documentation, CI, and dependency hygiene). - -## Methodological sources - -`autoFIPC()` implements fixed item parameter calibration (FIPC): common -items keep their old-form parameter values while the new form is -calibrated onto that scale (Kim, 2006). That is a different design from -separate calibration plus a Stocking and Lord (1983) or Haebara (1980) -characteristic-curve transformation, and from concurrent calibration -(Kolen & Brennan, 2014). This package does not estimate Stocking–Lord -or Haebara linking constants. - -Estimation is delegated to `mirt` MML-EM (Chalmers, 2012; Bock & -Aitkin, 1981). Linked-score interpretation is bounded by AERA, APA, and -NCME (2014). Full APA records and DOIs are in -`docs/papers/README.md`; method decisions are in `docs/adr/`. - -Kim, S. (2006). A comparative study of IRT fixed parameter calibration -methods. *Journal of Educational Measurement, 43*(4), 355–381. - - -Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in -item response theory. *Applied Psychological Measurement, 7*(2), -201–210. - -Haebara, T. (1980). Equating logistic ability scales by a weighted -least squares method. *Japanese Psychological Research, 22*(3), -144–149. - -Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and -linking: Methods and practices* (3rd ed.). Springer. - - -Chalmers, R. P. (2012). mirt: A multidimensional item response theory -package for the R environment. *Journal of Statistical Software, -48*(6), 1–29. - -Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood -estimation of item parameters: Application of an EM algorithm. -*Psychometrika, 46*(4), 443–459. - - -American Educational Research Association, American Psychological -Association, & National Council on Measurement in Education. (2014). -*Standards for educational and psychological testing*. American -Educational Research Association. - -## What this repository contains - -- `R/aFIPC.R`: core `autoFIPC()` implementation -- `DESCRIPTION`, `NAMESPACE`, `man/`: package metadata and generated docs -- `docs/adr/`: architecture decision records for FIPC and estimation -- `docs/fixed-parameter-item-calibration.md`: linking-contract restatement -- `docs/papers/README.md`: verified source papers and DOIs -- `packrat/`: historical dependency lock/vendor directory -- `.github/workflows/`: CI/security automation - -## Development status - -- Algorithmic core is legacy but trusted for historical outputs. -- Operational guardrails are now maintained via GitHub Actions and Dependabot. -- Legacy `packrat` bootstrap is opt-in via `AFIPC_ENABLE_PACKRAT=true`. -- Broken host-specific `packrat/lib-R` symlinks were removed for portable builds. -- Architectural and agent operation docs are available in: - - `ARCHITECTURE.md` - - `AGENTS.md` - - `CLAUDE.md` - - `CONTRIBUTING.md` - - `.github/SECURITY.md` - -## Collaboration workflow - -- Pull request template: `.github/PULL_REQUEST_TEMPLATE.md` -- Issue templates: `.github/ISSUE_TEMPLATE/` -- Code ownership: `.github/CODEOWNERS` -- Code quality checks: `.github/workflows/code-quality.yml` -- Security checks (private-safe): `.github/workflows/security-audit.yml` -- Secret-scan policy config: `.gitleaks.toml` -- CodeRabbit command reference: `docs/coderabbit/review-commands.md` -- Maintainer operations runbook: `docs/operations/maintenance-runbook.md` - -## Local package check +**Fixed-item IRT calibration for keeping test forms on a common score scale.** + +aFIPC automates fixed-item parameter calibration (FIPC) for linking and equating +workflows. It helps psychometric teams carry anchor-item information from a +reference form into calibration of a new form, inspect item-parameter drift, and +obtain linked model/score artifacts without turning the repository into a +general assessment platform. + +## When aFIPC fits + +Use aFIPC when you have a reference form, a newly administered form, and an +explicitly reviewed set of common items whose parameters should anchor the +linking design. The package owns the in-process calibration/linking workflow. +Test delivery, source-system data collection, operational score policy, and +downstream decision authority remain outside this repository. + +The numerical implementation is compatibility-sensitive. Changes to calibration +behavior should be backed by regression evidence rather than incidental +refactoring. + +## Core workflow + +1. Prepare old/reference-form and new-form response data, or compatible fitted + model objects. +2. Identify the corresponding common-item names on both forms. +3. Run `autoFIPC()` with the intended item model and explicit common-item + confirmation. +4. Review convergence, item-parameter-drift evidence, and linked outputs before + downstream use. + +A minimal API shape is: + +```r +result <- autoFIPC( + newformXData = new_form, + oldformYData = reference_form, + newformCommonItemNames = common_new, + oldformCommonItemNames = common_old, + confirmCommonItems = TRUE +) +``` + +`autoFIPC()` returns the base-form, new-form, and linked-model artifacts as an R +list. See `man/autoFIPC.Rd` for the complete argument contract. + +## Methodological boundary + +`autoFIPC()` implements FIPC: common items keep reference-form parameter values +while the new form is calibrated onto that scale (Kim, 2006). This differs from +separate calibration followed by Stocking-Lord or Haebara characteristic-curve +transformations, and from concurrent calibration. aFIPC does not estimate +Stocking-Lord or Haebara linking constants. + +The current estimation path delegates IRT estimation to `mirt` rather than +owning an independent numerical IRT engine. The accepted method decisions are +recorded in [the ADR index](docs/adr/README.md), and the verified APA records and +DOIs are maintained in [the research index](docs/papers/README.md). + +Key sources include: + +- Kim, S. (2006). A comparative study of IRT fixed parameter calibration + methods. *Journal of Educational Measurement, 43*(4), 355-381. + +- Chalmers, R. P. (2012). mirt: A multidimensional item response theory package + for the R environment. *Journal of Statistical Software, 48*(6), 1-29. + +- Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of + item parameters: Application of an EM algorithm. *Psychometrika, 46*(4), + 443-459. + +Linking quality still depends on anchor quality, model fit, calibration +assumptions, and the comparability of the forms being linked. Repository tests +and cited methodology are implementation evidence; they do not make arbitrary +forms automatically comparable. + +## Evaluate from source + +This repository currently provides a source package rather than a published +GitHub release. For contributor/evaluation checks, use a clean R profile and the +same `rcmdcheck` path exercised by repository automation: ```bash R_PROFILE_USER=/dev/null Rscript -e \ -'install.packages(c("rcmdcheck"), repos="https://cloud.r-project.org")' +'install.packages("rcmdcheck", repos="https://cloud.r-project.org")' + R_PROFILE_USER=/dev/null Rscript -e \ 'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning")' ``` -## Maintenance policy - -- Prefer preserving equation/calibration behavior over refactoring. -- Avoid silent behavioral changes in `autoFIPC()` without explicit regression - evidence. -- Keep CI green on supported runners and keep Actions pinned/updated. +The historical `packrat/` tree is retained for compatibility archaeology. Its +bootstrap is opt-in via `AFIPC_ENABLE_PACKRAT=true` and should not be treated as +the preferred dependency workflow. + +## Architecture and responsibility + +The runtime is single-process and fileless: callers provide in-memory R +data/model objects and receive R model artifacts. `R/aFIPC.R` owns the main +fixed-item linking workflow, while `R/surveyFA.R` contains supporting analytical +routines. Package metadata and generated reference docs live in `DESCRIPTION`, +`NAMESPACE`, and `man/`. + +The current calibration engine calls the external `mirt` package for IRT +estimation and parameter/model operations. That dependency is a material runtime +boundary, not an implementation detail that the repository license can override. + +See [ARCHITECTURE.md](ARCHITECTURE.md) for the component map and maintenance +boundaries. + +## Quality and change control + +Repository automation covers R package checks, code/document quality, +workflow/security auditing, dependency review where the platform supports it, +and supply-chain posture. Before modifying estimation/linking logic, read +[CONTRIBUTING.md](CONTRIBUTING.md) and preserve the documented regression and +review discipline. + +Current source metadata is `0.1.0`; the repository does not currently publish +GitHub Releases. A source version or passing development check is therefore not +presented as an immutable supported release. + +## Commercial licensing status + +**Not currently cleared for ContextualWisdomLab commercial +intake/distribution.** + +`DESCRIPTION` presently declares `GPL-3 | file LICENSE`, while the runtime +directly imports `mirt`, whose current CRAN distribution is GPL-family licensed. +ContextualWisdomLab's commercial intake policy does not accept GPL/LGPL/AGPL +family software as the normal dependency baseline. The repository-authored +source grant and third-party runtime obligation are separate questions; neither +can be made policy-compliant by README wording alone. + +Issue #320 owns the required source-provenance/relicensing review and replacement +of the GPL-family runtime path while preserving the actual fixed-item +calibration/linking estimand and regression behavior. Until that work is +integrated and verified, do not present this repository as Apache-2.0/MIT +cleared or commercially policy-compliant. + +## Documentation + +- [Public documentation home](docs/index.md) - product scope, workflow, + architecture, and change-control entry point. +- [FIPC linking contract](docs/fixed-parameter-item-calibration.md) - what is + fixed, what is estimated, and how the linked scale is defined. +- [Architecture decisions](docs/adr/README.md) - FIPC and estimation decisions. +- [Research sources](docs/papers/README.md) - verified methodological references + and DOIs. +- [Architecture](ARCHITECTURE.md) - component and responsibility boundaries. +- [Generated R reference](man/autoFIPC.Rd) - `autoFIPC()` API contract. +- [Contributing](CONTRIBUTING.md) - contributor workflow and verification + expectations. +- [.github/SECURITY.md](.github/SECURITY.md) - vulnerability-reporting guidance. +- [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/aFIPC) - + repository-aware navigation. + +For scientific or operational changes, open an issue or pull request with the +exact design assumptions, affected calibration behavior, and reproducible +verification evidence. From 06154dee1469120556b36faa59ca20afc0b6c11f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:10:27 +0900 Subject: [PATCH 09/20] docs: align public guide with canonical README --- docs/index.md | 102 +++++++++++++++++++++++++++++++++++++------------- 1 file changed, 77 insertions(+), 25 deletions(-) diff --git a/docs/index.md b/docs/index.md index 6dae9edb..919ad1be 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,40 +2,92 @@ [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/aFIPC) -**aFIPC** provides automated Fixed Item Parameter Calibration (FIPC) for item-response-theory test linking. It preserves common-item parameters from an old form while calibrating new-form items onto the established scale, with estimation delegated to `mirt`. +Automated Fixed Item Parameter Calibration for item-response-theory linking and +equating. -## Product scope +## Product responsibility -- `autoFIPC()` implements the package's FIPC linking workflow. -- Common items retain old-form parameter values during linked calibration. -- The package uses `mirt` marginal maximum-likelihood / EM estimation rather than implementing a separate numerical IRT engine. -- FIPC is intentionally distinct from separate calibration followed by Stocking–Lord or Haebara characteristic-curve transformations and from concurrent calibration. -- Item-parameter-drift / DIF screening is delegated to the supported `mirt` machinery documented by the repository. +aFIPC preserves a common measurement scale across test forms by combining +anchor-item information from a reference form with calibration of newly +administered items. It is intended for psychometric workflows where score +comparability across administrations matters and fixed-item linking is the +chosen design. -## Start here +The repository owns the in-process R calibration/linking workflow. Test +delivery, source-system data collection, operational score policy, and +downstream decision authority remain outside this package. -Install or inspect the package from the organization-owned repository: +## Core workflow -```r -# install.packages("remotes") -remotes::install_github("ContextualWisdomLab/aFIPC") +1. Prepare reference-form and new-form response data or compatible fitted model + objects. +2. Define the reviewed common-item correspondence between forms. +3. Run `autoFIPC()` with the intended item model and explicit common-item + confirmation. +4. Review convergence, item-parameter-drift evidence, and linked outputs before + downstream score reporting or operational use. + +## Method and architecture + +- `R/aFIPC.R` contains the main fixed-item linking workflow. +- `R/surveyFA.R` contains supporting analytical routines used by the package. +- `DESCRIPTION`, `NAMESPACE`, and `man/` define package metadata and generated + reference documentation. +- `docs/adr/` records reviewed method/architecture decisions. +- `docs/papers/` retains verified methodological references and DOIs. +- `.github/workflows/` provides continuous-integration and security checks. + +The current calibration engine directly uses the external `mirt` runtime for +IRT estimation and parameter/model operations. FIPC is distinct from separate +calibration plus Stocking-Lord/Haebara transformations and from concurrent +calibration. See the FIPC contract and ADR index for the precise boundary. + +## Onboarding and verification + +Clone the repository, use a current R toolchain, and run the package checks +before changing calibration behavior: + +```bash +R_PROFILE_USER=/dev/null Rscript -e \ +'install.packages("rcmdcheck", repos="https://cloud.r-project.org")' +R_PROFILE_USER=/dev/null Rscript -e \ +'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning")' ``` -For local development and verification, follow the repository README and contribution guidance rather than relying on the historical vendored Packrat installation tree. +For contributor expectations and architectural context, read `CONTRIBUTING.md` +and `ARCHITECTURE.md` before modifying estimation or linking logic. + +## Release and commercial-license status + +The repository currently has no published GitHub Release. Source metadata +`0.1.0` and development checks are not immutable release evidence. -## Documentation +The current source/dependency graph is also **not cleared for +ContextualWisdomLab commercial intake/distribution**: package metadata declares +`GPL-3 | file LICENSE`, and the runtime directly imports the GPL-family `mirt` +package. Issue #320 owns the source-provenance/relicensing review plus +replacement of that runtime dependency while preserving the actual fixed-item +calibration/linking contract. Until that work is complete, do not describe +aFIPC as Apache-2.0/MIT-cleared or commercially policy-compliant. -- [Repository README](https://github.com/ContextualWisdomLab/aFIPC/blob/master/README.md) — package purpose, development status, and local package checks. -- [Architecture](https://github.com/ContextualWisdomLab/aFIPC/blob/master/ARCHITECTURE.md) — runtime and maintenance boundaries. -- [FIPC linking contract](fixed-parameter-item-calibration.md) — what is fixed, what is estimated, and how the linked scale is defined. -- [Architecture decisions](adr/) — reviewed decisions for FIPC-only linking, `mirt` MML-EM estimation, and IPD/DIF delegation. -- [Research sources](papers/) — verified methodological references and DOIs. -- [Contributing](https://github.com/ContextualWisdomLab/aFIPC/blob/master/CONTRIBUTING.md) — development and verification expectations. -- [GitHub Releases](https://github.com/ContextualWisdomLab/aFIPC/releases) — release history when available. -- [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/aFIPC) — repository-aware questions about the code and documentation. +## Documentation and support -## Evidence boundary +- [Repository README](https://github.com/ContextualWisdomLab/aFIPC/blob/master/README.md) + - product, usage, status, and contributor entry point. +- [Architecture](https://github.com/ContextualWisdomLab/aFIPC/blob/master/ARCHITECTURE.md) + - runtime and maintenance boundaries. +- [FIPC linking contract](fixed-parameter-item-calibration.md) - linking design. +- [Architecture decisions](adr/README.md) - method and ownership decisions. +- [Research sources](papers/README.md) - verified references and DOIs. +- [Contributing](https://github.com/ContextualWisdomLab/aFIPC/blob/master/CONTRIBUTING.md) + - development and verification expectations. +- [Ask DeepWiki](https://deepwiki.com/ContextualWisdomLab/aFIPC) - repository-aware + documentation and code navigation. -aFIPC is psychometric research software. Linking quality depends on anchor quality, model fit, calibration assumptions, and the comparability of the forms being linked. Repository tests and cited methodology provide implementation evidence; they do not make every pair of operational test forms automatically comparable. +Linking quality depends on anchor quality, model fit, calibration assumptions, +and the comparability of the forms being linked. Repository tests and cited +methodology are implementation evidence; they do not make arbitrary test forms +automatically comparable. -This page is suitable as the source for a minimal GitHub Pages site once the organization-owned metadata/Pages reconciler can publish the repository safely. A source commit alone is not evidence that Pages is live. +This file is a Pages-ready source only. It is not evidence that GitHub Pages is +published; publication requires repository settings and live HTTPS verification. From dabe29d32ab1959fae1ee9f80d0dba6725b781a1 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:12:50 +0900 Subject: [PATCH 10/20] docs: describe mirt estimation paths precisely --- docs/adr/0002-mirt-mml-em-engine.md | 106 +++++++++++++++++----------- 1 file changed, 63 insertions(+), 43 deletions(-) diff --git a/docs/adr/0002-mirt-mml-em-engine.md b/docs/adr/0002-mirt-mml-em-engine.md index 271cd5c0..b310ecc4 100644 --- a/docs/adr/0002-mirt-mml-em-engine.md +++ b/docs/adr/0002-mirt-mml-em-engine.md @@ -1,4 +1,4 @@ -# ADR-0002: mirt MML-EM as the estimation engine +# ADR-0002: mirt as the estimation engine - Status: Accepted - Date: 2026-08-16 @@ -12,70 +12,90 @@ item parameters and the ability distribution. This package is an R orchestrator, not a new IRT estimator. Chalmers (2012) describes `mirt`, the package `DESCRIPTION` imports. -Bock and Aitkin (1981) give the marginal maximum likelihood EM -(MML-EM) algorithm that `mirt` implements when `method = "EM"`. -`autoFIPC()` uses that path when `tryEM` is true (the default) or when -the item type is nominal. +Bock and Aitkin (1981) give the marginal maximum-likelihood EM +(MML-EM) basis used when `mirt` is called with `method = "EM"`. + +The implementation does not use one estimation method for every model +artifact. The method policy has three distinct stages: + +1. When raw old/new form data must first be fitted, `autoFIPC()` starts with + the ordinary `mirt` path. If fit diagnostics remain unacceptable and + whole-form retry is enabled, it may retry QMCEM, then MHRM, and finally + `surveyFA()` recovery variants. These fallbacks are independent of the + later linked-fit `tryEM` choice. +2. The linked FIPC fit uses EM when `itemtype == "nominal"` or `tryEM` is + true. Otherwise it uses MHRM. +3. IPD/DIF screening follows the same EM-versus-MHRM selection rule as the + linked fit: EM for nominal items or `tryEM = TRUE`, MHRM otherwise. + +Accordingly, an `autoFIPC()` result can legitimately contain old/new form +models fitted by QMCEM or MHRM even when the linked model uses EM. Documentation +must not label every returned model as MML-EM. ## Decision -Use `mirt` as the estimation engine and MML-EM as the default -calibration method: - -- Separate old-form and new-form fits call `mirt::mirt`. -- The linked fit calls `mirt::mirt` with `pars` after the FIPC - copy-and-fix step. -- Default linked estimation uses `method = "EM"` (Bock & Aitkin, - 1981, as implemented by Chalmers, 2012). -- Scores and expected-score artifacts use `mirt` helpers such as - `fscores` and `expected.test`. - -This package does not reimplement the likelihood, quadrature, or EM -cycles. +Use `mirt` as the estimation engine while keeping the method of each model +artifact explicit: + +- Separate old-form and new-form raw-data fits call `mirt::mirt` and may use + the bounded recovery sequence described above when their initial fit is not + acceptable. +- The linked fit calls `mirt::mirt` with `pars` after the FIPC copy-and-fix + step. +- Default linked estimation uses `method = "EM"` because `tryEM` defaults to + true; non-nominal linked estimation uses `method = "MHRM"` when + `tryEM = FALSE`. +- IPD/DIF uses `mirt::multipleGroup` and `mirt::DIF` with the same explicit + EM/MHRM branch. +- Scores and expected-score artifacts use `mirt` helpers such as `fscores` + and `expected.test`. + +This package does not reimplement the likelihood, quadrature, EM, QMCEM, or +MHRM algorithms. ## Alternatives considered -- **A custom MML-EM implementation in `R/aFIPC.R`.** Would duplicate - a maintained estimator and risk silent numerical drift. Rejected. -- **A different IRT package.** Historical outputs were produced with - `mirt`. Changing engines would be a scientific behavior change, not - a docs fix. -- **`mirt` MHRM (`method = "MHRM"`).** `autoFIPC()` can take this - path when `tryEM` is false and the item type is not nominal. It is - an optional `mirt` method, not the default FIPC engine documented - here. +- **A custom estimator in `R/aFIPC.R`.** This would duplicate a maintained + estimator and risk silent numerical drift. Rejected. +- **A different IRT package.** Historical outputs were produced with `mirt`. + Changing engines would be a scientific behavior change, not a docs fix. +- **Calling every result “MML-EM.”** Rejected because source permits QMCEM and + MHRM for raw-form recovery and MHRM for the linked/IPD branch when the + explicit method policy selects it. ## Consequences -- Numerical changes in `mirt` can change `autoFIPC()` output even when - this repository's R sources are untouched. -- Formula-integrity reviews (see - `docs/fixed-parameter-item-calibration.md`) apply to orchestration - only. Estimation mathematics stay in `mirt`. -- `man/autoFIPC.Rd` remains roxygen-generated from `R/aFIPC.R`. - Method citations belong in these markdown ADRs and - `docs/papers/README.md` unless the roxygen `@references` block is - updated in the same change. +- Numerical changes in `mirt` can change `autoFIPC()` output even when this + repository's R sources are untouched. +- Evidence about a returned model should record its actual estimation path; + `tryEM = TRUE` alone does not prove the separately fitted old/new models used + EM after all recovery attempts. +- Formula-integrity reviews in + `docs/fixed-parameter-item-calibration.md` apply to orchestration only. + Estimation mathematics stay in `mirt`. +- `man/autoFIPC.Rd` remains roxygen-generated from `R/aFIPC.R`. Method + citations belong in these Markdown ADRs and `docs/papers/README.md` unless + the roxygen `@references` block is updated in the same change. ## Claim boundary -Choosing `mirt` and MML-EM is an engineering dependency decision. It -is not a claim that this package contributes a new estimation -algorithm, and it is not a claim about the relative accuracy of EM -versus other `mirt` methods. Linking-scale interpretation remains +Choosing `mirt` is an engineering dependency decision. The EM/MHRM method +selection and raw-form recovery sequence are implementation behavior, not a +claim that this package contributes a new estimation algorithm or that one +method is universally more accurate. Linking-scale interpretation remains bounded by ADR-0001 and AERA/APA/NCME (2014). ## References Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. *Journal of Statistical Software, -48*(6), 1–29. +48*(6), 1-29. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. -*Psychometrika, 46*(4), 443–459. +*Psychometrika, 46*(4), 443-459. Kim, S. (2006). A comparative study of IRT fixed parameter calibration -methods. *Journal of Educational Measurement, 43*(4), 355–381. +methods. *Journal of Educational Measurement, 43*(4), 355-381. From 217027a8a20b703705fb62b7159754be2e3ae5e7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:13:12 +0900 Subject: [PATCH 11/20] docs: align agent guidance with estimator fallbacks --- AGENTS.md | 174 +++++++++++++++++++++++------------------------------- 1 file changed, 74 insertions(+), 100 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0c215200..8085741c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,124 +1,98 @@ # AGENTS.md -## Mission +## Scope -Maintain this repository as a stable, reproducible R package for fixed-item -parameter calibration and test linking. +This repository contains the aFIPC R package and its supporting documentation, +CI, security, and historical dependency material. Preserve scientific behavior +unless a change is backed by explicit regression evidence. -## Non-negotiable guardrails +## Repository priorities -1. Preserve historical numerical behavior in `R/aFIPC.R` unless there is - explicit regression evidence and maintainer intent to change behavior. -2. Treat CI/security/docs hygiene as first-class maintenance work. -3. Keep changes minimal and auditable; prefer additive guardrails over broad - refactoring. +1. Preserve fixed-item calibration semantics and historical numerical behavior. +2. Keep security, package checks, and documentation truthful and reproducible. +3. Prefer small, auditable changes over broad refactors. +4. Keep generated/package metadata consistent with source and documentation. -## Required maintenance checks +## Source boundaries -- Keep `.github/workflows/` green and action SHAs pinned. -- Keep `.github/dependabot.yml` active for GitHub Actions updates. -- Keep `ARCHITECTURE.md` up to date when structure changes. -- Keep README accurate for local verification commands. -- Keep `.cursor/install.sh` revision-agnostic: provision R + CRAN deps only. - Do not `R CMD INSTALL` the current tree in `install` (environment builds - freeze that copy). Use `testthat::test_local()` / `rcmdcheck` on the checkout. +- `R/aFIPC.R` owns the main `autoFIPC()` orchestration flow. +- `R/surveyFA.R` contains supporting analytical recovery routines. +- `DESCRIPTION`, `NAMESPACE`, and `man/` are R package surfaces. +- `docs/adr/` records accepted method/architecture decisions. +- `docs/papers/README.md` records verified methodological sources. +- `packrat/` is historical compatibility material, not the preferred dependency + workflow. -## Editing priorities +Do not silently alter common-item matching, fixed/free parameter semantics, +latent-distribution handling, IPD/DIF screening, estimation method selection, +scoring, or expected-score output. -1. Safety and reproducibility -2. CI reliability -3. Documentation clarity -4. Feature changes +## Change discipline -## High-risk areas +- Treat `R/aFIPC.R` numerical behavior as compatibility-sensitive. +- Add focused regression evidence before intentional scientific behavior changes. +- Keep input validation and fail-closed behavior intact unless a stronger + evidence-backed contract replaces it. +- Do not replace a failing check with a weaker threshold or exclusion. +- Do not invent release, benchmark, customer, certification, or commercial + readiness claims from local or predecessor evidence. -- `R/aFIPC.R` contains long legacy logic with interactive prompts. -- Any changes around calibration/linking constraints can alter scientific output. +## Validation -## Preferred change strategy +Run the repository's actual package/documentation checks. The canonical local R +package check is: -- Add tests/fixtures first when behavior changes are required. -- Isolate operational fixes (workflow/docs/dependency policy) from algorithmic - edits. -- Document assumptions and risk in commit/PR summaries. +```bash +R_PROFILE_USER=/dev/null Rscript -e \ +'install.packages("rcmdcheck", repos="https://cloud.r-project.org")' +R_PROFILE_USER=/dev/null Rscript -e \ +'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning")' +``` -## Product design and Figma +Hosted exact-head checks remain authoritative for merge decisions. A push +invalidates predecessor-head check and review evidence. -- This repo has no frontend, route tree, Figma file, or implemented visual - surface. Do not create UI code or visual artifacts unless a real product - surface is introduced. -- Do not call future Figma work a "design system" unless it includes reusable - foundations, tokens, components, states, accessibility guidance, and code - mapping. Otherwise call it a UI kit, wireframe, or draft. +## Commercial license boundary + +The repository currently declares GPL-family package/runtime licensing and +imports `mirt`. Do not present the source as Apache-2.0/MIT-cleared or +commercially intake-compliant until the provenance/relicensing and dependency +replacement work tracked by issue #320 is integrated and verified. Repository +source licensing and third-party dependency licensing are separate obligations. -## Agent guidance (CWL governance) - -Applies to every agent (Claude, Codex, Cursor, opencode, ...) working in this repo. - -### Security & review gate - -- Every PR runs a central **Security Scan** required check: `osv-scan` + - `dependency-review` (diff-scoped) and `trivy-fs` (repo-wide, CRITICAL/HIGH, - fixable only). It runs against each PR base, **including stacked PRs**. -- A failing `trivy-fs` is a **REAL finding, not a flake.** Read the job log (it - prints each finding's rule id, severity, and file) or the run's SARIF results, - then **remediate**: bump the offending R dependency (this package vendors its - library tree under `packrat/`), or, only for a genuine false positive, add a - narrow, path-scoped, commented entry to `.trivyignore.yaml`. Never weaken or - disable the gate. This repo has no Dockerfile or k8s manifests, so misconfig - findings are not expected; a hit here is a dependency or secret finding. -- The vendored packrat openssl docs include a verified false-positive example - key at `packrat/lib/x86_64-pc-linux-gnu/3.4.1/openssl/doc/keys.html`; keep - its `private-key` suppression path-scoped in `.trivyignore.yaml` and do not - blanket-ignore the rule. -- A local `trivy` scan with a stale DB misses findings: run - `trivy --download-db-only` first, and scan the **merge ref**, not just the PR head. -- The org `code_scanning` ruleset is intentionally **CodeQL-only** (multiple - code-scanning tools cannot converge on one PR ref). Gating is by the Security - Scan **job result**, not the `code_scanning` rule; do not add tools to that rule. - -### Code exploration - -- There is no `.codegraph/` index in this repo, so use normal search - (grep/find) to locate and understand code. If a `.codegraph/` index is added - later, prefer CodeGraph (`codegraph explore ""` or the - code-review-graph MCP tools) before grep/find; it surfaces callers, callees, - and impact that text search misses. - -### This repo's role in the ecosystem - -- **aFIPC**: R IRT package for Fixed-Item Parameter Calibration; feeds - fast-mlsirm's psychometrics. -- The org is an ecosystem around **naruon** (the hub: an email/PIM app that - DOM-decomposes emails and files into a persisted knowledge graph). Every - component is a **standalone program that must ALSO work as a git submodule**, - grown separately and together. -- Sibling components: **waf-ids-ai-soc** (WAF/IDS/AI SOC/LB/APIM), - **clearfolio** (document viewer), **pg-erd-cloud** (ERD tool), - **contextual-orchestrator** (LLM cost/perf/upstream-LB gateway, beyond - LiteLLM), **codec-carver** (STT/omni-modal speech-video codec), - **fast-mlsirm** (LLM-as-a-Judge calibration + evaluation-item quality; uses - aFIPC FIPC + kaefa item-fit), **feelanet-adfs** (passwordless SSO: - OIDC/SCIM/ADFS/LDAP/FIDO2/OAuth2.1, eliminate passwords), **newsdom-api** - (PDF to DOM sidecar), **semantic-data-portal** (upper ontology/catalog/governance - plane with its own graph engine). +## ContextualWisdomLab operating context + +- Keep this component independently usable and composable. A sibling repository + may consume it through a released/versioned boundary, but do not create hidden + cross-repository implementation coupling. +- Sibling components include `fast-mlsirm`, which may consume aFIPC-style + fixed-item calibration concepts in broader psychometric workflows. aFIPC does + not become the owner of those sibling products' orchestration, data, or + deployment authority. +- Cross-product LLM/provider routing, if ever required, belongs to + `contextual-orchestrator`, not this numerical R package. ### Research grounding - For substantive calibration/linking changes, cite the relevant IRT and psychometrics literature. Commit paper PDFs only when redistribution is permitted; otherwise cite, link, and summarize. -- Method decisions are recorded in `docs/adr/`. Verified APA 7th records - and DOIs are in `docs/papers/README.md`. Do not invent bibliographic - records or leave empty `DOI:` placeholders. -- The implemented linking contract is FIPC (Kim, 2006): anchors keep - old-form values. `autoFIPC()` does not estimate a Stocking–Lord (1983) - or Haebara (1980) transformation (ADR-0001). Estimation is `mirt` - MML-EM (ADR-0002). IPD/DIF screening is delegated to `mirt` and is - not a published invariance claim (ADR-0003). -- Do not restore Kim and Kolen (2010), "Linking item parameters to a - base scale," *Journal of Educational Measurement*. That record is not - a real JEM article. The title is Kang and Petersen (2012). A real - Kim and Kolen FIPC paper is Kim and Kolen (2019). +- Method decisions are recorded in `docs/adr/`. Verified APA 7th records and + DOIs are in `docs/papers/README.md`. Do not invent bibliographic records or + leave empty `DOI:` placeholders. +- The implemented linking contract is FIPC (Kim, 2006): anchors keep old-form + values. `autoFIPC()` does not estimate a Stocking-Lord (1983) or Haebara + (1980) transformation (ADR-0001). +- `mirt` owns numerical estimation (ADR-0002), but do not flatten all returned + models into one method label. Raw old/new data fits can recover through QMCEM, + MHRM, and `surveyFA` paths when an initial fit is unacceptable. The linked + FIPC fit and IPD/DIF path use EM for nominal items or `tryEM = TRUE`, and MHRM + otherwise. Record the actual method path when evidence depends on it. +- IPD/DIF screening is delegated to `mirt` and is not a published invariance + claim (ADR-0003). +- Do not restore the earlier incorrect attribution of "Linking item parameters + to a base scale" to Kim and Kolen (2010) in the *Journal of Educational + Measurement*. The title belongs to Kang and Petersen (2012). Kim and Kolen + (2019) is a separate, real FIPC application paper. From 1db6d73965eb0f89e1cd7edea22490f5e7ecb608 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:13:43 +0900 Subject: [PATCH 12/20] docs: restore agent guidance and fix estimation wording --- AGENTS.md | 177 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 103 insertions(+), 74 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 8085741c..1a961475 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,98 +1,127 @@ # AGENTS.md -## Scope +## Mission -This repository contains the aFIPC R package and its supporting documentation, -CI, security, and historical dependency material. Preserve scientific behavior -unless a change is backed by explicit regression evidence. +Maintain this repository as a stable, reproducible R package for fixed-item +parameter calibration and test linking. -## Repository priorities +## Non-negotiable guardrails -1. Preserve fixed-item calibration semantics and historical numerical behavior. -2. Keep security, package checks, and documentation truthful and reproducible. -3. Prefer small, auditable changes over broad refactors. -4. Keep generated/package metadata consistent with source and documentation. +1. Preserve historical numerical behavior in `R/aFIPC.R` unless there is + explicit regression evidence and maintainer intent to change behavior. +2. Treat CI/security/docs hygiene as first-class maintenance work. +3. Keep changes minimal and auditable; prefer additive guardrails over broad + refactoring. -## Source boundaries +## Required maintenance checks -- `R/aFIPC.R` owns the main `autoFIPC()` orchestration flow. -- `R/surveyFA.R` contains supporting analytical recovery routines. -- `DESCRIPTION`, `NAMESPACE`, and `man/` are R package surfaces. -- `docs/adr/` records accepted method/architecture decisions. -- `docs/papers/README.md` records verified methodological sources. -- `packrat/` is historical compatibility material, not the preferred dependency - workflow. +- Keep `.github/workflows/` green and action SHAs pinned. +- Keep `.github/dependabot.yml` active for GitHub Actions updates. +- Keep `ARCHITECTURE.md` up to date when structure changes. +- Keep README accurate for local verification commands. +- Keep `.cursor/install.sh` revision-agnostic: provision R + CRAN deps only. + Do not `R CMD INSTALL` the current tree in `install` (environment builds + freeze that copy). Use `testthat::test_local()` / `rcmdcheck` on the checkout. -Do not silently alter common-item matching, fixed/free parameter semantics, -latent-distribution handling, IPD/DIF screening, estimation method selection, -scoring, or expected-score output. +## Editing priorities -## Change discipline +1. Safety and reproducibility +2. CI reliability +3. Documentation clarity +4. Feature changes -- Treat `R/aFIPC.R` numerical behavior as compatibility-sensitive. -- Add focused regression evidence before intentional scientific behavior changes. -- Keep input validation and fail-closed behavior intact unless a stronger - evidence-backed contract replaces it. -- Do not replace a failing check with a weaker threshold or exclusion. -- Do not invent release, benchmark, customer, certification, or commercial - readiness claims from local or predecessor evidence. +## High-risk areas -## Validation +- `R/aFIPC.R` contains long legacy logic with interactive prompts. +- Any changes around calibration/linking constraints can alter scientific output. -Run the repository's actual package/documentation checks. The canonical local R -package check is: +## Preferred change strategy -```bash -R_PROFILE_USER=/dev/null Rscript -e \ -'install.packages("rcmdcheck", repos="https://cloud.r-project.org")' -R_PROFILE_USER=/dev/null Rscript -e \ -'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning")' -``` +- Add tests/fixtures first when behavior changes are required. +- Isolate operational fixes (workflow/docs/dependency policy) from algorithmic + edits. +- Document assumptions and risk in commit/PR summaries. -Hosted exact-head checks remain authoritative for merge decisions. A push -invalidates predecessor-head check and review evidence. +## Product design and Figma -## Commercial license boundary - -The repository currently declares GPL-family package/runtime licensing and -imports `mirt`. Do not present the source as Apache-2.0/MIT-cleared or -commercially intake-compliant until the provenance/relicensing and dependency -replacement work tracked by issue #320 is integrated and verified. Repository -source licensing and third-party dependency licensing are separate obligations. +- This repo has no frontend, route tree, Figma file, or implemented visual + surface. Do not create UI code or visual artifacts unless a real product + surface is introduced. +- Do not call future Figma work a "design system" unless it includes reusable + foundations, tokens, components, states, accessibility guidance, and code + mapping. Otherwise call it a UI kit, wireframe, or draft. -## ContextualWisdomLab operating context - -- Keep this component independently usable and composable. A sibling repository - may consume it through a released/versioned boundary, but do not create hidden - cross-repository implementation coupling. -- Sibling components include `fast-mlsirm`, which may consume aFIPC-style - fixed-item calibration concepts in broader psychometric workflows. aFIPC does - not become the owner of those sibling products' orchestration, data, or - deployment authority. -- Cross-product LLM/provider routing, if ever required, belongs to - `contextual-orchestrator`, not this numerical R package. +## Agent guidance (CWL governance) + +Applies to every agent (Claude, Codex, Cursor, opencode, ...) working in this repo. + +### Security & review gate + +- Every PR runs a central **Security Scan** required check: `osv-scan` + + `dependency-review` (diff-scoped) and `trivy-fs` (repo-wide, CRITICAL/HIGH, + fixable only). It runs against each PR base, **including stacked PRs**. +- A failing `trivy-fs` is a **REAL finding, not a flake.** Read the job log (it + prints each finding's rule id, severity, and file) or the run's SARIF results, + then **remediate**: bump the offending R dependency (this package vendors its + library tree under `packrat/`), or, only for a genuine false positive, add a + narrow, path-scoped, commented entry to `.trivyignore.yaml`. Never weaken or + disable the gate. This repo has no Dockerfile or k8s manifests, so misconfig + findings are not expected; a hit here is a dependency or secret finding. +- The vendored packrat openssl docs include a verified false-positive example + key at `packrat/lib/x86_64-pc-linux-gnu/3.4.1/openssl/doc/keys.html`; keep + its `private-key` suppression path-scoped in `.trivyignore.yaml` and do not + blanket-ignore the rule. +- A local `trivy` scan with a stale DB misses findings: run + `trivy --download-db-only` first, and scan the **merge ref**, not just the PR head. +- The org `code_scanning` ruleset is intentionally **CodeQL-only** (multiple + code-scanning tools cannot converge on one PR ref). Gating is by the Security + Scan **job result**, not the `code_scanning` rule; do not add tools to that rule. + +### Code exploration + +- There is no `.codegraph/` index in this repo, so use normal search + (grep/find) to locate and understand code. If a `.codegraph/` index is added + later, prefer CodeGraph (`codegraph explore ""` or the + code-review-graph MCP tools) before grep/find; it surfaces callers, callees, + and impact that text search misses. + +### This repo's role in the ecosystem + +- **aFIPC**: R IRT package for Fixed-Item Parameter Calibration; feeds + fast-mlsirm's psychometrics. +- The org is an ecosystem around **naruon** (the hub: an email/PIM app that + DOM-decomposes emails and files into a persisted knowledge graph). Every + component is a **standalone program that must ALSO work as a git submodule**, + grown separately and together. +- Sibling components: **waf-ids-ai-soc** (WAF/IDS/AI SOC/LB/APIM), + **clearfolio** (document viewer), **pg-erd-cloud** (ERD tool), + **contextual-orchestrator** (LLM cost/perf/upstream-LB gateway, beyond + LiteLLM), **codec-carver** (STT/omni-modal speech-video codec), + **fast-mlsirm** (LLM-as-a-Judge calibration + evaluation-item quality; uses + aFIPC FIPC + kaefa item-fit), **feelanet-adfs** (passwordless SSO: + OIDC/SCIM/ADFS/LDAP/FIDO2/OAuth2.1, eliminate passwords), **newsdom-api** + (PDF to DOM sidecar), **semantic-data-portal** (upper ontology/catalog/governance + plane with its own graph engine). ### Research grounding - For substantive calibration/linking changes, cite the relevant IRT and psychometrics literature. Commit paper PDFs only when redistribution is permitted; otherwise cite, link, and summarize. -- Method decisions are recorded in `docs/adr/`. Verified APA 7th records and - DOIs are in `docs/papers/README.md`. Do not invent bibliographic records or - leave empty `DOI:` placeholders. -- The implemented linking contract is FIPC (Kim, 2006): anchors keep old-form - values. `autoFIPC()` does not estimate a Stocking-Lord (1983) or Haebara - (1980) transformation (ADR-0001). -- `mirt` owns numerical estimation (ADR-0002), but do not flatten all returned - models into one method label. Raw old/new data fits can recover through QMCEM, - MHRM, and `surveyFA` paths when an initial fit is unacceptable. The linked - FIPC fit and IPD/DIF path use EM for nominal items or `tryEM = TRUE`, and MHRM - otherwise. Record the actual method path when evidence depends on it. -- IPD/DIF screening is delegated to `mirt` and is not a published invariance - claim (ADR-0003). -- Do not restore the earlier incorrect attribution of "Linking item parameters - to a base scale" to Kim and Kolen (2010) in the *Journal of Educational - Measurement*. The title belongs to Kang and Petersen (2012). Kim and Kolen - (2019) is a separate, real FIPC application paper. +- Method decisions are recorded in `docs/adr/`. Verified APA 7th records + and DOIs are in `docs/papers/README.md`. Do not invent bibliographic + records or leave empty `DOI:` placeholders. +- The implemented linking contract is FIPC (Kim, 2006): anchors keep + old-form values. `autoFIPC()` does not estimate a Stocking–Lord (1983) + or Haebara (1980) transformation (ADR-0001). Numerical estimation is + delegated to `mirt` (ADR-0002), but the returned models are not all one + MML-EM path: raw old/new fits may recover through QMCEM, MHRM, and + `surveyFA` variants, while the linked fit and IPD/DIF path use EM for + nominal items or `tryEM = TRUE` and MHRM otherwise. IPD/DIF screening is + not a published invariance claim (ADR-0003). +- Do not restore Kim and Kolen (2010), "Linking item parameters to a + base scale," *Journal of Educational Measurement*. That attribution was + incorrect and was removed. The title is Kang and Petersen (2012). A real + Kim and Kolen FIPC paper is Kim and Kolen (2019). From 2c57f058584e36c35bfe8c0c9685dc98f3a4242e Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:14:23 +0900 Subject: [PATCH 13/20] docs: align architecture with actual estimator paths --- ARCHITECTURE.md | 49 +++++++++++++++++++++++++++++-------------------- 1 file changed, 29 insertions(+), 20 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f120619c..45433d76 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -43,13 +43,17 @@ R objects (data frame/matrix/model), and outputs are returned as an R list. - Path: `R/aFIPC.R` - Responsibility: - - Calibrate old/new forms using `mirt` + - Calibrate old/new forms using `mirt`, with bounded QMCEM/MHRM/`surveyFA` + recovery when an initial raw-data fit is unacceptable - Optionally detect item parameter drift (IPD) - Apply fixed common-item constraints for test linking + - Fit the linked model with EM for nominal items or `tryEM = TRUE`, and MHRM + otherwise - Produce linked model and score/theta artifacts - Key dependency: `mirt` -- Method decision: FIPC (Kim, 2006), not Stocking–Lord (1983) or - Haebara (1980) transformation estimation; see `docs/adr/0001-fipc-linking-contract.md` +- Method decision: FIPC (Kim, 2006), not Stocking-Lord (1983) or + Haebara (1980) transformation estimation; see + `docs/adr/0001-fipc-linking-contract.md` ### 3.2 Package Metadata and API Surface @@ -119,31 +123,36 @@ is an alternative to separate calibration plus Stocking and Lord concurrent calibration (Kolen & Brennan, 2014). This repository does not implement those transformation estimators. -Estimation lives in `mirt` MML-EM (Chalmers, 2012; Bock & Aitkin, -1981). Optional IPD screening calls `mirt::multipleGroup` and -`mirt::DIF`; it is not a published invariance claim (see -`docs/adr/0003-ipd-dif-screening-delegation.md`). Score-scale -interpretation is bounded by AERA, APA, and NCME (2014). - -A withdrawn cite attributed "Linking item parameters to a base scale" -to Kim and Kolen (2010) in JEM. That record is not kept. The title is -Kang and Petersen (2012). Kim and Kolen (2019) is a real later FIPC -application paper. +Numerical estimation lives in `mirt` (Chalmers, 2012). The linked FIPC +fit uses MML-EM (Bock & Aitkin, 1981) for nominal items or when +`tryEM = TRUE`, and MHRM otherwise. Separately fitted old/new raw-data +models can recover through QMCEM, MHRM, and `surveyFA` variants after +an unacceptable initial fit, so a returned aFIPC result must not be +summarized as if every model artifact used MML-EM. Optional IPD +screening calls `mirt::multipleGroup` and `mirt::DIF` with the same +EM-versus-MHRM selection rule as the linked fit; it is not a published +invariance claim (see `docs/adr/0003-ipd-dif-screening-delegation.md`). +Score-scale interpretation is bounded by AERA, APA, and NCME (2014). + +An earlier draft incorrectly attributed "Linking item parameters to a +base scale" to Kim and Kolen (2010) in JEM; that attribution was removed. +The title belongs to Kang and Petersen (2012). Kim and Kolen (2019) is +a separate, real later FIPC application paper. Full APA 7th records and DOIs: `docs/papers/README.md`. Accepted method ADRs: `docs/adr/`. Kim, S. (2006). A comparative study of IRT fixed parameter calibration -methods. *Journal of Educational Measurement, 43*(4), 355–381. +methods. *Journal of Educational Measurement, 43*(4), 355-381. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. *Applied Psychological Measurement, 7*(2), -201–210. +201-210. Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. *Japanese Psychological Research, 22*(3), -144–149. +144-149. Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and linking: Methods and practices* (3rd ed.). Springer. @@ -151,20 +160,20 @@ linking: Methods and practices* (3rd ed.). Springer. Kim, S., & Kolen, M. J. (2019). Application of IRT fixed parameter calibration to multiple-group test data. *Applied Measurement in -Education, 32*(4), 310–324. +Education, 32*(4), 310-324. Kang, T., & Petersen, N. S. (2012). Linking item parameters to a base -scale. *Asia Pacific Education Review, 13*(2), 311–321. +scale. *Asia Pacific Education Review, 13*(2), 311-321. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. *Journal of Statistical Software, -48*(6), 1–29. +48*(6), 1-29. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. -*Psychometrika, 46*(4), 443–459. +*Psychometrika, 46*(4), 443-459. American Educational Research Association, American Psychological From f6952158051f8b3dfa879c2fad0440dc05fb210c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 11:14:52 +0900 Subject: [PATCH 14/20] docs: bind FIPC note to actual mirt method paths --- docs/fixed-parameter-item-calibration.md | 127 ++++++++++++++--------- 1 file changed, 79 insertions(+), 48 deletions(-) diff --git a/docs/fixed-parameter-item-calibration.md b/docs/fixed-parameter-item-calibration.md index 500488c5..99f5ebad 100644 --- a/docs/fixed-parameter-item-calibration.md +++ b/docs/fixed-parameter-item-calibration.md @@ -9,16 +9,16 @@ should move onto that base scale. This follows the fixed parameter calibration framing in Kim (2006): old operational or anchor item parameters are treated as known values during the new-form calibration so the new form is calibrated directly on the established -scale. FIPC is one published linking design among others. Separate -calibration plus a Stocking and Lord (1983) or Haebara (1980) -characteristic-curve transformation, and concurrent calibration of both -forms, are alternatives surveyed by Kolen and Brennan (2014). -`autoFIPC()` implements FIPC only: it copies old-form anchor values, -holds them fixed, and re-estimates free new-form parameters in `mirt`. -It does not estimate a Stocking–Lord or Haebara linking transformation. +scale. FIPC is one published linking design among others. Separate calibration +plus a Stocking and Lord (1983) or Haebara (1980) characteristic-curve +transformation, and concurrent calibration of both forms, are alternatives +surveyed by Kolen and Brennan (2014). `autoFIPC()` implements FIPC only: it +copies old-form anchor values, holds them fixed, and re-estimates free new-form +parameters in `mirt`. It does not estimate a Stocking-Lord or Haebara linking +transformation. -The package test `test-fixed-parameter-calibration.R` reproduces this -contract with generated 2PL data: +The package test `test-fixed-parameter-calibration.R` reproduces this contract +with generated 2PL data: 1. Generate old-form and new-form responses from known true item parameters. The generated forms include all-zero and all-one response rows plus missing @@ -49,13 +49,32 @@ eligible when both items share the same number of scored response categories, i.e. `n_cat(x_j) == n_cat(y_j)` where `n_cat` counts distinct non-missing responses. +## Estimation-path boundary + +`mirt` owns the numerical estimation algorithms. aFIPC chooses among those +algorithms at distinct stages rather than applying one method label to every +returned model: + +- If old/new inputs are raw response data, `autoFIPC()` first builds separate + form models. When an initial fit is unacceptable and whole-form retry is + enabled, source can retry QMCEM, then MHRM, followed by `surveyFA()` recovery + variants. Those recovery choices can occur even when `tryEM = TRUE`. +- The linked FIPC model uses `method = "EM"` when the item type is nominal or + `tryEM = TRUE`; for non-nominal items with `tryEM = FALSE`, it uses MHRM. +- IPD/DIF screening follows the same EM-versus-MHRM selection rule as the + linked fit. + +Therefore, `tryEM = TRUE` is not evidence that the separately returned old/new +form models were ultimately fitted by MML-EM. Reproducibility evidence should +record the actual fitted-model path when method identity matters. + ## Formula-integrity audit of performance refactors -The estimation mathematics (item-response probabilities, the MML-EM cycles, -`fscores`, `expected.test`, and the DIF/IPD statistics) live in `mirt`; this -package only orchestrates the linking contract above. The following merged -performance refactors were reviewed against that contract and confirmed -**mathematically equivalent** (no term, margin, or constant changed): +The estimation mathematics (item-response probabilities, EM/QMCEM/MHRM +algorithms, `fscores`, `expected.test`, and DIF/IPD statistics) live in `mirt`; +this package only orchestrates the linking and recovery contracts above. The +following merged performance refactors were reviewed against that contract and +confirmed **mathematically equivalent** (no term, margin, or constant changed): - **#48 / #52** (`82fa77d`, `762b8a9`): hoist `fscores(..., method = 'MAP')` into a variable reused by `expected.test` instead of recomputing it. MAP @@ -82,43 +101,59 @@ contract (anchors fixed to old-form values, non-anchors left free) is pinned in Kolen and Brennan (2014) organize common IRT linking designs as: -- **Separate calibration + characteristic-curve transformation.** - Each form is calibrated freely. A linear transformation is then - chosen to match test characteristic curves (Stocking & Lord, 1983) - or item characteristic curves (Haebara, 1980). -- **Concurrent calibration.** Both forms are estimated in one run with - shared parameters for common items. -- **Fixed item parameter calibration (FIPC).** Anchor parameters from - the old form are treated as known and held fixed while the new form - is calibrated onto that scale (Kim, 2006; see also Kim & Kolen, - 2019, for a later multiple-group FIPC application). - -`R/aFIPC.R` implements the third design. There is no Stocking–Lord or -Haebara objective, and no post-calibration slope/intercept estimator. -A previous draft cited a non-existent Kim and Kolen (2010) *Journal of -Educational Measurement* article titled "Linking item parameters to a -base scale." That record is withdrawn. The title belongs to Kang and -Petersen (2012). - -Linked scores still fall under the interpretation limits in the -*Standards for Educational and Psychological Testing* (AERA, APA, & -NCME, 2014). Estimation of free parameters uses `mirt` MML-EM -(Chalmers, 2012; Bock & Aitkin, 1981). See `docs/adr/` for the -accepted method decisions. +- **Separate calibration + characteristic-curve transformation.** Each form is + calibrated freely. A linear transformation is then chosen to match test + characteristic curves (Stocking & Lord, 1983) or item characteristic curves + (Haebara, 1980). +- **Concurrent calibration.** Both forms are estimated in one run with shared + parameters for common items. +- **Fixed item parameter calibration (FIPC).** Anchor parameters from the old + form are treated as known and held fixed while the new form is calibrated + onto that scale (Kim, 2006; see also Kim & Kolen, 2019, for a later + multiple-group FIPC application). + +`R/aFIPC.R` implements the third design. There is no Stocking-Lord or Haebara +objective, and no post-calibration slope/intercept estimator. An earlier draft +incorrectly attributed "Linking item parameters to a base scale" to Kim and +Kolen (2010) in the *Journal of Educational Measurement*; that attribution was +removed. The title belongs to Kang and Petersen (2012). + +Linked scores still fall under the interpretation limits in the *Standards for +Educational and Psychological Testing* (AERA, APA, & NCME, 2014). The linked +fit defaults to `mirt` MML-EM because `tryEM` defaults to true, while the +explicit non-EM and raw-form recovery paths above remain valid implementation +behavior. See `docs/adr/` for the accepted method decisions. + +## mirt documentation/version evidence + +`DESCRIPTION` imports `mirt` without pinning an exact package version, so this +document must not imply one historical version is permanently authoritative. +At the 2026-09-02 documentation review, the current CRAN package was `mirt` +1.47. Reproducibility evidence should record the installed version actually +used for a calibration run. + +- CRAN package record: +- CRAN reference manual: + +The upstream `fixedCalib` documentation describes fixed-item calibration based +on Kim (2006) and points to `multipleGroup` for more flexible anchor-item +calibration. The Chalmers (2012) package citation below is the scientific +package reference; the CRAN record/manual are the executable documentation +locators. ## References Kim, S. (2006). A comparative study of IRT fixed parameter calibration -methods. *Journal of Educational Measurement, 43*(4), 355–381. +methods. *Journal of Educational Measurement, 43*(4), 355-381. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. *Applied Psychological Measurement, 7*(2), -201–210. +201-210. Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. *Japanese Psychological Research, 22*(3), -144–149. +144-149. Kolen, M. J., & Brennan, R. L. (2014). *Test equating, scaling, and linking: Methods and practices* (3rd ed.). Springer. @@ -126,27 +161,23 @@ linking: Methods and practices* (3rd ed.). Springer. Kim, S., & Kolen, M. J. (2019). Application of IRT fixed parameter calibration to multiple-group test data. *Applied Measurement in -Education, 32*(4), 310–324. +Education, 32*(4), 310-324. Kang, T., & Petersen, N. S. (2012). Linking item parameters to a base -scale. *Asia Pacific Education Review, 13*(2), 311–321. +scale. *Asia Pacific Education Review, 13*(2), 311-321. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. *Journal of Statistical Software, -48*(6), 1–29. +48*(6), 1-29. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. -*Psychometrika, 46*(4), 443–459. +*Psychometrika, 46*(4), 443-459. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). *Standards for educational and psychological testing*. American Educational Research Association. - -Chalmers, R. P. `mirt::fixedCalib` documentation. The implementation -note describes fixed-item calibration methods based on Kim (2006) and -points to `multipleGroup` for more flexible anchor-item calibration. From 9cd02c677194c3970948543217250bc015aea5a4 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 22:17:56 +0900 Subject: [PATCH 15/20] docs: make aFIPC onboarding non-interactive and license-honest --- README.md | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 220500f9..f6394087 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ refactoring. 4. Review convergence, item-parameter-drift evidence, and linked outputs before downstream use. -A minimal API shape is: +A non-interactive raw-data API shape is: ```r result <- autoFIPC( @@ -40,10 +40,17 @@ result <- autoFIPC( oldformYData = reference_form, newformCommonItemNames = common_new, oldformCommonItemNames = common_old, + newformBILOGprior = FALSE, + oldformBILOGprior = FALSE, confirmCommonItems = TRUE ) ``` +The explicit BILOG-prior choices matter for the default 3PL path: leaving either +choice as `NULL` can require interactive input when raw response data are fitted. +If callers use already fitted compatible model objects, review the complete +argument contract before omitting those raw-data choices. + `autoFIPC()` returns the base-form, new-form, and linked-model artifacts as an R list. See `man/autoFIPC.Rd` for the complete argument contract. @@ -77,11 +84,17 @@ assumptions, and the comparability of the forms being linked. Repository tests and cited methodology are implementation evidence; they do not make arbitrary forms automatically comparable. -## Evaluate from source +## Evaluate the current source + +This repository currently provides source rather than an immutable GitHub +release. It also has a known GPL-family runtime blocker: `DESCRIPTION` imports +`mirt`, and `rcmdcheck` does **not** install that dependency for the package under +check. Therefore this README does not present a fresh `install.packages("mirt")` +bootstrap as a commercially acceptable onboarding path. -This repository currently provides a source package rather than a published -GitHub release. For contributor/evaluation checks, use a clean R profile and the -same `rcmdcheck` path exercised by repository automation: +If you are maintaining the existing legacy development environment and its +current dependency graph has already been provisioned for license-diligence or +compatibility work, the repository check itself is: ```bash R_PROFILE_USER=/dev/null Rscript -e \ @@ -91,6 +104,12 @@ R_PROFILE_USER=/dev/null Rscript -e \ 'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning")' ``` +This is **not** a clean commercial-install recipe: `rcmdcheck` expects the +package's declared runtime dependencies, including the currently disallowed +`mirt`, to already exist. A commercially compatible clean setup is blocked until +issue #320 replaces/removes that runtime path and the final package graph is +revalidated. + The historical `packrat/` tree is retained for compatibility archaeology. Its bootstrap is opt-in via `AFIPC_ENABLE_PACKRAT=true` and should not be treated as the preferred dependency workflow. From 81f8a8155186a03931e293df758e454c77b4b16b Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 22:18:34 +0900 Subject: [PATCH 16/20] docs: keep aFIPC source checks license-honest --- docs/index.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/index.md b/docs/index.md index 919ad1be..21f91fbd 100644 --- a/docs/index.md +++ b/docs/index.md @@ -23,7 +23,8 @@ downstream decision authority remain outside this package. objects. 2. Define the reviewed common-item correspondence between forms. 3. Run `autoFIPC()` with the intended item model and explicit common-item - confirmation. + confirmation. For non-interactive raw-data use of the default 3PL path, + explicitly set both BILOG-prior choices instead of leaving them `NULL`. 4. Review convergence, item-parameter-drift evidence, and linked outputs before downstream score reporting or operational use. @@ -44,8 +45,13 @@ calibration. See the FIPC contract and ADR index for the precise boundary. ## Onboarding and verification -Clone the repository, use a current R toolchain, and run the package checks -before changing calibration behavior: +The current source graph is not commercially intake-cleared because it directly +imports GPL-family `mirt`. `rcmdcheck` does not install the package-under-check's +runtime dependencies, so there is no honest commercially compatible clean-R +bootstrap to advertise while issue #320 remains open. + +For maintainers working in an already provisioned legacy compatibility or +license-diligence environment, the repository check itself is: ```bash R_PROFILE_USER=/dev/null Rscript -e \ @@ -54,6 +60,12 @@ R_PROFILE_USER=/dev/null Rscript -e \ 'rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning")' ``` +That command assumes the package's existing declared dependencies are already +available; it is not a recommendation to add `mirt` to a new commercial stack. +A clean commercial setup becomes publishable only after #320 replaces/removes +the GPL-family runtime path and the resulting graph passes package, numerical, +security, and provenance verification. + For contributor expectations and architectural context, read `CONTRIBUTING.md` and `ARCHITECTURE.md` before modifying estimation or linking logic. From 80246f5b3d4fc6ad39689782ab773db334708b01 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 22:19:04 +0900 Subject: [PATCH 17/20] docs: separate aFIPC retry and surveyFA recovery gates --- docs/adr/0002-mirt-mml-em-engine.md | 33 +++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/docs/adr/0002-mirt-mml-em-engine.md b/docs/adr/0002-mirt-mml-em-engine.md index b310ecc4..2d327171 100644 --- a/docs/adr/0002-mirt-mml-em-engine.md +++ b/docs/adr/0002-mirt-mml-em-engine.md @@ -19,27 +19,36 @@ The implementation does not use one estimation method for every model artifact. The method policy has three distinct stages: 1. When raw old/new form data must first be fitted, `autoFIPC()` starts with - the ordinary `mirt` path. If fit diagnostics remain unacceptable and - whole-form retry is enabled, it may retry QMCEM, then MHRM, and finally - `surveyFA()` recovery variants. These fallbacks are independent of the - later linked-fit `tryEM` choice. + the ordinary `mirt` path. If the current fit is still unacceptable and the + corresponding `tryFitwholeOldItems` or `tryFitwholeNewItems` flag is true, + that flag gates only the direct QMCEM retry followed by the direct MHRM + retry. If the model remains unacceptable after that stage—or if the direct + whole-form retry flag is false—the later `surveyFA()` recovery sequence is + still evaluated independently: `forceUIRT`, then `forceNormalEM`, then the + `unstable` path, then `forceMHRM`, with each later step attempted only while + the current model remains unacceptable. These raw-form recovery gates are + independent of the later linked-fit `tryEM` choice. 2. The linked FIPC fit uses EM when `itemtype == "nominal"` or `tryEM` is true. Otherwise it uses MHRM. 3. IPD/DIF screening follows the same EM-versus-MHRM selection rule as the linked fit: EM for nominal items or `tryEM = TRUE`, MHRM otherwise. Accordingly, an `autoFIPC()` result can legitimately contain old/new form -models fitted by QMCEM or MHRM even when the linked model uses EM. Documentation -must not label every returned model as MML-EM. +models fitted by QMCEM, MHRM, or a `surveyFA()` recovery path even when the +linked model uses EM. Setting `tryFitwholeOldItems = FALSE` or +`tryFitwholeNewItems = FALSE` suppresses only the corresponding direct +QMCEM/MHRM whole-form retries; it does not suppress later `surveyFA()` recovery. +Documentation must not label every returned model as MML-EM. ## Decision Use `mirt` as the estimation engine while keeping the method of each model artifact explicit: -- Separate old-form and new-form raw-data fits call `mirt::mirt` and may use - the bounded recovery sequence described above when their initial fit is not - acceptable. +- Separate old-form and new-form raw-data fits call `mirt::mirt`. Their + `tryFitwhole*` flags gate the direct QMCEM-then-MHRM retries only; if the + current model is still unacceptable, `surveyFA()` recovery remains a + separate subsequent gate sequence. - The linked fit calls `mirt::mirt` with `pars` after the FIPC copy-and-fix step. - Default linked estimation uses `method = "EM"` because `tryEM` defaults to @@ -59,6 +68,9 @@ MHRM algorithms. estimator and risk silent numerical drift. Rejected. - **A different IRT package.** Historical outputs were produced with `mirt`. Changing engines would be a scientific behavior change, not a docs fix. +- **Treating `tryFitwhole* = FALSE` as disabling every later recovery.** + Rejected because current source places the `surveyFA()` recovery sequence + outside those direct retry gates. - **Calling every result “MML-EM.”** Rejected because source permits QMCEM and MHRM for raw-form recovery and MHRM for the linked/IPD branch when the explicit method policy selects it. @@ -70,6 +82,9 @@ MHRM algorithms. - Evidence about a returned model should record its actual estimation path; `tryEM = TRUE` alone does not prove the separately fitted old/new models used EM after all recovery attempts. +- Disabling `tryFitwholeOldItems` or `tryFitwholeNewItems` must not be described + as disabling all raw-form recovery unless runtime behavior is changed in a + dedicated behavior PR with regression evidence. - Formula-integrity reviews in `docs/fixed-parameter-item-calibration.md` apply to orchestration only. Estimation mathematics stay in `mirt`. From 5c7fb53a4631d39351d5657825f97e5797d22076 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 22:21:27 +0900 Subject: [PATCH 18/20] docs: broaden aFIPC estimation ADR title --- docs/adr/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/README.md b/docs/adr/README.md index be452207..6f4e25f8 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -17,7 +17,7 @@ behavior. | ID | Title | Status | | --- | --- | --- | | [ADR-0001](0001-fipc-linking-contract.md) | FIPC as the linking contract | Accepted | -| [ADR-0002](0002-mirt-mml-em-engine.md) | mirt MML-EM as the estimation engine | Accepted | +| [ADR-0002](0002-mirt-mml-em-engine.md) | mirt as the estimation engine | Accepted | | [ADR-0003](0003-ipd-dif-screening-delegation.md) | IPD/DIF screening is delegated to mirt | Accepted | ## Related documents From 47a37f7d6d948679cabb3cd02f9f0862102d6369 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 22:22:19 +0900 Subject: [PATCH 19/20] docs: clarify aFIPC raw-form recovery gates --- docs/fixed-parameter-item-calibration.md | 26 ++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/docs/fixed-parameter-item-calibration.md b/docs/fixed-parameter-item-calibration.md index 99f5ebad..e37654cc 100644 --- a/docs/fixed-parameter-item-calibration.md +++ b/docs/fixed-parameter-item-calibration.md @@ -56,17 +56,26 @@ algorithms at distinct stages rather than applying one method label to every returned model: - If old/new inputs are raw response data, `autoFIPC()` first builds separate - form models. When an initial fit is unacceptable and whole-form retry is - enabled, source can retry QMCEM, then MHRM, followed by `surveyFA()` recovery - variants. Those recovery choices can occur even when `tryEM = TRUE`. + form models. If a current form fit is unacceptable and the corresponding + `tryFitwholeOldItems` or `tryFitwholeNewItems` flag is true, that flag gates + only the direct QMCEM retry followed by the direct MHRM retry. +- If the current raw-form model remains unacceptable after that stage—or if the + corresponding direct whole-form retry flag is false—the later `surveyFA()` + recovery sequence is evaluated independently. Source proceeds through + `forceUIRT`, `forceNormalEM`, `unstable`, and `forceMHRM` variants as needed, + stopping when the current model becomes acceptable. These raw-form recovery + choices are independent of the later linked-fit `tryEM` choice. - The linked FIPC model uses `method = "EM"` when the item type is nominal or `tryEM = TRUE`; for non-nominal items with `tryEM = FALSE`, it uses MHRM. - IPD/DIF screening follows the same EM-versus-MHRM selection rule as the linked fit. Therefore, `tryEM = TRUE` is not evidence that the separately returned old/new -form models were ultimately fitted by MML-EM. Reproducibility evidence should -record the actual fitted-model path when method identity matters. +form models were ultimately fitted by MML-EM. Likewise, +`tryFitwholeOldItems = FALSE` or `tryFitwholeNewItems = FALSE` suppresses the +direct QMCEM/MHRM retry for that form, not every later `surveyFA()` recovery. +Reproducibility evidence should record the actual fitted-model path when method +identity matters. ## Formula-integrity audit of performance refactors @@ -128,9 +137,10 @@ behavior. See `docs/adr/` for the accepted method decisions. `DESCRIPTION` imports `mirt` without pinning an exact package version, so this document must not imply one historical version is permanently authoritative. -At the 2026-09-02 documentation review, the current CRAN package was `mirt` -1.47. Reproducibility evidence should record the installed version actually -used for a calibration run. +At the 2026-09-02 documentation review, current CRAN package/check metadata was +reverified as `mirt` 1.47. Reproducibility evidence should record the installed +version actually used for a calibration run rather than treating that review +snapshot as a permanent pin. - CRAN package record: - CRAN reference manual: From afc5a632543bcb671d792683166c279761c9649a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sun, 6 Sep 2026 12:06:43 +0900 Subject: [PATCH 20/20] docs: establish code-current product gap baseline --- docs/product-technical-gap-baseline.md | 78 ++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 docs/product-technical-gap-baseline.md diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md new file mode 100644 index 00000000..440e7f0c --- /dev/null +++ b/docs/product-technical-gap-baseline.md @@ -0,0 +1,78 @@ +# Product / Technical Gap Baseline + +This document records code-current commercial and technical gaps for aFIPC. It is not a roadmap promise and does not turn an open proposal into an accepted architecture decision. + +## Product boundary + +aFIPC owns fixed-item-parameter calibration/linking behavior and the domain meaning of its form, common-item, calibration, prior, linking, and recovery contracts. Shared psychometric kernels may be consumed through released/versioned boundaries when they preserve that product contract; aFIPC does not copy a sibling repository's source or depend on a mutable sibling head. + +Protected repository authority at this baseline is `master@f87c2324f1686135e57d8730c1b0b9420874f300`. Open PRs are evidence/proposals until they satisfy the live protected-branch gates and merge normally. + +## Commercial blocker: runtime and repository licensing + +Issue #320 is the current commercial-intake blocker. The package metadata declares GPL-3 and the core calibration path imports/calls `mirt`, whose CRAN license is GPL (>= 3). Those are separate obligations: changing repository prose or a root license file cannot make the current runtime dependency graph compatible with ContextualWisdomLab's no-GPL-family intake baseline. + +Acceptance for a commercially cleared successor is therefore behavioral and provenance-based, not merely textual: + +- establish first-party/contributor provenance before changing the repository grant; +- remove or replace GPL/LGPL/AGPL-family runtime, build, vendored, copied-source, or derivative obligations under the intended distribution model; +- preserve the actual fixed-item calibration/linking estimand rather than substituting a different psychometric method to obtain a different license; +- keep package metadata, root license bytes, dependency inventory, SBOM and release evidence mutually consistent; +- retain normal R compatibility only as an explicit boundary while any Rust-first replacement kernel is validated. + +A released `fast-mlsirm` capability may be evaluated as an optional canonical kernel boundary when it actually provides the required estimand and numerical contract. Until then it is a read-only dependency candidate, not a source-copy or mutable-head shortcut. + +## Numerical and psychometric evidence gap + +A replacement calibration runtime is not GREEN merely because unit tests execute. Acceptance must preserve observable aFIPC outputs over right-cleared representative product data and, where simulation is scientifically appropriate, report true-parameter recovery rather than synthetic-data success alone. + +Required evidence for the replacement/calibration boundary: + +- fixed common-item identities and ordering; +- linked item-parameter selection and scale behavior; +- convergence/recovery-path attribution across the currently supported estimation/recovery paths; +- reproducibility under pinned runtime/package/kernel versions; +- simulation studies that report true-parameter bias, RMSE and interval coverage with the data-generating design stated explicitly; +- real/right-cleared product-form replay for commercial acceptance, with synthetic data confined to unit/property/recovery tests. + +Any material numerical difference requires an explicit ADR with the estimand, constraint, alternatives, reason for acceptance/rejection, measured effect, migration risk and rollback path. + +## Current correctness lane: interactive choice domain + +The three interactive yes/no menu boundaries document `1` or `2`. PR #337 is the current repaired candidate that narrows admission before integer coercion and carries deterministic production-path regressions. It remains Draft until one unchanged exact head has terminal R CMD/test, applicable security/SAST/CodeQL evidence, current review findings resolved, protected-base compatibility, and qualifying independent review. + +The supported finding is input-contract/reliability correctness. Oversized decimal coercion to `NA` is not, without additional evidence, proof of integer overflow, memory corruption, remote exploitation, or a specific security severity. + +## Current performance lanes + +Several open branches propose R-level micro-optimizations in the calibration/linking path. Their admissible evidence is narrower than their generated performance language: + +- direct model-column lookup must preserve the source/model column-membership validation that the protected data-frame projection currently provides; +- distinct non-missing category refactors must preserve observed-category semantics, including NA/NaN/factor edge cases; +- local expression or allocation changes are not buyer-visible performance improvements until the actual `autoFIPC()` path is measured. + +Performance promotion requires representative item/respondent/form cardinalities, pinned R/runtime/dependency state, warm-up policy, repeated wall-time distribution including median and p95, allocation/GC or equivalent profile evidence, and numerical-equivalence checks. A unit test, coverage result, or isolated operation-count benchmark is correctness evidence only. + +If profiling shows a material psychometric hot path rather than incidental R overhead, prefer an auditable Rust-first kernel boundary with vector/linear/matrix operations and CPU multithreading, while preserving the product-owned R/API/domain contract through an adapter. Do not migrate a hot path on style grounds alone. + +## Release and rollback gap + +No open branch is a release by itself. A release-ready protected generation must have: + +- one exact protected head after normal reviewed merge; +- terminal required repository and organization checks on that generation; +- numerical/behavioral evidence appropriate to the changed domain boundary; +- version and CHANGELOG aligned with the shipped behavior; +- immutable tag/package plus SBOM and provenance; +- reproducibility instructions and a tested rollback/recovery path. + +Until the licensing blocker and the applicable correctness/numerical gates are resolved, documentation must not describe aFIPC as commercially intake-cleared or a replacement runtime as behaviorally equivalent. + +## Traceability + +- Commercial/runtime licensing owner: issue #320. +- Product/method documentation owner lane: PR #261. +- Interactive choice-domain repair candidate: PR #337. +- Model-column performance/validation candidate: PR #335. +- Protected repository authority recorded above remains the merge/release source of truth. +- CRAN `mirt` package metadata is the primary external license reference; Chalmers (2012), *Journal of Statistical Software, 48*(6), documents the `mirt` estimation framework. External references support the decision record but do not supersede live repository code, dependency bytes, or protected-branch rules.