diff --git a/AGENTS.md b/AGENTS.md index 5c6c8c8da..a99b4005c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -273,7 +273,7 @@ stops startup instead of leaving a healthy-looking partial schema, and application code must not compensate for a missing table. Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0119 / 0158 / 0162 / -0163 / 0164 / 0182 / 0185 / 0201 / 0233 / 0266 / 0267 / 0268 / 0269 / 0270 / 0271 / 0272 / 0273 / 0274 / 0275 / 0276 / 0277 / 0278 / 0279 / 0280 / 0281 / 0282 / 0283 / 0284 / 0285 / 0286 / 0287 / 0288 / 0289 / 0290 / 0291 / 0292) are computed in `lineageweave/leftover_pairs.py` from the +0163 / 0164 / 0182 / 0185 / 0201 / 0233 / 0266 / 0267 / 0268 / 0269 / 0270 / 0271 / 0272 / 0273 / 0274 / 0275 / 0276 / 0277 / 0278 / 0279 / 0280 / 0281 / 0282 / 0283 / 0284 / 0285 / 0286 / 0287 / 0288 / 0289 / 0290 / 0291 / 0292 / 0293) are computed in `lineageweave/leftover_pairs.py` from the residual after a real GRM/GPCM score, never invented. Distances are Euclidean on the two-dimensional Gabriel leftover map; missing cells stay out of the factorization. Closest and farthest post–criterion pairs @@ -398,6 +398,7 @@ axis share and does not invent `σ_k` from leftover-map axis share. ADR 0292 captions leftover-axis report badges with persisted leftover-map singular values `σ_k` when finite and non-negative, including rank-0 zero singular values; missing, non-finite, or negative singular values omit only the `σ` caption and never derive `σ_k` from leftover-map axis share. ADR 0290 captions the grouping comparison strip with persisted leftover-map post complete-case coverage only when the caller can see the full persisted grouping population. Partial-visibility groupings omit that aggregate; never reconstruct the psychometric denominator from visible members, leftover pairs, or plotted markers. ADR 0291 captions the grouping comparison strip with persisted leftover-map item complete-case coverage through leftoverMapItemCoverageCounts only when the caller can see the full persisted grouping population. Partial-visibility rows inherit ADR 0290's fail-closed coverage boundary and never recompute psychometric coverage from visible members, leftover pairs, or plotted criterion markers. +ADR 0293 captions the grouping comparison strip with persisted leftover-map incomplete-post count only under the same full-visible-grouping authorization boundary. Partial-visibility groupings omit the aggregate and never derive dropped posts from the visible subset or scored-minus-used. Persisted valid dropped `0` remains visible. When `R`, `R̂`, `U`, `x`, `s`, and `e` are finite, `e + s + x = 1`. When `Y`, `E`, and `R` are finite, `Y − E = R`. When `R`, `R̂`, and `U` are diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index b9325e8e6..e91d602bc 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -637,7 +637,7 @@ information at the group's mean θ (Lord, 1980 max-info CAT). Rankings persist to `report_item_information`. After those IRT main effects, residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021; ADR 0017 / 0048 / 0049 / 0119 / 0148 / 0158 / 0162 / 0163 / 0164 / 0168 / -0182 / 0185 / 0201 / 0233 / 0266 / 0267 / 0268 / 0269 / 0270 / 0271 / 0272 / 0273 / 0274 / 0275 / 0276 / 0277 / 0278 / 0279 / 0280 / 0281 / 0282 / 0283 / 0284 / 0285 / 0286 / 0287 / 0288 / 0289 / 0290 / 0291 / 0292) persist to `report_leftover_pair` with signed residual `R`, +0182 / 0185 / 0201 / 0233 / 0266 / 0267 / 0268 / 0269 / 0270 / 0271 / 0272 / 0273 / 0274 / 0275 / 0276 / 0277 / 0278 / 0279 / 0280 / 0281 / 0282 / 0283 / 0284 / 0285 / 0286 / 0287 / 0288 / 0289 / 0290 / 0291 / 0292 / 0293) persist to `report_leftover_pair` with signed residual `R`, observed `Y`, expected `E[Y|θ, item]`, full leftover-map rank, unexplained leftover, ADR 0201 reconstruction evidence, ADR 0185 cross-share evidence, ADR 0233 unexplained leftover share `s`, ADR 0266 explained leftover @@ -672,6 +672,7 @@ on the leftover pair list through leftoverMapCoverageCounts. ADR 0289 captions l graphic axes with persisted leftover-map singular values `σ_k`. ADR 0290 captions the grouping comparison strip with persisted leftover-map post complete-case coverage under a full-visible-grouping rule; partial-visibility groupings fail closed that aggregate to avoid disclosing a hidden population denominator. ADR 0291 captions grouping-comparison persisted item complete-case coverage through leftoverMapItemCoverageCounts only under ADR 0290's full-visible-grouping authorization boundary. ADR 0292 captions leftover-axis report badges with persisted finite, non-negative leftover-map singular values `σ_k`; missing or invalid singular values omit only that caption, and axis share remains a distinct persisted measure. +ADR 0293 captions grouping-comparison persisted incomplete-post count only for a fully caller-visible persisted grouping; partial visibility omits the aggregate rather than disclosing or recomputing a hidden-population count. Click a post marker or a pair button opens that post. Those ADRs are the normative mathematical, storage, and display contracts. Leftover-map axis share diff --git a/CHANGELOG.d/2.50.0-leftover-map-compare-incomplete-post.md b/CHANGELOG.d/2.50.0-leftover-map-compare-incomplete-post.md new file mode 100644 index 000000000..c41f8c939 --- /dev/null +++ b/CHANGELOG.d/2.50.0-leftover-map-compare-incomplete-post.md @@ -0,0 +1,37 @@ +## 2.50.0 — Leftover-map incomplete post coverage on the grouping comparison strip + +- Caption leftover-map incomplete post coverage on the grouping comparison + strip through leftoverMapIncompletePostCount (ADR 0293). After `make seed`, + closest and farthest leftover pairs sit above the member list with the + Gabriel biplot of already-named coordinates, leftover-map axis share when + finite, axis ticks that match `ξ (x, y) ζ (x, y)` on the pair row, pair + segments that match `d` on the pair row, pair segments that match `R̂` on + the pair row, pair segments that match `R̂²/R²` on the pair row, pair + segments that match `U²/R²` on the pair row, pair segments that match + `2R̂U/R²` on the pair row, pair segments that match `U` on the pair row, + pair segments that match `R` on the pair row, pair segments that match + `Y` on the pair row, pair segments that match `E` on the pair row, pair + segments that match leftover-map rank on the pair row, a plot caption + that matches leftover-map complete-case coverage above the pair list, a + plot caption that matches leftover-map item complete-case coverage, a + plot caption that matches leftover-map incomplete post coverage, a plot + caption that matches leftover-map incomplete item coverage, a pair-list + note that matches leftover-map post complete-case coverage, a pair-list + note that matches leftover-map item complete-case coverage, a pair-list + note that matches leftover-map incomplete post coverage, a pair-list + note that matches leftover-map incomplete item coverage, a grouping + comparison note that matches leftover-map post complete-case coverage, + a grouping comparison note that matches leftover-map item + complete-case coverage, and a grouping comparison note that matches + leftover-map incomplete post coverage; click a post marker or a pair + button opens that post. A missing, non-integer, or negative dropped + count, or a dropped count that contradicts usable complete-case + integers, omits that leftover-map comparison incomplete post note. + Dropped `0` is shown when that persisted dropped count is a + non-negative integer. Do not invent dropped posts from scored minus + used, leftover pair count, plotted marker count, leftover-map distance, + leftover-map rank, leftover-map post coverage, leftover-map item + coverage, leftover-map incomplete item coverage, or the count of unused + axes. The strip does not gain leftover-map incomplete item coverage or + the leftover-map graphic. Never invent a leftover score. Never invent a + theta. No new columns. diff --git a/CHANGELOG.md b/CHANGELOG.md index 04342510a..8340181a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ All notable changes to this project are documented here. Format follows ### Added +- Grouping comparison rows now show persisted leftover-map incomplete-post count through `leftoverMapIncompletePostCount` (ADR 0293 / v2.50.0) only for a fully caller-visible persisted grouping. Partial visibility omits the aggregate; valid persisted zero remains visible, and the UI never derives dropped posts from scored-minus-used. + - Period leftover pairs now caption leftover-axis report badges with persisted leftover-map singular values `σ_k` (ADR 0292 / v2.49.0). Missing, non-finite, or negative singular values omit only that `σ` caption; rank-0 zero remains `σ 0.00`, and axis share remains distinct. - Grouping comparison rows now name persisted leftover-map item complete-case coverage (ADR 0291 / v2.48.0) only when the caller can see the full persisted grouping population; partial-visibility rows omit the aggregate rather than leaking a denominator or recomputing subset coverage. diff --git a/CLAUDE.md b/CLAUDE.md index 3a6fbb514..1963b8114 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -49,7 +49,7 @@ cutoff. Global Ask optional `knowledge_cutoff` uses the same cover Create/start endpoint rules (ADR 0017 / 0021), tie-vs-miss similarity (ADR 0026), R&R catalog ids (ADR 0019 / 0027), leftover pairs -(ADR 0048–0164 / 0182 / 0185 / 0201 / 0233 / 0266 / 0267 / 0268 / 0269 / 0270 / 0271 / 0272 / 0273 / 0274 / 0275 / 0276 / 0277 / 0278 / 0279 / 0280 / 0281 / 0282 / 0283 / 0284 / 0285 / 0286 / 0287 / 0288 / 0289 / 0290 / 0291 / 0292), occupational construct catalog search +(ADR 0048–0164 / 0182 / 0185 / 0201 / 0233 / 0266 / 0267 / 0268 / 0269 / 0270 / 0271 / 0272 / 0273 / 0274 / 0275 / 0276 / 0277 / 0278 / 0279 / 0280 / 0281 / 0282 / 0283 / 0284 / 0285 / 0286 / 0287 / 0288 / 0289 / 0290 / 0291 / 0292 / 0293), occupational construct catalog search (ADR 0257), the text-channel embedding swap and cosine clamp (ADR 0190), per-edge channel-score persistence (ADR 0195), token-backed status notices (ADR 0220), diff --git a/docs/adr/0293-leftover-map-compare-incomplete-post.md b/docs/adr/0293-leftover-map-compare-incomplete-post.md new file mode 100644 index 000000000..31c018974 --- /dev/null +++ b/docs/adr/0293-leftover-map-compare-incomplete-post.md @@ -0,0 +1,169 @@ +# ADR 0293 — Name leftover-map incomplete post coverage on the grouping comparison strip + +**Decision status:** Proposed +**Date:** 2026-08-30 + +Amends leftover pairs on the grouping comparison strip +([ADR 0149](0149-leftover-pairs-on-comparison-strip.md)), leftover-map +complete-case coverage ([ADR 0168](0168-leftover-map-complete-case-coverage.md)), +leftover-map complete-case coverage on the grouping comparison strip +([ADR 0289](0289-leftover-map-compare-coverage.md)), and leftover-map item +complete-case coverage on the grouping comparison strip +([ADR 0290](0290-leftover-map-compare-item-coverage.md)). Independent of leftover-map +post complete-case coverage fail-closed on the pair list +([ADR 0288](0288-leftover-map-list-post-coverage-helper.md)), leftover-map +incomplete item coverage on the pair list +([ADR 0287](0287-leftover-map-list-incomplete-item.md)), leftover-map incomplete +post coverage on the pair list +([ADR 0286](0286-leftover-map-list-incomplete-post.md)), leftover-map item +complete-case coverage on the pair list +([ADR 0285](0285-leftover-map-list-item-coverage.md)), leftover-map incomplete +item coverage on the graphic display +([ADR 0284](0284-leftover-map-plot-incomplete-item.md)), leftover-map incomplete +post coverage on the graphic display +([ADR 0283](0283-leftover-map-plot-incomplete.md)), leftover-map item +complete-case coverage on the graphic display +([ADR 0282](0282-leftover-map-plot-item-coverage.md)), leftover-map complete-case +coverage on the graphic display +([ADR 0281](0281-leftover-map-plot-coverage.md)), leftover-map rank on pair +segments ([ADR 0280](0280-leftover-map-segment-rank.md)), leftover expected on pair +segments ([ADR 0279](0279-leftover-map-segment-expected.md)), leftover observed on +pair segments ([ADR 0278](0278-leftover-map-segment-observed.md)), leftover +residual on pair segments ([ADR 0277](0277-leftover-map-segment-residual.md)), +leftover-map unexplained leftover on pair segments +([ADR 0276](0276-leftover-map-segment-unexplained-leftover.md)), leftover-map +cross share on pair segments ([ADR 0275](0275-leftover-map-segment-cross-share.md)), +leftover-map unexplained leftover share on pair segments +([ADR 0274](0274-leftover-map-segment-unexplained-share.md)), leftover-map +explained leftover share on pair segments +([ADR 0273](0273-leftover-map-segment-explained-share.md)), leftover-map +reconstruction on pair segments +([ADR 0272](0272-leftover-map-segment-reconstruction.md)), leftover-map +distance on pair segments ([ADR 0271](0271-leftover-map-segment-distance.md)), +leftover-map coordinate ticks ([ADR 0270](0270-leftover-map-coordinate-ticks.md)), +leftover-map axis share on the graphic display +([ADR 0269](0269-leftover-map-axis-share-plot.md)), leftover residual +disclosure ([ADR 0162](0162-leftover-residual-disclosure.md)), leftover +observed `Y` / expected `E` ([ADR 0163](0163-leftover-observed-expected.md)), +leftover-map explained leftover share persistence +([ADR 0266](0266-leftover-map-explained-share.md)), leftover-map +unexplained leftover share persistence +([ADR 0233](0233-leftover-map-unexplained-share.md)), leftover-map +reconstruction persistence ([ADR 0201](0201-leftover-map-reconstruction.md)), +leftover-map cross share persistence +([ADR 0185](0185-leftover-map-cross-share.md)), leftover-map unexplained leftover +persistence ([ADR 0182](0182-leftover-map-unexplained.md)), leftover-map rank +persistence ([ADR 0164](0164-leftover-map-rank.md)), and leftover-map axis +share persistence ([ADR 0148](0148-leftover-map-axis-share.md)). + + +This aggregate inherits ADR 0290's full-visible-grouping authorization boundary. Persisted incomplete-post count is shown only when the caller can see the full persisted grouping population; partial-visibility groupings omit it. Never recalculate a psychometric dropped-post count from the visible subset. + +## Context + +ADR 0168 already persists `incomplete_post_count` on +`report_leftover_map_coverage`. ADR 0289 already captions leftover-map post +complete-case coverage on the grouping comparison strip. ADR 0290 already +captions leftover-map item complete-case coverage on the grouping comparison +strip. ADR 0283 already names persisted leftover-map incomplete post coverage +on the leftover-map graphic display. ADR 0286 already names it on the pair +list. The strip still does not name dropped incomplete posts, so a buyer who +compares leftover pairs can treat `used N of M scored posts (complete-case)` +on a grouping row as the full scored census even after incomplete rows were +dropped. Incomplete rows are dropped from Gabriel factorization; hiding +dropped posts on the strip lets a sparse post with one missing criterion +vanish without a next action. + +This increment captions persisted leftover-map incomplete post coverage on +each grouping comparison row through leftoverMapIncompletePostCount. It does +not add columns. It does not recompute dropped posts from scored minus used, +leftover pair count, plotted marker count, leftover-map distance, or +leftover-map rank. It does not persist leftover-map inner product, cosine, or +length as separate columns. Do not invent a leftover score. Do not invent a +theta. + +The dashboard stack already used neighbouring leftover facts under other +numbers. This protected increment uses **0291** so it does not collide with +leftover-map item complete-case coverage on the grouping comparison strip +(0290), leftover-map complete-case coverage on the grouping comparison strip +(0289), leftover-map post complete-case coverage fail-closed on the pair list +(0288), leftover-map incomplete item coverage on the pair list (0287), +leftover-map incomplete post coverage on the pair list (0286), leftover-map +item complete-case coverage on the pair list (0285), leftover-map incomplete +item coverage on the graphic display (0284), leftover-map incomplete post +coverage on the graphic display (0283), leftover-map item complete-case +coverage on the graphic display (0282), leftover-map complete-case coverage +on the graphic display (0281), leftover-map rank on pair segments (0280), +or the dashboard stacks. + +## Decision + +On the grouping comparison strip, caption leftover-map incomplete post +coverage as `Leftover map dropped {dropped} incomplete posts` only when +leftoverMapIncompletePostCount returns a usable dropped integer from +persisted `incomplete_post_count`. Use the distinct accessible name +`Leftover map comparison incomplete posts` so the strip caption is not the +pair-list note (`Leftover map incomplete posts`) and is not the graphic +caption (`Leftover-map graphic incomplete posts`). A missing coverage row, a +non-integer dropped count, a negative dropped count, or a dropped count that +contradicts usable complete-case integers (`dropped !== scored − used` when +`map_post_count` / `scored_post_count` are usable) omits that leftover-map +comparison incomplete post note and keeps the strip leftover-map post +coverage note, leftover-map item coverage note, leftover pairs, leftover-map +distance `d`, and any leftover-map captions on the pair list and graphic. +Dropped `0` is shown when that persisted dropped count is a non-negative +integer. Do not invent dropped posts from scored minus used, leftover pair +count, plotted marker count, leftover-map distance, leftover-map rank, +leftover-map post coverage, leftover-map item coverage, leftover-map +incomplete item coverage, or the count of unused axes. Do not caption +leftover-map incomplete item coverage on the strip in this increment. Do not +add the leftover-map graphic to the strip. Click a leftover pair on the strip +to open that post. + +Do not add SQL. Do not edit shipped migrations. Do not persist inner +product, cosine, or length as separate columns. Do not invent a leftover +score. Do not invent a theta. + +## Consequences + +After `make seed`, the grouping comparison strip names persisted leftover-map +post complete-case coverage, persisted leftover-map item complete-case +coverage, and persisted leftover-map incomplete post coverage on each grouping +row when leftoverMapCoverageCounts / leftoverMapItemCoverageCounts / +leftoverMapIncompletePostCount return usable integers, then names leftover +pairs with leftover-map distance `d`. Closest and farthest leftover pairs +still sit above the member list with the leftover-map graphic display; click a +post marker or a pair button opens that post. +Hidden posts stay hidden. + +## Related + +Independent of leftover interaction-map persistence, leftover-criterion +evaluation landing, leftover residual disclosure, leftover-map +complete-case coverage persistence, leftover-map axis share persistence, +leftover pairs on the grouping comparison strip, leftover-map complete-case +coverage on the grouping comparison strip, leftover-map item complete-case +coverage on the grouping comparison strip, leftover-map inner product, +leftover-map cosine, leftover-map length, leftover-map graphic display, +leftover-map item complete-case coverage on the graphic display, leftover-map +item complete-case coverage on the pair list, leftover-map incomplete post +coverage on the graphic display, leftover-map incomplete post coverage on the +pair list, leftover-map incomplete item coverage, and leftover-map post +complete-case coverage fail-closed on the pair list. + +## References + +Gabriel, K. R. (1971). The biplot graphic display of matrices with +application to principal component analysis. *Biometrika, 58*(3), +453–467. https://doi.org/10.1093/biomet/58.3.453 + +Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping +unobserved item–respondent interactions: A latent space item response +model with interaction map. *Psychometrika, 86*(2), 378–403. +https://doi.org/10.1007/s11336-021-09762-5 +(LSIRM interaction `−γ‖ξ_j − ζ_i‖` after main effects +`α_j − β_i`; typically `p = 2` for the interaction map. Incomplete +rows are dropped from the complete-case residual rectangle; grouping +comparison incomplete post coverage names how many scored posts stayed out +of that factorization for that grouping only when leftoverMapIncompletePostCount +returns a usable dropped integer.) diff --git a/frontend/package.json b/frontend/package.json index 8fa819fcf..497940a94 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "frontend", "private": true, - "version": "2.49.0", + "version": "2.50.0", "type": "module", "scripts": { "dev": "vite", diff --git a/frontend/src/App.test.tsx b/frontend/src/App.test.tsx index ec0b734f2..2f7a7c9b1 100644 --- a/frontend/src/App.test.tsx +++ b/frontend/src/App.test.tsx @@ -4285,6 +4285,26 @@ describe("App, authenticated", () => { expect( within(screen.getByLabelText("Grouping comparison")).queryByLabelText("Leftover map incomplete posts"), ).not.toBeInTheDocument(); + expect( + within(screen.getByLabelText("Grouping comparison")).getAllByLabelText( + "Leftover map comparison incomplete posts", + ), + ).toHaveLength(2); + expect( + within(screen.getByLabelText("Grouping comparison")).getAllByLabelText( + "Leftover map comparison incomplete posts", + )[0], + ).toHaveTextContent("Leftover map dropped 0 incomplete posts"); + expect( + within(screen.getByLabelText("Grouping comparison")).getAllByLabelText( + "Leftover map comparison incomplete posts", + )[1], + ).toHaveTextContent("Leftover map dropped 1 incomplete posts"); + expect( + within(screen.getByLabelText("Grouping comparison")).queryByLabelText( + "Leftover-map graphic incomplete posts", + ), + ).not.toBeInTheDocument(); expect( within(screen.getByLabelText("Grouping comparison")).queryByLabelText("Leftover map incomplete items"), ).not.toBeInTheDocument(); diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index e22eabc2d..fe3be602d 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -126,6 +126,7 @@ import { leftoverMapItemCoverageCounts, LEFTOVER_MAP_COMPARE_COVERAGE_LABEL, LEFTOVER_MAP_COMPARE_ITEM_COVERAGE_LABEL, + LEFTOVER_MAP_COMPARE_INCOMPLETE_POST_LABEL, LEFTOVER_MAP_LIST_COVERAGE_LABEL, LEFTOVER_MAP_LIST_INCOMPLETE_ITEM_LABEL, LEFTOVER_MAP_LIST_INCOMPLETE_POST_LABEL, @@ -4002,6 +4003,7 @@ function ReportsPanel({ {comparison.groupings.map((row) => { const comparisonCoverageCounts = leftoverMapCoverageCounts(row.leftover_map_coverage); const comparisonItemCoverageCounts = leftoverMapItemCoverageCounts(row.leftover_map_coverage); + const comparisonIncompletePostCount = leftoverMapIncompletePostCount(row.leftover_map_coverage); return (