Repository navigation
Spec2Contract Model Groups
This area covers the Model Groups subsystem: registry-backed group definitions, their two storage
scopes, the /model-groups TUI manager, the modality/constraint engine, spawn-time routing, and the
#group autocomplete + agent guidance. Left-space IDs are PB-GROUPS-01…PB-GROUPS-47 (47 rows).
Contracts are carried mainly by the dedicated fixture family T15–T24 (router, crud, constraints,
modalities, modality lexicon, display, autocomplete, integration, TUI, frontmatter), with cross-area
support from T01/T03 for spawn routing + identity and T48 for the compile-time type contract.
| Claim ID | Assertion (terse) | Fixture(s) | Status | Notes |
|---|---|---|---|---|
| PB-GROUPS-01 | A group maps to a list of provider/model refs, each optionally carrying a thinking level. | T16, T48 | ✅ asserted |
model-groups-crud.test.ts — "model groups store creates, round-trips, validates, renames, updates, deletes, and moves" (thinking "high" round-trips; absent thinkingLevel not persisted/invented). T48 tests/typecheck/model-groups-producer.ts pins ModelGroupModel {provider, modelId, thinkingLevel?}. |
| PB-GROUPS-02 | Global groups persist at ~/.pi/agent/pi-schematic/model-groups.json. |
T16 | ◐ partly asserted | missing: no independent assertion of the global path literal — tests write/read via modelGroupsPath("global", …) ("model groups load recovery handles malformed, schema-invalid, unsupported version, and backup failure"), so a wrong global root would still pass. Would be sealed by asserting modelGroupsPath("global", cwd) === path.join(homedir(), ".pi/agent/pi-schematic/model-groups.json"). |
| PB-GROUPS-03 | Project groups persist at <cwd>/.pi/pi-schematic/model-groups.json (Pi project config dir name). |
T16 | ✅ asserted |
"model groups use branded paths, global-only access, canonical own keys, and native max" asserts modelGroupsPath("project", cwd, "branded-pi") === path.join(cwd, "branded-pi", "pi-schematic", "model-groups.json"); the default segment is Pi's CONFIG_DIR_NAME. |
| PB-GROUPS-04 | Persisted document uses version 2 and a groups map. |
T16 | ✅ asserted |
"v1 migration is in-memory until the first successful mutation writes v2 without an invented override"; "model groups strictly partitions schema and legacy version domains"; "v2 constraint envelope preserves explicit empty and opaque slots canonically". |
| PB-GROUPS-05 | Names are trimmed of surrounding whitespace; empty-after-trim names are rejected. | T16, T23 | ✅ asserted |
"model groups direct save canonicalizes unique keys and rejects empty/colliding keys before write"; "model groups use branded paths…" (` ${name} ` → trimmed); model-groups-tui.test.ts — "decodes then canonicalizes prototype-sensitive names and rejects malformed escapes". |
| PB-GROUPS-06 | Project groups available only in a trusted project; otherwise access is global-only. | T16, T22 | ✅ asserted |
"model groups use branded paths, global-only access…" (project probe forbidden, project createGroup throws /global-only/); model-groups-integration.test.ts — "model groups untrusted root flow never probes project data and publishes global-only state to TUI". |
| PB-GROUPS-07 | A project group shadows a same-named global group when resolving an effective role. | T15, T16, T22 | ✅ asserted |
model-groups-router.test.ts — "effective model group names use project-over-global names"; crud round-trip asserts validation.shadowedByProject; integration notifies 1 project overrides. |
| PB-GROUPS-08 | Saves write a temporary JSON file and rename it into the selected scope's path. | T16 | ✅ asserted |
"model groups rename failure removes the generated temp file and preserves committed bytes" (.tmp name pattern, rename phase); "model groups persistence failures throw typed errors and preserve committed state" (temp-write/rename phases). |
| PB-GROUPS-09 |
/model-groups opens the interactive manager in TUI mode. |
T22 | ✅ asserted |
"…/model-groups command registers and opens ctx.ui.custom with live registry/cwd". |
| PB-GROUPS-10 | Manager cannot run in non-TUI mode; RPC receives a TUI-required notification. | T22 | ✅ asserted |
"root-recorded /model-groups rejects RPC and skips JSON/print before custom UI" (rpc → "/model-groups requires TUI mode"; json/print silent; custom never called). |
| PB-GROUPS-11 |
+ Add group immediately creates an empty group, defaulting to project scope when trusted and global otherwise. |
T23 | ✅ asserted |
model-groups-tui.test.ts — "computes unique new-group names and opens editor after create" (createGroup project, 0 models); "clamps final add row and enforces global-only access" (global-only policy → global). |
| PB-GROUPS-12 | The group editor allows moving a group between project and global locations. | T23 | ✅ asserted |
"move, wizard add, model thinking, and remove persist through store calls" (move:review:global); "notifies and preserves location on move collision". |
| PB-GROUPS-13 | Selecting the Name row activates inline input; committing renames the persisted group. | T23 | ✅ asserted |
"renders name editing inline and preserves edit/commit transitions" (Esc and Enter both commit renameGroup). |
| PB-GROUPS-14 | Adding a member walks through provider, model, and thinking selection. | T23 | ✅ asserted |
"wizard renders provider/model/thinking steps and preserves state on add failure"; "Esc and left-arrow share wizard back-step behavior". |
| PB-GROUPS-15 | The add-model picker offers only registry-available models with configured authentication. | T23 | ✅ asserted |
"Add-model picker shows capability chips per model" (gpt-no-auth absent); "model groups TUI wizard renders provider/model/thinking steps…"; "searchable Model step filters raw fields and handles no matches safely" (hidden absent). |
| PB-GROUPS-16 | Picker searches the complete selected provider's results by model ID, provider, provider/ID, and display name. | T23 | ✅ asserted |
"model groups TUI searchable Model step filters raw fields and handles no matches safely" (queries alpha-id, openai/alpha-id, Friendly, openai); "fuzzy search excludes synthetic provider-space-id matches". |
| PB-GROUPS-17 | Picker displays ten model rows at a time without limiting the underlying results to ten. | T23 | ✅ asserted |
"Model SelectList keeps all results behind a ten-row viewport and owns wrapping" (12 models, (1/12), navigable to model-11). |
| PB-GROUPS-18 | The thinking options the manager offers for a member are inherit, the levels the screen derives for the member's selected model, and the member's stored level when that level falls outside that set. |
T23 | ✅ asserted |
"model edit renders identity/status and filters thinking options" (claude → inherit only; gpt-5 → full list incl. max); "clamps final add row and enforces global-only access" offers native max; "MODEL_EDIT stale level on an available model keeps its row after inherit and Remove model last" and "MODEL_EDIT treats a stale off level on a non-reasoning model as current, unavailable" show the stored level added to the offered set. |
| PB-GROUPS-19 | The edit-model screen changes a member's thinking level or removes that member. | T23 | ✅ asserted |
"notifies and preserves model edit state when updateGroup fails" (thinking change then D remove); "move, wizard add, model thinking, and remove persist through store calls". |
| PB-GROUPS-20 | Deleting a group requires selecting Delete group rather than the default Keep group. |
T23 | ✅ asserted |
"selected markers and primary labels use accent token" (default selected Keep group, Down → Delete group); "list renders validation summary, health tags, add row, no Validate row, and confirmed D delete". |
| PB-GROUPS-21 | Deleting a same-named group in one scope leaves the other scope's group intact. | T16, T23 | ✅ asserted | crud "model groups store creates, round-trips, validates, renames, updates, deletes, and moves" (otherScopeHasOverride === true; other scope survives the delete/move); TUI "…confirmed D delete" asserts "Same-name group in the other scope remains unaffected". |
| PB-GROUPS-22 | Group rows identify scope, member count, thinking selections, capability letters, and validation warnings. | T23 | ✅ asserted |
"list renders validation summary, health tags, add row…" ([project], 1 models, ⚠ degraded, ✗ unavailable); "renders modality labels, warnings, and stale override choices" (T I, ⚠ no common modalities, ⚠ stale modality override: reasoning); "selected markers…" row label review [project] 1 models inherit. |
| PB-GROUPS-23 | The production capability registry contains exactly one constraint type, modalities. |
T17 | ✅ asserted |
model-groups-constraints.test.ts — "injected scalar traverses resolution, aggregation, persistence, reconciliation, and production isolation" (productionConstraintRegistry.descriptors keys ["modalities"]; injected key undefined). |
| PB-GROUPS-24 | The supported modality vocabulary is text, image, and reasoning. |
T17, T16, T19 | ✅ asserted | T17 "descriptor codecs report errors and retain vocabulary ordering"; crud "v2 config load rejects non-array, duplicate, and out-of-vocabulary modality constraints"; model-groups-modality.test.ts — "MODALITY_LETTER / MODALITY_FG cover the established single-letter lexicon". |
| PB-GROUPS-25 | A model's text/image capabilities follow its registry input modalities. |
T18 | ✅ asserted |
model-groups-modalities.test.ts — "derives ordered common, supported, and override-effective modalities from the live registry". |
| PB-GROUPS-26 | A model advertises reasoning when its registry model has reasoning: true. |
T18 | ✅ asserted | same "derives ordered common, supported…" test (rich reasoning:true → supported includes reasoning; text reasoning:false excluded); model-groups-constraints.test.ts — "engine preserves unresolved members as unknown facts". |
| PB-GROUPS-27 | Without an override, a group's advertised capabilities are the union of its resolved members' modalities. | T15, T18, T22 | ✅ asserted |
model-groups-router.test.ts — "automatic mixed groups default effective to the union so a capability routes to a capable member"; T18 "derives ordered common, supported, and override-effective modalities…"; model-groups-integration.test.ts — "before_agent_start exposes union-effective modalities for automatic mixed groups". |
| PB-GROUPS-28 | An override may narrow but cannot add a modality none of its members supports when creating/updating. | T16, T18, T15 | ✅ asserted | crud "low-level save permits shape-valid overrides while CRUD rejects union-cap-invalid overrides" and "CRUD rejects unsupported injected overrides before any write"; T18 "caps stale overrides without mutation and restores them when catalog support returns"; T15 "explicit override stays a subtractive ceiling even when a capable member exists". |
| PB-GROUPS-29 | Optional-media toggle edits image; text is a required base when supported; reasoning is not toggleable. | T23 | ✅ asserted |
"modality editor commits override and Automatic through updateGroup"; "text-only modalities screen is inert and states Automatic"; "editor rows show per-model capability chips and modalities screen shows dynamic Automatic/Override status" (T text [required], I image [on], no reasoning row). |
| PB-GROUPS-30 | Add-model and member rows show compact T/I chips; reasoning is omitted from these media chips. |
T19, T23 | ✅ asserted |
model-groups-modality.test.ts — "modalityLetterRun canonicalizes order and excludes reasoning by default"; TUI "Add-model picker shows capability chips per model" (T, T I); "editor rows show per-model capability chips…". |
| PB-GROUPS-31 | Group guidance includes only effective group names and capabilities, not membership or thinking levels. | T22 | ✅ asserted |
model-groups-integration.test.ts — "before_agent_start injects fresh names-and-effective-modalities guidance" (Available Model Groups: review (text, image, reasoning); doesNotMatch /gpt-5/, /model-groups\.json/). |
| PB-GROUPS-32 | A #-prefixed group mention autocompletes effective names and shows model/thinking details. |
T21 | ✅ asserted |
model-groups-autocomplete.test.ts — "#group autocomplete suggests effective live group names and delegates elsewhere" (openai/gpt-5 • high; anthropic/claude-sonnet-4 • inherit); "registerModelGroupAutocomplete uses ctx.ui.addAutocompleteProvider once". |
| PB-GROUPS-33 | Boot validation marks refs missing from the registry as unavailable and warns when counts are nonzero. | T16, T22 | ✅ asserted | crud "creates, round-trips…" (unavailableRefs: [{missing/nope}]); integration "index session_start stores model group validation and notifies load and validation issues" (notifies 1 unavailable model references, and "index session_start does not notify when load and validation issues are absent" seals the zero-count branch). |
| PB-GROUPS-33 | Boot validation marks present-but-unauthenticated refs as unavailable. | T15 | ◐ partly asserted | Router excludes unauthenticated usable members (model-groups-router.test.ts — "capability pre-selection excludes unauthenticated capable models and picks randomly from the capable set"), but no test drives validateModelGroups with a registry-present model whose hasConfiguredAuth is false to assert it enters validation.unavailableRefs. Would be sealed by such a boot-validation case. |
| PB-GROUPS-34 | Corrupt/schema-invalid JSON loads as an empty configuration for that scope with a backup attempt. | T16 | ✅ asserted |
model-groups-crud.test.ts — "model groups load recovery handles malformed, schema-invalid, unsupported version, and backup failure" (empty groups, .bak written, backupFailed branch). |
| PB-GROUPS-34 | Load issues are surfaced to the operator as a warning. | T22, T20 | ✅ asserted | integration "index session_start notifies corrupt/schema/unsupported load issues", "…schema-invalid load issues", "…includes backup-failure detail…", "…boot load-issue notifications escape hostile source, backup, and error detail fields"; model-groups-display.test.ts seals the escape codec the warning paths/messages pass through. |
| PB-GROUPS-35 |
spawn accepts an optional exact group name; omitting it inherits the parent's model and thinking. |
T15, T01 | ✅ asserted |
model-groups-router.test.ts — "omitted and unknown groups inherit parent route with fallback metadata" (status:"inherited"); spawn.test.ts — "spawn execute composes Model Group routing with readonly child guards" (explicit group routed) and "spawn forwards requested thinking and reports the session effective thinking" (no group → details.route { status: "inherited" }). |
| PB-GROUPS-36 | Unknown group names visibly fall back to the parent model/thinking rather than selecting a different group. | T15, T03 | ✅ asserted | T15 "omitted and unknown groups inherit parent route with fallback metadata" (status:"unknown-fallback", requestedGroup); spawn-render.test.ts — "spawn result identity formats default routed and unknown fallback" (rev? fallback → openai/parent • medium). |
| PB-GROUPS-37 | A known group with zero members fails before a child session is created. | T15, T01, T24 | ✅ asserted | T15 "known empty and all-unusable groups fail clearly" (reason:"empty"); spawn.test.ts — "executeSpawn propagates unusable-group errors before creating child work" (sessionFactory not called); model-group-frontmatter.test.ts — "empty model-group (SpawnRouteError) blocks execution with error". |
| PB-GROUPS-38 | A known group with no registry-resolved usable member fails before a child session is created. | T15, T01 | ✅ asserted | T15 "known empty and all-unusable groups fail clearly" (reason:"no-usable-models"); spawn.test.ts — "executeSpawn propagates unusable-group errors before creating child work" (no child registered). |
| PB-GROUPS-38 | A known group whose resolved members all lack configured authentication fails the same way. | T15 | ◐ partly asserted | T15's auth fixture has auth-configured members alongside the unauthenticated one, so it proves exclusion but not the all-unauthenticated → no-usable-models failure. Would be sealed by a group whose only member is registry-present with hasConfiguredAuth === false. |
| PB-GROUPS-39 | A usable routed member is drawn uniformly by entry position on every spawn; repeated calls can draw the same entry. | T15, T01 | ◐ partly asserted | T15 "routed empty modality requirements select by RNG across usable members" (exactly one RNG draw, index from rng) and "capability pre-selection excludes unauthenticated capable models and picks randomly from the capable set" (every selection consults the RNG, no cursor short-circuit). missing: no assertion of uniform distribution or that an unchanged pool can redraw the same entry across calls. Would be sealed by repeated-draw / repeat-entry assertions. |
| PB-GROUPS-40 |
spawn.constraints.modalities requires { required: [...] }, not a bare array; unknown/invalid keys/values are rejected. |
T01, T15 | ✅ asserted |
spawn.test.ts — "spawn tool schema validates constraints via Value.Check" (bare array, duplicates, out-of-vocab, non-object, stringified, array-with-embedded-required → false); "spawn requirements validate the canonical envelope and keep its shape"; "executeSpawn rejects unknown requirements before factory or session publication"; T15 "bare-array constraint requirements are rejected by the descriptor codec". |
| PB-GROUPS-41 | For declared capabilities, the router preferentially samples usable members satisfying the exact model requirement. | T15, T01 | ✅ asserted | T15 "required modalities prefer a capable member and still reject a wholly-incapable group" and "automatic mixed groups default effective to the union so a capability routes to a capable member"; spawn.test.ts — "spawn routes capability requirements to capable members via random selection on the filtered pool". |
| PB-GROUPS-42 | A declared capability must pass both the group's effective ceiling and the selected model's check; failure occurs before child creation. | T15, T01 | ✅ asserted | T15 "explicit override stays a subtractive ceiling even when a capable member exists" (missingFromGroup set, missingFromModel empty); spawn.test.ts — "executeSpawn propagates missing modalities before creating child work" and "registered spawn tool rejects missing modalities before creating child work" (sessionFactory/factory not called). |
| PB-GROUPS-43 | A routed member's thinking override wins over spawn.thinking/inherited; the result is clamped to the model's supported levels. |
T01, T24 | ✅ asserted |
spawn.test.ts — "spawn execute composes Model Group routing with readonly child guards" (group entry low beats requested xhigh); model-group-frontmatter.test.ts — "model-group-only frontmatter applies and records the group's effective thinking" ("group thinking is clamped for the routed model"). |
| PB-GROUPS-44 | Routed identity shows group → provider/model • thinking; unknown-group identity shows group? fallback → provider/model • thinking. |
T03 | ✅ asserted |
spawn-render.test.ts — "spawn result identity formats default routed and unknown fallback" (review → openai/gpt-routed • high, rev? fallback → openai/parent • medium; controlled chars escaped). |
| PB-GROUPS-45 | A #group-name mention directs the agent to route the spawn through that exact group (exact name, no fallback). |
— | No test asserts the #group-name sentence of modelGroupsPromptSection (index.ts:457-464); model-groups-integration.test.ts asserts only the section header, the Available Model Groups: … line, and /constraints/. A test would assert the guidance matches /#group-name/ and the exact-name/no-fallback sentence. |
|
| PB-GROUPS-46 | The edit-model screen opens with the cursor on the member's configured thinking level and marks that row ✓ current when the screen offers it. The marker reflects the screen's offered options, independently of authentication, which the Status: line reports separately. |
T23 | ✅ asserted |
model-groups-tui.test.ts — "MODEL_EDIT opens on the configured thinking level and preserves it on Enter", "MODEL_EDIT opens on a mid-list configured level and marks only that row current", "MODEL_EDIT marks the default inherit row current when no level is configured", "MODEL_EDIT opens on a supported level for a registered-but-unauthenticated model" (marker independent of Status: unavailable). |
| PB-GROUPS-47 | A member's stored thinking level that the edit screen does not offer for its model — a level the model no longer lists, off on a model that does not reason, or any level when the model is not in the registry — stays on screen as its own row immediately after inherit, marked ⚠ current, unavailable, and is preserved unless the operator deliberately chooses another option. |
T23 | ✅ asserted |
"MODEL_EDIT shows an unsupported/unavailable current level as current, unavailable and preserves it", "MODEL_EDIT stale level on an available model keeps its row after inherit and Remove model last", "MODEL_EDIT keeps an unsupported level after inherit when the model has other supported rows", "MODEL_EDIT stale level on an unavailable model keeps Status unavailable and preserves the value", "MODEL_EDIT treats a stale off level on a non-reasoning model as current, unavailable". |
No PB-GROUPS claim is entirely unrepresented except PB-GROUPS-45 (⚠️): no test asserts the #group-name mention sentence in the agent guidance. The prompt-section test hangs on the header, the names list, and /constraints/ only; it would be sealed by asserting the guidance matches /#group-name/ together with the exact-name/no-fallback sentence.
Partial surfaces (◐) and what would seal the missing portion:
-
PB-GROUPS-02 — no independent assertion of the global config path literal; tests reuse
modelGroupsPath("global", …). Seal by asserting the home-rooted path equals~/.pi/agent/pi-schematic/model-groups.json. -
PB-GROUPS-33 (auth aspect) — no boot-validation test with a registry-present, unauthenticated ref entering
validation.unavailableRefs. Seal by drivingvalidateModelGroupswithhasConfiguredAuth → false. -
PB-GROUPS-38 (auth aspect) — no test for a group whose only member is present but unauthenticated failing with
reason:"no-usable-models". Seal by that routing case. - PB-GROUPS-39 (uniformity aspect) — RNG-based selection is asserted deterministically, but uniform distribution and repeat-of-same-entry across calls are not. Seal by repeated-draw assertions over an unchanged pool.
| Claims | Junction rows | ✅ | ◐ | |
|---|---|---|---|---|
| 47 | 50 | 45 | 4 | 1 |