Target-repo-aware test floor — near-miss no-test diagnostics + per-phase testPattern re-evaluation (v0.14.53) - #1052
Merged
Conversation
…x 1-10 to 1-11 patch Round 1: 13 probes, 13/13 on-target, 0 blockers, 0 needsDecision. Aimed at stack-fresh drift (plan authored before plans 1-4 of its own campaign landed); the four-testPatternArg-site forward assumption holds exactly at 04edae0. One Minor auto-fixed: assert-test-in-diff.test.sh has eleven case families, not ten (case 11 FAIL-CLOSED FLOOR CLASSIFICATION #732 landed after authoring) - new near-miss cases append after case 11. 7 adjudication rows for args.adjudications. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ath (#1045) assert-test-in-diff.sh: when nothing matched the active pattern and the diff is non-empty, re-scan the SAME changed-file list against the fixed D5 set (*.test.*, *.spec.*, *_test.*, basename test_*) and name any hit on stderr beside the ACTIVE pattern set — the custom --pattern tokens when supplied, else the two defaults printed FROM the existing pattern_mjs/pattern_sh variables (never a third literal statement of the default set; red-team adjudication 5), plus a note that the *.test.sh union arm is always in force. Fail-open and diagnostic-only: stdout keeps its byte-empty summary (the refiner's read contract), the 0/1/2 exit contract is byte-preserved, no scan on exit 0 and none on the exit-2 die path. A pattern/target-repo mismatch is now visible on the FIRST no-test instead of after three add-test rounds (#983). Mapped test — case family 12 appended after case 11 (red-team adjudication 1: eleven pre-existing families, not ten; 1–11 unmodified and green): 12a runner/x.test.mjs bare -> exit 1, both defaults + path on stderr, stdout BYTE-EMPTY 12b same fixture --pattern '*.test.mjs' -> exit 0, empty stderr 12c docs-only exit-1 diff -> stderr empty (byte-identical to today) 12d bad-ref exit 2 -> die path only, near-miss marker ABSENT 12e src/foo.spec.ts --pattern '*.test.ts' -> exit 1, CUSTOM set named, defaults ABSENT Mutation-probed: removing the block reds 12a/12e; hardcoding the default set as the active set reds 12e. Sweep half (spec §4 / End state 9): the --pattern header note rewords from "threaded per-run from the run's overrides.testPattern" to per-phase-resolved, and the survey-derived straggler "whatever pattern is pinned" becomes "in force for the phase". Exit-code contract block records that stderr may carry the advisory diagnostic.
Three auditor-flagged absorb-disposition Nits, smallest mechanical fixes: 1. [Nit] The mandated '*.test.sh union arm is always in force' note line is unasserted by the new case family — the script printed the note but none of 12a-12e grepped for it, so deleting that single printf left all new checks green: a coverage hole against an explicitly named output element. Fix: fold `grep -qF 'union arm is always in force'` into case 12a's existing default-pattern if-condition. 2. [Nit] Case 4's comment still cites a script line number, now 29 lines stale after the insertion — "git diff reaches line 94" pointed at a call that had moved; CLAUDE.md's standing rule is to anchor by named construct, not line number. Fix: reword to "git diff reaches the `git diff --name-only` call" (line number dropped). 3. [Nit] 12d's NEAR_MARKER absence assert has no positive anchor — a message reword makes it silently vacuous: the marker was asserted only ABSENT (exit-2 path), so a reworded stderr line would keep 12d passing on a token the script no longer emits. Fix: assert NEAR_MARKER PRESENT in case 12a's condition (NEAR_MARKER was already defined above 12a), so the same token is proven present on exit 1 and absent on exit 2. Mutation-probed: deleting the union-arm printf now reds case 12a. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r-miss stderr to the add-test worker (#1046) The test floor's exit-1 stderr (the near-miss diagnostic from Task 1.1) now survives the merge-task boundary instead of dying in the refiner's shell. - MERGE_RESULT gains optional `floor_diagnostic` (string): the verbatim exit-1 stderr of assert-test-in-diff.sh. Orthogonal fail-open advisory field on the gate_failure_class precedent — never routed on, and NO status enum value, HARD_ESCALATION_REASONS member or KNOWN_LAND_DECISIONS member is added or changed (land-decision.mjs and both hand-mirrored enum blocks byte-untouched, ADR 0005). - Every dispatched merge-task floor site (four at this base: initial merge, floor-retry re-merge, environment-proceed, baseline-proceed) instructs verbatim-stderr capture into floor_diagnostic on the exit-1 path only; agents/war-refiner.md step 4 carries the same instruction (both surfaces, one commit). - The ADD_TEST fix prompt quotes a present diagnostic verbatim and tells the fix-worker to reconcile the diff's test files against the ACTIVE pattern — reporting blocked on a mismatch rather than adding a duplicate test. - The no-test/unpackaged exhaustion escalation carries the last diagnostic as `detail` on both the escalated and auditLog entries. - Absent/empty ⇒ every consumer is byte-/shape-identical to today. - overrides.testPattern prose rewords to per-phase-resolved; no enumerated site count survives in comments (the drift-guard discovers the sites). Tests: the validation-6 testPattern drift-guard is extended to DISCOVER every dispatched site in the template source (>= 3 non-vacuity floor, never an exact count) and assert per-site capture tokens plus the war-refiner.md mirror; plus the fixPrompt present/absent pair, the exhaustion-detail pair, and a schema lock pinning the status enum. Delete-the-feature probed: a three-of-four thread, a dropped fixPrompt append, a dropped exhaustion detail, a stripped war-refiner.md bullet, and a removed schema property each go RED.
…e-check (#1047) skills/war/SKILL.md, Task 1.3 of docs/plans/2026-07-22-test-floor-target-repo.md: - Setup step 3 `--afk` sanity floor: on fallback to null, the ledger note now records the rejected proposal verbatim (the full token set) plus its zero-match tokens — the pending-proposal record the per-phase re-check reads. - Test-floor pattern bullet: "floor subset-of gate is one Setup decision" stays true for the confirmation but is qualified — under --afk it is no longer the final word on the value. - "Run one Workflow per phase" paragraph: before threading a still-null overrides.testPattern, re-run the sanity floor's each-token-matches->=1-file Glob check against the current tree. A full match re-asserts the same glob-safe charset rule war-config.mjs's validate() enforces on overrides.testPattern (the injection guard only, not the separate glob-shape checks) before adopting — a charset-clean proposal is adopted monotonically (null -> proposal, once) as this and every subsequent phase's plan.testPattern with an adoption note; a charset-failing proposal is never adopted and the ledger note closes it (cannot self-heal); otherwise the proposal stays pending. Reworded "the run's pinned overrides.testPattern" to per-phase-resolved. Manual survey of Setup step 3 and the launch paragraph found one other "pinned" occurrence (Test-floor pattern bullet's "pinned beside the gate exactly as plan.gate does") — kept as-is: it describes config-threading position (overrides.X rides beside gate, same args.plan.X shape), not one-shot-per-run value semantics, and remains accurate post-change. Does not touch the Checkpoint gate_failed-routing region or the manual-land recipes (merge-land-resilience's regions). requiresTest: false per plan — criteria 9-10 stay audit-time greps (plan Notes), no test added. Gate: node --test 'skills/**/*.test.mjs' (877/877 pass) + the anchored hooks/+skills/ shell-test loop (26/26 files, all green).
… testPattern verdict; Setup bullets aligned (#1047) Applies four absorb-disposition findings on skills/war/SKILL.md: 1. [Minor] Charset re-assert excludes the glob-shape checks but names a command that runs them — a shape-violating proposal is closed as a 'charset failure' (line 75). validate() runs the charset test AND the per-token '**/'/'*word*' shape checks in one pass with one exit code, so the narrowed verdict the sentence specified cannot come from the named command. Fixed by dropping the exclusion clause and taking the validator's full overrides.testPattern verdict (charset + per-token shape): any error => never adopted, ledger note records it, proposal closed — matching the exemplar command exactly and the safer semantic. 2. [Nit] Setup sub-bullet stated adoption as token-match-sufficient, omitting the validator gate the launch paragraph adds (line 28) — the two statements of the same rule differed. Sentence now ends '...and adopts once every token matches and the value re-passes the validator's overrides.testPattern check.' 3. [Nit] Surviving 'pinned beside the gate' in the swept Setup step 3 bullet (line 27) — residual survey-token inconsistency with Task 1.4's rewording family, and imprecise since an --afk-adopted value is threaded per phase without a write-back. Now 'pinned beside the gate at Setup exactly as plan.gate does (an --afk-adopted value is threaded per phase, see the re-check below)'. 4. [Minor] Same defect as (1) from the second seat — the exemplar --stdin command surfaces the per-token shape errors too, and no route existed for the shape-failure case. Resolved by the same rewrite: one run, one non-zero exit, any validator error closes the proposal under an accurately-named reason. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…weep + two glossary terms (#1048) Task 1.4 of docs/plans/2026-07-22-test-floor-target-repo.md — comment/prose only, no executable surface touched. schemas.md: - MergeResult jsonc gains `floor_diagnostic?` (merge-task; verbatim exit-1 assert-test-in-diff.sh stderr) plus a definition bullet beside gate_log_path: fail-open advisory, consumers are the ADD_TEST fix prompt and the no-test exhaustion detail only, never routed on, and no status value / HARD_ESCALATION_REASONS / KNOWN_LAND_DECISIONS member added (ADR 0005). - overrides.testPattern comment: "the Lead threads the pinned value" -> per-phase-resolved (pinned at Setup; under --afk a sanity-floor-rejected proposal stays pending and is re-checked/adopted monotonically at each phase launch), and "at both merge-task invocation sites" -> "at every dispatched merge-task floor invocation site". The two-site claim was CONFIRMED STALE: the live template renders ${testPatternArg} at four dispatched sites (initial merge, floor-retry re-merge, baseline-proceed, environment-proceed), so no count is restated (red-team adjudication 2). war-config.mjs (COMMENT-ONLY — DEFAULTS value and validator behaviorally untouched, ADR 0019 null default byte-identical): the ONE-Setup-decision comment gains the same qualification, noting the --afk re-check is Lead-side with no write-back to this field. war-room/SKILL.md: the overrides.testPattern line's "pinned together with the gate" gains the same clause. assert-packaging-in-diff.sh (COMMENT-ONLY) — cross-reference adjudicated as REWORD, not confirm-correct: the note is a live surface under skills/war/ (the plan's historical-record carve-out covers docs/plans, docs/red-team, docs/learnings and ADR 0019, not script headers), and End state 9 forbids any live skills/war/ surface still describing overrides.testPattern as decided once per run. "gained a per-run overrides.testPattern" -> per-phase-resolved with the re-check clause; "needs no per-run knob" -> "needs no such knob" (this floor needs no knob at all — the adjective was doing no work). The audit's substance (epic #579 / #574 item 4 decision C: no override added) is preserved verbatim. CONTEXT.md: adds the two spec section-6 terms — "Pending testPattern proposal" and "Near-miss diagnostic" — each with an _Avoid_ line, and corrects the existing "test-floor pattern" entry's "The per-run glob set" to per-phase resolution (survey-derived straggler). Manual survey after the token grep (sanity floor, testPattern, per-run, pinned, one Setup decision) across the five files. Stragglers found and handled: CONTEXT.md "test-floor pattern" per-run (corrected, above) and war-config.mjs "the run's pinned test-floor glob set" -> "declared", aligning with schemas.md's established vocabulary for the config field. Kept as-is: schemas.md "floor subset-of gate survives any pinned pattern" (about the union arm, true for any value), CONTEXT.md "_Avoid_: ... floor subset-of gate is one decision" (the spec mandates this stays true for the confirmation), war-config.mjs validate() comments (charset/shape only, no lifetime claim), war-room "pinned run.provision" hits (unrelated construct). No hit outside the five files. requiresTest: false per plan (criteria 9-10 are audit-time greps, red-team adjudication 4). Gate: node --test 'skills/**/*.test.mjs' (881/881) + the shell-test loop (26/26 files) green.
… validator gate (#1048) [Nit] war-room's new --afk adoption clause states a weaker adoption condition than the landed rule (validator gate omitted) (skills/war-room/SKILL.md:25): the clause read "adopted monotonically once every token matches", but the landed rule (skills/war/SKILL.md, 10006f7) requires BOTH every token matching AND the value re-passing the validator's overrides.testPattern check (charset + per-token glob shape) before adoption; schemas.md:240 already states it correctly, so war-room was the odd surface out. Extended the clause with "and the value re-passes the validator's overrides.testPattern check". Same sweep: CONTEXT.md's "Pending testPattern proposal" entry omitted the other terminal state — now "until adopted (or closed — a validator-failing proposal cannot self-heal and is never re-checked)". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…geResult contract (#1048) Phase-close coherence sweep, phase 1 — one queued finding: - [Minor] war-refiner.md '## Return' MergeResult field enumeration omits floor_diagnostic? while step 4 mandates returning it (task 1.2, agents/war-refiner.md:166). Step 4's exit-1 bullet instructs capturing the near-miss stderr into floor_diagnostic, but the '## Return' contract line still enumerated the pre-feature field set. Since the file's convention is to list every optional field there (gate_log_path?, gate_failure_class?), the omission read as "this key is not part of the return" — a refiner obeying "return ONLY the MergeResult JSON: {...}" could prune the key step 4 told it to set, silently no-opping the feature. Adds floor_diagnostic? to the merge-task/land-phase enumeration in the schemas.md field order (after gate_base_sha?, before pr_number?), plus a short merge-task-only parenthetical pointing back at step 4. Doc-only; no status value or enum member touched (ADR 0005).
…; Return-contract enumeration duty Two hot type:project lessons from the test-floor-target-repo phase-1 servitor, Gate-2 verified at the landed tip 7d1922a (every asserted code fact re-checked). The servitor also edited an archive/ lesson (gate-exclusion mirroring, recurrence 5) — archived lessons are cold and are never promoted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…threading + per-phase testPattern re-check (#1051) Bump all four release slots in lock-step to the next free patch above the live integration base (0.14.52 at the phase-1 tip): - .claude-plugin/plugin.json version - .claude-plugin/marketplace.json metadata.version AND plugins[0].version - README.md the ## Status paragraph (replace-in-place) A partial bump is a silent no-op release — Claude Code dispatches plugin updates by the marketplace.json version string. version-slots.test.mjs is the arbiter and is green. Release blurb covers the three landed layers: the exit-1-only near-miss stderr diagnostic (exit codes and the byte-empty stdout contract unchanged), the new optional MergeResult.floor_diagnostic captured verbatim at every dispatched floor site and threaded into the ADD_TEST fix prompt + the no-test exhaustion detail, and the --afk pending-proposal re-check that re-checks the Setup proposal per phase and adopts it monotonically once every token matches and it re-passes the validator. No status enum value or enum member added (ADR 0005); the plugin-scoped null default stays byte-identical (ADR 0019). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…2026-07-22-test-floor-target-repo
…the 0.14.53 blurb The v0.14.53 Status blurb narrated the near-miss stderr block as something the exit-1 path prints, but assert-test-in-diff.sh guards the whole block behind `if [ -n "$near_misses" ]` — a docs-only no-test keeps stderr byte-identical to today (empty), which End state 2 case (c) asserts. Applies the auditing seat sfix wording verbatim. Verified after the edit: version-slots.test.mjs 3/3 and the full JS gate 881/881 green; all four release slots and the arbiters README-token extraction untouched. Promotes the servitors Recurrence 2 of release-blurb-overstates-guard-semantics, with its now-stale "remains unapplied" status superseded in place (the note was accurate at wrap-up time; Gate-2 resolved it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 23, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Campaign
2026-07-22-run-resilience-and-hardening, plan 5 of 9. Stacked ondev/2026-07-22-merge-land-resilience(#1042) per ADR 0011 stack-and-plow — merge bottom-up: #1008 → #1018 → #1024 → #1042 → this PR.Plan:
docs/plans/2026-07-22-test-floor-target-repo.md· Spec:docs/specs/2026-07-22-test-floor-target-repo-design.md· Red-team:docs/red-team/2026-07-22-test-floor-target-repo.md(CLEARED-WITH-NOTES, 1 round, 13/13 probes on-target, 0 blockers).What lands
Three fail-open layers, none touching the floor's 0/1/2 exit contract:
assert-test-in-diff.sh) — on the exit-1 path only, and only when the scan finds something, the floor names the active pattern set and each test-shaped file the pattern missed. Exit codes and the byte-empty stdout contract are untouched; a docs-onlyno-testkeeps stderr byte-identical to today.MERGE_RESULT.floor_diagnostic— a new optional, orthogonal, fail-open advisory field (thegate_failure_classprecedent). Captured verbatim at every dispatched merge-task floor site and threaded into the ADD_TEST fix-worker prompt and theno-testexhaustion escalation. No status enum value,HARD_ESCALATION_REASONSorKNOWN_LAND_DECISIONSmember added or changed —land-decision.mjsand both hand-mirrored enum blocks are byte-untouched (ADR 0005).--afkSetup proposal the sanity floor rejected is recorded verbatim with its zero-match tokens and re-checked at each phase launch, adopted monotonically once every token matches and it passes the validator's fulloverrides.testPatterncheck. Floor-time derivation stays rejected (D7); the plugin-scopednulldefault is byte-identical (ADR 0019).Plus the one-shot-prose sweep across SKILL.md / schemas.md / war-config.mjs / war-room / the floor headers, and two new
CONTEXT.mdglossary terms.Phases
7d1922a→ learningsdee7cfb5319492→ Gate-2c9bfad9Both phases'
deferredgate-audit verdicts were evidence-threading gaps, not unmet conditions, and were Lead-verified at the landed tips: 881/881 JS + 26/26 shell green at each,version-slots.test.mjs3/3.Follow-ups filed
near_miss()omits the gate'snode_modules/.git/.claudeexclusions its siblingmatch_sh_suite()mirrors, so the advisory blames the pattern for an excluded-location file. Four independent auditor seats converged on it. Note the obvious remedy is harmful:--pattern '*.test.sh'would breakfloor ⊆ gate.FLOOR_SITE_REterminates on the ternary false-arm shape rather than its own grammar; both escape modes reproduced by simulation (an inline-shaped 5th site passes vacuously and swallows a real site; an appended one never matches).Unexecuted backstops
Validations this plan deliberately deferred. All four are AI-declared (ADR 0014 — drafted by
/war-machine --afk, not operator-ratified):--afkrun on a docs-only target that grows matching test files by a later phase adopts the proposal and finishes without a test-floor relaunch--afkcross-repo run; read back via the ledger adoption note and/war-reviewno-test, and no invocation surface grows a2>/dev/nullthat eats itno-testpost-release/war-reviewafter cross-repo runs--afkscaffold-shaped runNote this run itself could not exercise the re-check live: the repo's test convention is the hardcoded floor default, so
testPatternstayednulland no proposal was ever minted — which is precisely what backstop 1 defers.