test(ci): cover the mechanism that actually catches the outage - #423
Merged
Conversation
…ust the newer bug Only fleet-refs-audit's RETIRED_HANDLES self-match had test coverage. The USES regex and the stale/missing/ok verdict decision — the part that would have caught all three real outages this tool exists for (2026-08-26/27/28) — had zero tests. A regex bug there would silently miss exactly the class of breakage the whole tool was built to catch, which is a bigger risk than the bug just fixed. Extracted verdictFor(slug, real) into fleet-refs-audit-lib.mjs as a pure decision function — no network call, so it's directly testable — replacing the inline if/else in fleet-refs-audit.mjs with the same behavior. 10 new assertions: USES correctly matches plain and path-qualified uses: lines, correctly excludes local (./…) and docker:// references (no owner to be wrong about), matches multiple lines independently, tolerates indentation; verdictFor correctly reports ok/stale/unreadable for all four real outcomes resolve() can produce, including the failed-lookup case, which must report unreadable rather than silently pass OR falsely fail. Documented one known, narrow limitation rather than silently claiming full coverage: USES matches by line shape, not YAML structural position, so a run: block whose first physical line happens to read literally like a uses: key would be misread. Not observed in this fleet; a real YAML parse would close it at the cost of a dependency this script deliberately avoids. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y9rKLxddothnXEtY6KDziN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
After fixing the RETIRED_HANDLES self-match (#421), only that newer check had test coverage. The USES regex + redirect-resolution verdict — the actual mechanism that caught all three real outages this tool exists for (2026-08-26/27/28) — had zero tests. A regex bug there would silently miss exactly the class of breakage the whole tool was built to catch.
Change
Extracted
verdictFor(slug, real)intofleet-refs-audit-lib.mjsas a pure decision function (no network), replacing the inline if/else with identical behavior. 10 new assertions coverUSESparsing (plain + path-qualified references, local/docker exclusions, multiple lines, indentation) andverdictFor's four real outcomes (ok / stale-redirect / stale-missing / unreadable-on-failed-lookup).Documented one known limitation rather than claiming full coverage:
USESmatches by line shape, not YAML structural position — narrow, not observed in this fleet.Verified
npx tsx scripts/test/fleet-refs-audit.ts— 15/15 assertions.npm run verifygreen (test:unit auto-discovers the file).Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01Y9rKLxddothnXEtY6KDziN