Skip to content

guard: ci_gate_runability verdicts are location-independent (removes surviving temp-dir fail-open) - #726

Open
matt82198 wants to merge 5 commits into
mainfrom
guard/runability-tempdir-failopen
Open

matt82198 wants to merge 5 commits into
mainfrom
guard/runability-tempdir-failopen

Conversation

@matt82198

Copy link
Copy Markdown
Owner

Problem (VERIFIED P1, regression lens)

tools/ci_gate_runability.py carried a second copy of the Windows temp-dir exemption that commit 6afb94ba claims to have deleted. 6afb94ba removed it from find_file_on_disk() but missed the copy in check_workflow():

repo_root_path = Path(repo_root).resolve()
is_temp_fixture = 'AppData' in str(repo_root_path) and 'Temp' in str(repo_root_path)

if step['run'] and not is_temp_fixture:   # <- check (c) silently skipped

The gate's verdict therefore depended on where the checkout lives on disk, not on what it contains. Byte-identical fixtures:

root verdict
C:\Users\...\AppData\Local\Temp\tmpXXXX [OK] All CI gates are runnable on PR eventsrc 0
C:\Users\...\aesop-repro-nontemp ... references missing file: tools/verify_definitely_missing.pyrc 1

find_file_on_disk()'s own comment (:253-259) forbids exactly this. The gate is a blocking step in ci.yml, so this is fail-open on any runner whose workspace sits under a temp-shaped path — the "green can mean never ran" incident class this Guardrail (G2.5) exists to prevent.

Why the exemption existed, and the correct replacement

Test fixtures built workflow YAML in temp dirs citing tools/*.py the fixture never created, so check (c) reported them missing. The exemption made the gate recognise its own fixtures.

6afb94ba already fixed this correctly on the test side: the fixture harness stubs every tools/*.py its workflow references (tests/test_ci_gate_runability.py:38-47). So the right replacement was already on disk — the surviving copy is pure dead fail-open. No --root change and no new skip flag is needed; a skip flag would just re-introduce the hole.

TDD

Reproduced first — TestCIGateRunabilityLocationIndependence was RED on 2 of 4 before the fix:

FAIL: test_missing_file_flagged_under_temp_shaped_root
  AssertionError: 0 != 1 : Gate must flag the missing file regardless of where the repo lives
FAIL: test_source_has_no_path_shape_sniffing
  AssertionError: ["314: is_temp_fixture = 'AppData' in str(repo_root_path) ...", "316: if step['run'] and not is_temp_fixture:"] != []

Roots are built at <tmp>/AppData/Temp/repo — trigger-shaped on every OS, so the repro is red on Linux CI too, not just Windows — and at <tmp>/ordinary/workspace/repo. The four tests assert:

  1. a missing file is flagged under a temp-shaped root;
  2. both roots yield identical verdicts and finding counts;
  3. a fixture that provides the file it cites stays clean (no inverted false positive);
  4. a static scan of the module source fails on any future copy of the path sniff (rules-as-code — the prose fix regressed once already because a second copy survived).

Verification

  • tests.test_ci_gate_runability: 23/23 OK
  • Python suite: Ran 4346 tests — OK (skipped=20), rc 0
  • Shell suite: 14/14, rc 0
  • Node suite: 308/309 — the single failure is a pre-existing node --test IPC flake (fleet-cli.test.mjs: "Unable to deserialize cloned data"); isolated re-run 7/7, rc 0
  • Gate battery all rc 0: ci_gate_runability (real repo clean), verify_test_suite_count --check, claudemd_lint, claudemd_contract, claudemd_sync_gate --check, encoding_lint, watcher_linter --check, verify_test_coverage --check, spec_contract_validator --check, workflow_model_linter --check, subprocess_guard --check --baseline .subprocess-guard-baseline.json
  • secret_scan --staged rc 0; pre-push gates green

tools/CLAUDE.md also corrects a stale claim on the same entry ("staged (wire into ci.yml after #596)" — it is wired, ci.yml:148) and records the location-independence invariant.

🤖 Generated with Claude Code

check_workflow() carried a second copy of the Windows temp-dir exemption that
6afb94b deleted from find_file_on_disk(): the missing-file check (c) ran only
when the resolved repo root did NOT contain both 'AppData' and 'Temp'. Two
byte-identical fixtures therefore produced different verdicts -- [OK] rc0 under
a temp-shaped path, "references missing file" rc1 outside it -- and any runner
whose workspace lives under such a path silently lost file-existence checking
entirely. The file's own comment at find_file_on_disk() forbids exactly this.

The exemption originally existed because tests/test_ci_gate_runability.py built
workflow fixtures in temp dirs that cited tools/*.py the fixture never created.
6afb94b already solved that correctly on the test side: the fixture harness
stubs every tools/*.py its workflow references. No CWD/path sniffing and no new
skip flag is needed -- a skip flag would just re-introduce the fail-open.

TDD: TestCIGateRunabilityLocationIndependence reproduces first. Roots are built
at <tmp>/AppData/Temp/repo (trigger-shaped on every OS, so the repro is red on
Linux CI too, not only Windows) and <tmp>/ordinary/workspace/repo, and the tests
assert identical verdicts, that a missing file is flagged under a temp-shaped
root, and that a present file stays clean (no inverted false positive). A static
scan of the module source fails on any future copy of the path sniff.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@matt82198 matt82198 added the merge-queue Queued for the merge-queue advancer daemon label Aug 3, 2026
matt82198 and others added 4 commits August 3, 2026 14:15
…: lines

#751 moved the per-tool index out of tools/CLAUDE.md into each tool's own
INDEX: header line, from which tools/INDEX.md is generated, because the inline
list was the top merge-queue conflict surface. This branch was cut before that
and still carried its documentation update as an edit to the old inline list.

Resolved by taking main's 33-line tools/CLAUDE.md and porting this branch's
updated description(s) into the tool's own INDEX: line, then regenerating
tools/INDEX.md: ci_gate_runability.py

No documentation content lost; only the obsolete inline index lines are gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-sniff scan

The extracted INDEX: docstring line quoted the removed temp-dir sniff verbatim,
so the tool's own static no-path-sniffing guard flagged its documentation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-queue Queued for the merge-queue advancer daemon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant