Skip to content

fix(runner): prevent scanner evidence path collisions - #54

Merged
steipete merged 1 commit into
mainfrom
fix/evidence-path-collisions
Sep 16, 2026
Merged

steipete merged 1 commit into
mainfrom
fix/evidence-path-collisions

Conversation

@steipete

@steipete steipete commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Scanning skills/a, skills/a!, and skills/a-2 wrote only two raw scanner report files: the generated a-2 suffix was not reserved, so a later target overwrote the earlier target's evidence. Valid custom scanner IDs such as custom and custom- also wrote the same sanitized filename.

Reserve every emitted path, including generated suffixes, and use the same allocator for scanner filenames within each run. Existing noncolliding paths stay unchanged. The artifact's outputPath always points to that result's own raw evidence. Operator docs and the Unreleased changelog describe the behavior.

Scope

  • CLI behavior

Security / Trust Impact

  • Security/trust impact explained

Prevents silent loss and misattribution of scanner evidence. No scanner execution, gate policy, or artifact schema changes.

Verification

  • go test -count=1 ./...
  • go vet ./...
  • Built CLI --help
  • Focused output-bundle regression and existing path-compatibility tests
  • make docs-site
  • Independent Codex review through P2

Red: the new regression tests fail on main, showing duplicate outputPath values and overwritten raw JSON. They cover a real suffix occurring both before and after a collision, profile/target collisions, and valid custom scanner IDs that sanitize to the same filename.

Built CLI proof with synthetic fixtures: before the fix, three discovered targets and three custom scanners each produced only two distinct evidence paths. The rebuilt CLI produces three distinct files for each case; every custom scanner file contains its matching scanner ID. Regression tests verify the exact raw bytes for every result. Custom scanner proof uses harmless local printf commands with explicit --sandbox off; no live scanner service or credentials were involved.

Notes

Generated build outputs are excluded. No release, tag, or version change.

@steipete
steipete requested review from a team and Patrick-Erichsen as code owners September 16, 2026 03:54
@clawsweeper

clawsweeper Bot commented Sep 16, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review in progress

ClawSweeper is reviewing this revision. This supersedes any previous blocked status.

View the workflow run.

@steipete
steipete merged commit 84ee99c into main Sep 16, 2026
9 checks passed
@steipete

Copy link
Copy Markdown
Contributor Author

Merged as 84ee99c.

Verified with go test -count=1 ./..., go vet ./..., focused output-bundle regression and existing path-compatibility tests, make docs-site, and built CLI --help. Independent Codex review was scoped-clean through P2.

Built CLI synthetic proof: skills/a, skills/a!, and skills/a-2 now retain three distinct scanner reports; before the fix there were two. Valid custom scanner IDs custom, custom-, and custom-2 likewise retain three distinct JSON files, each containing its matching scanner ID. Regression tests check exact raw bytes for every result, including opposite suffix orderings and profile collisions. The custom scanner fixture runs harmless local printf commands with explicit --sandbox off; no live scanner API or credentials were used.

Exact-head checks passed on d020d145283ffa6c1174b32fd11591876131b043: CI, CodeQL, and submission validation. No failures or retries. Main has been pulled and verified clean.

@steipete
steipete deleted the fix/evidence-path-collisions branch September 16, 2026 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant