Skip to content

research: name every flagged referenced file, not just divergent ones - #60

Merged
Ap6pack merged 1 commit into
mainfrom
claude/report-flagged-referenced-files
Sep 11, 2026
Merged

Ap6pack merged 1 commit into
mainfrom
claude/report-flagged-referenced-files

Conversation

@Ap6pack

@Ap6pack Ap6pack commented Sep 11, 2026

Copy link
Copy Markdown
Owner

What the first scan showed

Run 34654103313 (60 skills, seed 20260816):

skills scanned:              58
  naming referenced files:   29   (50%, consistent with step 1's 47.2%)
referenced paths tried:      120
  fetched:                   107
  unreachable (named, 404):   13

verdicts on referenced files:
  CLEAN        104
  SUSPICIOUS     3

DIVERGENCE (SKILL.md clean, referenced file flagged): 0

An empty divergence set is a real result and it argues against the hypothesis that motivated this tooling: the unread surface is confirmed large, but in this draw nothing is hiding in it.

Why this change

Three files were flagged and the report named none of them. They fall outside the divergence set because their skill's SKILL.md was itself non-clean — documentation and code agreeing, which is the honest case, and the opposite of concealment. That is a reason to label them, not to drop them. As written, the only way to recover which skills, which paths, and which rules was to re-run the scan.

Two additions:

  • All flagged referenced files, each with its parent doc verdict alongside the file verdict, so concordant and divergent cases are distinguishable on sight rather than by absence.
  • Named-but-unreachable paths, grouped by skill. The 13 404s are already kept out of the clean count, but a reader could not tell which skills carry that unknown. "We could not read it" and "there was nothing to read" are different facts and only one of them supports a clean claim.

Timeout goes to 120 minutes so a 300-skill scan fits: roughly 900 registry fetches at the 0.55s pacing, plus rule-engine time on every fetched file.

Not changed

No detection logic, no thresholds, no extraction rules. Report-only, so the numbers above stay comparable across runs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL


Generated by Claude Code

The first step-2 scan (60 skills, seed 20260816) returned a divergence set
of zero with three referenced files flagged SUSPICIOUS. Both facts matter,
but the report only printed the first, so the three were invisible: which
skills, which paths, which rules, all unrecoverable without re-running.

A file whose SKILL.md already flagged is not concealment -- documentation
and code agreeing is the honest case, and it is precisely why those three
are outside the divergence set. That is a reason to label them, not to
drop them. A flag you cannot see is a flag you cannot check.

Also lists the named-but-unreachable paths by skill. Thirteen of 120 paths
404'd; they are already excluded from the clean count, but a reader has no
way to tell which skills carry that unknown.

Raises the job timeout to 120 minutes so a 300-skill scan (~900 registry
fetches at 0.55s, plus rule-engine time per file) fits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DNoTXU8k3pfSBzR7aJubqL
@Ap6pack
Ap6pack merged commit 3520217 into main Sep 11, 2026
5 checks passed
@Ap6pack
Ap6pack deleted the claude/report-flagged-referenced-files branch September 11, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants