feat(validator): add project-template drift check #663
Merged
Conversation
potiuk
approved these changes
Jul 1, 2026
potiuk
left a comment
potiuk
left a comment
Member
There was a problem hiding this comment.
Oh. Cool - it will be nice for any future changes :)
…visory) Add validate_project_template_drift (check apache#15) to tools/skill-and-tool-validator to mechanically compare projects/_template/ with projects/non-asf-example/ for structural drift, closing the Known gap in specs/project-agnosticism.md. Three SOFT advisory checks under the new template-drift category: 1. README file-list coherence: every file linked in the ## Files section of non-asf-example/README.md must exist on disk. 2. Undocumented files: every .md file in non-asf-example/ (other than README.md) must be mentioned somewhere in its README. 3. Shared-file h2 alignment: for each .md file present in both profiles, h2 section headings are compared. project.md and README.md are excluded since they differ intentionally by organization profile (org-inherited blocks, narrative structure). 22 new tests in TestProjectTemplateDrift cover all checks, the live-repo clean-state assertion, DocToc-stripping, exclusion of parent-traversal and external URLs, and the project.md/README.md exemptions. The live tree produces zero template-drift violations. Updates specs/project-agnosticism.md Known gaps to document the new check and its scope. Generated-by: Claude (Opus 4.7)
potiuk
force-pushed
the
project-template-drift-check
branch
from
55e16e9 to
0ca030f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add validate_project_template_drift (check #15) to
tools/skill-and-tool-validator to mechanically compare
projects/_template/ with projects/non-asf-example/ for structural
drift, closing the Known gap in specs/project-agnosticism.md.
Three SOFT advisory checks under the new template-drift category:
section of non-asf-example/README.md must exist on disk.
than README.md) must be mentioned somewhere in its README.
profiles, h2 section headings are compared. project.md and
README.md are excluded since they differ intentionally by
organization profile (org-inherited blocks, narrative structure).
22 new tests in TestProjectTemplateDrift cover all checks, the
live-repo clean-state assertion, DocToc-stripping, exclusion of
parent-traversal and external URLs, and the project.md/README.md
exemptions. The live tree produces zero template-drift violations.
Updates specs/project-agnosticism.md Known gaps to document the
new check and its scope.
Generated-by: Claude (Opus 4.7)
Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespassesuv run pytest/ruff check/mypypasses(
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)