Skip to content

fix(skill): align author-saf-technique with the research validator - #232

Open
bishnubista wants to merge 1 commit into
secure-agentic-framework:mainfrom
bishnubista:fix/author-skill-validator-alignment
Open

fix(skill): align author-saf-technique with the research validator#232
bishnubista wants to merge 1 commit into
secure-agentic-framework:mainfrom
bishnubista:fix/author-skill-validator-alignment

Conversation

@bishnubista

@bishnubista bishnubista commented Sep 4, 2026

Copy link
Copy Markdown
Member

Summary

Aligns the author-saf-technique skill and the blank attestation template with what scripts/validate-technique-research.py actually enforces, and fixes one path and two portability problems in the clean-room instructions.

What changed

  • Required prohibited_inputs fragments documented. The validator requires six exact substrings in prohibited_inputs for generation_mode: clean_room (validate-technique-research.py:519-530). Four of them were absent from the skill, and previous conversation appeared nowhere in the skill or template, so an attestation written in the skill's own wording failed validation. references/clean-room-generation.md now lists the six strings verbatim, and research/templates/technique/clean-room-attestation.yml carries them as a commented block to uncomment when a packet is converted to clean_room (the standard-mode scaffold keeps its not-applicable placeholder, since those inputs are permitted in standard reruns).
  • Canonical paths corrected. SKILL.md listed traceability-ledger.yml as a bare path among otherwise-complete canonical paths; it lives at research/techniques/SAF-TXXXX/traceability-ledger.yml. references/methodology.md now names techniques/SAF-TXXXX/detection-rule.yml, the one packet artifact outside the research packet.
  • GHSA search reconciled with clean-room mode. references/breach-and-vulnerability-research.md mandated a GitHub Security Advisories search that clean-room-generation.md forbids; the bullet now carries the clean-room exception (open an exact advisory URL only after non-GitHub discovery).
  • Portable scratch root and a single retry convention. The frozen-bundle path was hardcoded to /private/tmp/..., which does not exist on the ubuntu-latest CI runner. It is now $SAF_CLEANROOM_ROOT, defaulting to ${TMPDIR:-/tmp}/saf-all-cleanroom, and must resolve to an absolute path outside the checkout. Attempt directories are created fail-closed (plain mkdir, next -rN on collision) so stale or concurrent attempts cannot mix under one FREEZE.sha256; freeze provenance is recorded in the bundle's integration-notes.yml. Retries and normalization re-freezes use SAF-TXXXX-rN; existing attestations had independently invented six different suffixes (-normalized, -retry, -r2, -retry3, -validator, -isolated).
  • "Strict validator" wording. No --strict flag exists; strict checking is the default and --draft disables it. Wording now says "default (non---draft) mode." The --draft help string said it only skipped "completion statuses"; it now states which strict-only checks it relaxes (including accepting the placeholder zero history SHA) and that schema, cross-file joins, and core clean-room assertions remain enforced.

Testing

  • python3 -m unittest discover -s scripts -p "test_*.py" — 15 tests, OK
  • validate-technique-research.py --all, validate-framework-model.py, generate-technique-catalog.py --check, validate-detection-registry.py, generate-detection-coverage.py --check, generate-mitigation-catalog.py --check — all exit 0
  • Reproduction of the original defect: in a scratch copy, rewriting SAF-T1309's prohibited_inputs in the skill's verbatim wording fails with four prohibited_inputs missing ... errors; the unmodified packet passes.
  • Template verified both ways in a scratch copy: a standard scaffold from scripts/new-technique.py produces zero clean-room errors; the same scaffold converted to clean_room per the documented procedure produces zero prohibited_inputs missing errors.

@bishnubista
bishnubista force-pushed the fix/author-skill-validator-alignment branch from e2e389f to 3d14485 Compare September 4, 2026 04:16
Document the six prohibited_inputs fragments the validator requires for
clean-room attestations and carry them in the blank template; correct the
canonical traceability-ledger and detection-rule paths; reconcile the GHSA
search instruction with clean-room discovery rules; replace the macOS-only
frozen-bundle path with a portable scratch root, a single -rN retry
convention, and fail-closed attempt-directory creation; fix "strict
validator" wording and the --draft help string to match the validator's
actual modes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: bishnubista <bista.developer@gmail.com>
@bishnubista
bishnubista force-pushed the fix/author-skill-validator-alignment branch from 3d14485 to 3a455d6 Compare September 4, 2026 04:17
@bishnubista
bishnubista requested a review from fkautz September 4, 2026 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant