Skip to content

About

A meta-skill framework for authoring, evaluating, and optimizing AI Agent Skills — with progressive disclosure, deterministic validation, and anti-hallucination guardrails.

Resources

Stars

1 star

Watchers

1 watching

Forks

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

skill-authoring

A meta-skill framework for authoring, evaluating, and optimizing AI Agent Skills — with progressive disclosure, deterministic validation, and anti-hallucination guardrails.

Skills are "operation manuals written for AI to read." This framework teaches Agents how to write high-quality Skill documents that other Agents can follow and execute reliably.


What is a Skill?

A Skill is a structured instruction set that enables an AI Agent to perform a specific task reliably. Unlike free-form prompts, Skills use:

  • YAML frontmatter for discovery metadata (name, triggers, scope boundaries)
  • Hard constraints (§0) with RFC 2119 keywords (MUST / SHOULD / MAY) for non-negotiable rules
  • Staged workflows with checklists, gates, and error recovery for step-by-step execution
  • Deterministic validation scripts to verify output without relying on LLM self-assessment
  • Progressive disclosure (3-layer loading) to minimize context window consumption

Why This Framework?

LLMs are powerful generators but unreliable self-validators. Common failure modes include:

Problem Framework Solution
Agent skips steps Anti-Skip Trio: Echoing + TICK Checklist + Gate blocks
Agent hallucinates success Deterministic validation via scripts/ (code verifies, not the model)
Context window overflow 3-layer progressive disclosure (L1 → L2 → L3)
Ambiguous instructions RFC 2119 constraint layering (MUST vs SHOULD vs MAY)
Inconsistent quality Automated scoring rubric + audit workflow

Quick Start

For Humans — Install the Skill

Copy the skill-authoring/ directory into your Agent's skill search path:

cp -r skill-authoring/ <your-workspace>/skills/skill-authoring/

For Agents — Use the Skill

Once installed, the Agent can discover and invoke this skill through standard skill search mechanisms. The YAML frontmatter triggers activation:

name: skill-authoring
description: >
  System-level meta-skill for writing, evaluating, and optimizing Agent Skill
  documents. Triggered when creating new Skills, refactoring existing Skills,
  reviewing SKILL.md quality, or understanding the Skill framework.
  Not applicable to executing specific business skills.

Workflows

ID Workflow Mode Description
001 Create New Skill 🤝 Collaborative 6-stage end-to-end process. Agent drafts, user approves at key gates.
002 Optimize Existing Skill 🤖 Autonomous 3-stage review cycle with 30-item scoring. Max 3 retry rounds.
003 Quality Check 🤖 Autonomous Scoring + Audit + Semantic Review → unified PASS/FAIL verdict.

Internal shared workflows (_shared/, 1xx series):

ID Workflow Called By
101 Scoring 003
102 Audit 003
103 Verify-Repair Loop 001, 002

Directory Structure

skill-authoring/
├── SKILL.md              ← Core instructions (≤500 lines, L2)
├── references/           ← Domain knowledge & patterns (L3, flat-indexed)
│   ├── 001.skill-template.md
│   ├── 002.checklist-patterns.md
│   ├── 003.quality-rubric.md
│   ├── 004.history-schema.md
│   ├── 005.deterministic-validation.md
│   ├── 006.harness-principles.md
│   └── 007.llm-validation-prompts.md
├── workflows/            ← Execution procedures
│   ├── 001.create-skill.md
│   ├── 002.optimize-skill.md
│   ├── 003.quality-check.md
│   └── _shared/
│       ├── 101.scoring.md
│       ├── 102.audit.md
│       └── 103.verify-repair-loop.md
├── scripts/              ← Deterministic validation (tiny CLIs)
│   ├── validate-skill.sh
│   ├── validate-metadata.sh
│   └── check-health.sh
├── assets/               ← Static resources
└── state/                ← Runtime state (live layer)
    ├── TODO.md
    └── history.jsonl

Three-Layer Architecture

Layer Files Purpose Load Timing
L1 Metadata YAML frontmatter Discovery & routing Agent startup
L2 Core SKILL.md Instructions & workflow routing On skill activation
L3 Reference references/, scripts/ Deep knowledge & validation On-demand per stage

Key Concepts

Hard Constraints (§0)

Twelve non-negotiable rules placed at the top of every Skill, enforced with <caution> blocks. These cover document size limits, structural requirements, terminology consistency, and more.

Anti-Skip Trio

Three mechanisms to prevent Agents from skipping steps:

  1. Echoing — Each stage opens with "Current execution: Stage X — <Name>"
  2. TICK Checklist — Binary - [ ] items requiring explicit YES/NO judgment
  3. Gate Blocks — <gate> elements that block progression until all checks pass

Deterministic Validation

Machine-verifiable rules are enforced by scripts, not by the model's self-assessment:

# Validate a Skill document
bash scripts/validate-skill.sh path/to/SKILL.md

# Validate YAML frontmatter only
bash scripts/validate-metadata.sh path/to/SKILL.md

# Run a full health check
bash scripts/check-health.sh path/to/skill-directory/

Verify-Repair Loop

An iterative quality convergence cycle:

Quality Check → PASS → Done
Quality Check → FAIL → Repair → Re-check (max 3 cycles)

Memory Architecture

File Memory Type Purpose
SKILL.md Procedural How to do things
workflows/ Procedural Detailed execution sequences
references/ Semantic Domain knowledge and rules
state/history.jsonl Episodic What happened before
state/TODO.md Working What to do next

Validation Scripts

All scripts follow a consistent interface convention:

  • Input: File paths via CLI arguments ($1, $2)
  • Success: stdout messages + exit code 0
  • Failure: stderr messages with actionable descriptions + exit code 1
  • No dependencies: Pure bash, no external libraries required
# Example output on failure
$ bash scripts/validate-skill.sh bad-skill/SKILL.md
FAIL: Line count is 623 (MUST ≤ 500)
FAIL: Missing §0 Hard Constraints section
VALIDATION FAILED: 2 error(s)

Design Philosophy

This framework is built on four principles from the LLM Harness Pattern:

  1. Deterministic Validation > Self-Review — Code verifies; the model generates.
  2. Execute-Verify Loop — Every stage output is validated; failures get injected feedback.
  3. Constraint Inheritance — §0 rules are global; sub-file <caution> rules are local additive.
  4. Agent Legibility — No tribal knowledge. Enums exhaustive, paths explicit, examples copy-pasteable.

Contributing

When contributing to this framework, ensure your changes:

  1. Pass all validation scripts: bash scripts/validate-skill.sh skill-authoring/SKILL.md
  2. Use RFC 2119 keywords for any new constraints
  3. Keep SKILL.md within the 500-line / 5000-token budget
  4. Follow flat-indexing (no chained references A→B→C)

Acknowledgments

This framework was inspired in part by:


License

MIT

About

A meta-skill framework for authoring, evaluating, and optimizing AI Agent Skills — with progressive disclosure, deterministic validation, and anti-hallucination guardrails.

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages