Config Validation

Config Validation

4-phase system to measure if your configuration is actually effective, not just present.

Phase 0: Structural (Coherence)

bash tests/test-config.sh

30 checks: hooks exist and are executable, rules have valid globs, deny list covers essentials, no contradictions.

Phase 1: Behavioral (Session Metrics)

Automatic after bootstrap. Every session generates:

block-destructive.sh → /tmp/counter → session-report.sh → ~/.claude/metrics/{slug}/{date}.json
lint-on-save.sh      → /tmp/counter ↗

Tracks: errors_added, hook_blocks, lint_blocks, files_touched, rules_matched, rule_coverage, commits.

Optional human-readable report: export FORGE_SESSION_REPORT=true.

Phase 2: Coverage (Rule Effectiveness)

/forge rule-check

Cross-references rule globs against git log --name-only:

Classification	Match rate	Action
Active	> 50%	Keep
Occasional	10-50%	Evaluate token cost
Inert	< 10%	Remove candidate

Also reports file coverage: % of touched files under at least one rule.

Phase 3: Practice Effectiveness

Binary recurrence model in practices/metrics.yml:

Practice activated → monitoring → 5 checks without recurrence → validated
                                  error recurred → failed

Phase 4: Comparative (Benchmark)

/forge benchmark

Same task, two worktrees:

• Full config — complete .claude/ setup
• Minimal config — only CLAUDE.md + basic settings.json

Compares: files created, tests passing, lint issues, errors.

Task definitions in tests/benchmark-tasks/ for python-fastapi, react-vite-ts, swift-swiftui, node-express, go-api, and generic.

Cost: 2 Claude Code executions. Always opt-in.