Enhancement: Skill Compact to Automatically Detect Duplicate/Overlapping Skills and Merge Them into a Single Better Skill

[Riseonelimit]

Overview
As a SkillNote registry grows, skill sprawl becomes inevitable. Teams end up with multiple skills that do the same thing — written at different times, by different people, with slightly different names. Skill Compact is an automated analysis feature that detects redundant or overlapping skills, shows you exactly how they overlap, and merges them into one consolidated, higher-quality skill.

---

Problem

Skill registries degrade over time in a predictable pattern:

Week 1:  write-tests
Week 3:  unit-testing-guide
Week 6:  how-to-write-tests
Week 9:  jest-test-conventions
Week 12: testing-best-practices

All five cover the same domain. Nobody notices because:

• Skills are created by different people with no visibility into what already exists
• There's no similarity check at creation time
• Search only works if you know what to search for
• The registry keeps growing, agents get more confused, and quality drops

The result: agents pick the wrong skill, get conflicting instructions from similar skills, or miss the best one entirely.

---

Proposed Feature: Skill Compact

Skill Compact is a registry health tool that runs an analysis pass over all skills, identifies clusters of overlapping or duplicate skills, and either auto-merges them or walks the user through a guided merge flow.

---

How It Works

Step 1 — Analysis

SkillNote analyzes every skill in the registry across three dimensions:

• Semantic similarity — do the skill bodies cover the same concepts and instructions?
• Description overlap — do the trigger descriptions match the same agent intents?
• Usage overlap — are these skills being called for the same tasks in analytics?

Each pair of skills gets a similarity score (0–100%). Pairs above a configurable threshold (default: 75%) are flagged as candidates for compaction.

Step 2 — Cluster Report

SkillNote groups flagged skills into clusters and presents a Compaction Report:

⚠️  Cluster: Testing (5 skills, avg similarity 87%)
    • write-tests              — 234 calls, ★ 4.1
    • unit-testing-guide       — 89 calls,  ★ 3.7
    • how-to-write-tests       — 12 calls,  ★ 3.2
    • jest-test-conventions    — 67 calls,  ★ 4.4
    • testing-best-practices   — 45 calls,  ★ 3.9

⚠️  Cluster: Code Review (3 skills, avg similarity 81%)
    • code-review-checklist    — 178 calls, ★ 4.6
    • pr-review-guide          — 55 calls,  ★ 4.0
    • review-conventions       — 23 calls,  ★ 3.5

Each cluster shows call counts and ratings so you immediately know which skills have been battle-tested and which are dead weight.

Step 3 — Merge

For each cluster, the user has three options:

Option A — Auto Merge
SkillNote uses an LLM pass to synthesize all skills in the cluster into one consolidated skill. It:

• Combines unique instructions from each skill
• Removes duplicated content
• Resolves contradictions by preferring higher-rated versions
• Generates a new, cleaner description
• Preserves the slug of the highest-rated skill
• Archives (not deletes) the source skills with a compacted_into reference

Option B — Guided Merge
A side-by-side diff editor showing all skills in the cluster. The user picks a base skill, cherry-picks sections from the others, and manually authors the final merged version. SkillNote handles archiving the rest.

Option C — Dismiss
Mark the cluster as intentionally separate (e.g. write-tests and jest-test-conventions are different enough on purpose). SkillNote won't flag this cluster again unless the skills change significantly.

---

Compaction Report UI

• Accessible from a "Run Compact" button in the dashboard header or Settings page
• Can be scheduled to run automatically (daily / weekly) and notify via the dashboard
• Each cluster shown as a card with an overlap heatmap, side-by-side preview, and merge options
• A Registry Health Score shown on the dashboard — a single number reflecting how compact and non-redundant your skill registry is (e.g. "87/100 — 3 clusters detected")
• After a compaction run, a summary is saved to history: "Compacted 5 skills → 1, freed 4 slots, registry health: 94/100"

---

What the Merged Skill Looks Like

Before (5 separate skills, conflicting and redundant):

# write-tests
Use Jest. Always test the happy path.

# jest-test-conventions  
Use describe/it blocks. Mock external deps with jest.mock().

# testing-best-practices
Cover edge cases. Aim for 80% coverage.

After (1 compacted skill):

# write-tests  ✦ compacted from 5 skills

Framework: Jest
Structure: Use describe/it blocks
Mocking: Use jest.mock() for all external dependencies
Coverage: Aim for 80% — cover happy path and edge cases
Always test: input validation, null inputs, error states

---

Archival — Not Deletion

Compacted skills are never hard-deleted. They get an archived status with a compacted_into: <slug> field. This means:

• Full version history is preserved
• Any agent that still calls the old skill slug gets transparently redirected to the new one
• You can undo a compaction and restore the originals at any time

---

Analytics Integration

• Post-compaction: track whether the merged skill's call count and rating outperforms the average of what it replaced
• Show a before/after comparison: "Registry went from 47 skills → 31 skills, avg rating improved from 3.8 → 4.2"
• Flag clusters that are growing (new similar skills being added repeatedly) as a signal that a skill is missing something

---

Why This Matters

Without Skill Compact	With Skill Compact
Registry grows unbounded	Registry stays lean and intentional
Agents pick conflicting/duplicate skills	One authoritative skill per domain
Quality diluted across many versions	Best knowledge consolidated in one place
No visibility into redundancy	Health score + cluster report at a glance
Manual cleanup is tedious and never happens	Automated analysis + one-click merge

---

Suggested Implementation Phases

Phase 1 — Detection only
Run similarity analysis, show the cluster report with overlap scores. No merging yet. Lets users see the problem before committing to a solution.

Phase 2 — Guided merge
Side-by-side diff editor. User manually authors the merged skill. SkillNote handles archival and slug redirection.

Phase 3 — Auto merge + Health Score
LLM-powered auto merge, registry health score on the dashboard, scheduled compaction runs.

---

Related

• Pairs directly with the 50+ skills scaling problem
• Complements Skill Workflows — compact skills first, then build workflows on a clean registry
• Health Score would be a natural addition to the existing analytics dashboard