Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,45 @@ This replaces the previous pattern of agents fetching `knowledge/*.md` files at

---

## Hooks — Deterministic Enforcement

The plugin includes hooks that run deterministic checks during migrations. Unlike skills and agent instructions (which the model can choose to ignore), hooks execute as shell commands at specific lifecycle points and can **block** operations or **inject warnings** into the agent's context.

### `hooks.json`

| Hook | Event | Matcher | What it does |
|------|-------|---------|-------------|
| Secret detection | `preToolUse` | `create\|edit` | Hard-denies file writes containing hardcoded secrets (passwords, tokens, API keys). Forces use of `${{ secrets.NAME }}`. Uses `permissionDecision: "deny"`. |
| File deletion guard | `preToolUse` | `bash` | Hard-denies `rm` operations outside `.github/ci-archive/`. Prevents accidental deletion of application source code. |
| Quality check + actionlint | `postToolUse` | `create\|edit` | After any workflow file write, injects `additionalContext` with: unpinned actions (tag vs SHA), placeholder text (TODO/FIXME), over-broad permissions (`write-all`), missing permissions block, and actionlint errors. The agent sees these on the same turn. |
| **Quality gate** | `agentStop` | — | Scans ALL workflow files when the agent finishes a turn. If any have issues, returns `decision: "block"` forcing the agent to take another turn to fix them. Safety valve releases after 3 attempts to prevent infinite loops. |
| **Migration scorecard** | `sessionEnd` | — | Appends an entry to `.github/MIGRATION-SCORECARD.md` with session ID, timestamp, completion reason, and workflow counts (total / clean / with-issues). Multiple passes show quality progression. Audit artifact for migration quality tracking. |

### Why hooks matter

The `migration-core` skill already contains guardrails as agent instructions. Hooks add a **deterministic layer** — the agent can't bypass them. This is the difference between "please don't delete files outside ci-archive" (instruction) and "the system will reject the tool call" (hook).

**actionlint** runs in three hooks: `postToolUse` (per-file, immediate feedback), `agentStop` (all files, blocks completion), and `sessionEnd` (final scorecard counts). The agent cannot skip or ignore lint errors — the quality gate blocks completion until they're fixed.

**The quality gate** (`agentStop`) is the key enforcement mechanism. Instead of just warning after each file write, it checks all workflows at the end of every agent turn and forces continuation until they pass. This works in both CLI interactive mode and cloud agent jobs.

### Enabling hooks

Hooks are installed automatically with the plugin. To verify:

```bash
copilot
/hooks list
```

To disable hooks temporarily (e.g., for debugging):

```bash
copilot --disable-hooks
```

---

## Customizing Skills

Customizing skills is the CLI plugin's equivalent of editing the `knowledge/` knowledge base in the [cloud-agent deployment](../docs/deployment.md). Because the plugin ships content **locally**, your edits take effect on the next `copilot plugin install ./plugin`—no `.github-private` push, no MCP round-trip.
Expand Down
46 changes: 46 additions & 0 deletions plugin/hooks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"version": 1,
"hooks": {
"preToolUse": [
{
"type": "command",
"description": "Block hardcoded secrets in file writes",
"matcher": "create|edit",
"timeoutSec": 10,
"bash": "INPUT=$(cat); CONTENT=$(echo \"$INPUT\" | jq -r '.toolArgs.content // .toolArgs.new_string // empty' 2>/dev/null); [ -z \"$CONTENT\" ] && echo '{}' && exit 0; FOUND=0; echo \"$CONTENT\" | while IFS= read -r line; do echo \"$line\" | grep -qiE '(password|secret|token|api[_-]?key)\\s*[:=]' 2>/dev/null || continue; echo \"$line\" | grep -qF '${' 2>/dev/null && continue; echo \"$line\" | grep -qiE '[:=]\\s*.{8,}' 2>/dev/null && echo 'HIT'; done | grep -q 'HIT' && echo '{\"permissionDecision\":\"deny\",\"permissionDecisionReason\":\"Blocked: hardcoded secret detected. Use GitHub Secrets (${{ secrets.NAME }}) instead.\"}' || echo '{}'"
},
{
"type": "command",
"description": "Guard against file deletion outside ci-archive",
"matcher": "bash",
"timeoutSec": 10,
"bash": "INPUT=$(cat); CMD=$(echo \"$INPUT\" | jq -r '.toolArgs.command // .toolArgs // empty' 2>/dev/null); if echo \"$CMD\" | grep -qE 'rm\\s+(-[rfi]+\\s+)*' 2>/dev/null; then TARGETS=$(echo \"$CMD\" | grep -oE '\\S+' | grep -v '^rm$' | grep -v '^-' || true); for t in $TARGETS; do if echo \"$t\" | grep -qF '..'; then echo '{\"permissionDecision\":\"deny\",\"permissionDecisionReason\":\"Blocked: path traversal (..) not allowed in delete operations.\"}'; exit 0; fi; done; for t in $TARGETS; do case \"$t\" in */.github/ci-archive/*) ;; *.github/ci-archive/*) ;; *) echo '{\"permissionDecision\":\"deny\",\"permissionDecisionReason\":\"Blocked: file deletion only allowed inside .github/ci-archive/.\"}'; exit 0 ;; esac; done; fi; echo '{}'"
}
],
"postToolUse": [
{
"type": "command",
"description": "Quality check and actionlint on workflow files after write",
"matcher": "create|edit",
"timeoutSec": 60,
"bash": "INPUT=$(cat); ARGS=$(echo \"$INPUT\" | jq -c '.toolArgs' 2>/dev/null); FILE=$(echo \"$ARGS\" | jq -r '.file_path // .path // empty' 2>/dev/null); echo \"$FILE\" | grep -q '.github/workflows/' || exit 0; [ -f \"$FILE\" ] || exit 0; if ! command -v actionlint >/dev/null 2>&1; then if [ \"$(uname)\" = 'Linux' ]; then V='1.7.11'; EXPECTED='900919a84f2229bac68ca9cd4103ea297abc35e9689ebb842c6e34a3d1b01b0a'; TGZ=\"/tmp/actionlint_${V}_linux_amd64.tar.gz\"; curl -fsSL -o \"$TGZ\" \"https://github.com/rhysd/actionlint/releases/download/v${V}/actionlint_${V}_linux_amd64.tar.gz\" 2>/dev/null; ACTUAL=$(sha256sum \"$TGZ\" 2>/dev/null | awk '{print $1}'); if [ \"$ACTUAL\" != \"$EXPECTED\" ]; then ESCAPED='actionlint install FAILED: checksum mismatch. Linting skipped — treat workflows as unverified.'; printf '{\"additionalContext\":\"%s\"}' \"$ESCAPED\"; exit 0; fi; tar xz -C /tmp -f \"$TGZ\" actionlint 2>/dev/null && install -m 755 /tmp/actionlint /usr/local/bin/actionlint 2>/dev/null; rm -f \"$TGZ\"; elif command -v brew >/dev/null 2>&1; then brew install actionlint 2>/dev/null; fi; fi; if ! command -v actionlint >/dev/null 2>&1; then ESCAPED='actionlint not available (install failed). Linting skipped — treat workflows as unverified.'; printf '{\"additionalContext\":\"%s\"}' \"$ESCAPED\"; exit 0; fi; W=''; grep -qE 'uses:\\s+[^@]+@v[0-9]' \"$FILE\" 2>/dev/null && W=\"${W}- Unpinned actions: use full SHA commit refs\\n\"; grep -qiE '(TODO|FIXME|CHANGEME|PLACEHOLDER|XXX)' \"$FILE\" 2>/dev/null && W=\"${W}- Placeholder text found: replace before merging\\n\"; grep -qE 'permissions:\\s*write-all' \"$FILE\" 2>/dev/null && W=\"${W}- Over-broad permissions: replace write-all with least-privilege\\n\"; grep -qE '^permissions:' \"$FILE\" 2>/dev/null || W=\"${W}- Missing top-level permissions block\\n\"; L=$(actionlint \"$FILE\" 2>&1 | head -5); if [ -n \"$W\" ] || [ -n \"$L\" ]; then MSG=\"MIGRATION QUALITY CHECK ($FILE):\\n${W}\"; [ -n \"$L\" ] && MSG=\"${MSG}actionlint errors:\\n${L}\\n\"; MSG=\"${MSG}Fix these issues now.\"; ESCAPED=$(printf '%s' \"$MSG\" | sed 's/\\\\/\\\\\\\\/g; s/\"/\\\\\"/g'); printf '{\"additionalContext\":\"%s\"}' \"$ESCAPED\"; fi"
}
],
"agentStop": [
{
"type": "command",
"description": "Migration quality gate — block completion if workflows have issues",
"timeoutSec": 60,
"bash": "INPUT=$(cat); CWD=$(echo \"$INPUT\" | jq -r '.cwd // \".\"' 2>/dev/null); COUNTER_FILE='/tmp/.migration-quality-gate'; COUNT=$(cat \"$COUNTER_FILE\" 2>/dev/null || echo 0); COUNT=$((COUNT + 1)); echo \"$COUNT\" > \"$COUNTER_FILE\"; if [ \"$COUNT\" -gt 3 ]; then rm -f \"$COUNTER_FILE\"; echo '{}'; exit 0; fi; if ! command -v actionlint >/dev/null 2>&1; then if [ \"$(uname)\" = 'Linux' ]; then V='1.7.11'; EXPECTED='900919a84f2229bac68ca9cd4103ea297abc35e9689ebb842c6e34a3d1b01b0a'; TGZ=\"/tmp/actionlint_${V}_linux_amd64.tar.gz\"; curl -fsSL -o \"$TGZ\" \"https://github.com/rhysd/actionlint/releases/download/v${V}/actionlint_${V}_linux_amd64.tar.gz\" 2>/dev/null; ACTUAL=$(sha256sum \"$TGZ\" 2>/dev/null | awk '{print $1}'); if [ \"$ACTUAL\" != \"$EXPECTED\" ]; then rm -f \"$TGZ\"; fi; tar xz -C /tmp -f \"$TGZ\" actionlint 2>/dev/null && install -m 755 /tmp/actionlint /usr/local/bin/actionlint 2>/dev/null; rm -f \"$TGZ\"; elif command -v brew >/dev/null 2>&1; then brew install actionlint 2>/dev/null; fi; fi; ISSUES=''; LINT_AVAIL=0; command -v actionlint >/dev/null 2>&1 && LINT_AVAIL=1; [ \"$LINT_AVAIL\" -eq 0 ] && ISSUES='actionlint-unavailable; '; for f in \"$CWD\"/.github/workflows/*.yml \"$CWD\"/.github/workflows/*.yaml; do [ -f \"$f\" ] || continue; FN=$(basename \"$f\"); W=''; grep -qE 'uses:\\s+[^@]+@v[0-9]' \"$f\" 2>/dev/null && W=\"${W}unpinned-actions \"; grep -qiE '(TODO|FIXME|CHANGEME|PLACEHOLDER|XXX)' \"$f\" 2>/dev/null && W=\"${W}placeholders \"; grep -qE 'permissions:\\s*write-all' \"$f\" 2>/dev/null && W=\"${W}write-all \"; grep -qE '^permissions:' \"$f\" 2>/dev/null || W=\"${W}no-permissions \"; [ \"$LINT_AVAIL\" -eq 1 ] && ! actionlint \"$f\" >/dev/null 2>&1 && W=\"${W}actionlint-errors \"; [ -n \"$W\" ] && ISSUES=\"${ISSUES}${FN}: ${W}; \"; done; if [ -n \"$ISSUES\" ]; then ESCAPED=$(printf '%s' \"$ISSUES\" | sed 's/\\\\/\\\\\\\\/g; s/\"/\\\\\"/g'); printf '{\"decision\":\"block\",\"reason\":\"Migration quality gate FAILED (attempt %d/3). Fix these workflow issues:\\n%s\"}' \"$COUNT\" \"$ESCAPED\"; else rm -f \"$COUNTER_FILE\"; echo '{}'; fi"
}
],
"sessionEnd": [
{
"type": "command",
"description": "Append migration scorecard entry",
"timeoutSec": 45,
"bash": "INPUT=$(cat); CWD=$(echo \"$INPUT\" | jq -r '.cwd // \".\"' 2>/dev/null); REASON=$(echo \"$INPUT\" | jq -r '.reason // \"unknown\"' 2>/dev/null); SESSION=$(echo \"$INPUT\" | jq -r '.sessionId // \"unknown\"' 2>/dev/null); if ! command -v actionlint >/dev/null 2>&1; then if [ \"$(uname)\" = 'Linux' ]; then V='1.7.11'; EXPECTED='900919a84f2229bac68ca9cd4103ea297abc35e9689ebb842c6e34a3d1b01b0a'; TGZ=\"/tmp/actionlint_${V}_linux_amd64.tar.gz\"; curl -fsSL -o \"$TGZ\" \"https://github.com/rhysd/actionlint/releases/download/v${V}/actionlint_${V}_linux_amd64.tar.gz\" 2>/dev/null; ACTUAL=$(sha256sum \"$TGZ\" 2>/dev/null | awk '{print $1}'); if [ \"$ACTUAL\" != \"$EXPECTED\" ]; then rm -f \"$TGZ\"; fi; tar xz -C /tmp -f \"$TGZ\" actionlint 2>/dev/null && install -m 755 /tmp/actionlint /usr/local/bin/actionlint 2>/dev/null; rm -f \"$TGZ\"; elif command -v brew >/dev/null 2>&1; then brew install actionlint 2>/dev/null; fi; fi; LINT_AVAIL=0; command -v actionlint >/dev/null 2>&1 && LINT_AVAIL=1; TOTAL=0; CLEAN=0; BAD=0; for f in \"$CWD\"/.github/workflows/*.yml \"$CWD\"/.github/workflows/*.yaml; do [ -f \"$f\" ] || continue; TOTAL=$((TOTAL + 1)); HAS=0; grep -qE 'uses:\\s+[^@]+@v[0-9]' \"$f\" 2>/dev/null && HAS=1; grep -qiE '(TODO|FIXME|CHANGEME|PLACEHOLDER|XXX)' \"$f\" 2>/dev/null && HAS=1; grep -qE 'permissions:\\s*write-all' \"$f\" 2>/dev/null && HAS=1; grep -qE '^permissions:' \"$f\" 2>/dev/null || HAS=1; [ \"$LINT_AVAIL\" -eq 1 ] && ! actionlint \"$f\" >/dev/null 2>&1 && HAS=1; [ \"$HAS\" -eq 0 ] && CLEAN=$((CLEAN + 1)) || BAD=$((BAD + 1)); done; rm -f /tmp/.migration-quality-gate; SC=\"$CWD/.github/MIGRATION-SCORECARD.md\"; [ -f \"$SC\" ] || printf '# Migration Scorecard\\n' > \"$SC\" 2>/dev/null; printf '\\n## %s\\n- Session: %s\\n- Reason: %s\\n- Workflows: %d total, %d clean, %d with issues\\n' \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\" \"$SESSION\" \"$REASON\" \"$TOTAL\" \"$CLEAN\" \"$BAD\" >> \"$SC\" 2>/dev/null; echo '{}'"
}
]
}
}