Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- `/tw-verify` Quality column + blocked footer (slice 28): Scan Status table is
**Name | Type | Status | Quality | Note**; Quality shows Tessl
`items.quality_score` as **`N/100`** (else `—`); shared
**Will be blocked when Tripwire is enabled** appears once under the table;
**Sources** line attributes Quality to Tessl and security Status to
Cisco AI Defense and Snyk. Shared helpers in `guard/verify.py`; contract SSOT
[frontline-output-contract.md](docs/user-guide/frontline-output-contract.md).
Re-run `tripwire setup-agent-hooks` to refresh installed skills.
- Docs: Wave **13-M** LLM usage / cost observability plan (slice 53) —
[design/llm-usage-tracking.md](docs/design/llm-usage-tracking.md), slice stub,
TRAIL/PROGRESS/STATUS DECIDED (not IMPLEMENTED). Historic collapsible Usage
Expand Down
4 changes: 3 additions & 1 deletion agent-hooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ from the repo directly — `tripwire setup-agent-hooks` installs copies:
stdout; every failure mode is an explicit deny (fail closed).
- **`hooks/_guard_entry.py`** — installed shim delegating to
`guard.entry.main()` (run via `uv run --project <repo_root> --extra guard`).
- **`skills/tw-verify`** — report scan status of named skills/MCP servers.
- **`skills/tw-verify`** — report scan status of named skills/MCP servers
(RAG + Tessl Quality as `N/100` when present; blocked message as table footer;
**Sources** line: Quality = Tessl, Status = Cisco AI Defense + Snyk).
- **`skills/tw-scan`** — submit named skills/MCP servers for scanning
(`--force` or bare `force` to rescan valid artifacts).
- **`skills/tw-enable` / `skills/tw-disable`** — flip the local `enable` key in
Expand Down
2 changes: 1 addition & 1 deletion agent-hooks/skills/tw-disable/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,5 +54,5 @@ If `created` is true, the config file was missing and was recreated with default
Tell the user, plainly:

- Tripwire enforcement is now **OFF** — the Claude Code hook allows every skill/MCP call without checking scan status. Enforcement is **fully bypassed**, including red-rated and never-scanned artifacts.
- Manual scanning and reporting still work while disabled: `/tw-scan` submits scans and `/tw-verify` reports status (its "will be blocked" column shows what enforcement WOULD do, marked as currently bypassed).
- Manual scanning and reporting still work while disabled: `/tw-scan` submits scans and `/tw-verify` reports status (its blocked **footer** shows what enforcement WOULD do, marked as currently bypassed).
- Re-enable at any time with `/tw-enable`.
20 changes: 13 additions & 7 deletions agent-hooks/skills/tw-self-check/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,19 @@ for ident in sys.argv[1:]:
blocked = not (rag == "green" or (rag == "amber" and threshold == "red"))
else:
blocked = True
item = s.get("item") or {}
q = item.get("quality_score")
quality_score = float(q) if isinstance(q, (int, float)) else None
rows.append({
"identifier": ident,
"state": state,
"rag": rag,
"scanned_at": s.get("scanned_at"),
"stale": state == "stale",
"errored": (s.get("item") or {}).get("heatmap_status") == "error",
"errored": item.get("heatmap_status") == "error",
"changed": changed,
"will_be_blocked": blocked,
"quality_score": quality_score,
})
print(json.dumps({
"config": {
Expand All @@ -82,16 +86,18 @@ print(json.dumps({

## Step 4 — Report (Scan Status table only)

Render exactly the tw-verify human table — same states, emoji, and labels (see `~/.claude/skills/tw-verify/SKILL.md` §Step 5). Parse the driver's JSON privately; do **not** dump a fenced `{config, artifacts}` block to the user. Summary of the row rules (N = `scan_validity_days`):
Render exactly the tw-verify human table — same columns **Name | Type | Status | Quality | Note**, emoji/labels, Quality as **`N/100`** or `—`, blocked-message **footer**, and **Sources** line (Tessl Quality; Cisco/Snyk Status) (see `~/.claude/skills/tw-verify/SKILL.md` §Step 5 and [frontline-output-contract.md](../../../docs/user-guide/frontline-output-contract.md)). Parse the driver's JSON privately; do **not** dump a fenced `{config, artifacts}` block to the user. Summary of the row rules (N = `scan_validity_days`):

- `fresh` + green → `🟢 GREEN (fresh)`, note `—`.
- `fresh` + amber → `🟠 AMBER`; `Reported but not blocked at current threshold` when threshold is `red`, else bold **Will be blocked when Tripwire is enabled**.
- `fresh` + red → `🔴 RED`; note ALWAYS bold **Will be blocked when Tripwire is enabled** — never omitted.
- `fresh` + amber → `🟠 AMBER`; `Reported but not blocked at current threshold` when threshold is `red`, else distinct amber note + footer when blocked.
- `fresh` + red → `🔴 RED`; note `rated red — at/above threshold` (blocked sentence is footer-only).
- `stale` → `⚠️ STALE`; `Last scanned >N days ago — blocked until rescanned (run /tw-scan <name>)`.
- `scanning` → `⏳ SCANNING`; `Scan in progress — check back shortly` (+ prior verdict if rag non-null).
- `unscanned` → `🚫 UNSCANNED`; `Never scanned — blocked when Tripwire is enabled` (or `Last scan errored — resubmit (run /tw-scan <name>)` when errored).
- `changed=true` → `✏️ CHANGED`; bold **content changed since last scan — run /tw-scan <name>** — takes precedence over every state-based row (the hook's tamper deny fires regardless of a green verdict), `will_be_blocked` true.
- missing install dir → `❓ NOT FOUND`; bold **Will be blocked when Tripwire is enabled** — `Not installed under ~/.claude/skills — run tripwire setup-agent-hooks`.
- `unscanned` → `🚫 UNSCANNED`; `Never scanned — offer /tw-scan <name>` (or `Last scan errored — resubmit (run /tw-scan <name>)` when errored).
- `changed=true` → `✏️ CHANGED`; bold **content changed since last scan — run /tw-scan <name>** — takes precedence over every state-based row, `will_be_blocked` true.
- missing install dir → `❓ NOT FOUND`; note `Not installed under ~/.claude/skills — run tripwire setup-agent-hooks` (footer covers blocked).

If any `will_be_blocked`, print **Will be blocked when Tripwire is enabled** once under the table.

If local `enable` is false, add the "currently bypassed" note; if `monitoring_enabled` is false, add the platform-switch warning.

Expand Down
56 changes: 35 additions & 21 deletions agent-hooks/skills/tw-verify/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Check the Tripwire scan status of Claude Code skills and MCP server

# tw-verify

Report the Tripwire scan status of one or more skills / MCP servers. Single pass over ALL requested names — never stop at the first problem; every requested name gets a row in the Scan Status table. Read-only: this skill never submits scans itself (it only offers to, at the end). Do **not** dump the driver's raw JSON (or any fenced `{config, artifacts}` block) to the user — that payload is for you to parse only.
Report the Tripwire scan status of one or more skills / MCP servers. Single pass over ALL requested names — never stop at the first problem; every requested name gets a row in the Scan Status table. Columns follow the Frontline dual-output contract ([frontline-output-contract.md](../../../docs/user-guide/frontline-output-contract.md)): **Name | Type | Status | Quality | Note**, with Tessl Quality as **`N/100`** when `items.quality_score` is present (else `—`), the shared blocked phrase once as a **table footer**, and a **Sources** line (Quality = Tessl; Status = Cisco AI Defense + Snyk). Shared helpers: `guard.verify.verify_artifacts` / `format_quality_cell`. Read-only: this skill never submits scans itself (it only offers to, at the end). Do **not** dump the driver's raw JSON (or any fenced `{config, artifacts}` block) to the user — that payload is for you to parse only.

**Hard rule — unscanned/blocked artifacts must not be executed.** If a row is `unscanned` (including errored), `stale`, `changed`, RED (or amber at threshold), or NOT FOUND: do **not** invoke that skill (`Skill` tool), do **not** call its `mcp__*` tools, and do **not** run its `install.sh` / scripts via Bash. Report that Tripwire will block those calls. Only offer `/tw-scan` via AskUserQuestion — never silently submit, and never “work around” a block by scanning so you can run the artifact in the same turn.

Expand Down Expand Up @@ -39,7 +39,7 @@ print(json.dumps({"resolutions": out}))

Parse the JSON privately. For each entry:

- `found=false` → ❓ NOT FOUND row (do not call the status driver for that name). Note: **Will be blocked when Tripwire is enabled** — no match in hook loci (`~/.claude/skills`, `.claude/skills`, `.mcp.json`, `~/.claude.json` incl. `projects["<cwd>"]`, `~/.tripwire/demo-mcp.json`, fixtures MCP manifest). Tip: demo skills are `safe-skill` / `vuln-skill` / `amber-skill`; demo MCP keys are `safe-tool` / `vuln-tool` / `amber-tool` (fixture names like `vuln-runtime-download` alias to those when demos are installed).
- `found=false` → ❓ NOT FOUND row (do not call the status driver for that name). Note: no match in hook loci (`~/.claude/skills`, `.claude/skills`, `.mcp.json`, `~/.claude.json` incl. `projects["<cwd>"]`, `~/.tripwire/demo-mcp.json`, fixtures MCP manifest) — do **not** repeat the blocked phrase in the Note (it goes in the table footer). Tip: demo skills are `safe-skill` / `vuln-skill` / `amber-skill`; demo MCP keys are `safe-tool` / `vuln-tool` / `amber-tool` (fixture names like `vuln-runtime-download` alias to those when demos are installed).
- `found=true` → use `identifier` (and `kind`) for Step 4. If `alias_of` is set, the Name column still shows the user’s requested name; optionally append `(as <resolved_as>)` in the Note.

An absolute path the user passed that exists as a skill directory is resolved by the driver — do not invent your own path logic.
Expand Down Expand Up @@ -79,15 +79,19 @@ for ident in sys.argv[1:]:
blocked = not (rag == "green" or (rag == "amber" and threshold == "red"))
else:
blocked = True
item = s.get("item") or {}
q = item.get("quality_score")
quality_score = float(q) if isinstance(q, (int, float)) else None
rows.append({
"identifier": ident,
"state": state,
"rag": rag,
"scanned_at": s.get("scanned_at"),
"stale": state == "stale",
"errored": (s.get("item") or {}).get("heatmap_status") == "error",
"errored": item.get("heatmap_status") == "error",
"changed": changed,
"will_be_blocked": blocked,
"quality_score": quality_score,
})
print(json.dumps({
"config": {
Expand All @@ -101,40 +105,50 @@ print(json.dumps({
' <identifier-1> <identifier-2> ...
```

The driver prints one JSON object `{config, artifacts}` — parse it privately; never paste it into the user-facing reply. NOT FOUND names never reach the driver (they get rows anyway). If the driver itself fails, report the failure and still render every row (status "unknown — status query failed"), never a partial silent result.
The driver prints one JSON object `{config, artifacts}` — parse it privately; never paste it into the user-facing reply. Each artifact includes nullable `quality_score` (0–100 from `items.quality_score`); human render owns the `/100` display. NOT FOUND names never reach the driver (they get rows anyway, `quality_score` null). If the driver itself fails, report the failure and still render every row (status "unknown — status query failed"), never a partial silent result.

## Step 5 — Render the human-readable table

One Markdown table, one row per REQUESTED name (selection rows count individually), all in a single pass. Use exactly these states, emoji, and labels (N = `scan_validity_days`):
One Markdown table, one row per REQUESTED name (selection rows count individually), all in a single pass. Fixed columns: **Name | Type | Status | Quality | Note**. Use exactly these states, emoji, and labels (N = `scan_validity_days`). Quality cell: **`N/100`** when `quality_score` is a number for a skill (Tessl skill-review, 0–100, higher better); otherwise `—` (MCP, unscanned, scanning, not-found, null, or missing). Never a bare integer or `Q N` alone.

| Name | Type | Status | Note |
|------|------|--------|------|
| `safe-skill` | skill | 🟢 GREEN (fresh) | — |
| `vuln-skill` | skill | 🔴 RED | **Will be blocked when Tripwire is enabled** |
| `safe-tool` | mcp | 🟠 AMBER | Reported but not blocked at current threshold |
| `unknown-skill` | — | ❓ NOT FOUND | **Will be blocked when Tripwire is enabled** — no match in ~/.claude/skills, .claude/skills, .mcp.json, ~/.claude.json, ~/.tripwire/demo-mcp.json, fixtures manifest |
| `old-skill` | skill | ⚠️ STALE | Last scanned >14 days ago — blocked until rescanned |
| `pending-skill` | skill | ⏳ SCANNING | Scan in progress — check back shortly |
| `new-skill` | skill | 🚫 UNSCANNED | Never scanned — blocked when Tripwire is enabled |
| `edited-skill` | skill | ✏️ CHANGED | **content changed since last scan — run /tw-scan <name>** |
| Name | Type | Status | Quality | Note |
|------|------|--------|---------|------|
| `safe-skill` | skill | 🟢 GREEN (fresh) | 91/100 | — |
| `vuln-skill` | skill | 🔴 RED | 12/100 | rated red — at/above threshold |
| `safe-tool` | mcp | 🟠 AMBER | — | Reported but not blocked at current threshold |
| `unknown-skill` | — | ❓ NOT FOUND | — | no match in ~/.claude/skills, .claude/skills, .mcp.json, ~/.claude.json, ~/.tripwire/demo-mcp.json, fixtures manifest |
| `old-skill` | skill | ⚠️ STALE | 80/100 | Last scanned >14 days ago — blocked until rescanned (run /tw-scan old-skill) |
| `pending-skill` | skill | ⏳ SCANNING | — | Scan in progress — check back shortly |
| `new-skill` | skill | 🚫 UNSCANNED | — | Never scanned — offer /tw-scan new-skill |
| `edited-skill` | skill | ✏️ CHANGED | 70/100 | **content changed since last scan — run /tw-scan edited-skill** |

**Blocked footer (de-dupe):** If **any** row has `will_be_blocked=true` (including NOT FOUND / RED / STALE / UNSCANNED / CHANGED / amber-at-threshold), print **once** under the table:

**Will be blocked when Tripwire is enabled**

Do **not** repeat that phrase in every Note. Row Notes keep *distinct* copy only.

**Sources attribution:** Always print **once** under the table (after the blocked footer when present):

*Sources: Quality is from Tessl (Review Quality). Security-related Status (GREEN/AMBER/RED) is from Cisco AI Defense and Snyk.*

Row rules (map driver output → row):

- `changed=true` → `✏️ CHANGED` — this row takes precedence over every state-based row (the hook's tamper deny fires regardless of a green verdict); note is the bold **content changed since last scan — run /tw-scan <name>**, and `will_be_blocked` is `true`.
- `state=fresh, rag=green` → `🟢 GREEN (fresh)`, note `—`.
- `state=fresh, rag=amber` → `🟠 AMBER`; note `Reported but not blocked at current threshold` when `threshold` is `red`, else **Will be blocked when Tripwire is enabled** (bold).
- `state=fresh, rag=red` → `🔴 RED`; note is ALWAYS the bold **Will be blocked when Tripwire is enabled** — never omitted, no exceptions.
- `state=fresh, rag=amber` → `🟠 AMBER`; note `Reported but not blocked at current threshold` when `threshold` is `red`, else note `amber at/above threshold` and `will_be_blocked=true` (footer covers the blocked sentence).
- `state=fresh, rag=red` → `🔴 RED`; note `rated red — at/above threshold` (blocked sentence is footer-only).
- `state=stale` → `⚠️ STALE`; note `Last scanned >N days ago — blocked until rescanned (run /tw-scan <name>)`.
- `state=scanning` → `⏳ SCANNING`; note `Scan in progress — check back shortly`; if `rag` is non-null append `(prior verdict: <rag>)`.
- `state=unscanned, errored=false` → `🚫 UNSCANNED`; note `Never scanned — blocked when Tripwire is enabled`.
- `state=unscanned, errored=false` → `🚫 UNSCANNED`; note `Never scanned — offer /tw-scan <name>`.
- `state=unscanned, errored=true` → `🚫 UNSCANNED`; note `Last scan errored — resubmit (run /tw-scan <name>)`.
- unresolved name → `❓ NOT FOUND`; note ALWAYS the bold **Will be blocked when Tripwire is enabled** plus `— no match in ~/.claude/skills, .claude/skills, .mcp.json, ~/.claude.json, ~/.tripwire/demo-mcp.json, fixtures manifest`. Fail-closed: the hook denies any Skill/mcp__* call it cannot resolve to a known locus (same as unscanned).
- unresolved name → `❓ NOT FOUND`; note `no match in ~/.claude/skills, .claude/skills, .mcp.json, ~/.claude.json, ~/.tripwire/demo-mcp.json, fixtures manifest`. Fail-closed: the hook denies any Skill/mcp__* call it cannot resolve to a known locus (same as unscanned); `will_be_blocked=true` → footer.

The `run /tw-scan <name>` remedy in the STALE, errored, and CHANGED notes actually works because tw-scan always submits with `--force` — without force the CLI would skip unchanged content and a stale/errored state could never clear.

After the table:
After the table (and after the blocked footer when present):

- If config `enabled` is `false`, add: `Note: Tripwire enforcement is currently DISABLED (/tw-disable) — "will be blocked" reports what enforcement would do when enabled; calls are currently bypassed.`
- If config `enabled` is `false`, add: `Note: Tripwire enforcement is currently DISABLED (/tw-disable) — the blocked footer reports what enforcement would do when enabled; calls are currently bypassed.`
- If `monitoring_enabled` is `false` while local `enable` is `true`, add a warning that the Supabase platform switch (`config.monitoring_enabled`) is OFF and still gates the guard — effective enforcement is local enable AND platform switch.

## Step 6 — Offer scans for blocked-but-fixable rows
Expand Down
Loading
Loading