Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Changed
- Dashboard detail drawer **Findings** section is collapsible: clickable heading with
a rotating ▸ chevron (expanded by default on item select). Same expand pattern as
Scanner outputs rows — `prototypes/dc-dashboard/Tripwire.dc.html` (IMPLEMENTED).
- Public docs UX + compaction (slice 44): README/QUICKSTART lead with plain language,
demo-first (Recommended) then Live (Advanced); badges behind disclosure; path map
absorbed into QUICKSTART. Merged `sie-setup` + `model-studio-setup` into
Expand Down Expand Up @@ -36,6 +39,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
avoids delayed native `title=` attributes.

### Added
- Tessl Eval auto-chain scanner row (slice 50): `run_tessl()` emits
`"Tessl: Eval"` as `blocked`, then auto-chains after Scenario Generation when
`<plugin>/evals/` has scenarios — `tessl eval run --runs 3 -y --json` +
`eval view` poll, `upstream_run_ids`, project create/repair preflight; scenario
re-run marks prior completed Eval `stale` (no cascade).
- Tessl Scenario Generation scanner row (slice 49): after Review (Quality),
`run_tessl()` runs plugin-path `tessl scenario generate --count 3`, downloads
into `<plugin>/evals/`, stamps `tessl_run_id` / `upstream_run_ids`, and persists
Expand Down
4 changes: 2 additions & 2 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ services — see [prerequisites](./user-guide/prerequisites.md).
| **Supabase** | Postgres + Realtime system of record | MVP Live | [supabase-setup](./user-guide/supabase-setup.md) → [env-vars](./user-guide/env-vars.md) |
| **Modal** | Isolated scanner sandbox compute | MVP Live | [modal-setup](./user-guide/modal-setup.md) → [env-vars](./user-guide/env-vars.md) |
| **Snyk** | Skill/MCP depth scanner | Full scanner coverage | [procurement](./user-guide/env-vars.md#vendor-procurement-quick-steps) |
| **Tessl** | Skill lint (auth-free) + review run quality (`TESSL_TOKEN` + `TESSL_WORKSPACE`) + Scenario Generation (`TESSL_TOKEN` + `.tessl-plugin/plugin.json`; IMPLEMENTED unit, slice 49) | Full scanner coverage | [procurement](./user-guide/env-vars.md#vendor-procurement-quick-steps) |
| **Tessl** | Skill lint (auth-free) + review / scenario / eval (`TESSL_TOKEN`; `--workspace` from optional `TESSL_WORKSPACE` or `whoami`+`workspace list`) + Scenario Generation (`.tessl-plugin/plugin.json`; IMPLEMENTED unit, slice 49) + Eval auto-chain (`tessl.json` project link; IMPLEMENTED unit, slice 50) | Full scanner coverage | [procurement](./user-guide/env-vars.md#vendor-procurement-quick-steps) |
| **Cisco AI Defense** | Skill Scanner / MCP Scanner / AI Defense APIs | Full scanner coverage | [procurement](./user-guide/env-vars.md#vendor-procurement-quick-steps) |
| **Superlinked SIE** | Cheap post-scan triage | Optional tiered router | [tiered-router-setup](./user-guide/tiered-router-setup.md) |
| **Alibaba Cloud Model Studio** | Escalation arbitration / triage | Optional tiered router | [tiered-router-setup](./user-guide/tiered-router-setup.md) |
Expand Down Expand Up @@ -289,7 +289,7 @@ separate `scan_run_scanners` row (slice 46 ✅ persist scan_run `a36cad9f`):
[#109](https://github.com/neomatrix369/tripwire/pull/109))
uses `tessl review run quality --json --workspace` and stamps `tessl_run_id`
from `tessl review view --last --json`, then seeds in-process
`_TesslIdContext["review_quality"]` for slices 49–51 (GWT-47.5). It is orthogonal to findings and to `risk_score`. **IMPLEMENTED (UI):** slice 48 synthesises "Not Available Yet" sentinel rows for Scenario Generation, Eval, and Security Review when those sources are absent from the scan_run (never stored as placeholders). **IMPLEMENTED (unit, slice 49):** Scenario Generation writes a real `scan_run_scanners` row (`scenario generate` → `download` into `<plugin>/evals/`, `resume_checkpoint`, mid-scan persist). **DECIDED (runner not implemented):** slices 50–51 write Eval + Security rows — see
`_TesslIdContext["review_quality"]` for slices 49–51 (GWT-47.5). It is orthogonal to findings and to `risk_score`. **IMPLEMENTED (UI):** slice 48 synthesises "Not Available Yet" sentinel rows for Scenario Generation, Eval, and Security Review when those sources are absent from the scan_run (never stored as placeholders). **IMPLEMENTED (unit, slice 49):** Scenario Generation writes a real `scan_run_scanners` row (`scenario generate` → `download` into `<plugin>/evals/`, `resume_checkpoint`, mid-scan persist). **IMPLEMENTED (unit, slice 50):** Eval starts `blocked`, auto-chains after Scenario Gen when `evals/` is populated (`eval run --runs 3 -y` + `eval view`; stale on scenario re-run; project create/repair preflight). **DECIDED (runner not implemented):** slice 51 writes Security Review rows — see
[design/tessl-5-row-expansion.md](./design/tessl-5-row-expansion.md) and slices
49–51; scenario→eval pipeline is generate → download → `eval run` on disk
`evals/` (sandbox-populated; host `evals/` is not a vuln-scan input — packing
Expand Down
39 changes: 24 additions & 15 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ Reachable through production entry points / config:
- `_acquire_target` dispatch (git clone, local copy, host→sandbox tar upload via
`local_entrypoint`, MCP introspection-only empty workdir) — `sandbox/`
- Dashboard Live/Mock with Supabase Realtime (~1s) + 8s poll fallback, SCANNING
in-flight UI, scanner console in drawer, partial-failed “n out of m scanners
in-flight UI, scanner console in drawer, collapsible Findings heading (rotating
chevron; expanded by default), partial-failed “n out of m scanners
unreachable” copy — `prototypes/dc-dashboard/`;
`scripts/serve-dashboard.mjs` / `scripts/sync-dashboard-config.sh`
- Dashboard visual identity v2 (FolderGate cream/tan × Tripwire HUD): paper
Expand Down Expand Up @@ -94,10 +95,12 @@ Reachable through production entry points / config:
— `sandbox/scanners.py` (slice 47 ✅,
[PR #109](https://github.com/neomatrix369/tripwire/pull/109); GWT-47.1–47.5)
- Dashboard Tessl "Not Available Yet" placeholders — Scanner Outputs always shows
five Tessl capability rows when any Tessl DB row exists; missing
Eval / Review (Security) (and Scenario Generation until a DB row exists) are
UI-only sentinels (`status: not_available_yet`), never written as placeholders,
and counted in the header. MCP scans are unchanged. IMPLEMENTED (unit) +
five Tessl capability rows when any Tessl DB row exists; missing sources among
Scenario Generation / Eval / Review (Security) are UI-only sentinels
(`status: not_available_yet`) when absent from the scan_run (never stored as
placeholders) and counted in the header. Runners 49–50 write Scenario Gen +
Eval rows when they run; Security remains sentinel until slice 51. MCP scans
unchanged. IMPLEMENTED (unit) +
VERIFIED (Mock UI 2026-08-24: `safe-changelog-writer` Scanner Outputs (7),
five Tessl rows, three NAY pills, no chevron; MCP `SCANNER OUTPUTS (3)`
unpadded) — `tripwire-status.js` `mergeTesslCapabilityRows`, `Tripwire.dc.html`
Expand All @@ -107,7 +110,13 @@ Reachable through production entry points / config:
`<plugin>/evals/`, `upstream_run_ids.review_quality` from ctx, `tessl_run_id`
stamp, `resume_checkpoint` + mid-scan persist via `on_scanner_progress`.
Missing token → `needs_setup`; missing `.tessl-plugin/plugin.json` → `failed`.
IMPLEMENTED (unit) — `sandbox/scanners.py` / `sandbox/scan_app.py` (slice 49 🔄)
IMPLEMENTED (unit) — `sandbox/scanners.py` / `sandbox/scan_app.py` (slice 49 ✅ #112)
- Tessl Eval auto-chain — `run_tessl()` emits `"Tessl: Eval"` as `blocked` before
Scenario Generation, then auto-chains to `queued`→`running` when generation
completes and `<plugin>/evals/` has scenarios; `tessl eval run --runs 3 -y
--json` + `eval view` poll; `upstream_run_ids` from ctx; project create/repair
preflight; scenario re-run marks prior completed Eval `stale` (no cascade).
IMPLEMENTED (unit) — `sandbox/scanners.py` / `sandbox/scan_app.py` (slice 50 🔨)
- Live dashboard latest-state read path — `dashboard_latest_runs` view (one row per
item) + batched child-table fetches in `tripwire-live.js`; replaces global
`scan_runs?limit=2000` page that could miss per-item newest runs and PostgREST
Expand Down Expand Up @@ -235,17 +244,17 @@ Not IMPLEMENTED — no production hook install path or `/tw-*` skills yet. ADR-0
Horizon A exclusion remains in force until Wave H lands and a superseding ADR
records the new production entry.

**Wave L — Tessl scenario generation + eval (2026-08-24):** Row 3
(`"Tessl: Scenario Generation"`) is **IMPLEMENTED (unit, slice 49)** —
**Wave L — Tessl scenario generation + eval (2026-08-25):** Row 3
(`"Tessl: Scenario Generation"`) is **IMPLEMENTED (unit, slice 49 ✅ #112)** —
`scenario generate <plugin-path> --count 3` → `scenario download <gen_id> -o
<plugin>/evals/` with `resume_checkpoint` + mid-scan persist. Row 4
(`"Tessl: Eval"`) remains **DECIDED** in slice 50 — not IMPLEMENTED. Eval
auto-chains from `blocked` when generation completes and `evals/` is
populated. Coverage Gap B (`scenario view <id>`) resolved; Gap C (agent-assisted
generation) open. **IMPLEMENTED (slice 48):** host `evals/` is not a vuln-scan
input — `_pack_local_dir` / `_copy_local` omit root `evals/` when the skill
root has `tessl.json` or `.tessl-plugin/`. Git clone and identity hash still
see on-disk `evals/`. Spec:
(`"Tessl: Eval"`) is **IMPLEMENTED (unit, slice 50)**starts `blocked`,
auto-chains when generation completes and `evals/` is populated; stale on
scenario re-run; resume via `eval view`. Coverage Gap B (`scenario view <id>`)
resolved; Gap C (agent-assisted generation) open. **IMPLEMENTED (slice 48):**
host `evals/` is not a vuln-scan input — `_pack_local_dir` / `_copy_local` omit
root `evals/` when the skill root has `tessl.json` or `.tessl-plugin/`. Git
clone and identity hash still see on-disk `evals/`. Spec:
[design/tessl-5-row-expansion.md](./design/tessl-5-row-expansion.md),
[slices 49–50](./plan/slices/12-L-tessl-5-row-expansion/).

Expand Down
14 changes: 7 additions & 7 deletions docs/design/tessl-5-row-expansion.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Design: Tessl 5-Row Expansion

**Status**: Schema IMPLEMENTED (slice 45 ✅). Lint adapter IMPLEMENTED (slice 46 ✅ #105). Review Quality run-ID + `_TesslIdContext` seed IMPLEMENTED unit (slice 47 ✅ #109). Rows 3–5 UI sentinels IMPLEMENTED (slice 48). Scenario Generation runner IMPLEMENTED unit (slice 49). Eval + Security runners remain DECIDED / not implemented.
**Status**: Schema IMPLEMENTED (slice 45 ✅). Lint adapter IMPLEMENTED (slice 46 ✅ #105). Review Quality run-ID + `_TesslIdContext` seed IMPLEMENTED unit (slice 47 ✅ #109). Rows 3–5 UI sentinels IMPLEMENTED (slice 48). Scenario Generation runner IMPLEMENTED unit (slice 49 ✅ #112). Eval auto-chain IMPLEMENTED unit (slice 50). Security runner remains DECIDED / not implemented.
**Date**: 2026-08-24
**Scope**: Design contract for replacing the single Tessl scanner row with 5 flat capability rows. Current-truth notes below mark what has shipped; remaining rows stay future-state.

Expand Down Expand Up @@ -255,7 +255,7 @@ Verified against [Tessl CLI reference](https://docs.tessl.io/reference/cli-comma
7. Security (slice 51) → upstream_run_ids={review_quality}; review run security; stamp tessl_run_id
```

**Not supported by Tessl CLI**: passing `gen_id` to `eval run`. Eval always consumes on-disk scenarios. **`--workspace`** on `scenario generate` is for repo mode (`org/repo --commits …`), not plugin-path generation.
**Not supported by Tessl CLI**: passing `gen_id` to `eval run`. Eval always consumes on-disk scenarios. **`--workspace`** is **required** outside interactive mode for plugin-path `scenario generate` (live CLI). Tripwire resolves it via optional `TESSL_WORKSPACE` or `tessl whoami` + `tessl workspace list` (personal workspace is usually the username).

### ID carry-forward contract (MUST — slices 47–51)

Expand Down Expand Up @@ -328,7 +328,7 @@ Each feature that reads from a prior feature's persisted state does so by:

**What is read**: Same Quality Review `tessl_run_id` lookup; `tessl review view <id> --json` to retrieve Quality findings.

**Threading findings into scenario generation**: The **plain CLI form** (`tessl scenario generate <plugin-path> [--count N]`) has no context-injection flag. `--workspace` applies to **repo** generation (`org/repo --commits …`), not plugin-path generation. To thread Quality findings into scenario generation, the **agent-assisted path** (`tessl install tessl-labs/tessl-skill-eval-scenarios`) is the only documented channel.
**Threading findings into scenario generation**: The **plain CLI form** (`tessl scenario generate <plugin-path> --workspace <ws> [--count N]`) has no context-injection flag for Quality findings. To thread Quality findings into scenario generation, the **agent-assisted path** (`tessl install tessl-labs/tessl-skill-eval-scenarios`) is the only documented channel.

**Caveat — agent-assisted path in headless sandbox**: This path is designed around an interactive agent prompt. Whether it can be scripted from Tripwire's headless Modal sandbox orchestration is **unverified** (Coverage Gap C). Until verified, the plain CLI form is used for scenario generation, and the Quality findings are surfaced in the UI as context for human review of the generated scenarios rather than injected into the CLI call.

Expand Down Expand Up @@ -358,7 +358,7 @@ The single existing `"Tessl"` row is replaced by 5 flat sibling rows, in this ex
| 1 | `Tessl: Lint` | live status (`completed` / `failed` / `unreachable`) | **IMPLEMENTED** (slice 46) — new row; auth-free `tessl skill lint` |
| 2 | `Tessl: Review (Quality)` | live status (`completed` / `needs_setup` / …) | **IMPLEMENTED** source string (slice 46); `tessl_run_id` + `_TesslIdContext["review_quality"]` **IMPLEMENTED unit** (slice 47 ✅ #109) via `review view --last --json` |
| 3 | `Tessl: Scenario Generation` | live status (`completed` / `failed` / `needs_setup` / `interrupted` / …) | **IMPLEMENTED unit (slice 49)** — `scenario generate` → download into `<plugin>/evals/`; `resume_checkpoint`; DB row replaces NAY sentinel |
| 4 | `Tessl: Eval` | `Not Available Yet` | **IMPLEMENTED (UI sentinel, slice 48)** — not written to DB |
| 4 | `Tessl: Eval` | live status (`blocked` / `queued` / `running` / `completed` / `stale` / …) | **IMPLEMENTED unit (slice 50)** — auto-chain after Scenario Gen + `evals/`; `upstream_run_ids`; project create/repair preflight; DB row replaces NAY sentinel |
| 5 | `Tessl: Review (Security)` | `Not Available Yet` | **IMPLEMENTED (UI sentinel, slice 48)** — not written to DB |

The 5 rows appear as a contiguous block where the single `"Tessl"` row used to be.
Expand All @@ -374,9 +374,9 @@ The dashboard derives the count from the **static constant list** of all 5 expec

> **DECIDED (slice 48):** include "Not Available Yet" rows in the Scanner Outputs count (consistent with Cisco credential-absent rows).

### "Not Available Yet" Rendering Rules (Eval + Security until slices 50–51)
### "Not Available Yet" Rendering Rules (Security until slice 51; Scenario Gen / Eval when absent)

The dashboard holds a **static ordered list** of all 5 Tessl `scanner_source` values. For each value absent from the DB rows for the current `scan_run_id`, the `scannersView` map emits a sentinel object with `status: 'not_available_yet'`.
The dashboard holds a **static ordered list** of all 5 Tessl `scanner_source` values. For each value absent from the DB rows for the current `scan_run_id`, the `scannersView` map emits a sentinel object with `status: 'not_available_yet'`. After slices 49–50, Scenario Generation and Eval normally write real rows (including `blocked` Eval before auto-chain); Security Review remains NAY until slice 51.

Rendering:
- Left accent bar: neutral/muted colour (not the status colours used for active rows).
Expand Down Expand Up @@ -411,7 +411,7 @@ Extends `scannerStatusColor` and `scannerStatusLabel` in the dashboard JS:

### `tesslQuality` Binding Scope Fix

The existing `tesslQuality` logic is implemented in `tesslInnerQuality` (`tripwire-status.js`) and is scoped to `scanner_source === "Tessl: Review (Quality)"`. Live attaches `output.quality_score` only for that source (`tripwire-live.js`). The quality score badge does not appear on Lint (slice 46 VERIFIED(unit)). Scenario Generation is written by the runner (slice 49); Eval and Security Review rows remain UI sentinels until slices 50–51 write real DB rows (slice 48 VERIFIED(unit)).
The existing `tesslQuality` logic is implemented in `tesslInnerQuality` (`tripwire-status.js`) and is scoped to `scanner_source === "Tessl: Review (Quality)"`. Live attaches `output.quality_score` only for that source (`tripwire-live.js`). The quality score badge does not appear on Lint (slice 46 VERIFIED(unit)). Scenario Generation (slice 49) and Eval (slice 50) are written by the runner; Security Review remains a UI sentinel until slice 51 writes a real DB row (slice 48 VERIFIED(unit) for the merge/sentinel path).

---

Expand Down
Loading
Loading