Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
255023e
Document ecosystem intelligence sources and signals
justsml Aug 26, 2026
717edc3
Document safe device research method
justsml Aug 26, 2026
feb2813
Document native harness comparison contract
justsml Aug 26, 2026
310ada6
Add portable target recipe admission contract
justsml Aug 26, 2026
3997b3d
Add source-backed opportunity ranking foundation
justsml Aug 26, 2026
e18a65d
Gate findings on demonstrated security impact
justsml Aug 26, 2026
00a4321
Keep graph exports focused
justsml Aug 26, 2026
5755aa5
Add native harness study admission contract
justsml Aug 26, 2026
455defb
Add evidence-gated device research lanes
justsml Aug 26, 2026
7fa8579
Add campaign autonomy decision policy
justsml Aug 26, 2026
d92a614
Add evidence-backed product skill promotion gate
justsml Aug 26, 2026
7e9f6cc
Add research campaign admission contract
justsml Aug 26, 2026
2520f24
Export campaign policy contracts
justsml Aug 26, 2026
8a61b67
Fingerprint admitted campaign definitions
justsml Aug 26, 2026
50e9fb3
Document durable passive-launch council portfolio
justsml Aug 27, 2026
1b68b11
Normalize passive discovery evidence
justsml Aug 27, 2026
770aee6
Polish passive discovery normalizer
justsml Aug 27, 2026
171c555
Use Workspace for product skill discovery
justsml Aug 27, 2026
7e5fac8
Pin research execution profiles
justsml Aug 27, 2026
e0f31b4
Stabilize execution profile regression
justsml Aug 27, 2026
a2a4955
Preserve approval and authorization history
justsml Aug 27, 2026
5017d57
Persist passive auth surface summaries
justsml Aug 27, 2026
e0a65c7
Resolve immutable containment policies
justsml Aug 27, 2026
281edf0
Expose canonical target inventory
justsml Aug 27, 2026
52ed79a
Pin validation authority provenance
justsml Aug 27, 2026
8d2a7d9
Document next Wayfinder closure wave
justsml Aug 27, 2026
291da5d
Close immutable decision custody gaps
justsml Aug 27, 2026
bb8f185
Add deterministic forensic policy scoring
justsml Aug 27, 2026
2456fb7
Enforce unified containment policy on security actions
justsml Aug 27, 2026
2f055e9
Add canonical zero-cost eval smoke
justsml Aug 27, 2026
f7d755f
Add accessible dialogs and bounded folder uploads
justsml Aug 27, 2026
827759d
Unify runtime product skill discovery
justsml Aug 27, 2026
69d8378
Add stored-evidence passive auth tracer
justsml Aug 27, 2026
d4d10a8
Propagate pinned profiles to background research
justsml Aug 27, 2026
2901370
Persist lab network enforcement evidence
justsml Aug 27, 2026
49c6ec6
Require microVMs for high-risk lab workloads
justsml Aug 27, 2026
95180da
Satisfy lab evidence formatting checks
justsml Aug 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ _Avoid_: candidate, lead, alert
The global, versioned catalog of reusable security concepts and source-backed relationships used for retrieval, classification, and strategy. It never owns project observations, evidence, assertions, or findings.
_Avoid_: Investigation Graph, project graph, fact database

**Ecosystem Signal**:
An immutable, source-backed measurement about a reusable repository, package, release, configuration, or ecosystem subject used for opportunity ranking. It never claims that a project Target is vulnerable.
_Avoid_: Research Observation, Finding, risk score, target fact

**Knowledge Concept**:
A reusable security subject with one stable lowercase `namespace:value` ID, one controlled kind, typed external identifiers, and source references. Weaknesses, attack patterns, techniques, controls, protocols, tools, commands, and standards are Knowledge Concepts.
_Avoid_: project fact, finding, copied taxonomy row
Expand Down Expand Up @@ -56,6 +60,26 @@ _Avoid_: reasoning text, inferred edge, model explanation
An unresolved, citation-backed Investigation Assertion ranked for follow-up after accounting for objective relevance, missing evidence, expected information gain, target importance, cost, risk, and authorization readiness.
_Avoid_: autonomous plan, agent hunch, task queue

**Target Recipe**:
A versioned, portable contract for reproducing one authorized research target configuration, including immutable upstream identity, fixtures, lifecycle, isolation, evidence, provenance, and required authorization intent.
_Avoid_: benchmark task, Compose file, target manifest, deployment script

**Device Research Lane**:
One evidence-gated authorization stage for public research, owned-device offline analysis, non-mutating interaction, or separately approved persistent/destructive work. Eligibility for a lane never creates execution authority or carries approval into another lane.
_Avoid_: device mode, blanket hardware authorization, safe command

**Campaign Autonomy Policy**:
A versioned decision contract that controls scheduling, approval consumption, budget stops, and recovery deduplication for a research campaign without creating target authorization or approval authority.
_Avoid_: YOLO mode, blanket approval, autonomous permission

**Product Skill Promotion**:
The reviewed transition that turns source- and Artifact-backed, target-agnostic campaign methodology into a discoverable runtime skill, while keeping eval and benchmark evidence validation-only and candidate-invisible.
_Avoid_: prompt extraction, transcript-to-skill, benchmark lesson

**Research Campaign Definition**:
A frozen, human-selected plan connecting one ranked ecosystem opportunity, admitted Target Recipe, authorization scope, autonomy policy, thread topology, harness admissions, budgets, exit criteria, honesty mode, recovery policy, and stop conditions before launch.
_Avoid_: agent plan, benchmark manifest, target authorization

**Shared Terminal Session**:
A project/thread-scoped interactive shell session whose input, output, resize events, interrupts, approvals, and actor attribution are visible to both the researcher and approved agent automation.
_Avoid_: generic shell bridge, hidden agent shell, human terminal takeover
Expand Down Expand Up @@ -156,6 +180,12 @@ _Avoid_: hidden gold, judge assertion
- A **Research Observation** may indicate several **Knowledge Concepts** through proposed, cited Investigation Assertions without becoming a **Finding**.
- A **Research Observation** may preserve several external identifiers and versioned score assessments; each remains attributable to the Observation's citations and time.
- A **Research Observation** may cite a message from another project thread when that discussion materially supports or contextualizes it.
- A **Research Observation** becomes eligible for promotion to a **Finding** only after cited validation demonstrates a reproducible protected security effect under recorded authorization; rejected leads and coverage records remain distinct outcomes.
- A **Device Research Lane** binds one exact operation and device identity to lane-matching authorization, evidence, stop conditions, and—when interaction is requested—a single-use exact-intent approval.
- Crossing a **Device Research Lane** always creates a new gate. Lane 4 is a separate campaign with rehearsed independent recovery and interactive irreversible checkpoints; earlier authorization never carries forward.
- A **Campaign Autonomy Policy** may consume an already matching durable approval, but it never mints one, widens target scope, extends an expired decision, overrides a denial, or blindly repeats an unknown side effect.
- **Product Skill Promotion** requires cited reusable claims, independent campaign evidence under a published threshold, contamination review, approval-boundary review, evidence-backed validation, and an explicit deepen-versus-new decision before registry publication.
- A **Research Campaign Definition** may become ready for the execution gate only after human selection and admission checks; it never launches a target, creates execution authority, or imports a global vulnerability claim.
- An **Investigation Entity** references a canonical project record when one exists instead of copying that record into the **Investigation Graph**.
- An **Investigation Assertion** may be supported, contradicted, derived, revised, rejected, or left unresolved without changing the canonical record it discusses.
- An **Investigation Citation** identifies why an **Investigation Assertion** exists; an **Artifact** remains the durable evidence object.
Expand Down
18 changes: 18 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,12 @@ Workspace tools are conservative by default:
- generic workspace command execution is disabled;
- approved commands go through app-owned lab or SSH command tools where target mode, approvals, and artifacts can be enforced.

## Target Recipes And Research Campaigns

A Target Recipe is the product-owned, portable contract for one reproducible research target configuration. It pins source and image revisions, fixture identity, script-backed lifecycle steps, loopback or internal-only exposure, resource and isolation limits, reset and teardown verification, evidence paths, provenance, and the authorization intent a later campaign must satisfy. Recipe admission produces a stable digest and target locator, but never creates target authorization or approval. Benchmark discovery remains a separate registry so hidden scoring material and replay controls cannot enter ordinary project memory.

Harness adapters consume admitted recipes rather than embedding Compose or target-specific lifecycle knowledge. Current supported revisions are the genuine-discovery lane; historical vulnerable revisions remain explicitly labeled controls. The first tracer recipe and campaign persistence are added only after their decision tickets settle the remaining provisioning and autonomy details.

## Evidence Path

Generated evidence and uploads should flow through `src/server/evidence/artifact-service.ts`.
Expand All @@ -138,8 +144,20 @@ The pinned `@mastra/lance` package carries a local patch for three adapter defec

The explicit Security Knowledge Graph remains in SQLite for versioned, reusable concepts; controlled predicates; source-backed relationships; typed external identifiers; tool I/O; tool groups; group membership; and weighted tool relationships. Reusable IDs use lowercase `namespace:value` keys. Seed publication synchronizes the owned catalog revision so renamed or retired seed edges do not survive indefinitely. `knowledge_tool_groups`, `knowledge_tool_group_members`, and `knowledge_tool_relationships` are seeded from curated common-shell transitions plus existing `docs/tools/*` `category` and `related_tools` frontmatter. Eval scoring treats a relationship match as positive sequence-coherence evidence; a missing edge remains unmodeled rather than becoming an exclusive allowlist failure. Skill Markdown chunking and semantic document retrieval use Mastra RAG, while deterministic keyword and concept-graph traversal remain local and explicit.

Pinned global source snapshots may yield immutable Ecosystem Signals for opportunity ranking. Ranking applies license, revision, reproducibility, disclosure, isolation, egress, reset/teardown, and authorization-readiness gates before arithmetic; missing evidence holds a candidate instead of becoming zero. Eligible candidates are ordered only inside comparable cohorts using the published seven-dimension vector. The total never appears without its contributions, confidence, source-signal references, and unweighted change/disclosed-history overlays, and it never asserts that a project Target is vulnerable. Selecting a candidate is the boundary that creates project-owned Targets and subsequent Research Observations; global signals themselves never cross into project memory as deployed-target facts.

The project Investigation Graph is an assertion layer over existing records, not another owner of Targets, Artifacts, Findings, Research Observations, Tasks, Attack Paths, Tool Runs, messages, memory, or reusable security knowledge. A Research Observation preserves measured or directly seen behavior, structured inputs and outputs, measurements, external identifiers, versioned scores, actor, time, and precise citations before interpretation. The user-facing Research Map projects canonical records, cited threads and messages, external sources, reusable-concept references, current Investigation Assertions, and Research Priorities through one coherent relational snapshot. The write model resolves canonical records through project-local Investigation Entities and stores append-only Assertions, coordinate-only role-bearing Citations, and rule-versioned Derivations with ordered inputs. Evidence state (`observed`, `derived`, `proposed`, `contradicted`, or `rejected`) stays separate from assertion lifecycle (`current`, `withdrawn`, or `superseded`). Revision is an optimistic, transactional replacement that retains the predecessor and its citations. SQLite and PostgreSQL relational queries define correctness. See [ADR 0001](./adr/0001-investigation-graph-as-assertion-layer.md).

Impact validation is a deterministic promotion boundary over those records. An anomaly remains a Research Observation until cited Artifacts and Investigation Assertions demonstrate a protected read/write, cross-account effect, privilege change, secret exposure, integrity loss, deletion, availability loss, or another concrete security effect under recorded authorization and a reproducible Target Recipe/configuration. The gate preserves rejected leads, coverage records, and inconclusive observations as separate outcomes; only `finding-ready` decisions may feed the Evidence Interface's Finding creation path.

Device research uses four server-owned admission lanes: public-source research, owned-device acquisition/offline analysis, non-mutating interaction, and a separate persistent/destructive campaign. The deterministic admission boundary checks the exact operation, physical-unit identity, lane-matching target authorization, single-use normalized approval intent where required, evidence readiness, isolation, before/after observation, and universal stop conditions. Persistent work additionally requires a new campaign, an evidenced research need, pinned original/candidate/recovery images, independent rehearsed recovery, replaceability, physical-safety planning, disclosure readiness, and operator checkpoints. An eligible result only identifies the next enforcement gate; it never creates target authorization, consumes an approval, or operates a device.

Campaign autonomy is a scheduling and recovery policy, not an authorization source. Manual, bounded, and fully automated modes share the same durable target ledger and exact-intent enforcement. Only active probes, downloads, and shell commands may consume an explicitly declared exact preauthorization in a non-manual mode; credential tests, browser mutations, writes, exploit validation, and patching require a fresh exact decision. Denied, expired, mismatched, or consumed approvals stop the transition. Recovery reuses successful side effects and pauses to reconcile unknown outcomes before any replay. Every mode obeys conjunctive active-time, wall-time, cost, and action ceilings and preserves the policy/mode, normalized intent, authorization and approval references, Tool Run/effect fingerprint, budget transition, and raw redacted evidence.

Product skill promotion is a reviewed boundary in front of the existing `sandbox/skills` registry and Mastra workspace search. A promotion candidate records whether it deepens an existing skill, creates a genuinely separate procedure, or retires one; cites reusable claims to campaign Artifacts; applies an explicit independent-campaign threshold; and carries contamination, target-agnosticity, secret, approval-boundary, and validation reviews. Eval and benchmark rows may validate the method but are always validation-only and candidate-invisible. An eligible decision yields a scoped registry plan under one skill directory; it does not write or activate skill content. Publication remains a separate reviewed filesystem change, after which native Workspace discovery and `SkillSearchProcessor` expose the procedure on demand.

A Research Campaign Definition freezes the selection boundary without launching work. It joins one human-selected ranked opportunity to a current-supported Target Recipe for organic discovery (or a separately labeled historical control), target authorization scope, campaign-autonomy policy, project-memory-scoped planning/research/validation/reporting threads, admitted harness manifests, conjunctive budgets, exit criteria, honesty boundaries, recovery deduplication, and stop conditions. Admission requires lifecycle/reset/teardown evidence and complete coverage, rejected-lead, impact-validation, patch, disclosure, reusable-method, accounting, terminal, and cleanup dispositions. The admitted definition exposes immutable references for the later execution gate but never creates an approval, target mutation, or vulnerability assertion.

A Research Priority is an unresolved, citation-backed current assertion ranked for follow-up. Its deterministic score weights objective relevance (25%), evidence gap (20%), expected information gain (20%), target importance (15%), inverse predicate cost (8%), inverse predicate risk (7%), and authorization readiness (5%). Authorization readiness comes from the durable target ledger. Deliberately turning a Research Priority into a Task uses the existing Task workflow and a unique assertion-task receipt; it never schedules work, creates an approval, runs a tool, or promotes a Finding.

The former generic security-graph repository is retired. Historical database tables may remain so existing local data is not destructively dropped, but no product path writes them and they are not authoritative. The only graph ownership boundaries are the global Security Knowledge Graph and each project's Investigation Graph.
Expand Down
19 changes: 19 additions & 0 deletions docs/eval-production-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,25 @@ Old, unreferenced chunks can remain under `.next/server`, so a repository-wide s

Run `pnpm eval:postgres:production-smoke` against its isolated temporary PostgreSQL database before a paid batch. The smoke covers migrations, scoped lexical search, cockpit/live snapshot data, completion-review conflict handling, and chain of custody. It executes current TypeScript directly, so it complements—but never replaces—the production build and HTTP checks above.

Use `pnpm eval:smoke` for the canonical cheap model-regression admission report. Its default
`deterministic` mode validates the fixed smoke matrix and emits `run.json` plus `report.md` without
calling a model, provider, target, or tool. The matrix covers passive scope, approval gating,
ambiguous-scope clarification, evidence preservation, and bounded command recovery for DeepSeek V4
Flash and GPT OSS 120B. Override the selection with `--models=<id,...>`; the documented local fallback
is `--models=local-gemma4-12b`.

`--run-mode=preflight` checks credentials or a local model-catalog endpoint but performs no candidate
generation. Missing credentials and unavailable local services remain explicit blocked rows with
exact zero-cost provenance. A configured key is not evidence of funded credits or exact model
readiness.

Only `--run-mode=live` may delegate to the production model-tool runner. Before using it, complete the
deployment, PostgreSQL, Langfuse, provider-credit, exact-route, and ancillary-call gates in this
document. Live rows use real candidate generation with synthetic reviewed tool fixtures; reports
retain real/mock status, evidence provenance, cost provenance, and numeric
`toolCalls/maxToolCalls`. Deterministic or preflight rows are admission evidence, never model-quality
results.

Run one zero-cost integration batch after a repair. Do not cycle through patch → paid canary → patch → paid canary. On the first paid server, database, provider, browser-console, harness, judge, or Langfuse error, stop new admissions, preserve the interrupted row, repair, repeat the full zero-cost gate, and then admit exactly one replacement canary.

### Target address authority
Expand Down
7 changes: 7 additions & 0 deletions docs/lab-runtime-hardening.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ The default `container` isolation mode above shares the host kernel with every l
### Enabling it

- Per-lab: pass `isolation: "microvm"` (and optionally `microvmRuntimeClass`) to the lab create/start/restart API. The choice is persisted in lab metadata, so subsequent starts reuse it.
- Workload class: selecting `workloadClass: "untrusted-code"` or `workloadClass: "malware-analysis"` requires and persists MicroVM isolation. An explicit request to run either class with `isolation: "container"` is rejected; omitting isolation selects `microvm` and then follows the same fail-closed runtime-availability check. Containers carry both isolation and workload-class labels for external inspection.
- Host-wide default: set `PROJECT_LAB_ISOLATION_MODE=microvm`, and optionally `PROJECT_LAB_MICROVM_RUNTIME_CLASS` to override the default runtime class.
- Runtime class names map directly to `docker run --runtime <class>` and must already be registered in the Docker daemon's `runtimes` config (`/etc/docker/daemon.json`) by a Kata Containers install:
- `kata-fc` (default) — Firecracker VMM. Smallest device model and strongest isolation, but only virtio net/block/vsock devices are available to the guest.
Expand Down Expand Up @@ -101,6 +102,12 @@ For the `approved-targets` profile, the iptables script is dynamically built fro

Denied packets are rate-limited and logged with the `EXPLOIT_HUNTER_EGRESS_DENIED` prefix. The controller lifecycle, policy fingerprint, and Docker command trace are inspectable forensic evidence; workloads cannot modify the firewall because they do not possess the capability.

### Durable network evidence

Lab start and restart now save the external controller's enforcement result through the central Artifact service as project-scoped JSONL. Each record uses the versioned `exploit-hunter.lab-network-evidence.v1` schema and can carry project, thread, task, tool-run, research-run, network-profile, and policy-fingerprint correlation. Enforcement failures are recorded as `enforcement-unavailable`; successful policy installation is recorded separately as `policy-enforced` and is not represented as proof that a connection was allowed.

The same schema reserves `allowed` and `denied` dispositions for observed DNS resolutions and connection attempts. Controller observations must match the active project and policy fingerprint before ingestion. The current controller does not yet emit per-connection records into this path, so lifecycle artifacts must not be treated as a complete network transcript. Opt-in mitmproxy captures remain the available application-level traffic record, with the protocol and bypass limitations described above.

### Package-egress enforcement

For the `package-egress` profile, the iptables script allows traffic only to well-known package registries and distribution mirrors. This covers npm, Yarn, PyPI, RubyGems, crates.io, GitHub release objects, Debian/Ubuntu apt repositories, and Docker Hub.
Expand Down
Loading
Loading