Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 39 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
# You only need one LLM provider key to run Opfor.
# .env is gitignored — never commit your real keys.

# ── LLM providers ────────────────────────────────────────────────────────────
# ── LLM providers — the attacker LLM for `opfor run` ─────────────────────────
# Any one of these is enough. NOTE: `opfor hunt` does not use these — its agents
# run on Claude only. See "Autonomous hunt" below.

# Groq (fast, free tier available — good default for getting started)
# GROQ_API_KEY=
Expand All @@ -22,6 +24,42 @@
# put the API key here if the endpoint requires one.
# OPFOR_API_KEY=

# ── Autonomous hunt (`opfor hunt`) — the attacker agents ─────────────────────
# The commander/operator/scout agents run on the Claude Agent SDK, so they are
# Claude-only regardless of which provider key you set above. Your TARGET can
# still be any model or agent.
#
# Credentials are resolved in this order — the first match wins:
#
# 1. ANTHROPIC_API_KEY pay-per-token API key
# 2. ANTHROPIC_BASE_URL + ANTHROPIC_AUTH_TOKEN gateway (LiteLLM, proxy, …)
# 3. CLAUDE_CODE_OAUTH_TOKEN from `claude setup-token`
# 4. ~/.claude/.credentials.json from `claude login` (Pro/Max)
#
# ANTHROPIC_API_KEY=

# Gateway / self-hosted proxy — SET BOTH OR NEITHER.
# ANTHROPIC_AUTH_TOKEN on its own is ignored (it is indistinguishable from a token
# inherited from a parent Claude Code session), and the run silently falls through
# to option 3 or 4 — e.g. billing your personal subscription instead of the gateway.
# ANTHROPIC_BASE_URL=https://your-gateway.example.com
# ANTHROPIC_AUTH_TOKEN=

# Subscription token, if you are not using an API key or gateway.
# CLAUDE_CODE_OAUTH_TOKEN=

# Optional — pin the `haiku` / `sonnet` / `opus` aliases to specific snapshots.
# Applies to --model / --operator-model / --scout-model and the --ui model pickers.
# ANTHROPIC_DEFAULT_HAIKU_MODEL=
# ANTHROPIC_DEFAULT_SONNET_MODEL=
# ANTHROPIC_DEFAULT_OPUS_MODEL=

# ── Target authentication (the system under test) ────────────────────────────
# Separate from everything above — this is the credential Opfor sends TO your
# target, not one it uses itself. Reference the variable NAME via
# `opfor hunt --target-key-env`, `apiKeyEnv` in a config, or the --ui setup form.
# TARGET_API_KEY=

# ── Telemetry enrichment (optional) ──────────────────────────────────────────

# Langfuse — pulls production traces to ground attack prompts
Expand Down
2 changes: 2 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@ The CLI loads `.env` from the current working directory automatically. Add `.env
opfor run --config .opfor/configs/opfor-config-....json --env .env.prod
```

> This key is for `opfor run`'s attacker LLM only. `opfor hunt` uses a separate, Claude-only credential — see [hunt.md § Authentication](hunt.md#authentication).

Telemetry credentials (Langfuse, Netra) also come from env vars — see [Trace-aware testing](#trace-aware-testing-agent-only).

> Add `.opfor/` to `.gitignore` — it contains configs and reports with embedded target metadata.
Expand Down
22 changes: 21 additions & 1 deletion docs/hunt.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,13 +150,33 @@ Credentials are resolved in order:

Options 2 and 3 require the [Claude Code CLI](https://docs.claude.com/claude-code) (`npm install -g @anthropic-ai/claude-code`).

**Gateway / self-hosted proxy** — set both together (a token without a base URL is ignored):
Note this is Claude-only, and independent of the provider key `opfor run` uses for its attacker LLM. Your target can still be any model or agent.

**Gateway / self-hosted proxy** — set both together:

```bash
ANTHROPIC_BASE_URL=https://your-gateway.example.com
ANTHROPIC_AUTH_TOKEN=...
```

> `ANTHROPIC_AUTH_TOKEN` on its own is **ignored**. A bare token is indistinguishable from one inherited from a parent Claude Code session, so it is stripped before the agents start — and the run silently falls through to the next credential in the list, which may mean billing your personal subscription instead of the gateway. `opfor hunt` warns about this at startup and on the `--ui` setup form.

The credential actually in use is printed at startup (`Authenticating via: …`) and shown on the `--ui` setup form.

**Skipping `.env` entirely** — the `--ui` setup form can also take an API key or gateway pair directly, if nothing is detected in the environment (or you'd rather not touch one at all). It's applied for that run only and never written to disk.

### Pinning model snapshots

`--model`, `--operator-model`, and `--scout-model` take the aliases `haiku` / `sonnet` / `opus`. To pin those aliases to specific snapshots — for a gateway that only exposes certain ids, or to freeze behaviour across runs — set:

```bash
ANTHROPIC_DEFAULT_HAIKU_MODEL=claude-haiku-4-5-20251001
ANTHROPIC_DEFAULT_SONNET_MODEL=claude-sonnet-4-6
ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-4-8
```

Unset, each alias falls back to a built-in default. Full model ids can also be passed directly to the `--*-model` flags, bypassing aliases entirely.

## Vulnerability Classes

`bias` · `harmful` · `accuracy` · `disclosure` · `injection` · `excessive-agency` · `brand-conduct` · `access-control` · `mcp-usage`
Expand Down
55 changes: 17 additions & 38 deletions runners/cli/src/commands/hunt.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import type { Command } from "commander";
import path from "node:path";
import { readFile } from "node:fs/promises";
import { createWriteStream, existsSync, mkdirSync, type WriteStream } from "node:fs";
import { homedir } from "node:os";
import { createWriteStream, mkdirSync, type WriteStream } from "node:fs";
import { consola } from "consola";
import type {
HuntOptions,
Expand All @@ -17,6 +16,7 @@ import {
} from "@keyvaluesystems/agent-opfor-core/autonomous/report/writeReport.js";
import { startUiServer } from "../ui/server.js";
import { mergeReporters } from "../ui/bridge.js";
import { resolveBrainAuth, noBrainAuthMessage } from "../lib/brainAuth.js";

/** Short HH:MM:SS timestamp for live log lines. */
function clock(): string {
Expand Down Expand Up @@ -107,35 +107,6 @@ function mapAgentTargetToAutonomous(t: ReturnType<typeof parseAgentTarget>): Tar
};
}

const NO_BRAIN_AUTH_MESSAGE =
"No Claude credentials found. Set ANTHROPIC_API_KEY, or run `claude login` / `claude setup-token` to use a Claude subscription.";

/**
* Resolve which credential the Claude Agent SDK will authenticate with, for a
* user-facing log line — or null if none is configured.
*
* The SDK resolves credentials itself (first match wins): ANTHROPIC_API_KEY →
* CLAUDE_CODE_OAUTH_TOKEN → a stored `~/.claude/.credentials.json` from a Claude
* subscription login (`claude setup-token` / `claude login`). This is a courtesy
* pre-check so we can emit an actionable message instead of a cryptic SDK error;
* it must therefore recognize the subscription path, not just env vars.
*/
function resolveBrainAuth(): string | null {
if (process.env.ANTHROPIC_API_KEY?.trim()) return "ANTHROPIC_API_KEY";
// ANTHROPIC_AUTH_TOKEN only counts alongside ANTHROPIC_BASE_URL: buildChildEnv()
// strips a bare token (it's treated as an inherited session token), so counting
// it here without a gateway URL would pass the gate then lose the credential.
if (process.env.ANTHROPIC_AUTH_TOKEN?.trim() && process.env.ANTHROPIC_BASE_URL?.trim()) {
return `gateway (${process.env.ANTHROPIC_BASE_URL})`;
}
if (process.env.CLAUDE_CODE_OAUTH_TOKEN?.trim()) return "CLAUDE_CODE_OAUTH_TOKEN";
// Claude subscription: credentials stored on disk by `claude setup-token` / `claude login`.
if (existsSync(path.join(homedir(), ".claude", ".credentials.json"))) {
return "Claude subscription (~/.claude/.credentials.json)";
}
return null;
}

function intOr(value: string | undefined, fallback: number): number {
const n = parseInt(value ?? "", 10);
return Number.isFinite(n) && n > 0 ? n : fallback;
Expand Down Expand Up @@ -237,13 +208,19 @@ export function registerHuntCommand(program: Command): void {
// --target-config, e.g. a local-script target), launch the setup wizard.
// Otherwise --ui means the live dashboard for the already-configured target.
if (opts.ui && !opts.endpoint && !opts.targetConfig) {
// Unlike the direct-run path below, the setup form can accept a brain-auth
// override (API key or gateway pair) for this run only — so a missing
// credential here is not fatal; the browser still opens and the form
// requires an override before it lets you start.
const brainAuth = resolveBrainAuth();
if (!brainAuth) {
consola.error(NO_BRAIN_AUTH_MESSAGE);
process.exitCode = 1;
return;
if (brainAuth) {
consola.info(`Authenticating via: ${brainAuth.method}`);
if (brainAuth.warning) consola.warn(brainAuth.warning);
} else {
consola.warn(
"No Claude credential detected — provide one on the setup page before starting."
);
}
consola.info(`Authenticating via: ${brainAuth}`);

const uiPort = intOr(opts.uiPort, 3847);

Expand Down Expand Up @@ -283,6 +260,7 @@ export function registerHuntCommand(program: Command): void {
},
setupMode: true,
initialConfig,
brainAuth: brainAuth ?? undefined,
openBrowser: true,
onLog: (line) => {
process.stdout.write(line + "\n");
Expand Down Expand Up @@ -324,11 +302,12 @@ export function registerHuntCommand(program: Command): void {

const brainAuth = resolveBrainAuth();
if (!brainAuth) {
consola.error(NO_BRAIN_AUTH_MESSAGE);
consola.error(noBrainAuthMessage());
process.exitCode = 1;
return;
}
consola.info(`Authenticating via: ${brainAuth}`);
consola.info(`Authenticating via: ${brainAuth.method}`);
if (brainAuth.warning) consola.warn(brainAuth.warning);

// Check endpoint is provided when not using setup UI (the endpoint may
// instead come from --target-config).
Expand Down
83 changes: 83 additions & 0 deletions runners/cli/src/lib/brainAuth.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// Which credential the Claude Agent SDK will authenticate the commander/operator/
// scout agents with. Shared by the CLI's startup precheck (hunt.ts) and the setup
// server's /api/brain-auth + override handling (ui/server.ts) — kept in its own
// module so neither has to import the other.

import { existsSync } from "node:fs";
import { homedir } from "node:os";
import path from "node:path";

/**
* Human-readable credential source, e.g. "ANTHROPIC_API_KEY". Never a secret value.
* `warning` is set when a configured credential was silently ignored (see below).
*/
export interface BrainAuthInfo {
method: string;
warning?: string;
}

const NO_BRAIN_AUTH_MESSAGE =
"No Claude credentials found. Set ANTHROPIC_API_KEY, or run `claude login` / `claude setup-token` to use a Claude subscription.";

const ORPHAN_GATEWAY_TOKEN_WARNING =
"ANTHROPIC_AUTH_TOKEN is set but ANTHROPIC_BASE_URL is not — the token is ignored and the run " +
"falls back to the next credential. Set both together to route through a gateway.";

/** True when ANTHROPIC_AUTH_TOKEN is set but its required pair, ANTHROPIC_BASE_URL, is not. */
function hasOrphanedGatewayToken(): boolean {
return Boolean(
process.env.ANTHROPIC_AUTH_TOKEN?.trim() && !process.env.ANTHROPIC_BASE_URL?.trim()
);
}

/**
* Resolve which credential the Claude Agent SDK will authenticate with, for a
* user-facing log line — or null if none is configured.
*
* The SDK resolves credentials itself (first match wins): ANTHROPIC_API_KEY →
* CLAUDE_CODE_OAUTH_TOKEN → a stored `~/.claude/.credentials.json` from a Claude
* subscription login (`claude setup-token` / `claude login`). This is a courtesy
* pre-check so we can emit an actionable message instead of a cryptic SDK error;
* it must therefore recognize the subscription path, not just env vars.
*/
export function resolveBrainAuth(): BrainAuthInfo | null {
// A gateway token without its base URL is stripped by buildChildEnv(), so the run
// silently proceeds on a *different* credential — e.g. billing a personal Claude
// subscription instead of the intended gateway. Surface that rather than let it pass.
const warning = hasOrphanedGatewayToken() ? ORPHAN_GATEWAY_TOKEN_WARNING : undefined;

if (process.env.ANTHROPIC_API_KEY?.trim()) return { method: "ANTHROPIC_API_KEY", warning };
// ANTHROPIC_AUTH_TOKEN only counts alongside ANTHROPIC_BASE_URL: buildChildEnv()
// strips a bare token (it's treated as an inherited session token), so counting
// it here without a gateway URL would pass the gate then lose the credential.
if (process.env.ANTHROPIC_AUTH_TOKEN?.trim() && process.env.ANTHROPIC_BASE_URL?.trim()) {
// Never interpolate the actual URL: it may carry userinfo or a signed query
// string, and this label is rendered in the setup UI, not just the terminal.
return { method: "gateway (ANTHROPIC_BASE_URL)" };
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
if (process.env.CLAUDE_CODE_OAUTH_TOKEN?.trim()) {
return { method: "CLAUDE_CODE_OAUTH_TOKEN", warning };
}
// Claude subscription: credentials stored on disk by `claude setup-token` / `claude login`.
if (existsSync(path.join(homedir(), ".claude", ".credentials.json"))) {
return { method: "Claude subscription (~/.claude/.credentials.json)", warning };
}
return null;
}

/**
* The error printed when resolveBrainAuth() finds nothing. Special-cased for the
* orphaned-gateway-token footgun — otherwise a user who DID set ANTHROPIC_AUTH_TOKEN
* sees "no credentials found" with no hint that what they configured was silently
* discarded for missing its required ANTHROPIC_BASE_URL pair.
*/
export function noBrainAuthMessage(): string {
if (hasOrphanedGatewayToken()) {
return (
"ANTHROPIC_AUTH_TOKEN is set but ANTHROPIC_BASE_URL is not, so it was ignored, and no " +
"other Claude credential was found. Set ANTHROPIC_BASE_URL alongside it, or set " +
"ANTHROPIC_API_KEY, or run `claude login` / `claude setup-token`."
);
}
return NO_BRAIN_AUTH_MESSAGE;
}
12 changes: 12 additions & 0 deletions runners/cli/src/ui/bridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,18 @@ export class UiBridge implements ProgressReporter {
this.clients.delete(client);
}

/** End every open SSE response so the HTTP server can finish closing. */
closeAllClients(): void {
for (const client of this.clients) {
try {
client.close();
} catch {
// Already torn down by the peer — nothing to do.
}
}
this.clients.clear();
}

snapshot(): UiRunState {
if (this.overrideState) return this.overrideState;
if (!this.runLog) {
Expand Down
Loading
Loading