Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,21 @@ security gaps; all critical/high findings shipped the same day:
foreign-key constraint and 500'd; `deletePolicy` now cascades its
`policy_results` in a transaction and reports the count in the audit log.

### Removed

- **The unused orchestration layer.** Jobs, the dispatcher, the orchestrator,
and coordination locks (core), their `jobs`/`locks` tables (db), and the
`job`, `lock`, and `dispatch` CLI commands were dead weight: nothing in the
hook-driven product path ever created a job or acquired a lock, dispatch
could never match a hook-created session, and the lock-expiry SQL was
broken. The dashboard's Jobs/Locks/Coordination pages were removed long
ago for the same reason. Databases created before this change keep their
empty `jobs`/`locks` tables; they are inert.
- **`agentops pr`.** It could never create a PR (`gh pr create` rejects the
`--json` flag it passed, and the error was swallowed), so every invocation
printed "Failed to create PR". `agentops link` (read-only PR/issue
linking) is unaffected.

### Added

#### Phase A — trial-blocking foundations
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ core ← db ← cli
```

- **@agentops/core** — Domain types, policy engine, scoring algorithm, and builder functions for all entities. No external dependencies. All types use `readonly` and branded ID types (RunId, JobId, SessionId, etc.) for type safety.
- **@agentops/db** — SQLite persistence via Drizzle ORM + better-sqlite3. Sixteen tables: `runs`, `policies`, `policy_results`, `run_metrics`, `jobs`, `sessions`, `events`, `locks`, `users`, `api_tokens`, `auth_sessions`, `device_codes`, `webhooks`, `webhook_deliveries`, `audit_log`, `user_budgets`. Complex fields stored as JSON columns. DB defaults to `~/.agentops/agentops.db` (override with `AGENTOPS_DB_PATH`). WAL mode with `foreign_keys = ON` — deletes of parent rows must cascade children first (see `deletePolicy`, `deleteOldRuns`).
- **@agentops/cli** — CLI entry point (`agentops`). Commands: `init`, `serve`, `setup`, `hook`, `login`, `doctor`, `user`, `admin`, `cleanup`, `run`, `policy`, `report`, `wrap`, `watch`, `link`, `pr`, `job`, `session`, `events`, `lock`, `dispatch`. Supports `--json` output and `--db-path` override. `init` bootstraps the DB (`--seed` for sample data, `--seed-policies` for the starter policy set, `--clean` to reset). `serve` starts the dashboard server (`--port` to override 3000). `setup` configures Claude Code hooks (`--global`, `--uninstall`, `--dry-run`). `hook` handles Claude Code hook events (session-start, pre-tool-use, post-tool-use, user-prompt-submit, stop, subagent-stop, session-end) — reads JSON from stdin, manages state under `~/.agentops/state/`, evaluates policies in real-time, can block risky tool calls (exit code 2), and reads cost/token usage from the Claude Code transcript (deduped by `message.id`; unknown models warn on stderr instead of pricing at $0). `login` runs the device-flow auth against a dashboard; `doctor` diagnoses a local install. Helper modules: `format.ts` (output formatting), `git.ts` (git integration), `github.ts` (GitHub API), `transcript.ts` (usage/cost from Claude Code transcripts), `pricing` lives in core. Note: `job`, `lock`, `dispatch`, `wrap`, and `watch` operate on orchestration machinery that Claude Code hooks never populate — treat them as experimental/vestigial.
- **@agentops/db** — SQLite persistence via Drizzle ORM + better-sqlite3. Fourteen tables: `runs`, `policies`, `policy_results`, `run_metrics`, `sessions`, `events`, `users`, `api_tokens`, `auth_sessions`, `device_codes`, `webhooks`, `webhook_deliveries`, `audit_log`, `user_budgets`. Complex fields stored as JSON columns. DB defaults to `~/.agentops/agentops.db` (override with `AGENTOPS_DB_PATH`). WAL mode with `foreign_keys = ON` — deletes of parent rows must cascade children first (see `deletePolicy`, `deleteOldRuns`). (Databases created before July 2026 may carry orphaned `jobs`/`locks` tables from the removed orchestration layer; they're inert.)
- **@agentops/cli** — CLI entry point (`agentops`). Commands: `init`, `serve`, `setup`, `hook`, `login`, `doctor`, `user`, `admin`, `cleanup`, `run`, `policy`, `report`, `wrap`, `watch`, `link`, `session`, `events`. Supports `--json` output and `--db-path` override. `init` bootstraps the DB (`--seed` for sample data, `--seed-policies` for the starter policy set, `--clean` to reset). `serve` starts the dashboard server (`--port` to override 3000). `setup` configures Claude Code hooks (`--global`, `--uninstall`, `--dry-run`). `hook` handles Claude Code hook events (session-start, pre-tool-use, post-tool-use, user-prompt-submit, stop, subagent-stop, session-end) — reads JSON from stdin, manages state under `~/.agentops/state/`, evaluates policies in real-time, can block risky tool calls (exit code 2), and reads cost/token usage from the Claude Code transcript (deduped by `message.id`; unknown models warn on stderr instead of pricing at $0). `login` runs the device-flow auth against a dashboard; `doctor` diagnoses a local install. Helper modules: `format.ts` (output formatting), `git.ts` (git integration), `github.ts` (GitHub API), `transcript.ts` (usage/cost from Claude Code transcripts), `pricing` lives in core.
- **@agentops/sdk** — Lightweight HTTP client for agent runtimes to talk to the AgentOps server. Depends only on `@agentops/core` for types. Uses native `fetch`. Provides `AgentOpsClient` class (via `createClient()` factory) with methods: `createSession`, `startRun`, `reportAction`, `reportArtifact`, `reportMetrics`, `checkPolicy`, `heartbeat`, `completeRun`, `failRun`, `terminateSession`. Also exports `PolicyMiddleware` for pre-flight policy checks before actions. Throws typed `AgentOpsError` with status codes.
- **@agentops/web** — Next.js 16 App Router dashboard with React 19, Tailwind CSS 4. API routes under `src/app/api/` organized by resource (runs, sessions, policies, events, analytics, admin, stats, sdk, auth, budgets, webhooks). Jobs, Locks, and Coordination pages were removed from the dashboard because hooks do not populate this data. Sidebar nav: Runs | Sessions | Events | Analytics | Usage | Policies | Settings. **Every API route is authenticated** (`src/lib/auth.ts`): bearer tokens or session cookies, `admin`/`member` roles, members are scoped to their own runs/sessions (`resolveViewScope`), non-owners get 404 (not 403) to avoid ID enumeration, mutations additionally pass `checkSameOrigin` CSRF checks — new routes must follow this pattern (regression tests live in `api/__tests__/auth-gaps.test.ts`). Inbound SDK routes under `/api/sdk/` use bearer auth + per-token rate limits. The Usage page (`/usage`) always shows the local hook-captured rollup (incl. Bedrock-vs-direct backend split and per-user attribution); when `ANTHROPIC_ADMIN_API_KEY` is set it additionally shows org-wide cost/tokens via `/api/admin/{status,cost,analytics}`, which normalize the Anthropic Admin API's reports (RFC 3339 `starting_at`, paginated, amounts are decimal-string cents) server-side. Server components access SQLite directly via a singleton lazy-loaded DB instance (`src/lib/db.ts`). Uses `serverExternalPackages: ["better-sqlite3"]` in next.config.ts. Path alias: `@/*` → `./src/*`. Dark theme by default.

Expand Down
19 changes: 1 addition & 18 deletions packages/cli/src/__tests__/commands-repo-normalization.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ import { mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { resolve } from "node:path";
import { Command } from "commander";
import { getDb, listRuns, listJobs } from "@agentops/db";
import { getDb, listRuns } from "@agentops/db";
import { registerRunCommands } from "../commands/run.js";
import { registerJobCommands } from "../commands/job.js";

// Proves the CLI write paths that accept an explicit --repo option
// (`run start`, `job submit`) canonicalize it on write, so an operator-typed
Expand Down Expand Up @@ -38,7 +37,6 @@ async function runCli(args: string[], dbPath: string): Promise<void> {
.option("--db-path <path>", "DB path", dbPath)
.option("--json");
registerRunCommands(program);
registerJobCommands(program);
await program.parseAsync(["node", "agentops", ...args]);
} finally {
console.log = originalLog;
Expand All @@ -58,21 +56,6 @@ describe("CLI --repo write-path normalization", () => {
expect(runs[0]!.environment.repo).toBe("iaj6/agentops");
});

it("`job submit --repo` canonicalizes a full remote URL", async () => {
const dir = makeTmpDir();
const dbPath = resolve(dir, "test.db");
const db = getDb(dbPath);

await runCli(
["job", "submit", "ship it", "--repo", "git@github.com:Iaj6/AgentOps.git"],
dbPath,
);

const jobs = listJobs(db, { limit: 10 });
expect(jobs).toHaveLength(1);
expect(jobs[0]!.environment.repo).toBe("iaj6/agentops");
});

it("the default --repo ('unknown') round-trips unchanged", async () => {
const dir = makeTmpDir();
const dbPath = resolve(dir, "test.db");
Expand Down
176 changes: 1 addition & 175 deletions packages/cli/src/__tests__/github.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { getLinkedPR, getIssue, createPR, addPRComment, createCheckRun, isGhAvailable } from "../github.js";
import { getLinkedPR, getIssue, isGhAvailable } from "../github.js";

vi.mock("node:child_process", () => ({
execFileSync: vi.fn(),
Expand All @@ -17,7 +17,7 @@
});

// Helper: make gh --version succeed (gh is available)
function mockGhAvailable() {

Check warning on line 20 in packages/cli/src/__tests__/github.test.ts

View workflow job for this annotation

GitHub Actions / Lint

'mockGhAvailable' is defined but never used. Allowed unused vars must match /^_/u
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
Expand Down Expand Up @@ -253,177 +253,3 @@
});
});

describe("createPR", () => {
it("returns null when gh is not available", () => {
mockGhUnavailable();
expect(createPR("title", "body")).toBeNull();
});

it("creates a PR and returns parsed data", () => {
const prData = {
number: 55,
title: "New feature",
url: "https://github.com/acme/app/pull/55",
state: "OPEN",
headRefName: "feat/new",
baseRefName: "main",
additions: 50,
deletions: 10,
changedFiles: 3,
};

mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
if (cmdStr.startsWith("gh pr create")) return JSON.stringify(prData) as any;
return "" as any;
});

const pr = createPR("New feature", "Description of the feature", "main");
expect(pr).not.toBeNull();
expect(pr!.number).toBe(55);
expect(pr!.title).toBe("New feature");
});

it("returns null when gh pr create fails", () => {
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
// All gh commands return empty (failure)
return "" as any;
});

expect(createPR("title", "body")).toBeNull();
});
});

describe("addPRComment", () => {
it("returns false when gh is not available", () => {
mockGhUnavailable();
expect(addPRComment(42, "comment")).toBe(false);
});

it("returns true on success", () => {
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
if (cmdStr.startsWith("gh pr comment")) return "https://github.com/acme/app/pull/42#comment" as any;
return "" as any;
});

expect(addPRComment(42, "Looks good!")).toBe(true);
});

it("returns false when comment fails", () => {
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
return "" as any;
});

expect(addPRComment(42, "comment")).toBe(false);
});
});

describe("createCheckRun", () => {
it("returns null when gh is not available", () => {
mockGhUnavailable();
expect(createCheckRun("test", "completed", "success")).toBeNull();
});

it("returns check object with provided values", () => {
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
if (cmdStr.startsWith("git rev-parse")) return "abc123def\n" as any;
if (cmdStr.startsWith("gh api")) return "{}" as any;
return "" as any;
});

const check = createCheckRun("CI Check", "completed", "success", "https://ci.example.com/run/1");
expect(check).not.toBeNull();
expect(check!.name).toBe("CI Check");
expect(check!.status).toBe("completed");
expect(check!.conclusion).toBe("success");
expect(check!.url).toBe("https://ci.example.com/run/1");
});

it("returns null when HEAD sha cannot be resolved", () => {
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
if (cmdStr.startsWith("git rev-parse")) throw new Error("not a git repo");
return "" as any;
});

expect(createCheckRun("test", "completed", "success")).toBeNull();
});
});

// Injection-fix guard: untrusted text (PR/comment bodies, check-run payloads)
// must travel via stdin (the execFileSync `input` option), never embedded in
// argv — and there must be no shell. These assertions FAIL against vulnerable
// inline-argv code like gh(['pr','create','--title',t,'--body',body]).
describe("argument safety (no shell injection surface)", () => {
// Each recorded call is [file, argsArray, options]; pull out file/args/input.
function calls() {
return mockExecSync.mock.calls.map((c) => ({
file: c[0] as string,
args: (c[1] as string[]) ?? [],
input: (c[2] as { input?: string } | undefined)?.input,
}));
}

it("createPR pipes the body via stdin, never argv", () => {
mockGhAvailable();
const body = "evil $(rm -rf /) `whoami`\nsecond line";
const title = "title with `backticks` and $(subshell)";
createPR(title, body, "main");

const call = calls().find((c) => c.args[0] === "pr" && c.args[1] === "create");
expect(call).toBeDefined();
expect(call!.args).toContain("--body-file");
expect(call!.args).toContain("-");
expect(call!.args).not.toContain(body); // body is NOT in argv
expect(call!.input).toBe(body); // body arrives via stdin
// Title is passed as a single argv element — no shell, so metacharacters
// are literal, not interpreted.
expect(call!.args).toContain(title);
});

it("addPRComment pipes the comment body via stdin, never argv", () => {
mockGhAvailable();
const body = "$(curl http://evil) `id` payload";
addPRComment(42, body);

const call = calls().find((c) => c.args[0] === "pr" && c.args[1] === "comment");
expect(call).toBeDefined();
expect(call!.args).toContain("--body-file");
expect(call!.args).toContain("-");
expect(call!.args).not.toContain(body);
expect(call!.input).toBe(body);
});

it("createCheckRun sends a JSON payload via stdin (--input -), never argv", () => {
mockExecSync.mockImplementation((file: string, args?: readonly string[]) => {
const cmdStr = [file, ...(args ?? [])].join(" ");
if (cmdStr === "gh --version") return "gh version 2.40.0\n" as any;
if (cmdStr.startsWith("git rev-parse")) return "abc123def\n" as any;
return "" as any;
});
createCheckRun("name `id`", "completed", "failure", "https://x/$(id)");

const call = calls().find((c) => c.args[0] === "api");
expect(call).toBeDefined();
expect(call!.args).toContain("--input");
expect(call!.args).toContain("-");
// Raw metacharacters never reach argv...
expect(call!.args.join(" ")).not.toContain("$(id)");
// ...they're carried as a well-formed JSON document over stdin.
const payload = JSON.parse(call!.input as string) as Record<string, unknown>;
expect(payload.name).toBe("name `id`");
expect(payload.head_sha).toBe("abc123def");
expect(payload.conclusion).toBe("failure");
expect(payload.details_url).toBe("https://x/$(id)");
});
});
64 changes: 0 additions & 64 deletions packages/cli/src/__tests__/pr.test.ts

This file was deleted.

Loading
Loading