diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ce5f5e0..d739346 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -144,9 +144,11 @@ app. Closing this needs the server-side blocking-message capture described above - **Scrollback comes from the transcript, not the terminal.** An agent's TUI runs on the *alternate screen* (`ESC[?1049h`), so the emulator keeps no scrollback ring and `pane.read` can never return more than the visible viewport — the live mirror physically cannot scroll back. Pane history is - therefore read from the agent's **own transcript file** off disk (`bridge/transcript.ts`, + therefore read from the agent's **own transcript file** off disk (`bridge/journal/`, `/api/pane/:id/history`), a separate source from the mirror with different fidelity: turns and - their text, not a replay of the screen. The client fetches the whole conversation in one request + their text, not a replay of the screen. Each harness writes a different log in a different place, + so this is a **per-agent adapter** (`bridge/journal/registry.ts` maps the pane's `agent` to one); + a harness with no adapter simply has no journal. The client fetches the whole conversation in one request and renders a window that grows upward, which is what lets find-in-history and jump-to-user-turn work across turns you haven't scrolled to. Rationale and the measured numbers are commented at the top of `web/src/routes/history.tsx`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 9534331..b37a5cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,19 @@ All notable changes to Collie are recorded here. The format follows `version` in `herdr-plugin.toml`, `package.json`, and `web/package.json` (enforced by `scripts/check-version.sh`). See [`CLAUDE.md`](./CLAUDE.md) → *Versioning* for the bump policy. +## [0.19.0] - 2026-07-29 + +### Added +- **Journal (pane history) is now per-harness, with Codex and pi support.** Reading an agent's own session log is an adapter keyed on the pane's agent (`bridge/journal/`), so a new harness is an adapter rather than a fork of the reader — Codex reads its date-partitioned `rollout-*.jsonl`, pi its per-cwd session log. Raised in #40 by @simonallfrey, who asked where to implement journaling for Codex (7e3b2bd) +- **`scripts/journal-probe.ts`** probes every adapter against the real logs on the host — the format-drift check unit tests can't make. It caught Codex 0.145 adding a `developer` message role the parser would have rendered as operator speech (7e3b2bd) + +### Fixed +- **pi could never have had history.** pi reports its session as a kind-`path` ref (an absolute path) and the bridge kept only kind-`id` refs, so a pi pane arrived with no session at all. Both kinds are kept now; a path ref is confined to that harness's root after symlink resolution (7e3b2bd) +- **A pane relaunched as a different agent served the previous agent's session ref.** Herdr keeps reporting the last session announced for a pane — a pane running pi still advertised a `herdr:claude` id. The ref is dropped unless its own `agent` matches the pane's (7e3b2bd) + +### Changed +- **A pane's session reference no longer goes to the browser.** `/api/snapshot` sends `hasSession` instead — for pi the reference is a filesystem path, and the History affordance only ever needed "may this pane have history?". It is now also gated on the harness actually having an adapter (7e3b2bd) + ## [0.18.0] - 2026-07-28 ### Added diff --git a/HERDR_API.md b/HERDR_API.md index 61303a1..7628ab4 100644 --- a/HERDR_API.md +++ b/HERDR_API.md @@ -200,6 +200,21 @@ Two sibling structural ops reorder objects. Both live-verified 2026-07-20 on the `agent_status` ∈ `idle | working | blocked | done | unknown`. Panes without an agent omit/null `agent`. +> **`agent_session` has TWO kinds, and it can outlive the agent that reported it** (live-verified +> 2026-07-29 against claude, codex and pi panes). Each harness's herdr integration reports through +> `pane.report_agent_session`, and what it reports differs: +> - `kind:"id"` + a uuid — claude (`source:"herdr:claude"`) and codex (`source:"herdr:codex"`, from +> codex's `SessionStart` hook; verified on codex 0.145.0). +> - `kind:"path"` + an **absolute path to the log file** — pi (`source:"herdr:pi"`). pi's integration +> prefers `agent_session_path` over an id whenever its session manager has a file open. +> +> Two consequences. A harness only reports at all once `herdr integration install ` has been +> run — pi's was missing on this host and the pane simply carried no session of its own. And Herdr +> keeps reporting the LAST session announced for a pane, so relaunching a pane's agent as a different +> harness leaves the previous one's ref behind: a pane running `pi` was observed still advertising a +> `herdr:claude` id. The record's own `agent` field is what distinguishes the two — compare it against +> the pane's `agent` before trusting the ref (`bridge/state-engine.ts`). + > **Pane records now carry `scroll`** (new in 0.7.2, live-verified 2026-07-07): `pane.list`, > `pane.get`, `pane.current`, and `session.snapshot` panes all include > `scroll: {offset_from_bottom, max_offset_from_bottom, viewport_rows} | null` (all `uint64`; diff --git a/bridge/config.test.ts b/bridge/config.test.ts index 245afc6..82f73e8 100644 --- a/bridge/config.test.ts +++ b/bridge/config.test.ts @@ -15,6 +15,12 @@ const KEYS = [ "COLLIE_READ_LINES", "COLLIE_TRANSCRIPT", "COLLIE_TRANSCRIPT_ROOT", + "COLLIE_CODEX_ROOT", + "COLLIE_PI_ROOT", + // Each harness's own home var participates in journal-root resolution, so the suite must own them + // too — otherwise a developer with CODEX_HOME set gets different results than CI. + "CODEX_HOME", + "PI_CODING_AGENT_DIR", "COLLIE_SUBMIT_KEYS", "COLLIE_TRUSTED_USER", "COLLIE_DEVICE_HEADER", @@ -60,7 +66,7 @@ describe("loadConfig", () => { expect(cfg.readLines).toBe(200); // Transcript history defaults ON — it's the only scrollback a Claude pane can ever have. expect(cfg.transcript).toBe(true); - expect(cfg.transcriptRoot).toEndWith("/.claude/projects"); + expect(cfg.journalRoots.claude).toEndWith("/.claude/projects"); expect(cfg.submitKeys).toEqual(["Enter"]); expect(cfg.trustedUser).toBe(""); expect(cfg.allowedOrigins).toEqual([]); @@ -127,9 +133,25 @@ describe("loadConfig", () => { expect(loadConfig().transcript).toBe(true); }); - test("COLLIE_TRANSCRIPT_ROOT relocates the transcript root", () => { + // COLLIE_TRANSCRIPT_ROOT predates the per-harness split and meant Claude's root — it keeps meaning + // exactly that, so an existing deployment's env survives the change untouched. + test("COLLIE_TRANSCRIPT_ROOT relocates the CLAUDE journal root", () => { process.env.COLLIE_TRANSCRIPT_ROOT = "/srv/claude/projects"; - expect(loadConfig().transcriptRoot).toBe("/srv/claude/projects"); + expect(loadConfig().journalRoots.claude).toBe("/srv/claude/projects"); + }); + + test("each harness's own home var relocates its journal root", () => { + process.env.CODEX_HOME = "/srv/codex"; + process.env.PI_CODING_AGENT_DIR = "/srv/pi"; + const cfg = loadConfig(); + expect(cfg.journalRoots.codex).toBe("/srv/codex/sessions"); + expect(cfg.journalRoots.pi).toBe("/srv/pi/sessions"); + }); + + test("an explicit COLLIE_* root beats the harness's home var", () => { + process.env.CODEX_HOME = "/srv/codex"; + process.env.COLLIE_CODEX_ROOT = "/elsewhere/rollouts"; + expect(loadConfig().journalRoots.codex).toBe("/elsewhere/rollouts"); }); test("reads the per-device auth header and allowlist", () => { diff --git a/bridge/config.ts b/bridge/config.ts index d4a9b25..21caf0e 100644 --- a/bridge/config.ts +++ b/bridge/config.ts @@ -2,6 +2,7 @@ import { homedir } from "node:os"; import { join } from "node:path"; import type { DialMode } from "./dial.ts"; +import type { JournalRoots } from "./journal/registry.ts"; // All bridge configuration, resolved once at startup. Env-driven so the systemd unit and the // plugin launcher can configure it without code changes. Defaults are safe for a single-user, @@ -108,17 +109,17 @@ export interface Config { readLines: number; /** * Serve agent conversation history from the agent's own on-disk session log. This is the only - * way to get scrollback for a Claude pane at all — Claude runs on the terminal's alternate + * way to get scrollback for most agent panes at all — they run on the terminal's alternate * screen, which has no scrollback ring, so Herdr retains nothing behind the viewport (see - * transcript.ts). Off disables the feature and its route wholesale. + * journal/claude.ts). Off disables the feature and its route wholesale, for every harness. */ transcript: boolean; /** - * Root of the agent's session logs — Claude Code's `~/.claude/projects`. Every transcript read is - * confined to this directory (after symlink resolution). Override only to relocate a non-default - * Claude home; it is never derived from a request. + * Where each harness keeps its session logs. Every read is confined to the matching root after + * symlink resolution, so these double as the security boundary for a feature that touches the + * filesystem — override only to relocate a non-default agent home, never from a request. */ - transcriptRoot: string; + journalRoots: JournalRoots; /** Key sequence sent to submit a reply after the text (agent-dependent; see HERDR_API.md). */ submitKeys: string[]; /** @@ -213,8 +214,19 @@ export function loadConfig(): Config { notifyDelayMs: envInt("COLLIE_NOTIFY_DELAY_MS", 30_000, { min: 0 }), readLines: envInt("COLLIE_READ_LINES", 200, { min: 1 }), transcript: envBool("COLLIE_TRANSCRIPT", true), - transcriptRoot: - process.env.COLLIE_TRANSCRIPT_ROOT ?? join(homedir(), ".claude", "projects"), + journalRoots: { + // COLLIE_TRANSCRIPT_ROOT predates the per-harness split and meant Claude's root, so it keeps + // meaning exactly that — an existing deployment's env keeps working untouched. + claude: process.env.COLLIE_TRANSCRIPT_ROOT ?? join(homedir(), ".claude", "projects"), + // Each harness's own home var is honoured first, so relocating the agent relocates its journal + // without a second Collie setting to keep in sync. + codex: + process.env.COLLIE_CODEX_ROOT ?? + join(process.env.CODEX_HOME ?? join(homedir(), ".codex"), "sessions"), + pi: + process.env.COLLIE_PI_ROOT ?? + join(process.env.PI_CODING_AGENT_DIR ?? join(homedir(), ".pi", "agent"), "sessions"), + }, submitKeys: submitKeys.length ? submitKeys : ["Enter"], trustedUser: process.env.COLLIE_TRUSTED_USER ?? "", deviceHeader: (process.env.COLLIE_DEVICE_HEADER ?? "").trim(), diff --git a/bridge/transcript.test.ts b/bridge/journal/claude.test.ts similarity index 78% rename from bridge/transcript.test.ts rename to bridge/journal/claude.test.ts index 64f29fc..ca52584 100644 Binary files a/bridge/transcript.test.ts and b/bridge/journal/claude.test.ts differ diff --git a/bridge/transcript.ts b/bridge/journal/claude.ts similarity index 57% rename from bridge/transcript.ts rename to bridge/journal/claude.ts index 4dae8a8..67ebae0 100644 --- a/bridge/transcript.ts +++ b/bridge/journal/claude.ts @@ -1,4 +1,4 @@ -// Agent transcript history — the scrollback Herdr structurally cannot give us. +// Claude Code's journal adapter. // // WHY THIS EXISTS. A pane running Claude sits on the terminal's ALTERNATE SCREEN, and the alternate // screen has no scrollback ring — Herdr's terminal core (Ghostty) keeps nothing behind the viewport. @@ -10,10 +10,9 @@ // // The history does exist, though — Claude Code writes every turn to its own session log at // `~/.claude/projects//.jsonl`, and Herdr hands us that uuid on the pane -// record (`agent_session.value`). This module turns that log into a bounded, typed transcript the -// phone can page through. It is strictly BETTER than terminal scrollback would have been: real -// message boundaries, timestamps, tool calls folded together with their results, and it survives the -// pane being closed. +// record (`agent_session.value`, kind `id`). It is strictly BETTER than terminal scrollback would +// have been: real message boundaries, timestamps, tool calls folded together with their results, and +// it survives the pane being closed. // // SHAPE OF THE SOURCE (verified against Claude Code 2.1.220, 2026-07-26): // {"type":"user", "message":{"role":"user","content":"..." | [ {type:"tool_result",...} ]}, ...} @@ -23,102 +22,28 @@ // not something the user typed — we fold those into the tool call that produced them rather than // rendering 705 fake "user" turns. `isSidechain` marks subagent traffic (dropped by default); // `isCompactSummary` marks the summary Claude writes when a session is compacted. -// -// SECURITY. This reads files, which nothing else in the bridge does, so the path is pinned shut: -// - the client never supplies a path — only a pane id, which we map to a session uuid server-side; -// - the uuid must match a strict v4-shaped pattern before it is ever concatenated into a path; -// - the resolved file must still be inside the transcript root after symlink resolution; -// - reads are byte-capped, so a pathological log can't balloon the bridge's memory. -// The transcript is exactly as sensitive as the pane mirror Collie already serves (it is the same -// conversation), but it reaches further back — `COLLIE_TRANSCRIPT=off` disables the feature wholesale. -import { readdir, realpath, stat } from "node:fs/promises"; -import { dirname, join, sep } from "node:path"; +import { readdir, stat } from "node:fs/promises"; +import { dirname, join } from "node:path"; -/** First bytes of a file — enough to find the root entry without reading a multi-megabyte log. */ -async function head(path: string, bytes = 64 * 1024): Promise { - return Bun.file(path).slice(0, bytes).text(); -} +import { containedRealpath, exists, head, loadTail, statFile } from "./files.ts"; +import { clamp, MAX_RESULT_CHARS, MAX_TEXT_CHARS, stripAnsi, summarizeToolInput } from "./text.ts"; +import type { + AgentSessionRef, + JournalAdapter, + TranscriptEntry, + TranscriptPart, + TranscriptSource, +} from "./types.ts"; /** A session uuid as Claude writes it — canonical 8-4-4-4-12 hex. Anything else never touches fs. */ const SESSION_ID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; -/** Most bytes we will ever pull off one transcript. Beyond this we keep the TAIL (newest turns). */ -const MAX_TRANSCRIPT_BYTES = 32 * 1024 * 1024; // 32 MB - -/** Per-tool-result cap. Tool output is unbounded (a 2 MB file read); the phone only needs a gist. */ -const MAX_RESULT_CHARS = 2000; - -/** Per-text-part cap. Generous — assistant prose is the thing you actually came to read. */ -const MAX_TEXT_CHARS = 20_000; - -/** How many parsed transcripts to keep hot. Each is re-parsed only when the file's size/mtime moves. */ -const CACHE_MAX = 4; - -/** One renderable piece of a turn. Deliberately small — the phone renders these as text nodes. */ -export type TranscriptPart = - | { kind: "text"; text: string; truncated?: boolean } - /** - * Extended-thinking text. In practice this NEVER fires: Claude Code writes `thinking` blocks whose - * `thinking` field is an empty string, keeping only the encrypted `signature` (verified 2026-07-26: - * 3944 of 3944 blocks across 40 session logs were empty). The branch stays because the block type is - * real and not ours to control — if a future version persists the text, it renders instead of vanishing. - */ - | { kind: "thinking"; text: string; truncated?: boolean } - /** A tool call. `result` is filled in from the `tool_result` row that answers it, when one exists. */ - | { - kind: "tool"; - name: string; - /** One-line gist of the call's input (the file read, the command run) — never the whole input. */ - summary: string; - result?: { text: string; truncated?: boolean; isError?: boolean }; - }; - -/** - * One turn of the conversation. - * - * `user`/`assistant` are speech. The other two are NOT, and are rendered set apart so they can't be - * mistaken for it: `summary` is Claude's own compaction summary, and `note` is machine-injected - * content that still belongs on screen (a background task finishing, a local command's output). - */ -export interface TranscriptEntry { - /** The source row's uuid — stable, and the paging cursor (`before`). */ - uuid: string; - /** ISO timestamp from the log; empty when the row carried none. */ - ts: string; - role: "user" | "assistant" | "summary" | "note"; - parts: TranscriptPart[]; -} - -/** What the history endpoint answers with. */ -export interface TranscriptPage { - paneId: string; - /** Oldest-first, ready to render top-down. */ - entries: TranscriptEntry[]; - /** True when older turns exist before `entries[0]` — drives "load older". */ - hasMore: boolean; - /** Total turns available in the parsed window (after sidechain filtering). */ - total: number; - /** True when the on-disk log exceeded the byte cap and we kept only its tail. */ - fileTruncated: boolean; -} - -// ── pure parsing ────────────────────────────────────────────────────────────── - -/** Guard used before any path work. Exported so the route can 400 on a malformed id explicitly. */ +/** Guard used before any path work. Exported so tests can pin the shape the fs layer relies on. */ export function isSessionId(value: string): boolean { return SESSION_ID_RE.test(value); } -// CSI/SGR and two-character escapes. Transcript text is NOT a terminal mirror — nothing downstream -// interprets escapes, so a `\x1b[2m` left in place renders as garbage glyphs on the phone. -const ANSI_RE = /\[[0-9;?]*[ -/]*[@-~]|[@-Z\\-_]/g; - -/** Strip terminal escapes from log text. Exported for the parser's tests. */ -export function stripAnsi(text: string): string { - return text.replace(ANSI_RE, ""); -} - /** Inner text of the first `…`, trimmed; null when the tag isn't present. */ function inner(tag: string, text: string): string | null { const m = new RegExp(`<${tag}>([\\s\\S]*?)`).exec(text); @@ -175,39 +100,6 @@ export function classifyUserText( return text.trim() === "" ? null : { role: "user", text }; } -function clamp(text: string, max: number): { text: string; truncated?: boolean } { - if (text.length <= max) return { text }; - return { text: text.slice(0, max), truncated: true }; -} - -/** - * Collapse a tool call's input into one readable line. The well-known tools get their defining - * argument (the path, the command, the pattern); anything else falls back to the first string-ish - * value, so a tool this code has never heard of still reads as something rather than "{...}". - */ -export function summarizeToolInput(input: unknown): string { - if (input === null || typeof input !== "object") return ""; - const o = input as Record; - const pick = (...keys: string[]): string | undefined => { - for (const k of keys) { - const v = o[k]; - if (typeof v === "string" && v.trim() !== "") return v; - } - return undefined; - }; - // Order matters: the MOST defining argument first. Grep carries both `pattern` and `path`, and the - // pattern is what you actually searched for; Task carries both `description` and `prompt`, and the - // description is already the one-line form. - const chosen = - pick("file_path", "command", "pattern", "query", "url", "path", "description", "prompt") ?? - // Unknown tool: first string value wins, so the line is never empty for no reason. - Object.values(o).find((v): v is string => typeof v === "string" && v.trim() !== ""); - if (chosen === undefined) return ""; - // Tool inputs can be multi-line (a Bash heredoc); a summary is one line by definition. - const oneLine = chosen.replace(/\s+/g, " ").trim(); - return oneLine.length > 200 ? `${oneLine.slice(0, 200)}…` : oneLine; -} - /** Flatten a `tool_result.content`, which is either a plain string or a list of text blocks. */ function toolResultText(content: unknown): string { if (typeof content === "string") return content; @@ -239,7 +131,7 @@ interface RawRow { * `includeSidechains` defaults false: subagent traffic is a different conversation and would swamp * the thread you opened. */ -export function parseTranscript( +export function parseClaudeTranscript( text: string, opts: { includeSidechains?: boolean } = {}, ): TranscriptEntry[] { @@ -337,30 +229,6 @@ export function parseTranscript( return entries; } -/** - * Page a parsed transcript, newest-anchored: with no cursor you get the LAST `limit` turns (the phone - * opens at the recent end, like the mirror it replaces); `before` walks backwards from a turn you - * already hold. Returned entries stay oldest-first so the view renders top-down either way. - */ -export function pageEntries( - entries: TranscriptEntry[], - opts: { limit: number; before?: string }, -): { window: TranscriptEntry[]; hasMore: boolean } { - // An unknown cursor (log rewritten under us, or a stale client) degrades to "newest", never to an - // empty page — the user asked for older history and must still see something. - const end = - opts.before === undefined - ? entries.length - : (() => { - const i = entries.findIndex((e) => e.uuid === opts.before); - return i === -1 ? entries.length : i; - })(); - const start = Math.max(0, end - opts.limit); - return { window: entries.slice(start, end), hasMore: start > 0 }; -} - -// ── fs-backed store ─────────────────────────────────────────────────────────── - /** * The uuid of a log's first entry — its conversation ROOT. * @@ -387,14 +255,6 @@ export function conversationRoot(text: string): string | null { return null; } -/** The fs seam. Real implementation below; tests inject a fake so no temp files are needed. */ -export interface TranscriptSource { - /** Absolute path of the log for `sessionId`, or null when it isn't on disk. */ - resolve(sessionId: string): Promise; - /** Tail-read a log. `complete` is false when the byte cap clipped the head. */ - load(path: string): Promise<{ text: string; complete: boolean; size: number; mtimeMs: number }>; -} - /** * Real filesystem source rooted at Claude's projects directory. * @@ -409,8 +269,11 @@ export class ClaudeTranscriptSource implements TranscriptSource { constructor(private readonly root: string) {} - async resolve(sessionId: string): Promise { - if (!isSessionId(sessionId)) return null; // never build a path from an unvalidated id + async resolve(ref: AgentSessionRef): Promise { + // Claude always reports an id. A path-kind ref for this agent is not something we've ever seen, + // and inventing a meaning for it would widen the fs surface for no gain. + if (ref.kind !== "id" || !isSessionId(ref.value)) return null; + const sessionId = ref.value; const cached = this.pathCache.get(sessionId); if (cached !== undefined) { // Re-verify: a cached path can vanish when a session is deleted. @@ -433,12 +296,8 @@ export class ClaudeTranscriptSource implements TranscriptSource { for (const dir of dirs) { const candidate = join(this.root, dir, file); if (!(await exists(candidate))) continue; - // Containment check AFTER symlink resolution: a project dir symlinked outside the root must not - // become a way to read arbitrary files, even though the id itself is already pattern-validated. - const real = await realpath(candidate).catch(() => null); - const realRoot = await realpath(this.root).catch(() => null); - if (real === null || realRoot === null) return null; - if (real !== realRoot && !real.startsWith(realRoot + sep)) return null; + const real = await containedRealpath(candidate, this.root); + if (real === null) return null; this.pathCache.set(sessionId, real); return this.followContinuation(real); } @@ -496,75 +355,16 @@ export class ClaudeTranscriptSource implements TranscriptSource { return best.path; } - async load(path: string): Promise<{ text: string; complete: boolean; size: number; mtimeMs: number }> { - const st = await stat(path); - const size = st.size; - const complete = size <= MAX_TRANSCRIPT_BYTES; - const file = Bun.file(path); - // Over the cap we keep the TAIL — the newest turns are the ones worth having. The clipped first - // line is a partial JSON object; parseTranscript skips it by design. - const text = complete - ? await file.text() - : await file.slice(size - MAX_TRANSCRIPT_BYTES).text(); - return { text, complete, size, mtimeMs: st.mtimeMs }; - } -} + stat = statFile; -async function exists(path: string): Promise { - try { - await stat(path); - return true; - } catch { - return false; - } + load = loadTail; } -interface CacheEntry { - size: number; - mtimeMs: number; - complete: boolean; - entries: TranscriptEntry[]; -} - -/** - * Reads + caches parsed transcripts. History is fetched ON DEMAND (it is not on the 1.5 s poll path), - * so the cost that matters is the repeat visit, not the first — a parse is reused until the log's - * size or mtime moves. - */ -export class TranscriptStore { - private readonly cache = new Map(); - - constructor(private readonly source: TranscriptSource) {} - - /** Null when this session has no log on disk (never started, or a non-Claude agent). */ - async page( - sessionId: string, - opts: { limit: number; before?: string }, - ): Promise | null> { - const path = await this.source.resolve(sessionId); - if (path === null) return null; - - const { text, complete, size, mtimeMs } = await this.source.load(path); - const cached = this.cache.get(path); - let entries: TranscriptEntry[]; - if (cached && cached.size === size && cached.mtimeMs === mtimeMs) { - entries = cached.entries; - } else { - entries = parseTranscript(text); - this.cache.set(path, { size, mtimeMs, complete, entries }); - if (this.cache.size > CACHE_MAX) { - const oldest = this.cache.keys().next().value; - if (oldest !== undefined) this.cache.delete(oldest); - } - } - - const { window, hasMore } = pageEntries(entries, opts); - return { - entries: window, - // A clipped file always has more behind it, even at the window's start. - hasMore: hasMore || (!complete && window.length > 0 && window[0] === entries[0]), - total: entries.length, - fileTruncated: !complete, - }; - } +/** Claude's journal adapter. `agent` matches the Herdr snapshot's `agent` string. */ +export function claudeJournal(root: string): JournalAdapter { + return { + agent: "claude", + source: new ClaudeTranscriptSource(root), + parse: (text) => parseClaudeTranscript(text), + }; } diff --git a/bridge/journal/codex.test.ts b/bridge/journal/codex.test.ts new file mode 100644 index 0000000..2fb6f6b --- /dev/null +++ b/bridge/journal/codex.test.ts @@ -0,0 +1,218 @@ +import { describe, expect, test } from "bun:test"; + +import { codexCursor, codexToolOutput, isCodexSessionId, parseCodexTranscript } from "./codex.ts"; + +// Row builders mirroring the verified on-disk shape (codex rollout logs, cli 0.32.0, 2026-07-29). +// `{timestamp,type,payload}` are the only top-level keys — note the absence of any per-row id, which +// is the whole reason this adapter synthesises a cursor. +const item = (payload: Record, ts = "2026-07-29T10:00:00.000Z") => + JSON.stringify({ timestamp: ts, type: "response_item", payload }); + +const message = (role: "user" | "assistant", text: string) => + item({ + type: "message", + role, + content: [{ type: role === "user" ? "input_text" : "output_text", text }], + }); + +const event = (payload: Record) => + JSON.stringify({ timestamp: "2026-07-29T10:00:00.000Z", type: "event_msg", payload }); + +const meta = () => + JSON.stringify({ + timestamp: "2026-07-29T10:00:00.000Z", + type: "session_meta", + payload: { id: "116ee214-d563-4bcc-95f2-f03c5330d354", cwd: "/repo", cli_version: "0.32.0" }, + }); + +describe("isCodexSessionId", () => { + test.each([ + ["a canonical uuid", "116ee214-d563-4bcc-95f2-f03c5330d354", true], + ["a traversal attempt", "../../../etc/passwd", false], + ["a uuid with a path glued on", "116ee214-d563-4bcc-95f2-f03c5330d354/../x", false], + ["empty", "", false], + ])("%s → %s", (_label, value, expected) => { + expect(isCodexSessionId(value)).toBe(expected); + }); +}); + +describe("parseCodexTranscript", () => { + test("reads a user turn and an assistant turn", () => { + const entries = parseCodexTranscript( + [meta(), message("user", "fix the types"), message("assistant", "I'll open the file.")].join("\n"), + ); + expect(entries.map((e) => e.role)).toEqual(["user", "assistant"]); + expect(entries[0]!.parts).toEqual([{ kind: "text", text: "fix the types" }]); + expect(entries[0]!.ts).toBe("2026-07-29T10:00:00.000Z"); + }); + + // THE trap in this format: every turn is written twice, once per family. Parsing both renders the + // whole conversation double. + test("drops the event_msg family — the conversation is double-booked", () => { + const entries = parseCodexTranscript( + [ + message("user", "fix the types"), + event({ type: "user_message", message: "fix the types" }), + event({ type: "agent_message", message: "on it" }), + event({ type: "token_count", info: {} }), + message("assistant", "on it"), + ].join("\n"), + ); + expect(entries).toHaveLength(2); + expect(entries.map((e) => e.parts[0]).map((p) => (p as { text: string }).text)).toEqual([ + "fix the types", + "on it", + ]); + }); + + test("session_meta and other bookkeeping rows render nothing", () => { + expect(parseCodexTranscript(meta())).toEqual([]); + }); + + // Live-verified against codex 0.145: three `developer` rows carrying injected system prompts + // (permissions, multi-agent instructions) precede the first real turn. Mapping "not assistant" to + // "user" — which the parser used to do — rendered those as things the operator had said. + test.each(["developer", "system", "tool"])("a %s role is plumbing and renders nothing", (role) => { + const entries = parseCodexTranscript( + item({ + type: "message", + role, + content: [{ type: "input_text", text: "…" }], + }), + ); + expect(entries).toEqual([]); + }); + + test.each([ + ["world_state", { type: "world_state", state: {} }], + ["turn_context", { type: "turn_context", cwd: "/repo" }], + ])("the 0.145 row type %s renders nothing", (_label, payload) => { + expect(parseCodexTranscript(item(payload))).toEqual([]); + }); + + test("a reasoning summary becomes a thinking part (unlike Claude, this one has text)", () => { + const entries = parseCodexTranscript( + item({ + type: "reasoning", + summary: [{ type: "summary_text", text: "**Inspecting TypeScript errors**" }], + content: null, + encrypted_content: "gAAAAA…", + }), + ); + expect(entries[0]!.parts).toEqual([ + { kind: "thinking", text: "**Inspecting TypeScript errors**" }, + ]); + }); + + test("an encrypted-only reasoning row renders nothing rather than an empty bubble", () => { + const entries = parseCodexTranscript( + item({ type: "reasoning", summary: [], content: null, encrypted_content: "gAAAAA…" }), + ); + expect(entries).toEqual([]); + }); + + // `arguments` is a JSON STRING here (pi passes an object), and a shell call's `command` is an argv + // ARRAY — both were places a naive reuse of the Claude summariser produced an empty line. + test("a shell call summarises to its joined argv", () => { + const entries = parseCodexTranscript( + item({ + type: "function_call", + name: "shell", + arguments: JSON.stringify({ command: ["bash", "-lc", "ls -la"], timeout_ms: 120000 }), + call_id: "call_1", + }), + ); + expect(entries[0]!.parts[0]).toMatchObject({ + kind: "tool", + name: "shell", + summary: "bash -lc ls -la", + }); + }); + + test("malformed arguments still summarise to something", () => { + const entries = parseCodexTranscript( + item({ type: "function_call", name: "shell", arguments: '{"command": ["bash"', call_id: "c" }), + ); + expect((entries[0]!.parts[0] as { summary: string }).summary).not.toBe(""); + }); + + test("an output folds onto the call that produced it", () => { + const entries = parseCodexTranscript( + [ + item({ type: "function_call", name: "shell", arguments: "{}", call_id: "call_1" }), + item({ + type: "function_call_output", + call_id: "call_1", + output: JSON.stringify({ output: "total 0\n", metadata: {} }), + }), + ].join("\n"), + ); + // One entry, not two: the result attaches to its call rather than becoming its own turn. + expect(entries).toHaveLength(1); + expect(entries[0]!.parts[0]).toMatchObject({ + kind: "tool", + result: { text: "total 0\n" }, + }); + }); + + test("an orphan output is kept unattached so the window never drops output", () => { + const entries = parseCodexTranscript( + item({ type: "function_call_output", call_id: "gone", output: '{"output":"stranded"}' }), + ); + expect(entries[0]!.parts[0]).toMatchObject({ kind: "tool", name: "result" }); + }); + + test("injected environment context is dropped, not rendered as something you said", () => { + const entries = parseCodexTranscript( + [ + message("user", "\n /repo\n"), + message("user", "help me fix the typescript errors"), + ].join("\n"), + ); + expect(entries).toHaveLength(1); + expect(entries[0]!.parts[0]).toMatchObject({ text: "help me fix the typescript errors" }); + }); + + test("a clipped or partial line is skipped, not thrown on", () => { + const entries = parseCodexTranscript( + ['{"timestamp":"2026","type":"response_i', message("user", "hi")].join("\n"), + ); + expect(entries).toHaveLength(1); + }); + + test("every entry gets a cursor, and identical rows still get distinct ones", () => { + const dup = item({ type: "function_call", name: "shell", arguments: "{}", call_id: "c" }); + const entries = parseCodexTranscript([dup, dup].join("\n")); + expect(entries).toHaveLength(2); + expect(entries[0]!.uuid).not.toBe(entries[1]!.uuid); + expect(entries.every((e) => e.uuid !== "")).toBe(true); + }); +}); + +describe("codexToolOutput", () => { + test("unwraps the JSON envelope codex writes", () => { + expect(codexToolOutput('{"output":"hello","metadata":{}}')).toBe("hello"); + }); + + test("a non-JSON output is its own text rather than nothing", () => { + expect(codexToolOutput("plain text")).toBe("plain text"); + }); + + test("JSON without an output field falls back to the raw string", () => { + expect(codexToolOutput('{"other":1}')).toBe('{"other":1}'); + }); +}); + +describe("codexCursor", () => { + test("is deterministic for the same row", () => { + expect(codexCursor("a", new Map())).toBe(codexCursor("a", new Map())); + }); + + test("depends on content, not position — the point of hashing rather than counting", () => { + const seen = new Map(); + codexCursor("filler", seen); + codexCursor("filler", seen); + // "a" is the third row here but the first anywhere else; its cursor must not encode that. + expect(codexCursor("a", seen)).toBe(codexCursor("a", new Map())); + }); +}); diff --git a/bridge/journal/codex.ts b/bridge/journal/codex.ts new file mode 100644 index 0000000..e6756ac --- /dev/null +++ b/bridge/journal/codex.ts @@ -0,0 +1,313 @@ +// Codex's journal adapter. +// +// SHAPE OF THE SOURCE (verified against on-disk rollouts from codex 0.32.0 AND 0.145.0, 2026-07-29 — +// the path layout and the row envelope are identical across that span; 0.145 adds `world_state` and +// `turn_context` row types, and a `developer` message role, all of which this parser ignores): +// ~/.codex/sessions/YYYY/MM/DD/rollout--.jsonl +// {"timestamp":"…","type":"session_meta","payload":{"id":"","cwd":"…","cli_version":"…"}} +// {"timestamp":"…","type":"response_item","payload":{"type":"message"|"reasoning"| +// "function_call"|"function_call_output", …}} +// {"timestamp":"…","type":"event_msg","payload":{"type":"user_message"|"agent_message"| +// "agent_reasoning"|"token_count", …}} +// `{timestamp,type,payload}` are the ONLY top-level keys. +// +// THE TRAP: ROWS ARE DOUBLE-BOOKED. The same conversation is written twice — once as `response_item` +// (the API-shaped record) and once as `event_msg` (the UI event stream). Measured on one session: 29 +// `response_item` user messages against 28 `event_msg` user_messages, and the same for assistant +// turns. Parse both families and every turn renders twice. We take `response_item` and drop +// `event_msg` wholesale, because only `response_item` carries tool RESULTS +// (`function_call_output`) — the event stream has the calls' narration but not their output. +// +// Where Herdr's id comes from: Codex's `SessionStart` hook reports `session_id` to +// `pane.report_agent_session` (herdr integration `codex`, version 6), so the pane record carries a +// kind-`id` ref exactly like Claude's. It needs `herdr integration install codex`; without the hook +// there is no id and the journal correctly reports "no-session". + +import { readdir } from "node:fs/promises"; +import { join } from "node:path"; + +import { containedRealpath, exists, loadTail, statFile } from "./files.ts"; +import { clamp, MAX_RESULT_CHARS, MAX_TEXT_CHARS, oneLine, stripAnsi, summarizeToolInput } from "./text.ts"; +import type { + AgentSessionRef, + JournalAdapter, + TranscriptEntry, + TranscriptPart, + TranscriptSource, +} from "./types.ts"; + +/** Codex names sessions with the same canonical uuid shape Claude does. */ +const SESSION_ID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; + +export function isCodexSessionId(value: string): boolean { + return SESSION_ID_RE.test(value); +} + +/** + * A stable per-row cursor, synthesised because Codex rows carry NO id of their own — and + * `TranscriptEntry.uuid` is the paging cursor (`?before=`). + * + * Row POSITION is the obvious candidate and the wrong one: a log over the byte cap is tail-read, so + * the window's first row is not the file's first row, and the offset moves as the file grows. Hashing + * the row's own bytes instead makes the cursor a property of the CONTENT, so it survives a window + * that starts somewhere else. The occurrence counter disambiguates rows that are byte-identical + * (two identical shell calls); an unknown cursor degrades to "newest" rather than to an empty page, + * so the rare miss is a re-render, never a dead end. + * + * KNOWN FAILURE MODE, accepted. If a >32 MB log's tail window shifts between two requests AND an + * earlier byte-identical row falls out of it, the occurrence counters renumber: a cursor the client + * holds as `cx--2` can then name what used to be `cx--3`. That pages to a slightly wrong + * position rather than degrading to "newest" — the one case where the miss is silent. It needs a + * multi-megabyte log, duplicate rows identical to the byte, and a window shift between two taps; the + * harm is a misplaced page in a view you scroll anyway, so it isn't worth a per-row index the format + * doesn't give us. + * + * INVARIANT this relies on: `seen` is advanced for every `response_item` row, INCLUDING rows that + * emit no entry (a tool output that folds onto its call, an empty reasoning row). Numbering must be a + * function of the parsed window alone — if it depended on which rows happened to render, adding a + * renderable row would renumber the ones before it. + */ +export function codexCursor(line: string, seen: Map): string { + // djb2 — we need determinism and speed, not collision resistance; a collision costs a re-render. + let hash = 5381; + for (let i = 0; i < line.length; i++) hash = ((hash << 5) + hash + line.charCodeAt(i)) | 0; + const key = (hash >>> 0).toString(36); + const n = seen.get(key) ?? 0; + seen.set(key, n + 1); + return n === 0 ? `cx-${key}` : `cx-${key}-${n}`; +} + +/** Flatten a Codex content list (`input_text` / `output_text` / `text` blocks) into plain text. */ +function blockText(content: unknown): string { + if (typeof content === "string") return content; + if (!Array.isArray(content)) return ""; + return content + .map((b) => + b && typeof b === "object" && typeof (b as { text?: unknown }).text === "string" + ? (b as { text: string }).text + : "", + ) + .filter(Boolean) + .join("\n"); +} + +/** + * Unwrap a `function_call_output.output`, which is a JSON STRING wrapping `{"output": "…"}` rather + * than the output itself. Falls back to the raw string when it isn't that shape — a tool whose + * output isn't JSON should still show its output rather than nothing. + */ +export function codexToolOutput(raw: unknown): string { + if (typeof raw !== "string") return ""; + try { + const parsed: unknown = JSON.parse(raw); + if (parsed && typeof parsed === "object" && typeof (parsed as { output?: unknown }).output === "string") { + return (parsed as { output: string }).output; + } + } catch { + // not JSON — the raw string IS the output + } + return raw; +} + +/** `arguments` arrives as a JSON string, not an object — parse before summarising. */ +function codexToolSummary(args: unknown): string { + if (typeof args !== "string") return summarizeToolInput(args); + try { + return summarizeToolInput(JSON.parse(args)); + } catch { + return oneLine(args); // malformed/partial arguments still say something useful + } +} + +/** + * Injected context Codex sends as a user turn. Rendering it as "You" would be actively wrong — the + * operator never typed it — so it is dropped exactly like Claude's `system-reminder`. + */ +function isInjectedContext(text: string): boolean { + return text.trimStart().startsWith(""); +} + +interface CodexRow { + timestamp?: unknown; + type?: unknown; + payload?: unknown; +} + +/** + * Parse a Codex rollout log into oldest-first turns. PURE — no fs, no clock. + * + * Unparseable lines are skipped: the log is appended to live, so the last line can be a partial + * write, and a tail-read window starts mid-line by construction. + */ +export function parseCodexTranscript(text: string): TranscriptEntry[] { + const entries: TranscriptEntry[] = []; + const seen = new Map(); + // call_id → the part awaiting its output, so a `function_call_output` lands on its own call. + const pendingTools = new Map>(); + + for (const line of text.split("\n")) { + if (line.trim() === "") continue; + let row: CodexRow; + try { + row = JSON.parse(line) as CodexRow; + } catch { + continue; + } + // The double-booking guard: everything the UI stream carries is already in `response_item`. + if (row.type !== "response_item") continue; + const payload = row.payload; + if (payload === null || typeof payload !== "object") continue; + const p = payload as Record; + const ts = typeof row.timestamp === "string" ? row.timestamp : ""; + const uuid = codexCursor(line, seen); + + if (p.type === "message") { + // Roles are matched EXPLICITLY, never "assistant or else user". Codex 0.145 writes `developer` + // rows carrying the injected system prompts (permissions, multi-agent instructions) — three of + // them before the first real turn — and treating an unknown role as speech would render those + // as things the operator said. Anything that isn't user or assistant is plumbing: drop it. + if (p.role !== "user" && p.role !== "assistant") continue; + const role = p.role; + const body = stripAnsi(blockText(p.content)); + if (body.trim() === "") continue; + if (role === "user" && isInjectedContext(body)) continue; + entries.push({ uuid, ts, role, parts: [{ kind: "text", ...clamp(body, MAX_TEXT_CHARS) }] }); + continue; + } + + if (p.type === "reasoning") { + // Unlike Claude — whose persisted `thinking` text is empty every time — Codex writes a real + // reasoning summary here, so this branch actually renders. + const summary = Array.isArray(p.summary) + ? p.summary + .map((s) => + s && typeof s === "object" && typeof (s as { text?: unknown }).text === "string" + ? (s as { text: string }).text + : "", + ) + .filter(Boolean) + .join("\n\n") + : ""; + if (summary.trim() === "") continue; // encrypted-only reasoning row — nothing to show + entries.push({ + uuid, + ts, + role: "assistant", + parts: [{ kind: "thinking", ...clamp(stripAnsi(summary), MAX_TEXT_CHARS) }], + }); + continue; + } + + if (p.type === "function_call") { + const part: Extract = { + kind: "tool", + name: typeof p.name === "string" ? p.name : "tool", + summary: codexToolSummary(p.arguments), + }; + if (typeof p.call_id === "string") pendingTools.set(p.call_id, part); + entries.push({ uuid, ts, role: "assistant", parts: [part] }); + continue; + } + + if (p.type === "function_call_output") { + const id = typeof p.call_id === "string" ? p.call_id : ""; + const target = pendingTools.get(id); + const outputText = stripAnsi(codexToolOutput(p.output)); + if (target) { + // Mutated in place — the part already sits in an emitted entry, which is exactly why results + // attach without reordering anything. + pendingTools.delete(id); + target.result = clamp(outputText, MAX_RESULT_CHARS); + } else if (outputText.trim() !== "") { + // Orphan output (its call fell outside a tail-read window) — kept unattached so the window + // never silently drops output. + entries.push({ + uuid, + ts, + role: "assistant", + parts: [ + { kind: "tool", name: "result", summary: "", result: clamp(outputText, MAX_RESULT_CHARS) }, + ], + }); + } + } + } + + return entries; +} + +/** + * Real filesystem source rooted at Codex's `sessions` directory. + * + * Resolution is a targeted walk rather than Claude's flat scan, because the uuid is in the FILENAME + * under date-partitioned directories (`YYYY/MM/DD/rollout--.jsonl`). We walk newest-date + * first, so a live session is found after reading a handful of directory entries rather than the + * whole year. The hit is cached; a cached path is re-verified before use, since a session can be + * deleted while the bridge is up. + * + * No continuation-following, deliberately: Codex reports its session on the `SessionStart` hook, so a + * resumed conversation re-reports its NEW id and the pane record follows it. That's the failure + * Claude's followContinuation exists to paper over, and Codex's hook simply doesn't have it. + */ +export class CodexTranscriptSource implements TranscriptSource { + private readonly pathCache = new Map(); + + constructor(private readonly root: string) {} + + async resolve(ref: AgentSessionRef): Promise { + if (ref.kind !== "id" || !isCodexSessionId(ref.value)) return null; + const sessionId = ref.value; + const cached = this.pathCache.get(sessionId); + if (cached !== undefined) { + if (await exists(cached)) return cached; + this.pathCache.delete(sessionId); + } + + const suffix = `-${sessionId.toLowerCase()}.jsonl`; + // Newest first at every level: a session being read is almost always today's. + for (const year of await descending(this.root)) { + for (const month of await descending(join(this.root, year))) { + for (const day of await descending(join(this.root, year, month))) { + const dir = join(this.root, year, month, day); + let names: string[]; + try { + names = await readdir(dir); + } catch { + continue; + } + const hit = names.find( + (n) => n.startsWith("rollout-") && n.toLowerCase().endsWith(suffix), + ); + if (hit === undefined) continue; + const real = await containedRealpath(join(dir, hit), this.root); + if (real === null) return null; + this.pathCache.set(sessionId, real); + return real; + } + } + } + return null; + } + + stat = statFile; + + load = loadTail; +} + +/** Directory entries, newest-name first. Empty when the directory doesn't exist. */ +async function descending(dir: string): Promise { + try { + return (await readdir(dir)).sort().reverse(); + } catch { + return []; + } +} + +/** Codex's journal adapter. `agent` matches the Herdr snapshot's `agent` string. */ +export function codexJournal(root: string): JournalAdapter { + return { + agent: "codex", + source: new CodexTranscriptSource(root), + parse: parseCodexTranscript, + }; +} diff --git a/bridge/journal/files.ts b/bridge/journal/files.ts new file mode 100644 index 0000000..1136739 --- /dev/null +++ b/bridge/journal/files.ts @@ -0,0 +1,77 @@ +// The filesystem half of the journal, shared by every adapter. +// +// SECURITY. Reading session logs is the only thing in the bridge that touches the filesystem, so the +// path is pinned shut here rather than re-argued per harness: +// - the client never supplies a path — only a pane id, which the route maps to a session ref; +// - an `id` ref is pattern-validated by its adapter before it is ever concatenated into a path; +// - a `path` ref (pi reports one) is attacker-shaped by construction — it arrives over the socket +// from a process we don't control — so it is confined to the harness's own root the same way; +// - EVERY resolved path is re-checked for containment AFTER symlink resolution, so a log or project +// directory symlinked out of the root cannot become a way to read arbitrary files; +// - reads are byte-capped, so a pathological log can't balloon the bridge's memory. +// A journal is exactly as sensitive as the pane mirror Collie already serves (it is the same +// conversation), but it reaches further back — `COLLIE_TRANSCRIPT=off` disables the feature wholesale. + +import { realpath, stat } from "node:fs/promises"; +import { sep } from "node:path"; + +/** Most bytes we will ever pull off one log. Beyond this we keep the TAIL (newest turns). */ +export const MAX_TRANSCRIPT_BYTES = 32 * 1024 * 1024; // 32 MB + +/** True when the path exists at all. Cheap pre-check before the more expensive realpath work. */ +export async function exists(path: string): Promise { + try { + await stat(path); + return true; + } catch { + return false; + } +} + +/** + * Resolve `candidate` and return it only if it is still inside `root` afterwards. + * + * The check runs on the REAL paths of both sides, which is the whole point: comparing the strings we + * were handed would be satisfied by a symlink pointing anywhere. Null means "not ours to read" — + * callers treat that identically to "no log", so a containment failure is never distinguishable from + * an absent file by anything the client can see. + */ +export async function containedRealpath(candidate: string, root: string): Promise { + const real = await realpath(candidate).catch(() => null); + const realRoot = await realpath(root).catch(() => null); + if (real === null || realRoot === null) return null; + return real === realRoot || real.startsWith(realRoot + sep) ? real : null; +} + +/** Size + mtime, or null when the file is gone. The store's cache-validity probe (see types.ts). */ +export async function statFile(path: string): Promise<{ size: number; mtimeMs: number } | null> { + try { + const st = await stat(path); + return { size: st.size, mtimeMs: st.mtimeMs }; + } catch { + return null; + } +} + +/** First bytes of a file — enough to identify a log without reading a multi-megabyte one. */ +export async function head(path: string, bytes = 64 * 1024): Promise { + return Bun.file(path).slice(0, bytes).text(); +} + +/** + * Tail-read a log under the byte cap. Shared by every adapter's `load` — the cap and the "keep the + * newest end" policy are properties of the journal, not of any one harness. + * + * Over the cap the clipped first line is a partial JSON object; every parser skips unparseable lines + * by design, so the window simply starts one turn later. + */ +export async function loadTail( + path: string, +): Promise<{ text: string; complete: boolean; size: number; mtimeMs: number }> { + const st = await stat(path); + const size = st.size; + const complete = size <= MAX_TRANSCRIPT_BYTES; + const file = Bun.file(path); + const text = complete ? await file.text() : await file.slice(size - MAX_TRANSCRIPT_BYTES).text(); + return { text, complete, size, mtimeMs: st.mtimeMs }; +} diff --git a/bridge/journal/pi.test.ts b/bridge/journal/pi.test.ts new file mode 100644 index 0000000..189d656 --- /dev/null +++ b/bridge/journal/pi.test.ts @@ -0,0 +1,215 @@ +import { describe, expect, test } from "bun:test"; +import { mkdir, rm, symlink } from "node:fs/promises"; +import { tmpdir } from "node:os"; + +import { isPiSessionId, parsePiTranscript, PiTranscriptSource } from "./pi.ts"; + +// Row builders mirroring the verified on-disk shape (pi session logs, session format v3, 2026-07-29). +// Every row carries its own `id`, so unlike Codex there is nothing to synthesise for paging. +const row = (id: string, message: Record) => + JSON.stringify({ + type: "message", + id, + parentId: "p0", + timestamp: "2026-07-29T10:00:00.000Z", + message, + }); + +const speech = (id: string, role: "user" | "assistant", text: string) => + row(id, { role, content: [{ type: "text", text }] }); + +const header = () => + JSON.stringify({ + type: "session", + version: 3, + id: "019f1827-bf99-7927-9684-76318de905b5", + timestamp: "2026-07-29T10:00:00.000Z", + cwd: "/repo", + }); + +describe("isPiSessionId", () => { + test.each([ + ["a v4 uuid", "715d7796-b4de-4f46-a11c-fbbdd8ca965b", true], + ["a v7 uuid", "019f4665-7df0-7540-a64f-7068335f21af", true], + ["a traversal attempt", "../../secrets", false], + ])("%s → %s", (_label, value, expected) => { + expect(isPiSessionId(value)).toBe(expected); + }); +}); + +describe("parsePiTranscript", () => { + test("reads speech turns and ignores the session header", () => { + const entries = parsePiTranscript( + [header(), speech("a", "user", "go ahead"), speech("b", "assistant", "on it")].join("\n"), + ); + expect(entries.map((e) => [e.uuid, e.role])).toEqual([ + ["a", "user"], + ["b", "assistant"], + ]); + }); + + test.each([ + ["model_change", { type: "model_change", id: "m", modelId: "x", provider: "y" }], + ["thinking_level_change", { type: "thinking_level_change", id: "t", thinkingLevel: "high" }], + ])("%s is bookkeeping and renders nothing", (_label, r) => { + expect(parsePiTranscript(JSON.stringify(r))).toEqual([]); + }); + + test("a thinking block renders — pi persists real reasoning text", () => { + const entries = parsePiTranscript( + row("a", { + role: "assistant", + content: [ + { type: "thinking", thinking: "**Identifying single subagent call**", thinkingSignature: "{}" }, + ], + }), + ); + expect(entries[0]!.parts).toEqual([ + { kind: "thinking", text: "**Identifying single subagent call**" }, + ]); + }); + + test("a toolCall summarises from its arguments OBJECT (no JSON string, unlike codex)", () => { + const entries = parsePiTranscript( + row("a", { + role: "assistant", + content: [ + { type: "toolCall", id: "call_1", name: "read", arguments: { path: "/repo/SKILL.md" } }, + ], + }), + ); + expect(entries[0]!.parts[0]).toMatchObject({ + kind: "tool", + name: "read", + summary: "/repo/SKILL.md", + }); + }); + + // pi puts a tool result in its OWN row (Claude nests it inside a user turn), linked by toolCallId. + test("a toolResult row folds onto the call that produced it", () => { + const entries = parsePiTranscript( + [ + row("a", { + role: "assistant", + content: [{ type: "toolCall", id: "call_1", name: "read", arguments: { path: "/x" } }], + }), + row("b", { + role: "toolResult", + toolCallId: "call_1", + toolName: "read", + content: [{ type: "text", text: "file contents" }], + }), + ].join("\n"), + ); + expect(entries).toHaveLength(1); + expect(entries[0]!.parts[0]).toMatchObject({ kind: "tool", result: { text: "file contents" } }); + }); + + test("an errored toolResult keeps its error flag", () => { + const entries = parsePiTranscript( + [ + row("a", { + role: "assistant", + content: [{ type: "toolCall", id: "c", name: "read", arguments: {} }], + }), + row("b", { + role: "toolResult", + toolCallId: "c", + toolName: "read", + isError: true, + content: [{ type: "text", text: "ENOENT" }], + }), + ].join("\n"), + ); + expect(entries[0]!.parts[0]).toMatchObject({ result: { text: "ENOENT", isError: true } }); + }); + + test("an orphan toolResult is kept unattached so the window never drops output", () => { + const entries = parsePiTranscript( + row("b", { + role: "toolResult", + toolCallId: "gone", + toolName: "read", + content: [{ type: "text", text: "stranded" }], + }), + ); + expect(entries[0]!.parts[0]).toMatchObject({ kind: "tool", name: "read", result: { text: "stranded" } }); + }); + + test("a clipped or partial line is skipped, not thrown on", () => { + expect(parsePiTranscript(['{"type":"mess', speech("a", "user", "hi")].join("\n"))).toHaveLength(1); + }); +}); + +// pi is the harness that reports a kind-`path` ref: its herdr integration prefers +// `agent_session_path` (an absolute path chosen by a process we don't control) over an id. That path +// is treated as hostile input, so containment is the security boundary and it needs real files. +describe("PiTranscriptSource — path refs are confined to the root", () => { + const SID = "019f4665-7df0-7540-a64f-7068335f21af"; + + /** + * Everything lives under one `base` so cleanup takes the "outside" file with it: + * base/sessions/--repo--/_.jsonl the real log + * base/outside.jsonl a file the root must never reach + * base/sessions/--repo--/sneaky.jsonl → ../../outside.jsonl a symlink out of the root + */ + async function fixture() { + const base = `${tmpdir()}/collie-pi-${Math.floor(performance.now() * 1000)}`; + const root = `${base}/sessions`; + const project = `${root}/--var-home-you-repo--`; + await mkdir(project, { recursive: true }); + const log = `${project}/2026-07-29T10-00-00-000Z_${SID}.jsonl`; + await Bun.write(log, speech("a", "user", "hi")); + const outside = `${base}/outside.jsonl`; + await Bun.write(outside, speech("z", "user", "secrets")); + const sneaky = `${project}/2026-07-29T11-00-00-000Z_${OUTSIDE_SID}.jsonl`; + await symlink(outside, sneaky); + return { base, root, log, sneaky }; + } + + const OUTSIDE_SID = "ffffffff-1111-2222-3333-444444444444"; + + test("resolves a path ref that really is inside the root", async () => { + const { base, root, log } = await fixture(); + expect(await new PiTranscriptSource(root).resolve({ kind: "path", value: log })).toBe(log); + await rm(base, { recursive: true, force: true }); + }); + + test("refuses a path ref pointing outside the root", async () => { + const { base, root, log } = await fixture(); + const escape = `${log}/../../../../etc/hosts`; + expect(await new PiTranscriptSource(root).resolve({ kind: "path", value: escape })).toBeNull(); + await rm(base, { recursive: true, force: true }); + }); + + // Symlink resolution is the ENTIRE reason containment runs on realpaths rather than on the strings + // we were handed: `..` traversal would be caught by plain normalisation, this would not. The file + // sits inside the root, has a plausible session filename, and still must not be readable. + test("refuses a symlink inside the root that points outside it", async () => { + const { base, root, sneaky } = await fixture(); + const src = new PiTranscriptSource(root); + expect(await src.resolve({ kind: "path", value: sneaky })).toBeNull(); + // …and the id fallback must not be a way around the same check. + expect(await src.resolve({ kind: "id", value: OUTSIDE_SID })).toBeNull(); + await rm(base, { recursive: true, force: true }); + }); + + test("refuses a path ref that isn't a session log at all", async () => { + const { base, root } = await fixture(); + expect(await new PiTranscriptSource(root).resolve({ kind: "path", value: "/etc/passwd" })).toBeNull(); + await rm(base, { recursive: true, force: true }); + }); + + test("resolves the id fallback by scanning the per-cwd directories", async () => { + const { base, root, log } = await fixture(); + expect(await new PiTranscriptSource(root).resolve({ kind: "id", value: SID })).toBe(log); + await rm(base, { recursive: true, force: true }); + }); + + test("an unknown id resolves to null rather than guessing", async () => { + const { base, root } = await fixture(); + const src = new PiTranscriptSource(root); + expect(await src.resolve({ kind: "id", value: "ffffffff-ffff-ffff-ffff-ffffffffffff" })).toBeNull(); + await rm(base, { recursive: true, force: true }); + }); +}); diff --git a/bridge/journal/pi.ts b/bridge/journal/pi.ts new file mode 100644 index 0000000..2b5e824 --- /dev/null +++ b/bridge/journal/pi.ts @@ -0,0 +1,221 @@ +// pi's journal adapter. +// +// SHAPE OF THE SOURCE (verified against on-disk sessions, 2026-07-29): +// ~/.pi/agent/sessions/----/_.jsonl +// {"type":"session","version":3,"id":"","timestamp":"…","cwd":"…"} ← header, first row +// {"type":"message","id":"…","parentId":"…","timestamp":"…","message":{ … }} +// {"type":"model_change" | "thinking_level_change", …} ← bookkeeping +// Every row carries its OWN `id`, so unlike Codex there is nothing to synthesise for paging. +// +// `message.role` is one of `user` | `assistant` | `toolResult`. The first two carry a `content` list +// of `text` / `thinking` / `toolCall` blocks; a `toolResult` row is its own row (not a block inside a +// user turn, the way Claude does it) and links back by `toolCallId`. pi's `thinking` blocks carry +// REAL text — usually a short bolded title — so the thinking branch renders here. +// +// HOW HERDR NAMES THE SESSION — the one that's different. pi's integration reports +// `agent_session_path` in preference to `agent_session_id` (herdr integration `pi`, version 6: +// `withSessionRef` returns the path whenever `sessionManager.getSessionFile()` gave one). So a pi +// pane arrives as a kind-`path` ref: an ABSOLUTE PATH chosen by a process we don't control. It is +// treated as hostile input and confined to pi's own sessions root like everything else — see +// journal/files.ts. The id fallback is supported too, since the hook uses it when no file is open yet. + +import { readdir } from "node:fs/promises"; +import { join } from "node:path"; + +import { containedRealpath, exists, loadTail, statFile } from "./files.ts"; +import { clamp, MAX_RESULT_CHARS, MAX_TEXT_CHARS, stripAnsi, summarizeToolInput } from "./text.ts"; +import type { + AgentSessionRef, + JournalAdapter, + TranscriptEntry, + TranscriptPart, + TranscriptSource, +} from "./types.ts"; + +/** pi's session ids are uuids (v4 and v7 both observed) — validated before any path work. */ +const SESSION_ID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; + +export function isPiSessionId(value: string): boolean { + return SESSION_ID_RE.test(value); +} + +/** Flatten a pi content list into text, keeping only `text` blocks. */ +function textBlocks(content: unknown): string { + if (typeof content === "string") return content; + if (!Array.isArray(content)) return ""; + return content + .map((b) => + b && typeof b === "object" && (b as { type?: unknown }).type === "text" && + typeof (b as { text?: unknown }).text === "string" + ? (b as { text: string }).text + : "", + ) + .filter(Boolean) + .join("\n"); +} + +interface PiRow { + type?: unknown; + id?: unknown; + timestamp?: unknown; + message?: unknown; +} + +/** + * Parse a pi session log into oldest-first turns. PURE — no fs, no clock. + * + * Unparseable lines are skipped: the log is appended to live, so the last line can be a partial + * write, and a tail-read window starts mid-line by construction. + */ +export function parsePiTranscript(text: string): TranscriptEntry[] { + const entries: TranscriptEntry[] = []; + // toolCall id → the part awaiting its result, so a later `toolResult` row lands on its own call. + const pendingTools = new Map>(); + + for (const line of text.split("\n")) { + if (line.trim() === "") continue; + let row: PiRow; + try { + row = JSON.parse(line) as PiRow; + } catch { + continue; + } + // `session`, `model_change`, `thinking_level_change` are bookkeeping — nothing to render. + if (row.type !== "message") continue; + const message = row.message; + if (message === null || typeof message !== "object") continue; + const m = message as Record; + const uuid = typeof row.id === "string" ? row.id : ""; + const ts = typeof row.timestamp === "string" ? row.timestamp : ""; + + if (m.role === "toolResult") { + const id = typeof m.toolCallId === "string" ? m.toolCallId : ""; + const target = pendingTools.get(id); + const resultText = stripAnsi(textBlocks(m.content)); + const isError = m.isError === true; + if (target) { + // Mutated in place — the part already sits in an emitted entry, which is why results attach + // without reordering anything. + pendingTools.delete(id); + target.result = { + ...clamp(resultText, MAX_RESULT_CHARS), + ...(isError ? { isError: true } : {}), + }; + } else if (resultText.trim() !== "") { + // Orphan result (its call fell outside a tail-read window) — kept unattached so the window + // never silently drops output. + entries.push({ + uuid, + ts, + role: "assistant", + parts: [ + { + kind: "tool", + name: typeof m.toolName === "string" ? m.toolName : "result", + summary: "", + result: { ...clamp(resultText, MAX_RESULT_CHARS), ...(isError ? { isError: true } : {}) }, + }, + ], + }); + } + continue; + } + + const role: TranscriptEntry["role"] = m.role === "assistant" ? "assistant" : "user"; + const parts: TranscriptPart[] = []; + const content = Array.isArray(m.content) ? m.content : []; + for (const block of content) { + if (block === null || typeof block !== "object") continue; + const b = block as Record; + if (b.type === "text" && typeof b.text === "string") { + if (b.text.trim() !== "") + parts.push({ kind: "text", ...clamp(stripAnsi(b.text), MAX_TEXT_CHARS) }); + } else if (b.type === "thinking" && typeof b.thinking === "string") { + if (b.thinking.trim() !== "") + parts.push({ kind: "thinking", ...clamp(stripAnsi(b.thinking), MAX_TEXT_CHARS) }); + } else if (b.type === "toolCall") { + const part: Extract = { + kind: "tool", + name: typeof b.name === "string" ? b.name : "tool", + // pi passes `arguments` as a real object (Codex passes a JSON string) — no parse needed. + summary: summarizeToolInput(b.arguments), + }; + if (typeof b.id === "string") pendingTools.set(b.id, part); + parts.push(part); + } + } + + if (parts.length === 0) continue; // a row with nothing renderable + entries.push({ uuid, ts, role, parts }); + } + + return entries; +} + +/** + * Real filesystem source rooted at pi's `sessions` directory. + * + * Two ref kinds, because pi's hook reports whichever it has: + * - `path` — the common case. Confined to the root after symlink resolution, so a path pointing + * anywhere else resolves to null and reads to the client as an ordinary "no log". + * - `id` — the fallback. The uuid is the filename SUFFIX (`_.jsonl`) inside a + * per-cwd directory, so this is a scan of the project dirs, cached after the first hit. + */ +export class PiTranscriptSource implements TranscriptSource { + private readonly pathCache = new Map(); + + constructor(private readonly root: string) {} + + async resolve(ref: AgentSessionRef): Promise { + if (ref.kind === "path") { + // No shape validation is possible on a free-form path — containment IS the validation. + if (!ref.value.endsWith(".jsonl")) return null; + if (!(await exists(ref.value))) return null; + return containedRealpath(ref.value, this.root); + } + + if (!isPiSessionId(ref.value)) return null; + const sessionId = ref.value; + const cached = this.pathCache.get(sessionId); + if (cached !== undefined) { + if (await exists(cached)) return cached; + this.pathCache.delete(sessionId); + } + + const suffix = `_${sessionId.toLowerCase()}.jsonl`; + let dirs: string[]; + try { + dirs = await readdir(this.root); + } catch { + return null; // no sessions root at all — nothing to serve + } + for (const dir of dirs) { + let names: string[]; + try { + names = await readdir(join(this.root, dir)); + } catch { + continue; + } + const hit = names.find((n) => n.toLowerCase().endsWith(suffix)); + if (hit === undefined) continue; + const real = await containedRealpath(join(this.root, dir, hit), this.root); + if (real === null) return null; + this.pathCache.set(sessionId, real); + return real; + } + return null; + } + + stat = statFile; + + load = loadTail; +} + +/** pi's journal adapter. `agent` matches the Herdr snapshot's `agent` string. */ +export function piJournal(root: string): JournalAdapter { + return { + agent: "pi", + source: new PiTranscriptSource(root), + parse: parsePiTranscript, + }; +} diff --git a/bridge/journal/registry.test.ts b/bridge/journal/registry.test.ts new file mode 100644 index 0000000..b2c45b9 --- /dev/null +++ b/bridge/journal/registry.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, test } from "bun:test"; + +import { adapterFor, buildJournalRegistry, journalAgents } from "./registry.ts"; + +// The registry is the SINGLE decision site for "which agents have a journal". These tests pin the +// two properties that keep it from rotting: keys come from the adapters themselves, and a hostile +// agent name can't resolve to something that isn't an adapter. + +const roots = { claude: "/c", codex: "/x", pi: "/p" }; + +describe("buildJournalRegistry", () => { + test("serves the three verified harnesses", () => { + expect(journalAgents(buildJournalRegistry(roots))).toEqual(["claude", "codex", "pi"]); + }); + + test("every key IS its adapter's own agent string — the map can't drift from the adapters", () => { + const registry = buildJournalRegistry(roots); + for (const [key, adapter] of Object.entries(registry)) expect(adapter.agent).toBe(key); + }); +}); + +describe("adapterFor", () => { + const registry = buildJournalRegistry(roots); + + test.each(["claude", "codex", "pi"])("resolves %s", (agent) => { + expect(adapterFor(registry, agent)?.agent).toBe(agent); + }); + + test("an agent with no journal is undefined, not a throw", () => { + expect(adapterFor(registry, "opencode")).toBeUndefined(); + expect(adapterFor(registry, undefined)).toBeUndefined(); + }); + + // The agent string comes from Herdr, but it ORIGINATES in an agent's own report — so an inherited + // Object.prototype key must not resolve to a function masquerading as an adapter. + test.each(["toString", "constructor", "__proto__", "hasOwnProperty"])( + "%s does not resolve to a non-adapter", + (key) => { + expect(adapterFor(registry, key)).toBeUndefined(); + }, + ); +}); diff --git a/bridge/journal/registry.ts b/bridge/journal/registry.ts new file mode 100644 index 0000000..c940c46 --- /dev/null +++ b/bridge/journal/registry.ts @@ -0,0 +1,56 @@ +// The journal registry — the SINGLE decision site for "which agents have a readable history". +// +// Maps a Herdr snapshot `agent` string to its JournalAdapter; anything absent from the map has no +// journal, which the history route reports as an ordinary `no-session` rather than an error. Adding a +// harness is a one-line change to the list below plus its adapter module — never a new branch in the +// route or the store. +// +// This deliberately mirrors `web/src/lib/harness/registry.ts`, but the two are NOT the same seam and +// must not be conflated: the frontend harness registry owns block grammars and the send guard for the +// LIVE MIRROR; this one owns reading an on-disk log. A harness can plausibly have one without the +// other. + +import { claudeJournal } from "./claude.ts"; +import { codexJournal } from "./codex.ts"; +import { piJournal } from "./pi.ts"; +import type { JournalAdapter } from "./types.ts"; + +/** Where each harness keeps its logs. Every path is a containment root, never a request input. */ +export interface JournalRoots { + /** Claude Code's `~/.claude/projects`. */ + claude: string; + /** Codex's `$CODEX_HOME/sessions`. */ + codex: string; + /** pi's `$PI_CODING_AGENT_DIR/sessions`. */ + pi: string; +} + +/** + * Build the registry for a set of roots. + * + * The map is built FROM each adapter's own `agent` field (not a hand-written literal), so a key can + * never drift from the adapter it points at — the same guarantee the frontend registry gives. + */ +export function buildJournalRegistry(roots: JournalRoots): Record { + const adapters = [claudeJournal(roots.claude), codexJournal(roots.codex), piJournal(roots.pi)]; + return Object.fromEntries(adapters.map((a) => [a.agent, a])); +} + +/** + * The adapter for `agent`, or undefined when the agent has no journal. + * + * `Object.hasOwn` rather than a truthy lookup, so an inherited Object.prototype key ("toString", + * "constructor", "__proto__", …) arriving as an agent name can't resolve to a non-adapter and crash + * the read path. The agent string comes from Herdr, but it originates in an agent's own report. + */ +export function adapterFor( + registry: Record, + agent: string | undefined, +): JournalAdapter | undefined { + return agent !== undefined && Object.hasOwn(registry, agent) ? registry[agent] : undefined; +} + +/** The agents this build can serve a journal for — used by the probe script and by tests. */ +export function journalAgents(registry: Record): string[] { + return Object.keys(registry).sort(); +} diff --git a/bridge/journal/store.test.ts b/bridge/journal/store.test.ts new file mode 100644 index 0000000..c4cdaab --- /dev/null +++ b/bridge/journal/store.test.ts @@ -0,0 +1,155 @@ +import { describe, expect, test } from "bun:test"; + +import { pageEntries, TranscriptStore } from "./store.ts"; +import type { JournalAdapter, TranscriptEntry, TranscriptSource } from "./types.ts"; + +// The store is harness-BLIND: it resolves through whatever adapter it's handed, caches by absolute +// path, and pages. So it's tested with a fake adapter rather than any real grammar — if a test here +// needs to know what Claude writes, the seam has leaked. + +const entry = (uuid: string): TranscriptEntry => ({ + uuid, + ts: "", + role: "user", + parts: [{ kind: "text", text: uuid }], +}); + +/** Fake adapter over one in-memory log, counting stat/load so caching is observable. */ +function fakeAdapter(lines: string[], opts: { complete?: boolean } = {}) { + let text = lines.join("\n"); + let mtimeMs = 1000; + const calls = { resolve: 0, stat: 0, load: 0, parse: 0 }; + const source: TranscriptSource = { + async resolve(ref) { + calls.resolve++; + return ref.kind === "id" && ref.value !== "unknown" ? `/fake/${ref.value}.jsonl` : null; + }, + async stat() { + calls.stat++; + return { size: text.length, mtimeMs }; + }, + async load() { + calls.load++; + return { text, complete: opts.complete ?? true, size: text.length, mtimeMs }; + }, + }; + const adapter: JournalAdapter = { + agent: "fake", + source, + parse: (t) => { + calls.parse++; + return t.split("\n").filter(Boolean).map(entry); + }, + }; + return { + adapter, + calls, + /** Simulate the agent appending a turn — new size AND mtime, as a real write would give. */ + append: (uuid: string) => { + text = `${text}\n${uuid}`; + mtimeMs += 1; + }, + }; +} + +const REF = { kind: "id", value: "s1" } as const; + +describe("TranscriptStore", () => { + test("pages the newest turns and reports the total", async () => { + const { adapter } = fakeAdapter(["u1", "u2", "u3"]); + const page = await new TranscriptStore().page(adapter, REF, { limit: 2 }); + expect(page).not.toBeNull(); + expect(page!.entries.map((e) => e.uuid)).toEqual(["u2", "u3"]); + expect(page!.hasMore).toBe(true); + expect(page!.total).toBe(3); + expect(page!.fileTruncated).toBe(false); + }); + + test("an unresolvable ref is null, not an error", async () => { + const { adapter } = fakeAdapter(["u1"]); + const page = await new TranscriptStore().page(adapter, { kind: "id", value: "unknown" }, { + limit: 10, + }); + expect(page).toBeNull(); + }); + + // The regression this guards: the store used to read the file BEFORE consulting its cache, so + // every "load older" tap on a multi-megabyte journal paid a full re-read to discover it already + // had the parse. Validity is a stat; only a moved size/mtime may cost a read. + test("a repeat read stats but does not re-read or re-parse", async () => { + const { adapter, calls } = fakeAdapter(["u1", "u2", "u3"]); + const store = new TranscriptStore(); + await store.page(adapter, REF, { limit: 10 }); + await store.page(adapter, REF, { limit: 10 }); + expect(calls.load).toBe(1); + expect(calls.parse).toBe(1); + expect(calls.stat).toBe(2); + }); + + test("a moved file is re-read and re-parsed", async () => { + const { adapter, calls, append } = fakeAdapter(["u1", "u2"]); + const store = new TranscriptStore(); + await store.page(adapter, REF, { limit: 10 }); + append("u3"); + const after = await store.page(adapter, REF, { limit: 10 }); + expect(calls.load).toBe(2); + expect(after!.total).toBe(3); + expect(after!.entries.map((e) => e.uuid)).toEqual(["u1", "u2", "u3"]); + }); + + test("a log that vanishes between resolve and read is null, not a throw", async () => { + const { adapter } = fakeAdapter(["u1"]); + const gone: JournalAdapter = { + ...adapter, + source: { ...adapter.source, stat: async () => null }, + }; + expect(await new TranscriptStore().page(gone, REF, { limit: 10 })).toBeNull(); + }); + + test("a byte-capped file reports fileTruncated and keeps hasMore at its head", async () => { + const { adapter } = fakeAdapter(["u1", "u2"], { complete: false }); + const page = await new TranscriptStore().page(adapter, REF, { limit: 10 }); + expect(page!.fileTruncated).toBe(true); + // The window covers every parsed entry, but the log itself was clipped — there IS more behind it. + expect(page!.hasMore).toBe(true); + }); +}); + +describe("pageEntries", () => { + const entries = ["e1", "e2", "e3", "e4", "e5"].map(entry); + + test("with no cursor, anchors at the NEWEST turns", () => { + const { window, hasMore } = pageEntries(entries, { limit: 2 }); + expect(window.map((e) => e.uuid)).toEqual(["e4", "e5"]); + expect(hasMore).toBe(true); + }); + + test("a limit past the start yields everything and stops offering more", () => { + expect(pageEntries(entries, { limit: 10 }).window.map((e) => e.uuid)).toEqual([ + "e1", + "e2", + "e3", + "e4", + "e5", + ]); + expect(pageEntries(entries, { limit: 10 }).hasMore).toBe(false); + }); + + test("a cursor walks backwards from the turn the client already holds", () => { + const { window, hasMore } = pageEntries(entries, { limit: 2, before: "e4" }); + expect(window.map((e) => e.uuid)).toEqual(["e2", "e3"]); + expect(hasMore).toBe(true); + }); + + // An unknown cursor is expected in normal operation — a rewritten log, a stale client, or a + // synthesised Codex cursor whose row fell out of the window. Degrading to "newest" re-renders; + // degrading to empty would look like the history had been lost. + test("an unknown cursor degrades to the newest page, never to empty", () => { + const { window } = pageEntries(entries, { limit: 2, before: "gone" }); + expect(window.map((e) => e.uuid)).toEqual(["e4", "e5"]); + }); + + test("an empty journal pages to nothing without erroring", () => { + expect(pageEntries([], { limit: 10 })).toEqual({ window: [], hasMore: false }); + }); +}); diff --git a/bridge/journal/store.ts b/bridge/journal/store.ts new file mode 100644 index 0000000..c54ce80 --- /dev/null +++ b/bridge/journal/store.ts @@ -0,0 +1,93 @@ +// Reads + caches parsed journals, for whichever adapter the pane's agent selects. +// +// History is fetched ON DEMAND (it is not on the 1.5 s poll path), so the cost that matters is the +// repeat visit, not the first — a parse is reused until the log's size or mtime moves. The cache is +// keyed by absolute path, which is why one store can serve every agent and every herdr session at +// once: two sessions fronting panes whose agents write into the same root still hit the same entry. + +import type { AgentSessionRef, JournalAdapter, TranscriptEntry, TranscriptPage } from "./types.ts"; + +/** How many parsed journals to keep hot. Each is re-parsed only when its file's size/mtime moves. */ +const CACHE_MAX = 4; + +interface CacheEntry { + size: number; + mtimeMs: number; + complete: boolean; + entries: TranscriptEntry[]; +} + +/** + * Page a parsed journal, newest-anchored: with no cursor you get the LAST `limit` turns (the phone + * opens at the recent end, like the mirror it replaces); `before` walks backwards from a turn you + * already hold. Returned entries stay oldest-first so the view renders top-down either way. + */ +export function pageEntries( + entries: TranscriptEntry[], + opts: { limit: number; before?: string }, +): { window: TranscriptEntry[]; hasMore: boolean } { + // An unknown cursor (log rewritten under us, a stale client, or a synthesised cursor whose row + // fell out of the window) degrades to "newest", never to an empty page — the user asked for older + // history and must still see something. + const end = + opts.before === undefined + ? entries.length + : (() => { + const i = entries.findIndex((e) => e.uuid === opts.before); + return i === -1 ? entries.length : i; + })(); + const start = Math.max(0, end - opts.limit); + return { window: entries.slice(start, end), hasMore: start > 0 }; +} + +export class TranscriptStore { + private readonly cache = new Map(); + + /** + * Read one page of a pane's journal. + * + * Null means "nothing to serve" for every reason the client is allowed to distinguish: the ref + * names no file, the adapter refused the ref's shape, or the path failed containment. Those stay + * indistinguishable on purpose — a containment failure must not be probeable. + */ + async page( + adapter: JournalAdapter, + ref: AgentSessionRef, + opts: { limit: number; before?: string }, + ): Promise | null> { + const path = await adapter.source.resolve(ref); + if (path === null) return null; + + // Cache check BEFORE the read: a journal can be 32 MB and paging walks the same file repeatedly, + // so validity is decided by a stat. Only a moved size/mtime costs a read + parse. + const meta = await adapter.source.stat(path); + if (meta === null) return null; // vanished between resolve and read + const cached = this.cache.get(path); + let entry: CacheEntry; + if (cached && cached.size === meta.size && cached.mtimeMs === meta.mtimeMs) { + entry = cached; + // Re-set to move it to the end: eviction below is insertion-ordered, so touching a hit keeps + // the hot journal from being evicted underneath a colder one. + this.cache.delete(path); + this.cache.set(path, entry); + } else { + const { text, complete, size, mtimeMs } = await adapter.source.load(path); + entry = { size, mtimeMs, complete, entries: adapter.parse(text) }; + this.cache.set(path, entry); + if (this.cache.size > CACHE_MAX) { + const oldest = this.cache.keys().next().value; + if (oldest !== undefined) this.cache.delete(oldest); + } + } + const { entries, complete } = entry; + + const { window, hasMore } = pageEntries(entries, opts); + return { + entries: window, + // A clipped file always has more behind it, even at the window's start. + hasMore: hasMore || (!complete && window.length > 0 && window[0] === entries[0]), + total: entries.length, + fileTruncated: !complete, + }; + } +} diff --git a/bridge/journal/text.ts b/bridge/journal/text.ts new file mode 100644 index 0000000..f138f2f --- /dev/null +++ b/bridge/journal/text.ts @@ -0,0 +1,79 @@ +// Text handling every adapter shares: the caps that keep one pathological log from ballooning the +// bridge, and the escape-stripping that keeps a terminal's colour codes out of a view that renders +// text nodes rather than interpreting them. + +/** Per-tool-result cap. Tool output is unbounded (a 2 MB file read); the phone only needs a gist. */ +export const MAX_RESULT_CHARS = 2000; + +/** Per-text-part cap. Generous — assistant prose is the thing you actually came to read. */ +export const MAX_TEXT_CHARS = 20_000; + +/** Longest one-line tool summary. Past this the line stops being a summary. */ +const MAX_SUMMARY_CHARS = 200; + +// CSI/SGR and two-character escapes. Journal text is NOT a terminal mirror — nothing downstream +// interprets escapes, so a `\x1b[2m` left in place renders as garbage glyphs on the phone. +const ANSI_RE = /\x1b\[[0-9;?]*[ -/]*[@-~]|\x1b[@-Z\\-_]/g; + +/** Strip terminal escapes from log text. */ +export function stripAnsi(text: string): string { + return text.replace(ANSI_RE, ""); +} + +/** Cap a string, flagging the cut so the view can say so rather than silently lying. */ +export function clamp(text: string, max: number): { text: string; truncated?: boolean } { + if (text.length <= max) return { text }; + return { text: text.slice(0, max), truncated: true }; +} + +/** Collapse to a single capped line — what a tool-call summary is by definition. */ +export function oneLine(value: string): string { + const line = value.replace(/\s+/g, " ").trim(); + return line.length > MAX_SUMMARY_CHARS ? `${line.slice(0, MAX_SUMMARY_CHARS)}…` : line; +} + +/** + * Collapse a tool call's input object into one readable line. + * + * The well-known arguments get picked by name (the path, the command, the pattern); anything else + * falls back to the first string-ish value, so a tool this code has never heard of still reads as + * something rather than "{...}". Shared across harnesses because tool vocabularies overlap heavily — + * every one of them has a `read`, a `shell`, and a `grep` under some spelling. + */ +export function summarizeToolInput(input: unknown): string { + if (input === null || typeof input !== "object") return ""; + const o = input as Record; + const pick = (...keys: string[]): string | undefined => { + for (const k of keys) { + const v = o[k]; + if (typeof v === "string" && v.trim() !== "") return v; + // Codex spells a shell call's `command` as an ARGV ARRAY (["bash","-lc","ls -la"]), and pi + // passes arrays for multi-file tools — join rather than skip, or the defining argument of the + // most common call in any log goes missing. + if (Array.isArray(v)) { + const joined = v.filter((x): x is string => typeof x === "string").join(" ").trim(); + if (joined !== "") return joined; + } + } + return undefined; + }; + // Order matters, and it is load-bearing: Grep carries both `pattern` and `path`, and the pattern is + // what you actually searched for, so `pattern` MUST outrank the bare `path` (a test pins this). A + // subagent call carries both `description`/`task` and `prompt`, and the short one is already the + // one-line form. + const chosen = + pick( + "file_path", + "command", + "pattern", + "query", + "url", + "path", + "description", + "task", + "prompt", + ) ?? + // Unknown tool: first string value wins, so the line is never empty for no reason. + Object.values(o).find((v): v is string => typeof v === "string" && v.trim() !== ""); + return chosen === undefined ? "" : oneLine(chosen); +} diff --git a/bridge/journal/types.ts b/bridge/journal/types.ts new file mode 100644 index 0000000..1c51011 --- /dev/null +++ b/bridge/journal/types.ts @@ -0,0 +1,112 @@ +// The journal's shared vocabulary — the shape every harness adapter must produce, and the seams the +// store drives them through. Nothing agent-specific lives here. +// +// WHY A JOURNAL EXISTS AT ALL. A pane running an agent usually sits on the terminal's ALTERNATE +// SCREEN, which has no scrollback ring — Herdr's terminal core keeps nothing behind the viewport, so +// `pane.read` can never return more than one screenful (see journal/claude.ts for the measurements). +// The history does exist, though: every harness writes its own session log. This module's job is to +// make "read that log" a per-harness decision behind one interface, so a new harness is an adapter +// rather than a fork of the reader. + +/** + * How an agent named its session, straight off Herdr's `agent_session` record. + * + * Two kinds are in the wild and they are NOT interchangeable: + * - `id` — an opaque session id (Claude, Codex). The adapter must find the file itself, so the + * value never touches a path until the adapter has validated its shape. + * - `path` — an absolute path to the log, reported by the agent (pi). Convenient, but it is + * attacker-shaped input by construction: it arrives over the socket from a process we + * do not control, so the adapter must still confine it to its own root. + */ +export interface AgentSessionRef { + kind: "id" | "path"; + value: string; +} + +/** One renderable piece of a turn. Deliberately small — the phone renders these as text nodes. */ +export type TranscriptPart = + | { kind: "text"; text: string; truncated?: boolean } + /** + * Extended-thinking text. Whether this ever carries anything is per-harness: Claude Code persists + * `thinking` blocks with the text stripped (empty every time), while Codex and pi both write real + * reasoning summaries. The branch is universal; only the harnesses that fill it differ. + */ + | { kind: "thinking"; text: string; truncated?: boolean } + /** A tool call. `result` is filled in from the result row that answers it, when one exists. */ + | { + kind: "tool"; + name: string; + /** One-line gist of the call's input (the file read, the command run) — never the whole input. */ + summary: string; + result?: { text: string; truncated?: boolean; isError?: boolean }; + }; + +/** + * One turn of the conversation. + * + * `user`/`assistant` are speech. The other two are NOT, and are rendered set apart so they can't be + * mistaken for it: `summary` is a compaction summary the agent wrote about its own history, and + * `note` is machine-injected content that still belongs on screen (a background task finishing, a + * local command's output). + */ +export interface TranscriptEntry { + /** + * The paging cursor (`?before=`), and it must be stable across reads of the same log. + * + * Where a harness gives rows their own id (Claude, pi) this IS that id. Codex rows carry none, so + * its adapter synthesises one — see journal/codex.ts for why that synthetic form has to be + * anchored to the END of the file. + */ + uuid: string; + /** ISO timestamp from the log; empty when the row carried none. */ + ts: string; + role: "user" | "assistant" | "summary" | "note"; + parts: TranscriptPart[]; +} + +/** What the history endpoint answers with, minus the pane id the route adds. */ +export interface TranscriptPage { + paneId: string; + /** Oldest-first, ready to render top-down. */ + entries: TranscriptEntry[]; + /** True when older turns exist before `entries[0]` — drives "load older". */ + hasMore: boolean; + /** Total turns available in the parsed window (after sidechain filtering). */ + total: number; + /** True when the on-disk log exceeded the byte cap and we kept only its tail. */ + fileTruncated: boolean; +} + +/** + * The fs seam. Real implementations live beside each adapter; tests inject a fake so no temp files + * are needed (the repo convention — see sessions.test.ts / state-engine.test.ts). + */ +export interface TranscriptSource { + /** Absolute path of the log this ref names, or null when it isn't on disk / isn't ours to read. */ + resolve(ref: AgentSessionRef): Promise; + /** + * Size + mtime of a log, WITHOUT reading it — the store's cache-validity check. + * + * Split out from `load` on purpose: a journal can be 32 MB, and paging back through a long + * conversation asks for the same file over and over. Reading it to discover the cache was already + * valid made every "load older" tap a full re-read. + */ + stat(path: string): Promise<{ size: number; mtimeMs: number } | null>; + /** Tail-read a log. `complete` is false when the byte cap clipped the head. */ + load(path: string): Promise<{ text: string; complete: boolean; size: number; mtimeMs: number }>; +} + +/** + * One harness's journal support: how to find its log, and how to read its grammar. + * + * `agent` is matched against the Herdr snapshot's `agent` string, and it is also the registry key — + * the map is built FROM this field so the two can never drift (journal/registry.ts). An agent with + * no adapter simply has no journal, which the route reports as an ordinary "no-session". + * + * `parse` is PURE — no fs, no clock — so every harness's grammar is table-testable under `bun test`. + */ +export interface JournalAdapter { + readonly agent: string; + readonly source: TranscriptSource; + parse(text: string): TranscriptEntry[]; +} diff --git a/bridge/server.test.ts b/bridge/server.test.ts index 824a7ce..f70c078 100644 --- a/bridge/server.test.ts +++ b/bridge/server.test.ts @@ -44,7 +44,7 @@ function cfg(overrides: Partial = {}): Config { notifyDelayMs: 30_000, readLines: 200, transcript: true, - transcriptRoot: "/tmp/claude-projects", + journalRoots: { claude: "/tmp/claude-projects", codex: "/nope/codex", pi: "/nope/pi" }, submitKeys: ["Enter"], trustedUser: "", deviceHeader: "", diff --git a/bridge/server.ts b/bridge/server.ts index 11b8a7d..de8642b 100644 --- a/bridge/server.ts +++ b/bridge/server.ts @@ -16,7 +16,10 @@ import { herdTagFor, type SessionRegistry } from "./sessions.ts"; import type { Snooze } from "./snooze.ts"; import type { UpdateMonitor } from "./update.ts"; import type { StateEngine } from "./state-engine.ts"; -import { ClaudeTranscriptSource, TranscriptStore } from "./transcript.ts"; +import { adapterFor, buildJournalRegistry } from "./journal/registry.ts"; +import { TranscriptStore } from "./journal/store.ts"; +import type { JournalAdapter } from "./journal/types.ts"; +import { toPaneWire } from "./types.ts"; import type { ActionResponse, BridgeConfig, @@ -106,12 +109,14 @@ export function startServer(opts: { audit: AuditLog; }) { const { cfg, registry, push, snooze, notifyPrefs, updateMonitor, audit } = opts; - // One transcript store for the process: it caches parsed session logs across requests, and the - // cache is keyed by absolute path, so sharing it across herdr sessions is correct (two sessions - // can front panes whose agents write into the same ~/.claude/projects root). - const transcripts = cfg.transcript - ? new TranscriptStore(new ClaudeTranscriptSource(cfg.transcriptRoot)) - : null; + // One journal registry + store for the process. The store's cache is keyed by absolute path, so + // sharing it across herdr sessions AND across harnesses is correct — two sessions can front panes + // whose agents write into the same root. Which harnesses have journals at all is decided in + // journal/registry.ts, never here. + const journals = cfg.transcript ? buildJournalRegistry(cfg.journalRoots) : null; + const transcripts = cfg.transcript ? new TranscriptStore() : null; + /** Does this agent have a journal at all — the snapshot's History-affordance gate. */ + const hasJournal = (agent: string) => adapterFor(journals ?? {}, agent) !== undefined; // Per-session background notifications live in each session's runtime (built by the factory in // index.ts, wired to its StateEngine transitions). The routes here only fan preference changes and // snooze-clears across every live session's coordinator. @@ -149,8 +154,12 @@ export function startServer(opts: { bridge, // Only report device state when the feature is on, so an off deployment sends nothing new. ...(device.enforced ? { device } : {}), - agents, - shellPanes, + // The one place a pane leaves the bridge: the session ref is stripped to a presence flag + // here, so an agent-reported filesystem path never reaches a browser (see toPaneWire). + // The flag is computed against the registry, so a harness Herdr detects but Collie has no + // journal for doesn't advertise a History button that can only ever come back empty. + agents: agents.map((p) => toPaneWire(p, hasJournal)), + shellPanes: shellPanes.map((p) => toPaneWire(p, hasJournal)), workspaces, tabs, sessions: registry.list(), @@ -211,7 +220,7 @@ export function startServer(opts: { if (!action && req.method === "GET") return readPane(herdr, cfg, paneId, url, req); if (action === "history" && req.method === "GET") - return paneHistory(cfg, transcripts, rt.engine, paneId, url, req); + return paneHistory(cfg, journals, transcripts, rt.engine, paneId, url, req); if (action === "reply" && req.method === "POST") return replyPane(herdr, cfg, paneId, req, audit, device, session); if (action === "keys" && req.method === "POST") return keysPane(herdr, cfg, paneId, req, audit, device, session); if (action === "upload" && req.method === "POST") return uploadPane(cfg, paneId, req, audit, device, session); @@ -444,14 +453,16 @@ export function historyParams(url: URL): { limit: number; before?: string } { } /** - * GET /api/pane/:id/history — the conversation history a Claude pane's terminal cannot provide. + * GET /api/pane/:id/history — the conversation history the pane's terminal cannot provide. * - * The session id is resolved HERE, from the live snapshot, keyed by pane id — the client never sends + * The session ref is resolved HERE, from the live snapshot, keyed by pane id — the client never sends * one. That is the whole safety story for a route that reads files: the only client-controlled inputs - * are a pane id (a Map lookup) and an opaque cursor (an array lookup). + * are a pane id (a Map lookup) and an opaque cursor (an array lookup). Which harness knows how to + * read the log is the registry's decision, so this route stays agent-agnostic. */ async function paneHistory( cfg: Config, + journals: Record | null, transcripts: TranscriptStore | null, engine: StateEngine, paneId: string, @@ -462,16 +473,20 @@ async function paneHistory( const unavailable = (reason: "disabled" | "no-session" | "no-log") => json({ paneId, available: false, reason } satisfies PaneHistoryResponse, accept); - if (!cfg.transcript || transcripts === null) return unavailable("disabled"); + if (!cfg.transcript || transcripts === null || journals === null) return unavailable("disabled"); const { agents, shellPanes } = engine.current(); const pane = [...agents, ...shellPanes].find((a) => a.paneId === paneId); - // No pane, or an agent that reported no id-kind session (a shell, or a harness that doesn't keep - // one): there is nothing to read, and that's an ordinary answer rather than an error. - if (!pane?.agentSessionId) return unavailable("no-session"); + // No pane, or an agent that named no session (a shell, or a harness whose integration isn't + // installed): nothing to read, and that's an ordinary answer rather than an error. + if (!pane?.agentSession) return unavailable("no-session"); + // An agent with no adapter has no journal. Same answer — the UI shouldn't distinguish "this + // harness isn't supported" from "this pane never started one"; both mean there's nothing to show. + const adapter = adapterFor(journals, pane.agent); + if (adapter === undefined) return unavailable("no-session"); try { - const page = await transcripts.page(pane.agentSessionId, historyParams(url)); + const page = await transcripts.page(adapter, pane.agentSession, historyParams(url)); if (page === null) return unavailable("no-log"); return json({ paneId, available: true, ...page } satisfies PaneHistoryResponse, accept); } catch (err) { diff --git a/bridge/state-engine.test.ts b/bridge/state-engine.test.ts index 9374450..1e028ab 100644 --- a/bridge/state-engine.test.ts +++ b/bridge/state-engine.test.ts @@ -505,26 +505,70 @@ describe("StateEngine — poke / cadence / onUpdate", () => { // record, and both must stay ABSENT rather than defaulting when the server doesn't report them — // an older Herdr should read as "unknown", not as "zero scrollback" or "no transcript". describe("StateEngine — pane capability fields", () => { - test("maps an id-kind agent session to agentSessionId", async () => { + test("keeps an id-kind agent session (claude, codex)", async () => { const { herdr, engine, poll } = makeEngine(); const p = pane("w1:p1", "w1", "idle", "claude"); p.agent_session = { source: "herdr:claude", agent: "claude", kind: "id", value: "abc-123" }; herdr.panes = [p]; await poll(); - expect(engine.current().agents[0]!.agentSessionId).toBe("abc-123"); + expect(engine.current().agents[0]!.agentSession).toEqual({ kind: "id", value: "abc-123" }); + }); + + // The regression that kept pi journal-less: pi's herdr integration reports `agent_session_path` + // in preference to an id, and this mapper used to keep ONLY kind "id" — so a pi pane arrived with + // no session at all and its history could never be offered. Which kinds are meaningful is the + // journal adapter's call now, not this function's. + test("keeps a path-kind agent session (pi)", async () => { + const { herdr, engine, poll } = makeEngine(); + const p = pane("w1:p1", "w1", "idle", "pi"); + p.agent_session = { + source: "herdr:pi", + agent: "pi", + kind: "path", + value: "/home/you/.pi/agent/sessions/--repo--/2026-07-29T10-00-00-000Z_abc.jsonl", + }; + herdr.panes = [p]; + await poll(); + expect(engine.current().agents[0]!.agentSession).toEqual({ + kind: "path", + value: "/home/you/.pi/agent/sessions/--repo--/2026-07-29T10-00-00-000Z_abc.jsonl", + }); + }); + + // Live-observed on a demo pane: Herdr keeps reporting the LAST session announced for a pane, so + // relaunching it as a different harness leaves the previous agent's ref behind — a pane running + // `pi` still advertised a `herdr:claude` id. Routing by pane agent would then hand pi's adapter a + // Claude uuid. + test("drops a session ref left behind by a different harness", async () => { + const { herdr, engine, poll } = makeEngine(); + const p = pane("w1:p1", "w1", "idle", "pi"); + p.agent_session = { source: "herdr:claude", agent: "claude", kind: "id", value: "abc-123" }; + herdr.panes = [p]; + await poll(); + expect(engine.current().agents[0]!.agentSession).toBeUndefined(); + }); + + test("keeps a ref from an older Herdr that reports no owning agent", async () => { + const { herdr, engine, poll } = makeEngine(); + const p = pane("w1:p1", "w1", "idle", "claude"); + p.agent_session = { kind: "id", value: "abc-123" }; + herdr.panes = [p]; + await poll(); + expect(engine.current().agents[0]!.agentSession).toEqual({ kind: "id", value: "abc-123" }); }); test.each([ - ["a non-id session kind", { kind: "name", value: "my-session" }], + ["an unrecognised session kind", { kind: "name", value: "my-session" }], ["a session with no value", { kind: "id" }], + ["a session with an empty value", { kind: "id", value: "" }], ["no agent_session at all", undefined], - ])("omits agentSessionId for %s", async (_label, session) => { + ])("omits the session for %s", async (_label, session) => { const { herdr, engine, poll } = makeEngine(); const p = pane("w1:p1", "w1", "idle", "claude"); if (session) p.agent_session = session; herdr.panes = [p]; await poll(); - expect(engine.current().agents[0]!.agentSessionId).toBeUndefined(); + expect(engine.current().agents[0]!.agentSession).toBeUndefined(); }); test("readableLines is scrollback depth PLUS the viewport (what a recent read can return)", async () => { diff --git a/bridge/state-engine.ts b/bridge/state-engine.ts index 8e5c5fe..83e1ecc 100644 --- a/bridge/state-engine.ts +++ b/bridge/state-engine.ts @@ -213,10 +213,27 @@ export class StateEngine { kind, // A user-set pane label (herdr pane.rename); omitted when unset so "absent stays absent". ...(typeof p.label === "string" && p.label.length > 0 ? { paneLabel: p.label } : {}), - // The agent's own session id — only the "id" kind names an on-disk transcript. Omitted - // otherwise, so "no history for this pane" is simply the field being absent. - ...(p.agent_session?.kind === "id" && typeof p.agent_session.value === "string" - ? { agentSessionId: p.agent_session.value } + // How the agent named its session. BOTH kinds are kept: Claude and Codex report an `id`, + // while pi reports a `path` (its herdr integration prefers `agent_session_path` whenever + // the session manager has a file open). Keeping only `id` — as this did until journals + // became per-agent — silently denied pi any history at all. Which kinds are meaningful is + // now the adapter's call, not this function's; anything else is omitted, so "no history + // for this pane" stays simply the field being absent. + // + // The ref must also BELONG to the agent currently in the pane. Herdr keeps reporting the + // last session announced for a pane, so relaunching a pane's agent as a different harness + // leaves the old one's ref behind — live-observed: a pane running `pi` still advertising + // `{source:"herdr:claude", kind:"id"}` from the claude that had been there before. Serving + // that would hand pi's adapter a Claude uuid; harmless today (it resolves to nothing) but + // only by luck. `agent_session.agent` is compared when Herdr reports it, and absence stays + // permissive so an older server that omits the field still works. + ...((p.agent_session?.kind === "id" || p.agent_session?.kind === "path") && + typeof p.agent_session.value === "string" && + p.agent_session.value !== "" && + (typeof p.agent_session.agent !== "string" || + p.agent_session.agent === "" || + p.agent_session.agent === agent) + ? { agentSession: { kind: p.agent_session.kind, value: p.agent_session.value } } : {}), // Scrollback depth + viewport = what a `recent` read can yield. Omitted when the server // predates `scroll`, so an older Herdr simply reads as "unknown" rather than "zero". diff --git a/bridge/types.ts b/bridge/types.ts index 74e139e..5130466 100644 --- a/bridge/types.ts +++ b/bridge/types.ts @@ -1,11 +1,11 @@ // Domain model for the bridge. These are OUR types, decoupled from Herdr's wire shapes // (which live only in herdr-client.ts). The rest of the app talks in these terms. -import type { TranscriptEntry } from "./transcript.ts"; +import type { AgentSessionRef, TranscriptEntry } from "./journal/types.ts"; // Re-exported so the wire surface has ONE import site: a consumer of PaneHistoryResponse gets the -// entry shape from here too, without reaching into the parser module. -export type { TranscriptEntry, TranscriptPart } from "./transcript.ts"; +// entry shape from here too, without reaching into an adapter module. +export type { TranscriptEntry, TranscriptPart } from "./journal/types.ts"; export type AgentStatus = "idle" | "working" | "blocked" | "done" | "unknown"; @@ -35,12 +35,14 @@ export interface AgentView { */ sessionName?: string; /** - * The agent's own session id (Herdr `agent_session.value`), present only when the agent reported - * an id-kind session. Its presence is what tells the UI a transcript may exist for this pane, so - * the History affordance can be offered without a speculative fetch. The history endpoint - * re-derives this server-side from the pane id and NEVER trusts a client-supplied value. + * How the agent named its session (Herdr `agent_session`), when it named one at all. + * + * SERVER-SIDE ONLY — stripped before this pane goes on the wire (see {@link PaneWire}), because + * pi reports a kind-`path` ref whose value is an absolute filesystem path, and the client has no + * use for it: the history endpoint re-derives the ref from the pane id and never trusts a + * client-supplied one. What the client gets is the presence flag `hasSession`. */ - agentSessionId?: string; + agentSession?: AgentSessionRef; /** * Upper bound on the lines a `recent` read of this pane can return — Herdr's scrollback depth plus * the viewport. This is the ONLY reliable "is there more scrollback" signal: `PaneRead.truncated` @@ -51,6 +53,37 @@ export interface AgentView { readableLines?: number; } +/** + * A pane as the BROWSER sees it: every {@link AgentView} field except the session ref, which is + * replaced by a presence flag. + * + * The two shapes differ on purpose. `agentSession.value` is either an opaque id (Claude, Codex) or an + * absolute path (pi) — the client needs neither, and shipping the path would hand out filesystem + * layout for nothing. All the UI ever asked of that field was "may this pane have history?", which is + * what `hasSession` answers, so the History affordance still shows without a speculative fetch. + * + * NOTE the `Omit` is opt-OUT: a future server-only field on AgentView goes on the wire unless it is + * added to the omit list here. If you add one, strip it here in the same change. + */ +export type PaneWire = Omit & { + /** True when this pane's history is actually offerable: the agent named a session AND its harness + * has a journal adapter. Says nothing about whether the log is readable — a named session whose + * file is missing still answers `available:false` with reason `no-log`. */ + hasSession?: boolean; +}; + +/** + * Strip a pane down to its wire shape. The one place the session ref leaves the bridge's hands. + * + * `hasJournal` is asked rather than assumed: a harness can name a session while having no adapter to + * read it (Herdr detects more agents than Collie has journals for). Keying the flag on the ref alone + * would advertise a History affordance that always comes back empty, so the registry gets a vote. + */ +export function toPaneWire(pane: AgentView, hasJournal: (agent: string) => boolean): PaneWire { + const { agentSession, ...rest } = pane; + return agentSession && hasJournal(pane.agent) ? { ...rest, hasSession: true } : rest; +} + /** A Herdr workspace ("space") — a project-scoped container of tabs. From `workspace.list`. */ export interface WorkspaceView { workspaceId: string; @@ -115,9 +148,9 @@ export interface SnapshotResponse { /** Per-device authorisation for the requesting client; absent when the feature is off. */ device?: DeviceAuth; /** Agent-bearing panes, triage-sorted (the home list). */ - agents: AgentView[]; + agents: PaneWire[]; /** Bare shell panes (no agent) — surfaced so freshly-created tabs/spaces are reachable. */ - shellPanes: AgentView[]; + shellPanes: PaneWire[]; /** All spaces (workspaces) and their tabs, for the space/tab navigator. */ workspaces: WorkspaceView[]; tabs: TabView[]; diff --git a/herdr-plugin.toml b/herdr-plugin.toml index f145bd4..0b322d5 100644 --- a/herdr-plugin.toml +++ b/herdr-plugin.toml @@ -1,6 +1,6 @@ id = "herdr.collie" name = "Collie" -version = "0.18.0" +version = "0.19.0" min_herdr_version = "0.7.0" description = "Mobile web UI to monitor and reply to your agent herd, served over Tailscale" platforms = ["linux", "macos"] diff --git a/package.json b/package.json index 46645f6..70e37b2 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "collie", - "version": "0.18.0", + "version": "0.19.0", "private": true, "license": "MIT", "description": "Collie — a mobile web UI to monitor and reply to your Herdr agent herd over Tailscale", diff --git a/scripts/journal-probe.ts b/scripts/journal-probe.ts new file mode 100644 index 0000000..1c60a50 --- /dev/null +++ b/scripts/journal-probe.ts @@ -0,0 +1,151 @@ +#!/usr/bin/env bun +// Probe every journal adapter against the REAL logs on this machine. +// +// The unit tests pin each grammar against builders that mirror the on-disk shape; this answers the +// different question those can't — does the log actually exist where the adapter looks, and does a +// real one (with its unannounced version drift, its odd rows, its size) still parse? Run it after +// touching an adapter, and on any machine where a journal is unexpectedly empty: +// +// bun scripts/journal-probe.ts +// +// It reads only, prints only counts and roles — never transcript content, so its output is safe to +// paste into an issue. A harness you don't have installed reports `no logs found`, which is not a +// failure: exit code is non-zero only when a log EXISTS and the adapter couldn't resolve or parse it. + +import { readdir, stat } from "node:fs/promises"; +import { join } from "node:path"; + +import { loadConfig } from "../bridge/config.ts"; +import { buildJournalRegistry } from "../bridge/journal/registry.ts"; +import type { AgentSessionRef, JournalAdapter, TranscriptEntry } from "../bridge/journal/types.ts"; + +/** + * Every `.jsonl` under `dir`, newest first. + * + * We try candidates in order rather than trusting the single newest, because the newest log is often + * a dud through no fault of the adapter: a session someone opened and abandoned parses to zero turns + * quite correctly (Codex writes a `session_meta` plus one injected `` turn, both + * of which are meant to be dropped). Only "no candidate at all worked" is a real failure. + */ +async function logsNewestFirst(dir: string, depth = 4): Promise { + const found: { path: string; mtimeMs: number }[] = []; + const walk = async (d: string, left: number): Promise => { + let names: string[]; + try { + names = await readdir(d); + } catch { + return; + } + for (const name of names) { + const p = join(d, name); + let st: Awaited>; + try { + st = await stat(p); + } catch { + continue; + } + if (st.isDirectory()) { + if (left > 0) await walk(p, left - 1); + } else if (name.endsWith(".jsonl")) { + found.push({ path: p, mtimeMs: st.mtimeMs }); + } + } + }; + await walk(dir, depth); + return found.sort((a, b) => b.mtimeMs - a.mtimeMs).map((f) => f.path); +} + +/** How many candidates to try before calling a harness unreadable. */ +const MAX_CANDIDATES = 12; + +/** + * Rebuild the session ref Herdr would have reported for this log, per harness. + * + * This is the part worth probing: each adapter's resolve() is a different strategy (Claude scans flat + * project dirs, Codex walks date partitions, pi takes the path straight), and each derives from a + * differently-shaped filename. + */ +function refFor(agent: string, path: string): AgentSessionRef | null { + const file = path.slice(path.lastIndexOf("/") + 1).replace(/\.jsonl$/, ""); + const uuid = /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/i.exec(file)?.[0]; + if (agent === "pi") return { kind: "path", value: path }; + return uuid ? { kind: "id", value: uuid } : null; +} + +function summarise(entries: TranscriptEntry[]): string { + const roles = new Map(); + let parts = 0; + let results = 0; + for (const e of entries) { + roles.set(e.role, (roles.get(e.role) ?? 0) + 1); + for (const p of e.parts) { + parts++; + if (p.kind === "tool" && p.result !== undefined) results++; + } + } + const byRole = [...roles].map(([r, n]) => `${r}:${n}`).join(" "); + return `${entries.length} turns (${byRole}), ${parts} parts, ${results} tool results`; +} + +async function probe(adapter: JournalAdapter, root: string): Promise<"ok" | "empty" | "fail"> { + const label = adapter.agent.padEnd(7); + const candidates = await logsNewestFirst(root); + if (candidates.length === 0) { + console.log(`${label} — no logs found under ${root} (harness not installed here?)`); + return "empty"; + } + + let tried = 0; + let lastProblem = "no candidate produced turns"; + for (const log of candidates.slice(0, MAX_CANDIDATES)) { + const ref = refFor(adapter.agent, log); + if (ref === null) { + // Claude keeps subagent logs under `subagents/` with no uuid in the name — not a session, so + // not something Herdr would ever name. Skip rather than fail. + lastProblem = `no session ref derivable from ${log}`; + continue; + } + tried++; + + const resolved = await adapter.source.resolve(ref); + if (resolved === null) { + lastProblem = `ref ${ref.kind}:${ref.value.slice(0, 60)} did not resolve`; + continue; + } + + const { text, complete } = await adapter.source.load(resolved); + const entries = adapter.parse(text); + if (entries.length === 0) { + lastProblem = `resolved a log but parsed 0 turns from ${text.length} bytes`; + continue; + } + + const cursors = new Set(entries.map((e) => e.uuid)); + const dupes = entries.length - cursors.size; + console.log( + `${label} ✓ ${summarise(entries)}${complete ? "" : " [tail-clipped]"}` + + `${dupes > 0 ? ` ⚠ ${dupes} duplicate cursors` : ""}`, + ); + console.log(`${" ".repeat(9)}${resolved} (candidate ${tried} of ${candidates.length})`); + return "ok"; + } + + // Every candidate failed: either the logs moved, or a format drifted under the parser. + console.log(`${label} ✗ ${tried} candidate(s) tried, none readable — last: ${lastProblem}`); + return "fail"; +} + +const cfg = loadConfig(); +const registry = buildJournalRegistry(cfg.journalRoots); +// Keyed lookup rather than a cast: JournalRoots is a closed shape on purpose (adding a harness +// should be a type error here until its root is wired), so widen it explicitly. +const roots = new Map(Object.entries(cfg.journalRoots)); + +console.log("journal adapters — probing real logs\n"); +const results = await Promise.all( + Object.entries(registry).map(([agent, adapter]) => probe(adapter, roots.get(agent) ?? "")), +); +const failed = results.filter((r) => r === "fail").length; +const ok = results.filter((r) => r === "ok").length; +console.log(`\n${ok} ok, ${results.length - ok - failed} with no logs, ${failed} failed`); +process.exit(failed > 0 ? 1 : 0); diff --git a/web/package.json b/web/package.json index 70276e1..85af38f 100644 --- a/web/package.json +++ b/web/package.json @@ -1,6 +1,6 @@ { "name": "collie-web", - "version": "0.18.0", + "version": "0.19.0", "private": true, "license": "MIT", "type": "module", diff --git a/web/src/components/agent-chat.test.tsx b/web/src/components/agent-chat.test.tsx index f26207d..00c0197 100644 --- a/web/src/components/agent-chat.test.tsx +++ b/web/src/components/agent-chat.test.tsx @@ -447,13 +447,13 @@ describe("AgentChat — shared header: stale-status dimming", () => { // lead to an empty screen. describe("AgentChat — history affordance", () => { it("is offered when the pane reports an agent session id", () => { - const agent = { ...fixtureAgents[0]!, agentSessionId: "d7e62e23-8576-4c63-98ba-ec1b02902c6b" }; + const agent = { ...fixtureAgents[0]!, hasSession: true }; renderChat({ agent, agents: [agent] }); expect(screen.getByRole("button", { name: /conversation history/i })).toBeInTheDocument(); }); it("is hidden when the pane has no agent session (a shell, or a harness without one)", () => { - renderChat(); // fixture agents carry no agentSessionId + renderChat(); // fixture agents carry no session expect(screen.queryByRole("button", { name: /conversation history/i })).not.toBeInTheDocument(); }); @@ -462,7 +462,7 @@ describe("AgentChat — history affordance", () => { // // (The top-of-mirror affordance is covered separately below.) it("sits to the LEFT of the status pill", () => { - const agent = { ...fixtureAgents[0]!, agentSessionId: "d7e62e23-8576-4c63-98ba-ec1b02902c6b" }; + const agent = { ...fixtureAgents[0]!, hasSession: true }; renderChat({ agent, agents: [agent] }); const history = screen.getByRole("button", { name: /conversation history/i }); const pill = screen.getByText("needs you"); // fixtureAgents[0] is blocked → "needs you" @@ -476,13 +476,12 @@ describe("AgentChat — history affordance", () => { // working signal is `readableLines` (scrollback depth + viewport), and which button appears is // decided by what the pane can actually offer — the two are never simultaneously possible. describe("AgentChat — top-of-mirror history affordance", () => { - const SESSION_ID = "d7e62e23-8576-4c63-98ba-ec1b02902c6b"; const showHistory = () => screen.queryByRole("button", { name: /show entire history/i }); const loadOlder = () => screen.queryByRole("button", { name: /load older/i }); it("an agent pane with a transcript offers the full history, not scrollback paging", () => { // A Claude pane: alt-screen, so readableLines is just its viewport — there IS no scrollback. - const agent = { ...fixtureAgents[0]!, agentSessionId: SESSION_ID, readableLines: 51 }; + const agent = { ...fixtureAgents[0]!, hasSession: true, readableLines: 51 }; renderChat({ agent, agents: [agent], requestedLines: 600 }); expect(showHistory()).toBeInTheDocument(); expect(loadOlder()).not.toBeInTheDocument(); @@ -517,7 +516,7 @@ describe("AgentChat — top-of-mirror history affordance", () => { }); it("a transcript wins even when the pane also reports scrollback", () => { - const agent = { ...fixtureAgents[0]!, agentSessionId: SESSION_ID, readableLines: 6946 }; + const agent = { ...fixtureAgents[0]!, hasSession: true, readableLines: 6946 }; renderChat({ agent, agents: [agent], requestedLines: 600 }); expect(showHistory()).toBeInTheDocument(); expect(loadOlder()).not.toBeInTheDocument(); diff --git a/web/src/components/agent-chat.tsx b/web/src/components/agent-chat.tsx index 562c7b2..db6a018 100644 --- a/web/src/components/agent-chat.tsx +++ b/web/src/components/agent-chat.tsx @@ -239,7 +239,7 @@ export function AgentChat({ // `moreScrollback`: Herdr says this pane can still yield lines beyond the window we've asked for, // AND we're under the cap Herdr's own read clamp imposes. `readableLines` is undefined on an older // bridge/Herdr; treat that as "no idea" and stay hidden rather than offer a tap that fetches nothing. - const historyAvailable = Boolean(agent?.agentSessionId); + const historyAvailable = Boolean(agent?.hasSession); const moreScrollback = agent?.readableLines !== undefined && requestedLines < agent.readableLines && @@ -528,7 +528,7 @@ export function AgentChat({ rightLead={ agent ? ( <> - {agent.agentSessionId && ( + {agent.hasSession && (