Platform reframe: layered engine, tmux agents, workflow packages

[mattleaverton]

Summary

This PR reframes Kilroy as a layered platform: a core graph-execution engine (L0), an agent-execution layer that drives CLI tools via tmux (L1), and workflow packages that ship as self-contained directories with graphs, scripts, and manifests (L2). It adds the primitives and server infrastructure needed to run multi-agent workflows reliably from a headless process: explicit loop and concurrent split/join nodes, a SQLite run database, a per-run structured activity log, a REST API with SSE streaming, and an embedded dashboard UI. Three ready-to-use workflow packages ship with the branch: quick-launch, pr-review, and build-test.

This unblocks building agentic automation that can be launched from other agents (the quick-launch package is already used that way), observed in real time via the log API, and extended by adding new workflow packages without touching engine code.

What's new

Layered architecture (L0/L1/L2)

• Engine package (internal/attractor/engine/) is now L0-only: graph execution, node dispatch, lifecycle hooks. No agent or git specifics inside.
• L1 (internal/attractor/agents/) provides TmuxAgentHandler, which is registered as the agent handler at startup.
• L2 (internal/attractor/workflows/) provides GitHook, HumanGateHandler, and the package loader — registered at startup in cmd/kilroy/main.go via newLayeredRegistry().
• GitOps interface in the engine means git mode is fully optional; the engine runs in plain-directory mode when GitOps is nil, enabling non-git workflows.
• codergen has been renamed to agent throughout (types, files, test names).

Workflow packages and CLI flags

• --package <dir>: loads a self-contained workflow from a directory containing graph.dot, optional workflow.toml, and scripts/ / prompts/ subdirectories. Package scripts and prompts are materialized into .kilroy/package/ inside the workspace before execution.
• --workspace <dir>: sets the execution directory independently of the source repo.
• --input <path-or-json>: passes structured inputs as a JSON/YAML file or inline JSON. Values are injected as KILROY_INPUT_<KEY> env vars and written to .kilroy/INPUT.md.
• --prompt-file <path>: stages a file as the node prompt.
• --label KEY=VALUE: attaches metadata to a run; queryable via runs list and runs show.
• --tmux: enables the tmux agent handler for the run.
• runs show, runs wait: wait on and inspect a run by ID or prefix; --latest with --label filters to pick the most recent matching run.

Engine primitives

• Loop (internal/attractor/engine/loop.go): explicit single-node or multi-node iteration. Termination conditions: loop_count, loop_max, loop_until_file, loop_until_file_contains, loop_while_outcome=fail. Each iteration gets its own attempt number and DB row for observability. Nested loops are rejected at validation.
• Concurrent split/join (internal/attractor/engine/concurrent.go): runs independent node chains concurrently in the same workspace, waits for all branches. Configured via concurrent_id node attribute; allow_partial=true permits branch failures. Nested concurrent scopes are rejected.
• Output contracts: nodes declare expected output files via outputs= attribute; the engine enforces their existence after the node completes and writes .kilroy/FEEDBACK.md on failure.
• .kilroy/ file conventions (internal/attractor/engine/kilroy_files.go): INPUT.md, CONTEXT.md, TASK.md, FEEDBACK.md written by the engine as standard inter-node data files. KILROY_STAGE_STATUS_PATH / KILROY_STAGE_STATUS_FALLBACK_PATH env vars let scripts signal completion status.

Agent execution via tmux

• TmuxAgentHandler (internal/attractor/agents/tmux_handler.go) spawns each agent node in a named tmux session on the kilroy socket. Sessions persist after completion for inspection; pane output is tailed in real time.
• Session manager (internal/attractor/agents/tmux/) handles create, destroy, wait, and health checks. Process group kill (SIGTERM then SIGKILL) on context cancel — no orphan processes.
• Invocation templates (internal/attractor/agents/templates/) define per-tool startup, env, and structured-output behavior for claude, codex, opencode, and gemini. Adding a new CLI tool requires only a new template, no handler code.
• Auth isolation: codex sessions write a per-run auth.json into an isolated CODEX_HOME; the user's ~/.codex/config.toml is explicitly excluded so user-local settings (model, reasoning effort) cannot bleed into kilroy runs.
• Conversation log parsers (internal/attractor/agents/agentlog/): structured AgentEvent extraction from Claude JSONL, Codex JSONL, and OpenCode JSONL. A live tailer reads the log during execution and emits events to the RunLog.

RunLog and observability

• RunLog (internal/attractor/engine/runlog.go): newline-delimited JSON activity log written to {run_logs_root}/run.log. Every lifecycle event (run start/end, node start/end, edge decision, checkpoint, git activity, tool stdout/stderr line, agent conversation event) is emitted as a timestamped RunLogEvent.
• GET /runs/{id}/log: HTTP endpoint that serves the run.log with optional query filters (?node=, ?source=, ?event=, ?since=, ?tail=N) and SSE streaming (?stream=true) for live tail without polling.
• Git activity events (worktree.created, commit) appear in the RunLog so the full timeline is in one place.

Server and REST API

• internal/server/ now covers a full run-management API:
  • POST /runs: accepts a workflow package, inputs, workspace, and labels to start a run.
  • GET /runs, GET /runs/{id}: run list and detail from the RunDB (no filesystem-only fallback for completed runs).
  • GET /runs/{id}/log: RunLog with SSE.
  • GET /runs/{id}/nodes/{nodeId}/diff: per-node git diff.
  • GET /runs/{id}/files/{path...}, GET /runs/{id}/workspace/{path...}: file browser.
  • GET /workflows: lists available packages.
  • /pipelines/... kept as a backward-compatibility alias.
• Embedded dashboard UI at /ui/ (internal/server/ui/index.html, viz.js, viz-render.js) — compiled into the binary via //go:embed. No separate asset server needed.
• Prefix ID resolution: runs show 01KPB resolves to the unique matching run, same as the CLI.
• SQLite WAL mode and 5-second busy timeout (internal/attractor/rundb/rundb.go) for safe concurrent server writes.

Workflows shipped

• workflows/quick-launch/: single-agent task runner. Accepts prompt, optional context_file or context. Three graph variants: graph.dot (Claude), graph.codex.dot (Codex), graph.gemini.dot (Gemini). Ships with a kilroy slash-command skill and install script.
• workflows/pr-review/: full PR review pipeline — checkout, build/test, per-file code review (Claude), holistic review, combined report. Accepts pr_repo and pr_number inputs; emits review-report.md.
• workflows/build-test/: build-and-test workflow for CI-style validation.
• workflows/multi-tool-exercise/: multi-agent graph used to validate concurrent tool execution and observability.

Reliability fixes

• Subprocess group kill on cancel: tool handler spawns each subprocess in its own process group (Setpgid: true) and sends SIGTERM to the group on context cancellation, falling back to SIGKILL if needed (internal/attractor/engine/process_group_unix.go). Prevents orphaned child processes on run cancel.
• Validation rejects nested concurrent and loop scopes at graph load time.
• Stall watchdog now resets from the agent log tailer's last-seen event, not just stdout — prevents false stall detection during long LLM reasoning pauses.
• Run failure is recorded in the RunDB on engine error return (previously only terminal nodes wrote their own state).
• Server reconciles in-flight runs as stale on startup to prevent permanently pending entries after a crash.

Breaking changes

• codergen is renamed to agent throughout internal types and test file names. Any code outside this repo that imports internal/attractor/engine types named Codergen* will need to be updated.
• --graph remains the primary flag; --package is additive. No existing graph-based invocations change.
• The /pipelines/ server API is retained as an alias but the canonical path is /runs/.
• internal/attractor/workflows/ now contains HumanGateHandler (moved out of the engine package). Callers that registered a wait.human handler directly into the engine registry will need to use the L2 package instead, or register workflows.NewHumanGateHandler() themselves.

Known gaps and follow-ups

• The supervisor prototype (internal/attractor/workflows/supervisor.go) is present but not wired to any endpoint or scheduler yet.
• Windows process-group kill is a no-op stub (process_group_windows.go); subprocess cleanup on cancel is not guaranteed on Windows.
• The dashboard UI is functional for run inspection but does not yet support launching runs or answering human-gate questions from the browser.
• The CXDB integration boundary is documented (engine/cxdb_hook.go, engine/document CXDB integration boundary) but CXDB remains embedded in the engine; extraction to a separate service is a planned follow-up.
• Gemini support in quick-launch (graph.gemini.dot) exists but the Gemini template is minimal and untested end-to-end.

Test plan

• go build ./... compiles cleanly.
• go test ./... passes (unit and integration tests; integration tests require external services to be available or will skip).
• kilroy attractor run --graph demo/tmux-agent-test/graph.dot --tmux completes without error; check that the tmux session is cleaned up.
• kilroy attractor run --package workflows/quick-launch --input '{"prompt":"say hello"}' --workspace /tmp/ql-test --tmux produces a result.md in the workspace.
• kilroy attractor runs list shows the run; kilroy attractor runs show --latest prints its detail.
• kilroy attractor serve starts; visit http://localhost:8080/ui/ and confirm the dashboard loads and lists the completed runs.
• GET http://localhost:8080/runs/<id>/log returns the NDJSON log; add ?stream=true and confirm it stays open while a run is in progress.
• Run a graph with a loop_count=3 node and verify three distinct attempt rows appear in runs show --json output.
• Cancel a running job mid-execution and confirm no orphan tmux sessions or subprocesses remain (tmux ls -L kilroy should be empty).