Idea: Cate as a spatial command center for a fleet of AI agents

[architawr]

Pitch for discussion, not a finished plan. I'd love maintainers' feedback before writing any code. Some of this likely overlaps with your own roadmap — let's mark what resonates and what doesn't.

Context / why this direction

Cate already has the expensive layer built — agent observation:

• agentScreenDetector + agentScreenDetectorLogic derive each agent's state (running / waitingForInput / finished / notRunning) from the OSC title the CLI writes itself (spinner in title/body), with no byte-rate heuristics.
• agentTitleParser extracts the current task from the title; agentLogos gives the agent name/logo (Claude / Codex / pi).
• useAgentInfoByPanel already serves { state, name, logo } per panelId.
• ParallelWorkTab promoted worktrees to first-class "parallel branches" with "Open agent here", panels are color-bound to their worktree, there's an OS "agent needs input" notification, and a minimap.

In other words, Cate already knows what each agent is doing and where it sits on the canvas. Almost no mainstream IDE shows this. Layer orchestration on top and you get something nobody else does: a swarm of agents, visible and steerable on an infinite canvas. That, to me, is Cate's core wow.

Below is a grouped list. For each: why, what it builds on, rough size (S/M/L).

🛰️ Agent orchestration

1. Mission Control + Inbox (new sidebar tab) — S/M
One tab with all agents across worktrees/workspaces: state, current task (from title), name/logo. At the top, an "inbox": who is waiting on you (waitingForInput). Click a row → camera zooms to that panel on the canvas.

• Why: the real pain at 5+ agents — "who's waiting on me, and where are they physically."
• Builds on: useAgentInfoByPanel, agentScreenDetector, spatial-nav. Low risk — it's an aggregating view over data that already exists.

2. Dispatch — spatial task spawn — L
A "describe a task" card → Cate creates a worktree + terminal, launches an agent with that prompt, and fans the panels out on the canvas. An evolution of "Open agent here".

• Why: the most spectacular scenario — "hand out 3 tasks, watch the swarm work."
• Risk: higher — worktree/PTY lifecycle, passing the prompt into the CLI, layout.

3. Follow-mode / auto-camera — S
Toggle: the canvas flies to whichever agent just flipped to waitingForInput. Queue if several are waiting.

• Why: a pure spatial superpower, tiny in code. Builds on state transitions from agentScreenDetector + the spatial-nav camera.

🗂️ Sidebar tabs / review

4. Auto-diff on finish — M
Agent flips to finished → a diff of its changes pops up next to its panel (GitPanel already exists). In Mission Control, a badge: "N agents have changes ready to review."

• Why: closes the "agent did it → I review it" loop without leaving the canvas.

5. Session Timeline / Activity — M
A timeline of which agent did what, which files it touched, which commits — a mini "time machine" of the session.

6. Spatial Scenes / camera bookmarks — S
Named canvas positions ("frontend cluster", "this bug", "review"), jump via Cmd+1..9.

• Why: on an infinite canvas, navigation is the main pain; this solves it.

Suggested first step

Start with #1 Mission Control + Inbox as the core, designed so #3 (follow-mode), #4 (auto-diff) and #6 (scenes) plug in as small modules around it — a coherent "command center" narrative rather than scattered features, with a low-risk first slice on existing foundations. #2 (Dispatch) is a separate larger milestone once the core settles.

Questions for maintainers

1. Does "command center for a fleet of agents" resonate as direction fedora 44 - cant install it with npm #1, or is your priority elsewhere (collaboration / perf / editor)?
2. Is waitingForInput reliable enough in practice to build an "inbox" UX on, or does it false-fire? (Related: Agent running detection in terminal is inconsistent #140 "Agent running detection in terminal is inconsistent".)
3. Mission Control — a separate sidebar tab, or folded into ParallelWorkTab?
4. What of this is already on your roadmap, so we don't duplicate?

[Anton-Horn]

We had this idea before and even tried it out with a custom CLI and skills. Didn't work out so well. Would only try this again as a lightweight integration on top of features we already have. For example Parallel Work (Worktrees) and one Agent writing to the Session/Workspace File orchestrating the agent team.

[Snailflyer]

The lightweight integration path sounds right here. For Mission Control / Inbox, I would avoid treating waitingForInput alone as the proof; it is the right trigger, but the row still needs a receipt that names the live target.

A row-level shape I would trust:

• panel id, worktree, and current agent title
• state transition source (OSC title, terminal detector, or workspace file)
• last output cursor and last input-request id
• route target for a follow-up command
• state: actionable, observed_only, stale_panel, or needs_rebind

If one agent writes to a Session/Workspace file to orchestrate the team, each file item should still name the live panel/session expected to consume the next action. Otherwise Mission Control can drift into a beautiful project board that is detached from the runtime.

Adjacent reference: I maintain Faryo, a project/mobile workbench for tmux-backed AI coding sessions. The same product boundary shows up there: the project card is not enough; the command has to land in the same live session.
https://github.com/Snailflyer/faryo/releases/tag/v1.0.8