From ab5c492ea84934e88f6ef4fc7733bc97928db35d Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 16:59:23 +0400 Subject: [PATCH 1/8] rfc: agent-composed UI via A2UI protocol Proposes adopting Google's A2UI protocol as the rendering contract between capsules, the agent, and frontends. The agent owns the full layout composition with no predefined zones or hardcoded regions. The TUI is a dumb renderer. Key decisions: - A2UI adopted as-is, no custom extensions - Bus-based component discovery (same pattern as tools) - Agent composes the entire component tree - Natural language layout customization - Frontend-agnostic (CLI/web/Discord all render same descriptions) Tracks astrid#629. --- text/0002-agent-composed-ui.md | 461 +++++++++++++++++++++++++++++++++ 1 file changed, 461 insertions(+) create mode 100644 text/0002-agent-composed-ui.md diff --git a/text/0002-agent-composed-ui.md b/text/0002-agent-composed-ui.md new file mode 100644 index 0000000..3cca572 --- /dev/null +++ b/text/0002-agent-composed-ui.md @@ -0,0 +1,461 @@ +- Feature Name: `agent_composed_ui` +- Start Date: 2026-03-25 +- RFC PR: [rfcs#0000](https://github.com/unicity-astrid/rfcs/pull/0000) +- Tracking Issue: [astrid#629](https://github.com/unicity-astrid/astrid/issues/629) + +# Summary +[summary]: #summary + +Adopt Google's A2UI (Agent-to-User Interface) protocol as the rendering contract between capsules, +the agent, and frontends. The agent owns the full layout composition — there are no predefined zones, +slots, or hardcoded regions. The TUI (and any future frontend) is a dumb renderer that draws whatever +component tree the agent describes. Capsules provide components and data sources; the agent decides +what goes where. + +# Motivation +[motivation]: #motivation + +The current CLI/TUI has a hardcoded layout: chat area, input box, status bar. Colors are hardcoded +for dark terminals. Status indicators (model name, context usage) are fragile because the CLI owns +their rendering but doesn't own the data. There is no way for capsules to present structured UI +(forms, tables, panels) back to the user. And there is no way for the user to customize the layout +without code changes. + +These are all symptoms of the same root problem: **the frontend owns both structure and content.** +The frontend should own neither. Content comes from capsules. Structure comes from the agent. + +The agent is uniquely positioned to compose UI because it understands: +- What capsules are loaded and what components they offer +- What the user is currently doing (coding, chatting, configuring, debugging) +- What the user has asked for ("show me tools on the right", "minimal interface") +- What information is relevant right now vs. noise + +By making the agent the UI composer, we get: +- **Adaptive layouts** — the agent reshapes the UI based on task context +- **Natural language customization** — "put my config on the right" just works +- **No hardcoded structure to maintain** — the default layout is just the agent's starting opinion +- **Multi-frontend for free** — each frontend renders A2UI components natively; the agent's layout + description is frontend-agnostic +- **Personality-driven defaults** — different agent identities can have different default layouts + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## For capsule developers + +Your capsule exports **components** — UI building blocks that the agent can place anywhere. You +declare what you can provide; you do not decide where it goes. + +A component is registered via IPC and described as an A2UI component type with a data model: + +```rust +#[astrid::ui_component("model-status")] +fn model_status() -> UiComponent { + UiComponent::new("model-status") + .catalog_type("Text") + .data_model(json!({ + "model": "unknown", + "context_usage": 0.0 + })) +} +``` + +The agent discovers available components and composes them into a layout. Your capsule updates its +data model via IPC events — the component re-renders automatically through A2UI's data binding. + +```rust +// React capsule updates model status after each LLM response +sdk::publish("ui.v1.data_model.update", json!({ + "surfaceId": "root", + "patch": [ + { "op": "replace", "path": "/model-status/model", "value": "gpt-5.4" }, + { "op": "replace", "path": "/model-status/context_usage", "value": 0.47 } + ] +})); +``` + +## For users + +The UI adapts to you. On first launch, you get a sensible default layout. But you can reshape it: + +``` +You: show me loaded tools in a sidebar +Agent: [restructures layout — adds a right column with the tool list] + +You: actually, put that at the bottom instead +Agent: [moves tool list below the chat area] + +You: minimal mode — just chat +Agent: [collapses to chat + input only] +``` + +Your layout preferences persist across sessions. Different agent personalities may have different +default layouts. + +## For frontend implementors + +Your frontend is a **dumb A2UI renderer**. You receive a component tree and render it using your +native widget toolkit. You do not decide layout, structure, or what content appears where. + +Responsibilities: +- Map A2UI component types to native widgets (`Row` → flex row, `Text` → text widget, etc.) +- Handle user input events and route them back via IPC +- Advertise your component catalog (what A2UI types you can render) +- Gracefully degrade for unsupported component types (render as text or omit) + +The CLI/TUI maps A2UI to ratatui. A future web frontend would map to React/HTML. A Discord frontend +would map to Discord components. Same agent layout description, different native rendering. + +## The agent's role + +The agent owns a single root A2UI surface that IS the entire screen. It composes the full component +tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) and capsule-provided +components. + +On startup, the agent emits a default layout — this looks like a traditional TUI: + +```json +{ + "createSurface": { + "surfaceId": "root", + "catalogId": "astrid/cli/v1" + } +} +``` + +```json +{ + "updateComponents": { + "surfaceId": "root", + "components": [ + { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, + { "id": "status", "component": "Row", "children": ["breadcrumb", "model-status"], "justify": "spaceBetween" }, + { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, + { "id": "model-status", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, + { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, + { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, + { "id": "input", "component": "TextField", "placeholder": "Message..." } + ] + } +} +``` + +This is the current TUI — recreated purely from A2UI primitives. But the agent can restructure it +at any time. When the user asks for a sidebar, the agent replaces the `main` component with a `Row` +containing the chat column and a new sidebar column. No code change. No predefined zones. Just a +different component tree. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## Protocol + +We adopt A2UI v0.10 (or latest stable at implementation time) as the wire protocol. No modifications +to the core spec. The protocol defines: + +### Message types (A2UI standard) + +| Message | Purpose | +|---------|---------| +| `createSurface` | Create a new rendering surface with a catalog reference | +| `updateComponents` | Add, modify, or remove components in a surface | +| `updateDataModel` | Patch the surface's data model (RFC 6902 JSON Patch) | +| `deleteSurface` | Destroy a surface | + +### Component catalog + +The CLI frontend advertises a catalog of A2UI component types it can render. The catalog is the +security boundary — only allowlisted types are rendered. The initial CLI catalog: + +| A2UI Component | ratatui Mapping | Notes | +|----------------|-----------------|-------| +| `Row` | `Layout::horizontal` | `justify`, `align`, `weight` supported | +| `Column` | `Layout::vertical` | `justify`, `align`, `weight` supported | +| `Text` | `Paragraph` | `variant` maps to styling (h1=bold, caption=dim) | +| `List` | `List` / scrollable | Vertical or horizontal | +| `Card` | `Block` with borders | Single child | +| `Tabs` | `Tabs` widget | Tab switching via keybind | +| `Button` | Keybind-triggered action | Rendered as `[label]`, activated by key | +| `TextField` | Input widget | Single-line text entry | +| `TextArea` | Multi-line input | For message composition | +| `ProgressBar` | `Gauge` | 0.0–1.0 value | +| `Table` | `Table` | Headers + rows | +| `Badge` | Styled `Span` | Inline colored label | +| `Select` | Popup list | Arrow-key selection | +| `Checkbox` | Toggle | `[x]` / `[ ]` | +| `Modal` | Overlay `Clear` + centered `Block` | Focus trap | + +Unsupported components degrade to `Text` showing the component's text content, or are omitted. + +### IPC integration + +A2UI messages travel over the Astrid IPC event bus under the `ui.v1.*` topic namespace: + +| IPC Topic | Payload | Direction | +|-----------|---------|-----------| +| `ui.v1.surface.create` | A2UI `createSurface` | Agent → Frontend | +| `ui.v1.components.update` | A2UI `updateComponents` | Agent → Frontend | +| `ui.v1.data_model.update` | A2UI `updateDataModel` | Capsule/Agent → Frontend | +| `ui.v1.surface.delete` | A2UI `deleteSurface` | Agent → Frontend | +| `ui.v1.action` | `{ surfaceId, componentId, action, data }` | Frontend → Agent | +| `ui.v1.catalog.query` | `{}` | Agent → Frontend | +| `ui.v1.catalog.response` | `{ components: [...] }` | Frontend → Agent | + +### Component discovery — bus-based, same pattern as tools + +UI component discovery follows the exact same pattern as tool discovery. Today, tools work like this: + +1. Prompt-builder triggers `tool.v1.request.describe` (hook fan-out to all capsules) +2. Each capsule's `tool_describe` interceptor responds with its tool schemas +3. Prompt-builder collects and deduplicates + +UI components work identically: + +1. Agent (or a coordinator) triggers `ui.v1.request.describe` (hook fan-out) +2. Each capsule's `ui_describe` interceptor responds with its available components +3. Agent collects the component registry + +No manifest extension is needed. The `Capsule.toml` declares the interceptor and IPC capabilities +(just like tools do today), and the actual component definitions are returned at runtime: + +```toml +# In Capsule.toml — same pattern as tool_describe +[[interceptor]] +event = "ui.v1.request.describe" +action = "ui_describe" + +[capabilities] +ipc_publish = ["ui.v1.response.describe.*"] +``` + +The capsule's `ui_describe` handler returns what it can provide: + +```json +{ + "components": [ + { + "id": "identity-config-form", + "display_name": "Identity Configuration", + "description": "Agent identity settings — callsign, class, tone, backstory", + "schema": { + "properties": { + "callsign": { "type": "string", "description": "Agent's display name" }, + "class": { "type": "string", "description": "Agent class/role" }, + "tone": { "type": "string", "description": "Communication style" } + } + }, + "data_topic": "identity.v1.config", + "actions": ["save", "reset"] + }, + { + "id": "identity-status", + "display_name": "Identity Status", + "description": "Current agent identity — name and class as text", + "schema": { + "properties": { + "callsign": { "type": "string" }, + "class": { "type": "string" } + } + }, + "data_topic": "identity.v1.status" + } + ] +} +``` + +This tells the agent: "I have a config form and a status widget you can place anywhere. Here are +their schemas and the IPC topics for live data." The agent decides if, when, and where to use them. + +A capsule can offer multiple components of different shapes — a full config form for a sidebar, a +compact status line for a header bar, a detailed dashboard for a modal. Same data, different +presentations. The agent picks what fits the current layout. + +### Agent composition flow + +1. **Boot**: Frontend publishes `ui.v1.catalog.response` with supported A2UI component types + (what the renderer can draw: Row, Column, Text, TextField, etc.). +2. **Discovery**: Agent triggers `ui.v1.request.describe` — all capsules respond with their + available components (what data/interactions they can provide). +3. **Compose**: Agent maps capsule components onto A2UI primitives and emits `createSurface` + + `updateComponents` to render the default layout. The agent decides the tree structure. +4. **React**: On user input ("show tools sidebar"), agent emits new `updateComponents` with a + restructured tree. Capsule components move, resize, appear, or disappear. +5. **Live data**: Capsules publish `ui.v1.data_model.update` to push new values to their + components. Frontend re-renders affected components automatically via A2UI data binding. +6. **Persist**: Agent saves the current layout composition to session state. On reconnect, it + restores the last layout rather than resetting to default. + +There are two catalogs in play: + +- **Frontend catalog** (A2UI standard): what primitive types the renderer supports (Row, Text, etc.) +- **Capsule component registry** (Astrid-specific): what data/interactions capsules can provide + +The agent bridges the two — it takes capsule components and composes them into A2UI primitives that +the frontend can render. + +### Pre-baked default + +The agent's system prompt includes a default layout composition. This is what renders on first boot +before any user customization. It replicates the current hardcoded TUI: + +- Top row: breadcrumb path (left), model name + context usage (right) +- Main area: scrollable message list +- Bottom: text input +- Status indicators update via data model patches from the react capsule + +This default is not special. It is not hardcoded in the frontend. It is just the agent's opening +move, expressed as A2UI components. The agent can replace it entirely at any time. + +### Error handling + +- **Unknown component type**: Frontend logs a warning and renders as `Text` with the component's + `id` as content, or omits. The agent sees a `ui.v1.error` event and can adapt. +- **Invalid tree structure**: Frontend validates parent-child relationships (e.g., `weight` only + valid as direct child of `Row`/`Column`). Invalid structures are rejected with a + `ui.v1.error` event; the last valid tree is retained. +- **Agent not ready**: Frontend renders a minimal loading state (spinner + "composing...") until + the first `createSurface` arrives. This is the only hardcoded UI in the frontend. + +## Capability gating + +A2UI surface creation and component updates are capability-gated. Only the orchestrating agent (or +capsules with explicit `ui.surface.write` capability) can emit `createSurface` and +`updateComponents`. Any capsule can emit `updateDataModel` for components it owns (scoped by +component ID prefix matching its capsule namespace). + +This prevents a rogue capsule from hijacking the layout while still allowing capsules to update +their own data. + +# Drawbacks +[drawbacks]: #drawbacks + +- **Agent latency on boot**: The first render depends on the agent composing a layout. Until the + LLM responds, users see a loading state. This adds perceived startup time. Mitigation: cache + the last layout and restore it immediately; the agent can update it once ready. + +- **LLM token cost**: The component registry, current layout state, and A2UI grammar all consume + context window tokens on every agent turn. For simple chat interactions, this is overhead that + a hardcoded TUI doesn't have. + +- **Layout instability**: A poorly prompted agent could produce jarring layout changes mid- + conversation. Mitigation: layout changes should only happen on explicit user request or major + context shifts, not on every turn. The system prompt should instruct the agent to be + conservative with layout changes. + +- **A2UI dependency**: We take a dependency on an external spec (Google, Apache 2.0). If the spec + evolves in directions incompatible with our needs, we'd need to fork or freeze at a version. + Mitigation: pin to a specific version; the spec is Apache 2.0 so forking is always an option. + +- **Testing complexity**: UI tests must now account for dynamic layouts rather than fixed structure. + Snapshot testing becomes harder when the layout can change between runs. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +## Why A2UI over a custom protocol? + +A2UI is a well-designed, open-source (Apache 2.0) protocol that solves the component description +problem. Its component catalog, surface lifecycle, data binding, and security model are exactly what +we need. Building a custom protocol would duplicate this work and cut us off from potential ecosystem +interop. + +## Why not MCP Apps? + +MCP Apps (Anthropic + OpenAI) sends HTML/JS rendered in sandboxed iframes. This is web-centric and +fundamentally incompatible with terminal rendering. A2UI's declarative approach maps naturally to any +native widget toolkit, including ratatui. + +## Why not predefined zones/slots? + +Zones impose structure that the agent should own. A zone system means every frontend must agree on a +zone vocabulary, every capsule must target specific zones, and layout customization is limited to +"which zones are visible." The agent-composed approach is strictly more flexible — the agent CAN +create a zone-like layout if it chooses to, but it is not constrained to one. + +## Why not keep the hardcoded TUI? + +The hardcoded TUI cannot adapt to context, cannot be customized by users without code changes, +creates tight coupling between the CLI and capsule data, and must be maintained as a parallel +rendering system alongside any future A2UI support. Starting from A2UI avoids this dual-maintenance +problem. + +## What if the agent produces bad layouts? + +The frontend validates the component tree before rendering. Invalid structures are rejected and the +last valid tree is retained. The system prompt instructs the agent to be conservative with layout +changes. And the user can always say "reset to default" to get back to the pre-baked layout. + +# Prior art +[prior-art]: #prior-art + +- **A2UI (Google)** — The protocol we adopt. Declarative JSON component descriptions, catalog-based + security, surface lifecycle management. v0.8+ public preview, Apache 2.0. Our contribution is + the agent-as-composer pattern on top of A2UI's primitives. + +- **MCP Apps (Anthropic + OpenAI)** — HTML/JS in sandboxed iframes. Richer rendering but requires a + web runtime. Not viable for terminal UIs. The "opaque payload" approach vs. A2UI's "declarative + catalog" approach. + +- **AG-UI (CopilotKit)** — Event-based protocol for real-time agent-frontend communication. Covers + streaming, shared state, and human-in-the-loop. Complementary to A2UI (AG-UI = transport, A2UI = + component format). Worth evaluating as the IPC transport layer in future work. + +- **Zellij / tmux** — Terminal multiplexers that let users compose panes. Similar concept (user + decides layout) but manual rather than agent-driven. Our approach is the AI-native version of + terminal pane composition. + +- **Emacs** — The original "editor as a Lisp-driven UI canvas." Windows, buffers, and frames are + composed programmatically. Astrid's agent-composed UI is philosophically similar, with the agent + replacing Emacs Lisp as the composition layer. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +- **A2UI version pinning**: Which exact A2UI version do we adopt? v0.10 is the latest draft but not + yet stable. We may need to pin v0.8 (public preview) and upgrade later. + +- **Streaming component trees**: A2UI supports incremental JSONL streaming. How does this interact + with ratatui's double-buffered rendering? Do we render partial trees or wait for a complete update? + +- **Focus management**: When the agent restructures the layout, where does keyboard focus go? The + agent should be able to specify a focus target, but A2UI doesn't have a focus primitive. We may + need a small extension or convention (e.g., a `focused: true` property). + +- **Layout persistence format**: How is the "last known layout" serialized for session restore? + The raw A2UI component list, or a higher-level representation? + +- **Chat message rendering**: Is the message stream itself an A2UI `List` of `Text` components, or + does it remain a special-cased renderer? Full A2UI message rendering enables richer messages + (inline tables, cards, images on supported terminals) but adds complexity. + +- **Input handling**: A2UI defines actions routed from client to agent. How does this map to the + existing IPC event system? Is a keystroke in a `TextField` an A2UI action or an IPC event? + +- **Performance budget**: How many components can ratatui render at 60fps? Large component trees + (hundreds of items in a tool list) may need virtualization. + +# Future possibilities +[future-possibilities]: #future-possibilities + +- **Saveable layouts**: Users save named layouts ("coding", "debugging", "minimal") and switch + between them. The agent restores a saved layout by name. + +- **Layout marketplace**: Capsule authors publish recommended layouts alongside their capsules. + "Install the monitoring capsule and its dashboard layout." + +- **Multi-surface**: Instead of one root surface, multiple surfaces for multi-monitor or tabbed + interfaces. Each tab is an independent A2UI surface. + +- **Sixel/Kitty graphics**: Terminals that support image protocols could render A2UI `Image` + components as inline graphics. The catalog advertises this capability. + +- **Web frontend**: A web-based Astrid frontend renders A2UI components as React/HTML. The same + agent layout description works across CLI and web — the rendering quality just improves. + +- **Collaborative UI**: Multiple users connected to the same session see the same layout. The agent + composes for the group, not just one user. + +- **Theme as data model**: The agent controls the color palette via A2UI data model updates. "Dark + mode", "light mode", and "high contrast" become data model patches, not hardcoded theme structs. + This solves the terminal background detection problem (#626) — the agent detects or asks, then + patches the theme. From 50b6b62d849802e2926c3d421739b02e44e9436a Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 17:16:08 +0400 Subject: [PATCH 2/8] fix: rename RFC to 0000 (number assigned at merge time) --- text/{0002-agent-composed-ui.md => 0000-agent-composed-ui.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename text/{0002-agent-composed-ui.md => 0000-agent-composed-ui.md} (99%) diff --git a/text/0002-agent-composed-ui.md b/text/0000-agent-composed-ui.md similarity index 99% rename from text/0002-agent-composed-ui.md rename to text/0000-agent-composed-ui.md index 3cca572..408a545 100644 --- a/text/0002-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -1,6 +1,6 @@ - Feature Name: `agent_composed_ui` - Start Date: 2026-03-25 -- RFC PR: [rfcs#0000](https://github.com/unicity-astrid/rfcs/pull/0000) +- RFC PR: [rfcs#25](https://github.com/unicity-astrid/rfcs/pull/25) - Tracking Issue: [astrid#629](https://github.com/unicity-astrid/astrid/issues/629) # Summary From 83d0fef4c66cf1d143b8ea26190169f3cee235af Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 17:19:34 +0400 Subject: [PATCH 3/8] rfc: strip current-system descriptions, focus on proposal --- text/0000-agent-composed-ui.md | 293 +++++++++++++-------------------- 1 file changed, 111 insertions(+), 182 deletions(-) diff --git a/text/0000-agent-composed-ui.md b/text/0000-agent-composed-ui.md index 408a545..747ed9a 100644 --- a/text/0000-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -8,21 +8,14 @@ Adopt Google's A2UI (Agent-to-User Interface) protocol as the rendering contract between capsules, the agent, and frontends. The agent owns the full layout composition — there are no predefined zones, -slots, or hardcoded regions. The TUI (and any future frontend) is a dumb renderer that draws whatever -component tree the agent describes. Capsules provide components and data sources; the agent decides -what goes where. +slots, or hardcoded regions. Frontends are dumb renderers that draw whatever component tree the agent +describes. Capsules provide components and data sources; the agent decides what goes where. # Motivation [motivation]: #motivation -The current CLI/TUI has a hardcoded layout: chat area, input box, status bar. Colors are hardcoded -for dark terminals. Status indicators (model name, context usage) are fragile because the CLI owns -their rendering but doesn't own the data. There is no way for capsules to present structured UI -(forms, tables, panels) back to the user. And there is no way for the user to customize the layout -without code changes. - -These are all symptoms of the same root problem: **the frontend owns both structure and content.** -The frontend should own neither. Content comes from capsules. Structure comes from the agent. +The frontend should not own structure or content. Content comes from capsules. Structure comes from +the agent. The agent is uniquely positioned to compose UI because it understands: - What capsules are loaded and what components they offer @@ -30,13 +23,15 @@ The agent is uniquely positioned to compose UI because it understands: - What the user has asked for ("show me tools on the right", "minimal interface") - What information is relevant right now vs. noise -By making the agent the UI composer, we get: +By making the agent the UI composer: - **Adaptive layouts** — the agent reshapes the UI based on task context - **Natural language customization** — "put my config on the right" just works -- **No hardcoded structure to maintain** — the default layout is just the agent's starting opinion +- **No hardcoded structure** — the default layout is just the agent's starting opinion - **Multi-frontend for free** — each frontend renders A2UI components natively; the agent's layout description is frontend-agnostic - **Personality-driven defaults** — different agent identities can have different default layouts +- **Theme as data** — colors and styling become data model patches the agent controls, adapting to + terminal capabilities rather than assuming a dark background # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -46,7 +41,7 @@ By making the agent the UI composer, we get: Your capsule exports **components** — UI building blocks that the agent can place anywhere. You declare what you can provide; you do not decide where it goes. -A component is registered via IPC and described as an A2UI component type with a data model: +A component is registered via a `ui_describe` interceptor (same pattern as `tool_describe`): ```rust #[astrid::ui_component("model-status")] @@ -61,10 +56,9 @@ fn model_status() -> UiComponent { ``` The agent discovers available components and composes them into a layout. Your capsule updates its -data model via IPC events — the component re-renders automatically through A2UI's data binding. +data model via IPC — the component re-renders automatically through A2UI data binding: ```rust -// React capsule updates model status after each LLM response sdk::publish("ui.v1.data_model.update", json!({ "surfaceId": "root", "patch": [ @@ -74,6 +68,9 @@ sdk::publish("ui.v1.data_model.update", json!({ })); ``` +A capsule can offer multiple components of different shapes — a full config form, a compact status +line, a detailed dashboard. Same data, different presentations. The agent picks what fits. + ## For users The UI adapts to you. On first launch, you get a sensible default layout. But you can reshape it: @@ -89,13 +86,12 @@ You: minimal mode — just chat Agent: [collapses to chat + input only] ``` -Your layout preferences persist across sessions. Different agent personalities may have different -default layouts. +Layout preferences persist across sessions. Different agent personalities may have different defaults. ## For frontend implementors Your frontend is a **dumb A2UI renderer**. You receive a component tree and render it using your -native widget toolkit. You do not decide layout, structure, or what content appears where. +native widget toolkit. Responsibilities: - Map A2UI component types to native widgets (`Row` → flex row, `Text` → text widget, etc.) @@ -103,16 +99,13 @@ Responsibilities: - Advertise your component catalog (what A2UI types you can render) - Gracefully degrade for unsupported component types (render as text or omit) -The CLI/TUI maps A2UI to ratatui. A future web frontend would map to React/HTML. A Discord frontend -would map to Discord components. Same agent layout description, different native rendering. - ## The agent's role The agent owns a single root A2UI surface that IS the entire screen. It composes the full component tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) and capsule-provided components. -On startup, the agent emits a default layout — this looks like a traditional TUI: +On startup, the agent emits a default layout: ```json { @@ -129,9 +122,9 @@ On startup, the agent emits a default layout — this looks like a traditional T "surfaceId": "root", "components": [ { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, - { "id": "status", "component": "Row", "children": ["breadcrumb", "model-status"], "justify": "spaceBetween" }, + { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info"], "justify": "spaceBetween" }, { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, - { "id": "model-status", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, + { "id": "model-info", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, { "id": "input", "component": "TextField", "placeholder": "Message..." } @@ -140,32 +133,33 @@ On startup, the agent emits a default layout — this looks like a traditional T } ``` -This is the current TUI — recreated purely from A2UI primitives. But the agent can restructure it -at any time. When the user asks for a sidebar, the agent replaces the `main` component with a `Row` -containing the chat column and a new sidebar column. No code change. No predefined zones. Just a -different component tree. +This is not special. It is not hardcoded. It is the agent's opening move. When the user asks for a +sidebar, the agent restructures the tree — replacing `main` with a `Row` containing the chat column +and a new sidebar column. No code change. No predefined zones. Just a different component tree. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation ## Protocol -We adopt A2UI v0.10 (or latest stable at implementation time) as the wire protocol. No modifications -to the core spec. The protocol defines: +Adopt A2UI (latest stable at implementation time) as the wire protocol. No modifications to the +core spec. ### Message types (A2UI standard) | Message | Purpose | |---------|---------| -| `createSurface` | Create a new rendering surface with a catalog reference | +| `createSurface` | Create a rendering surface with a catalog reference | | `updateComponents` | Add, modify, or remove components in a surface | | `updateDataModel` | Patch the surface's data model (RFC 6902 JSON Patch) | | `deleteSurface` | Destroy a surface | ### Component catalog -The CLI frontend advertises a catalog of A2UI component types it can render. The catalog is the -security boundary — only allowlisted types are rendered. The initial CLI catalog: +Each frontend advertises a catalog of A2UI component types it can render. The catalog is the +security boundary — only allowlisted types are rendered. + +CLI catalog (A2UI → ratatui mapping): | A2UI Component | ratatui Mapping | Notes | |----------------|-----------------|-------| @@ -185,11 +179,11 @@ security boundary — only allowlisted types are rendered. The initial CLI catal | `Checkbox` | Toggle | `[x]` / `[ ]` | | `Modal` | Overlay `Clear` + centered `Block` | Focus trap | -Unsupported components degrade to `Text` showing the component's text content, or are omitted. +Unsupported components degrade to `Text` or are omitted. -### IPC integration +### IPC topics -A2UI messages travel over the Astrid IPC event bus under the `ui.v1.*` topic namespace: +A2UI messages travel over the IPC event bus under `ui.v1.*`: | IPC Topic | Payload | Direction | |-----------|---------|-----------| @@ -201,25 +195,17 @@ A2UI messages travel over the Astrid IPC event bus under the `ui.v1.*` topic nam | `ui.v1.catalog.query` | `{}` | Agent → Frontend | | `ui.v1.catalog.response` | `{ components: [...] }` | Frontend → Agent | -### Component discovery — bus-based, same pattern as tools +### Component discovery -UI component discovery follows the exact same pattern as tool discovery. Today, tools work like this: +UI component discovery follows the same bus-based pattern as tool discovery: -1. Prompt-builder triggers `tool.v1.request.describe` (hook fan-out to all capsules) -2. Each capsule's `tool_describe` interceptor responds with its tool schemas -3. Prompt-builder collects and deduplicates - -UI components work identically: - -1. Agent (or a coordinator) triggers `ui.v1.request.describe` (hook fan-out) +1. Agent triggers `ui.v1.request.describe` (hook fan-out to all capsules) 2. Each capsule's `ui_describe` interceptor responds with its available components 3. Agent collects the component registry -No manifest extension is needed. The `Capsule.toml` declares the interceptor and IPC capabilities -(just like tools do today), and the actual component definitions are returned at runtime: +The `Capsule.toml` declares the interceptor and IPC capabilities: ```toml -# In Capsule.toml — same pattern as tool_describe [[interceptor]] event = "ui.v1.request.describe" action = "ui_describe" @@ -228,7 +214,7 @@ action = "ui_describe" ipc_publish = ["ui.v1.response.describe.*"] ``` -The capsule's `ui_describe` handler returns what it can provide: +The `ui_describe` handler returns what the capsule can provide: ```json { @@ -250,7 +236,7 @@ The capsule's `ui_describe` handler returns what it can provide: { "id": "identity-status", "display_name": "Identity Status", - "description": "Current agent identity — name and class as text", + "description": "Current agent identity — compact name and class display", "schema": { "properties": { "callsign": { "type": "string" }, @@ -263,199 +249,142 @@ The capsule's `ui_describe` handler returns what it can provide: } ``` -This tells the agent: "I have a config form and a status widget you can place anywhere. Here are -their schemas and the IPC topics for live data." The agent decides if, when, and where to use them. - -A capsule can offer multiple components of different shapes — a full config form for a sidebar, a -compact status line for a header bar, a detailed dashboard for a modal. Same data, different -presentations. The agent picks what fits the current layout. +The agent sees: "identity capsule has a config form and a status widget. Here are their schemas and +data topics." The agent decides if, when, and where to place them. -### Agent composition flow - -1. **Boot**: Frontend publishes `ui.v1.catalog.response` with supported A2UI component types - (what the renderer can draw: Row, Column, Text, TextField, etc.). -2. **Discovery**: Agent triggers `ui.v1.request.describe` — all capsules respond with their - available components (what data/interactions they can provide). -3. **Compose**: Agent maps capsule components onto A2UI primitives and emits `createSurface` + - `updateComponents` to render the default layout. The agent decides the tree structure. -4. **React**: On user input ("show tools sidebar"), agent emits new `updateComponents` with a - restructured tree. Capsule components move, resize, appear, or disappear. -5. **Live data**: Capsules publish `ui.v1.data_model.update` to push new values to their - components. Frontend re-renders affected components automatically via A2UI data binding. -6. **Persist**: Agent saves the current layout composition to session state. On reconnect, it - restores the last layout rather than resetting to default. - -There are two catalogs in play: +Two catalogs are in play: - **Frontend catalog** (A2UI standard): what primitive types the renderer supports (Row, Text, etc.) -- **Capsule component registry** (Astrid-specific): what data/interactions capsules can provide - -The agent bridges the two — it takes capsule components and composes them into A2UI primitives that -the frontend can render. +- **Capsule component registry** (bus-discovered): what data/interactions capsules provide -### Pre-baked default +The agent bridges the two — mapping capsule components onto A2UI primitives. -The agent's system prompt includes a default layout composition. This is what renders on first boot -before any user customization. It replicates the current hardcoded TUI: +### Composition flow -- Top row: breadcrumb path (left), model name + context usage (right) -- Main area: scrollable message list -- Bottom: text input -- Status indicators update via data model patches from the react capsule - -This default is not special. It is not hardcoded in the frontend. It is just the agent's opening -move, expressed as A2UI components. The agent can replace it entirely at any time. +1. **Boot**: Frontend publishes `ui.v1.catalog.response` with supported A2UI component types. +2. **Discovery**: Agent triggers `ui.v1.request.describe` — capsules respond with available + components. +3. **Compose**: Agent emits `createSurface` + `updateComponents` with the default layout. +4. **Interact**: On user input ("show tools sidebar"), agent emits `updateComponents` with a + restructured tree. +5. **Live data**: Capsules publish `ui.v1.data_model.update` to push values to their components. + Frontend re-renders via A2UI data binding. +6. **Persist**: Agent saves layout composition to session state. On reconnect, restores last layout. ### Error handling -- **Unknown component type**: Frontend logs a warning and renders as `Text` with the component's - `id` as content, or omits. The agent sees a `ui.v1.error` event and can adapt. -- **Invalid tree structure**: Frontend validates parent-child relationships (e.g., `weight` only - valid as direct child of `Row`/`Column`). Invalid structures are rejected with a - `ui.v1.error` event; the last valid tree is retained. -- **Agent not ready**: Frontend renders a minimal loading state (spinner + "composing...") until - the first `createSurface` arrives. This is the only hardcoded UI in the frontend. +- **Unknown component type**: Frontend renders as `Text` with the component's `id`, or omits. + Agent sees `ui.v1.error` and can adapt. +- **Invalid tree structure**: Rejected with `ui.v1.error`; last valid tree retained. +- **Agent not ready**: Frontend renders a minimal loading state until the first `createSurface` + arrives. This is the only hardcoded UI in the frontend. -## Capability gating +### Capability gating -A2UI surface creation and component updates are capability-gated. Only the orchestrating agent (or -capsules with explicit `ui.surface.write` capability) can emit `createSurface` and -`updateComponents`. Any capsule can emit `updateDataModel` for components it owns (scoped by -component ID prefix matching its capsule namespace). +Surface creation and component updates are capability-gated. Only the orchestrating agent (or +capsules with `ui.surface.write` capability) can emit `createSurface` and `updateComponents`. +Any capsule can emit `updateDataModel` for components it owns (scoped by component ID prefix +matching its capsule namespace). -This prevents a rogue capsule from hijacking the layout while still allowing capsules to update -their own data. +This prevents a rogue capsule from hijacking the layout while allowing capsules to update their data. # Drawbacks [drawbacks]: #drawbacks -- **Agent latency on boot**: The first render depends on the agent composing a layout. Until the - LLM responds, users see a loading state. This adds perceived startup time. Mitigation: cache - the last layout and restore it immediately; the agent can update it once ready. +- **Agent latency on boot**: First render depends on the agent composing a layout. Mitigation: cache + the last layout and restore it immediately; the agent updates once ready. -- **LLM token cost**: The component registry, current layout state, and A2UI grammar all consume - context window tokens on every agent turn. For simple chat interactions, this is overhead that - a hardcoded TUI doesn't have. +- **LLM token cost**: Component registry, layout state, and A2UI grammar consume context window + tokens. For simple chat, this is overhead. -- **Layout instability**: A poorly prompted agent could produce jarring layout changes mid- - conversation. Mitigation: layout changes should only happen on explicit user request or major - context shifts, not on every turn. The system prompt should instruct the agent to be - conservative with layout changes. +- **Layout instability**: A poorly prompted agent could produce jarring layout changes. Mitigation: + system prompt instructs conservative layout behavior; changes only on explicit user request. -- **A2UI dependency**: We take a dependency on an external spec (Google, Apache 2.0). If the spec - evolves in directions incompatible with our needs, we'd need to fork or freeze at a version. - Mitigation: pin to a specific version; the spec is Apache 2.0 so forking is always an option. +- **A2UI dependency**: External spec (Google, Apache 2.0). Mitigation: pin to a version; Apache 2.0 + permits forking. -- **Testing complexity**: UI tests must now account for dynamic layouts rather than fixed structure. - Snapshot testing becomes harder when the layout can change between runs. +- **Testing complexity**: Dynamic layouts make snapshot testing harder. # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives -## Why A2UI over a custom protocol? +## Why A2UI? -A2UI is a well-designed, open-source (Apache 2.0) protocol that solves the component description -problem. Its component catalog, surface lifecycle, data binding, and security model are exactly what -we need. Building a custom protocol would duplicate this work and cut us off from potential ecosystem -interop. +A2UI solves the component description problem with a well-designed catalog, surface lifecycle, data +binding, and security model. Building a custom protocol would duplicate this work. A2UI is +Apache 2.0, open-source, and has ecosystem adoption. ## Why not MCP Apps? -MCP Apps (Anthropic + OpenAI) sends HTML/JS rendered in sandboxed iframes. This is web-centric and -fundamentally incompatible with terminal rendering. A2UI's declarative approach maps naturally to any -native widget toolkit, including ratatui. +MCP Apps sends HTML/JS in sandboxed iframes — web-centric, incompatible with terminal rendering. +A2UI's declarative approach maps to any native widget toolkit. ## Why not predefined zones/slots? -Zones impose structure that the agent should own. A zone system means every frontend must agree on a -zone vocabulary, every capsule must target specific zones, and layout customization is limited to -"which zones are visible." The agent-composed approach is strictly more flexible — the agent CAN -create a zone-like layout if it chooses to, but it is not constrained to one. - -## Why not keep the hardcoded TUI? - -The hardcoded TUI cannot adapt to context, cannot be customized by users without code changes, -creates tight coupling between the CLI and capsule data, and must be maintained as a parallel -rendering system alongside any future A2UI support. Starting from A2UI avoids this dual-maintenance -problem. +Zones impose structure the agent should own. The agent CAN create a zone-like layout if it chooses, +but is not constrained to one. Structure is an agent choice, not a system constraint. ## What if the agent produces bad layouts? -The frontend validates the component tree before rendering. Invalid structures are rejected and the -last valid tree is retained. The system prompt instructs the agent to be conservative with layout -changes. And the user can always say "reset to default" to get back to the pre-baked layout. +The frontend validates component trees. Invalid structures are rejected; the last valid tree is +retained. Users can always say "reset to default." # Prior art [prior-art]: #prior-art - **A2UI (Google)** — The protocol we adopt. Declarative JSON component descriptions, catalog-based - security, surface lifecycle management. v0.8+ public preview, Apache 2.0. Our contribution is - the agent-as-composer pattern on top of A2UI's primitives. + security, surface lifecycle. Apache 2.0. Our contribution is the agent-as-composer pattern. - **MCP Apps (Anthropic + OpenAI)** — HTML/JS in sandboxed iframes. Richer rendering but requires a - web runtime. Not viable for terminal UIs. The "opaque payload" approach vs. A2UI's "declarative - catalog" approach. + web runtime. Not viable for terminals. -- **AG-UI (CopilotKit)** — Event-based protocol for real-time agent-frontend communication. Covers - streaming, shared state, and human-in-the-loop. Complementary to A2UI (AG-UI = transport, A2UI = - component format). Worth evaluating as the IPC transport layer in future work. +- **AG-UI (CopilotKit)** — Event-based agent-frontend streaming protocol. Complementary to A2UI + (transport layer vs. component format). Worth evaluating for IPC transport in future work. -- **Zellij / tmux** — Terminal multiplexers that let users compose panes. Similar concept (user - decides layout) but manual rather than agent-driven. Our approach is the AI-native version of - terminal pane composition. +- **Zellij / tmux** — Terminal multiplexers with user-composed panes. Same concept but manual. This + RFC is the AI-native version of terminal pane composition. -- **Emacs** — The original "editor as a Lisp-driven UI canvas." Windows, buffers, and frames are - composed programmatically. Astrid's agent-composed UI is philosophically similar, with the agent - replacing Emacs Lisp as the composition layer. +- **Emacs** — Editor as a Lisp-driven UI canvas. Windows, buffers, and frames composed + programmatically. Philosophically similar, with the agent replacing Emacs Lisp. # Unresolved questions [unresolved-questions]: #unresolved-questions -- **A2UI version pinning**: Which exact A2UI version do we adopt? v0.10 is the latest draft but not - yet stable. We may need to pin v0.8 (public preview) and upgrade later. +- **A2UI version pinning**: Which version do we adopt? v0.10 is latest draft. May need to pin + v0.8 (public preview) and upgrade. -- **Streaming component trees**: A2UI supports incremental JSONL streaming. How does this interact - with ratatui's double-buffered rendering? Do we render partial trees or wait for a complete update? +- **Streaming component trees**: How does incremental JSONL interact with double-buffered + rendering? Render partial trees or wait for complete updates? -- **Focus management**: When the agent restructures the layout, where does keyboard focus go? The - agent should be able to specify a focus target, but A2UI doesn't have a focus primitive. We may - need a small extension or convention (e.g., a `focused: true` property). +- **Focus management**: When the agent restructures the layout, where does keyboard focus go? + A2UI lacks a focus primitive. May need a convention (e.g., `focused: true` property). -- **Layout persistence format**: How is the "last known layout" serialized for session restore? - The raw A2UI component list, or a higher-level representation? +- **Layout persistence**: How is the last-known layout serialized? Raw A2UI component list or a + higher-level representation? -- **Chat message rendering**: Is the message stream itself an A2UI `List` of `Text` components, or - does it remain a special-cased renderer? Full A2UI message rendering enables richer messages - (inline tables, cards, images on supported terminals) but adds complexity. +- **Chat message rendering**: Is the message stream a `List` of A2UI `Text` components, or a + special-cased renderer? Full A2UI enables richer messages but adds complexity. -- **Input handling**: A2UI defines actions routed from client to agent. How does this map to the - existing IPC event system? Is a keystroke in a `TextField` an A2UI action or an IPC event? +- **Input handling**: How do A2UI actions map to the IPC event system? Is a keystroke in a + `TextField` an A2UI action or an IPC event? -- **Performance budget**: How many components can ratatui render at 60fps? Large component trees - (hundreds of items in a tool list) may need virtualization. +- **Performance**: How many components can ratatui render at 60fps? Large trees may need + virtualization. # Future possibilities [future-possibilities]: #future-possibilities -- **Saveable layouts**: Users save named layouts ("coding", "debugging", "minimal") and switch - between them. The agent restores a saved layout by name. +- **Saveable layouts**: Named layouts ("coding", "debugging", "minimal") the user switches between. - **Layout marketplace**: Capsule authors publish recommended layouts alongside their capsules. - "Install the monitoring capsule and its dashboard layout." -- **Multi-surface**: Instead of one root surface, multiple surfaces for multi-monitor or tabbed - interfaces. Each tab is an independent A2UI surface. +- **Multi-surface**: Multiple surfaces for multi-monitor or tabbed interfaces. -- **Sixel/Kitty graphics**: Terminals that support image protocols could render A2UI `Image` - components as inline graphics. The catalog advertises this capability. +- **Sixel/Kitty graphics**: Terminals with image protocol support render `Image` components inline. -- **Web frontend**: A web-based Astrid frontend renders A2UI components as React/HTML. The same - agent layout description works across CLI and web — the rendering quality just improves. +- **Web frontend**: A2UI components rendered as React/HTML. Same layout description, richer output. -- **Collaborative UI**: Multiple users connected to the same session see the same layout. The agent - composes for the group, not just one user. +- **Collaborative UI**: Multiple users in one session see the same agent-composed layout. -- **Theme as data model**: The agent controls the color palette via A2UI data model updates. "Dark - mode", "light mode", and "high contrast" become data model patches, not hardcoded theme structs. - This solves the terminal background detection problem (#626) — the agent detects or asks, then - patches the theme. +- **Theme as data model**: Colors and styling as data model patches. Dark/light/high-contrast + become agent-controlled, adapting to terminal capabilities. From a0ab5fed9e1229d07fde48366cbbc61a89ed18a9 Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 17:48:12 +0400 Subject: [PATCH 4/8] rfc: rename IPC topics to a2ui.*, add versioning section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Topics carry no version — A2UI messages self-describe via the version field in every payload. Decouples IPC routing from spec evolution. --- text/0000-agent-composed-ui.md | 46 +++++++++++++++++++--------------- 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/text/0000-agent-composed-ui.md b/text/0000-agent-composed-ui.md index 747ed9a..411d88f 100644 --- a/text/0000-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -59,7 +59,7 @@ The agent discovers available components and composes them into a layout. Your c data model via IPC — the component re-renders automatically through A2UI data binding: ```rust -sdk::publish("ui.v1.data_model.update", json!({ +sdk::publish("a2ui.data_model.update", json!({ "surfaceId": "root", "patch": [ { "op": "replace", "path": "/model-status/model", "value": "gpt-5.4" }, @@ -183,23 +183,32 @@ Unsupported components degrade to `Text` or are omitted. ### IPC topics -A2UI messages travel over the IPC event bus under `ui.v1.*`: +A2UI messages travel over the IPC event bus under `a2ui.*`: | IPC Topic | Payload | Direction | |-----------|---------|-----------| -| `ui.v1.surface.create` | A2UI `createSurface` | Agent → Frontend | -| `ui.v1.components.update` | A2UI `updateComponents` | Agent → Frontend | -| `ui.v1.data_model.update` | A2UI `updateDataModel` | Capsule/Agent → Frontend | -| `ui.v1.surface.delete` | A2UI `deleteSurface` | Agent → Frontend | -| `ui.v1.action` | `{ surfaceId, componentId, action, data }` | Frontend → Agent | -| `ui.v1.catalog.query` | `{}` | Agent → Frontend | -| `ui.v1.catalog.response` | `{ components: [...] }` | Frontend → Agent | +| `a2ui.surface.create` | A2UI `createSurface` | Agent → Frontend | +| `a2ui.components.update` | A2UI `updateComponents` | Agent → Frontend | +| `a2ui.data_model.update` | A2UI `updateDataModel` | Capsule/Agent → Frontend | +| `a2ui.surface.delete` | A2UI `deleteSurface` | Agent → Frontend | +| `a2ui.action` | `{ surfaceId, componentId, action, data }` | Frontend → Agent | +| `a2ui.catalog.query` | `{}` | Agent → Frontend | +| `a2ui.catalog.response` | `{ components: [...] }` | Frontend → Agent | + +### Versioning + +IPC topics carry no version number. A2UI messages self-describe their version via the `version` +field in every payload (e.g., `"version": "v0.10"`). Topics are stable routing addresses; the A2UI +spec version is a payload concern. + +When A2UI evolves, the frontend reads the `version` field and handles accordingly. No topic changes, +no capsule manifest changes, no subscriber updates. This decouples IPC routing from spec evolution. ### Component discovery UI component discovery follows the same bus-based pattern as tool discovery: -1. Agent triggers `ui.v1.request.describe` (hook fan-out to all capsules) +1. Agent triggers `a2ui.request.describe` (hook fan-out to all capsules) 2. Each capsule's `ui_describe` interceptor responds with its available components 3. Agent collects the component registry @@ -207,11 +216,11 @@ The `Capsule.toml` declares the interceptor and IPC capabilities: ```toml [[interceptor]] -event = "ui.v1.request.describe" +event = "a2ui.request.describe" action = "ui_describe" [capabilities] -ipc_publish = ["ui.v1.response.describe.*"] +ipc_publish = ["a2ui.response.describe.*"] ``` The `ui_describe` handler returns what the capsule can provide: @@ -261,21 +270,21 @@ The agent bridges the two — mapping capsule components onto A2UI primitives. ### Composition flow -1. **Boot**: Frontend publishes `ui.v1.catalog.response` with supported A2UI component types. -2. **Discovery**: Agent triggers `ui.v1.request.describe` — capsules respond with available +1. **Boot**: Frontend publishes `a2ui.catalog.response` with supported A2UI component types. +2. **Discovery**: Agent triggers `a2ui.request.describe` — capsules respond with available components. 3. **Compose**: Agent emits `createSurface` + `updateComponents` with the default layout. 4. **Interact**: On user input ("show tools sidebar"), agent emits `updateComponents` with a restructured tree. -5. **Live data**: Capsules publish `ui.v1.data_model.update` to push values to their components. +5. **Live data**: Capsules publish `a2ui.data_model.update` to push values to their components. Frontend re-renders via A2UI data binding. 6. **Persist**: Agent saves layout composition to session state. On reconnect, restores last layout. ### Error handling - **Unknown component type**: Frontend renders as `Text` with the component's `id`, or omits. - Agent sees `ui.v1.error` and can adapt. -- **Invalid tree structure**: Rejected with `ui.v1.error`; last valid tree retained. + Agent sees `a2ui.error` and can adapt. +- **Invalid tree structure**: Rejected with `a2ui.error`; last valid tree retained. - **Agent not ready**: Frontend renders a minimal loading state until the first `createSurface` arrives. This is the only hardcoded UI in the frontend. @@ -350,9 +359,6 @@ retained. Users can always say "reset to default." # Unresolved questions [unresolved-questions]: #unresolved-questions -- **A2UI version pinning**: Which version do we adopt? v0.10 is latest draft. May need to pin - v0.8 (public preview) and upgrade. - - **Streaming component trees**: How does incremental JSONL interact with double-buffered rendering? Render partial trees or wait for complete updates? From fe74a7a94dc51f305cd454a95539c7cf5a8f31a1 Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 17:49:57 +0400 Subject: [PATCH 5/8] rfc: clarify LLM interacts via tools, not IPC topics The LLM calls tools (compose_ui, delete_surface). A UI capsule bridges tool calls to A2UI IPC messages. IPC topics are internal plumbing between capsules and frontends, not the LLM's interface. Adds layers section, tool definitions, and updates composition flow. --- text/0000-agent-composed-ui.md | 103 ++++++++++++++++++++++++--------- 1 file changed, 76 insertions(+), 27 deletions(-) diff --git a/text/0000-agent-composed-ui.md b/text/0000-agent-composed-ui.md index 411d88f..8fe0a5a 100644 --- a/text/0000-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -105,44 +105,91 @@ The agent owns a single root A2UI surface that IS the entire screen. It composes tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) and capsule-provided components. -On startup, the agent emits a default layout: +The LLM interacts with A2UI through **tools**, not IPC topics directly. A UI capsule exposes tools +that the LLM calls; the capsule translates tool calls into A2UI IPC messages: -```json -{ - "createSurface": { - "surfaceId": "root", - "catalogId": "astrid/cli/v1" - } -} ``` +LLM calls `compose_ui` tool + → UI capsule receives tool call + → publishes A2UI messages to `a2ui.*` IPC topics + → frontend renders +``` + +On startup, the agent calls `compose_ui` with a default layout: ```json { - "updateComponents": { - "surfaceId": "root", - "components": [ - { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, - { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info"], "justify": "spaceBetween" }, - { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, - { "id": "model-info", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, - { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, - { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, - { "id": "input", "component": "TextField", "placeholder": "Message..." } - ] - } + "action": "update", + "components": [ + { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, + { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info"], "justify": "spaceBetween" }, + { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, + { "id": "model-info", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, + { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, + { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, + { "id": "input", "component": "TextField", "placeholder": "Message..." } + ] } ``` This is not special. It is not hardcoded. It is the agent's opening move. When the user asks for a -sidebar, the agent restructures the tree — replacing `main` with a `Row` containing the chat column -and a new sidebar column. No code change. No predefined zones. Just a different component tree. +sidebar, the agent calls `compose_ui` again with a restructured tree. No code change. No predefined +zones. Just a different component tree. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation +## Layers + +There are three layers: + +1. **Tools** — the LLM's interface. The LLM calls tools like `compose_ui` to describe layouts. +2. **IPC topics** (`a2ui.*`) — internal bus plumbing. A UI capsule translates tool calls into A2UI + IPC messages. Capsules also publish data model updates here directly. +3. **A2UI protocol** — the wire format for messages on the bus. Frontends consume these. + +The LLM never publishes to IPC topics directly. It calls tools. A capsule bridges the gap. + +## Tools + +A UI capsule exposes tools for the LLM to compose and manage the interface: + +| Tool | Purpose | +|------|---------| +| `compose_ui` | Create or update the component tree on a surface | +| `delete_surface` | Remove a surface | + +`compose_ui` input schema: + +```json +{ + "type": "object", + "properties": { + "action": { + "type": "string", + "enum": ["create", "update"], + "description": "Create a new surface or update an existing one" + }, + "surface_id": { + "type": "string", + "description": "Surface identifier (default: 'root')" + }, + "components": { + "type": "array", + "description": "A2UI component tree — flat list with id, component type, and children references", + "items": { "type": "object" } + } + }, + "required": ["action", "components"] +} +``` + +The UI capsule receives this tool call, wraps it in A2UI `createSurface`/`updateComponents` +messages, and publishes to the `a2ui.*` IPC topics. + ## Protocol -Adopt A2UI (latest stable at implementation time) as the wire protocol. No modifications to the +Adopt A2UI (latest stable at implementation time) as the wire format. No modifications to the core spec. ### Message types (A2UI standard) @@ -273,11 +320,13 @@ The agent bridges the two — mapping capsule components onto A2UI primitives. 1. **Boot**: Frontend publishes `a2ui.catalog.response` with supported A2UI component types. 2. **Discovery**: Agent triggers `a2ui.request.describe` — capsules respond with available components. -3. **Compose**: Agent emits `createSurface` + `updateComponents` with the default layout. -4. **Interact**: On user input ("show tools sidebar"), agent emits `updateComponents` with a +3. **Compose**: Agent calls `compose_ui` tool with the default layout. The UI capsule publishes + the A2UI messages to the bus. +4. **Interact**: On user input ("show tools sidebar"), agent calls `compose_ui` again with a restructured tree. -5. **Live data**: Capsules publish `a2ui.data_model.update` to push values to their components. - Frontend re-renders via A2UI data binding. +5. **Live data**: Capsules publish `a2ui.data_model.update` directly to push values to their + components. Frontend re-renders via A2UI data binding. (This is the one case where capsules + publish to `a2ui.*` topics directly — data updates, not layout changes.) 6. **Persist**: Agent saves layout composition to session state. On reconnect, restores last layout. ### Error handling From 815ad50c1842d0c3f608a1b12a9e53a31f4305df Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 18:19:58 +0400 Subject: [PATCH 6/8] rfc: agent as bus participant, no component registration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Major revision: - Agent observes existing IPC traffic directly, no ui_describe needed - Capsules don't change — their existing events ARE the data sources - Agent publishes to a2ui.* as a bus participant, not through tools - Default layout on boot without LLM (static, cached on subsequent boots) - Multi-principal surfaces — each agent gets its own A2UI scope - Removed tool definitions, component discovery protocol, capsule changes --- text/0000-agent-composed-ui.md | 343 ++++++++++++--------------------- 1 file changed, 124 insertions(+), 219 deletions(-) diff --git a/text/0000-agent-composed-ui.md b/text/0000-agent-composed-ui.md index 8fe0a5a..52bd4bd 100644 --- a/text/0000-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -6,10 +6,10 @@ # Summary [summary]: #summary -Adopt Google's A2UI (Agent-to-User Interface) protocol as the rendering contract between capsules, -the agent, and frontends. The agent owns the full layout composition — there are no predefined zones, -slots, or hardcoded regions. Frontends are dumb renderers that draw whatever component tree the agent -describes. Capsules provide components and data sources; the agent decides what goes where. +Adopt Google's A2UI (Agent-to-User Interface) protocol as the rendering contract between the agent +and frontends. The agent is a first-class bus participant — it observes IPC traffic, understands what +data capsules are producing, and composes A2UI surfaces directly on the bus. Frontends are dumb +renderers. Capsules don't change — their existing IPC events are the data sources. # Motivation [motivation]: #motivation @@ -17,59 +17,35 @@ describes. Capsules provide components and data sources; the agent decides what The frontend should not own structure or content. Content comes from capsules. Structure comes from the agent. -The agent is uniquely positioned to compose UI because it understands: -- What capsules are loaded and what components they offer -- What the user is currently doing (coding, chatting, configuring, debugging) -- What the user has asked for ("show me tools on the right", "minimal interface") -- What information is relevant right now vs. noise +The agent is uniquely positioned to compose UI because it: +- Observes the IPC bus and sees what capsules are communicating +- Understands what data is flowing and what's relevant to show +- Knows what the user is doing and what they've asked for +- Can adapt the layout to context without code changes By making the agent the UI composer: - **Adaptive layouts** — the agent reshapes the UI based on task context - **Natural language customization** — "put my config on the right" just works -- **No hardcoded structure** — the default layout is just the agent's starting opinion -- **Multi-frontend for free** — each frontend renders A2UI components natively; the agent's layout - description is frontend-agnostic -- **Personality-driven defaults** — different agent identities can have different default layouts -- **Theme as data** — colors and styling become data model patches the agent controls, adapting to - terminal capabilities rather than assuming a dark background +- **No hardcoded structure** — the default layout is the agent's starting opinion +- **Multi-frontend for free** — each frontend renders A2UI natively +- **Personality-driven defaults** — different agent identities can have different layouts +- **Theme as data** — colors and styling become data the agent controls +- **Zero capsule changes** — existing IPC data structures are the data sources # Guide-level explanation [guide-level-explanation]: #guide-level-explanation ## For capsule developers -Your capsule exports **components** — UI building blocks that the agent can place anywhere. You -declare what you can provide; you do not decide where it goes. +Nothing changes. Your capsule already publishes data on IPC topics with defined structures. The +agent observes the bus, sees your data, and composes UI from it. You don't register UI components. +You don't declare what you can provide. Your existing IPC events are the data sources. -A component is registered via a `ui_describe` interceptor (same pattern as `tool_describe`): +For example, the identity capsule already publishes spark config on `spark.v1.response.ready`. The +react capsule already publishes model info and usage on `agent.v1.response`. The agent sees this +traffic and can present it however it wants — as a status bar, a sidebar panel, a modal. -```rust -#[astrid::ui_component("model-status")] -fn model_status() -> UiComponent { - UiComponent::new("model-status") - .catalog_type("Text") - .data_model(json!({ - "model": "unknown", - "context_usage": 0.0 - })) -} -``` - -The agent discovers available components and composes them into a layout. Your capsule updates its -data model via IPC — the component re-renders automatically through A2UI data binding: - -```rust -sdk::publish("a2ui.data_model.update", json!({ - "surfaceId": "root", - "patch": [ - { "op": "replace", "path": "/model-status/model", "value": "gpt-5.4" }, - { "op": "replace", "path": "/model-status/context_usage", "value": 0.47 } - ] -})); -``` - -A capsule can offer multiple components of different shapes — a full config form, a compact status -line, a detailed dashboard. Same data, different presentations. The agent picks what fits. +If you want the agent to have richer data to display, publish richer IPC events. That's it. ## For users @@ -101,91 +77,55 @@ Responsibilities: ## The agent's role -The agent owns a single root A2UI surface that IS the entire screen. It composes the full component -tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) and capsule-provided -components. +The agent is a first-class participant on the IPC bus. It: -The LLM interacts with A2UI through **tools**, not IPC topics directly. A UI capsule exposes tools -that the LLM calls; the capsule translates tool calls into A2UI IPC messages: +1. **Observes** — subscribes to IPC topics and sees what data capsules produce +2. **Composes** — publishes A2UI messages (`a2ui.*`) to describe the UI layout +3. **Reacts** — restructures the UI based on bus traffic, user requests, or context changes -``` -LLM calls `compose_ui` tool - → UI capsule receives tool call - → publishes A2UI messages to `a2ui.*` IPC topics - → frontend renders -``` +The agent owns a single root A2UI surface that IS the entire screen. It composes the full component +tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) and data from the bus. -On startup, the agent calls `compose_ui` with a default layout: +On startup, the agent publishes a default layout: ```json { - "action": "update", - "components": [ - { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, - { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info"], "justify": "spaceBetween" }, - { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, - { "id": "model-info", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, - { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, - { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, - { "id": "input", "component": "TextField", "placeholder": "Message..." } - ] + "updateComponents": { + "surfaceId": "root", + "components": [ + { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, + { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info"], "justify": "spaceBetween" }, + { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, + { "id": "model-info", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, + { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, + { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, + { "id": "input", "component": "TextField", "placeholder": "Message..." } + ] + } } ``` This is not special. It is not hardcoded. It is the agent's opening move. When the user asks for a -sidebar, the agent calls `compose_ui` again with a restructured tree. No code change. No predefined -zones. Just a different component tree. +sidebar, the agent restructures the tree. No code change. No predefined zones. Just a different +component tree. + +The agent can publish this default layout on boot without an LLM call — it's a static starting +point. The LLM is consulted when the user requests layout changes, not for routine rendering. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation ## Layers -There are three layers: - -1. **Tools** — the LLM's interface. The LLM calls tools like `compose_ui` to describe layouts. -2. **IPC topics** (`a2ui.*`) — internal bus plumbing. A UI capsule translates tool calls into A2UI - IPC messages. Capsules also publish data model updates here directly. -3. **A2UI protocol** — the wire format for messages on the bus. Frontends consume these. - -The LLM never publishes to IPC topics directly. It calls tools. A capsule bridges the gap. - -## Tools +Two layers: -A UI capsule exposes tools for the LLM to compose and manage the interface: - -| Tool | Purpose | -|------|---------| -| `compose_ui` | Create or update the component tree on a surface | -| `delete_surface` | Remove a surface | - -`compose_ui` input schema: - -```json -{ - "type": "object", - "properties": { - "action": { - "type": "string", - "enum": ["create", "update"], - "description": "Create a new surface or update an existing one" - }, - "surface_id": { - "type": "string", - "description": "Surface identifier (default: 'root')" - }, - "components": { - "type": "array", - "description": "A2UI component tree — flat list with id, component type, and children references", - "items": { "type": "object" } - } - }, - "required": ["action", "components"] -} -``` +1. **IPC bus** (`a2ui.*` topics) — the agent publishes A2UI messages and observes capsule traffic + directly on the bus. The bus is the agent's native communication layer. +2. **A2UI protocol** — the wire format for rendering messages. Frontends consume these. -The UI capsule receives this tool call, wraps it in A2UI `createSurface`/`updateComponents` -messages, and publishes to the `a2ui.*` IPC topics. +The agent does not need tools to compose UI. It publishes to `a2ui.*` topics directly, just as any +capsule publishes to its own topics. The LLM may influence layout decisions (via the react loop), +but the agent capsule handles A2UI publishing as a bus participant. ## Protocol @@ -230,13 +170,13 @@ Unsupported components degrade to `Text` or are omitted. ### IPC topics -A2UI messages travel over the IPC event bus under `a2ui.*`: +A2UI messages travel on the bus under `a2ui.*`: | IPC Topic | Payload | Direction | |-----------|---------|-----------| | `a2ui.surface.create` | A2UI `createSurface` | Agent → Frontend | | `a2ui.components.update` | A2UI `updateComponents` | Agent → Frontend | -| `a2ui.data_model.update` | A2UI `updateDataModel` | Capsule/Agent → Frontend | +| `a2ui.data_model.update` | A2UI `updateDataModel` | Agent → Frontend | | `a2ui.surface.delete` | A2UI `deleteSurface` | Agent → Frontend | | `a2ui.action` | `{ surfaceId, componentId, action, data }` | Frontend → Agent | | `a2ui.catalog.query` | `{}` | Agent → Frontend | @@ -249,111 +189,66 @@ field in every payload (e.g., `"version": "v0.10"`). Topics are stable routing a spec version is a payload concern. When A2UI evolves, the frontend reads the `version` field and handles accordingly. No topic changes, -no capsule manifest changes, no subscriber updates. This decouples IPC routing from spec evolution. - -### Component discovery - -UI component discovery follows the same bus-based pattern as tool discovery: - -1. Agent triggers `a2ui.request.describe` (hook fan-out to all capsules) -2. Each capsule's `ui_describe` interceptor responds with its available components -3. Agent collects the component registry - -The `Capsule.toml` declares the interceptor and IPC capabilities: - -```toml -[[interceptor]] -event = "a2ui.request.describe" -action = "ui_describe" - -[capabilities] -ipc_publish = ["a2ui.response.describe.*"] -``` - -The `ui_describe` handler returns what the capsule can provide: - -```json -{ - "components": [ - { - "id": "identity-config-form", - "display_name": "Identity Configuration", - "description": "Agent identity settings — callsign, class, tone, backstory", - "schema": { - "properties": { - "callsign": { "type": "string", "description": "Agent's display name" }, - "class": { "type": "string", "description": "Agent class/role" }, - "tone": { "type": "string", "description": "Communication style" } - } - }, - "data_topic": "identity.v1.config", - "actions": ["save", "reset"] - }, - { - "id": "identity-status", - "display_name": "Identity Status", - "description": "Current agent identity — compact name and class display", - "schema": { - "properties": { - "callsign": { "type": "string" }, - "class": { "type": "string" } - } - }, - "data_topic": "identity.v1.status" - } - ] -} -``` +no capsule manifest changes, no subscriber updates. -The agent sees: "identity capsule has a config form and a status widget. Here are their schemas and -data topics." The agent decides if, when, and where to place them. +### Data sources — the bus IS the component registry -Two catalogs are in play: +Capsules don't need to register UI components. Their existing IPC events are the data sources. The +agent observes the bus and knows what data is available: -- **Frontend catalog** (A2UI standard): what primitive types the renderer supports (Row, Text, etc.) -- **Capsule component registry** (bus-discovered): what data/interactions capsules provide +| Existing IPC traffic | Data available | UI possibilities | +|---------------------|----------------|------------------| +| `spark.v1.response.ready` | Agent identity (callsign, class, tone) | Status badge, config panel | +| `agent.v1.response` | Model name, response text, completion status | Status bar, chat stream | +| `llm.v1.stream.*` | Token deltas, usage stats, tool calls | Progress bar, token counter | +| `tool.v1.request.describe` | Tool schemas from all capsules | Tool list panel | +| `registry.v1.active_model_changed` | Active model/provider switch | Model indicator | +| `session.v1.response.get_messages` | Conversation history | Chat view | -The agent bridges the two — mapping capsule components onto A2UI primitives. +The agent maps this data onto A2UI components. No new protocol for capsules. No `ui_describe` +interceptor. Capsules keep doing exactly what they do today. ### Composition flow -1. **Boot**: Frontend publishes `a2ui.catalog.response` with supported A2UI component types. -2. **Discovery**: Agent triggers `a2ui.request.describe` — capsules respond with available - components. -3. **Compose**: Agent calls `compose_ui` tool with the default layout. The UI capsule publishes - the A2UI messages to the bus. -4. **Interact**: On user input ("show tools sidebar"), agent calls `compose_ui` again with a - restructured tree. -5. **Live data**: Capsules publish `a2ui.data_model.update` directly to push values to their - components. Frontend re-renders via A2UI data binding. (This is the one case where capsules - publish to `a2ui.*` topics directly — data updates, not layout changes.) -6. **Persist**: Agent saves layout composition to session state. On reconnect, restores last layout. +1. **Boot**: Agent publishes a default A2UI layout on the bus. No LLM call needed — this is a + static default the agent capsule emits on load. +2. **Observe**: Agent subscribes to relevant IPC topics and sees capsule data flowing. +3. **Update**: Agent publishes `a2ui.data_model.update` as bus data changes (e.g., model name + changes, context usage updates). +4. **Interact**: When the user requests a layout change, the LLM is consulted (via the react + loop). The react loop's response includes A2UI updates that the agent publishes. +5. **Persist**: Layout composition saved to KV. On reconnect, restored without LLM involvement. ### Error handling - **Unknown component type**: Frontend renders as `Text` with the component's `id`, or omits. Agent sees `a2ui.error` and can adapt. - **Invalid tree structure**: Rejected with `a2ui.error`; last valid tree retained. -- **Agent not ready**: Frontend renders a minimal loading state until the first `createSurface` - arrives. This is the only hardcoded UI in the frontend. ### Capability gating Surface creation and component updates are capability-gated. Only the orchestrating agent (or -capsules with `ui.surface.write` capability) can emit `createSurface` and `updateComponents`. -Any capsule can emit `updateDataModel` for components it owns (scoped by component ID prefix -matching its capsule namespace). +capsules with `a2ui.surface.write` capability) can emit `createSurface` and `updateComponents`. + +This prevents a rogue capsule from hijacking the layout. + +## Multi-principal surfaces + +Each principal (`home/agent1`, `home/agent2`) gets its own capsule instances, bus subscriptions, and +A2UI surfaces. The agent for each principal composes its own UI independently. + +This means multiple agents can coexist in the same runtime, each with their own layout and data +sources. A2A (Agent-to-Agent) communication between principals is native IPC on the same bus — +agents can observe each other's traffic and coordinate. -This prevents a rogue capsule from hijacking the layout while allowing capsules to update their data. +A principal's A2UI surface is scoped to its own frontend connections. `home/agent1`'s UI doesn't +bleed into `home/agent2`'s frontend. # Drawbacks [drawbacks]: #drawbacks -- **Agent latency on boot**: First render depends on the agent composing a layout. Mitigation: cache - the last layout and restore it immediately; the agent updates once ready. - -- **LLM token cost**: Component registry, layout state, and A2UI grammar consume context window - tokens. For simple chat, this is overhead. +- **LLM token cost**: When layout changes are requested, the A2UI grammar and current layout state + consume context window tokens. - **Layout instability**: A poorly prompted agent could produce jarring layout changes. Mitigation: system prompt instructs conservative layout behavior; changes only on explicit user request. @@ -369,8 +264,8 @@ This prevents a rogue capsule from hijacking the layout while allowing capsules ## Why A2UI? A2UI solves the component description problem with a well-designed catalog, surface lifecycle, data -binding, and security model. Building a custom protocol would duplicate this work. A2UI is -Apache 2.0, open-source, and has ecosystem adoption. +binding, and security model. Building a custom protocol would duplicate this. A2UI is Apache 2.0 +and has ecosystem adoption. ## Why not MCP Apps? @@ -382,28 +277,35 @@ A2UI's declarative approach maps to any native widget toolkit. Zones impose structure the agent should own. The agent CAN create a zone-like layout if it chooses, but is not constrained to one. Structure is an agent choice, not a system constraint. -## What if the agent produces bad layouts? +## Why bus-native instead of tools? + +Tools are an abstraction over the bus for LLM interaction. But UI composition is not purely an LLM +concern — the agent publishes a default layout on boot (no LLM), updates data bindings as bus +traffic flows (no LLM), and only consults the LLM for user-requested layout changes. Making the +agent a direct bus participant avoids forcing every UI update through the tool call → LLM → response +cycle. + +## Why no component registration? -The frontend validates component trees. Invalid structures are rejected; the last valid tree is -retained. Users can always say "reset to default." +Capsules already publish structured data on IPC topics. The agent observes the bus and knows what +data exists. Adding a separate UI component registration protocol would duplicate what the bus +already provides and require every capsule to implement a new interceptor for no gain. # Prior art [prior-art]: #prior-art - **A2UI (Google)** — The protocol we adopt. Declarative JSON component descriptions, catalog-based - security, surface lifecycle. Apache 2.0. Our contribution is the agent-as-composer pattern. + security, surface lifecycle. Apache 2.0. Our contribution is the agent-as-bus-participant + composition model. -- **MCP Apps (Anthropic + OpenAI)** — HTML/JS in sandboxed iframes. Richer rendering but requires a - web runtime. Not viable for terminals. +- **MCP Apps (Anthropic + OpenAI)** — HTML/JS in sandboxed iframes. Not viable for terminals. -- **AG-UI (CopilotKit)** — Event-based agent-frontend streaming protocol. Complementary to A2UI - (transport layer vs. component format). Worth evaluating for IPC transport in future work. +- **AG-UI (CopilotKit)** — Event-based agent-frontend streaming protocol. Complementary to A2UI. -- **Zellij / tmux** — Terminal multiplexers with user-composed panes. Same concept but manual. This - RFC is the AI-native version of terminal pane composition. +- **Zellij / tmux** — Terminal multiplexers with user-composed panes. Same concept but manual. -- **Emacs** — Editor as a Lisp-driven UI canvas. Windows, buffers, and frames composed - programmatically. Philosophically similar, with the agent replacing Emacs Lisp. +- **Emacs** — Editor as a Lisp-driven UI canvas. Philosophically similar, with the agent replacing + Emacs Lisp as the composition layer. # Unresolved questions [unresolved-questions]: #unresolved-questions @@ -414,32 +316,35 @@ retained. Users can always say "reset to default." - **Focus management**: When the agent restructures the layout, where does keyboard focus go? A2UI lacks a focus primitive. May need a convention (e.g., `focused: true` property). -- **Layout persistence**: How is the last-known layout serialized? Raw A2UI component list or a - higher-level representation? +- **Layout persistence**: How is the last-known layout serialized for session restore? What + survives session clear vs. daemon restart? - **Chat message rendering**: Is the message stream a `List` of A2UI `Text` components, or a - special-cased renderer? Full A2UI enables richer messages but adds complexity. + special-cased renderer? -- **Input handling**: How do A2UI actions map to the IPC event system? Is a keystroke in a - `TextField` an A2UI action or an IPC event? +- **Input handling**: How do A2UI actions map to the IPC event system? - **Performance**: How many components can ratatui render at 60fps? Large trees may need virtualization. +- **Multi-principal surface routing**: How does the frontend/proxy know which principal's A2UI + surface to render for a given connection? + # Future possibilities [future-possibilities]: #future-possibilities - **Saveable layouts**: Named layouts ("coding", "debugging", "minimal") the user switches between. +- **A2A-driven UI**: Agent1 observes Agent2's bus traffic and composes a dashboard of Agent2's + activity. Multi-agent monitoring via the same A2UI surface. + - **Layout marketplace**: Capsule authors publish recommended layouts alongside their capsules. - **Multi-surface**: Multiple surfaces for multi-monitor or tabbed interfaces. - **Sixel/Kitty graphics**: Terminals with image protocol support render `Image` components inline. -- **Web frontend**: A2UI components rendered as React/HTML. Same layout description, richer output. - -- **Collaborative UI**: Multiple users in one session see the same agent-composed layout. +- **Web frontend**: A2UI components rendered as React/HTML. Same layout, richer output. - **Theme as data model**: Colors and styling as data model patches. Dark/light/high-contrast become agent-controlled, adapting to terminal capabilities. From 91e76b33e533ec445c101df5cb9974140cb50549 Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 18:27:00 +0400 Subject: [PATCH 7/8] =?UTF-8?q?rfc:=20direct=20IPC=20event=20bindings=20?= =?UTF-8?q?=E2=80=94=20no=20intermediate=20data=20model?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Components bind to IPC events via {topic, path} references. The runtime subscribes to referenced topics and routes payload values to bound components automatically. The agent describes structure + bindings, the runtime handles plumbing. No agent involvement in routine data updates. --- text/0000-agent-composed-ui.md | 84 +++++++++++++++++++--------------- 1 file changed, 46 insertions(+), 38 deletions(-) diff --git a/text/0000-agent-composed-ui.md b/text/0000-agent-composed-ui.md index 52bd4bd..c55479d 100644 --- a/text/0000-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -77,16 +77,19 @@ Responsibilities: ## The agent's role -The agent is a first-class participant on the IPC bus. It: +The agent publishes A2UI layout descriptions on the bus. It describes **structure and bindings** — +which components exist, how they're arranged, and which IPC events drive their values. The runtime +handles the data plumbing. -1. **Observes** — subscribes to IPC topics and sees what data capsules produce -2. **Composes** — publishes A2UI messages (`a2ui.*`) to describe the UI layout -3. **Reacts** — restructures the UI based on bus traffic, user requests, or context changes +1. **Composes** — publishes A2UI messages (`a2ui.*`) describing layout and IPC bindings +2. **Restructures** — when the user requests changes, the LLM produces new structure + bindings The agent owns a single root A2UI surface that IS the entire screen. It composes the full component -tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) and data from the bus. +tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) with IPC event bindings. -On startup, the agent publishes a default layout: +On startup, the agent publishes a default layout. Components bind directly to IPC events — not +to an intermediate data model the agent manually patches. The agent describes **structure and +bindings**. The runtime wires IPC events to components automatically: ```json { @@ -94,9 +97,10 @@ On startup, the agent publishes a default layout: "surfaceId": "root", "components": [ { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, - { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info"], "justify": "spaceBetween" }, - { "id": "breadcrumb", "component": "Text", "text": "~ > dev > astrid", "variant": "caption" }, - { "id": "model-info", "component": "Text", "text": "gpt-5.4 | 47%", "variant": "caption" }, + { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info", "context-bar"], "justify": "spaceBetween" }, + { "id": "breadcrumb", "component": "Text", "text": {"topic": "workspace.v1.path", "path": "/display_path"}, "variant": "caption" }, + { "id": "model-info", "component": "Text", "text": {"topic": "registry.v1.active_model_changed", "path": "/model_id"}, "variant": "caption" }, + { "id": "context-bar", "component": "ProgressBar", "value": {"topic": "llm.v1.stream.*", "path": "/event/usage/ratio"} }, { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, { "id": "input", "component": "TextField", "placeholder": "Message..." } @@ -105,9 +109,12 @@ On startup, the agent publishes a default layout: } ``` -This is not special. It is not hardcoded. It is the agent's opening move. When the user asks for a -sidebar, the agent restructures the tree. No code change. No predefined zones. Just a different -component tree. +When `registry.v1.active_model_changed` fires on the IPC bus, the runtime extracts `/model_id` +from the event payload and updates the `model-info` component. No agent involvement. No data model +patches. The binding IS the wiring — IPC events flow directly to components. + +The agent describes structure and bindings (what goes where, what IPC data drives it). The runtime +handles the plumbing. Layout changes require the LLM. Data updates are automatic. The agent can publish this default layout on boot without an LLM call — it's a static starting point. The LLM is consulted when the user requests layout changes, not for routine rendering. @@ -119,13 +126,14 @@ point. The LLM is consulted when the user requests layout changes, not for routi Two layers: -1. **IPC bus** (`a2ui.*` topics) — the agent publishes A2UI messages and observes capsule traffic - directly on the bus. The bus is the agent's native communication layer. +1. **IPC bus** (`a2ui.*` topics) — the agent publishes A2UI layout descriptions. Components bind + directly to other IPC topics via `{topic, path}` references. The runtime wires events to + components automatically. 2. **A2UI protocol** — the wire format for rendering messages. Frontends consume these. -The agent does not need tools to compose UI. It publishes to `a2ui.*` topics directly, just as any -capsule publishes to its own topics. The LLM may influence layout decisions (via the react loop), -but the agent capsule handles A2UI publishing as a bus participant. +The agent publishes to `a2ui.*` topics directly, just as any capsule publishes to its own topics. +The LLM may influence layout decisions (via the react loop), but routine data updates flow from +IPC events to components through bindings without agent involvement. ## Protocol @@ -191,33 +199,33 @@ spec version is a payload concern. When A2UI evolves, the frontend reads the `version` field and handles accordingly. No topic changes, no capsule manifest changes, no subscriber updates. -### Data sources — the bus IS the component registry +### Data sources — IPC event bindings -Capsules don't need to register UI components. Their existing IPC events are the data sources. The -agent observes the bus and knows what data is available: +Capsules don't register UI components. Their existing IPC events are the data sources. Components +bind directly to IPC topics and extract values via JSON path: -| Existing IPC traffic | Data available | UI possibilities | -|---------------------|----------------|------------------| -| `spark.v1.response.ready` | Agent identity (callsign, class, tone) | Status badge, config panel | -| `agent.v1.response` | Model name, response text, completion status | Status bar, chat stream | -| `llm.v1.stream.*` | Token deltas, usage stats, tool calls | Progress bar, token counter | -| `tool.v1.request.describe` | Tool schemas from all capsules | Tool list panel | -| `registry.v1.active_model_changed` | Active model/provider switch | Model indicator | -| `session.v1.response.get_messages` | Conversation history | Chat view | +| Binding target | IPC topic | Payload path | Component | +|---------------|-----------|-------------|-----------| +| Active model | `registry.v1.active_model_changed` | `/model_id` | `Text` | +| Context usage | `llm.v1.stream.*` | `/event/usage/ratio` | `ProgressBar` | +| Agent name | `spark.v1.response.ready` | `/callsign` | `Badge` | +| Conversation | `session.v1.response.get_messages` | `/messages` | `List` | +| Tool list | `tool.v1.request.describe` | `/tools` | `Table` | -The agent maps this data onto A2UI components. No new protocol for capsules. No `ui_describe` -interceptor. Capsules keep doing exactly what they do today. +The runtime subscribes to the referenced topics and routes payload values to bound components. +Components never contain hardcoded values — they contain bindings to IPC event structure. Capsules +keep doing exactly what they do today. ### Composition flow -1. **Boot**: Agent publishes a default A2UI layout on the bus. No LLM call needed — this is a - static default the agent capsule emits on load. -2. **Observe**: Agent subscribes to relevant IPC topics and sees capsule data flowing. -3. **Update**: Agent publishes `a2ui.data_model.update` as bus data changes (e.g., model name - changes, context usage updates). -4. **Interact**: When the user requests a layout change, the LLM is consulted (via the react - loop). The react loop's response includes A2UI updates that the agent publishes. -5. **Persist**: Layout composition saved to KV. On reconnect, restored without LLM involvement. +1. **Boot**: Agent publishes a default A2UI layout with IPC bindings. No LLM call needed — this + is a static default the agent capsule emits on load. +2. **Wire**: The runtime subscribes to all topics referenced in bindings. As IPC events fire, + values flow to bound components automatically. No agent involvement. +3. **Interact**: When the user requests a layout change, the LLM is consulted (via the react + loop). The LLM describes new structure and bindings — not values. +4. **Persist**: Layout composition (structure + bindings) saved to KV. On reconnect, restored + without LLM involvement. Bindings re-activate as bus traffic resumes. ### Error handling From de5b8775663af98be79b130ebf5680acffb056ad Mon Sep 17 00:00:00 2001 From: "Joshua J. Bouw" Date: Wed, 25 Mar 2026 18:33:55 +0400 Subject: [PATCH 8/8] rfc: precise IPC binding model with real payload structures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Traced through actual IpcPayload types and wire format: - Custom payloads: field path is /payload/data/ - Typed payloads (AgentResponse etc): path is /payload/ Added end-to-end flow examples: - Model name update: registry event → binding extraction → component render - Layout change: user request → LLM → new structure+bindings → render Binding syntax: {topic, path} where path is a JSON Pointer into the full IPC message as it arrives on the wire. --- text/0000-agent-composed-ui.md | 140 +++++++++++++++++++++++++++------ 1 file changed, 115 insertions(+), 25 deletions(-) diff --git a/text/0000-agent-composed-ui.md b/text/0000-agent-composed-ui.md index c55479d..b0c0053 100644 --- a/text/0000-agent-composed-ui.md +++ b/text/0000-agent-composed-ui.md @@ -87,9 +87,64 @@ handles the data plumbing. The agent owns a single root A2UI surface that IS the entire screen. It composes the full component tree using A2UI layout primitives (`Row`, `Column`, `Card`, `Tabs`, `List`) with IPC event bindings. -On startup, the agent publishes a default layout. Components bind directly to IPC events — not -to an intermediate data model the agent manually patches. The agent describes **structure and -bindings**. The runtime wires IPC events to components automatically: +On startup, the agent publishes a default layout. Components bind directly to IPC event fields. +The agent describes **structure and bindings**. The runtime wires IPC events to components. + +### How IPC messages look on the wire + +Every IPC message has an envelope with `topic` and `payload`. The `payload` is an `IpcPayload` — +a tagged enum. Capsule-published data arrives as the `Custom` variant with the data nested under +`data`. Typed kernel payloads (like `AgentResponse`) have fields at the top level of `payload`. + +For example, when the registry capsule publishes the active model, the message on the wire is: + +```json +{ + "topic": "registry.v1.active_model_changed", + "payload": { + "type": "custom", + "data": { + "id": "gpt-5.4", + "description": "OpenAI GPT-5.4 (gpt-5.4)", + "request_topic": "llm.v1.request.generate.openai", + "stream_topic": "llm.v1.stream.openai", + "capabilities": ["text", "tools", "vision"], + "context_window": 1050000, + "max_output_tokens": 128000 + } + }, + "source_id": "...", + "timestamp": "..." +} +``` + +And when the react capsule publishes a response, it looks like: + +```json +{ + "topic": "agent.v1.response", + "payload": { + "type": "agent_response", + "text": "Hello! How can I help?", + "is_final": true, + "session_id": "abc-123" + } +} +``` + +### Binding syntax + +A binding references an IPC topic and a JSON Pointer path into the message. The path is relative +to the full message object: + +```json +{ "topic": "", "path": "" } +``` + +For Custom payloads (capsule-published), the data is at `/payload/data/`. +For typed payloads (kernel-originated), the fields are at `/payload/`. + +### Default layout with bindings ```json { @@ -97,10 +152,23 @@ bindings**. The runtime wires IPC events to components automatically: "surfaceId": "root", "components": [ { "id": "root", "component": "Column", "children": ["status", "main", "input"] }, - { "id": "status", "component": "Row", "children": ["breadcrumb", "model-info", "context-bar"], "justify": "spaceBetween" }, - { "id": "breadcrumb", "component": "Text", "text": {"topic": "workspace.v1.path", "path": "/display_path"}, "variant": "caption" }, - { "id": "model-info", "component": "Text", "text": {"topic": "registry.v1.active_model_changed", "path": "/model_id"}, "variant": "caption" }, - { "id": "context-bar", "component": "ProgressBar", "value": {"topic": "llm.v1.stream.*", "path": "/event/usage/ratio"} }, + { + "id": "status", + "component": "Row", + "children": ["model-info", "context-bar"], + "justify": "spaceBetween" + }, + { + "id": "model-info", + "component": "Text", + "text": { "topic": "registry.v1.active_model_changed", "path": "/payload/data/id" }, + "variant": "caption" + }, + { + "id": "context-bar", + "component": "ProgressBar", + "value": { "topic": "llm.v1.stream.openai", "path": "/payload/event/Usage/input_tokens" } + }, { "id": "main", "component": "Column", "weight": 1, "children": ["messages"] }, { "id": "messages", "component": "List", "direction": "vertical", "children": [] }, { "id": "input", "component": "TextField", "placeholder": "Message..." } @@ -109,11 +177,33 @@ bindings**. The runtime wires IPC events to components automatically: } ``` -When `registry.v1.active_model_changed` fires on the IPC bus, the runtime extracts `/model_id` -from the event payload and updates the `model-info` component. No agent involvement. No data model -patches. The binding IS the wiring — IPC events flow directly to components. - -The agent describes structure and bindings (what goes where, what IPC data drives it). The runtime +### End-to-end flow: model name update + +1. User runs `/model gpt-5.4-mini` in the CLI. +2. Registry capsule handles the command, sets the active model. +3. Registry calls `ipc::publish_json("registry.v1.active_model_changed", &provider_entry)`. +4. The kernel wraps this as `IpcMessage { topic, payload: Custom { data: ProviderEntry } }` and + publishes on the event bus. +5. The runtime sees that `model-info` has a binding to topic `registry.v1.active_model_changed` + at path `/payload/data/id`. +6. The runtime extracts `"gpt-5.4-mini"` from the message at that JSON pointer path. +7. The runtime updates the `model-info` component's `text` property to `"gpt-5.4-mini"`. +8. The frontend re-renders the component. + +No LLM call. No agent involvement. No layout change. The IPC event flows through the binding to +the component automatically. + +### End-to-end flow: layout change + +1. User types "show me a sidebar with loaded tools". +2. React loop processes the prompt → LLM generates a response. +3. The LLM's response includes a layout restructure — it publishes an `updateComponents` that + replaces `main` with a `Row` containing the chat column and a new tools sidebar, with the + sidebar's table bound to `tool.v1.request.describe` at `/payload/data/tools`. +4. The frontend receives the new component tree and re-renders. +5. Tool data flows through the new binding automatically. + +The LLM describes structure and bindings (what goes where, what IPC data drives it). The runtime handles the plumbing. Layout changes require the LLM. Data updates are automatic. The agent can publish this default layout on boot without an LLM call — it's a static starting @@ -202,19 +292,19 @@ no capsule manifest changes, no subscriber updates. ### Data sources — IPC event bindings Capsules don't register UI components. Their existing IPC events are the data sources. Components -bind directly to IPC topics and extract values via JSON path: - -| Binding target | IPC topic | Payload path | Component | -|---------------|-----------|-------------|-----------| -| Active model | `registry.v1.active_model_changed` | `/model_id` | `Text` | -| Context usage | `llm.v1.stream.*` | `/event/usage/ratio` | `ProgressBar` | -| Agent name | `spark.v1.response.ready` | `/callsign` | `Badge` | -| Conversation | `session.v1.response.get_messages` | `/messages` | `List` | -| Tool list | `tool.v1.request.describe` | `/tools` | `Table` | - -The runtime subscribes to the referenced topics and routes payload values to bound components. -Components never contain hardcoded values — they contain bindings to IPC event structure. Capsules -keep doing exactly what they do today. +bind directly to IPC topics and extract fields via JSON Pointer paths into the message: + +| IPC topic | Payload type | Pointer path | Extracted value | +|-----------|-------------|-------------|-----------------| +| `registry.v1.active_model_changed` | `Custom` | `/payload/data/id` | `"gpt-5.4"` | +| `registry.v1.active_model_changed` | `Custom` | `/payload/data/context_window` | `1050000` | +| `llm.v1.stream.*` | `LlmStreamEvent` | `/payload/event/Usage/input_tokens` | `4521` | +| `spark.v1.response.ready` | `Custom` | `/payload/data/callsign` | `"Nova"` | +| `agent.v1.response` | `AgentResponse` | `/payload/text` | `"Hello!"` | + +The runtime subscribes to the referenced topics and routes extracted values to bound components. +Components never contain hardcoded values — they contain bindings to IPC event structure. +Capsules keep doing exactly what they do today. ### Composition flow