Skip to content

Commit f0ff080

Browse files
committed
feat: integrate excalidraw-mcp as first-class diagramming feature
Wire up MCPAppsMiddleware on the LangGraph agent so Excalidraw widget HTML is properly sent to the frontend (fixes blank canvas). Add progressive iframe streaming via postMessage to preserve JS state during updates, and a 75% focused modal for expanding widgets. Include an Excalidraw diagram skill for the agent with camera reveal patterns, color grammar, and element reference snippets. Closes #11
1 parent 1fde890 commit f0ff080

3 files changed

Lines changed: 358 additions & 66 deletions

File tree

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
# Excalidraw Diagram Skill
2+
3+
Create beautiful, professional, animated Excalidraw diagrams with progressive camera reveals, color-coded zones, and polished visual design. Use this skill whenever a user asks to diagram, visualize, map, chart, illustrate, or draw anything — including architecture diagrams, flowcharts, sequence diagrams, concept explainers, system maps, process flows, and technical overviews. Also trigger for requests like "show me how X works", "draw a diagram of", "create a visual for", "make an Excalidraw of", or any time a visual explanation would be clearer than text alone.
4+
5+
---
6+
7+
## Step 1 — Always call read_me first
8+
9+
Before emitting ANY elements, call `Excalidraw:read_me`. Do not skip this step, even for simple diagrams. It provides the color palette, camera sizes, font rules, and element syntax required to produce clean output.
10+
11+
```
12+
Excalidraw:read_me()
13+
```
14+
15+
Then proceed directly to `Excalidraw:create_view` with your elements array — no narration about the read_me call.
16+
17+
---
18+
19+
## Step 2 — Plan the diagram before writing elements
20+
21+
Before writing elements, mentally sketch:
22+
23+
1. **What are the layers / zones?** (e.g. Frontend / Backend / Database, or Input / Process / Output)
24+
2. **What color grammar makes sense?** Assign one color per layer and keep it consistent throughout
25+
3. **How many camera positions do I need?** Plan 3–6 camera stops minimum for a reveal effect
26+
4. **What's the reading order?** Left-to-right or top-to-bottom; pick one and stick to it
27+
28+
---
29+
30+
## Step 3 — Core design rules (MUST follow)
31+
32+
### Camera rules
33+
- **Always start with `cameraUpdate` as the first element**
34+
- Camera sizes MUST be exact 4:3 ratios: `400x300`, `600x450`, `800x600`, `1200x900`, `1600x1200`
35+
- Use **multiple cameraUpdates** throughout the array — pan to each section as you draw it
36+
- Leave padding: if content is 500px wide, use 800x600 camera
37+
- Final element should be a wide cameraUpdate showing the full diagram
38+
39+
### Color grammar (use consistently)
40+
41+
| Zone / Role | Fill | Stroke |
42+
|---------------------|---------------|-----------|
43+
| UI / Frontend | `#dbe4ff` | `#4a9eed` |
44+
| Logic / Agent | `#e5dbff` | `#8b5cf6` |
45+
| Data / Storage | `#d3f9d8` | `#22c55e` |
46+
| External / API | `#ffd8a8` | `#f59e0b` |
47+
| Error / Alert | `#ffc9c9` | `#ef4444` |
48+
| Notes / Decisions | `#fff3bf` | `#f59e0b` |
49+
50+
Zone background rectangles: use `opacity: 40`, `fillStyle: "solid"`
51+
52+
Node shapes: use pastel fills (`#a5d8ff`, `#b2f2bb`, `#d0bfff`, `#ffd8a8`, `#c3fae8`, `#eebefa`)
53+
54+
### Typography rules
55+
- Title: `fontSize: 26–28`, `strokeColor: "#1e1e1e"`
56+
- Subtitle / annotation: `fontSize: 16`, `strokeColor: "#757575"`
57+
- Shape labels: `fontSize: 16–18` via `label` property on the shape
58+
- NEVER use fontSize below 14
59+
- NEVER use light gray on white backgrounds (minimum text color: `#757575`)
60+
61+
### Shape rules
62+
- Use `label: { "text": "...", "fontSize": 16 }` directly on shapes — no separate text elements
63+
- Minimum shape size: `120x60` for labeled boxes
64+
- Add `roundness: { type: 3 }` for rounded corners (preferred for nodes)
65+
- Leave 20–30px gaps between elements
66+
67+
### Drawing order (z-order, critical)
68+
Emit in this sequence per section:
69+
1. Zone background rectangle (drawn first = sits behind)
70+
2. Zone label text
71+
3. Node shapes (with labels)
72+
4. Arrows between nodes
73+
5. Then next section
74+
75+
NEVER dump all rectangles, then all text, then all arrows.
76+
77+
### Arrow rules
78+
- Always include `endArrowhead: "arrow"` for directional flow
79+
- Use `strokeStyle: "dashed"` for responses, return values, optional paths
80+
- Keep arrow labels short (under 20 chars) or omit — long labels overflow
81+
- Use `startBinding` / `endBinding` with `fixedPoint` to attach to shapes
82+
83+
---
84+
85+
## Step 4 — Diagram type patterns
86+
87+
### Architecture / System Diagram
88+
Zones as swim lanes (left-to-right or top-to-bottom). Each zone = one architectural layer. Arrows show data/request flow between layers. End with a full-width cameraUpdate.
89+
90+
**Camera pattern:** Title zoom (M) → pan right zone by zone (S/M) → final overview (XL)
91+
92+
### Sequence / Flow Diagram
93+
Actors as header boxes with dashed vertical lifelines. Horizontal arrows show messages. Pan camera downward as messages progress.
94+
95+
**Camera pattern:** Title (M) → pan right per actor drawing header + lifeline → zoom out (L) → pan down per message group → final overview (XL)
96+
97+
### Concept Explainer
98+
Start zoomed on the title, then reveal parts of the concept one at a time. Use annotations (`#fff3bf` boxes) as callouts. Simple left-to-right flow.
99+
100+
**Camera pattern:** Title zoom (S) → zoom out (M) → pan section by section → final (L)
101+
102+
### Process / Flowchart
103+
Diamonds for decisions, rectangles for steps. Top-to-bottom flow. Color-code by stage (e.g. initiation=blue, processing=purple, output=green).
104+
105+
**Camera pattern:** Top zoom → pan down per stage group → final overview
106+
107+
---
108+
109+
## Step 5 — The camera reveal technique (what makes diagrams feel alive)
110+
111+
The secret to great Excalidraw diagrams is **drawing section by section with camera moves**:
112+
113+
```json
114+
// 1. Start with title, zoomed in
115+
{"type":"cameraUpdate","width":600,"height":450,"x":100,"y":0},
116+
{"type":"text","id":"t1","x":200,"y":20,"text":"My Diagram","fontSize":28},
117+
118+
// 2. Pan to first zone and draw it
119+
{"type":"cameraUpdate","width":400,"height":300,"x":20,"y":60},
120+
{"type":"rectangle","id":"zone1", ...zone background...},
121+
{"type":"rectangle","id":"node1", ...node with label...},
122+
123+
// 3. Pan to second zone
124+
{"type":"cameraUpdate","width":400,"height":300,"x":280,"y":60},
125+
{"type":"rectangle","id":"zone2", ...},
126+
{"type":"rectangle","id":"node2", ...},
127+
128+
// 4. Draw connecting arrows (camera stays or pans to show both ends)
129+
{"type":"cameraUpdate","width":800,"height":600,"x":0,"y":40},
130+
{"type":"arrow","id":"a1", ...arrow from node1 to node2...},
131+
132+
// 5. Final wide overview
133+
{"type":"cameraUpdate","width":1200,"height":900,"x":-20,"y":-10}
134+
```
135+
136+
This creates the "drawing itself" animation effect users love.
137+
138+
---
139+
140+
## Step 6 — Common mistakes to avoid
141+
142+
- **No cameraUpdate first** → diagram appears un-framed, elements clip
143+
- **Wrong aspect ratio** → `700x500` causes distortion; use `800x600`
144+
- **All elements at once, no panning** → loses the reveal animation
145+
- **Overlapping elements** → check y-coordinates leave 60–80px between rows
146+
- **Long arrow labels** → overflow the arrow; keep under 20 chars or use a note box instead
147+
- **Emoji in text** → don't render in Excalidraw's font
148+
- **Light text on white** → `#b0b0b0` on white is invisible; minimum `#757575`
149+
- **Zone label covered by nodes** → put zone label text at top-left of zone (y + 8px from zone top), nodes start 40px below
150+
- **Title not centered** → estimate `text.length x fontSize x 0.5` for width, then set `x = diagramCenterX - estimatedWidth/2`
151+
152+
---
153+
154+
## Step 7 — Quality checklist before emitting
155+
156+
- [ ] `Excalidraw:read_me` called
157+
- [ ] First element is `cameraUpdate`
158+
- [ ] All camera sizes are valid 4:3 ratios
159+
- [ ] Minimum 3 camera positions used (more = better animation)
160+
- [ ] Color grammar is consistent across zones
161+
- [ ] All shape labels use `label` property, not separate text elements
162+
- [ ] No font sizes below 14
163+
- [ ] Zone backgrounds are drawn BEFORE the nodes inside them
164+
- [ ] Arrows drawn AFTER both source and target shapes
165+
- [ ] Final element is a wide cameraUpdate revealing the full diagram
166+
- [ ] No emoji in any text strings
167+
168+
---
169+
170+
## Reference: Element snippets
171+
172+
**Zone background:**
173+
```json
174+
{"type":"rectangle","id":"zone_bg","x":20,"y":80,"width":220,"height":380,"backgroundColor":"#dbe4ff","fillStyle":"solid","roundness":{"type":3},"strokeColor":"#4a9eed","strokeWidth":1,"opacity":40}
175+
```
176+
177+
**Zone label:**
178+
```json
179+
{"type":"text","id":"zone_lbl","x":40,"y":88,"text":"FRONTEND","fontSize":14,"strokeColor":"#2563eb"}
180+
```
181+
182+
**Node:**
183+
```json
184+
{"type":"rectangle","id":"n1","x":60,"y":130,"width":150,"height":55,"backgroundColor":"#a5d8ff","fillStyle":"solid","roundness":{"type":3},"strokeColor":"#4a9eed","strokeWidth":2,"label":{"text":"API Gateway","fontSize":16}}
185+
```
186+
187+
**Arrow (solid, directed):**
188+
```json
189+
{"type":"arrow","id":"a1","x":210,"y":157,"width":100,"height":0,"points":[[0,0],[100,0]],"strokeColor":"#1e1e1e","strokeWidth":2,"endArrowhead":"arrow","startBinding":{"elementId":"n1","fixedPoint":[1,0.5]},"endBinding":{"elementId":"n2","fixedPoint":[0,0.5]}}
190+
```
191+
192+
**Arrow (dashed, response):**
193+
```json
194+
{"type":"arrow","id":"a2","x":310,"y":157,"width":-100,"height":0,"points":[[0,0],[-100,0]],"strokeColor":"#757575","strokeWidth":2,"strokeStyle":"dashed","endArrowhead":"arrow"}
195+
```
196+
197+
**Annotation note:**
198+
```json
199+
{"type":"rectangle","id":"note1","x":80,"y":200,"width":200,"height":36,"backgroundColor":"#fff3bf","fillStyle":"solid","roundness":{"type":3},"strokeColor":"#f59e0b","strokeWidth":1,"opacity":80,"label":{"text":"Caches for 5 min","fontSize":14}}
200+
```
201+
202+
**Title text:**
203+
```json
204+
{"type":"text","id":"title","x":150,"y":15,"text":"System Architecture","fontSize":28,"strokeColor":"#1e1e1e"}
205+
```
206+
207+
**Stick figure (user icon):**
208+
```json
209+
{"type":"ellipse","id":"fig_head","x":58,"y":110,"width":20,"height":20,"backgroundColor":"#a5d8ff","fillStyle":"solid","strokeColor":"#4a9eed","strokeWidth":2},
210+
{"type":"rectangle","id":"fig_body","x":57,"y":132,"width":22,"height":26,"backgroundColor":"#a5d8ff","fillStyle":"solid","roundness":{"type":3},"strokeColor":"#4a9eed","strokeWidth":2}
211+
```

apps/app/src/app/api/copilotkit/route.ts

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import {
44
copilotRuntimeNextJSAppRouterEndpoint,
55
} from "@copilotkit/runtime";
66
import { LangGraphAgent } from "@copilotkit/runtime/langgraph";
7+
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";
78
import { NextRequest } from "next/server";
89

910
// 1. Define the agent connection to LangGraph
@@ -13,21 +14,25 @@ const defaultAgent = new LangGraphAgent({
1314
langsmithApiKey: process.env.LANGSMITH_API_KEY || "",
1415
});
1516

17+
// 2. Wire up MCP apps middleware so widget HTML is sent to the frontend
18+
defaultAgent.use(
19+
new MCPAppsMiddleware({
20+
mcpServers: [{
21+
type: "http",
22+
url: process.env.MCP_SERVER_URL || "https://mcp.excalidraw.com",
23+
serverId: "example_mcp_app",
24+
}],
25+
})
26+
);
27+
1628
// 3. Define the route and CopilotRuntime for the agent
1729
export const POST = async (req: NextRequest) => {
1830
const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
1931
endpoint: "/api/copilotkit",
2032
serviceAdapter: new ExperimentalEmptyAdapter(),
2133
runtime: new CopilotRuntime({
22-
agents: { default: defaultAgent, },
34+
agents: { default: defaultAgent },
2335
a2ui: { injectA2UITool: true },
24-
mcpApps: {
25-
servers: [{
26-
type: "http",
27-
url: process.env.MCP_SERVER_URL || "https://mcp.excalidraw.com",
28-
serverId: "example_mcp_app",
29-
}],
30-
},
3136
}),
3237
});
3338

0 commit comments

Comments
 (0)