This document covers the real-time WebSocket streaming system, including the backend Hub architecture, frontend connection handling, event flow, and backpressure management.
- Architecture Overview
- WebSocket Hub
- Event Types
- Frontend Connection
- Backpressure Management
- Error Handling
ChatML uses a hub-and-spoke WebSocket architecture where a central Hub broadcasts events to all connected clients.
graph TB
subgraph Backend
AgentMgr[Agent Manager]
Hub[WebSocket Hub]
Client1[Client 1]
Client2[Client 2]
Client3[Client N]
end
subgraph Frontend
WS1[WebSocket Hook]
WS2[WebSocket Hook]
WS3[WebSocket Hook]
end
AgentMgr -->|Events| Hub
Hub -->|Broadcast| Client1
Hub -->|Broadcast| Client2
Hub -->|Broadcast| Client3
Client1 <-->|WebSocket| WS1
Client2 <-->|WebSocket| WS2
Client3 <-->|WebSocket| WS3
File: backend/server/websocket.go:107-127
type Hub struct {
clients map[*Client]bool // Connected clients
broadcast chan []byte // Pre-serialized JSON (buffer: 1024)
register chan *Client // New client registration
unregister chan *Client // Client disconnection (buffer: 64)
mu sync.RWMutex
metrics *HubMetrics
lastWarningTime atomic.Int64 // Rate-limit warning emissions
}
type Client struct {
hub *Hub
conn *websocket.Conn
send chan []byte // Buffered channel (256 messages)
}File: backend/server/websocket.go:42-48
type Event struct {
Type string `json:"type"`
AgentID string `json:"agentId,omitempty"`
SessionID string `json:"sessionId,omitempty"`
ConversationID string `json:"conversationId,omitempty"`
Payload interface{} `json:"payload,omitempty"`
}flowchart TB
subgraph HubRun["Hub.Run() - Main Loop"]
Start([Start])
Wait{Select on channels}
subgraph Register
R1[Add client to map]
R2[Update metrics]
end
subgraph Unregister
U1[Remove from map]
U2[Close send channel]
U3[Update metrics]
end
subgraph Broadcast
B1[Check buffer utilization]
B2{> 75% full?}
B3[Log warning]
B4[Send to all clients]
B5{Client buffer full?}
B6[Spawn eviction goroutine]
B7[Queue message]
end
end
Start --> Wait
Wait -->|register| R1 --> R2 --> Wait
Wait -->|unregister| U1 --> U2 --> U3 --> Wait
Wait -->|broadcast| B1 --> B2
B2 -->|Yes| B3 --> B4
B2 -->|No| B4
B4 --> B5
B5 -->|Yes| B6 --> Wait
B5 -->|No| B7 --> Wait
File: backend/server/websocket.go:203-262
sequenceDiagram
participant Mgr as Agent Manager
participant Hub
participant Broadcast as Broadcast Channel
participant Client as Client Buffers
Mgr->>Hub: Broadcast(Event)
Hub->>Hub: JSON Marshal event
Hub->>Hub: Check buffer utilization
alt Buffer > 75%
Hub->>Hub: Log backpressure warning
end
Hub->>Broadcast: Send with 2s timeout
alt Timeout
Hub->>Hub: Emit rate-limited warning
Hub-->>Mgr: Return timeout result
end
Broadcast->>Hub: Hub.Run receives
loop For each client
Hub->>Client: Non-blocking send
alt Client buffer full
Hub->>Hub: Spawn eviction goroutine
Hub->>Client: Close and unregister
end
end
| Event Type | Source | Direction | Description |
|---|---|---|---|
init |
Agent | Hub → Client | SDK initialization with model/tools info |
assistant_text |
Agent | Hub → Client | Streamed response text |
thinking_start |
Agent | Hub → Client | Extended thinking began |
thinking_delta |
Agent | Hub → Client | Thinking text chunk |
thinking |
Agent | Hub → Client | Complete thinking block |
tool_start |
Agent | Hub → Client | Tool execution started |
tool_end |
Agent | Hub → Client | Tool execution completed |
tool_progress |
Agent | Hub → Client | Tool still executing |
todo_update |
Agent | Hub → Client | TodoWrite tool executed |
name_suggestion |
Agent | Hub → Client | AI-suggested conversation name |
result |
Agent | Hub → Client | Final result with stats |
complete |
Agent | Hub → Client | Stream completed |
error |
Agent | Hub → Client | Execution error |
conversation_status |
Manager | Hub → Client | Status change (active/idle/completed) |
checkpoint_created |
Agent | Hub → Client | File checkpoint saved |
permission_mode_changed |
Agent | Hub → Client | Permission mode changed |
session_name_update |
Manager | Hub → Client | Session name changed |
session_stats_update |
Manager | Hub → Client | Session stats updated |
session_pr_update |
Manager | Hub → Client | PR status changed |
streaming_warning |
Hub | Hub → Client | Backpressure warning |
flowchart LR
subgraph Agent["Agent Runner (stdout)"]
A1[assistant_text]
A2[tool_start]
A3[tool_end]
A4[result]
A5[thinking]
end
subgraph Parser["Agent Parser"]
P[ParseAgentLine]
end
subgraph Manager["Agent Manager"]
M1[handleConversationOutput]
M2[ConversationEventHandler]
end
subgraph Hub["WebSocket Hub"]
H[Broadcast]
end
subgraph Clients["All Clients"]
C1[Client 1]
C2[Client 2]
C3[Client N]
end
A1 --> P
A2 --> P
A3 --> P
A4 --> P
A5 --> P
P --> M1
M1 --> M2
M2 --> H
H --> C1
H --> C2
H --> C3
File: src/hooks/useWebSocket.ts
// URL Construction (Lines 54-60)
const wsUrl = isTauri
? `ws://localhost:${port}/ws`
: process.env.NEXT_PUBLIC_WS_URL || 'ws://localhost:9876/ws';
// Auth token appended (Line 351)
const urlWithAuth = `${wsUrl}?token=${authToken}`;stateDiagram-v2
[*] --> Disconnected
Disconnected --> Connecting: Connect attempt
Connecting --> Connected: WebSocket open
Connecting --> Disconnected: Connection failed
Connected --> Disconnected: Connection lost
Disconnected --> Connecting: Auto-reconnect (after delay)
Connected --> Connected: Message received
note right of Connecting
WebSocket upgrade
Token validation
end note
note right of Connected
Messages flow
Ping/pong active
end note
note right of Disconnected
Reconnect scheduled
WEBSOCKET_RECONNECT_DELAY_MS
end note
File: src/hooks/useWebSocket.ts:104-314
flowchart TB
WS[WebSocket onmessage]
Parse[JSON.parse]
Type{Event Type?}
subgraph TextEvents["Text Events"]
AT[assistant_text]
AT --> AppendText[appendStreamingText]
AT --> ClearThink[clearThinking]
end
subgraph ThinkEvents["Thinking Events"]
TS[thinking_start]
TD[thinking_delta]
TH[thinking]
TS --> SetThink[setThinking true]
TD --> AppendThink[appendThinkingText]
TH --> AppendThink
end
subgraph ToolEvents["Tool Events"]
TST[tool_start]
TEN[tool_end]
TST --> AddTool[addActiveTool]
TEN --> CompTool[completeActiveTool]
end
subgraph ResultEvents["Result Events"]
RES[result]
COM[complete]
RES --> Finalize[finalizeStreamingMessage]
COM --> Clear[clearStreamingText]
end
WS --> Parse --> Type
Type -->|assistant_text| TextEvents
Type -->|thinking_*| ThinkEvents
Type -->|tool_*| ToolEvents
Type -->|result/complete| ResultEvents
flowchart TB
subgraph Level1["Level 1: Hub Broadcast Channel"]
L1A[1024-message buffer]
L1B[2-second send timeout]
L1C[Graceful timeout handling]
end
subgraph Level2["Level 2: Client Send Buffer"]
L2A[256-message per-client buffer]
L2B[Non-blocking send check]
L2C[Slow client detection]
end
subgraph Level3["Level 3: Client Eviction"]
L3A[Async eviction goroutine]
L3B[Connection close]
L3C[Metrics tracking]
end
L1A --> L1B --> L1C
L1C -->|Buffer OK| L2A
L1C -->|Timeout| Warning1[Rate-limited warning]
L2A --> L2B --> L2C
L2C -->|Buffer OK| Deliver[Deliver message]
L2C -->|Buffer Full| L3A
L3A --> L3B --> L3C
File: backend/server/websocket.go
// Backend: max 1 warning per 5 seconds
const warningCooldown = 5 * time.Second
func (h *Hub) emitBackpressureWarning() {
now := time.Now().Unix()
last := h.lastWarningTime.Load()
if now-last < int64(warningCooldown.Seconds()) {
return // Rate limited
}
if h.lastWarningTime.CompareAndSwap(last, now) {
// Emit warning event
}
}File: src/components/StreamingWarningHandler.tsx:15-22
// Frontend: max 1 toast per 10 seconds
const TOAST_COOLDOWN_MS = 10000;
let lastToastTime = 0;
function showWarning() {
const now = Date.now();
if (now - lastToastTime < TOAST_COOLDOWN_MS) return;
lastToastTime = now;
toast.warning('Streaming data may have been lost');
}File: backend/server/websocket.go:59-105
type HubMetrics struct {
messagesDelivered atomic.Uint64 // Successful transmissions
messagesDropped atomic.Uint64 // Clients removed due to slowness
messagesTimedOut atomic.Uint64 // Broadcast channel timeouts
clientsDropped atomic.Uint64 // Slow client disconnections
broadcastBackpressure atomic.Uint64 // High buffer utilization events
peakClients atomic.Uint64 // Maximum concurrent connections
currentClients atomic.Uint64 // Active connections
}sequenceDiagram
participant FE as Frontend
participant WS as WebSocket
participant Hub as Hub
FE->>WS: Connect
WS->>Hub: Upgrade request
Hub-->>WS: Connection established
WS-->>FE: onopen
Note over FE,Hub: Normal operation...
WS-xHub: Connection lost
WS-->>FE: onclose
FE->>FE: Schedule reconnect
Note over FE: Wait RECONNECT_DELAY_MS
FE->>WS: Reconnect
WS->>Hub: Upgrade request
Hub-->>WS: Connection established
WS-->>FE: onopen
FE->>FE: Clear reconnect timeout
File: backend/server/websocket.go:312-351
flowchart TB
Start([Start writePump])
Wait{Wait on channel}
subgraph Message["Message Received"]
M1[Set 10s write deadline]
M2{Write to WebSocket}
M3[Continue]
M4[Close connection]
end
subgraph Ticker["Ping Ticker ~54s"]
T1[Set write deadline]
T2{Send ping}
T3[Continue]
T4[Close connection]
end
Start --> Wait
Wait -->|send channel| M1
M1 --> M2
M2 -->|Success| M3 --> Wait
M2 -->|Error| M4
Wait -->|ticker| T1
T1 --> T2
T2 -->|Success| T3 --> Wait
T2 -->|Error| T4
File: backend/server/websocket.go:355-381
func (c *Client) readPump() {
defer func() {
c.hub.unregister <- c
c.conn.Close()
}()
c.conn.SetReadLimit(512) // Small limit - only pongs expected
c.conn.SetReadDeadline(time.Now().Add(pongWait)) // 60 seconds
c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(pongWait))
return nil
})
for {
if _, _, err := c.conn.ReadMessage(); err != nil {
break // Connection closed
}
}
}| Metric | Value | Notes |
|---|---|---|
| Hub broadcast buffer | 1024 messages | Pre-serialized JSON |
| Client send buffer | 256 messages | Per-client |
| Broadcast timeout | 2 seconds | Before warning |
| Ping interval | ~54 seconds | 90% of pong wait |
| Pong timeout | 60 seconds | Connection health |
| Write deadline | 10 seconds | Per message |
| Warning cooldown (backend) | 5 seconds | Rate limiting |
| Warning cooldown (frontend) | 10 seconds | Toast rate limiting |