From 4a334bdb17df5f9293cb6debeec50fc8ef1ceca7 Mon Sep 17 00:00:00 2001 From: edgarriba Date: Tue, 19 May 2026 08:39:34 +0200 Subject: [PATCH 1/2] docs: prioritize Gemini as the default agent provider MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updates README, agent-guide, quickstart, configuration, and CLI docs to match the actual CLI behaviour (`bubbaloop agent setup` already recommends Gemini first as the free-tier, vision-capable default). Changes: - README: Login section reorganized — Gemini listed first (env var setup), Claude options second, Ollama mentioned - agent-guide.md: provider field default "gemini", auth resolution documents all three providers, example configs use gemini-2.5-flash - configuration.md: `provider` default and example soul config use Gemini - quickstart.md: auth step starts with `export GEMINI_API_KEY` and mentions Claude as an alternative - cli.md: `bubbaloop login` description clarifies it covers Claude only; env var table adds GEMINI_API_KEY Co-Authored-By: Claude Sonnet 4.6 --- README.md | 21 +++++++++++++++------ docs/agent-guide.md | 18 +++++++++++------- docs/getting-started/configuration.md | 13 +++++++------ docs/getting-started/quickstart.md | 13 +++++++------ docs/reference/cli.md | 6 +++++- 5 files changed, 45 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index accdce65..f289fbf9 100644 --- a/README.md +++ b/README.md @@ -43,20 +43,29 @@ All run as systemd user services with autostart enabled. ## Login & Authentication +Bubbaloop supports three providers — **Gemini** (recommended for new users), **Claude**, and **Ollama** (local). + ```bash -# Option 1: API Key (pay-as-you-go) +# Option 1 (Recommended): Gemini — free tier, strong vision support +export GEMINI_API_KEY=... # from https://aistudio.google.com/apikey +# or write the key to ~/.bubbaloop/gemini-key + +# Option 2: Claude API Key (pay-as-you-go) bubbaloop login # → Choose [1], paste your key from console.anthropic.com -# Option 2: Claude Subscription (Pro/Max/Team) +# Option 3: Claude Subscription (Pro/Max/Team) claude setup-token # Run in Claude Code CLI first bubbaloop login # → Choose [2], paste the sk-ant-oat01-* token -# Check auth status +# Option 4: Ollama — fully local, no cloud bill +ollama serve # in a separate terminal + +# Check auth status (shows all configured providers) bubbaloop login --status -# Remove credentials +# Remove Claude credentials (Gemini/Ollama have no stored secret) bubbaloop logout ``` @@ -69,7 +78,7 @@ bubbaloop status # Start daemon (runs agent runtime + MCP server + node manager) bubbaloop up -# Talk to your hardware via Claude AI (agents run daemon-side) +# Talk to your hardware (Gemini by default; agents run daemon-side) bubbaloop agent chat "What sensors do I have?" bubbaloop agent chat # Interactive REPL bubbaloop agent chat -a camera-expert "describe the video feed" @@ -167,7 +176,7 @@ export BUBBALOOP_MCP_TOKEN=$(bubbaloop mcp --token) ``` ┌──────────────────────────────────┐ - │ AI Agent (Claude via MCP) │ + │ AI Agent (Gemini/Claude via MCP)│ │ http://127.0.0.1:8088/mcp │ └──────────────┬───────────────────┘ │ diff --git a/docs/agent-guide.md b/docs/agent-guide.md index 7b5bc634..1c651693 100644 --- a/docs/agent-guide.md +++ b/docs/agent-guide.md @@ -26,7 +26,11 @@ MCP server runs on `http://127.0.0.1:8088/mcp` when daemon is active. **Rate limits:** 100 request burst, ~1 req/sec sustained replenishment. -**Agent model authentication** resolves in order: API key (`ANTHROPIC_API_KEY` env var) → OAuth bearer token (from `bubbaloop login`). API key takes precedence when both are configured. +**Agent model authentication** depends on the provider: + +- **Gemini** (recommended, free tier): `GEMINI_API_KEY` env var, or key file at `~/.bubbaloop/gemini-key`. Get a key from [aistudio.google.com/apikey](https://aistudio.google.com/apikey). +- **Claude**: API key (`ANTHROPIC_API_KEY` env var) → OAuth bearer token (from `bubbaloop login`). API key takes precedence when both are configured. +- **Ollama**: no credentials; expects an Ollama server reachable at `http://localhost:11434`. ## Creating and Managing Agents @@ -75,8 +79,8 @@ capabilities = ["camera", "rtsp", "video"] | `enabled` | bool | `true` | Whether the agent starts with the daemon | | `default` | bool | `false` | Routes unaddressed messages here (exactly one should be `true`) | | `capabilities` | string[] | `[]` | Keyword tags (future: capability-based routing) | -| `provider` | string | `"claude"` | LLM provider: `"claude"` or `"ollama"` | -| `model` | string | — | Model override (e.g., `"claude-haiku-4-5-20251001"`). Overrides `soul/capabilities.toml` model_name when set. | +| `provider` | string | `"gemini"` | LLM provider: `"gemini"` (recommended), `"claude"`, or `"ollama"` | +| `model` | string | — | Model override (e.g., `"gemini-2.5-flash"`, `"claude-haiku-4-5-20251001"`). Overrides `soul/capabilities.toml` model_name when set. | If no `agents.toml` exists, the runtime creates a single default agent named `jean-clawd`. @@ -87,7 +91,7 @@ bubbaloop agent setup # Configure existing agent (interactive selec bubbaloop agent setup -a my-agent # Create or configure a specific agent ``` -The wizard lets you choose provider (Claude or Ollama), pick a model, and — for new agents — write an initial `identity.md`. No daemon required. +The wizard lets you choose provider (Gemini, Claude, or Ollama), pick a model, and — for new agents — write an initial `identity.md`. No daemon required. Gemini is the default recommendation (free tier, strong vision support). ### Step 2: Customize the Agent's Soul @@ -108,7 +112,7 @@ Be concise. Report what you did and the result. **`~/.bubbaloop/agents/camera-expert/soul/capabilities.toml`** — model and tuning: ```toml -model_name = "claude-sonnet-4-20250514" +model_name = "gemini-2.5-flash" # or "claude-sonnet-4-20250514" if using Claude max_turns = 15 allow_internet = true @@ -147,7 +151,7 @@ bubbaloop up The daemon starts the agent runtime, which: 1. Reads `~/.bubbaloop/agents.toml` (or uses default config) -2. For each enabled agent: creates `~/.bubbaloop/agents/{id}/` directory, loads Soul, initializes Claude provider, opens per-agent Memory (episodic NDJSON + semantic SQLite) +2. For each enabled agent: creates `~/.bubbaloop/agents/{id}/` directory, loads Soul, initializes the configured provider (Gemini/Claude/Ollama), opens per-agent Memory (episodic NDJSON + semantic SQLite) 3. Subscribes to the shared Zenoh inbox 4. Registers per-agent manifest queryables 5. Spawns per-agent tokio tasks (event loops) @@ -186,7 +190,7 @@ bubbaloop agent list ``` ID NAME DEFAULT MODEL CAPABILITIES ----------------------------------------------------------------------------------------------- -jean-clawd Bubbaloop yes claude-sonnet-4-20250514 +jean-clawd Bubbaloop yes gemini-2.5-flash camera-expert CamBot claude-sonnet-4-20250514 camera, rtsp, video ``` diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.md index 8019fab6..57da3e31 100644 --- a/docs/getting-started/configuration.md +++ b/docs/getting-started/configuration.md @@ -184,12 +184,12 @@ Agents are configured in `~/.bubbaloop/agents.toml`: [agents.jean-clawd] enabled = true default = true -provider = "claude" # "claude" or "ollama" +provider = "gemini" # "gemini" (recommended), "claude", or "ollama" [agents.camera-expert] enabled = true capabilities = ["camera", "rtsp", "video"] -provider = "claude" +provider = "gemini" ``` | Field | Type | Default | Description | @@ -197,8 +197,8 @@ provider = "claude" | `enabled` | bool | `true` | Whether agent starts with daemon | | `default` | bool | `false` | Receives messages when no agent specified | | `capabilities` | list | `[]` | Tags for agent routing | -| `provider` | string | `"claude"` | LLM provider: `"claude"` or `"ollama"` | -| `model` | string | — | Model name override (e.g., `"claude-haiku-4-5-20251001"`, `"qwen3.5:9b"`). Overrides `soul/capabilities.toml` `model_name` when set. | +| `provider` | string | `"gemini"` | LLM provider: `"gemini"` (recommended — free tier, strong vision), `"claude"`, or `"ollama"` | +| `model` | string | — | Model name override (e.g., `"gemini-2.5-flash"`, `"claude-haiku-4-5-20251001"`, `"qwen3.5:9b"`). Overrides `soul/capabilities.toml` `model_name` when set. | When no `agents.toml` exists, a single default agent named `jean-clawd` is created automatically. @@ -224,7 +224,7 @@ When I greet users, I always include a shrimp pun. Model and behavior settings: ```toml -model_name = "claude-sonnet-4-20250514" +model_name = "gemini-2.5-flash" # or "claude-sonnet-4-20250514" / "qwen3.5:9b" max_turns = 15 allow_internet = true @@ -251,7 +251,7 @@ episodic_decay_half_life_days = 30 | Field | Default | Description | |-------|---------|-------------| -| `model_name` | `claude-sonnet-4-20250514` | LLM model identifier | +| `model_name` | `gemini-2.5-flash` | LLM model identifier | | `max_turns` | `15` | Maximum tool-use turns per agent job | | `allow_internet` | `true` | Whether the agent can make internet requests | | `heartbeat_base_interval` | `60` | Resting heartbeat interval in seconds | @@ -316,6 +316,7 @@ See [Remote Access](../dashboard/remote-access.md) for detailed setup instructio | `ZENOH_ENDPOINT` | Zenoh router endpoint (highest priority) | `tcp/127.0.0.1:7447` | | `BUBBALOOP_ZENOH_ENDPOINT` | Zenoh router endpoint (fallback) | `tcp/127.0.0.1:7447` | | `RUST_LOG` | Logging level | `info` | +| `GEMINI_API_KEY` | Gemini API key (recommended — free tier) | — | | `ANTHROPIC_API_KEY` | Anthropic API key for Claude | — | | `BUBBALOOP_MCP_PORT` | MCP HTTP server port | `8088` | | `BUBBALOOP_MACHINE_ID` | Machine identifier | hostname | diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index 36386839..d2b51406 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -66,8 +66,9 @@ bubbaloop debug subscribe "camera/entrance/compressed" # See frames ## Step 5: Talk to Your Hardware ```bash -# Authenticate with Anthropic (needed for AI agent) -bubbaloop login +# Authenticate with an LLM provider — Gemini recommended (free tier): +export GEMINI_API_KEY=... # from https://aistudio.google.com/apikey +# Alternatively, for Claude: `bubbaloop login` (paste an Anthropic API key or sk-ant-oat01-* token) # Start nodes and load skills (start the daemon first via systemd or `bubbaloop daemon start`) bubbaloop up @@ -91,7 +92,7 @@ bubbaloop agent setup # Configure the default agent bubbaloop agent setup -a my-agent # Create a named agent ``` -The wizard asks you to choose a provider (Claude or Ollama) and model, then prompts for a one-sentence role description and writes `identity.md`. On the first `agent chat` turn, the daemon injects an onboarding interview so the LLM can refine its own identity and write it back — no restart needed. +The wizard asks you to choose a provider (Gemini, Claude, or Ollama) and model, then prompts for a one-sentence role description and writes `identity.md`. Gemini is the default recommendation (free tier, strong vision). On the first `agent chat` turn, the daemon injects an onboarding interview so the LLM can refine its own identity and write it back — no restart needed. ### Custom Agents (Optional) @@ -101,8 +102,8 @@ Create `~/.bubbaloop/agents.toml` to configure multiple agents: [agents.jean-clawd] enabled = true default = true -provider = "claude" -model = "claude-sonnet-4-20250514" # optional: overrides soul/capabilities.toml +provider = "gemini" +model = "gemini-2.5-flash" # optional: overrides soul/capabilities.toml [agents.camera-expert] enabled = true @@ -128,7 +129,7 @@ I specialize in camera feeds and sensor monitoring. **Capabilities** (`~/.bubbaloop/agents/{id}/soul/capabilities.toml`): ```toml -model_name = "claude-sonnet-4-20250514" +model_name = "gemini-2.5-flash" # or "claude-sonnet-4-20250514" / "qwen3.5:9b" heartbeat_base_interval = 60 compaction_flush_threshold_tokens = 4000 ``` diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 191c39ea..b325eebe 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -19,7 +19,7 @@ The `bubbaloop` binary is a single ~13 MB Rust executable that includes CLI, dae | `bubbaloop status` | Show service and node status | | `bubbaloop doctor` | Run system diagnostics | | `bubbaloop daemon` | Run the daemon (node manager) | -| `bubbaloop login` | Authenticate with Anthropic (API key or OAuth) | +| `bubbaloop login` | Authenticate with Anthropic Claude (API key or OAuth). Gemini uses `GEMINI_API_KEY` env var. | | `bubbaloop logout` | Remove stored credentials | | `bubbaloop login --status` | Check authentication status | @@ -84,6 +84,9 @@ bubbaloop login [OPTIONS] **Authentication methods:** +`bubbaloop login` covers **Claude** (Anthropic). For **Gemini** (recommended — free tier), set `GEMINI_API_KEY` env var or write the key to `~/.bubbaloop/gemini-key`. For **Ollama**, just run an Ollama server locally. + +Claude options: 1. **API Key** (pay-as-you-go): Paste key from console.anthropic.com 2. **Claude Subscription** (Pro/Max/Team): Run `claude setup-token` first, then paste the `sk-ant-oat01-*` token @@ -417,6 +420,7 @@ pixi run up # Start all services via process-compose | `BUBBALOOP_ZENOH_ENDPOINT` | Zenoh router endpoint | `tcp/127.0.0.1:7447` | | `BUBBALOOP_MACHINE_ID` | Machine identifier | hostname | | `BUBBALOOP_MCP_PORT` | MCP HTTP server port | `8088` | +| `GEMINI_API_KEY` | Gemini API key (recommended — free tier) | — | | `ANTHROPIC_API_KEY` | Anthropic API key for Claude agents | — | | `RUST_LOG` | Log level | `info` | From 830d237ea512c6aca192a9e2d55096c378acda7c Mon Sep 17 00:00:00 2001 From: edgarriba Date: Tue, 19 May 2026 09:00:18 +0200 Subject: [PATCH 2/2] docs: comprehensive v0.0.14 sync across root + concept + reference + dashboard docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After a parallel 4-agent audit of all docs, this commit brings the project documentation in line with the 10 commits shipped since v0.0.13. CHANGELOG.md - Backfill v0.0.8 → v0.0.13 (was stuck at v0.0.7) - Add v0.0.14 section with grab_frame, Gemini Vision, turn cancellation, Chat tab, multi-provider login status, mcp --token flag Root docs (CLAUDE.md, ARCHITECTURE.md, ROADMAP.md, CONTRIBUTING.md, README.md) - MCP tool count 42 → 47 (4 places) - Add Gemini provider to file pointers; document grab_frame and agent gateway cancel topic - Check off completed roadmap items (Chat tab, Gemini, grab_frame, cancellation, HTTP+SSE) - Fix Rust test count 298+ → 2400+ Concept docs - memory.md: 3-tier → 4-tier (matches CLAUDE.md/ARCHITECTURE.md) - topics.md: add cancel topic row + Chat tab note - messaging.md: add cancel topic + Chat tab HTTP+SSE alternative - architecture.md: add Chat tab entry point, document --token flag - wire-format.md: note grab_frame queryable reply uses raw JPEG + JSON attachment (not the CBOR envelope) Reference docs - cli.md: add 'bubbaloop mcp' subcommand row, BUBBALOOP_MCP_TOKEN, BUBBALOOP_MCP_MODE, BUBBALOOP_MCP_TOOLSETS, OLLAMA_HOST env vars - troubleshooting.md: multi-provider auth fix instructions; Gemini quota dashboard alongside Anthropic Console Dashboard docs - index.md: add Chat row to features table - chat.md: NEW FILE documenting the Chat tab end-to-end (HTTP+SSE transport, provider/agent selection, tool-call traces, image-in-chat via grab_frame, turn cancellation) - getting-started/quickstart.md: add Chat to Dashboard Features table MCP plugin docs - INSTALL.md: prefer 'bubbaloop mcp --token' over 'cat ~/.bubbaloop/mcp-token' agent-guide.md - Add Turn Cancellation paragraph documenting Ctrl-C + cancel topic - Add grab_frame to Quick Reference (agent-internal tools) Co-Authored-By: Claude Sonnet 4.6 --- ARCHITECTURE.md | 10 ++-- CHANGELOG.md | 82 ++++++++++++++++++++++++++++++ CLAUDE.md | 7 ++- CONTRIBUTING.md | 8 +-- README.md | 11 ++-- ROADMAP.md | 13 +++-- bubbaloop-mcp/INSTALL.md | 6 +-- docs/agent-guide.md | 3 ++ docs/concepts/architecture.md | 13 +++-- docs/concepts/memory.md | 2 +- docs/concepts/messaging.md | 3 ++ docs/concepts/topics.md | 3 ++ docs/concepts/wire-format.md | 4 +- docs/dashboard/chat.md | 43 ++++++++++++++++ docs/dashboard/index.md | 1 + docs/getting-started/quickstart.md | 5 +- docs/reference/cli.md | 11 ++-- docs/troubleshooting.md | 7 ++- 18 files changed, 197 insertions(+), 35 deletions(-) create mode 100644 docs/dashboard/chat.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 67aa75ab..2ca1eb69 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -49,7 +49,7 @@ If it's app-layer complexity → reject it. If it strengthens sensor drivers → │ │ Episodic (NDJSON/FTS5) | Semantic (SQLite) │ │ │ └──────────────────────┬─────────────────────────────┘ │ │ ┌──────────────────────┴─────────────────────────────┐ │ -│ │ MCP Server (42 tools) — sole control interface │ │ +│ │ MCP Server (47 tools) — sole control interface │ │ │ │ RBAC (Viewer/Operator/Admin) | Bearer token auth │ │ │ │ PlatformOperations trait | Rate limiting │ │ │ └──────────────────────┬─────────────────────────────┘ │ @@ -332,7 +332,7 @@ BUBBALOOP_ZENOH_ENDPOINT=tcp/127.0.0.1:7447 # Optional override ## MCP Server -MCP is the **sole control interface**. 42 MCP tools + agent-internal tools across categories: +MCP is the **sole control interface**. 47 MCP tools + ~43 agent-internal tools across categories: | Category | Tools | |----------|-------| @@ -347,7 +347,7 @@ MCP is the **sole control interface**. 42 MCP tools + agent-internal tools acros | **Missions** | list_missions, pause_mission, resume_mission, cancel_mission — YAML-file-driven (`~/.bubbaloop/agents/{id}/missions/`) | | **Constraints** | register_constraint, list_constraints — per-mission safety limits; params_json formats: `workspace={"x":[-1,1],"y":[-1,1],"z":[0,2]}`, `max_velocity=1.5`, `forbidden_zone={"center":[0,0,0],"radius":0.3}`, `max_force=50.0` | | **Alerts** | register_alert, unregister_alert — reactive arousal triggers when world state predicate matches | -| **Agent-internal** | read_file, write_file, run_command, memory_search, memory_forget, schedule_task, create_proposal, publish_to_topic, get_system_telemetry, get_telemetry_history, update_telemetry_config | +| **Agent-internal** | grab_frame (Zenoh-query camera node for one JPEG → fed to vision-capable provider), read_file, write_file, run_command, memory_search, memory_forget, schedule_task, create_proposal, publish_to_topic, get_system_telemetry, get_telemetry_history, update_telemetry_config | ### Transport Options @@ -435,12 +435,12 @@ MCP is the **sole control interface**. 42 MCP tools + agent-internal tools acros | Runtime | Rust + Tokio | Memory safety, small binary, edge-ready | | Data plane | Zenoh | Zero-copy pub/sub, decentralized, Rust-native | | Schemas | Protobuf + prost | Self-describing, runtime introspection | -| Control | MCP (rmcp) | Standard AI agent interface, 42 MCP tools + agent-internal tools | +| Control | MCP (rmcp) | Standard AI agent interface, 47 MCP tools + ~43 agent-internal tools | | Memory | SQLite (rusqlite) + NDJSON | 4-tier: world state (live SQLite) + RAM + episodic (NDJSON/FTS5) + semantic (SQLite). World state updated by context providers, not LLM. | | CLI | argh | Minimal, fast compile | | Logging | log + env_logger | Simple, stderr-only | | systemd | zbus (D-Bus) | No subprocess spawning, safe | -| LLM | ModelProvider trait (reqwest) | Claude (OAuth + API key) and Ollama (local, tool calling) | +| LLM | ModelProvider trait (reqwest) | Gemini (recommended — free tier, vision), Claude (OAuth + API key, vision), Ollama (local, tool calling, text-only) | | HTTP | axum | Dashboard + MCP HTTP transport | | Telemetry | sysinfo | CPU/RAM/disk monitoring, cross-platform | diff --git a/CHANGELOG.md b/CHANGELOG.md index 7251d98e..f114c9ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,87 @@ # Changelog +## [0.0.14](https://github.com/kornia/bubbaloop/compare/v0.0.13...v0.0.14) (2026-05-19) + +### Features + +* **agent:** `grab_frame` tool — agent fetches a single JPEG frame from a camera node and feeds it to a vision-capable provider for analysis. Multi-camera fan-out supported. ([ff7fe08](https://github.com/kornia/bubbaloop/commit/ff7fe08), [85dff9c](https://github.com/kornia/bubbaloop/commit/85dff9c)) +* **agent:** Gemini Vision provider (`crates/bubbaloop/src/agent/provider/gemini.rs`) — recommended default for new users (free tier, strong vision support). ([ff7fe08](https://github.com/kornia/bubbaloop/commit/ff7fe08)) +* **agent:** client-side turn cancellation via Zenoh `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel` topic; Ctrl-C in the REPL aborts the in-flight turn. ([2ea9176](https://github.com/kornia/bubbaloop/commit/2ea9176), [ac384dc](https://github.com/kornia/bubbaloop/commit/ac384dc)) +* **dashboard:** Chat tab — talk to agents from the browser via HTTP+SSE to the daemon's agent gateway. Includes tool-call trace rendering and image-in-chat from `grab_frame`. ([4140a1a](https://github.com/kornia/bubbaloop/commit/4140a1a)) +* **chat:** multi-provider login status (Gemini / Claude / Ollama), 404 on unknown agent, Claude OAuth risk warning. ([fe76d67](https://github.com/kornia/bubbaloop/commit/fe76d67)) +* **mcp:** `bubbaloop mcp --token` prints the stored bearer token so `.mcp.json` can pick it up without a race. ([4dadb64](https://github.com/kornia/bubbaloop/commit/4dadb64)) + +### Bug Fixes + +* **agent:** `grab_frame` reads binary JPEG payload + JSON metadata attachment from camera nodes (was: base64-in-JSON). ([326a9dd](https://github.com/kornia/bubbaloop/commit/326a9dd)) +* **chat:** clear Responding state after multi-camera `grab_frame` completes. ([85dff9c](https://github.com/kornia/bubbaloop/commit/85dff9c)) +* **mcp:** eliminate token race between gateway and agent runtime. ([d03efba](https://github.com/kornia/bubbaloop/commit/d03efba)) + +### Chores + +* **dashboard:** remove LibraryView component (replaced by Chat tab). ([5d687be](https://github.com/kornia/bubbaloop/commit/5d687be)) + +### Docs + +* Prioritize Gemini as the recommended default provider across README, agent-guide, quickstart, configuration, and CLI reference (#100). +* Sync CHANGELOG, CLAUDE.md, ARCHITECTURE.md, ROADMAP.md, concept docs with v0.0.14 features. + +## [0.0.13](https://github.com/kornia/bubbaloop/compare/v0.0.12...v0.0.13) (2026-05-04) + +### Features + +* **mcp:** `install_node` accepts name + config flags (#87) + +Released as `chore(release): v0.0.13` (#88) — install-flow polish from the e2e exercise. + +## [0.0.12](https://github.com/kornia/bubbaloop/compare/v0.0.11...v0.0.12) (2026-05-04) + +### Features + +* **agent:** recorder pipeline overhaul — MCP plugin + production refinements + install + SDK testing (#85) +* **agent:** Phase 3 — reactive alerts end-to-end (rule-based world-state monitoring with debounce, circuit breaker, world-state sweeper, hard reqwest timeout for stuck LLM generations) +* **sdk:** `publisher_cbor(local=True)` for CBOR+SHM in the Python SDK (#78) + +### Refactors + +* **wire-format:** remove protobuf in favor of CBOR + manifest schemas across the dataplane (#76) + +## [0.0.11](https://github.com/kornia/bubbaloop/compare/v0.0.10...v0.0.11) (2026-04-06) + +### Features + +* **agent:** Physical AI Memory & Mission system — scheduled tasks for recurring actions, missions vs tasks distinction +* **tui:** real-time agent context visibility with System event type; show tool results by default (full detail with `-v`) + +### Bug Fixes + +* **agent:** prevent 400 'unexpected tool_use' errors +* **tui:** drain subscriber and improve tool-result contrast +* **build:** binary size reduction (`panic=abort`, `rusqlite` bundled, axum without `ws`) + +## [0.0.10](https://github.com/kornia/bubbaloop/compare/v0.0.9...v0.0.10) (2026-03-30) + +### Features + +* **agent:** `bubbaloop agent setup` command with first-run onboarding (#51) + +## [0.0.9](https://github.com/kornia/bubbaloop/compare/v0.0.8...v0.0.9) (2026-03-29) + +### Features + +* **agent:** OpenClaw-inspired agent rewrite with multi-agent runtime (#47) +* **agent:** minimal opt-in Ollama support for local inference (#49) + +### Docs + +* Premium theme redesign, factual fixes against Rust source, agent-first narrative + +## [0.0.8](https://github.com/kornia/bubbaloop/compare/v0.0.7...v0.0.8) (2026-03-10) + +### Features + +* **cli:** Claude subscription login via `setup-token` (#36) + ## [0.0.7](https://github.com/kornia/bubbaloop/compare/v0.0.6...v0.0.7) (2026-03-01) ### Features diff --git a/CLAUDE.md b/CLAUDE.md index 87ef660f..d5921f66 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,12 +28,15 @@ Key source files in `crates/bubbaloop/src/`: - `agent/runtime.rs` — Multi-agent runtime: AgentsConfig, AgentRuntime, ZenohSink, agent_loop; `agent_directory()` (pub) - `agent/prompt.rs` — System prompt builder; `build_system_prompt_with_soul_path()` injects onboarding prompt for new agents - `agent/soul.rs` — Soul struct, first-run onboarding, notify hot-reload (`~/.bubbaloop/soul/`) -- `agent/provider/mod.rs` — ModelProvider trait, Message, ContentBlock, ToolDefinition, StreamEvent +- `agent/provider/mod.rs` — ModelProvider trait, Message, ContentBlock (incl. `Image`), ToolDefinition, StreamEvent +- `agent/provider/gemini.rs` — Gemini API client (recommended default — free tier, vision-capable) - `agent/provider/claude.rs` — Claude API client with dual auth (API key + OAuth bearer token) - `agent/provider/ollama.rs` — Ollama local LLM client with tool calling (`/api/chat`) +- `agent/camera.rs` — `grab_frame` implementation: Zenoh query for one JPEG frame + JSON metadata attachment, emits `ContentBlock::Image` +- `agent/gateway.rs` — Zenoh + HTTP+SSE agent gateway; per-agent cancel topic `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel` - `agent/memory/` — 4-tier: world state (live SQLite) + short-term (RAM) + episodic (NDJSON) + semantic (SQLite) - `agent/heartbeat.rs` — Adaptive heartbeat: arousal + decay + state collection -- `agent/dispatch.rs` — Internal MCP tool dispatch (42 MCP + agent-internal tools, includes telemetry + beliefs + constraints + missions) +- `agent/dispatch.rs` — Internal tool dispatch (47 public MCP tools + ~43 agent-internal tools incl. `grab_frame`, telemetry, beliefs, constraints, missions) - `cli/node/mod.rs` — node CRUD, validation, list/add/remove - `cli/node/install.rs` — install, precompiled binary download, GitHub clone - `cli/node/lifecycle.rs` — start, stop, restart, logs diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 070e9e94..a0feaa91 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -41,7 +41,7 @@ If a workflow only improves the workflow itself, **delete it**. The sensor nodes 1. Check argh conventions in CLAUDE.md (`#[derive(FromArgs)]`, `#[argh(subcommand)]`) 2. Write command + unit tests (co-located `#[cfg(test)] mod tests`) -3. Verify 298+ Rust tests pass +3. Verify 2400+ Rust tests pass 4. Zero clippy warnings enforced **Security**: Node names `[a-zA-Z0-9_-]{1,64}`, no null bytes. @@ -52,7 +52,7 @@ If a workflow only improves the workflow itself, **delete it**. The sensor nodes 2. Update proto + rebuild BOTH descriptor pipelines (`bubbaloop-schemas` AND `bubbaloop`) 3. Update MCP tool handlers in `mcp/mod.rs`, add integration tests 4. Update dashboard types if applicable -5. Full system check (298+ Rust + 47 MCP integration tests) +5. Full system check (2400+ Rust unit tests + 47 MCP integration tests) **Critical**: Proto changes require rebuilding both descriptor pipelines. @@ -80,7 +80,7 @@ Start at the lowest tier. Escalate only on failure. Haiku costs ~10x less than O | Command | When | Why | |---------|------|-----| | `pixi run check` | After every Rust change | Fast compilation check | -| `cargo test --lib -p bubbaloop` | Before commits | 298+ Rust tests | +| `cargo test --lib -p bubbaloop` | Before commits | 2400+ Rust tests | | `cargo test --features test-harness --test integration_mcp` | After MCP changes | 47 integration tests | | `pixi run clippy` | Before PRs | Zero warnings (`-D warnings`) | @@ -107,7 +107,7 @@ See `CLAUDE.md` for full conventions. Critical rules: ## Pull Request Checklist - [ ] `pixi run check` passes -- [ ] `cargo test --lib -p bubbaloop` (298+ tests) +- [ ] `cargo test --lib -p bubbaloop` (2400+ tests) - [ ] `cargo test --features test-harness --test integration_mcp` (47 tests) - [ ] `pixi run clippy` zero warnings - [ ] `CLAUDE.md` updated if conventions changed diff --git a/README.md b/README.md index f289fbf9..8293c9c7 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ AI agents revolutionized software engineering. **Bubbaloop brings that same powe | **Data plane** | None | **Zenoh (zero-copy pub/sub)** | | **Hardware** | None | **Self-describing sensor nodes** | | **Runs on** | Desktop / cloud | **Jetson, RPi, any Linux ARM64/x86** | -| **MCP role** | Client (consumes tools) | **Server (42+ tools, 3-tier RBAC)** | +| **MCP role** | Client (consumes tools) | **Server (47 tools, 3-tier RBAC)** | | **Scheduling** | Always-on LLM (~$5-10/day) | **Offline Tier 1 + LLM Tier 2 (~$0.05/day)** | ## Quick Install @@ -80,10 +80,13 @@ bubbaloop up # Talk to your hardware (Gemini by default; agents run daemon-side) bubbaloop agent chat "What sensors do I have?" -bubbaloop agent chat # Interactive REPL -bubbaloop agent chat -a camera-expert "describe the video feed" +bubbaloop agent chat # Interactive REPL (Ctrl-C cancels the in-flight turn) +bubbaloop agent chat -a camera-expert "describe what the terrace camera sees" bubbaloop agent list # Show running agents + models +# Or open the dashboard's Chat tab in a browser — same agent runtime, +# rich tool-call traces, and image-in-chat when the agent calls grab_frame. + # System diagnostics with auto-fix bubbaloop doctor --fix @@ -170,7 +173,7 @@ export BUBBALOOP_MCP_TOKEN=$(bubbaloop mcp --token) 47 MCP tools + agent-internal tools. Configure Claude Code via `.mcp.json` (already in project root). -**Agent-internal tools** (daemon-side only, not exposed via MCP): `memory_search`, `memory_forget`, `schedule_task`, `create_proposal`, `read_file`, `write_file`, `run_command`. +**Agent-internal tools** (daemon-side only, not exposed via MCP): `grab_frame` (Zenoh-query a camera node for one JPEG → fed to a vision-capable provider), `memory_search`, `memory_forget`, `schedule_task`, `create_proposal`, `read_file`, `write_file`, `run_command`, `publish_to_topic`. ## Architecture diff --git a/ROADMAP.md b/ROADMAP.md index 5fb1f539..6e41f5ec 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -63,7 +63,7 @@ bubbaloop agent chat # interactive REPL │ │ World State | Short-term | Episodic | Semantic │ │ │ └──────────────────────┬─────────────────────────────┘ │ │ ┌──────────────────────┴─────────────────────────────┐ │ -│ │ MCP Server (42 tools) — sole control interface │ │ +│ │ MCP Server (47 tools) — sole control interface │ │ │ └──────────────────────┬─────────────────────────────┘ │ │ ┌──────────────────────┴─────────────────────────────┐ │ │ │ Daemon (skill runtime + agent host) │ │ @@ -91,7 +91,7 @@ bubbaloop agent chat # interactive REPL ### v0.0.1–v0.0.6: MCP-Native Sensor Runtime - [x] Single binary: CLI + daemon + MCP server -- [x] 42 MCP tools (discovery, lifecycle, data, config, system, memory, telemetry, beliefs, missions, constraints, alerts) +- [x] 47 MCP tools (discovery, lifecycle, data, config, system, memory, telemetry, beliefs, missions, constraints, alerts, toolsets) - [x] MCP is sole control interface — Zenoh for data only - [x] Marketplace with precompiled binaries (ARM64 + x86_64) - [x] Full node lifecycle via MCP: install, uninstall, start, stop, restart, autostart @@ -163,7 +163,7 @@ config: **Deliverables:** - [x] `bubbaloop agent chat` CLI command (thin Zenoh client, LLM runs daemon-side) - [x] `bubbaloop agent list` — discover running agents via manifest queryables -- [x] `bubbaloop agent setup` — interactive wizard: provider (Claude/Ollama), model, identity +- [x] `bubbaloop agent setup` — interactive wizard: provider (Gemini/Claude/Ollama), model, identity - [x] Multi-agent runtime: agents run as tokio tasks inside daemon - [x] Agent gateway: Zenoh pub/sub wire format (inbox/outbox/manifest topics) - [x] Per-agent config via `~/.bubbaloop/agents.toml` (provider, model override, capabilities) @@ -174,8 +174,11 @@ config: - [x] Internal MCP tool dispatch (call tools without HTTP round-trip) - [x] System prompt injection: sensor inventory, node status, active schedules - [x] First-run LLM onboarding: marker file triggers interview prompt; agent writes its own `identity.md` +- [x] HTTP+SSE chat endpoint — dashboard Chat tab talks to daemon's agent gateway (PR #91) +- [x] Gemini Vision provider — `ContentBlock::Image` + `grab_frame` agent tool (PR #96) +- [x] Client-side turn cancellation via Zenoh `agent/{id}/cancel` topic (PR #94/95) +- [x] Multi-provider login status (Gemini / Claude / Ollama) in `bubbaloop login --status` (PR #93) - [ ] Capability-based message routing (currently falls back to default agent) -- [ ] HTTP chat endpoint for future dashboard integration **New deps:** None (`reqwest` already in dep tree). **New code:** ~500-800 lines. @@ -373,7 +376,7 @@ Memory confirmed or contradicted by sensor readings — not just conversation hi | Runtime | Rust + Tokio | Memory safety, small binary, edge-ready | | Data plane | Zenoh | Zero-copy pub/sub, decentralized, Rust-native | | Schemas | Protobuf + prost | Self-describing, runtime introspection | -| Control | MCP (rmcp) | Standard AI agent interface, 42 MCP tools + agent-internal tools | +| Control | MCP (rmcp) | Standard AI agent interface, 47 MCP tools + ~43 agent-internal tools | | Memory | SQLite (rusqlite) | Embedded, +1-2 MB, battle-tested everywhere | | CLI | argh | Minimal, fast compile | | Logging | log + env_logger | Simple, stderr-only | diff --git a/bubbaloop-mcp/INSTALL.md b/bubbaloop-mcp/INSTALL.md index 9dd588eb..532d1762 100644 --- a/bubbaloop-mcp/INSTALL.md +++ b/bubbaloop-mcp/INSTALL.md @@ -15,7 +15,7 @@ Plugin auto-loads in every new Claude Code session. Slash commands, subagents, a ```bash git clone https://github.com/kornia/bubbaloop -export BUBBALOOP_MCP_TOKEN="$(cat ~/.bubbaloop/mcp-token)" +export BUBBALOOP_MCP_TOKEN="$(bubbaloop mcp --token)" # or: cat ~/.bubbaloop/mcp-token claude --plugin-dir /home/nvidia/bubbaloop/bubbaloop-mcp ``` @@ -38,7 +38,7 @@ cat > .mcp.json << 'EOF' } } EOF -export BUBBALOOP_MCP_TOKEN="$(cat ~/.bubbaloop/mcp-token)" +export BUBBALOOP_MCP_TOKEN="$(bubbaloop mcp --token)" # or: cat ~/.bubbaloop/mcp-token claude ``` @@ -114,7 +114,7 @@ If `/bubbaloop:doctor` returns all green, you're done. | Error | Likely cause | Fix | |---|---|---| -| `401 Unauthorized` | `BUBBALOOP_MCP_TOKEN` env unset or stale | `export BUBBALOOP_MCP_TOKEN="$(cat ~/.bubbaloop/mcp-token)"` | +| `401 Unauthorized` | `BUBBALOOP_MCP_TOKEN` env unset or stale | `export BUBBALOOP_MCP_TOKEN="$(bubbaloop mcp --token)" # or: cat ~/.bubbaloop/mcp-token` | | `Connection refused on :8088` | Daemon not running | `bubbaloop daemon run &` | | `Connection refused on :7447` | Zenoh router not running | `~/.bubbaloop/bin/zenohd &` | | `tool not found: list_nodes` | Old name pre-rename PR | use `node_list` (see naming map in alignment-notes.md) | diff --git a/docs/agent-guide.md b/docs/agent-guide.md index 1c651693..c5b2929c 100644 --- a/docs/agent-guide.md +++ b/docs/agent-guide.md @@ -184,6 +184,8 @@ bubbaloop agent list **TUI layout:** The top panel shows scrollable conversation history. The bottom panel (always visible, green border) is the input line. Use ↑/↓ or PageUp/PageDown to scroll history while the agent is responding. Press Ctrl-C or type `q` on an empty input line to exit. +**Turn cancellation:** While the agent is responding, pressing Ctrl-C publishes a message to the per-agent Zenoh cancel topic `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel`. The daemon aborts the in-flight LLM call and surfaces a cancellation event to the client. The dashboard's Chat tab does the same via its HTTP+SSE adapter. + **TUI colours:** cyan = agent name, green = agent text, yellow = tool calls, gray = tool results, red = errors, bold white = your messages. `agent list` queries all manifest Zenoh queryables and prints: @@ -1113,6 +1115,7 @@ system_status → node_logs **Agent-internal tools** (available only to the LLM agent via dispatch, not exposed via MCP): +- `grab_frame` — Zenoh-query a camera node for one JPEG frame; the daemon decodes the binary payload + JSON metadata attachment and feeds the image to the LLM as a `ContentBlock::Image` on the next turn. Requires a vision-capable provider (Gemini / Claude). Multi-camera fan-out: the agent can call `grab_frame` N times in one turn to cover multiple cameras. - `memory_search`, `memory_forget`, `schedule_task`, `create_proposal`, `read_file`, `write_file`, `run_command`, `publish_to_topic` Note: `telemetry_get`, `telemetry_history`, and `telemetry_config_update` are available via both the MCP server and the agent dispatch. diff --git a/docs/concepts/architecture.md b/docs/concepts/architecture.md index 3c9f1e97..34ca55d2 100644 --- a/docs/concepts/architecture.md +++ b/docs/concepts/architecture.md @@ -34,19 +34,25 @@ One binary. Four subsystems. One data plane. --- -## Three Entry Points +## Entry Points -All three share the same daemon-side agent runtime and MCP tools. +All entry points share the same daemon-side agent runtime and MCP tools. **CLI** (`bubbaloop agent chat`) - Thin Zenoh client. No LLM on the CLI side. - Publishes to the agent's inbox topic, subscribes to its outbox. +- Ctrl-C during a turn publishes to the cancel topic and aborts the in-flight LLM call. - All LLM processing happens inside the daemon. +**Dashboard Chat tab** (browser → daemon HTTP+SSE) +- Same agent runtime, different transport: the browser talks to the daemon's `/api/agent/...` endpoints over HTTP, and streams events back via SSE. +- Renders tool-call traces inline; surfaces images returned by `grab_frame` as inline chat entries (Gemini Vision). +- Multi-provider login status (Gemini / Claude / Ollama) visible from the tab. + **MCP stdio** (`bubbaloop mcp --stdio`) - For Claude Code and local AI agents. - No auth required — inherits user permissions (Admin tier). -- Launch: add `bubbaloop mcp --stdio` to your Claude Code config. +- Launch: add `bubbaloop mcp --stdio` to your Claude Code config. Use `bubbaloop mcp --token` to print the bearer token if you need it for `.mcp.json`. **MCP HTTP** (daemon auto-starts on `:8088`) - For remote agents, dashboards, and external integrations. @@ -73,6 +79,7 @@ CLI client Daemon (agent runtime) - Shared inbox topic: `bubbaloop/global/{machine_id}/agent/inbox` - Per-agent outbox: `bubbaloop/global/{machine_id}/agent/{agent_id}/outbox` +- Per-agent cancel topic: `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel` (clients publish here to abort an in-flight turn) - Wire format: JSON (`AgentMessage`, `AgentEvent`) **Per-agent state** diff --git a/docs/concepts/memory.md b/docs/concepts/memory.md index 4bc5804c..f80ab817 100644 --- a/docs/concepts/memory.md +++ b/docs/concepts/memory.md @@ -1,5 +1,5 @@ --- -description: "Bubbaloop 3-tier agent memory system. Short-term RAM, episodic NDJSON with FTS5 search, and long-term SQLite semantic memory for persistent context." +description: "Bubbaloop 4-tier agent memory system. Live world state (SQLite), short-term RAM, episodic NDJSON with FTS5 search, and long-term SQLite semantic memory for persistent context." --- # Memory & Mission Engine diff --git a/docs/concepts/messaging.md b/docs/concepts/messaging.md index 6195fd1c..7e4872a9 100644 --- a/docs/concepts/messaging.md +++ b/docs/concepts/messaging.md @@ -216,8 +216,11 @@ The daemon hosts a multi-agent runtime. All LLM processing is daemon-side. The C bubbaloop/global/{machine}/agent/inbox <- shared inbox (all agents) bubbaloop/global/{machine}/agent/{agent_id}/outbox <- per-agent event stream bubbaloop/global/{machine}/agent/{agent_id}/manifest <- queryable: agent metadata +bubbaloop/global/{machine}/agent/{agent_id}/cancel <- client cancels in-flight turn ``` +The dashboard's Chat tab speaks HTTP+SSE to the daemon's agent gateway instead of Zenoh, but emits the same `AgentMessage` / `AgentEvent` shapes and shares the same cancel semantics. + ### Wire Format Messages are JSON. diff --git a/docs/concepts/topics.md b/docs/concepts/topics.md index e4f5b5c1..2e45f530 100644 --- a/docs/concepts/topics.md +++ b/docs/concepts/topics.md @@ -69,6 +69,9 @@ The agent runtime uses dedicated topics for multi-agent messaging: | `bubbaloop/global/{machine}/agent/inbox` | CLI → Daemon | Shared intake for all agent messages | | `bubbaloop/global/{machine}/agent/{agent_id}/outbox` | Daemon → CLI | Per-agent streamed responses | | `bubbaloop/global/{machine}/agent/{agent_id}/manifest` | Queryable | Agent capabilities and model info | +| `bubbaloop/global/{machine}/agent/{agent_id}/cancel` | CLI → Daemon | Abort the in-flight turn for this agent (Ctrl-C in REPL) | + +The dashboard's Chat tab uses HTTP+SSE to the daemon's agent gateway instead of Zenoh. Both transports surface the same `AgentEvent` stream and share the same cancel semantics. ## Topic Discovery diff --git a/docs/concepts/wire-format.md b/docs/concepts/wire-format.md index 73de2036..999699ce 100644 --- a/docs/concepts/wire-format.md +++ b/docs/concepts/wire-format.md @@ -4,7 +4,9 @@ description: "Canonical CBOR envelope used by Bubbaloop nodes on the data plane. # Wire Format -Every payload published on the data plane is a **CBOR-encoded envelope**. The envelope splits transport metadata (delivery, ordering, source) from schema metadata (sensor timestamps, frame IDs, codec). This separation is what lets the same MCAP recorder, dashboard subscriber, or replayer node consume frames from any sensor without baking in per-sensor decoders. +Every **streaming** payload published on the data plane is a **CBOR-encoded envelope**. The envelope splits transport metadata (delivery, ordering, source) from schema metadata (sensor timestamps, frame IDs, codec). This separation is what lets the same MCAP recorder, dashboard subscriber, or replayer node consume frames from any sensor without baking in per-sensor decoders. + +> **Queryable exception**: the `grab_frame` queryable reply on `{camera}/grab_frame` is **not** a CBOR envelope. The reply payload is raw JPEG bytes and the Zenoh `attachment` carries JSON metadata (camera name, capture timestamp, dimensions). This keeps the agent's vision path zero-copy from the camera publisher all the way to the LLM, without the envelope overhead. Replayers and MCAP recorders should detect the encoding via Zenoh's `Encoding` field rather than assuming CBOR everywhere. For the topic-key layout (`bubbaloop/{global,local}/{machine}/{instance}/{suffix}`) and Zenoh routing, see [Messaging → Topic Convention](messaging.md#topic-convention). This document focuses on what's *inside* a payload at any of those keys. diff --git a/docs/dashboard/chat.md b/docs/dashboard/chat.md new file mode 100644 index 00000000..d19dc523 --- /dev/null +++ b/docs/dashboard/chat.md @@ -0,0 +1,43 @@ +--- +description: "Talk to your hardware from the browser. Multi-agent chat via HTTP+SSE with tool-call traces, image-in-chat (Gemini Vision via grab_frame), and Ctrl-C-style turn cancellation." +--- + +# Chat Tab + +The dashboard ships a Chat tab that talks to the same multi-agent runtime as `bubbaloop agent chat`, but over **HTTP+SSE** instead of Zenoh. The browser is a thin client — every LLM call runs daemon-side. No SDK or API key lives in the browser. + +``` +Browser Daemon (agent runtime) LLM provider + Chat tab ── HTTP ──> /api/agent/* ── SSE ── AgentEvent stream + │ │ + │ ├── agent loop (per-agent tokio task) + │ ├── tool dispatch (47 MCP + 43 internal tools) + │ └── ContentBlock::Image for vision turns + │ + └── POST /cancel ─> abort the in-flight turn (same semantics as Ctrl-C) +``` + +## What you can do + +- **Pick an agent.** The picker is populated from the daemon's per-agent `manifest` queryables. Unknown agent IDs surface a 404. +- **Pick a provider.** The status indicator shows which providers are configured. Gemini is the recommended default (free tier, strong vision); Claude works once `bubbaloop login` is done; Ollama works if `ollama serve` is running locally. If you choose a Claude OAuth subscription, the UI shows a risk warning — see the `bubbaloop login --status` docs. +- **Use tools.** Tool calls show as collapsible chips inline; the result follows. Tool-call traces help you debug what the agent actually did vs. what it said it did. +- **See images.** When the agent calls `grab_frame`, the daemon decodes the JPEG payload + JSON metadata attachment from the camera node and emits a `ContentBlock::Image` for the next LLM turn. The Chat tab renders the image inline so you can compare what the model saw vs. what it said. +- **Cancel a turn.** Press the cancel button (or just close the tab) — the dashboard posts to the per-agent cancel endpoint, which publishes to `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel`. The daemon aborts the in-flight LLM call. + +## Transport + +- HTTP for request/control, SSE for the response stream. Connects to the daemon's MCP HTTP port (`8088` by default — override with `BUBBALOOP_MCP_PORT`). +- Bearer token auth: the token is read from `~/.bubbaloop/mcp-token` on the daemon side and is picked up automatically when the dashboard is served from the same host. +- The wire shapes mirror `AgentMessage` / `AgentEvent` from the Zenoh gateway. See [Concepts → Messaging](../concepts/messaging.md#agent-gateway-protocol). + +## When to use Chat vs. `bubbaloop agent chat` + +- **Chat tab** — best for visual workflows, multi-camera vision, remote (LAN) operators, anyone who already has the dashboard open. +- **`bubbaloop agent chat`** — best for scripting (pipe-friendly stdout), SSH sessions, and quick one-liners. Same agent, same memory, same tools. + +## Related + +- [`grab_frame` tool](../agent-guide.md#quick-reference) — the agent-side primitive that produces the image-in-chat entries +- [Agent Guide → Turn Cancellation](../agent-guide.md#interact-via-cli) — the cancel semantics in detail +- [Remote Access](remote-access.md) — exposing the dashboard (and Chat tab) over LAN with HTTPS diff --git a/docs/dashboard/index.md b/docs/dashboard/index.md index 02688de8..e2a31395 100644 --- a/docs/dashboard/index.md +++ b/docs/dashboard/index.md @@ -39,6 +39,7 @@ flowchart LR | Topic discovery | Auto-discover available topics | | WebCodecs decoding | Hardware-accelerated H264 decoding | | HTTPS support | Secure remote access | +| Chat tab | Multi-agent chat via HTTP+SSE — tool-call traces, image-in-chat (e.g. `grab_frame` → Gemini Vision), in-flight turn cancellation. See [Chat tab](chat.md). | | Responsive design | Works on desktop and mobile | ## Quick Start diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index d2b51406..8e4cb19b 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -174,8 +174,9 @@ Access at: http://localhost:5173 ### Dashboard Features -| Panel | Description | -|-------|-------------| +| Tab / Panel | Description | +|-------------|-------------| +| Chat | Talk to agents in the browser via HTTP+SSE — tool-call traces, image-in-chat (`grab_frame` → Gemini Vision), turn cancellation. See [Chat tab](../dashboard/chat.md). | | Cameras | Live H264 video streams | | Nodes | Service management | | Weather | Current conditions and forecasts | diff --git a/docs/reference/cli.md b/docs/reference/cli.md index b325eebe..3d18088f 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -19,9 +19,10 @@ The `bubbaloop` binary is a single ~13 MB Rust executable that includes CLI, dae | `bubbaloop status` | Show service and node status | | `bubbaloop doctor` | Run system diagnostics | | `bubbaloop daemon` | Run the daemon (node manager) | -| `bubbaloop login` | Authenticate with Anthropic Claude (API key or OAuth). Gemini uses `GEMINI_API_KEY` env var. | -| `bubbaloop logout` | Remove stored credentials | -| `bubbaloop login --status` | Check authentication status | +| `bubbaloop mcp` | MCP server. Flags: `--stdio` (local), `-p ` (HTTP), `--token` (print bearer token for `.mcp.json`) | +| `bubbaloop login` | Authenticate Claude (API key or OAuth). Gemini uses `GEMINI_API_KEY` env var; Ollama needs no credentials. | +| `bubbaloop logout` | Remove stored Claude credentials | +| `bubbaloop login --status` | Check status across all providers (Gemini / Claude / Ollama) | ### Agent Commands @@ -420,8 +421,12 @@ pixi run up # Start all services via process-compose | `BUBBALOOP_ZENOH_ENDPOINT` | Zenoh router endpoint | `tcp/127.0.0.1:7447` | | `BUBBALOOP_MACHINE_ID` | Machine identifier | hostname | | `BUBBALOOP_MCP_PORT` | MCP HTTP server port | `8088` | +| `BUBBALOOP_MCP_TOKEN` | MCP HTTP bearer token (consumed by `.mcp.json`); read with `bubbaloop mcp --token` | — | +| `BUBBALOOP_MCP_MODE` | MCP transport mode (`stdio` / `http`) | — | +| `BUBBALOOP_MCP_TOOLSETS` | Comma-separated list of MCP toolsets to expose (default: all 11) | — | | `GEMINI_API_KEY` | Gemini API key (recommended — free tier) | — | | `ANTHROPIC_API_KEY` | Anthropic API key for Claude agents | — | +| `OLLAMA_HOST` | Ollama server base URL | `http://localhost:11434` | | `RUST_LOG` | Log level | `info` | **Examples:** diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index ebd56ac9..c4d15cae 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -1061,7 +1061,10 @@ systemctl --user restart bubbaloop-daemon bubbaloop login --status ``` -**Fix:** Authenticate with `bubbaloop login`. Choose API key or Claude subscription. +**Fix:** Pick a provider: +- **Gemini** (recommended, free tier): `export GEMINI_API_KEY=...` or write the key to `~/.bubbaloop/gemini-key` +- **Claude**: `bubbaloop login` and choose API key or subscription +- **Ollama**: run `ollama serve` (no credentials needed) ### Agent stuck / not finishing turn @@ -1070,7 +1073,7 @@ Turn timeout is 120s, tool call timeout is 30s. Check daemon logs: journalctl --user -u bubbaloop-daemon | grep -i timeout ``` -May indicate API rate limiting (HTTP 429). Wait and retry, or check your Anthropic usage dashboard. +May indicate API rate limiting (HTTP 429). Wait and retry, or check the provider quota dashboard — [Google AI Studio](https://aistudio.google.com/) for Gemini, [Anthropic Console](https://console.anthropic.com/) for Claude. ### Agent lost context