Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ If it's app-layer complexity → reject it. If it strengthens sensor drivers →
│ │ Episodic (NDJSON/FTS5) | Semantic (SQLite) │ │
│ └──────────────────────┬─────────────────────────────┘ │
│ ┌──────────────────────┴─────────────────────────────┐ │
│ │ MCP Server (42 tools) — sole control interface │ │
│ │ MCP Server (47 tools) — sole control interface │ │
│ │ RBAC (Viewer/Operator/Admin) | Bearer token auth │ │
│ │ PlatformOperations trait | Rate limiting │ │
│ └──────────────────────┬─────────────────────────────┘ │
Expand Down Expand Up @@ -332,7 +332,7 @@ BUBBALOOP_ZENOH_ENDPOINT=tcp/127.0.0.1:7447 # Optional override

## MCP Server

MCP is the **sole control interface**. 42 MCP tools + agent-internal tools across categories:
MCP is the **sole control interface**. 47 MCP tools + ~43 agent-internal tools across categories:

| Category | Tools |
|----------|-------|
Expand All @@ -347,7 +347,7 @@ MCP is the **sole control interface**. 42 MCP tools + agent-internal tools acros
| **Missions** | list_missions, pause_mission, resume_mission, cancel_mission — YAML-file-driven (`~/.bubbaloop/agents/{id}/missions/`) |
| **Constraints** | register_constraint, list_constraints — per-mission safety limits; params_json formats: `workspace={"x":[-1,1],"y":[-1,1],"z":[0,2]}`, `max_velocity=1.5`, `forbidden_zone={"center":[0,0,0],"radius":0.3}`, `max_force=50.0` |
| **Alerts** | register_alert, unregister_alert — reactive arousal triggers when world state predicate matches |
| **Agent-internal** | read_file, write_file, run_command, memory_search, memory_forget, schedule_task, create_proposal, publish_to_topic, get_system_telemetry, get_telemetry_history, update_telemetry_config |
| **Agent-internal** | grab_frame (Zenoh-query camera node for one JPEG → fed to vision-capable provider), read_file, write_file, run_command, memory_search, memory_forget, schedule_task, create_proposal, publish_to_topic, get_system_telemetry, get_telemetry_history, update_telemetry_config |

### Transport Options

Expand Down Expand Up @@ -435,12 +435,12 @@ MCP is the **sole control interface**. 42 MCP tools + agent-internal tools acros
| Runtime | Rust + Tokio | Memory safety, small binary, edge-ready |
| Data plane | Zenoh | Zero-copy pub/sub, decentralized, Rust-native |
| Schemas | Protobuf + prost | Self-describing, runtime introspection |
| Control | MCP (rmcp) | Standard AI agent interface, 42 MCP tools + agent-internal tools |
| Control | MCP (rmcp) | Standard AI agent interface, 47 MCP tools + ~43 agent-internal tools |
| Memory | SQLite (rusqlite) + NDJSON | 4-tier: world state (live SQLite) + RAM + episodic (NDJSON/FTS5) + semantic (SQLite). World state updated by context providers, not LLM. |
| CLI | argh | Minimal, fast compile |
| Logging | log + env_logger | Simple, stderr-only |
| systemd | zbus (D-Bus) | No subprocess spawning, safe |
| LLM | ModelProvider trait (reqwest) | Claude (OAuth + API key) and Ollama (local, tool calling) |
| LLM | ModelProvider trait (reqwest) | Gemini (recommended — free tier, vision), Claude (OAuth + API key, vision), Ollama (local, tool calling, text-only) |
| HTTP | axum | Dashboard + MCP HTTP transport |
| Telemetry | sysinfo | CPU/RAM/disk monitoring, cross-platform |

Expand Down
82 changes: 82 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,87 @@
# Changelog

## [0.0.14](https://github.com/kornia/bubbaloop/compare/v0.0.13...v0.0.14) (2026-05-19)

### Features

* **agent:** `grab_frame` tool — agent fetches a single JPEG frame from a camera node and feeds it to a vision-capable provider for analysis. Multi-camera fan-out supported. ([ff7fe08](https://github.com/kornia/bubbaloop/commit/ff7fe08), [85dff9c](https://github.com/kornia/bubbaloop/commit/85dff9c))
* **agent:** Gemini Vision provider (`crates/bubbaloop/src/agent/provider/gemini.rs`) — recommended default for new users (free tier, strong vision support). ([ff7fe08](https://github.com/kornia/bubbaloop/commit/ff7fe08))
* **agent:** client-side turn cancellation via Zenoh `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel` topic; Ctrl-C in the REPL aborts the in-flight turn. ([2ea9176](https://github.com/kornia/bubbaloop/commit/2ea9176), [ac384dc](https://github.com/kornia/bubbaloop/commit/ac384dc))
* **dashboard:** Chat tab — talk to agents from the browser via HTTP+SSE to the daemon's agent gateway. Includes tool-call trace rendering and image-in-chat from `grab_frame`. ([4140a1a](https://github.com/kornia/bubbaloop/commit/4140a1a))
* **chat:** multi-provider login status (Gemini / Claude / Ollama), 404 on unknown agent, Claude OAuth risk warning. ([fe76d67](https://github.com/kornia/bubbaloop/commit/fe76d67))
* **mcp:** `bubbaloop mcp --token` prints the stored bearer token so `.mcp.json` can pick it up without a race. ([4dadb64](https://github.com/kornia/bubbaloop/commit/4dadb64))

### Bug Fixes

* **agent:** `grab_frame` reads binary JPEG payload + JSON metadata attachment from camera nodes (was: base64-in-JSON). ([326a9dd](https://github.com/kornia/bubbaloop/commit/326a9dd))
* **chat:** clear Responding state after multi-camera `grab_frame` completes. ([85dff9c](https://github.com/kornia/bubbaloop/commit/85dff9c))
* **mcp:** eliminate token race between gateway and agent runtime. ([d03efba](https://github.com/kornia/bubbaloop/commit/d03efba))

### Chores

* **dashboard:** remove LibraryView component (replaced by Chat tab). ([5d687be](https://github.com/kornia/bubbaloop/commit/5d687be))

### Docs

* Prioritize Gemini as the recommended default provider across README, agent-guide, quickstart, configuration, and CLI reference (#100).
* Sync CHANGELOG, CLAUDE.md, ARCHITECTURE.md, ROADMAP.md, concept docs with v0.0.14 features.

## [0.0.13](https://github.com/kornia/bubbaloop/compare/v0.0.12...v0.0.13) (2026-05-04)

### Features

* **mcp:** `install_node` accepts name + config flags (#87)

Released as `chore(release): v0.0.13` (#88) — install-flow polish from the e2e exercise.

## [0.0.12](https://github.com/kornia/bubbaloop/compare/v0.0.11...v0.0.12) (2026-05-04)

### Features

* **agent:** recorder pipeline overhaul — MCP plugin + production refinements + install + SDK testing (#85)
* **agent:** Phase 3 — reactive alerts end-to-end (rule-based world-state monitoring with debounce, circuit breaker, world-state sweeper, hard reqwest timeout for stuck LLM generations)
* **sdk:** `publisher_cbor(local=True)` for CBOR+SHM in the Python SDK (#78)

### Refactors

* **wire-format:** remove protobuf in favor of CBOR + manifest schemas across the dataplane (#76)

## [0.0.11](https://github.com/kornia/bubbaloop/compare/v0.0.10...v0.0.11) (2026-04-06)

### Features

* **agent:** Physical AI Memory & Mission system — scheduled tasks for recurring actions, missions vs tasks distinction
* **tui:** real-time agent context visibility with System event type; show tool results by default (full detail with `-v`)

### Bug Fixes

* **agent:** prevent 400 'unexpected tool_use' errors
* **tui:** drain subscriber and improve tool-result contrast
* **build:** binary size reduction (`panic=abort`, `rusqlite` bundled, axum without `ws`)

## [0.0.10](https://github.com/kornia/bubbaloop/compare/v0.0.9...v0.0.10) (2026-03-30)

### Features

* **agent:** `bubbaloop agent setup` command with first-run onboarding (#51)

## [0.0.9](https://github.com/kornia/bubbaloop/compare/v0.0.8...v0.0.9) (2026-03-29)

### Features

* **agent:** OpenClaw-inspired agent rewrite with multi-agent runtime (#47)
* **agent:** minimal opt-in Ollama support for local inference (#49)

### Docs

* Premium theme redesign, factual fixes against Rust source, agent-first narrative

## [0.0.8](https://github.com/kornia/bubbaloop/compare/v0.0.7...v0.0.8) (2026-03-10)

### Features

* **cli:** Claude subscription login via `setup-token` (#36)

## [0.0.7](https://github.com/kornia/bubbaloop/compare/v0.0.6...v0.0.7) (2026-03-01)

### Features
Expand Down
7 changes: 5 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,15 @@ Key source files in `crates/bubbaloop/src/`:
- `agent/runtime.rs` — Multi-agent runtime: AgentsConfig, AgentRuntime, ZenohSink, agent_loop; `agent_directory()` (pub)
- `agent/prompt.rs` — System prompt builder; `build_system_prompt_with_soul_path()` injects onboarding prompt for new agents
- `agent/soul.rs` — Soul struct, first-run onboarding, notify hot-reload (`~/.bubbaloop/soul/`)
- `agent/provider/mod.rs` — ModelProvider trait, Message, ContentBlock, ToolDefinition, StreamEvent
- `agent/provider/mod.rs` — ModelProvider trait, Message, ContentBlock (incl. `Image`), ToolDefinition, StreamEvent
- `agent/provider/gemini.rs` — Gemini API client (recommended default — free tier, vision-capable)
- `agent/provider/claude.rs` — Claude API client with dual auth (API key + OAuth bearer token)
- `agent/provider/ollama.rs` — Ollama local LLM client with tool calling (`/api/chat`)
- `agent/camera.rs` — `grab_frame` implementation: Zenoh query for one JPEG frame + JSON metadata attachment, emits `ContentBlock::Image`
- `agent/gateway.rs` — Zenoh + HTTP+SSE agent gateway; per-agent cancel topic `bubbaloop/global/{machine_id}/agent/{agent_id}/cancel`
- `agent/memory/` — 4-tier: world state (live SQLite) + short-term (RAM) + episodic (NDJSON) + semantic (SQLite)
- `agent/heartbeat.rs` — Adaptive heartbeat: arousal + decay + state collection
- `agent/dispatch.rs` — Internal MCP tool dispatch (42 MCP + agent-internal tools, includes telemetry + beliefs + constraints + missions)
- `agent/dispatch.rs` — Internal tool dispatch (47 public MCP tools + ~43 agent-internal tools incl. `grab_frame`, telemetry, beliefs, constraints, missions)
- `cli/node/mod.rs` — node CRUD, validation, list/add/remove
- `cli/node/install.rs` — install, precompiled binary download, GitHub clone
- `cli/node/lifecycle.rs` — start, stop, restart, logs
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ If a workflow only improves the workflow itself, **delete it**. The sensor nodes

1. Check argh conventions in CLAUDE.md (`#[derive(FromArgs)]`, `#[argh(subcommand)]`)
2. Write command + unit tests (co-located `#[cfg(test)] mod tests`)
3. Verify 298+ Rust tests pass
3. Verify 2400+ Rust tests pass
4. Zero clippy warnings enforced

**Security**: Node names `[a-zA-Z0-9_-]{1,64}`, no null bytes.
Expand All @@ -52,7 +52,7 @@ If a workflow only improves the workflow itself, **delete it**. The sensor nodes
2. Update proto + rebuild BOTH descriptor pipelines (`bubbaloop-schemas` AND `bubbaloop`)
3. Update MCP tool handlers in `mcp/mod.rs`, add integration tests
4. Update dashboard types if applicable
5. Full system check (298+ Rust + 47 MCP integration tests)
5. Full system check (2400+ Rust unit tests + 47 MCP integration tests)

**Critical**: Proto changes require rebuilding both descriptor pipelines.

Expand Down Expand Up @@ -80,7 +80,7 @@ Start at the lowest tier. Escalate only on failure. Haiku costs ~10x less than O
| Command | When | Why |
|---------|------|-----|
| `pixi run check` | After every Rust change | Fast compilation check |
| `cargo test --lib -p bubbaloop` | Before commits | 298+ Rust tests |
| `cargo test --lib -p bubbaloop` | Before commits | 2400+ Rust tests |
| `cargo test --features test-harness --test integration_mcp` | After MCP changes | 47 integration tests |
| `pixi run clippy` | Before PRs | Zero warnings (`-D warnings`) |

Expand All @@ -107,7 +107,7 @@ See `CLAUDE.md` for full conventions. Critical rules:
## Pull Request Checklist

- [ ] `pixi run check` passes
- [ ] `cargo test --lib -p bubbaloop` (298+ tests)
- [ ] `cargo test --lib -p bubbaloop` (2400+ tests)
- [ ] `cargo test --features test-harness --test integration_mcp` (47 tests)
- [ ] `pixi run clippy` zero warnings
- [ ] `CLAUDE.md` updated if conventions changed
Expand Down
32 changes: 22 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ AI agents revolutionized software engineering. **Bubbaloop brings that same powe
| **Data plane** | None | **Zenoh (zero-copy pub/sub)** |
| **Hardware** | None | **Self-describing sensor nodes** |
| **Runs on** | Desktop / cloud | **Jetson, RPi, any Linux ARM64/x86** |
| **MCP role** | Client (consumes tools) | **Server (42+ tools, 3-tier RBAC)** |
| **MCP role** | Client (consumes tools) | **Server (47 tools, 3-tier RBAC)** |
| **Scheduling** | Always-on LLM (~$5-10/day) | **Offline Tier 1 + LLM Tier 2 (~$0.05/day)** |

## Quick Install
Expand All @@ -43,20 +43,29 @@ All run as systemd user services with autostart enabled.

## Login & Authentication

Bubbaloop supports three providers — **Gemini** (recommended for new users), **Claude**, and **Ollama** (local).

```bash
# Option 1: API Key (pay-as-you-go)
# Option 1 (Recommended): Gemini — free tier, strong vision support
export GEMINI_API_KEY=... # from https://aistudio.google.com/apikey
# or write the key to ~/.bubbaloop/gemini-key

# Option 2: Claude API Key (pay-as-you-go)
bubbaloop login
# → Choose [1], paste your key from console.anthropic.com

# Option 2: Claude Subscription (Pro/Max/Team)
# Option 3: Claude Subscription (Pro/Max/Team)
claude setup-token # Run in Claude Code CLI first
bubbaloop login
# → Choose [2], paste the sk-ant-oat01-* token

# Check auth status
# Option 4: Ollama — fully local, no cloud bill
ollama serve # in a separate terminal

# Check auth status (shows all configured providers)
bubbaloop login --status

# Remove credentials
# Remove Claude credentials (Gemini/Ollama have no stored secret)
bubbaloop logout
```

Expand All @@ -69,12 +78,15 @@ bubbaloop status
# Start daemon (runs agent runtime + MCP server + node manager)
bubbaloop up

# Talk to your hardware via Claude AI (agents run daemon-side)
# Talk to your hardware (Gemini by default; agents run daemon-side)
bubbaloop agent chat "What sensors do I have?"
bubbaloop agent chat # Interactive REPL
bubbaloop agent chat -a camera-expert "describe the video feed"
bubbaloop agent chat # Interactive REPL (Ctrl-C cancels the in-flight turn)
bubbaloop agent chat -a camera-expert "describe what the terrace camera sees"
bubbaloop agent list # Show running agents + models

# Or open the dashboard's Chat tab in a browser — same agent runtime,
# rich tool-call traces, and image-in-chat when the agent calls grab_frame.

# System diagnostics with auto-fix
bubbaloop doctor --fix

Expand Down Expand Up @@ -161,13 +173,13 @@ export BUBBALOOP_MCP_TOKEN=$(bubbaloop mcp --token)

47 MCP tools + agent-internal tools. Configure Claude Code via `.mcp.json` (already in project root).

**Agent-internal tools** (daemon-side only, not exposed via MCP): `memory_search`, `memory_forget`, `schedule_task`, `create_proposal`, `read_file`, `write_file`, `run_command`.
**Agent-internal tools** (daemon-side only, not exposed via MCP): `grab_frame` (Zenoh-query a camera node for one JPEG → fed to a vision-capable provider), `memory_search`, `memory_forget`, `schedule_task`, `create_proposal`, `read_file`, `write_file`, `run_command`, `publish_to_topic`.

## Architecture

```
┌──────────────────────────────────┐
│ AI Agent (Claude via MCP)
│ AI Agent (Gemini/Claude via MCP)│
│ http://127.0.0.1:8088/mcp │
└──────────────┬───────────────────┘
Expand Down
13 changes: 8 additions & 5 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ bubbaloop agent chat # interactive REPL
│ │ World State | Short-term | Episodic | Semantic │ │
│ └──────────────────────┬─────────────────────────────┘ │
│ ┌──────────────────────┴─────────────────────────────┐ │
│ │ MCP Server (42 tools) — sole control interface │ │
│ │ MCP Server (47 tools) — sole control interface │ │
│ └──────────────────────┬─────────────────────────────┘ │
│ ┌──────────────────────┴─────────────────────────────┐ │
│ │ Daemon (skill runtime + agent host) │ │
Expand Down Expand Up @@ -91,7 +91,7 @@ bubbaloop agent chat # interactive REPL
### v0.0.1–v0.0.6: MCP-Native Sensor Runtime

- [x] Single binary: CLI + daemon + MCP server
- [x] 42 MCP tools (discovery, lifecycle, data, config, system, memory, telemetry, beliefs, missions, constraints, alerts)
- [x] 47 MCP tools (discovery, lifecycle, data, config, system, memory, telemetry, beliefs, missions, constraints, alerts, toolsets)
- [x] MCP is sole control interface — Zenoh for data only
- [x] Marketplace with precompiled binaries (ARM64 + x86_64)
- [x] Full node lifecycle via MCP: install, uninstall, start, stop, restart, autostart
Expand Down Expand Up @@ -163,7 +163,7 @@ config:
**Deliverables:**
- [x] `bubbaloop agent chat` CLI command (thin Zenoh client, LLM runs daemon-side)
- [x] `bubbaloop agent list` — discover running agents via manifest queryables
- [x] `bubbaloop agent setup` — interactive wizard: provider (Claude/Ollama), model, identity
- [x] `bubbaloop agent setup` — interactive wizard: provider (Gemini/Claude/Ollama), model, identity
- [x] Multi-agent runtime: agents run as tokio tasks inside daemon
- [x] Agent gateway: Zenoh pub/sub wire format (inbox/outbox/manifest topics)
- [x] Per-agent config via `~/.bubbaloop/agents.toml` (provider, model override, capabilities)
Expand All @@ -174,8 +174,11 @@ config:
- [x] Internal MCP tool dispatch (call tools without HTTP round-trip)
- [x] System prompt injection: sensor inventory, node status, active schedules
- [x] First-run LLM onboarding: marker file triggers interview prompt; agent writes its own `identity.md`
- [x] HTTP+SSE chat endpoint — dashboard Chat tab talks to daemon's agent gateway (PR #91)
- [x] Gemini Vision provider — `ContentBlock::Image` + `grab_frame` agent tool (PR #96)
- [x] Client-side turn cancellation via Zenoh `agent/{id}/cancel` topic (PR #94/95)
- [x] Multi-provider login status (Gemini / Claude / Ollama) in `bubbaloop login --status` (PR #93)
- [ ] Capability-based message routing (currently falls back to default agent)
- [ ] HTTP chat endpoint for future dashboard integration

**New deps:** None (`reqwest` already in dep tree). **New code:** ~500-800 lines.

Expand Down Expand Up @@ -373,7 +376,7 @@ Memory confirmed or contradicted by sensor readings — not just conversation hi
| Runtime | Rust + Tokio | Memory safety, small binary, edge-ready |
| Data plane | Zenoh | Zero-copy pub/sub, decentralized, Rust-native |
| Schemas | Protobuf + prost | Self-describing, runtime introspection |
| Control | MCP (rmcp) | Standard AI agent interface, 42 MCP tools + agent-internal tools |
| Control | MCP (rmcp) | Standard AI agent interface, 47 MCP tools + ~43 agent-internal tools |
| Memory | SQLite (rusqlite) | Embedded, +1-2 MB, battle-tested everywhere |
| CLI | argh | Minimal, fast compile |
| Logging | log + env_logger | Simple, stderr-only |
Expand Down
Loading
Loading