Important
AI Assist Note (Knowledge Heritage): This document is part of the "Sovereign Reality" documentation.
- @docs ARCHITECTURE:Core
- Failure Path: Information drift, legacy terminology, or documentation mismatch.
- Telemetry Link: Search
[README]in audit logs.
Core technical resource for the Autonomous Agentic Tadpole OS runtime, dashboard, Rust engine, and Python execution layer.
Traceability via execution/parity_guard.py.
A local-first operating system for autonomous agent swarms, governed execution, model operations, and live mission telemetry.
Quick Start β’ Architecture β’ API β’ Configuration β’ Verification β’ π Wiki
β οΈ ***Warning: With great power comes great responsibility. SANDBOX USE ONLY*** β οΈ
Caution
Host Execution Notice: Autonomous Agentic Tadpole is designed for local developer workstations and isolated bunker lab environments. While WebAssembly skill isolation and Docker container options exist, local Python skills and verification scripts execute within host process boundaries under path-containment controls. Do not execute untrusted agent workflows without dedicated machine or container isolation.
Most agent frameworks stop at prompt orchestration. Autonomous Agentic Tadpole goes further: it combines a Rust control plane, a production-grade React operations console, SQLite-backed persistence, governance gates, skill execution, MCP tooling, model-provider management, and live telemetry into one local-first system.
The result is a desktop-ready agent operations platform that can run missions, manage agent state, inspect model providers, schedule continuity jobs, surface security posture, and keep a durable audit trail without depending on a hosted control plane.
| Dimension | Marketplace Frameworks (CrewAI, AutoGen, LangChain) | Autonomous Agentic Tadpole | Enterprise Impact |
|---|---|---|---|
| Core Kernel | Python / Node.js (Interpreted, GIL locks) | Compiled Rust (server-rs) + Axum + Tokio |
10x β 100x Throughput & Sub-ms Latency |
| Token Optimization | Naive string counts / uncached BPE | Model-Aware TokenizerService (< 1Β΅s DashMap LRU) |
Sub-Microsecond Zero-Allocation Token Tracking |
| Context Compression | Truncation or basic sliding window | 3-Zone Adaptive Context Slicer + 2-Tier Compressor | 30% β 50% Token Cost Reduction with Strict Budget Enforcement |
| RAG Architecture | Single Naive Vector DB (Chroma/Pinecone) | Hybrid RAG Triad RRF (Vector + TrustGraph + BM25) | Sub-ms Exact Code Symbol & Multi-Hop RAG with Fusion Scoring |
| Workflow Durability | Stateless re-execution on crash | SQLite-Native Durable Step Memoization (SHA-256) | Zero Token Waste on Engine Restart |
| Task Orchestration | Linear chains or flat fan-out | Dynamic DAG Engine (petgraph) + Shared Blackboard |
Parallel Task Execution with Deadlock-Free Failure Cascading |
| Actor Supervision | No supervision / manual restart | OTP-Style Supervision Tree (OneForOne / OneForAll) | Erlang-Grade Fault Isolation & Auto-Recovery |
| Model Routing | Single model per request | Tiered Cascade Router (Flash β Reasoning Escalation) | 2xβ5x Cost Reduction on Routine Turns |
| Mutation Safety | Trust-the-agent / no verification | Aletheia Zero-Trust Verification Gate | Independent Verifier Blast-Radius Audit |
| Financial Safety | Loose per-call limits / unconstrained loops | A2E-01 2PC Ledger + 24h Rolling Cap + Lock Awareness | Zero Risk of Runaway API Overruns |
| Self-Healing | Uncaught exceptions / endless loops | tool_loop_guard.py + Boot DB Reconciler + Annealing |
Deterministic Circuit Breakers |
-
Runs a Rust agent engine with Axum, Tokio, background actors, lifecycle workers, and protected
/v1APIs. - Ships a real operations dashboard built with React 19, TypeScript, Vite, Tailwind CSS, Zustand, and React Router.
- Persists operational state locally with SQLite migrations through SQLx.
-
Executes skills and tools through Python scripts, JSON manifests, modular
execution/core/skills, and MCP endpoints. - Manages providers and models across local and cloud backends, with Ollama support and provider-key configuration.
- Streams telemetry live through WebSocket routes, heartbeat events, system logs, and dashboard health panels.
- Enforces governance and security controls through bearer-token auth, request middleware, shell scanning, audit trails, quotas, oversight queues, and privacy-mode behavior.
- Visualizes codebase interdependencies with an interactive 2D Force-Directed Knowledge Graph and down-stream blast-radius impact analysis.
-
OTP Actor Supervision Tree: Erlang/OTP-style supervisor with
OneForOneandOneForAllrestart strategies,AbortHandledeterministic shutdown, and stability-based exponential backoff. -
Hybrid RAG Triad Fusion: Reciprocal Rank Fusion (RRF) combining LanceDB Vector, BM25 Lexical, and TrustGraph Entity search via
GET /v1/memory/search/hybrid. - Durable Workflow Step Memoization: SQLite-native crash-resilient workflow engine with SHA-256 parameter hashing; completed steps fast-forward on restart with zero token waste.
-
Dynamic DAG Task Engine: Directed task dependency graph (
petgraph::StableDiGraph) with topological cycle rejection, parallel ready-queue extraction, and deadlock-free failure cascading via BFSSkippedpropagation. -
Shared Mission Blackboard: High-performance
DashMap+Arc<BlackboardEntry>in-memory scratchpad for multi-agent data exchange, replacing prompt string passing with O(1) key pointers. - Tiered Model Cascade Router: Dynamic turn routing between Tier 1 Fast (Ollama/Groq/Gemini Flash) and Tier 2 Frontier Reasoning (Gemini Pro/Claude/GPT-4o) with capability-aware error escalation.
-
Aletheia Verification Gate: Zero-trust dual-agent verification for high-impact mutations with independent Verifier blast-radius calculation and O(1)
HashSetsensitive skill gating. -
Adaptive Context Slicer: Cognitive 3-zone prompt assembly (Pinned Anchors,
<grounded_context>RAG, Sliding Active Window) with stricttiktokenBPE token budget enforcement. - Institutional Knowledge Store (IKS): Provides cross-cluster, persistent semantic memory backed by local SQLite databases and optional LanceDB vectors, complete with TTL evictions and time-aware confidence decay.
- Unified Graph Analysis: Implements off-thread, lock-protected codebase dependency tracking with O(1) reverse-obfuscated path lookup for secure blast-radius calculations.
- Modular Sovereign UI Layouts: Refactored the monolithic chat console into highly cohesive frontend sub-components and specialized React hooks.
- Automated Self-Healing State Reversion: Integrates benchmark telemetry with sovereign time-travel state rollbacks, autonomously reverting missions to the last known green state on regression.
- Hardened Sandboxed Execution: Spawns dynamic agent-generated skills inside secure WebAssembly (Wasmtime) or Docker micro-containers with strict CPU/memory limits and filesystem token isolation.
- Tiered Cognitive Memory consolidation: Consolidates episodic short-term memories into semantic long-term knowledge via a background summarization loop, with a fully database-backed SQLite fallback when vector features are disabled.
- Inter-Agent Communication Protocol (IACP): Enables agents to negotiate tasks, delegate sub-tasks, peer-review skills, and transfer operational micro-budgets.
- Autonomous Browser Sentinel Daemon: Background idle loop (~25s interval) executing non-intrusive DOM entropy audits, automated client-side health checks, and self-healing with triple circuit breakers (tab visibility, VRAM critical pressure, and pipeline concurrency locks).
-
In-Browser ONNX/WebGPU Inference & DLP Shield: Zero-cloud in-browser reasoning with quantized 4-bit ONNX models (
SmolLM-360M-Instruct) via@huggingface/transformers, WebGPU hardware acceleration with WASM fallback, explicit WebGPU buffer disposal, and zero-leakage regex DLP pre-flight screening for high-entropy secrets and keys. -
Autonomous-Agentic Cognition Spine: Enforces hard pre-turn budget cap checks prior to LLM invocation, single-pass grounded refinement with structured tool diagnostic feedback (
retryable,suggested_action), and balanced-brace function call recovery for open-source models (<function=...>, hallucinated brackets/parentheses). -
Pinned-Tail Context Compactor & Tool Overflow: Permanently pins the system prompt and the last 3 conversational turns uncompressed while offloading large tool payloads (>2,000 characters) to
.tmp/tool_overflow/<call_id>.txtwith a disk pointer URI, preserving tool failure visibility verbatim. -
Lock-Free Atomic Execution Metrics: Real-time per-mission accounting (turns, token throughput, USD cost, tool failures) via lock-free
AtomicU64counters, warning operators if summarizations loop ($\ge 3$ ). -
Concurrent Edit Conflict Manager & Dependency Guard: 30-second TTL file write lease locks preventing multi-agent race conditions during concurrent file writes, paired with thread-safe cached cross-platform binary discovery (
git,python,node,docker,cargo). -
RFC 9457 Problem Details & Auto-Pruning Telemetry: Standardized error responses (
application/problem+json) with W3Ctraceparentpropagation, alongside a 7-day auto-rotating JSONL telemetry file sink. -
Clean-Room WASM Pulse Codec & Multi-Agent Starter Kits: High-efficiency MIT-licensed Postcard pulse serialization (
crates/wasm-codec) and 4 plug-and-play domain starter kits (starter_kits/) covering Customer Success, Explorer Scout, Finance Compliance, and Marketing Growth. -
Token-Budgeted AST Symbol Context Injection: Native agent tool (
get_symbol_context) providing syntax-tree context extraction directly within prompt token limits, eliminating blind edits without blowing token windows. -
Selective Blast-Radius Test Targeting: Intelligent test suite selection via
GET /v1/intelligence/impacted-testsandget_impacted_tests, mapping code changes directly to dependent unit/integration tests for sub-second feedback loops. -
Native Multi-Role ChatMessage Protocol & Ephemeral Caching: Decouples multi-turn conversation from fragile flat strings using a typed
ChatMessage(System,User,Assistant,Tool) pipeline. Structures Anthropic and Gemini system prompts with ephemeral prompt caching (cache_control: {"type": "ephemeral"}), slashing turn input costs by up to 90% and accelerating reasoning. -
Context Window Token Guard & Overflow Containment: Evaluates total session tokens via model-aware
TokenizerService. EmitsAppError::ContextOverflow(HTTP 413, classified as Repairable/Turn) before dispatch, triggering dynamic context compaction without dropping active mission state. -
Tool Repetition Loop Guard & Financial Cap Fallbacks: Tracks consecutive identical
(tool_name, args_hash)calls, issuing actionable warnings at 3 attempts and deterministically aborting viaRunnerError::RecursionBlockedat 5. Bounded by a reliable$10.00pre-turn budget cap fallback (DEFAULT_BUDGET_LIMIT_USD). -
Deterministic Artifact Verification: Enforces physical disk existence of all declared files during
complete_missionbefore acknowledging completion, preventing premature completion hallucinations. -
20-Task Ground Truth Evaluation Suite: Automated benchmark suite (
evals/tasks.jsonandevals/run.py/npm run test:evals) asserting 20 architectural invariants with 100% ground-truth pass rate. -
Swarm Delegation Cycle & Deadlock Prevention: Enforces topological BFS acyclicity checks during directive persistence (
save_directive), rejecting circular agent delegation graphs at the API boundary. -
Actionable Swarm Topology & Quick Command Bar: Live active mission graph synthesis with dangling edge filtering and direct directive dispatch directly from the interactive
Swarm_Visualizercanvas. -
Shared Tool Cache with Monotonic MTime Invalidation: In-memory read-only cache with strictly bounded allowlist (
read_file,list_files, etc.), size/mtime freshness checks, and instant mutation invalidation on disk writes or shell execution. - DoomLoopDetector with Period-4 Cyclic Cycle Interception: JSON-normalized SHA-256 tool call hashing detecting multi-step alternating oscillation loops (periods 1 through 4) across agent turns, preventing runaway execution traps.
- Two-Stage Monologue Compaction: Active context compaction trimming embedded tool execution logs (>2,000 chars) and dense LLM-driven summarization for turns outside the hot tail.
- SSRFGuard with Direct IP Resolution (DNS Rebinding Defense): Validates public HTTP/HTTPS URLs against comprehensive RFC private, loopback, CGNAT, link-local, and IPv6 unique-local blocks, directly pinning the resolved IP in HTTP transport to defeat Time-Of-Check-Time-Of-Use (TOCTOU) DNS rebinding attacks.
- SpanWatchdog with Provider-Aware Leaked Span Reaping: Background telemetry watchdog sweeping open spans with provider-specific TTLs (60s cloud, 300s local Ollama), emitting synthetic timeout events to prevent orphan spans from corrupting mission observability.
-
Mission Benchmark Harness: 5-scenario reproducible benchmark test harness (
tests/mission_bench/run.pyandtests/mission_bench/judge.py) asserting tool caching, loop prevention, monologue compression, SSRF security, and span watchdog timeout tracking. - System Event Bus: Combines Tokio channels with system triggers (file modifications, system load, webhooks) to spawn Continuity Jobs proactively.
- Git Worktree Sandboxing: Lightweight VCS-level disk isolation (server-rs/src/adapter/worktree.rs) spawning temporary worktrees in .tmp/worktrees/<task_id>, ensuring collision-free concurrent agent branch editing, instantaneous rollbacks, and atomic diff-based review before merge.
- Actor Idle Reaping & Hibernation: Automated periodic sweep (server-rs/src/agent/reaper.rs) transitioning inactive swarm agents (>300s) from idle to suspended/hibernating state, reclaiming Tokio scheduler slots and in-memory caches.
- Decoupled Background Context Compaction: Dispatches Tier 1 & Tier 2 context compaction onto background worker tasks (server-rs/src/agent/context_manager.rs), eliminating turn latency spikes during operator sessions.
- Keeps optional heavy features explicit: vector memory and neural audio are Cargo features, disabled by default for lightweight local builds.
flowchart LR
Dashboard["React + TypeScript Dashboard"] --> API["/v1 REST + WebSocket API"]
API --> Engine["Rust Axum Engine"]
Engine --> State["AppState Hubs"]
State --> DB["SQLite + SQLx Migrations + Durable Steps"]
State --> Actors["Audit, Memory, Security, Skill Actors"]
Engine --> Supervisor["OTP Supervisor Tree"]
Supervisor --> Actors
Engine --> Orchestration["DAG + Blackboard + Cascade Router"]
Orchestration --> Gate["Aletheia Verification Gate"]
Orchestration --> Slicer["Adaptive Context Slicer"]
Engine --> RAG["Hybrid RAG Triad RRF"]
RAG --> VectorDB["LanceDB Vector"]
RAG --> BM25["BM25 Lexical"]
RAG --> TrustGraph["TrustGraph Entity"]
Engine --> Workers["Telemetry, Scheduler, Discovery, Reaper"]
Engine --> MCP["MCP Host + Skill Registry"]
MCP --> Execution["Python Scripts + JSON Skill Manifests"]
Dashboard --> BrowserRuntime["Browser Inference + VRAM Monitor"]
Prerequisites:
- Node.js and npm compatible with the checked-in
package-lock.json. - Rust toolchain for
server-rs. - Python 3 for scripts in
execution/. - Optional: Ollama or cloud provider API keys for model execution.
Install dependencies:
npm installCreate local environment config:
cp .env.example .envSet engine security and execution parameters in .env:
NEURAL_TOKEN=your-secret-token
# Skill Execution Sandboxing (Default-deny bare host execution)
USE_SANDBOX_DOCKER=true # Recommended: containerized execution
# USE_SANDBOX_WASM=true # Alternative: Wasmtime sandbox
# ALLOW_HOST_SKILL_EXECUTION=true # Explicitly permit unsandboxed host fallbackStart the Rust engine:
npm run engineStart the dashboard in a second terminal:
npm run devOpen:
- Dashboard:
http://localhost:5173 - Engine health:
http://127.0.0.1:8000/v1/engine/health
On Windows, these helpers wrap the same flow:
start_AA_tadpole.batstart_backend.batstart_frontend.batstop_AA_tadpole.bat
To run the containerized stack (Engine + Prometheus + Grafana + Jaeger):
docker compose up --build -dAutonomous Agentic Tadpole has six practical runtime layers.
| Layer | Code | Responsibility |
|---|---|---|
| Interface | src/ |
Dashboard shell, pages, stores, services, Browser Sentinel & DLP Shield, browser monitoring, provider sync, detached views |
| Engine | server-rs/src/ |
Axum routes, AppState, actors, middleware, telemetry, agent runner, security, startup workers |
| Token & Context | server-rs/src/agent/tokenizer.rs, server-rs/src/agent/context_manager.rs, server-rs/src/agent/context_slicer.rs |
Model-aware BPE counting (< 1Β΅s), 2-Tier compression, adaptive 3-zone context slicing |
| Supervision & Durability | server-rs/src/system/actors/supervisor.rs, server-rs/src/agent/durable.rs |
OTP supervision tree, crash-resilient step memoization |
| Swarm Orchestration | server-rs/src/agent/dag.rs, server-rs/src/agent/blackboard.rs, server-rs/src/agent/cascade_router.rs, server-rs/src/agent/verification_gate.rs |
DAG parallelism, shared blackboard, tiered model routing, zero-trust verification |
| Execution | execution/ |
Python tools, MCP server, JSON skill definitions, verification scripts, modular skill framework |
The engine boot path starts in server-rs/src/main.rs, initializes environment and tracing, creates AppState, starts background workers, spawns system actors under the OTP supervisor tree, launches the orchestrator, and binds Axum on 127.0.0.1:8000 unless configured otherwise.
server-rs/src/router.rs assembles all /v1 API routes. The public engine health route (GET /v1/engine/health) remains open; management routes and WebSocket streams require authentication (Authorization: Bearer <NEURAL_TOKEN> or Sec-WebSocket-Protocol: bearer.<token>). When dist/ exists, the same Rust process serves the built React app with SPA fallback.
| Path | Purpose |
|---|---|
src/ |
React dashboard, stores, hooks, components, services, contracts, pages, tests |
server-rs/src/ |
Rust API, agent runner, AppState hubs, actors, middleware, routes, telemetry, security |
server-rs/migrations/ |
SQLite schema migrations |
execution/ |
Python execution layer, MCP host, skill manifests, verification and audit utilities |
execution/core/ |
Modular skill framework foundation |
directives/ |
Governance, identity, orchestration, and provider operating instructions |
docs/ |
Architecture, operations, API reference, OpenAPI, and security docs |
data/ |
Local runtime data (initialized on first boot with SQLite database tadpole.db and agent/model seed configs) |
crates/ |
Workspace crates: WASM binary pulse codec (crates/wasm-codec), Rust AST guard (crates/tadpole-guard), .gitleaks.toml-backed secret scanner (crates/tadpole-sec), and experimental unwired graph physics prototype (crates/wasm-graph-physics) |
starter_kits/ |
Ready-to-deploy multi-agent swarms and domain workflows |
scripts/ |
Database backup, recovery, and persistence verification utilities |
dist/ |
Production dashboard build served by the Rust engine |
tests/ |
Shared frontend test setup and e2e support |
Navigation is defined in src/constants/routes.ts.
| URL | View |
|---|---|
/dashboard |
Operations dashboard |
/org-chart |
Hierarchy |
/standups |
Standups |
/workspaces |
Workspaces |
/missions |
Missions |
/models |
Model manager |
/agents |
Agent manager |
/engine |
Engine dashboard |
/oversight |
Oversight |
/skills |
Skills |
/benchmarks |
Benchmarks |
/scheduled-jobs |
Jobs |
/infra/model-store |
Intelligence store |
/docs |
Documentation |
/settings |
Settings |
/store |
Template store |
/security |
Security |
/governance |
Governance |
Detached windows are available at /detached-view, /detached/swarm-pulse, and /detached/chat.
The Rust engine binds to 127.0.0.1:8000 by default. Application APIs are nested under /v1 alongside root operational endpoints /health and /metrics.
Public routes:
| Method | Route | Purpose |
|---|---|---|
GET |
/health |
Engine health check (root unauthenticated ping) |
GET |
/v1/engine/health |
Engine health check (unauthenticated ping) |
Protected route groups and WebSocket streams:
| Prefix / Route | Purpose |
|---|---|
/metrics |
Prometheus metrics scrape endpoint (Authorization: Bearer <token>) |
/v1/engine/ws |
Live mission telemetry WebSocket stream (Sec-WebSocket-Protocol: bearer.<token>) |
/v1/engine/live-voice |
Live voice audio WebSocket stream (Sec-WebSocket-Protocol: bearer.<token>) |
/v1/agents |
Agent CRUD, graph, tasks, pause/resume, memory, ledger, claims, receipts |
/v1/a2a |
Agent-to-Agent communication protocols and inter-agent delegation |
/v1/oversight |
Decisions, governance settings, ledger, quotas, audit trail, health, policy, token burn |
/v1/infra |
Node discovery and infrastructure nodes |
/v1/model-manager |
Providers, models, catalog, pulls, provider tests |
/v1/skills |
Skill manifests, MCP tools, imports, promotion, scripts, workflows, hooks |
/v1/benchmarks |
Benchmark definitions, runs, and history |
/v1/continuity |
Scheduled jobs and workflows |
/v1/docs |
Knowledge docs and operations manual |
/v1/system |
Compute profile and system introspection |
/v1/governance |
Blueprints and sovereign manifest |
/v1/sovereign |
Mission session tree and branch state |
/v1/cas |
Lean read-only file revision history for diff observability |
/v1/missions |
Mission run catalog, cryptographic flight dossier, and retention pinning |
/v1/intelligence |
High-fidelity symbol graph mapping and dependent blast-radius analytics |
/v1/knowledge |
Sovereign knowledge base nodes, edges, peers, and synthesis |
/v1/iacp |
Inter-Agent Communication Protocol (negotiate, hire) |
/v1/search/memory |
Global memory search |
/v1/memory/search/bm25 |
High-speed BM25 lexical repository and doc search |
/v1/memory/search/hybrid |
Hybrid RAG Triad RRF fusion (Vector + BM25 + TrustGraph) |
/v1/env-schema |
Runtime environment schema |
/v1/engine/* |
Deploy, kill, shutdown, transcribe, speak, template install |
/v1/mcp/* |
MCP SSE and message bridge |
Protected routes require:
Authorization: Bearer <NEURAL_TOKEN>For WebSocket connections (/v1/engine/ws, /v1/engine/live-voice), pass the token via the subprotocol header:
Sec-WebSocket-Protocol: bearer.<NEURAL_TOKEN>BM25 lexical search (/v1/memory/search/bm25), TrustGraph entity traversal, and SQLite fallback memories are active by default. Semantic vector k-NN search and LanceDB embeddings return 501 Not Implemented unless the Rust vector-memory Cargo feature is enabled.
| Command | Purpose |
|---|---|
npm run dev |
Start Vite on port 5173 |
npm run engine |
Run the Rust engine via Cargo |
npm run build |
Type-check and build the frontend |
npm run lint |
Run ESLint |
npm run typecheck:ts7 |
Type-check frontend with TypeScript 7 compiler |
npm run test |
Run Vitest |
npm run test:invariants |
Run executable system invariant gates in tests/invariants/ |
npm run test:evals |
Run 20-task ground truth architectural evaluation benchmark suite |
npm run test:coverage |
Run Vitest with coverage |
npm run test:e2e |
Run Playwright end-to-end suite |
npm run test:py |
Run Python unit tests in tests/unit/ |
npm run test:rust |
Run backend Rust engine test suite via Cargo |
npm run test:wasm |
Run WASM codec test suites |
npm run wasm:build |
Build WASM binary package with wasm-pack |
npm run preview |
Preview the Vite build |
npm run docs:api |
Regenerate docs/openapi.yaml and docs/API_REFERENCE.md from server-rs/src/router.rs |
npm run docs:parity |
Run documentation/API/version parity checks |
npm run db:backup |
Hot backup SQLite database via WAL-safe connection .backup() API |
npm run db:restore |
Restore SQLite database with SHA-256 verification |
npm run audit:sovereign |
Run 7-pillar sovereign security, memory, and AST audit |
npm run context:verify |
Verify AI assist tags (@docs) across entire repository |
npm run tauri:dev |
Start Tauri dev mode |
npm run tauri:build |
Build Tauri app |
npm run docs:dev |
Start VitePress docs |
npm run docs:build |
Build VitePress docs |
npm run docs:preview |
Preview built docs |
npm run version:sync |
Sync version.json into manifests and docs |
Common environment variables:
| Variable | Default | Purpose |
|---|---|---|
NEURAL_TOKEN |
Required | Bearer token for protected API routes |
NEURAL_TOKEN_OLD |
Optional | Previous token kept valid during zero-downtime rotation |
NEURAL_TOKEN_NEW |
Optional | Replacement token staged during rotation grace window |
NEURAL_ENGINE_ACCESS_TOKEN |
Required alternative | Alternate auth token accepted by the engine |
ALLOWED_ORIGINS |
Local dev origins | Comma-separated origin allow-list; use * only for wildcard troubleshooting with credentials disabled |
PORT |
8000 |
Engine port |
BIND_ADDRESS |
127.0.0.1 |
Engine bind address |
STATIC_DIR |
dist |
Production static asset directory |
WORKSPACE_ROOT |
Current directory | Base path for data, scripts, skills, and panic logs |
DATABASE_URL |
sqlite:<workspace>/data/tadpole.db |
SQLx database URL |
RESOURCE_ROOT |
Optional | Static/model resource root |
HEARTBEAT_INTERVAL_SECS |
3 |
Engine health heartbeat cadence |
SKIP_DB_SEED |
false |
Skip baseline database seeding |
PRIVACY_MODE |
true |
Restrict execution toward local-only providers (zero-trust sovereign mode) |
AUTO_APPROVE_SAFE_SKILLS |
false |
When false, enforces human oversight for all skill executions |
ALLOW_HOST_SKILL_EXECUTION |
false |
When false, denies bare-metal host execution if container sandboxes are unavailable |
USE_SANDBOX_DOCKER |
false |
Run dynamic agent skills in isolated Docker micro-containers |
USE_SANDBOX_WASM |
false |
Run dynamic agent skills in isolated WebAssembly (Wasmtime) sandboxes |
TADPOLE_ALLOW_LOCAL_HTTP |
unset | Allows insecure local HTTP model-provider calls when set |
TADPOLE_NULL_PROVIDERS |
unset | Forces null providers for tests and integration runs |
DISABLE_TELEMETRY |
false |
Disables OpenTelemetry stdout exporter when true |
TOKIO_WORKER_THREADS |
available CPUs (min 4) | Tokio async runtime worker threads count |
TOKIO_MAX_BLOCKING_THREADS |
32 |
Tokio max blocking threads for blocking tasks |
TOKIO_THREAD_STACK_SIZE_MB |
4 |
Stack size per worker thread in MB |
Provider keys supported by .env.example:
OPENAI_API_KEYANTHROPIC_API_KEYGOOGLE_API_KEYGROQ_API_KEYDEEPSEEK_API_KEYREPLICATE_API_KEY
Build frontend assets:
npm run buildStart the engine:
npm run engineIf dist/ exists, the Rust router serves it automatically and falls back to dist/index.html for dashboard routes.
Override the static asset directory:
STATIC_DIR=distDefault Cargo features are intentionally empty to keep local Windows and lightweight machines from pulling in heavy native dependencies.
Enable vector memory:
cargo run --manifest-path server-rs/Cargo.toml --features vector-memoryEnable neural audio:
cargo run --manifest-path server-rs/Cargo.toml --features neural-audioEnable both:
cargo run --manifest-path server-rs/Cargo.toml --features vector-memory,neural-audio- Default database:
data/tadpole.db. - Rust migrations:
server-rs/migrations/. - Agent data is loaded from SQLite and related registry files under the workspace data directory.
- Provider and model registries are persisted on graceful shutdown by
AppState. - Panic diagnostics are written to
sidecar_panic.logunderWORKSPACE_ROOTwhen possible.
Frontend tests:
npm run testFrontend type-check and production build:
npm run buildRust tests:
cargo test --manifest-path server-rs/Cargo.tomlPython-side verification utilities:
execution/verify_all.pyexecution/verify_ai_context.pyexecution/parity_guard.pyexecution/sovereign_audit.py
| Document | Path |
|---|---|
| Architecture | docs/ARCHITECTURE.md |
| Operations | docs/OPERATIONS_MANUAL.md |
| API reference | docs/API_REFERENCE.md |
| OpenAPI | docs/openapi.yaml |
| Security | docs/SECURITY.md |
| System map | SYSTEM_MAP.md |
The wiki/ directory contains a comprehensive knowledge base designed for developers and operators:
| Page | Description |
|---|---|
| π Home | Overview, system diagram, quick-start TL;DR |
| π Getting Started | Prerequisites, install, first run, Windows shortcuts |
| ποΈ Architecture | Engine boot, AppState hubs, router stack, agent pipeline, repo map |
| π‘ API Reference | All 50+ REST/WebSocket endpoints with schemas |
| βοΈ Configuration | Every env var, Cargo feature flag, and provider key |
| π Security Model | Auth, token rotation, MCP sandboxing, audit trail, budget guard |
| π₯ Operations Manual | Backup/restore, token rotation runbook, Grafana, GDPR, shutdown |
| π§ͺ Testing Guide | Rust/Python/React test suites, CI, common pitfalls |
| π» Development Guide | Local dev setup, conventions, adding endpoints, migrations |
| π§ Troubleshooting | 12 common issues with exact step-by-step fixes |
- Dual-Plane Sovereign Hierarchy (
directives/IDENTITY.md): Strictly partitions operational authority into Plane 1 (Sovereign Codebase Lifecycle Engine β recursively operates, diagnoses, fixes, and maintains the codebase) and Plane 2 (Company-Agnostic SMB AI Digital Twin β provides brand voice, customer workflows, and messy data ingestion without tenant PII leakage into engine source code). - FR-11 Permission Firewall (
server-rs/src/agent/runner/tools/security.rs,server-rs/src/agent/types/mod.rs): Enforces zero-trust execution boundaries. Agents operating under Plane 2 (SmbDigitalTwin) are strictly prohibited from host shell tool execution (execute_shell,run_command,terminal) and prohibited from mutating engine codebase paths (server-rs/,src/,execution/,directives/,.agent/).
- 24KB Tool Observation Compactor (
server-rs/src/agent/runner/turn_compactor.rs,server-rs/src/agent/runner/intelligence.rs): Automatically bounds inline tool observations. Outputs exceeding 24KB (24,000 characters) are flushed to.tmp/tool_overflow/with safe UTF-8 codepoint boundary slicing, preserving a 300-character preview and exact file reference pointer to eliminate context window overflow crashes. - Sliding-Window Cyclic Loop Detector (
server-rs/src/agent/runner/loop_detector.rs):DoomLoopDetectortracks a 16-signature history of SHA-256 normalized tool calls across periods 1 to 4. Features a 2-stage defensive circuit breaker:LoopCheckResult::Warninginjects steering advisories, andLoopCheckResult::Terminalimmediately halts repetitive execution loops to prevent runaway token exhaustion. - Pre-Commit UI Quality Guard (
execution/ui_quality_guard.py): Static zero-dependency pre-commit scanner auditing all 134 frontend TSX components againstDESIGN.mdcontracts (pure black avoidance, untinted grays, low-contrast badge text, missing button/SVGaria-labelaccessibility, and card inception).
- Persistent Telemetry Event Bridging (
server-rs/src/telemetry/mod.rs,server-rs/src/telemetry/bridge.rs):TelemetryLayer::on_eventbroadcasts structuredtrace:eventsignals toTELEMETRY_TX, which persists error events to SQLitemission_logs. - Trace Status Immutability (
src/stores/trace_store.ts): Hardened span lifecycle handling to ensure error spans remain marked asstatus: 'error'when closed. - WAL Concurrency Lock Timeout (
execution/mission_diagnose.py,execution/verify_telemetry.py): Added 30.0-second SQLite timeout andPRAGMA busy_timeout = 30000;preventing lock contention with active server writers during diagnostic scans.
- Monolith Decomposition: Decomposed legacy multi-thousand-line monoliths into clean, cohesive sub-packages:
server-rs/src/routes/oversight/(12 modules),server-rs/src/routes/templates/(7 modules),server-rs/src/routes/missions/(7 modules), andserver-rs/src/agent/runner/synthesis/(6 modules). - Convoy-Free Agent Persistence (
server-rs/src/state/persist.rs): Replaced global 64-stripe mutex lock acquisition with lock-freeDashMapsnapshotting during 30-second periodic flushes, eliminating swarm-wide convoy stalling while preserving SQLite OCC safety (WHERE agents.id = excluded.id AND agents.version = ?). - Zeroized Thread-Safe Key Management (
server-rs/src/security/audit.rs): Removed process-levelstd::env::set_varkey exports, preventing multi-threaded libc memory races and eliminating Ed25519 private key leakage to child subprocesses and/proc/self/environ. - Bounded AST Traversal Ceilings & Cooperative Cancellation (
crates/tadpole-intelligence/src/graph/engine/analysis.rs,server-rs/src/intelligence/service.rs): Implemented cooperativeAtomicBoolcancellation and traversal step ceilings (MAX_TRAVERSAL_STEPS) in BFS blast radius analysis, preventing CPU runaway and thread pool starvation in Tokiospawn_blockingworkers during dense graph traversals. - Workspace Build Acceleration (
Cargo.toml,crates/tadpole-types/,crates/tadpole-intelligence/): Extracted pure domain DTOs and the Tree-sitter/petgraphintelligence engine into cached workspace crates and tuned[profile.dev]/[profile.test](opt-level = 0,debug = "line-tables-only") for rapid incremental dev iteration. - Overlord 10/10 Command & Day-1 Onboarding (
execution/tadpole_init.py,src/components/onboarding/FirstRun_Wizard.tsx,src/components/layout/Overlord_Inbox_Bar.tsx,src/components/model/Readiness_Card.tsx): Addednpm run initsecret-safe bootstrapping, 4-step First-Run Wizard, progressive Focus (6-view) vs Expert (18-view) cockpit (Ctrl+Shift+E), keyboard-first Oversight Inbox (j/k/a/r/g/Shift+Alow-risk batch approve) with live WebSocket push, tieredOperatorScopeRBAC (NEURAL_TOKEN_ADMIN/NEURAL_TOKEN_READONLY), and nativePOST /v1/benchmarks/readinesspre-flight scoring.
- OTP Actor Supervision Tree (
server-rs/src/system/actors/supervisor.rs): Erlang/OTP-style supervision engine supportingOneForOne(restart individual) andOneForAll(cascade restart all siblings) strategies,AbortHandledeterministic hard shutdown, stability-based exponential backoff with auto-reset, and locklessDashMapchild registry. - Hybrid RAG Triad Fusion (
server-rs/src/services/rag_fusion.rs): Reciprocal Rank Fusion (RRF) combining LanceDB Vector (w=0.40), BM25 Lexical (w=0.35), and TrustGraph Entity (w=0.25) with deduplication and multi-engine intersection boosting. Exposed viaGET /v1/memory/search/hybrid. - Durable Workflow Step Memoization (
server-rs/src/agent/durable.rs): SQLite-native crash-resilient execution engine. Completed workflow steps are SHA-256 hashed and cached; on engine restart, previously completed steps are fast-forwarded from SQLite with zero token waste. Mutation-aware: automatically re-executes when input parameters change.
-
Zero-Turn Socratic Context Contracts (
server-rs/src/agent/socratic.rs): Deterministic 4-Pillar envelope auto-injection (Scope, Performance Threshold, Architecture Mode, Pre-Cleared Failure Policies) with typedBlastRadiusLevel(Level1ReadOnly, Level2WorkspaceLocal, Level3SystemWide), configurable policy profiles (SocraticDefaults), micro-cent budget precision (budget_cents_cap), and zero-allocationCow<'a, str>prompt injection to eliminate conversational permission stalling in autonomous agent swarms. -
Dynamic Swarm Pulse Telemetry & Edge Hierarchy (
server-rs/src/telemetry/pulse.rs): Real-time MessagePack pulse stream calculating dynamic node progress (progress: f32) from agent reasoning turns/lifecycle and auto-constructing parent-child hierarchical edge connections (PulseConnection) fromparent_agent_idorlead_agent_id. -
Recursion-Safe Tiered Recruitment & Synthesis Window Guard (
server-rs/src/agent/runner/swarm/mod.rs): Multi-tier recruitment (User Specialist$\rightarrow$ AI Pool$\rightarrow$ Keyword Fabrication) with lineage checks (ctx.lineage.contains(&sub_agent_id)), dynamicactive_model_slotresolution from agent config, and safe UTF-8 synthesis context truncation (MAX_SYNTHESIS_PAYLOAD_CHARS) to prevent context overflows. -
Local-First Model Slot Optimizer (
execution/optimize_local_slot_routing.py): Deterministic Layer 3 utility auditing installed Ollama models and non-destructively configuring Slot 1 (Strategic Lead Reasoning) and Slot 2 (Fast Sub-Worker Execution) compliant with.envPRIVACY_MODE. -
Shared Mission Blackboard (
server-rs/src/agent/blackboard.rs): High-performance thread-safe in-memory scratchpad (DashMap+Arc<BlackboardEntry>) for multi-agent data exchange, replacing large prompt string passing with lightweight O(1) key pointers. UTF-8 safe truncation and generic tag filtering. -
Dynamic DAG Task Engine (
server-rs/src/agent/dag.rs): Directed task dependency graph built onpetgraph::StableDiGraphwith topological cycle rejection, parallel ready-queue extraction, state transition validation, and deadlock-free failure cascading via BFSSkippedpropagation. -
Tiered Model Cascade Router (
server-rs/src/agent/cascade_router.rs): Dynamic turn routing between Tier 1 Fast (Ollama/Groq/Gemini Flash) and Tier 2 Frontier Reasoning (Gemini Pro/Claude/GPT-4o). Externalizedcritical_keywordsinCascadePolicy. Capability-aware error escalation (skips auth errors, escalates on JSON/format failures). StructuredRoutingDecisionreturn type. -
Aletheia Verification Gate (
server-rs/src/agent/verification_gate.rs): Zero-trust GeneratorβVerifier triad for high-impact mutations. O(1)HashSetsensitive skill gating, independent Verifier blast-radius evaluation (mitigates the "Honesty Problem" of generator self-reporting), and calibrated threshold (default: 15 symbols). -
Adaptive Context Slicer (
server-rs/src/agent/context_slicer.rs): Cognitive 3-zone prompt assembly (Pinned Anchors,<grounded_context>XML RAG, Sliding Active Window) with stricttiktokenBPE token budget enforcement and pre-allocated heap buffers viastd::fmt::Write.
- Zero-Downtime Token Rotation: Auth middleware now supports dual-token validation via
NEURAL_TOKEN_OLD/NEURAL_TOKEN_NEWenv vars, enabling safe production token rotation with a configurable grace window. Documented indocs/SECURITY.md. - MCP Subprocess Sandboxing: Hardened
execution/tadpole_mcp_server.pywithasyncio.create_subprocess_exec(noshell=True),shlex-based command splitting, an environment variable allowlist (_ALLOWED_ENV_VARS), JSON Schema input validation, hard 30-second execution timeout, andresource.setrlimitCPU/memory constraints on Linux/Docker.
- Lock-Free Concurrent Graph Engine: Replaced
parking_lot::RwLockwith lock-free concurrent access viaarc_swap::ArcSwapinResourceHubto guarantee non-blocking reads during background graph rebuilds. - Path Traversal Hardening & Error Segregation: Tightened
/v1/intelligence/blast-radiusand/v1/intelligence/resolveto validate input paths unconditionally. Missing paths return404 Not Found(IntelPathUnknown), whereas path boundary violations (like directory traversal attempts) return403 Forbidden(Forbidden). - Oversight Ledger Pagination Fix: Resolved a critical double-pagination bug in
/v1/oversight/ledgerand/v1/oversight/security/audit-trailthat capped the approved card count in the dashboard at 100 entries. - SHA-256 Hashing for Code Review Graph: Replaced MD5 hashing with SHA-256 in
graph_store::scan_files::file_hashand bumped the schema version to 10 with a dual-writefile_hash_sha256migration window. - Git Resolution at Boot: Cached
gitexecutable absolute path at boot viawhichto ensure hermetic and safe execution of sub-processes. - Axum Request Limit: Added a global 16 MiB default request limit in router (
DefaultBodyLimit::max(16 * 1024 * 1024)) to safely accommodate large AST code-graph payloads while preventing Denial-of-Service attacks. - Robustness Cap Warning and Truncation: Demoted
MAX_NODESandMAX_EDGESexceeding checks from fatal errors to early-termination warning logs, leaving the previous graph intact, and resolved workspace-wide file bloat viaMAX_DISCOVERED_FILEStruncation.
- Hot SQLite Backup & Restore: Added
execution/backup_sqlite.pyandexecution/restore_sqlite.pywith WAL-safe online backups (.backup()API), SHA-256 integrity hashing, andPRAGMA integrity_checkverification. - DB helper API in Rust: Added
run_backup()andcheck_integrity()helpers toserver-rs/src/db/init.rsfor programmatic backup orchestration. - Durable Workflow Migration: Added
server-rs/migrations/20260822000100_durable_workflows.sqlfor step memoization table.
- Extended Health Endpoint:
GET /v1/engine/healthnow returns structureddatabase(WAL size, pool stats),budget(spend, limit),swarm(agent count, status), anduptime_secondsfields. - Monitoring Dashboards: Added
monitoring/alerts.yml(Prometheus alert rules) andmonitoring/grafana/dashboards/tadpole_dashboard.json(pre-built Grafana dashboard).
- Rust Integration Tests (
server-rs/src/routes/):backup_restore_tests.rsβ hot backup +PRAGMA integrity_checktoken_rotation_tests.rsβ dual-token grace window authhealth_endpoint_tests.rsβ extended health schema validationshutdown_orchestrator_tests.rsβ graceful shutdown + emergency kill switch
- Python Unit Tests (
tests/unit/):test_backup_integrity.py,test_token_rotation.py,test_mcp_sandbox.pytest_snapshot_state.py,test_verify_ai_context_fix.py
docs/OPERATIONS_MANUAL.md: Added Token Rotation Runbook, Database Backup/Restore procedure, Sandbox Snapshotting guide, MCP Security Hardening details, Prometheus/Grafana alerting setup, and GDPR cascading deletion runbook.docs/SECURITY.md: Updated with dual-token rotation mechanism, MCP subprocess boundary hardening, and RCE mitigation details.docs/ARCHITECTURE.md: Added Swarm Orchestration Engine, Hybrid RAG Triad Fusion, and Sovereign Engine Hardening sections.SYSTEM_MAP.md: Registered all 8 new subsystems in the subsystem table.directives/rust_engine.md: Expanded Subsystem Registry from 6 to 14 subsystems.directives/AUTONOMY_MANIFEST.md: Added Priority 7 (Swarm Orchestration Engine verification).
-
Native Multi-Role ChatMessage Protocol: Transitioned conversation dispatch from flat strings to a strongly typed
ChatMessage(System,User,Assistant,Tool) pipeline across Anthropic, OpenAI, Gemini, and Groq providers, eliminating prompt injection surface and instruction drift. -
Native Ephemeral Prompt Caching: Places
cache_control: {"type": "ephemeral"}on structured Anthropic and Gemini system blocks, drastically cutting repetitive multi-turn reasoning costs by up to 90%. -
Token-Based Context Window Accounting &
AppError::ContextOverflow: Evaluates total token usage viaTokenizerServicebefore provider dispatch, safely returning a repairable HTTP 413 error to trigger dynamic compaction without dropping session state. -
Deterministic Tool Repetition & Loop Guard: Tracks consecutive identical
(tool_name, args_hash)invocations, injecting[TOOL_WARNING]at 3 repetitions and safely terminating recursion at 5 withRunnerError::RecursionBlocked. -
Mission Artifact Verification: Validates physical disk presence of all declared output files during
complete_mission, rejecting hallucinations with structuredToolExecutionError::Validation. -
Heartbeat SQLite Cooldown: Throttles agent heartbeat writes to a
$\ge 10$ -second cooldown for SQLite persistence while maintaining immediate in-memory registry updates. -
20-Task Ground Truth Evaluation Suite: Automated benchmark suite (
evals/tasks.jsonandevals/run.py, runnable vianpm run test:evals) asserting 20 architectural invariants with 100% ground-truth pass rate.
- CI Pipeline Fix: Corrected missing
uses:keyword in.github/workflows/ci.ymlthat caused workflow parse failures. - Compiler Warning Cleanup: Eliminated all 70 Rust compiler warnings β unused imports (feature-gated re-exports, test fixtures), unused variables prefixed with
_, and dead code in scaffolded actor subsystems suppressed with targeted#[allow(...)]attributes.
- Shared Tool Cache with Monotonic MTime Invalidation (
server-rs/src/agent/runner/tools/cache.rs): Replaces indiscriminate SQLite caching with an in-memory, thread-safe LRU tool cache. Restricts caching strictly to idempotent read-only tools (read_file,list_files,list_file_symbols,search_mission_knowledge,fetch_status). Validates file modification time (mtime) and byte length before serving hits, and triggers path-specific or global cache invalidation on mutating tools (write_file,patch_file,replace_file_content,run_command, etc.). - Cyclic DoomLoopDetector (
server-rs/src/agent/runner/loop_detector.rs): Detects complex multi-step cyclical tool loops (periods 1, 2, 3, and 4) using normalized JSON key ordering and SHA-256 signatures of tool name, arguments, and output hashes. Intercepts oscillating patterns (e.g., A β B β A β B) before exhausting mission budgets. - Two-Stage Monologue Compaction (
server-rs/src/agent/runner/monologue.rs): Implements 2-stage compaction: (1) trimming verbose embedded code blocks and tool logs (>2,000 chars) from intermediate agent reasoning, and (2) dense LLM summarization of older conversational turns while strictly preserving the hot tail (last 4 turns) verbatim. Includes deterministic fallback summarization when LLM calls are unavailable. - SSRFGuard with DNS Rebinding Defense (
server-rs/src/security/ssrf_guard.rs): Hardens outbound HTTP tool requests (fetch_web_content) against server-side request forgery. Enforces RFC 1918 private, RFC 5737 documentation, loopback, CGNAT (100.64.0.0/10), link-local, and IPv6 unique-local CIDR rejections. Pins the resolved IP directly intoreqwest::ClientBuildervia DNS pre-resolution to eliminate Time-of-Check to Time-of-Use (TOCTOU) DNS rebinding vulnerabilities. - SpanWatchdog with Provider-Aware Leaked Span Reaping (
server-rs/src/telemetry/span_watchdog.rs): Background system service sweeping active telemetry spans. Enforces provider-aware TTLs (60s for cloud providers, 300s for local Ollama), automatically closing orphaned spans with syntheticSPAN_TIMEOUT_REAPEDevents to safeguard mission observability. - Mission Benchmark Harness (
tests/mission_bench/): Standardized 5-scenario benchmark test suite (tests/mission_bench/run.pyandtests/mission_bench/judge.py) validating tool caching, doom loop detection, monologue compression, SSRF protection, and span leak recovery.
- Institutional Knowledge Store (IKS): Implemented cross-restart, cross-cluster durable memory storing agent playbooks, decision history, and curated facts. Uses SQLite metadata indexing + LanceDB k-NN semantic search, complete with a time-aware confidence decay rate (0.01 per day) and automatic TTL eviction sweeps.
- Path Traversal Hardened Graph Engine: Re-engineered
/v1/intelligence/blast-radiusand/v1/intelligence/resolveto look up paths via an obfuscated-to-real-path registry, enforcing strict directory checks against workspace boundaries to prevent traversal attacks. - Sovereign Chat UI Modularization: Monolithic chat layouts have been split into standalone, testable React components under
src/components/chat/and custom hooks (use_chat_dispatch,use_chat_voice). - Observability Tag Alignment: Unified telemetry tags (
[okf_gate],[graph],[IKS],[intelligence]) across comments and tracing logs to ensure 100% synchronization and prevent documentation drift.
- The default engine token is intentionally required outside tests.
- The frontend stores the API token locally through the settings flow and sends it as a bearer token to protected endpoints.
- Vite runs on port
5173; the Rust engine runs on port8000. - The Rust engine can serve the built dashboard directly from
dist/, so production mode does not require a separate Vite server. vector-memoryandneural-audioare opt-in Cargo features because they can introduce heavier native dependencies.- SQLite backups use the online
.backup()API (and programmaticVACUUM INTOin engine helpers) and must target a file path (not in-memory). Runexecution/backup_sqlite.pybefore any major migration or deployment. - Token rotation uses a two-token grace window: set
NEURAL_TOKEN_OLDto the current token andNEURAL_TOKEN_NEWto the replacement before restarting the engine.