Voice-first AI coding companion with animated Skia sphere interface — and the mature decision to contribute upstream instead of building alone.
Role: Solo builder — full-stack mobile + backend
Status: Private prototype. Case study tells the engineering story and the strategic pivot.
Timeline: May 2026
NOVA was a voice-first mobile interface for a personal remote coding agent. You talk to it. It talks back. It does real work — creating repos, writing files, executing code — through an Anthropic-powered backend with GitHub integration.
The app had two modes:
- Talk Mode — a full-screen animated Skia sphere that listens, processes, and speaks. Voice-first interaction with tap-to-talk, auto-submit on silence, and tap-to-interrupt.
- Work Mode — the trust surface showing active jobs, task timelines, approval cards for risky operations, and completed results.
I wanted a coding agent I could talk to — not type at. Something that felt like a real companion: present, responsive, and safety-aware. The sphere UI was the physical embodiment of that idea: a glowing orb that shifts between idle, waiting, listening, thinking, and speaking states, giving continuous visual feedback.
- Skia sphere animations — real-time visual feedback for voice states using React Native Skia and Reanimated. Particles, audio reactivity, smooth transitions.
- Voice pipeline — speech recognition via
expo-speech-recognition, text-to-speech viaexpo-speech, silence-based auto-submit, tap-to-interrupt, long-press cancel. - Conversation Manager — 712-line state machine orchestrating full voice flow: listen → transcribe → send to agent → receive response → speak.
- Backend Service — WebSocket client with reconnection, message routing, and job-aware protocol support.
- Zustand state management — centralized app state (interface mode, sphere state, conversation state, job state).
- Approval UI — Work Mode renders approval cards for risky operations. Voice approval supported: saying "approve" or "cancel" routes to backend approval flow.
- WebSocket server —
/wsendpoint for real-time bidirectional communication with the mobile app. - Agent integration — 978-line agent module integrating Anthropic's tool-use API. Handles conversation turns, tool calling, and structured responses.
- GitHub client — 487-line module wrapping direct GitHub API operations (create repos, manage files, handle visibility).
- Backend-enforced approval gate — risky operations (delete repo, delete file, overwrite file, change visibility) require explicit backend approval. The backend stores the exact pending action and only executes on explicit approval — the model cannot bypass this.
- Telemetry — local JSONL event logging with a
/telemetrymonitoring snapshot endpoint. - Deterministic evals — backend test suite verifying agent tool loop behavior.
The project had genuine documentation discipline:
- Architecture and product behavior doc
- Setup and testing guide
- Deployment guide (Render + EAS/TestFlight)
- Stabilization plan (what needed to be solid before remote workers)
- Sprint summaries tracking progress
.brain/persistent memory directory for cross-session context
Around the same time, I started using Hermes — an open-source AI agent with a CLI, TUI, multi-platform gateway, plugin system, cron scheduling, and a real engineering team behind it.
I realized Hermes was quite literally the final version of what NOVA was trying to become.
NOVA was a voice interface for one agent. Hermes is the agent itself — with more tools, more platforms, more maturity, and a community contributing to it.
The choice became clear: I could spend months building NOVA toward feature parity with something that already existed, or I could take what I learned from building NOVA and contribute it upstream to Hermes.
I chose the latter.
Building NOVA taught me several things that directly apply to Hermes contributions:
-
Voice UX is hard. Silence detection, interruption handling, state transitions, TTS timing — voice interaction is a deep UX problem, not just an API integration. I built all of it from scratch and understand the edge cases.
-
Mobile agent interfaces matter. A touch-first, voice-first mobile companion feels qualitatively different from a terminal or chat window. The sphere UI wasn't just decoration — it gave presence and continuous feedback that text can't.
-
Safety gates must be backend-enforced. I designed NOVA's approval flow so the backend stores the exact pending action and only executes on explicit human approval. Model instructions alone are insufficient — the server must be the authority.
-
Solo building has limits. You can build a lot alone with obsession and skill, but at some point the surface area of a good AI agent exceeds what one person can maintain. Recognizing that and joining an open-source project is not giving up — it's the rational engineering decision.
-
Documentation is infrastructure. NOVA's docs (architecture, setup, deployment, stabilization plan, sprint summaries) meant I could context-switch effectively. This discipline carries directly into open-source contribution.
I built a voice-first AI coding companion from scratch — React Native with an animated Skia sphere interface, FastAPI backend with Anthropic integration and a GitHub client with backend-enforced safety gates. I learned a ton about voice UX, mobile agent interfaces, and safety architecture. Then I discovered Hermes — an open-source agent that was the evolved version of exactly what I was building — and made the strategic decision to contribute upstream rather than duplicate effort. I took everything I learned from NOVA and am now applying it to Hermes contributions.
The NOVA source code is a private prototype. This case study describes the architecture and engineering decisions without exposing implementation details that could be sensitive. The code remains private; the learnings are public.
Built by Mohamed Abdelgelil, May 2026. Case study created 2026-06-10T13:38:01.