EchoAI

EchoAI is a Windows local AI agent workspace. It grew out of a desktop clipboard workflow and is being rebuilt into a local companion-style AI system with a clipboard bar, dynamic island, ZED music radio, voice conversation, screen context, tool calling, local model runtime templates, and a long-term memory roadmap.

EchoAI is not trying to be only a chat window. The goal is a desktop-resident assistant that can understand the current work context, respond through voice, be interrupted naturally, call local tools, and gradually develop useful memory with explicit user control.

Current Status

Implemented and ready for local validation:

• Baseline event spine for voice, subtitles, TTS, interruption, screen context, and regression evidence.
• Context builder seam for ZED/avatar runtime context, making clipboard, screen, window timeline, todo, and memory sources easier to connect.
• Unified abort boundary for stopping AI replies, TTS playback, subtitle flow, and active session work through one path.
• Initial on-demand screen context and OCR diagnostics.
• ZED companion UI direction: visible identity changed from Lumi to ZED, old menu/status/dock/settings surfaces reduced, and the AI panel made more compact.
• Voice reply rhythm tuning: default spoken replies are limited to about five sentences to reduce long unfinished playback.

Still unfinished:

• Long-term memory on the live path. The packages/echoai-lumi-kernel/src/memory foundation exists, but stable retrieval, write-back, deduplication, confidence, user controls, and acceptance gates still need a dedicated phase.
• Streaming STT mainline. The current voice flow is usable, but a full streaming STT path is still future work.
• Screen context accuracy. The first version can collect and diagnose screen context, but multi-monitor coverage, OCR timeout behavior, and content accuracy still need tuning.

Features

ZED AI Companion

• Desktop AI entry point for text and voice.
• Persona prompt and visible identity centered on ZED.
• Tool calling, task execution, voice diagnostics, interruption, and evidence reports.
• Internal compatibility names such as lumi may still exist in code paths to avoid breaking IPC and historical data.

Voice And Digital Human Flow

• STT, TTS, subtitles, voice state, and playback evidence.
• Manual and audio-triggered interruption paths.
• Small sentinel path for reducing accidental interruption from short noise.
• Local and cloud TTS/STT provider adapters with diagnostics.

Screen Context

• Window and desktop context collection.
• OCR path and timeout diagnostics.
• On-demand screen context for prompts such as asking what is currently on screen.

Clipboard Bar And Dynamic Island

• Clipboard history and desktop material entry point.
• Lightweight desktop control surface and AI status display.
• Future settings should move toward small island controls instead of a large settings window.

ZED Music Radio

• Music/radio windows and playback-related services.
• Netease sidecar integration and smoke checks.
• Music intent and radio/avatar coordination paths.

Local Tools And Models

• Electron main-process services for local automation and diagnostics.
• Runtime templates for Whisper, TTS, SAPI, Edge TTS, and cloned TTS-compatible servers.
• GPT-SoVITS setup scripts are included as integration helpers, but model weights and local voice samples are not committed.

Tech Stack

• Electron
• Node.js / JavaScript
• Native HTML/CSS/JS
• React + Vite for newer renderer surfaces
• sql.js / SQLite WASM
• Electron preload IPC bridge
• PowerShell and Node.js scripts for local validation
• electron-builder for Windows packaging

Repository Layout

EchoAI/
├── main.js                    # Electron main process and core IPC
├── preload.js                 # Safe renderer bridge
├── renderer/                  # Original desktop UI
├── renderer-island/           # Native dynamic island UI
├── renderer-island-react/     # React dynamic island UI
├── renderer-avatar-react/     # AI avatar / companion UI
├── renderer-music*/           # ZED music radio UI
├── renderer-settings-react/   # Settings and diagnostics UI
├── services/                  # Local services and reports
├── scripts/                   # Regression, smoke, acceptance, and dev scripts
├── packages/echoai-lumi-kernel/
│   └── src/                   # Kernel primitives for turn, memory, TTS, STT, tools, session
├── models/                    # Runtime templates and small model metadata
├── tools/                     # EchoAI integration helpers only
├── docs/                      # Requirements, plans, architecture, and task cards
├── dev-logs/                  # Development logs
└── data/                      # Local runtime data, ignored by Git

Local Development

git clone https://github.com/Zedking18/EchoAI.git
cd EchoAI
npm install
npm run dev:local

API keys and local secrets should be configured on the developer's own machine. Copy .env.example to .env if you want to use environment variables, or configure provider keys through the EchoAI settings UI.

Useful commands:

npm run dev:status
npm run verify:voice
npm run test:installed

Voice and digital human checks:

npm run voice:session-report
npm run digital-human:voice-rhythm-check
npm run digital-human:barge-in-preflight

Screen context checks:

node scripts/check-screen-context-regression.js
node scripts/check-lumios-desktop-relay-screen-regression.js

Public Source Boundary

This repository is the public EchoAI source tree. It intentionally excludes:

• node_modules/
• dist/
• data/
• .env files and local secrets
• local databases and logs
• local voice samples and generated audio
• local model weights
• third-party upstream clones and large local tool workspaces
• external reference source snapshots that are useful for local research but should not be presented as EchoAI code

See docs/open-source-boundary.md for the full boundary.

Roadmap

1. Finish long-term memory on the live conversation path.
2. Continue improving screen context, especially multi-monitor and OCR timeout behavior.
3. Move small settings such as language switching into the dynamic island.
4. Improve natural voice interruption and reduce accidental noise-triggered stops.
5. Add clearer installation and validation flows for local model runtimes.

License

EchoAI source code is released under the MIT License. Third-party dependencies, optional external tools, and reference materials keep their own licenses.