Project J.A.R.V.I.S. 🤖

A sci-fi AI assistant with a holographic React + Three.js interface and a FastAPI brain.
Voice-activated · LLM-agnostic · Real macOS system actions · Zero-shot voice cloning

---

Screenshot / Demo

---

Table of Contents

• Features
• Prerequisites
• Security Notice
• Project Structure
• Quick Start
• Manual Installation
  • Backend
  • Frontend
  • Voice Cloning
  • Coqui TTS (optional)
• License

---

Features

Feature	Description
🎙 Voice activation	Hold-to-talk or toggle mode, configurable hotkey
🌐 Multi-provider LLM	Groq · Mistral · OpenRouter · Ollama (auto-fallback)
🔊 Voice cloning	Pocket-TTS zero-shot cloning from a 5-second reference WAV
🖥 macOS system actions	Open URLs/apps, create/read/write/delete files, screenshots, sysinfo
📡 Telegram bot	Optional webhook integration
🎨 Holographic HUD	Microphone-reactive Three.js blob, FFT bands, provider status

---

Prerequisites

• OS: macOS (system actions use screencapture, pmset, open)
• Node.js ≥ 18
• Python ≥ 3.10
• API keys (optional): Groq, OpenRouter, Mistral, or Hugging Face — or run fully locally with Ollama

---

Security Notice

Warning

Never commit .env files or hard-code API keys.

• All .env files and virtual environments are in .gitignore.
• Manage keys at runtime via the Settings → API Keys tab in the frontend.

---

Project Structure

Jarvisai/
├── backend/
│   ├── app/
│   │   ├── main.py          # FastAPI app, routes, CORS
│   │   ├── brain.py         # LLM provider logic + fallback chain
│   │   ├── tools.py         # macOS system-action dispatcher
│   │   ├── tts.py           # TTS synthesis (HuggingFace / Coqui / gTTS)
│   │   ├── voice_engine.py  # Pocket-TTS voice cloning engine
│   │   ├── telegram_bot.py  # Optional Telegram webhook
│   │   └── models.py        # Pydantic request/response models
│   ├── voices/              # Generated audio output (git-ignored)
│   ├── .env.example
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── App.tsx          # Main application shell
│   │   ├── components/      # MicBlobScene, SettingsModal, TerminalLog
│   │   ├── hooks/           # useSpeechToText, useDubbedAudio
│   │   └── config.ts        # API_BASE configuration
│   └── package.json
└── start.sh                 # One-command launcher

---

Quick Start

git clone https://github.com/satiricalguru/Jarvis.git
cd Jarvis

# Install backend deps
cd backend && python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt && cp .env.example .env
cd ..

# Install frontend deps
cd frontend && npm install && cd ..

# Launch both servers
chmod +x start.sh && ./start.sh

Open http://localhost:5173 in your browser.

---

Manual Installation

1. Backend Setup

cd backend
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env        # then fill in your keys
uvicorn app.main:app --reload --port 8000

2. Frontend Setup

cd frontend
npm install
npm run dev                 # runs at http://localhost:5173

Zero-shot Voice Cloning Setup

1. Place a 5-second WAV sample of your target voice at backend/voices/jarvis.wav.
2. Add your HUGGINGFACE_TOKEN to backend/.env and accept the pocket-tts model terms.
3. Chat replies will automatically be dubbed in that voice.

Optional: Coqui TTS Integration

cd backend && source .venv/bin/activate
pip install -r requirements-tts-coqui.txt

---

License

This project is licensed under the MIT License — see LICENSE for details.