An offline, AI-powered Halloween fortune teller prop. Visitors approach a crystal ball, ask a question aloud, and receive a spooky spoken fortune. Runs entirely on a local mini PC with no internet required.
Stack: Faster-Whisper (speech-to-text) + Ollama / Llama 3.2 (fortune generation) + Piper TTS (text-to-speech)
The crystal ball runs as a continuous state machine designed to operate unattended as a prop:
RESTING ──(wake trigger)──> GREETING ──> LISTENING ──> THINKING ──> SPEAKING ──┐
^ │
│ ┌──(silence timeout or max questions reached)─────────────────┘
│ v
│ FAREWELL
└──────────────┘
- Rest - The ball sleeps with a dim purple glow, waiting for a wake trigger (button press, keyboard, foot pedal)
- Greet - A visitor triggers the ball, which wakes with a dramatic light flash and speaks its greeting
- Listen - Faster-Whisper with Silero VAD captures the visitor's question via microphone
- Think - Ollama generates an in-character fortune using a local LLM (a filler phrase plays while it "consults the spirits")
- Speak - Piper TTS speaks the fortune aloud through the speakers
- Loop or Farewell - After each answer, the ball listens for the next question. The session ends automatically after a configurable number of questions or a silence timeout, then the ball fades back to sleep.
Expected end-to-end latency is 2-4 seconds — a natural "communing with the spirits" pause. Ctrl+C or SIGTERM triggers a clean shutdown (LEDs off, resources released) from any state.
| Persona | Character | Style |
|---|---|---|
| voss | Madam Voss | Playfully spooky, warm, British accent |
| mordecai | Baron Mordecai | Brooding, ominous, grave baritone |
| barnacle | Captain Barnacle | Salty sea captain, gruff, pirate-adjacent |
Linux / macOS:
./run.sh # Interactive persona selection
./run.sh voss # Madam Voss by name
./run.sh personas/mordecai.yaml # Baron Mordecai by path
./run.sh barnacle --wake-device /dev/input/event5 # Wake on USB button/keyboard press
./run.sh voss --max-questions 5 # 5 questions per session
./run.sh voss --model mistral # Use Mistral 7B instead of Llama 3.2
./run.sh voss --length-scale 1.4 # Slower, more dramatic speech
./run.sh voss --wled-host 192.168.4.1 # Enable WLED LEDs (auto-infers --led-type wled)Windows (PowerShell):
venv\Scripts\python crystal_ball.py --debug # Interactive persona selection
venv\Scripts\python crystal_ball.py --debug voss # Madam Voss by name
venv\Scripts\python crystal_ball.py --debug mordecai # Baron Mordecai
venv\Scripts\python crystal_ball.py --debug voss --max-questions 5 # 5 questions per session
venv\Scripts\python crystal_ball.py --debug voss --model mistral # Use Mistral 7B instead of Llama 3.2
venv\Scripts\python crystal_ball.py --debug voss --length-scale 1.4 # Slower, more dramatic speech
venv\Scripts\python crystal_ball.py --debug voss --wled-host 192.168.4.1 # Enable WLED LEDs (auto-infers --led-type wled)Note: The
--wake-deviceflag usesevdev, which is Linux-only. On Windows, use the default stdin wake trigger (press Enter to wake).
Linux / macOS:
git clone https://github.com/GallionConsulting/halloweenoracle.git
cd halloweenoracle
python3 -m venv venv
source venv/bin/activateWindows (PowerShell):
git clone https://github.com/GallionConsulting/halloweenoracle.git
cd halloweenoracle
python -m venv venv
venv\Scripts\Activate.ps1Linux (Debian/Ubuntu):
sudo apt update
sudo apt install -y ffmpeg portaudio19-dev python3-pipWindows:
# Install ffmpeg (pick one):
winget install Gyan.FFmpeg # via winget
# or: choco install ffmpeg # via Chocolatey
# PortAudio is bundled with the sounddevice Python package on Windows — no separate install needed.
# python3-pip is included with the Python installer from python.org (ensure "Add to PATH" is checked).pip install -r requirements.txtLinux:
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3bWindows:
Download and run the installer from ollama.com/download, then:
ollama pull llama3.2:3bLinux: Add these to /etc/systemd/system/ollama.service under [Service]:
Environment="OLLAMA_VULKAN=1"
Environment="OLLAMA_FLASH_ATTENTION=1"Then reload: sudo systemctl daemon-reload && sudo systemctl restart ollama
Windows (PowerShell — set as persistent user environment variables):
[Environment]::SetEnvironmentVariable("OLLAMA_VULKAN", "1", "User")
[Environment]::SetEnvironmentVariable("OLLAMA_FLASH_ATTENTION", "1", "User")Then restart the Ollama application (quit from the system tray and relaunch, or restart the Ollama service in Task Manager > Services).
Voice model files (.onnx + .onnx.json) are not included in this repo due to their size (~375MB total). Download them into the voices/ directory.
Browse and preview voices: Piper Voice Samples
Download from: Piper Voices on Hugging Face
For each voice, you need both the .onnx model file and its .onnx.json config file. Quick download example:
Linux / macOS:
mkdir -p voices
cd voices
# Default voice for Madam Voss
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/alba/medium/en_GB-alba-medium.onnx
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/alba/medium/en_GB-alba-medium.onnx.json
# Default voice for Baron Mordecai and Captain Barnacle (multi-speaker)
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/semaine/medium/en_GB-semaine-medium.onnx
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/semaine/medium/en_GB-semaine-medium.onnx.json
cd ..Windows (PowerShell):
New-Item -ItemType Directory -Force -Path voices
cd voices
# Default voice for Madam Voss
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/alba/medium/en_GB-alba-medium.onnx" -OutFile "en_GB-alba-medium.onnx"
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/alba/medium/en_GB-alba-medium.onnx.json" -OutFile "en_GB-alba-medium.onnx.json"
# Default voice for Baron Mordecai and Captain Barnacle (multi-speaker)
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/semaine/medium/en_GB-semaine-medium.onnx" -OutFile "en_GB-semaine-medium.onnx"
Invoke-WebRequest -Uri "https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/semaine/medium/en_GB-semaine-medium.onnx.json" -OutFile "en_GB-semaine-medium.onnx.json"
cd ..All recommended voices:
| Voice | Description | Used By |
|---|---|---|
en_GB-alba-medium |
British female, warm | Madam Voss (default) |
en_GB-semaine-medium |
British RP, multi-speaker | Baron Mordecai (speaker 2 "obadiah"), Captain Barnacle (speaker 1 "spike") |
en_GB-jenny_dioco-medium |
British female, expressive | Optional |
en_GB-northern_english_male-medium |
Northern English male, deep | Optional |
en_GB-alan-medium |
British male | Optional |
en_US-lessac-medium |
American, neutral | Optional |
Linux / macOS:
# Validate persona files (fast, no heavy dependencies)
venv/bin/python3 crystal_ball.py --validate-persona voss
# Test all components at once
venv/bin/python3 test_components.py
# Or test individually:
ollama run llama3.2:3b "Say hello in a spooky voice"
echo "The spirits are listening" | piper --model voices/en_GB-alba-medium.onnx --length-scale 1.2 --output-raw | aplay -r 22050 -f S16_LEWindows (PowerShell):
# Validate persona files (fast, no heavy dependencies)
venv\Scripts\python crystal_ball.py --validate-persona voss
# Test all components at once
venv\Scripts\python test_components.py
# Or test individually:
ollama run llama3.2:3b "Say hello in a spooky voice"
echo "The spirits are listening" | piper --model voices/en_GB-alba-medium.onnx --length-scale 1.2 --output_file test.wav
# Then play test.wav with your default media player, or:
# [System.Media.SoundPlayer]::new("test.wav").PlaySync()PERSONA Fortune teller persona name or path to YAML file (omit for interactive selection)
--model MODEL Ollama model (overrides persona default)
--whisper-model MODEL Whisper model size (overrides persona default)
--voice VOICE Piper voice model path (overrides persona default)
--length-scale N Speech speed, higher = slower (overrides persona default)
--sentence-silence N Pause between sentences in seconds (overrides persona default)
--speaker ID Speaker ID for multi-speaker voice models (overrides persona default)
--mic-device ID Microphone device index
--list-devices List audio devices and exit
--debug Show timing, state transitions, and debug info
--led-type TYPE LED controller: wled, serial, dummy, auto (default: dummy)
--wled-host HOST WLED device IP address (auto-infers --led-type wled)
--no-leds Disable LEDs (same as --led-type dummy)
--wake-device PATH evdev input device for wake trigger (e.g. /dev/input/event5)
--list-input-devices List available evdev input devices and exit
--validate-persona NAME Validate a persona file and exit (checks YAML syntax, required fields, types,
message placeholders, voice file, LED config, comma_filter_words; no heavy deps)
--max-questions N Max questions per session before farewell (default: 3)
--silence-timeout SECS Seconds of silence before ending session (default: 20)
--llm-timeout SECS Seconds to wait for LLM before error (default: 45)
--length-scale |
Effect |
|---|---|
0.75 |
Fast (good for testing) |
1.0 |
Normal speed |
1.2 |
Slightly slow (default) |
1.4 |
Dramatic |
1.6 |
Maximum spookiness |
Linux / macOS:
# Compare all downloaded voices
TEXT="The mists swirl... I see a journey in your future."
for voice in voices/*.onnx; do
echo "--- $voice ---"
echo "$TEXT" | piper --model "$voice" --length-scale 1.2 --output-raw | aplay -r 22050 -f S16_LE
doneWindows (PowerShell):
# Compare all downloaded voices
$text = "The mists swirl... I see a journey in your future."
foreach ($voice in Get-ChildItem voices\*.onnx) {
Write-Host "--- $($voice.Name) ---"
echo $text | piper --model $voice.FullName --length-scale 1.2 --output_file test.wav
[System.Media.SoundPlayer]::new("$PWD\test.wav").PlaySync()
}./run.sh --list-devices # Find your mic's index
./run.sh --mic-device 3 # Use that indexThe crystal ball sleeps between visitors and wakes on a physical trigger. Any USB HID device works: keyboard, arcade button, foot pedal, or PIR sensor with a keyboard adapter.
# List available input devices
./run.sh --list-input-devices
# Use a specific device
./run.sh --wake-device /dev/input/event5Without --wake-device, the ball falls back to stdin (press Enter to wake) — useful for development.
Permission: Your user needs read access to /dev/input/eventN. Either add your user to the input group:
sudo usermod -aG input $USER
# Log out and back in for group change to take effectOr create a udev rule for your specific device.
Dependency: The wake trigger uses the evdev Python package (pip install evdev), which is Linux-only. It is only imported when --wake-device is used or --list-input-devices is called.
Windows: The --wake-device and --list-input-devices flags are not available on Windows. Use the default stdin wake trigger (press Enter) instead.
Each visitor session is bounded by two limits to keep the prop moving between visitors:
| Setting | Default | CLI Flag | Persona YAML Key |
|---|---|---|---|
| Max questions per session | 3 | --max-questions |
max_questions |
| Silence timeout (no speech) | 20s | --silence-timeout |
silence_timeout |
| LLM response timeout | 45s | --llm-timeout |
— |
When either limit is reached, the ball speaks a farewell message and returns to the resting state. Conversation history is cleared between sessions.
Persona YAML values are used as defaults and can be overridden by CLI flags.
| Model | RAM | Notes |
|---|---|---|
| llama3.2:3b | ~2GB | Default, fast |
| phi4-mini | ~2.5GB | Strong reasoning for its size |
| gemma3:4b | ~3GB | Good balance of quality and speed |
| mistral | ~4GB | Higher quality |
| qwen3:8b | ~5GB | Strong reasoning & multilingual |
| llama3.1:8b | ~5GB | Best quality, slower |
Personas are defined in YAML files in the personas/ directory. To create a new one, copy an existing file and edit it:
cp personas/voss.yaml personas/witch.yaml
# Edit personas/witch.yaml with your character's details
./run.sh witchYou can also load a persona from any path:
./run.sh /path/to/custom.yamlValidate your persona file before running (checks YAML syntax, required/optional fields, types, message placeholders, voice file existence, LED config structure, and comma_filter_words entries):
./run.sh --validate-persona witchEach persona YAML controls the character's identity, voice, LLM settings, system prompt, filler phrases, LED effects, TTS tuning, and all UI messages. See personas/voss.yaml for the full schema.
The crystal ball supports WS2812B/NeoPixel LED strips via WLED firmware. LEDs change color/effect at each stage of the fortune-telling loop (idle, listening, thinking, dramatic reveal, speaking, goodbye).
By default LEDs are disabled (dummy controller, silent). To enable:
# WLED over WiFi (recommended) — --led-type wled is auto-inferred from --wled-host
./run.sh voss --wled-host 192.168.4.1
# Explicit type (equivalent)
./run.sh voss --led-type wled --wled-host 192.168.4.1
# Auto-detect (probes WLED then serial, falls back to dummy)
./run.sh --led-type auto
# Test LED effects standalone
venv/bin/python3 led_integration.py --type dummy --demoWLED setup:
- Get a WLED-compatible controller (ESP8266/ESP32)
- Flash WLED firmware: https://install.wled.me/
- Connect your LED strip and configure WiFi
- Pass the device IP with
--wled-host
Each persona can define its own LED colors and effects via the optional leds section in the persona YAML:
leds:
brightness: 150 # Default brightness (0-255)
transition_time: 0.5 # Default transition in seconds
colors:
idle: [128, 0, 255] # RGB for each state
listening: [0, 255, 128]
thinking: [0, 100, 255]
speaking: [255, 100, 0]
dramatic: [255, 255, 255]
goodbye: [100, 0, 150]
states:
sleeping:
effect: breathe
brightness: 20
idle:
effect: breathe
brightness: 100
listening:
effect: fairy
brightness: 180
thinking:
effect: aurora
brightness: 200
palette: purple_blue # WLED color palette name
speaking:
effect: candle
brightness: 220
dramatic_reveal:
effect: solid
brightness: 255
transition: 1
goodbye:
effect: dissolve
brightness: 150If the leds section is omitted, built-in defaults (purple/mystical theme) are used. See led_integration.py for the full list of available effects and palettes.
Serial (Arduino/Pico) controllers are also supported via --led-type serial. See led_integration.py for the single-character protocol.
LLM output is automatically cleaned before being sent to the TTS engine:
<think>...</think>blocks are stripped (for models like Qwen that use thinking tags)- Single-word action descriptors like
*grumble*or*chuckle*are removed entirely (multi-word emphasis like*very important*is kept as plain text) - Markdown emphasis (
*bold*,_italic_) is stripped to plain text - Emojis are removed
- Ellipses (
...) are replaced with commas for natural TTS pauses - Double spaces are collapsed
Per-persona comma filtering: Piper TTS inserts a long pause at every comma. For personas with frequent vocative words (e.g. ", matey"), this sounds stilted. Add comma_filter_words to a persona's YAML to remove commas before specific words:
comma_filter_words:
- matey
- ye
- lad
- me heartiesThis turns "Blow me down, matey!" into "Blow me down matey!" for smoother speech. Only affects the persona that defines the list — other personas are unchanged.
- Cloud LLM - See
.planning/cloud_api_example.pyfor Claude and OpenAI API integration examples (requires internet)
- Mini PC: Ryzen 5 / 16GB RAM (or similar)
- Microphone: USB or 3.5mm
- Speakers: Any powered speakers
- Optional: WS2812B LED strip + ESP8266 with WLED firmware
- Optional: Wake trigger — USB arcade button, foot pedal, keyboard, or PIR sensor with keyboard HID adapter
| Problem | Fix |
|---|---|
| Whisper hallucinating on silence | VAD is enabled by default (vad_filter=True) — check mic sensitivity |
| Responses too slow | Enable Vulkan for Ollama; use llama3.2:3b; reduce max_tokens |
| Voice too fast/slow | Adjust --length-scale (higher = slower) |
| Audio not working | Run ./run.sh --list-devices and try --mic-device |
| Ollama connection refused | Run ollama serve; verify with curl http://localhost:11434/api/tags |
| Missing voice models | See Download Voice Models above |
| Wake trigger permission denied | Add user to input group: sudo usermod -aG input $USER then re-login |
| evdev not installed | pip install evdev (only needed for --wake-device, Linux-only) |
| Ollama slow on AMD GPU (Windows) | Set OLLAMA_VULKAN=1 as a user environment variable and restart Ollama |
piper not found (Windows) |
Ensure venv is activated; run as venv\Scripts\piper or python -m piper |
halloweenoracle/
├── README.md
├── run.sh # Launch script (activates venv, runs app)
├── run-claude.sh # Alternative launch script
├── crystal_ball.py # Main application
├── led_integration.py # LED controllers (WLED/serial/dummy)
├── test_components.py # Verify all components work (STT, LLM, TTS, mic, LEDs, etc.)
├── voices/ # Piper TTS voice models (not in repo, see install step 5)
├── personas/ # Persona definitions (YAML)
│ ├── voss.yaml # Madam Voss
│ ├── mordecai.yaml # Baron Mordecai
│ └── barnacle.yaml # Captain Barnacle
└── requirements.txt # Python dependencies
MIT