A sophisticated AI agent built with Google's Agent Development Kit (ADK) that creates personalized music experiences based on your emotional state. The agent captures your voice, analyzes your mood, generates custom music using Suno AI, and synchronizes smart bulbs with the music's wavelengths.
- 🎤 Voice Input: Captures audio from your microphone
- 🧠 Mood Detection: Uses Gemini's multimodal AI to analyze sentiment from speech
- 🎵 Music Generation: Creates custom music matching your mood via Suno AI
- 🎧 Audio Playback: Plays generated music automatically
- 🌈 Smart Lighting: Syncs bulb colors with music frequencies
- 🤖 Autonomous Orchestration: Agent handles the entire workflow intelligently
The system uses Google ADK's agent-based architecture with custom tools:
┌─────────────────────────────────────────────────┐
│ Mood Orchestrator Agent │
│ (Gemini 2.0 Flash) │
│ │
│ Orchestrates the entire workflow: │
│ 1. Capture voice input from microphone │
│ 2. Transcribe and analyze mood from speech │
│ 3. Generate music based on detected mood │
│ 4. Play music and analyze audio │
│ 5. Control bulbs based on wavelengths │
└─────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────┐
│ Custom Tools │
├─────────────────────────────┤
│ • record_voice_input │
│ • transcribe_and_analyze │
│ • generate_music │
│ • play_music │
│ • analyze_audio_wavelengths │
│ • control_bulb_colors │
└─────────────────────────────┘
- Python 3.10 or higher
- Google Gemini API key (Get it here)
- Suno AI API key (Get it here)
- Microphone for voice input
- (Optional) Smart bulb server running on localhost:3000
-
Clone or navigate to the project directory:
cd moodagent -
Install dependencies (using uv):
uv sync
-
Set up environment variables:
cp .env.example .env
Edit
.envand add your API keys:GEMINI_API_KEY=your_gemini_api_key_here SUNO_API_KEY=your_suno_api_key_here BULB_SERVER_URL=http://localhost:3000Getting API Keys:
- Gemini API: Visit Google AI Studio to get your free API key
- Suno AI API: Visit SunoAPI.org to obtain your API key
Run the agent:
uv run python main.py🎵 MOOD ORCHESTRATOR AGENT 🎵
============================================================
Welcome! I'll capture your voice, detect your mood,
generate personalized music, and sync your smart lights!
============================================================
🎤 Ready to start! The agent will guide you through the process.
Type 'start' to begin, or 'quit' to exit.
You: start
🤖 Agent:
------------------------------------------------------------
I'll help you create a personalized music experience! Let me start by
recording your voice to understand your mood.
🔧 Executing tool: record_voice_input
🎤 Recording for 5 seconds... Speak now!
✓ Recording complete!
🔧 Executing tool: transcribe_and_analyze_mood
📝 Transcription: I'm feeling really happy and energetic today!
😊 Detected Mood: energetic (confidence: 0.95)
💭 Reasoning: High vocal energy, upbeat tone, positive words
Great! I detected you're feeling energetic! Let me generate some
upbeat music for you...
🔧 Executing tool: generate_music
🎵 Generating music for mood: energetic
⏳ Task created. Waiting for generation...
✓ Music generated successfully!
🔧 Executing tool: play_music
🎵 Now playing: generated_music_energetic.mp3
🔧 Executing tool: analyze_audio_wavelengths
🎼 Dominant frequencies: 523.3Hz, 1046.5Hz, 2093.0Hz
🔧 Executing tool: control_bulb_colors
💡 Setting bulb color: RGB(128, 64, 255) for 523.3Hz
Perfect! I've created energetic music for you and synced your lights
with the music's rhythm. Enjoy! 🎵
------------------------------------------------------------
- Captures audio from microphone
- Duration: 5 seconds (configurable)
- Output: WAV file
- Transcribes speech using Gemini
- Analyzes mood from vocal tone and content
- Classifies: happy, sad, calm, energetic, angry
- Returns confidence score
- Uses Suno AI API (api.sunoapi.org)
- Non-custom mode for simplicity
- Maps mood to music style
- Model: V4_5ALL (better song structure, max 8 min)
- Polls for completion (up to 4 minutes)
- Returns 2 songs per request (uses first one)
- Downloads MP3 file
- Stream URL available in 30-40 seconds
- Full quality download in 2-3 minutes
- Plays audio using pygame
- Non-blocking playback
- FFT analysis of audio
- Extracts dominant frequencies
- Returns frequency spectrum
- Maps frequencies to RGB colors
- Sends HTTP requests to bulb server
- Graceful fallback if server unavailable
Edit config.py to customize:
- Audio settings: Sample rate, channels, recording duration
- Mood mappings: Customize music styles for each mood
- Bulb server: Change URL or endpoint
- Suno API: Model version (V4_5ALL - better song structure, max 8 min)
moodagent/
├── main.py # Main application with ADK runner
├── agent.py # Agent definition and instructions
├── config.py # Configuration and constants
├── tools/ # Custom ADK tools
│ ├── voice_tool.py # Voice recording
│ ├── mood_tool.py # Mood analysis
│ ├── music_tool.py # Music generation
│ ├── audio_tool.py # Playback and analysis
│ └── bulb_tool.py # Smart bulb control
├── utils/ # Utility functions
│ └── helpers.py # File operations
├── temp/ # Temporary files
├── pyproject.toml # Dependencies
├── .env # Environment variables (create from .env.example)
└── README.md # This file
- google-adk: Agent Development Kit
- google-genai: Gemini API for multimodal AI
- requests: HTTP client for Suno API
- sounddevice: Microphone recording
- scipy: Audio file I/O
- pygame: Audio playback
- numpy: Audio analysis
- python-dotenv: Environment variables
- pydantic: Type validation
- Ensure microphone permissions are granted
- Check default audio device settings
- Try adjusting
AUDIO_SAMPLE_RATEin config.py
- Verify API keys are correct in
.env - Check API quota/limits
- Ensure internet connection
- The agent works without bulb server (graceful fallback)
- Adjust endpoint in
bulb_tool.pyto match your server API
- Real-time audio streaming for live mood detection
- Multiple music generation options
- Advanced audio-to-color algorithms
- Support for multiple bulb systems
- Web UI for easier interaction
- Music library and history
MIT License
Built with: