A voice-first iOS AI assistant built on the ElevenLabs Conversational AI agent platform, extended with:
- Persistent memory across sessions via a lightweight Notion-based summary architecture
- Custom agentic tools for workspace actions: semantic search, page read/edit, task tracker
- In-conversation web search via OpenAI
- Post-call transcript archiving with AI-generated summaries via AWS Bedrock
- Pipedream-orchestrated webhooks for every tool call — no backend to run
- Post-call continuation in text via AWS Bedrock (Claude Opus/Sonnet/Haiku) using the voice transcript as message history
Built as a hands-on exploration of the ElevenLabs agent SDK and custom-tool system.
ElevenLabs conversational agents are stateless between sessions by default. Each call starts with no knowledge of the previous one. For a voice agent you talk to daily — on walks, between meetings, while thinking out loud — that's the single biggest UX gap.
The fix is architectural, not model-level: store a compact summary of each conversation in Notion at hang-up, and retrieve the latest N summaries at the start of the next call. The agent gets real continuity, the user's data stays in their own workspace, and the whole thing runs on Pipedream workflows instead of a dedicated backend.
???????????????????????????????
? React Native iOS app ?
? (Expo Router + ElevenLabs ?
? React Native SDK) ?
???????????????????????????????
? WebRTC (LiveKit under the hood)
?
???????????????????????????????
? ElevenLabs Conversational ?
? AI Agent (Claude Sonnet) ?
? + custom tools via webhook ?
???????????????????????????????
? webhook POST
?
???????????????????????????????
? Pipedream workflows ?
? (get_memory, save_memory, ?
? search_notion, read_page, ?
? add_task, search_web, …) ?
???????????????????????????????
? Notion API / OpenAI API
?
???????????????????????????????
? User's Notion workspace ?
? (memory DB, task DB, ?
? project pages) ?
???????????????????????????????
After a call ends, the transcript is also available for text-mode continuation via AWS Bedrock — the same conversation, different medium.
See ARCHITECTURE.md for the deeper design notes (why Pipedream vs. own backend, why Notion for memory, tool-call latency trade-offs).
.
├── README.md ← this file
├── ARCHITECTURE.md ← design notes
├── SETUP.md ← step-by-step deploy guide
├── TROUBLESHOOTING.md ← common issues and fixes
├── .env.example ← root-level env vars reference
│
├── voice-agent-app/ ← React Native iOS app (Expo)
│ ├── app/ ← Expo Router screens
│ ├── components/ ← UI components
│ ├── hooks/ ← useVoiceCall (ElevenLabs session)
│ ├── lib/ ← storage, Bedrock chat, transcript sync
│ └── …
│
├── elevenlabs-config/
│ ├── system_prompt.template.md
│ └── tools.template.json ← webhook URLs as placeholders
│
└── pipedream-workflows/
├── README.md
├── search_notion.js ← the main non-trivial workflow (scored 2-layer search)
├── post_call_save.md ← saves full transcripts with AI summaries
└── *.md ← per-workflow setup notes
In: all the source code and architecture needed to deploy your own instance.
Not in: any single reference to a specific user, Notion database ID, webhook URL, conversation transcript, or deployed model ARN. Everything is templated. You bring your own Notion workspace, Pipedream account, ElevenLabs agent, and (optionally) AWS Bedrock inference profile.
See SETUP.md to deploy your own instance end-to-end (~1€“2 hours, mostly Pipedream wiring).
This is a reference implementation / architectural demo, not a maintained product. The iOS app builds and runs; the Pipedream workflows are deployable; the ElevenLabs agent configuration is complete. Use as a starting point for your own voice agent deployments.
MIT. See LICENSE.