Notetaker + NotebookLLM

A local-first, bilingual (中文 / English) AI meeting notetaker for macOS — with a private, self-hosted NotebookLM-style knowledge layer over all your meetings.

It listens to your online meetings in the background — Google Meet, Zoom, 腾讯会议, any app — produces a live transcript and a running summary + action items, saves polished notes, and then lets you ask questions across every meeting you've ever had, with cited answers.

Audio is transcribed entirely on-device and never leaves your Mac; only transcript text ever reaches the cloud. Works across every meeting app because it taps system audio — no bot joins the call.

✨ The headline feature: ask your meetings anything

Each finished meeting is automatically pushed into a local open-notebook instance (a self-hosted NotebookLM) and embedded with a local multilingual model. Then, from the menu bar's Ask my meetings, you ask things like "上个月关于定价我们定了什么?" and get a synthesized answer with citations back to the exact meeting — all running on your machine. See Knowledge layer below.

How it works

Layer	What	Where it runs
Ears — speech → text	sherpa-onnx streaming Paraformer (bilingual zh-en, int8)	On-device. ¥0/min. Audio never leaves your Mac.
Brain — text → notes & answers	DeepSeek (summary, action items, Q&A)	Cloud. Only transcript text is sent — never audio.
Memory — search & RAG	open-notebook + local bge-m3 embeddings	Self-hosted (Docker). Cross-meeting Q&A with citations.

One local Paraformer model handles Chinese, English, and code-switching with no language toggling; DeepSeek turns transcripts into notes for pennies per meeting; open-notebook makes every meeting searchable. See docs/plans/ for the full design and rationale.

Features

• 🎙️ Background dual-track capture (you vs. others) across any meeting app, via system-audio loopback
• 🌏 Bilingual on-device transcription (zh/en + code-switching), int8 for low memory, with punctuation
• 🪄 Live panel — watch the transcript + a running summary update during the call
• 📝 Auto summary + notes saved as markdown on stop (DeepSeek)
• 🔎 Ask my meetings — cross-meeting Q&A and semantic search with citations (open-notebook)
• 🛟 Preflight check — warns before recording if capture won't actually work
• ✍️ Editable corrections — fix recurring name/term mis-transcriptions
• 🖱️ One-click menu-bar app, open-at-login

The menu-bar items in detail: Start / Stop a meeting; Show panel (live transcript, during a meeting); Ask my meetings (Q&A page anytime); Sync to notebook (push any unsynced notes); Quit. Feedback is delivered via macOS notifications — allow them for AI Notetaker in System Settings → Notifications.

Implementation status / history

The thin end-to-end loop works: menu-bar Start → background dual-track capture → on-device transcription → Stop → DeepSeek summary → a saved markdown notes file. Transcription uses the int8 model (lower memory) with punctuation restored.

Also: a preflight check warns before a meeting if capture won't work; a live panel (menu → Show panel) shows the running transcript + summary in the browser; a user-editable corrections.toml fixes recurring name/term mis-transcriptions; and an optional open-notebook knowledge layer gives cross-meeting Q&A / search (see below).

Requirements

• macOS (Apple Silicon recommended)
• Python 3.11+
• BlackHole 2ch virtual audio driver — used to capture meeting audio (see Audio setup below)
• A DeepSeek API key
• (For the knowledge layer) Docker + Ollama — see Knowledge layer

Note: this started as a personal tool; the launcher and runbook assume the repo lives at ~/ai-notetaker. Adjust paths if you clone elsewhere.

Audio setup (one-time)

The app captures the meeting through a virtual loopback so it works with any meeting app. Set it up once:

1. Install BlackHole 2ch: brew install blackhole-2ch (then log out/in, or reboot).
2. Open Audio MIDI Setup → + → Create Multi-Output Device.
3. In the Multi-Output Device, check both your speakers/headphones and BlackHole 2ch. (This lets you still hear the meeting while it is routed to BlackHole.)
4. Set that Multi-Output Device as your Mac's system output (Sound settings, or the menu bar).
5. Leave your microphone as the normal input.

The app reads your mic as you and BlackHole as others. If BlackHole isn't found, Start shows an error pointing back here.

Setup

# 1. Create a virtual environment and install
python3 -m venv .venv
source .venv/bin/activate
pip install -e .

# 2. Download the on-device ASR + punctuation models (~once, into ./models)
python scripts/download_models.py

# 3. Configure
cp config.example.toml config.toml
#   then edit config.toml and set your DeepSeek key,
#   or export it:  export DEEPSEEK_API_KEY=sk-...

# 4. Run the menu-bar app
ai-notetaker            # or: python -m notetaker.app

Development

pip install -e ".[dev]"
pytest

Knowledge layer — open-notebook (optional)

Turn your meetings into a searchable, ask-anything knowledge base (a local NotebookLM):

1. Stand up open-notebook locally — see integrations/open-notebook/README.md (docker compose up -d, then configure DeepSeek chat + local bge-m3 embeddings in its web UI).
2. In config.toml, set [opennotebook] enabled = true and the password (from integrations/open-notebook/.env).
3. Each meeting now auto-pushes to open-notebook on Stop. Backfill old notes with python scripts/backfill_notebook.py, or use the menu's Sync to notebook.
4. Open open-notebook's web UI (127.0.0.1:8502) to ask cross-meeting questions with citations, search semantically, and generate audio overviews.

The notetaker only talks to open-notebook's REST API — it does not embed its code.

Privacy & cost

• Audio stays on your Mac. Transcription is fully on-device.
• Only text leaves the machine, to DeepSeek, costing pennies per meeting.
• A planned follow-up lets you route summaries to a local LLM (e.g. your own Hermes server) so even the text stays local. See the plan's Future Direction section.