Your screen, explained.
Ask about anything on your screen. Buddy explains it out loud and draws on it while it talks.
Learn something new, get through an app, fix an error, or check your own work, from your Mac's notch.
projects-hacks.github.io/buddy
A real answer, sped up. The words under the notch are what Buddy is saying.
Hold Control + Option and ask, or type. Buddy answers out loud and draws on your screen while it talks. To cut in, just say "Buddy, wait…".
|
LEARN Explains anything on your screen Code, a paper, a dashboard, a lecture video. It rings what it is talking about, one sentence at a time, and on LeetCode gives a hint before the answer. Try: "why does this crash on the last element?" |
LEARN Teaches a topic from scratch, then checks you The screen turns into a whiteboard and Buddy teaches it part by part. Explain it back and it marks what you missed. Try: "teach me TCP from scratch" |
|
GET IT DONE Walks you through any app One step at a time, pointing at each button, and waits while you click. Ask again later and it replays from memory. Try: "how do I export this as a PDF?" |
FIX Explains the error you just hit An error in a terminal or editor makes the notch glow. One tap and Buddy rings the line that caused it. Try: tap Control + Option when it glows |
|
REVIEW Reviews your own work In Excalidraw, draw.io, Miro or Freeform it marks your boxes and sketches what is missing beside them. Try: "what am I missing in this design?" |
REMEMBER Remembers what you learn Every lesson becomes a notebook page, and a map shows which ideas are fading. All on your Mac. Try: open Map from the notch |
 |
 |
| Taught from scratch on a whiteboard | Your explanation back, gaps marked in red |
 |
 |
| Every lesson, a notebook page | A map of what you know |
Each question goes to the model measured best for it, Claude Sonnet, Opus or Gemini Argon. A question costs one to six cents; hover Spent today in Settings to see where it went.
The download window shows these same steps. macOS asks for Open Anyway once, because Buddy is not notarized. Setup then asks for your name and keys and walks through the permissions it needs; after that Buddy updates itself.
Or with Homebrew:
brew tap projects-hacks/buddy https://github.com/projects-hacks/buddy
brew trust projects-hacks/buddy
brew install --cask projects-hacks/buddy/buddyStep by step, with troubleshooting: docs/install.md.
| Keys | Does |
|---|---|
| Hold Control + Option | Ask out loud; your words show under the notch |
| Control + Option + T | Type a question; Return sends, Shift + Return adds a line |
| Say "Buddy, wait…" while it talks | Cut in with a question, or "Buddy, hold on", "go on", "Buddy, slower"; anything said without its name is ignored |
| Hover the notch | The last lesson with Replay, the day's chat, and Notebook, Map and Settings in its top row |
| Hover the glow, then Explain this error (or tap Control + Option) | When the notch glows red and says "An error on screen.", explain the error that just appeared |
| Esc | Clear the drawing, or stop a walkthrough |
Everything else is in the guide: How to use Buddy.
flowchart TD
ask([Spoken or typed question]):::input --> prep
subgraph prep [On the Mac, in about 0.7 s]
capture[Screen capture and text index]:::device
sort[Question sorter]:::device
recall[Learning memory]:::device
end
prep --> route[Router]:::logic --> model{{Sonnet, Opus or Argon}}:::cloud
model -->|drawing program, streamed| play[Voice and pen, sentence by sentence]:::device
play --> you([You hear and see it]):::input
classDef input fill:#fde68a,stroke:#b45309,color:#111
classDef device fill:#bbf7d0,stroke:#15803d,color:#111
classDef logic fill:#bfdbfe,stroke:#1d4ed8,color:#111
classDef cloud fill:#e9d5ff,stroke:#7e22ce,color:#111
style prep fill:#f0fdf4,stroke:#15803d
The model answers with a short drawing program, and the first sentence is spoken while the rest is still being written. Details in docs/how-it-works.md.
| What | Where it goes |
|---|---|
| Screen | Captured whole only while you hold the keys or ask, and dropped if no question follows, or after a click while a walkthrough waits; those checks are compared on the Mac and discarded |
| Errors | While a terminal, editor, Xcode or a coding judge's page is in front, after Return, a run shortcut or a click, the front window alone is read on the Mac (its accessibility text, or text recognition on the part that changed) to spot an error. Nothing is stored; fingerprints of errors already seen stay in memory until Buddy quits, and nothing is sent until you accept the offer |
| Voice | The microphone is open while the keys are held and, so you can talk over it, while an answer plays; then only words said to Buddy by name are acted on. A small mic beside "explaining" in the notch shows it is open, Esc or Stop ends the answer and closes it, and with a Bluetooth headset as the microphone it listens through the Mac's own microphone instead, so the headset is never switched to call quality. Heard by Deepgram when a Deepgram key is set; otherwise transcribed on the Mac |
| Teach it back | After you ask to explain a topic ("let me explain X back to you"), the microphone stays open hands free until you are done: you say "I'm done", press Return, go quiet for eight seconds, or three minutes pass. The red meter and "Listening" show the whole time, and Esc or Cancel throws it away. Heard like any question (Deepgram with a key, on the Mac without); what you said goes to the model that answers board turns (Sonnet first with an Anthropic key) to be drawn and checked, and is kept on the board's Notebook page and, as every answer is, in the diagnostic log |
| Lectures | When you ask while a video is in front, the minute before the playback time is read from the page's transcript or YouTube's captions (only the video's id goes to YouTube). Where there is none, and once you allow System Audio Recording, the front app's own sound is turned into words on your Mac while it plays, never sent anywhere for this, filed by the video's playback time so a question at any point you watched gets the minute before it. The words stay in memory for that app while Buddy runs, and the last 90 seconds of sound until you switch apps (a small screen glyph shows in the notch while sound is kept), never on disk. They are used only when you ask about the video, meaning a player's time is on screen or you ask what was said, and keep the video's voice from being taken for yours. Only when too little was heard on your Mac, as right after a video starts, or your Mac cannot recognise speech itself, is the newest 65 seconds of sound sent to be transcribed when you ask (by Deepgram with a key). For a browser that is all its tabs. Never the microphone; never calls, meetings or music apps, and any app using a microphone counts as a call |
| Sent | The screenshot and question, to the model that answers, with the lecture's last minute when there is one |
| Kept | Transcripts, memory and how-tos in ~/Library/Application Support/Buddy; keys in the Keychain |
| Notebook | Each finished lesson as a page in ~/Library/Application Support/Buddy/Notebook: the question, what was said, the cards drawn, and a small JPEG of only the part of the screen that was marked, cut from that question's screenshot (never the whole screen). A board lesson or a teach-back is one page when you leave the board, the whole board kept as one drawing with what was said. About 70 to 150 KB a page, the picture cut from the display's own pixels so code stays sharp, under 2 KB for an answer without a drawing. A cancelled question keeps nothing. Never uploaded; the Notebook window shows each page's size, deletes a page, a day or everything, and can stop keeping new lessons |
| Map | Drawn on the Mac from memory. The first time it opens, ideas with no links on record yet (memory from before 2.0) have only their names and one-sentence summaries sent to Claude Haiku, 20 ideas a call, once, to find how they relate; no lesson, question or screen goes with them |
| Analytics | None |
Building from source
Needs Xcode 16 and a self-signed code signing certificate named Clicky Local Dev (Keychain Access > Certificate Assistant > Create a Certificate, Self Signed Root, Code Signing). The name predates Buddy and has to stay: installed copies only accept updates signed with it.
git clone https://github.com/projects-hacks/buddy.git
cd buddy
zsh scripts/build-local.shLogs go to ~/Library/Logs/Buddy.log. Releases: docs/releasing.md. Source map: AGENTS.md. Tests: scripts/.
Started as a fork of Clicky by Farza (MIT). Drawing uses rough.js and perfect-freehand (MIT). Licensed under MIT.