Repository navigation
Phone as a microphone for PC apps (the reverse of "Phone speaker") #7
Description
The problem
NectarLink can send the phone's audio to the PC ("Phone speaker"), but there is no way to go the other way and use the phone's microphone as an input device for Windows apps (Discord, Zoom, OBS, Teams). Without a headset I have to fall back to a separate app for that, which is exactly the kind of thing NectarLink is meant to replace.
I know this is already on the radar: docs/PLAN.md §2.5 "Phone as Live PC Microphone" and §1.3 step 2 ("live PCM streaming ... so the PC can monitor or route the phone mic in real time"), both noted as needing a virtual audio driver.
Proposal
If a signed driver really is unavoidable, could this be split into two steps?
- Ship the streaming half first: phone microphone → PC as low-latency 48 kHz PCM (
audio.pcm), which §1.3 already calls for. On its own this is useful for live monitoring, recordings, and for users who already run a virtual audio cable (that could be documented). - Then add the virtual endpoint, either as a WHQL-signed driver or as an optional component users download separately, so the main installer stays driver-free.
Question for the maintainer: on current Windows 11, is a user-mode virtual microphone endpoint possible at all, or is a kernel WDM/WaveRT driver still required? If a driver is required, would you accept an optional, separately-downloaded driver component?
Alternatives
- Document a third-party virtual audio cable and stream PCM into it.
- Bluetooth HFP (phone acts as a headset) — poor quality, and it does not work for arbitrary PC apps.
- Keep it out of scope, but then the user guide could say so explicitly next to "Phone speaker" so people know it is not coming.
Activity
thtbee commentedon Oct 16, 2026
Agree with your split. The plan:
Not in the next release, but it's on the list. Thanks for laying it out so clearly.
jochenkirstaetter commentedon Oct 16, 2026
Hi,
I didn't check the details just yet. Does the video stream from phone to PC not include audio? If yes, would this be a potential route to solve the access to the phone microphone? Maybe I'll give it a try and report back on that one.
Going the virtual audio driver is bumpy in regards to getting it into the Windows ecosystem, and macOS and Linux (perhaps less here), if that kind of cross-platform is on the roadmap?
Cheers
thtbee commentedon Oct 16, 2026
Good question. The sound that comes with the screen stream is what the phone plays, not its microphone: Android's playback capture only returns audio that apps play. So the mic needs a stream of its own, which is step 1 of the plan: send it to the PC as low-latency audio.
Showing it to other apps as a microphone is step 2, and that part depends on the OS:
pw-loopbackwithmedia.class = Audio/Source.So Linux may actually be the easiest place to start. Curious what you find.
Phone as microphone: the protocol, the core and the phone
jochenkirstaetter commentedon Oct 18, 2026
Hi @thtbee
Actually yes, it's happening through a virtual input source and I might have it already available in the Linux branch.
However this also needs changes in the app UI and in Android app to provide user access to that functionality. I haven't touched that yet, not to delay the port too much though.
See commit da34f29 (as part of milestone 6) which shall contain the code changes.
Cheers
jochenkirstaetter commentedon Oct 18, 2026
Hey @thtbee
How about that one?
Probably needs better labeling. I'm going to check your commit d3a73b3 - to see what you are up to.
Cheers
thtbee commentedon Oct 18, 2026
That looks great, a real microphone without any driver.
On the label: the description goes through PulseAudio's module arguments, so a value with spaces needs quotes inside the argument, for example
source_properties=device.description='NectarLink Phone Microphone'. The phone's name might be even better, such as "Pixel 9a (NectarLink)", so two phones don't look the same.About d3a73b3: that's the phone side of #7, and 41c70c2 is the PC side. Both are on main and will be in the next release. The PC sends
mic.start, the phone asks its user, then streams 48 kHz mono 16-bit PCM in 10 ms packets (docs/protocol/mic.md). On Windows the PC plays it into an output of your choice, VB-CABLE when it's installed. The PC receives it throughPlatform::mic_sink, so on Linux your pipe source can be that sink. Its format already matches what you load the module with, and the Settings UI and the Android side are done, the part you mentioned you hadn't started.One small thing for the Linux PR: a FIFO at a fixed path in /tmp can be created first by another user on the machine.
$XDG_RUNTIME_DIRis private to your user, and it's where PipeWire keeps its own sockets.jochenkirstaetter commentedon Oct 18, 2026
Hi @thtbee
I might be slightly ahead of you regarding this functionality. See branch
phone-micin my repo.Here are a few aspects that I already have in place.
Architectural / Behavioral Tensions to Be Aware Of:
Phone-Initiated vs. PC-Initiated Start (HomeScreen button vs. docs/protocol/mic.md):
• In d3a73b3's protocol design (mic.rs), the PC holds no special "waiting" gate required to accept an incoming mic/pcm stream — whenever the phone calls core.micOpen(pcId) (as MicService does), the PC's handle_stream accepts mic/pcm, calls platform.mic_sink(peer), and emits NodeEvent::Mic { peer, live: true }.
• That means tapping "Use as mic" on the phone (phone-mic branch) works seamlessly even if the user didn't click Start on the PC first!
What phone-mic b3355b9 Resolved Better / Fixed Over d3a73b3
• In d3a73b3, MicService.streaming was defined as a public StateFlow<String?>, and MicService.stop(context) did not set EXTRA_FROM_USER = true (so calling MicService.stop() from Android UI code wouldn't send mic.stop to the PC!).
• phone-mic added MicService.stopFromUser(context) (which sets EXTRA_FROM_USER = true so core.micStop(pcId) notifies the PC) and wired MicService.streaming into MainActivity.kt:783 and HomeScreen.kt:459-469.
• d3a73b3 added the per-device "mic" toggle check in core/nectarlink-core/src/mic.rs (returning Error::Denied when off), and the PC desktop app even had an error message for it ("The microphone is turned off for this PC on %1"), but d3a73b3 forgot to add the toggle to the Android app, making it impossible for a phone user to actually toggle it!
• phone-mic completed this in Core.kt:1005-1018, CoreState.kt:146-147, and SettingsScreen.kt:504-522, and also stops any active stream immediately if the user flips the toggle off mid-session.
• In d3a73b3, if the user had Android notification permission disabled (POST_NOTIFICATIONS denied) or was already looking at the NectarLink home screen, MicRequests.show() either failed or only posted a system shade notification with no in-app UI, unlike webcamRequest and storageRequestedFrom.
• phone-mic tracks micRequestFrom in CoreState.kt:216-217, clears it when the PC stops/disconnects (onMicStop in PhonePlatform.kt:215-219) or when MicService.start() begins (MicService.kt:77-79), and shows an in-app request card with Start and Not now (which calls dismissMicRequest → micStop(pcId) so the PC stops waiting immediately instead of hanging in "asking" state).
• On Windows (41c70c2), the user must manually install third-party VB-CABLE software, otherwise the phone's mic plays out of the PC's speakers.
• On Linux (phone-mic), phone_mic.rs:149-293 injects and auto-selects NectarLink Phone Mic (Virtual Microphone) via VirtualMic, creating a real PipeWire/PulseAudio source (nectarlink_mic) out of the box.
Hope this helps.
Cheers
jochenkirstaetter commentedon Oct 18, 2026
And on review of 41c70c2 - there are two additional notes.
Subtle Edge Cases / Gaps Found in 41c70c2 (Feedback & Opportunities)
Inspecting 41c70c2's phone_mic.rs alongside mic.rs reveals three subtle behavioral gaps in 41c70c2 that we can also improve if you want:
Gap 1: Declining a PC request on the phone ("Not now") leaves the PC stuck in Phase::Asking
• What happens in 41c70c2 + d3a73b3:
• shared.stop_mic(&peer) does not emit NodeEvent::Mic { device: peer, on: false } because self.mic_stops.remove(peer) is None (stream never opened)!
• Even if NodeEvent::Mic { device, on: false } were emitted, phone_mic.rs:209 in 41c70c2 has: NodeEvent::Mic { device, on: false } if phase() == Phase::Live(*device) => set_phase(Phase::Off), which explicitly ignores Phase::Asking(*device)!
• Result: When the phone user declines with Not now (or turns off the per-PC mic toggle while the PC is asking), the PC UI stays stuck showing "Tap Start on " (Phase::Asking) until the user manually clicks Stop on the PC.
• How to fix cleanly:
Gap 2: Changing the Output device on the PC while Phase::Live does not hot-switch the active stream
• What happens in 41c70c2:
• In phone_mic.rs:147-151, set_output(id: String) updates OUTPUT and saves settings, and the QML text immediately updates to "Live from Pixel, playing into ".
• However, Sink::config(&self, config: MirrorAudioConfig) is only called once when the phone opens the mic/pcm stream. So changing the output dropdown while Phase::Live updates the label in the UI, but audio keeps playing on the old output device until the user stops and restarts the stream!
• If Sink stores the last MirrorAudioConfig (or checks a generation counter / re-initializes when OUTPUT changes), switching outputs in the PC UI while live would hot-swap between VirtualMic and speaker endpoints immediately.
Hope this helps.
Cheers
thtbee commentedon Oct 18, 2026
Thanks for going through it so carefully, Jochen.
Both gaps are real: when the phone declines or ignores the request, the PC stays on "Tap Start on your phone", and changing the output while the mic is live doesn't move the sound. You're also right that there's no request inside the app, so with notifications turned off the user never sees it.
MicService.stopis fine as it is: it's only called when the PC asked to stop, so it shouldn't tell the PC again. The notification's Stop does tell it. What's missing is a Stop inside the app.I'll build these on
mainand credit you:Starting the mic from the phone without the PC asking isn't planned for now: the PC asks first, the same way the webcam works. On Linux, your pipe source as the
MicSinkwould fit right in once #12 is in.jochenkirstaetter commentedon Oct 18, 2026
Hi @thtbee
Oh! "Starting the mic from the phone without the PC asking isn't planned for now: the PC asks first, the same way the webcam works." - let me double-check this one. I was under the impression this could be initiated from the phone to provide the phone's webcam to the PC directly. Maybe I didn't observe this properly. Hence the reason I added same process for offering the phone's microphone to the connected PC.
Weird... I'll check and back to you on this one.
Yes, yes, you're spot on. This is coming on top of #12 only.
Cheers
thtbee commentedon Oct 18, 2026
You're right, Jochen, and I had that wrong. The webcam can be started from the phone's Webcam screen without the PC asking first (
docs/protocol/webcam.md, section 2).So the mic will work the same way. I'll add a "Use as microphone" button on the phone for each PC, along with the other pieces from my last comment. The PC shows it as live with a Stop button, and the per-PC
mictoggle still refuses it when it's off.Sorry for the confusion.
Phone as microphone: a declined request reaches the PC
PC: phone microphone stops waiting, and switches output live
Android: phone microphone requests on Home, Stop, and a per-PC switch
Phone as microphone: start it from the phone, as the webcam can
jochenkirstaetter commentedon Oct 18, 2026
Hi @thtbee
All good, good to discuss these things and keep consistency up in the application(s).
And I'm glad you take feedback like this easily.
Crossing fingers that the rebase onto
phone-micbranch is going smooth. 😆Cheers
jochenkirstaetter commentedon Oct 24, 2026
Hi @thtbee
Now, with the Linux port being merged this feature is partially available.
Any updates on this in terms of Windows or macOS support?
Cheers