Audio output redirection in Realtime OpenAI conversation

[ana-fillthegapp]

Hey there!

First of all, congrats for the great work with this SDK ☺️

I was wondering if you guys are planning to implement the audio output redirection when the user is talking in Realtime with the AI and suddenly headphones are connected to the device and viceversa.
Audio is a bit tricky in iOS 🥲

Thanks in advance!

[lzell]

While I would love to add that, it's unfortunately down very far on the list. I'm happy to take a PR if a community member wants to give it a shot!

I won't lie though, the work is tricky to get right with multi-platform (e.g. working on macOS, iOS, watchOS)

It took a long time to make the existing realtime audio work equally well on multiple platforms, and such that the model wouldn't hear itself if no headphones were in use. So I'm reluctant to sink a bunch more time into it at the moment

[ana-fillthegapp]

I hear you... 😅 Thanks a lot for answering!

[richarddas]

FWIW I'd consider this a downstream (client) responsibility, as in: AIProxy covers getting requests over the wire, and audio playback. Was the suggestion that this be baked into AIP because it owns the audio interface at this point, and there's no way to build your own support on top of it?