Add audio device control: stop/start/state APIs with isolate offloading - #504
Add audio device control: stop/start/state APIs with isolate offloading#504Colton127 wants to merge 12 commits into
Conversation
Match waveform sources to the engine sample rate
fix: Waveform audio sources do not match engine sample rate
On Android, soloud_miniaudio_pause() was compiled to a no-op, so the miniaudio device (and the audioserver AudioMix partial wakelock it holds, attributed to the app's UID) stayed alive even when all voices were paused. This counts against Google Play's excessive-partial-wake-locks metric (alnitak#250). Add a runtime flag (default false) that lets the Android branch of soloud_miniaudio_pause() stop the device when idle, like every other native platform. It stays off by default to preserve the historical behavior that avoids the rare stale-buffer glitch on rapid stop->play (alnitak#446). - src/soloud/.../soloud_miniaudio.cpp: add atomic gAndroidPauseDeviceWhenIdle flag + miniaudio_setAndroidPauseDeviceWhenIdle setter; un-gate the Android pause branch behind it (Web branch unchanged). - src/soloud/include/soloud_internal.h: declare the new setter. - src/bindings.cpp + src/ffi_gen_tmp.h: FFI export mirroring setAndroidAAudioAttributes. - Dart bindings (abstract/FFI/web) + public SoLoud.setAndroidPauseDeviceWhenIdle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mf2uYriAipRTFF5zCXDMDw
Previously the setter only stored the backend flag, so enabling it while the engine was already idle left the device (and its AudioMix wakelock) running until the next pause/stop event. Route the FFI export through a new Player::setAndroidPauseDeviceWhenIdle passthrough that also acts on the current runtime state: - enable + no active voices -> schedule the deferred device stop now via pauseEngine(), reusing the same ~500 ms coalescing delay as a normal idle-pause (so the delay is preserved). - disable -> restart the device immediately if a prior idle-pause stopped it (resume() is a no-op when already running). bindings.cpp now calls the Player passthrough instead of the backend directly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mf2uYriAipRTFF5zCXDMDw
Expose a way to stop and restart the miniaudio output device without
tearing down the engine. Stopping only calls ma_device_stop() on the
existing device; SoLoud is not deinitialised and its loaded sources,
active voices and initialized state are all preserved, so startAudioDevice()
resumes rendering exactly where it left off.
Unlike the idle-pause path, the stop is unconditional: it runs regardless
of platform idle policy or whether voices are actively playing (needed for
testing). Both methods are idempotent via ma_device_get_state().
- src/soloud/.../soloud_miniaudio.cpp: miniaudio_stopAudioDevice()/
miniaudio_startAudioDevice() using ma_device_stop/start with state checks.
- src/soloud/include/soloud_internal.h: declarations.
- src/player.{h,cpp}: Player::stopAudioDevice()/startAudioDevice() with the
usual mInited/PlayerErrors handling.
- src/bindings.cpp + src/ffi_gen_tmp.h: FFI exports mirroring changeDevice.
- Dart bindings (abstract/FFI/web + js_extension) and public
SoLoud.stopAudioDevice()/startAudioDevice().
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D7UTvguQ31vuiRqcPeMBTC
The native startAudioDevice()/stopAudioDevice() FFI calls were invoked synchronously on the main (UI) isolate. ma_device_start() blocks for tens of milliseconds while the OS restarts the device (~90ms on Android/AAudio), freezing the entire app for that window. Run the blocking native call on a background isolate via Isolate.run. A raw C function pointer address is a plain int and is sendable across isolates, so the already-resolved pointer's address is passed to the worker, the pointer rebuilt there, and the function called off the UI isolate. No native changes are needed: miniaudio serializes device start/stop internally and the returned future still completes only once the device has actually started/stopped (including the idempotent no-op). - bindings_player.dart: methods are now Future<PlayerErrors>. - bindings_player_ffi.dart: run via Isolate.run using the pointer address. - bindings_player_web.dart: async wrappers over the synchronous wasm calls (web is single-threaded; the device change is instant). - soloud.dart: await the now-async binding calls. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D7UTvguQ31vuiRqcPeMBTC
Add opt-in Android idle device-pause (setAndroidPauseDeviceWhenIdle) Fixes frame jank during audio device start and stop Expose manual control over audio device lifecycle
Expose a public `getAudioDeviceState()` that returns an `AudioDeviceState` enum reflecting the current state of the audio output device. The native implementation reads miniaudio's `ma_device_get_state(&gDevice)` and the enum values mirror `ma_device_state` (uninitialized, stopped, started, starting, stopping). Wired through the full stack: the miniaudio backend, Player, the FFI export, and the Dart bindings (FFI + web), surfaced as a synchronous `SoLoud.getAudioDeviceState()`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ESvHS7kXkJ7omwTmJanUfR
…-i1aqkt Add getAudioDeviceState() to query the audio device state
SoLoud.init() runs the native initEngine() FFI call synchronously on the Dart main (UI) isolate. On Android the heavy AAudio device init/start blocks for seconds on low/mid-range devices, tripping the ANR watchdog (alnitak#481). Extend the off-thread pattern already used for start/stopAudioDevice(): a raw C function pointer address is a plain int and is sendable across isolates, so the resolved pointer's address (plus the primitive args) is passed into Isolate.run, the pointer rebuilt in the worker, and the native call executed off the UI isolate. All Dart state and NativeCallable registration stay on the main isolate, which is why this works where wrapping the whole init() in an isolate does not. miniaudio's AAudio/OpenSL init needs no JNI, so running the C call on a worker thread is safe on Android; init_deinit_mutex serializes it. init() is already Future<void>, so this is not a public API break. deinit() stays synchronous (its synchronous contract is tested and used at app exit); a new non-blocking deinitAsync() is added that runs only the native teardown off-thread while keeping the isolate-bound callable disposal on the main isolate. - bindings_player.dart: initEngine -> Future<PlayerErrors>; add deinitAsync(). - bindings_player_ffi.dart: initEngine/deinitAsync run via Isolate.run using the function pointer address; add worker entrypoints. - bindings_player_web.dart: async wrappers over the synchronous wasm calls. - soloud.dart: await initEngine; add deinitAsync(); factor shared deinit steps. Fixes alnitak#481. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D7UTvguQ31vuiRqcPeMBTC
|
@alnitak This is still a work in progress. Running initEngine in an isolate is the right call and should have no tangible impact on performance/latency, it just prevents the UI thread from freezing. I encountered some trouble with the miniaudio's audio device lifecycle. Keeping the device active (outputting silence even when no sound is playing) is essential for low-latency sound playback. Otherwise, there is a measurable delay to begin playback. However, keeping the audio device active even when idle introduces two major problems:
The fix to 2 solves the first, My preferred solution is probably:
stopAudioDevice/startAudioDevice as well as getAudioDeviceState is added weight to the API but genuinely helpful. In fact it's a necessity for me as I need to init soloud and create audio sources and handles before playing anything. |
|
@Colton127, thanks for your effort on this PR. The changes look good to me. Also, your PR 10 looks good and I think it deserves to be included (here on in another PR). My concern from the beginning has been to keep the engine as simple as possible for the user, but depending on the platform, that isn't always feasible. For the same reason, I'd also like to keep the documentation as simple (and hopefully less scary 😄) as possible. Also, just let me know when this PR is no longer a Work In Progress, and I'll be happy to give it a final review and merge it. |
|
@alnitak I appreciate the prompt response! Colton127#10 is the way to go. It replaces the need for How should the existing android lifecycle be handled?
I'm learning toward 2 because it guarantees consistent, low latency across all platforms. The new API's in the PR allow full control over the lifecycle if needed. |
|
@Colton127 thanks a lot for all this! I am not very familiar with the audio behavior on mobile, so your effort is very welcome! The "Pause the audio device after 500ms of inactivity" is a workaround I did without noticing the delay when starting the device again. So it's worth considering pausing the audio device after 500ms of inactivity only when the app is in the background? But I don't think the app lifecycle can be monitored from the plugin (in Dart), and maybe this should be made using method channels. Of course, your |
|
closed; replaced with #508 |
3 participants
Description
This PR adds new APIs for controlling miniaudio's audio output device:
stopAudioDevice()/startAudioDevice(): Stop and restart the audio device while preserving loaded sounds, active voices, and engine state. Both are idempotent and run off the UI thread via isolates to prevent app freezing.getAudioDeviceState(): Query the current device state (uninitialized, stopped, started, starting, stopping). Synchronous and cheap since it's just an atomic read.deinitAsync(): Async variant ofdeinit()that runs the blocking native teardown off the UI thread, complementing the existing synchronousdeinit()for contexts where awaiting is possible.setAndroidPauseDeviceWhenIdle(): Lets the Android implementation stop the audio device when idle, mirroring every other platform. Defaults to false to preserve the fix for fix: small sound bleed after stop and play #446Additionally,
initEngine()is now async and runs off the UI thread via isolates, fixing ANR (Application Not Responding) issues on Android when device initialization takes several seconds (#481).Key Implementation Details
initEngine,startAudioDevice,stopAudioDevice,deinitAsync) are wrapped in top-level functions (_invokeInitEngine,_invokeDeviceLifecycle,_invokeVoidNative) that can run inIsolate.run()workers. Only sendable primitives (ints) cross the isolate boundary; native function pointers are reconstructed inside the worker.AudioDeviceStatemirrors miniaudio'sma_device_statefor type-safe device state reporting.miniaudio_stopAudioDevice,miniaudio_startAudioDevice,miniaudio_getAudioDeviceState) and Player methods expose device lifecycle control._predeinit()and_postdeinit()extracted to avoid duplication betweendeinit()anddeinitAsync().Type of Change
initEngine()signature changed from sync to async)Breaking Changes
initEngine()now returnsFuture<PlayerErrors>instead ofPlayerErrors. Callers must await the result.