QVAC-23201 feat[api]: integrate @qvac/audiogen-ggml 0.2.1 into the SDK

[RamazTs]

🎯 What problem does this PR solve?

• @qvac/audiogen-ggml 0.2.1 (QVAC-22886 feat[audiogen-ggml]: add reference and cover audio inputs #3782, QVAC-23306 feat[audiogen-ggml]: align generation with official ACE-Step #3743, QVAC-23558: fix[audiogen] harden package lifecycle and distribution #3822, Bump tts-ggml to 0.7.0 and audiogen-ggml to 0.2.1 #3877) added reference/source audio inputs, the cover-nofsq task, LM sampling + Haar DCW controls, and official mobile prebuilds (Android Vulkan/CPU, iOS CPU), but the SDK still pinned ^0.1.0 and only forwarded the original caption/lyrics/musical controls.
• SDK users could not generate music from a reference song (timbre conditioning) or re-render an existing track as a cover through audioGen().

📝 How does it solve it?

• Bumps @qvac/audiogen-ggml to ^0.2.1.
• audioGen() gains referenceAudio / sourceAudio, taskType (text2music | cover-nofsq), audioCoverStrength, coverNoiseStrength, LM sampling (lmTemperature, lmTopP, lmTopK, lmCfgScale, lmPhase1) and DCW (dcwEnabled, dcwScaler, dcwHighScaler) controls; the plugin forwards them 1:1 to AudioGen.run().
• Audio inputs accept a file path or in-memory bytes (same shape as transcribe's audioChunk): paths with a decodable extension are decoded server-side through the existing FFmpeg decoder (now with a 48 kHz output option) and duplicated onto both channels; other paths (streamed asynchronously) and buffers are read as raw interleaved stereo 48 kHz Float32 LE PCM. Every input is bounded by AUDIOGEN_INPUT_MAX_SECONDS (600 s, enforced before/while materializing) and checked for finite samples; malformed or oversized PCM fails with the new INVALID_AUDIO_INPUT (52422) before the addon is invoked; missing files surface AUDIO_FILE_NOT_FOUND. Aligned buffers are viewed as Float32Array without copying.
• Cancellation arriving while reference/source audio is still decoding is honoured: the abort state is re-checked after decode so model.run() never starts for a cancelled request.
• Cover tasks require sourceAudio, and cover-nofsq rejects any explicit audioCoverStrength other than 1 (the engine has no partial source context yet) — validated client-side (before RPC) and on the wire schema.
• decodeAudioToStream() now takes an options object; its timeout is inactivity-based (re-armed per decoded chunk, default 10 s, decoder unloaded on timeout) instead of a fixed 10 s total, so long inputs decode up to the 600 s limit. The response→stream bridge is exported as decoderResponseToStream() for tests.
• New root exports AUDIOGEN_TASK_TYPES, AUDIOGEN_INPUT_SAMPLE_RATE, AUDIOGEN_INPUT_CHANNELS, AUDIOGEN_INPUT_MAX_SECONDS (+ AudioGenTaskType/AudioGenAudioInput types); AudioGenTaskType registered in the constants registry for cross-language codegen. Contract + generated Python client regenerated.
• Docs: music-generation guide (reference audio & covers, sampling/DCW table, mobile support note), addon page, new examples/audiogen/generate-cover.ts.
• Deliberately not exposed: audioCodes (frozen semantic codes; no SDK path produces them) and the reserved, unimplemented FSQ cover task.
• Mobile e2e keeps skipping AudioGen (four ~3.3 GB GGUFs); addon-level mobile smoke tests cover Android Vulkan/iOS.

🧪 How was it tested?

• bun run build (lint + typecheck), bun run test:unit (109/109 files), bun run contract:check, sdk-python scripts/generate.py --check, e2e npm run typecheck.
• New bare tests (test/bare/audio-gen-input.test.ts, run under brittle-bare with the 0.2.1 prebuild): raw base64/raw file inputs, real WAV → FFmpeg → 48 kHz stereo float decode (24000 frames for 0.5 s @ 44.1 kHz mono), option forwarding to AudioGen.run(), fail-fast on malformed PCM / NaN samples / over-limit clips (sparse file) without invoking the addon, and cancel-during-decode never starting the native run; test/bare/audio-decoder-timeout.test.ts proves the inactivity semantics (decode outliving the window, hung decoder, 5-minute WAV end to end); existing audio-gen-plugin cancellation tests still pass on 0.2.1.
• New unit tests: schema bounds for the new controls, client normalization of path/Buffer/Uint8Array inputs, cover-requires-source refinement, root export + registry contract.
• New desktop e2e definitions: audio-gen-reference-audio (bundled WAV via file path), audio-gen-cover-nofsq (synthesized 1 s stereo tone as raw PCM buffer), audio-gen-cover-missing-source-error (client validation). Not run locally (needs the ~3.3 GB model set + GPU); to be exercised via the test-e2e-* label.

🔌 API Changes

const cover = audioGen({
  modelId,
  caption: 'orchestral arrangement with dramatic strings',
  lyrics: '[Instrumental]',
  taskType: 'cover-nofsq',
  sourceAudio: '/path/to/source.wav',        // decoded server-side, or a Buffer of raw stereo 48 kHz f32le PCM
  referenceAudio: '/path/to/reference.mp3',  // optional timbre reference
  audioCoverStrength: 1,
  coverNoiseStrength: 0.75,
  lmTemperature: 0.85,
  dcwEnabled: true
})

[github-actions]

License compliance — findings detected (warn-only)

Critical: 0 · High: 1 · Medium: 0

Dependency	License	Scope	Severity	Outcome
@qvac/audiogen-ggml@^0.2.1	(none detected)	runtime	High	blocks

How to resolve a blocking finding:

• Remove or replace the disallowed dependency, or
• If the license is genuinely acceptable, run the compliance SKILL and record the decision in .github/license-allowlist.yml (CODEOWNERS-reviewed), or
• For a one-off, a maintainer can apply the license-override label (High findings only; Critical cannot be overridden).

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

• ./.github/actions/release-merge-guard
• ./docs/website
• ./packages/ggml-coload-smoke
• ./packages/fabric/test/integration
• ./packages/inference-addon-cpp/mobile
• ./packages/sdk/e2e
• ./packages/llm-llamacpp/benchmarks/performance
• ./packages/llm-llamacpp/benchmarks/server
• ./packages/vla-ggml/sim/server
• ./packages/embed-llamacpp/benchmarks/performance
• ./packages/embed-llamacpp/benchmarks/server
• ./packages/asr-ggml/benchmarks/server

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​qvac/​audiogen-ggml@​0.2.2

View full report

[github-actions]

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

[github-actions]

QVAC E2E — android — ✅ all tests passed (92/106, 1987s)

Config: suite=smoke · filter=(none) · exclude=(none)
Device pool: Samsung S25 Ultra - Android 15 — ☀️ highly available
View run · Artifacts: reports · Device Farm logs

[github-actions]

QVAC E2E — ios — ✅ all tests passed (92/106, 1026s)

Config: suite=smoke · filter=(none) · exclude=(none)
Device pool: iPhone 16 Pro - iOS 26 — ☀️ highly available
View run · Artifacts: reports · Device Farm logs

[github-actions]

QVAC E2E — windows — ✅ all tests passed (106/106, 283s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run

[github-actions]

QVAC E2E — linux — ✅ all tests passed (106/106, 283s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run

[github-actions]

QVAC E2E — macos — ✅ all tests passed (106/106, 283s)

Config: suite=smoke · filter=(none) · exclude=(none)
View run · Artifacts: reports

[GustavoA1604]

[medium] packages/sdk/server/bare/plugins/audiogen-ggml/ops/audio-gen-input.ts — line 92 (const stream = await decodeAudioToStream(filePath, 'f32le', AUDIOGEN_INPUT_SAMPLE_RATE))

decodeAudioToStream() has a fixed 10-second total timeout that is not reset when output arrives. Consequently, valid reference or source files can fail long before the advertised 600-second limit. Make the timeout configurable or inactivity-based and add a long-running decode regression test.

[medium] packages/sdk/schemas/audio-gen.ts — line 134 (audioCoverStrength: unitIntervalSchema)

This accepts values below 1 for cover-nofsq, although the engine currently rejects them. Add conditional validation requiring audioCoverStrength === 1 for that task and test the unsupported-value branch.

[RamazTs]

@GustavoA1604 thanks — both addressed in 202f9e0:

• decodeAudioToStream timeout — now inactivity-based: the timer is re-armed on every decoded chunk (default still 10 s between chunks, configurable via the new DecodeAudioOptions.inactivityTimeoutMs), and the decoder is unloaded on the timeout path as well. The response→stream bridge is extracted as decoderResponseToStream() so the semantics are testable without FFmpeg timing luck. New test/bare/audio-decoder-timeout.test.ts covers: a decode whose total time outlives the window still completes, a decoder that goes silent fails with the inactivity error (decoder released once), and a 5-minute WAV decodes end to end (14.4M frames) with a 100 ms window.
• audioCoverStrength for cover-nofsq — the cover refinement now rejects any explicit value other than 1 (omitted is fine) on both the client params and the wire schema; text2music is unaffected. Unit test covers the accepted/rejected branches; docs updated.

Also rebased on the main merge that landed on the branch.

[opaninakuffo]

exposes audiogen-ggml 0.2.1 through audioGen() (reference/source, cover-nofsq, LM/DCW). abort-during-decode recheck is the right shape; gustavo's timeout + cover-strength notes look addressed. CI green.

two leftover decode-path issues: destroy doesn't cancel ffmpeg, and the 600s cap doesn't bound decoder-audio's full-file slurp. new e2e defs aren't in smoke (needs a full desktop e2e label to actually run).