Audio-domain consumer feasibility question: 0.21.4 → 0.21.12 upgrade (context from DeepFilterNet revert)

[czoli1976]

First — thank you for tract. The fact that a full DeepFilterNet 3
speech-enhancement graph runs end-to-end in WebAssembly inside a
browser worker is a direct testament to tract's design: fast, small,
and actually practical for real-world deployment. The investment in
ONNX coverage + NNEF + the new facade layer is genuinely appreciated
from the consumer side.

Context

We're evaluating a browser/WASM noise-suppression library built on
DeepFilterNet 3 (libDF). libDF has been pinned to tract 0.21.4
since Rikorose/DeepFilterNet#558
(May 2024) and we're looking at whether to upgrade further within
the 0.21 line — specifically toward 0.21.12, which looks attractive
for WASM targets.

One data point we'd appreciate your perspective on: in June 2023,
DeepFilterNet reverted from tract 0.20.x back to 0.19.4 after
observing "audio artifacts" in enhanced output
(Rikorose/DeepFilterNet#405).
They later moved to 0.21 (Jan 2024) without incident and have been
stable on 0.21.4 for nearly two years.

We've also opened a question on the DeepFilterNet side about their
plans: Rikorose/DeepFilterNet#682.

Questions

1. The 2023 revert: are you aware of the 0.19 → 0.20 audio-
   artifact issue, and if so, is there any context you can share
   about what changed in that transition that might have produced
   it? (No pressure if it's been too long — understandable.)

2. The 0.21 line since 0.21.4: releases that look particularly
   relevant for a WASM speech-enhancement workload include:

   • 0.21.6 — "WASM f32 4x4 kernel" and multithreaded matmul
     runner
   • 0.21.8 — MMM kits + element-wise binary op optimizations
   • 0.21.10 — reduce optimizations impacting modern
     normalization layers
   • 0.21.12 — multithread-mm feature flag in tract-linalg

   Have these been exercised against audio / RNN-heavy ONNX graphs
   in production by other consumers you're aware of? Any known
   regressions or gotchas?

3. Upgrade feasibility: from the tract side, are there specific
   changes within 0.21.4 → 0.21.12 you'd call out to a consumer
   doing audio speech-enhancement inference, either as concerns to
   validate against or as wins worth capturing?

The goal isn't to pressure anyone — we're collecting signal to
decide whether to drive the upgrade ourselves (with DNSMOS parity
validation on a clip corpus) or park the idea. Your perspective
would be really helpful regardless of which direction it points us.

Thank you again for what you've built.

[kali]

Hello, thanks for the kind words.

I don't remember hearing about regressions between 0.19 and 0.20. I guess I would have investigated them. We are talking with accuracy issue, not performance here, right ? These "artifacts" are actually wrong computation, not latency increase ?

What I can share: Sonos is (obviously) doing a lot of audio, heavy on CNN and RNN with a clear preference for NNEF inputs. We are not very invested in the wasm target, so the CI coverage is lighter on this front. A first high level diagnostic would be to check if the regression are WASM side only or if they also appear on a native cpu target.

I would strongly encourage your team to migrate to 0.22, and/or prepare to migrate to 0.23 even if the new facade api is not fully stabilized yet (it may be getting there, i have no know issue or frustration about it at this stage but... it's been just a week). I will stop maintaining 0.21 pretty soon, and I consider 0.20 and 0.19 legacy. 0.23 and the facade API is very important, as I am trying to find an API I can actually commit to instead of the old "everything is essentially public, I'm just doing my best to not break it too much" stance.

If there are observable accuracy or performance regressions and you can share details with me (privately if required), I'm prepared to look into them.

[czoli1976]

Kali, thanks for the quick reply, according to Rikorose/DeepFilterNet#405 there were audio artefacts, not latency. I did not attempt at upgrading yet from the version that Rikorose has been pinning. Asked them if they tried more recently and will let you know here if they reply!

I think I will give it a try anyways because I am curious to see if I can unlock further WASM Improvements, will have to do careful DNSMOS parity validation. I am new to such things, It is a side project for me ATM and I am starting to understand how tricky these things are, my hunger for speed improvements is a slippery slope as need to watch out for output precision regressions.

Given DeepFilterNet is very popular and use tract, would it make sense to consider adding a test for it in your CI ?

[kali]

Absolutely open for a test case, as long as I can have a the model on tract s3 bucket to avoid adding an external dep. Does not necessarily need to be a trained model.

[czoli1976]

Thanks kali, this is really helpful guidance — especially the
deprecation-of-0.21 signal. That reshapes our plan materially.

Rescoping what we're going to do:

• Extend our validation sweep to target 0.22.x at minimum rather
  than stopping in the 0.21 line. 0.23 stable when it lands is also on
  the list, but we'll let the facade API settle a bit more before
  targeting it.
• Your diagnostic hint about native CPU vs WASM comparison is
  exactly the right first check — we'll add that as a Phase 0 step so
  any regression we find gets cleanly attributed (tract-core math vs
  WASM kernel behavior). Cheap to add, high diagnostic value.

On the CI test case — yes please, we'd love to contribute that. Our
working plan:

1. Finish our upgrade validation sweep with full DNSMOS parity +
   native-CPU diff + WASM per-frame RTT measurements on a stratified
   ~100-clip corpus.
2. Based on what we learn, propose a minimal DFN3-shaped test case
   (graph topology + a small set of representative inputs), targeting
   your S3 bucket for the model artifact as you requested.
3. Open a PR against tract with the test case + run-it-in-CI wiring,
   plus a brief note on what signal it's designed to catch (kernel
   parity, shape inference, WASM target correctness).

Will keep this thread updated as we progress. If there's a specific op
coverage area you'd like us to emphasize (multi-frame filtering, GRU
stacks, ERB-domain reductions), let me know and we'll weight the test
inputs accordingly.

Thanks again — much appreciated.

[czoli1976]

@kali — quick update on our DFN3 validation sweep, with thanks for your earlier guidance which shaped the whole approach.

1. tract 0.21.x → 0.22.1 path is clean for DFN3. Output is SHA256-identical across 0.21.4 → 0.21.5 → 0.21.12 → 0.21.15 → 0.22.0 → 0.22.1 on a 40-clip VoiceBank-DEMAND set, and DNSMOS-OVR is bit-stable at 3.324. We're committing to 0.22.1 as our potential ship target (this is exploratory work) per your guidance about 0.21 deprecation — really appreciated knowing that signal early.

2. The WASM speedup is real and meaningful. Per-frame inference: 0.466 ms on 0.21.15 vs 0.659 ms on 0.21.4 baseline (Node v20, M1 Pro), preserved in Chrome 147. ~25% faster on real audio. Your WASM kernel investment is paying off concretely for streaming-audio downstream consumers — thank you for that work even if WASM isn't the primary investment area.

3. Found a 0.23.0-dev.5 regression — but it's PulsedModel-specific. Filed as a separate dedicated issue so this thread stays focused: #2157 . TL;DR: same model, same SimpleState, same deterministic inputs → byte-identical frame-0 output but divergent GRU-recurrence outputs from frame 1 onward, only when PulsedModel-wrapped. Self-contained Rust repro in the new issue + linked gist.

I followed your suggestion of native-vs-WASM cross-check first — confirmed the regression appears identically on both targets, so it's tract-core / tract-pulse rather than WASM kernel.

On the CI test case offer you made earlier: the new repro is exactly the right shape (small ONNX + deterministic inputs + golden outputs). Once #2157 gets resolved, happy to PR it as a regression-prevention check, however you'd like it structured in tract's tree.

Thanks again for the engagement here.

— CK (Ckristian Zoli, Vonage - part of Ericsson)