Skip to content

feat: expand audio degradation scenarios with new presets and compound chains #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
karamouche merged 3 commits into from
May 22, 2026
Merged

feat: expand audio degradation scenarios with new presets and compound chains #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
02e5e1d
feat: expand audio degradation scenarios with new presets, enhance CL…
karamouche May 22, 2026
6447345
refactor: rename preset to telecom, update related doc, and add input…
karamouche May 22, 2026
27c0e6f
docs: update README to clarify dataset generation for noise-robust AS…
karamouche May 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 93 additions & 16 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,17 @@

**Always update this file when making notable changes** (new commands, new presets, architectural decisions, scoring changes, dependency additions).

**Always update README.md** when changing presets, CLI flags, output format, or any user-facing behavior.

## Project

`noisekit` is a `uvx`-compatible Python CLI that generates degraded speech datasets from clean HuggingFace corpora. It simulates three real-world audio degradation scenarios — telecommunication (G.711 calls), bad audio encoding (low-bitrate codecs), and noisy environments (real ambient noise) — for ASR noise-robustness benchmarking. A `clean_reference` control completes the catalog.
`noisekit` is a `uvx`-compatible Python CLI that generates degraded speech datasets from clean HuggingFace corpora. It simulates seven atomic audio degradation scenarios — telecom (G.711 calls), bad audio encoding (low-bitrate codecs), noisy environments (real ambient noise), far-field reverb, transmission dropout, and clipping distortion — plus compound multi-condition scenarios built by chaining atomic presets. Designed for ASR noise-robustness benchmarking. A `clean_reference` control completes the catalog.

## Package Management

Use **UV** for everything: `uv add`, `uv run`, `uv sync`. Never use pip directly.

Key runtime dependencies: `audiomentations>=0.38`, `lameenc>=1.4` (pure-Python MP3 encoder used by `Mp3Compression` in `telecommunication` and `bad_audio_encoding`; no system ffmpeg needed), `torchmetrics>=1.7.0` (NISQA scoring — downloads ~50 MB model weights to `~/.torchmetrics/NISQA/` on first use).
Key runtime dependencies: `audiomentations>=0.38`, `lameenc>=1.4` (pure-Python MP3 encoder used by `Mp3Compression` in `telecom` and `bad_audio_encoding`; no system ffmpeg needed), `torchmetrics>=1.7.0` (NISQA scoring — downloads ~50 MB model weights to `~/.torchmetrics/NISQA/` on first use), `pyroomacoustics` (room acoustics simulation for `reverb_far_field` — now a core dependency, no extra install needed).

## Architecture

Expand Down Expand Up @@ -61,29 +63,78 @@ transforms:

Built-in presets:

| Preset | Scenario | Bandwidth | PESQ mode | Target MOS |
| -------------------- | -------------------------------------------- | ------------------- | --------- | ---------- |
| `clean_reference` | Minimal gain normalization (PESQ ceiling) | full | WB 16 kHz | 4.0-4.5 |
| `telecommunication` | G.711 call + low-bitrate MP3 codec artifacts | 300-3400 Hz @ 8 kHz | NB 8 kHz | 2.0-3.5 |
| `bad_audio_encoding` | Aggressive low-bitrate MP3 (16-32 kbps) | 80-7500 Hz @ 16 kHz | WB 16 kHz | 1.5-2.5 |
| `noisy_environment` | Real ambient noise via `AddBackgroundNoise` | up to 8-12 kHz | WB 16 kHz | 2.0-3.5 |
### Atomic Presets

| Preset | Scenario | Bandwidth | PESQ mode | Target MOS |
| ---------------------- | ----------------------------------------------------- | ------------------- | --------- | ---------- |
| `clean_reference` | Minimal gain normalization (PESQ ceiling) | full | WB 16 kHz | 4.0-4.5 |
| `telecom` | G.711 call + low-bitrate MP3 codec artifacts | 300-3400 Hz @ 8 kHz | NB 8 kHz | 2.0-3.5 |
| `bad_audio_encoding` | Aggressive low-bitrate MP3 (16-32 kbps) | 80-7500 Hz @ 16 kHz | WB 16 kHz | 1.5-2.5 |
| `noisy_environment` | Real ambient noise via `AddBackgroundNoise` | up to 8-12 kHz | WB 16 kHz | 2.0-3.5 |
| `clipping_distortion` | Microphone overload / ADC saturation (`ClippingDistortion` 10-25%) | full | WB 16 kHz | 2.0-3.5 |
| `transmission_dropout` | VoIP packet loss: 1-3 silent dropout windows | full | WB 16 kHz | 1.5-3.0 |
| `reverb_far_field` | Far-field reverberant room via `RoomSimulator` | full | WB 16 kHz | 2.0-3.5 |

`telecom` and any compound preset ending with `telecom` use the 8 kHz PESQ NB scoring split (see below). All other presets score in PESQ WB at 16 kHz.

### Compound Presets

Compound presets chain two or more atomic presets together. Noise is added first (acoustic environment), then codec/dropout (digital processing of the already-degraded signal).

| Preset | Chain | Requires | PESQ mode | Target MOS |
| ------------------ | ----------------------------------------- | ------------- | --------- | ---------- |
| `noisy_telecom` | `noisy_environment` → `telecom` | `--noise-dir` | NB 8 kHz | 1.5-2.5 |
| `reverb_noisy` | `reverb_far_field` → `noisy_environment` | `--noise-dir` | WB 16 kHz | 1.0-2.5 |
| `clipping_telecom` | `clipping_distortion` → `telecom` | — | NB 8 kHz | 1.0-2.5 |

### Compound Preset YAML Format

A preset can use `chain:` instead of `transforms:` to apply multiple atomic presets sequentially:

`telecommunication` uses the 8 kHz PESQ NB scoring split (see below). All other presets score in PESQ WB at 16 kHz.
```yaml
name: my_compound
description: "..."
chain:
- atomic_preset_a
- atomic_preset_b
```

Rules:
- `chain` and `transforms` are mutually exclusive.
- Chained entries must be names of built-in atomic presets (no nesting chains).
- `${NOISE_DIR}` resolution and the PESQ NB scoring split are detected automatically across the full concatenated chain.
- `reverb_far_field` uses `pyroomacoustics` (bundled as a core dependency — no extra install needed).

### Why no white noise

The catalog deliberately avoids `AddGaussianSNR` — white Gaussian noise sounds artificial and doesn't reflect real production audio. Instead:

- `telecommunication` and `bad_audio_encoding` rely on `Mp3Compression` at 16-32 kbps for realistic codec smearing/pre-echo.
- `telecom` and `bad_audio_encoding` rely on `Mp3Compression` at 16-32 kbps for realistic codec smearing/pre-echo.
- `noisy_environment` uses `AddBackgroundNoise` over a user-supplied WAV corpus (MUSAN/DEMAND/FSD50K), so the noise floor matches the real environment you care about.

## PESQ Scoring — Important Design Decision

For `telecommunication`, PESQ is computed at **8 kHz narrowband** on the audio **before** the final `Resample(16000)` restoration step. Output WAV files are still saved at 16 kHz.
For `telecom`, PESQ is computed at **8 kHz narrowband** on the audio **before** the final `Resample(16000)` restoration step. Output WAV files are still saved at 16 kHz.

**Why:** Computing PESQ NB by downsampling the 16 kHz output (8k→16k→8k round-trip) collapses all telephony scores to ~1.1 regardless of noise level. Scoring at the 8 kHz intermediate stage gives proper stratification.

**BitCrush + Normalize:** `telecommunication` inserts `Normalize(p=1.0)` immediately before `BitCrush`. HuggingFace speech datasets (e.g., FLEURS) often have very low peak amplitude (~0.001-0.02). At 8-bit depth the quantization step is 0.0078 — a peak below one step rounds the entire signal to zero. Normalizing to ±1 before quantization ensures all 256 levels are used.
**BitCrush + Normalize:** `telecom` inserts `Normalize(p=1.0)` immediately before `BitCrush`. HuggingFace speech datasets (e.g., FLEURS) often have very low peak amplitude (~0.001-0.02). At 8-bit depth the quantization step is 0.0078 — a peak below one step rounds the entire signal to zero. Normalizing to ±1 before quantization ensures all 256 levels are used.

## Input Normalization — Global Pipeline Decision

`pipeline.py` peak-normalizes every input sample to amplitude 1.0 immediately after resampling to 16 kHz, before any preset transforms run:

```python
peak = np.abs(ref_16k).max()
if peak > 1e-9:
ref_16k = ref_16k / peak
```

**Why:** HuggingFace datasets often have peaks as low as 0.001–0.02. Without normalization, `AddBackgroundNoise` (relative SNR mode) scales noise proportional to that tiny signal RMS — both speech and noise end up inaudible, and 16-bit PCM quantization noise dominates. Peak normalization guarantees all presets receive a full-scale signal.

**Safety:** The same normalized `ref_16k` is used as both the transform input and the PESQ/SNR reference, so all quality metrics remain valid relative comparisons. The mid-chain `Normalize` inside `telecom.yaml` (before `BitCrush`) is still needed separately — the bandpass filter removes energy and that step re-normalizes before quantization.

**`noisy_environment` also pre-normalizes:** `noisy_environment.yaml` adds a `Normalize` as its first transform. This handles the `reverb_noisy` compound case: `RoomSimulator` can attenuate the signal by ~10× at large mic distances; without the mid-chain normalize, `AddBackgroundNoise` would see the attenuated level and mix noise too quietly. All compound presets using `noisy_environment` inherit this fix automatically.

`transforms.py` auto-detects this split: if the last transform is `Resample(16000)`, it creates a `scoring` Compose (all-but-last) alongside the `full` Compose.

Expand All @@ -97,11 +148,11 @@ Uses `datasets` with `Audio(decode=False)` + manual `soundfile` decoding — avo

```json
{
"file_name": "audio/common_voice_en_23136613_telecommunication.wav",
"file_name": "audio/common_voice_en_23136613_telecom.wav",
"source": "common_voice_en_23136613.mp3",
"dataset": "google/fleurs",
"language": "en-US",
"preset": "telecommunication",
"preset": "telecom",
"transcript": "...",
"snr_db": 1.8,
"pesq_mos": 2.86,
Expand All @@ -126,16 +177,40 @@ uv run noisekit list-presets --verbose
uv run noisekit generate \
--dataset google/fleurs \
--config en_us --split test \
--samples 3 --presets clean_reference telecommunication bad_audio_encoding \
--samples 3 --presets clean_reference telecom bad_audio_encoding \
--output ./test_out --seed 42
cat test_out/metadata.jsonl

# New atomic presets — no external dependencies
uv run noisekit generate \
--dataset google/fleurs --config en_us --split test \
--samples 3 --presets clipping_distortion transmission_dropout \
--no-nisqa --output ./test_atomic --seed 42

# noisy_environment — auto-downloads MUSAN noise-only clips on first run
uv run noisekit generate \
--dataset google/fleurs --config en_us --split test \
--samples 3 --presets noisy_environment \
--output ./test_noise --seed 42

# Compound presets (auto-downloads MUSAN noise on first run)
uv run noisekit generate \
--dataset google/fleurs --config en_us --split test \
--samples 3 --presets noisy_telecom \
--no-nisqa --output ./test_compound --seed 42

# clipping_telecom — no noise dir needed
uv run noisekit generate \
--dataset google/fleurs --config en_us --split test \
--samples 3 --presets clipping_telecom \
--no-nisqa --output ./test_clipping_telecom --seed 42

# Far-field reverb
uv run noisekit generate \
--dataset google/fleurs --config en_us --split test \
--samples 3 --presets reverb_far_field reverb_noisy \
--no-nisqa --output ./test_reverb --seed 42

# noisy_environment with your own noise corpus (skips auto-download)
uv run noisekit generate \
--dataset google/fleurs --config en_us --split test \
Expand All @@ -144,6 +219,8 @@ uv run noisekit generate \
--output ./test_noise --seed 42
```

Expected PESQ spread: clean ~4.6, telecommunication ~2.5-3.5 (NB), bad_audio_encoding ~1.5-2.5 (WB), noisy_environment ~1.0-2.5 (WB).
Expected PESQ spread: clean ~4.6, telecom ~2.5-3.5 (NB), bad_audio_encoding ~1.5-2.5 (WB), noisy_environment ~1.0-2.5 (WB), clipping_distortion ~2.0-3.5 (WB), transmission_dropout ~1.5-3.0 (WB), reverb_far_field ~2.0-3.5 (WB).

Compound preset PESQ: noisy_telecom ~1.5-2.5 (NB), clipping_telecom ~1.0-2.5 (NB), reverb_noisy ~1.0-2.5 (WB).

Expected NISQA spread: clean ~4.0-4.5, degraded presets ~1.5-3.0. NISQA model weights (~50 MB) are downloaded on first run.
72 changes: 50 additions & 22 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,23 @@

<br/>

Generate noise-stratified speech datasets for ASR benchmark studies.
Generate degraded speech datasets for noise-robust ASR benchmarking.

Takes a clean speech-to-text dataset from HuggingFace, applies real-world degradation presets via [audiomentations](https://github.com/iver56/audiomentations), and scores each output with PESQ + SNR + NISQA — producing a JSONL manifest ready for noise-robustness benchmarking.
Takes a clean HuggingFace speech dataset, applies real-world degradation presets via [audiomentations](https://github.com/iver56/audiomentations), and scores each output with PESQ, SNR, and NISQA, producing a JSONL manifest ready for noise-robustness benchmarking.

Three scenarios are covered out of the box: **telecommunication** (G.711 + low-bitrate MP3 codec artifacts), **bad audio encoding** (aggressive low-bitrate compression), and **noisy environment** (real ambient noise from a user-supplied corpus).
Seven atomic degradation scenarios are built in: telephony (G.711 + low-bitrate codec), wideband codec compression, ambient noise, clipping distortion, transmission dropout, and far-field reverb. Atomic presets compose into compound multi-condition scenarios.

> [!NOTE]
> Degradations are programmatically simulated. Scores may not generalize to genuine production recordings; validate final benchmarks on annotated real-world data.

## How it works

```mermaid
flowchart LR
A[("HuggingFace\nDataset")] --> B["noisekit generate"]
B --> C["telecommunication\nG.711 + MP3"]
B --> D["bad_audio_encoding\n16-32 kbps MP3"]
B --> E["noisy_environment\nReal ambient noise"]
B --> F["clean_reference\nControl"]
C & D & E & F --> G[("WAVs +\nmetadata.jsonl\nPESQ · SNR · NISQA")]
B --> C["7 atomic presets\ncodec · noise · reverb\ndropout · clipping"]
B --> D["3 compound presets\nmulti-condition chains"]
C & D --> E[("WAVs + metadata.jsonl\nPESQ · SNR · NISQA")]
```

## Install
Expand Down Expand Up @@ -55,7 +56,7 @@ uvx noisekit generate \
--config en_us \
--split test \
--samples 300 \
--presets telecommunication bad_audio_encoding \
--presets telecom bad_audio_encoding \
--output ./benchmark_dataset \
--seed 42
```
Expand All @@ -76,7 +77,7 @@ Output:
benchmark_dataset/
├── metadata.jsonl # one entry per generated file (AudioFolder format)
└── audio/
├── sample_0000_telecommunication.wav
├── sample_0000_telecom.wav
├── sample_0001_bad_audio_encoding.wav
└── ...
```
Expand All @@ -92,11 +93,11 @@ Each `metadata.jsonl` entry:

```json
{
"file_name": "audio/sample_0042_telecommunication.wav",
"file_name": "audio/sample_0042_telecom.wav",
"source": "common_voice_en_23136613.mp3",
"dataset": "google/fleurs",
"language": "en-US",
"preset": "telecommunication",
"preset": "telecom",
"transcript": "the cat sat on the mat",
"snr_db": 5.2,
"pesq_mos": 2.78,
Expand Down Expand Up @@ -130,19 +131,46 @@ uvx noisekit list-presets --verbose # show full transform stack

## Presets

Four built-in presets — three real-world scenarios plus a clean control. None use synthetic white noise; codec artifacts and real ambient recordings produce the degradation instead.
Ten built-in presets: seven atomic scenarios, three compound multi-condition presets, and a clean reference control. None use synthetic white noise; codec artifacts, real ambient recordings, and room simulation produce the degradation instead.

### Atomic presets

| Preset | Description | PESQ |
| ---------------------- | ------------------------------------------------------------------------ | ---------- |
| `clean_reference` | Minimal processing (PESQ ceiling / control) | 4.0-4.5 |
| `telecom` | G.711-style call: 8 kHz bandpass + 8-bit BitCrush + 16-32 kbps MP3 codec | NB 2.0-3.5 |
| `bad_audio_encoding` | Wideband audio crushed by 16-32 kbps MP3 compression | WB 1.5-2.5 |
| `noisy_environment` | Real ambient noise from `--noise-dir` mixed in at SNR 5-15 dB | WB 1.0-2.5 |
| `clipping_distortion` | Microphone overload: clips the loudest 10-25% of samples | WB 2.0-3.5 |
| `transmission_dropout` | VoIP packet loss: 1-3 silent dropout windows (60-180 ms each) | WB 1.5-3.0 |
| `reverb_far_field` | Far-field room reverb at 1-3 m mic distance | WB 2.0-3.5 |

| Preset | Description | PESQ |
| -------------------- | ------------------------------------------------------------------------ | ---------- |
| `clean_reference` | Minimal processing (PESQ ceiling / control) | 4.0-4.5 |
| `telecommunication` | G.711-style call: 8 kHz bandpass + 8-bit BitCrush + 16-32 kbps MP3 codec | NB 2.0-3.5 |
| `bad_audio_encoding` | Wideband audio crushed by 16-32 kbps MP3 compression | WB 1.5-2.5 |
| `noisy_environment` | Real ambient noise from `--noise-dir` mixed in at SNR 3-20 dB | WB 1.0-2.5 |
`telecom` is scored with PESQ narrowband at 8 kHz (before the final upsample); all other presets are scored wideband at 16 kHz.

`telecommunication` is scored with PESQ narrowband at 8 kHz (before the final upsample); all other presets are scored wideband at 16 kHz.
All atomic presets require no noise corpus. All dependencies, including `pyroomacoustics` (used by `reverb_far_field`), are bundled with no extra install needed.

`noisy_environment` requires `--noise-dir` pointing at a directory of background-noise WAVs (e.g. MUSAN, DEMAND, FSD50K). If omitted, noisekit auto-downloads a small MUSAN noise-only subset (~120 MB) from HuggingFace on first use.

### Compound presets

Compound presets chain two atomic presets together. Noise is applied first (acoustic environment), then codec or dropout (digital processing on the already-degraded signal).

| Preset | Chain | Requires | PESQ |
| ------------------ | ---------------------------------------- | ------------- | ---------- |
| `noisy_telecom` | `noisy_environment` → `telecom` | `--noise-dir` | NB 1.5-2.5 |
| `clipping_telecom` | `clipping_distortion` → `telecom` | (none) | NB 1.0-2.5 |
| `reverb_noisy` | `reverb_far_field` → `noisy_environment` | `--noise-dir` | WB 1.0-2.5 |

You can also define your own compound preset with a `chain:` key in a YAML file:

```yaml
name: my_compound
description: "Noisy environment then telephony codec"
chain:
- noisy_environment
- telecom
```

### Custom presets

Pass your own YAML file with `--preset-file`:
Expand Down Expand Up @@ -179,10 +207,10 @@ transforms:
p: 1.0
```

Any transform from [audiomentations](https://github.com/iver56/audiomentations) is supported. Use `${NOISE_DIR}` as a placeholder for `--noise-dir` inside your preset YAML.
Any transform from [audiomentations](https://github.com/iver56/audiomentations) is supported. Use `${NOISE_DIR}` as a placeholder for `--noise-dir` inside your preset YAML. Use `chain:` instead of `transforms:` to compose built-in atomic presets sequentially.

## Requirements

- Python ≥ 3.10
- [uv](https://docs.astral.sh/uv/) for `uvx` usage
- No system dependencies — MP3 encoding uses pure-Python `lameenc`, no ffmpeg needed
- No system dependencies: MP3 encoding uses pure-Python `lameenc`, no ffmpeg needed
2 changes: 1 addition & 1 deletion assets/banner.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading