Skip to content

feat(zoo): load the Higgs Audio v2 codec encoder from int8 and int4 checkpoints - #104

Merged
xxpopielxx merged 1 commit into
mainfrom
feat/quant-encoder
Nov 13, 2026
Merged

xxpopielxx merged 1 commit into
mainfrom
feat/quant-encoder

Conversation

@xxpopielxx

Copy link
Copy Markdown
Member

Description

Stacked on #102 (base feat/quant-codec).

Loads the Higgs Audio v2 codec encoder from int8 and int4 checkpoints (format version 2).

  • Conv1d: a layer that no scale group fits on the quantized conv (in channels not a multiple of 4, or kernel·in not a multiple of 8, e.g. one input channel) loads a quantized weight as its dequantized floats. A wrong group size on a layer that fits still throws.
  • HuBERT's position conv takes a folded conv.weight next to the weight norm, so a quantized file loads; each group runs quantized.
  • New dequantize() weight helper; the codec decoder's fold uses it instead of its own loop (touches feat(zoo): load the Higgs Audio v2 codec decoder from int8 and int4 checkpoints #102's decoder.ts).

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Performance (kernel tuning, graph building, anything related to performance)
  • Other (chores, tests, code style improvements etc.)

Testing instructions

  • pnpm --filter runntime exec vitest run test/core/nn/layers.test.ts test/core/weights/transforms.test.ts test/zoo/higgs
  • The GPU test loads a small quantized encoder (int8 convs, int8 or int4 Linears) and checks it against the same encoder loaded with the dequantized floats.
  • Checked locally on the codec's int8 and int4 encoder files: the quantized run matches the same files dequantized bit for bit; codes vs the float encoder 0.93–1.00 (int8) and 0.88–1.00 (int4) per codebook.
  • pnpm bench:ab kernels -b feat/quant-codec -k conv,matmul: 90 cases, all same, max abs 0.

Screenshots

Related issues

#97, #102

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

@xxpopielxx xxpopielxx added enhancement New feature or request core Engine: graph ops, dispatch, buffer pool, weight loading zoo Task runners and model loaders in src/zoo labels Nov 5, 2026
@xxpopielxx
xxpopielxx force-pushed the feat/quant-encoder branch 2 times, most recently from 416fed3 to 6acae1b Compare November 12, 2026 09:03
@xxpopielxx
xxpopielxx added this pull request to stack #114 November 12, 2026 13:21
Base automatically changed from feat/quant-codec to main November 13, 2026 14:07
…heckpoints

A Conv1d the quantized matmul cannot run, such as one on a single input channel, loads a
quantized weight as its dequantized floats.. The position conv takes a
folded weight next to the weight norm. dequantize() gives a QuantTensor's values, and the
codec decoder folds its quantized weights with it.
@xxpopielxx
xxpopielxx merged commit af0413c into main Nov 13, 2026
2 checks passed
@xxpopielxx
xxpopielxx deleted the feat/quant-encoder branch November 13, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Engine: graph ops, dispatch, buffer pool, weight loading enhancement New feature or request zoo Task runners and model loaders in src/zoo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants