Skip to content

fix(loader): pass model_type to TasksManager for CTC ASR model class resolution#1164

Closed
codykk wants to merge 1 commit into
mainfrom
yongyue/add-wav2vec2-ctc-support
Closed

fix(loader): pass model_type to TasksManager for CTC ASR model class resolution#1164
codykk wants to merge 1 commit into
mainfrom
yongyue/add-wav2vec2-ctc-support

Conversation

@codykk

@codykk codykk commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Fix incorrect model class resolution for CTC-based ASR models (Wav2Vec2, Hubert, Data2Vec-Audio). When automatic-speech-recognition maps to multiple candidate classes, Optimum defaults to AutoModelForSpeechSeq2Seq without model_type context, causing all *ForCTC models to fail. Passing model_type enables correct per-family resolution.

Add verified CPU recipes for jonatasgrosman/wav2vec2-large-xlsr-53-russian (fp32 + fp16).

Model metadata

  • What it does: Large-scale cross-lingual speech representation model fine-tuned for Russian automatic speech recognition (CTC decoding)
  • Architecture: Wav2Vec2ForCTC (wav2vec2) — CNN feature encoder → transformer encoder → CTC linear head
  • Task: automatic-speech-recognition
  • Vocab size: 39 (Russian characters + CTC blank)

Validation and support evidence

Baseline (main, 4daf0f1): FAIL — Unrecognized configuration class Wav2Vec2Config for AutoModelForSpeechSeq2Seq

Fix: Pass model_type=model_type_norm to TasksManager.get_model_class_for_task() (1-line change)

EP Device Precision Build Perf (avg latency) Throughput
CPU cpu fp32 ✅ PASS 87.10 ms 11.48 samples/sec
CPU cpu fp16 ✅ PASS 9.49 samples/sec

Generalization confirmed: HubertForCTC, Data2VecAudioForCTC also resolve correctly. WhisperForConditionalGeneration (Seq2Seq) unaffected.

Reproduce:

winml build -m jonatasgrosman/wav2vec2-large-xlsr-53-russian -o out --ep cpu --device cpu
winml perf -m out/model.onnx --ep cpu --device cpu

…lass resolution

Add regression recipe for jonatasgrosman/wav2vec2-large-xlsr-53-russian
(cpu/fp32 and cpu/fp16).
@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 21, 2026
@codykk codykk closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants