Skip to content

Enable Online FP8&NF4 on Minimax-H3 - #25

Merged
lzx1413 merged 5 commits into
Tele-AI:mainfrom
Uxito-Ada:minimax_h3_nf4_fp8
Aug 7, 2026
Merged

Enable Online FP8&NF4 on Minimax-H3#25
lzx1413 merged 5 commits into
Tele-AI:mainfrom
Uxito-Ada:minimax_h3_nf4_fp8

Conversation

@Uxito-Ada

@Uxito-Ada Uxito-Ada commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add online DiT quantization for MiniMax H3 with two backends:

  • TorchAO FP8 dynamic activation and weight quantization
  • bitsandbytes NF4 weight-only quantization with BF16 compute

They convert 258 Linear layers of MiniMax H3 across the main and token-refiner transformer blocks while preserving the reference dtype of the FP32 projections, text encoder, and VAEs.

Motivation

MiniMax H3's BF16 DiT profile requires most of an 80 GB H100. Online quantization provides a practical single-GPU
deployment option.

Implementation

  • Add MiniMaxH3DiT.enable_quant() dispatch for TorchAO FP8 and BNB NF4.
  • Add quantization loading for CUDA, single-GPU execution, and FSDP exclusion.
  • Add dedicated H100 examples:
    • examples/minimax_h3/minimax_h3_fl2va_torchao_fp8_h100.py
    • examples/minimax_h3/minimax_h3_fl2va_bnb_nf4_h100.py
  • Add --quantization to the existing FL2VA, Ref2VA, and JSON request examples.
  • Documents.

Benchmark

The matched benchmark uses one NVIDIA H100 80 GB, MiniMax H3 FL2VA, 768p 16:9 output, five seconds, 50 inference
steps, seed 0, and the prompt:

image

tf-kernel FP8 shows 5.6% faster of denoising and 9.9% less memory than torchao FP8, while its one-time quantization is much slower than torchao, which leads to higher cost of a single e2e inference. @lzx1413

Generated Video Comparison

prompt: "Steam rises from the ramen while the family talks in the background."

Precision Video
BF16 https://github.com/user-attachments/assets/72afe0cf-99b0-4a07-903f-94a412ef43d9
torchao FP8 https://github.com/user-attachments/assets/1361ecb6-0a62-48a3-8b7b-2b3b9c7c55d5
NF4 https://github.com/user-attachments/assets/1993e6c0-fe70-40bf-baed-8b16f6442fc3
tf-kernel FP8 https://github.com/user-attachments/assets/8e958cd5-3ad9-45fa-a3df-c9a4c103d098

It is seen that tf-kernel FP8 presents much more vibrant views than BNB NF4, but it is inconsistent to the original BF16 output, whose content diverges from the prompt. I suggest torchao as the default FP8 for Minimax-H3, and tf-kernel as an option. @lzx1413

Checklist

  • Code follows the project's coding standards (ruff)
  • Pre-commit hooks pass (pre-commit run --all-files)
  • All tests pass (pytest tests/)
  • New tests added for new functionality
  • Documentation updated (README, CLAUDE.md, docstrings)
  • Commit messages are clear and descriptive
  • PR title follows the convention: [TYPE] Brief description

@Uxito-Ada
Uxito-Ada requested a review from lzx1413 August 6, 2026 02:46
@lzx1413
lzx1413 force-pushed the minimax_h3_nf4_fp8 branch from 424c468 to 9bb9277 Compare August 7, 2026 08:48
@lzx1413

lzx1413 commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

LGTM

@lzx1413
lzx1413 merged commit ed98dee into Tele-AI:main Aug 7, 2026
5 checks passed
@Uxito-Ada
Uxito-Ada deleted the minimax_h3_nf4_fp8 branch August 7, 2026 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants