Enable Online FP8&NF4 on Minimax-H3 - #25
Merged
Merged
Conversation
lzx1413
force-pushed
the
minimax_h3_nf4_fp8
branch
from
August 7, 2026 08:48
424c468 to
9bb9277
Compare
Collaborator
|
LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add online DiT quantization for MiniMax H3 with two backends:
They convert 258 Linear layers of MiniMax H3 across the main and token-refiner transformer blocks while preserving the reference dtype of the FP32 projections, text encoder, and VAEs.
Motivation
MiniMax H3's BF16 DiT profile requires most of an 80 GB H100. Online quantization provides a practical single-GPU
deployment option.
Implementation
MiniMaxH3DiT.enable_quant()dispatch for TorchAO FP8 and BNB NF4.quantizationloading for CUDA, single-GPU execution, and FSDP exclusion.examples/minimax_h3/minimax_h3_fl2va_torchao_fp8_h100.pyexamples/minimax_h3/minimax_h3_fl2va_bnb_nf4_h100.py--quantizationto the existing FL2VA, Ref2VA, and JSON request examples.Benchmark
The matched benchmark uses one NVIDIA H100 80 GB, MiniMax H3 FL2VA, 768p 16:9 output, five seconds, 50 inference
steps, seed
0, and the prompt:tf-kernelFP8 shows 5.6% faster of denoising and 9.9% less memory than torchao FP8, while its one-time quantization is much slower than torchao, which leads to higher cost of a single e2e inference. @lzx1413Generated Video Comparison
prompt: "Steam rises from the ramen while the family talks in the background."
It is seen that tf-kernel FP8 presents much more vibrant views than BNB NF4, but it is inconsistent to the original BF16 output, whose content diverges from the prompt. I suggest torchao as the default FP8 for Minimax-H3, and tf-kernel as an option. @lzx1413
Checklist
ruff)pre-commit run --all-files)pytest tests/)[TYPE] Brief description