Making AI inference faster, more accurate, and easier to understand.
I'm an AI engineer at Intel focused on low-bit quantization, efficient attention, and GPU kernel optimization.
I maintain AutoRound and LLM-Compressor, contribute to open-source inference frameworks, and create interactive visualizations to explain how these systems work.
| Project | My focus |
|---|---|
| AutoRound | Maintainer; quantization algorithms and efficient low-bit inference |
| LLM-Compressor | Maintainer; quantization workflows, Intel GPU support, CI/CD and community contributions |
| vLLM | LLM inference optimization, quantization, and GPU backend integration |
| vLLM-Omni | Multimodal quantization, attention optimization, and model integration |
| compressed-tensors | Low-bit quantization formats and backend integration |
- Low-bit inference: INT4, FP8, MXFP4/MXFP8, NVFP4, and accuracy-performance tradeoffs.
- Efficient attention: SageAttention, block-sparse attention, and attention quantization error.
- GPU kernels: Performance profiling, compiler behavior, and low-level optimization.
- Multimodal inference: Image and video generation with efficient memory and compute.
- AI for systems: Agents for GPU kernel development and performance optimization.
I build interactive explanations of GPU architecture, quantization, attention mechanisms, and inference optimization.
Explore 40+ interactive visualizations โ ยท Source code: Inference Viz
- Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs โ Findings of EMNLP 2024.
- PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization โ TPAMI.
I'm interested in practical ideas that make AI inference more efficient.
If you're working on quantization, attention algorithms, GPU kernels, or inference frameworks, I'd be happy to exchange ideas and collaborate through open source.
- ๐ง Email: yi4.liu@intel.com
- ๐ผ LinkedIn: Randall Liu
