A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
Add a description, image, and links to the fp4 topic page so that developers can more easily learn about it.
To associate your repository with the fp4 topic, visit your repo's landing page and select "manage topics."