FOCUS: New FP4 Quantization Framework for Efficient LLM Deployment

A novel framework for post-training quantization, named FOCUS, has been introduced to tackle the accuracy issues associated with FP4 quantization in large language models (LLMs). This framework is elaborated in a paper available on arXiv (2608.01847) and features Coupled-Relaxation Scaling (CRS) along with Dual-Granularity Scaling for optimizing scale parameters. Although FP4 quantization formats such as MXFP4 and NVFP4 are supported by modern hardware accelerators, achieving accuracy has proven challenging. The primary limitation stems from the tight coupling of quantization and dequantization scales, which must adhere to discrete low-precision formats like E8M0 in MXFP4. FOCUS addresses this by employing end-to-end scale learning, enhancing accuracy and facilitating efficient deployment. This research, announced as a new submission on arXiv, is pivotal for minimizing the computational and memory demands of LLMs, thus broadening their applicability in real-world scenarios.