InfoQuant: Optimizing Activation Distributions for Low-Bit LLM Quantization

A recent study published on arXiv (2605.26175), entitled 'InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization,' tackles the issue of low-bit quantization of activations in large language models (LLMs). The researchers contend that current post-training quantization (PTQ) techniques do not adequately define which activation distributions are conducive to discretization, resulting in significant quantization errors despite seemingly smoother numerical activations. They suggest redefining activation transformation as a design for quantizer-oriented distributions and examine quantization errors through an information-theoretic lens. Their findings indicate that activations ideal for quantization should possess both a reduced numerical range and adequate variability.