LBLLM Framework Enables Efficient Binarization of Large Language Models for Resource-Constrained Environments

A novel lightweight binarization framework named LBLLM has been introduced to tackle the challenges of computation and memory when deploying large language models in environments with limited resources. This framework utilizes a three-phase quantization approach that effectively achieves W(1+1)A4 quantization. Initially, it establishes a high-quality quantized model via post-training quantization (PTQ). Next, it quantizes binarized weights, group-wise bitmaps, and quantization parameters through layer-wise distillation, keeping activations at full precision. Finally, it trains adjustable activation quantization factors to convert activations to 4 bits dynamically. This separated design minimizes the interference between weight and activation quantization, enhancing training stability and inference accuracy. Notably, LBLLM was trained with only 0.016B tokens on a single GPU, outperforming current leading binarization techniques in W2A4 settings. The findings were shared on arXiv, identifier arXiv:2604.19167v1, highlighting significant progress in AI model compression methods.