Task-Stratified Knowledge Scaling Laws for Post-Training Quantized LLMs
A recent study presents Task-Stratified Knowledge Scaling Laws to examine the impact of post-training quantization (PTQ) on various knowledge abilities in large language models (LLMs). This framework categorizes capabilities into memorization, application, and reasoning, integrating model size, bit-width, group size, and calibration set size. Tested across 293 PTQ configurations, the findings indicate that reasoning is sensitive to precision, application responds to scale, and memorization is influenced by calibration. The study emphasizes the necessity of fine-tuning specific factors in low-bit contexts.
Key facts
- Post-Training Quantization (PTQ) is critical for efficient LLM deployment.
- Existing scaling laws overlook fine-grained factors and differential impacts on knowledge capabilities.
- The framework stratifies capabilities into memorization, application, and reasoning.
- It unifies model size, bit-width, group size, and calibration set size.
- Validated on 293 diverse PTQ configurations.
- Demonstrates strong fit and cross-architecture consistency.
- Reasoning is precision-critical, application is scale-responsive, memorization is calibration-sensitive.
- Optimizing fine-grained factors is essential in low-bit scenarios.
Entities
—