XFP: Dynamic Weight Quantizer for Efficient LLM Inference

XFP has introduced a groundbreaking weight quantization method for large language models that redefines existing strategies. This approach eliminates the necessity for manual bit-width selection and calibration data, instead establishing quality metrics grounded in cosine similarity for individual channels. It maintains strict criteria for attention and shared experts while allowing greater flexibility in routed-expert mixture-of-experts scenarios. The technique independently handles codebook size, outlier allocation, and layer arrangement, utilizing a unique configuration that separates weight matrices into sparse outliers and dense indices. During testing with Qwen3.5-122B-A10B, it achieved a remarkable 138 tokens per second, outperforming Marlin INT4 by 49%.