Hyperfitting Enhances LLM Output Diversity Beyond Temperature Scaling

A recent study published on arXiv (2605.22579) explores the concept of "Hyperfitting," which refers to the enhancement of open-ended generation quality and a decrease in repetition when Large Language Models are fine-tuned to achieve nearly zero training loss on limited datasets. The findings indicate that hyperfitting differs from mere temperature scaling, as entropy-matched controls reveal that temperature scaling fails to achieve similar diversity improvements. Additionally, the research disproves the idea of static vocabulary reweighting, uncovering a dynamic mechanism for rank reordering that depends on context. A layer-wise examination identifies this phenomenon as a "Terminal Expansion" occurring in the model's final layers.