Data Difficulty and Generalization-Extrapolation Tradeoff in LLM Fine-Tuning

A recent preprint on arXiv (2605.12906) explores the impact of data difficulty on the supervised fine-tuning (SFT) of large language models (LLMs). The researchers conclude that there isn't a single best difficulty level; rather, as the data budget expands, the ideal difficulty tends to favor more challenging data. Through controlled synthetic experiments, they identify a balance between in-distribution generalization and extrapolation gaps. The investigation rigorously analyzes heuristics such as perplexity, difficulty, and length, highlighting discrepancies with previous studies. This research combines empirical and theoretical approaches, concentrating on strategies for data selection.