Survey Reveals Complex Scaling Dynamics in LLM Reasoning

A new survey paper on arXiv (2504.02181) examines the scaling of reasoning capabilities in large language models (LLMs). Unlike the straightforward performance gains from scaling data and model size, scaling reasoning is more complex and can sometimes degrade performance, posing challenges for alignment and robustness. The survey categorizes scaling into multiple dimensions, including input size, reasoning steps, and others, analyzing how each contributes to reasoning improvement. It explores how larger input contexts enable better utilization of information, while increased reasoning steps enhance multi-step inference and logical consistency. The paper highlights that naive scaling may introduce new issues, requiring careful consideration of model alignment and robustness.