New Research Challenges Chain-of-Thought as Primary Mechanism in LLM Reasoning

A recent position paper on arXiv contends that the reasoning process in large language models should primarily be seen as the formation of latent-state trajectories, rather than as a reliable chain-of-thought at the surface level. The authors identify three interconnected factors and propose three competing hypotheses: H1 asserts that reasoning is driven by latent-state trajectories, H2 claims it relies on explicit surface chains of thought, and H0 argues that the observed reasoning improvements can be better attributed to generic serial computation instead of any specialized representational object. This differentiation is crucial for discussions regarding faithfulness, interpretability, reasoning benchmarks, and inference-time interventions. The paper reinterprets recent empirical, mechanistic, and survey studies within this framework, introducing compute-audited examples that decompose surface traces, latent interventions, and comparable baselines. The findings question core assumptions about evaluating reasoning abilities in AI systems.