Controlled Ablation Reveals Action Routing Drives LLM Self-Reflection Gains

A recent preprint available on arXiv (2608.12322) details a controlled ablation study involving six conditions that examines four elements of self-reflection in large language models (LLMs): evidence exposure, diagnostic scaffolding, taxonomy vocabulary, and action routing. This research, which centers on forecasting armed conflict, reveals that structured diagnostic inquiries do not enhance outcomes compared to unstructured reflection (F1 = 0.296 vs 0.297, p = 1.000, 95% CI [-0.041, +0.040]). Moreover, using the complete uncertainty taxonomy while limiting the action space to one generic action shows no benefit (ΔF1 = +0.008, overlapping 95% CIs). In contrast, typed action routing consistently yields positive results (F1 = 0.379 vs 0.296), with a conservative estimate of ΔF1 = +0.075 when accounting for taxonomy vocabulary. The study concludes that the improvements in LLM reasoning stem from action routing rather than diagnostic scaffolding or taxonomy vocabulary, highlighting the need for more effective self-reflection strategies in AI development.