Chain-of-Thought Prompting Mechanism Revealed via Information Flow Tracing
A recent study published on arXiv (submitted July 2025) explores the enhancement of model reasoning through Chain-of-Thought (CoT) prompting by examining the flow of information during decoding, projection, and activation stages. The researchers discovered that CoT serves as a pruner for decoding space, employing answer templates to steer the output generation process, with a stronger adherence to templates linked to improved performance. Interestingly, CoT influences neuron activation differently based on the task: it diminishes activation in open-domain tasks while amplifying it in closed-domain ones. This research offers a framework for mechanistic interpretability aimed at creating more effective prompts. The associated code and data are available to the public.
Key facts
- Study submitted to arXiv in July 2025.
- Analyzes CoT's operational principles by reverse tracing information flow.
- CoT serves as a decoding space pruner using answer templates.
- Higher template adherence correlates with improved performance.
- CoT reduces neuron activation in open-domain tasks.
- CoT increases neuron activation in closed-domain tasks.
- Framework enables targeted CoT interventions for prompt design.
- Code and data released at provided URL.
Entities
Institutions
- arXiv