Tree of Thoughts as Classical Heuristic Search: Formal Foundations
So, there's this new paper on arXiv, numbered 2605.28566, that presents a framework called Tree-of-Thoughts, or ToT, specifically for Large Language Models. The authors argue that the usual method of predicting tokens one by one is often limited and inaccurate. ToT creates a broader search space that allows for intermediate reasoning, which helps in exploring options and revisiting previous steps. This study brings together various ToT research from natural language processing and automated planning into a unified system. It connects LLM reasoning with classic search components, like how thoughts are represented, how new prompts are generated, and how progress is assessed. The paper also reviews current studies in this area.
Key facts
- arXiv paper 2605.28566
- Tree-of-Thoughts (ToT) framework
- Large Language Models (LLMs)
- auto-regressive token prediction is myopic
- ToT creates search space over reasoning steps
- unified taxonomy based on classical heuristic search
- state representation, successor generation, heuristic evaluation
- analyzes existing work in context of taxonomy
Entities
Institutions
- arXiv