DORA Explorer: Training-Free Algorithm Boosts LLM Agent Exploration

A recent study presents DORA Explorer (Diversity-Oriented Ranking of Actions), an algorithm that operates at inference time and does not require training, aimed at enhancing exploration in large language model (LLM) agents. This paper, accessible on arXiv (2604.17244v2), tackles the challenge of LLM agents generating varied outputs for sequential decision-making tasks, which can result in limited exploration and repetitive actions. The authors note that current sampling methods, such as temperature scaling, only diversify at the token level rather than the sequence level where actions are formulated. DORA Explorer creates several candidate actions, evaluates them using sequence-level log-probability metrics, and selects an action based on a customizable exploration parameter. Initial tests in the Multi-Armed Bandit framework showed significant improvements over temperature-based sampling. The primary assessment occurred in the Text Adventure Learning Environment Suite (TALES), where traditional prompting methods failed to explore effectively, while DORA excelled. This preprint has been flagged for revision and contributes to advancements in AI and machine learning, particularly in enhancing LLM agents' decision-making without additional training.