Clearing the Fog: Enhancing Proactive Exploration in LLM Agents

A recent paper published on arXiv presents a strategy aimed at enhancing the proactive exploration skills of large language model (LLM) agents. The research pinpoints two key obstacles that limit this capability and introduces a new methodology named \ours, which includes two main elements: Exploratory Data Construction and RL Optimization with Contrastive Signal Guidance. The first element generates exploration-rich trajectories to address the hindsight bias found in typical demonstrations, while the second uses contrastive trajectory pairs to differentiate between effective exploration and unnecessary wandering. Comprehensive experiments validate the method's efficacy and shed light on the nature of proactive exploration. The paper can be found on arXiv under the identifier 2608.14339, and the source code is available to the public.