PROGRESS: Coverage-Guided RL for Search-Augmented LLM Agents

A novel reinforcement learning technique named PROGRESS has been introduced to enhance the training of search-augmented large language model (LLM) agents. This approach, outlined in an arXiv paper (2608.00969), tackles a significant drawback of current methods that depend on outcome-level rewards, which offer minimal guidance on search behavior and fail to utilize the agent's capacity to break down complex queries. PROGRESS employs teacher-guided coverage rewards to directly influence the generation of decomposed queries by the policy model. During the training phase, static teacher models segment intricate queries into fundamental search queries that steer the policy model's search actions. Incorporated into an R1-style training framework, PROGRESS delivers lightweight direction on query decomposition choices without requiring extensive process-level supervision. Experiments indicate that coverage-guided RL enhances overall task performance, underscoring the necessity of explicit supervision in query decomposition. The paper has been submitted to arXiv.