ProGPO: New Method to Train LLM Agents on Long-Horizon Tasks
Researchers propose Progress-conditioned Group Policy Optimization (ProGPO) to address the credit trap in training large language model (LLM) agents on long-horizon tasks. Standard group-based policy optimization suffers from sampling imbalance where repeated low-effect actions dominate, leading to all-failed rollout groups that provide no corrective signal. ProGPO uses first-visit observation coverage when all samples in a group receive zero outcome reward, breaking the self-reinforcing loop. The method is detailed in arXiv:2607.22724.
Key facts
- ProGPO stands for Progress-conditioned Group Policy Optimization.
- It addresses the credit trap in LLM agent training.
- Standard group-based policy optimization suffers from sampling imbalance.
- Repeated low-effect actions dominate high-probability regions.
- All-failed rollout groups provide no outcome-based correction.
- ProGPO uses first-visit observation coverage when all samples get zero reward.
- The method is described in arXiv paper 2607.22724.
- The paper was announced on arXiv with type cross.
Entities
Institutions
- arXiv