HyGAE: Hybrid Advantage Estimation for VLM Agentic RL

A recent study published on arXiv (2607.23605) presents HyGAE, a novel actor-critic framework designed for reinforcement learning within Vision-Language Model (VLM) agents. This approach tackles the complexities of multi-turn decision-making by formulating theoretical foundations for both token-wise and turn-wise optimization. It introduces a hybrid advantage that caters to both goals and demonstrates that a unified critic model can effectively assess values at both levels using a suitable discount factor and learning target. HyGAE optimally balances token and turn objectives. The paper includes comprehensive evaluations across five different environments, although specific findings are not disclosed in the abstract. The goal is to enhance coherent reasoning and decision-making in settings where VLMs function as agents.