EnvACE: World Rehearsal for Agentic Reinforcement Learning
A new method for training large language model agents, called EnvACE, replaces external environment interaction with world rehearsal. The policy alternates between acting and rehearsing: it generates a tool call, then simulates the environment's response, and conditions subsequent decisions on that rehearsed response. Both roles are optimized end-to-end using task-success rewards. This approach internalizes environment dynamics in the model's parameters, creating an agent world model that supports decision making. The method was evaluated on BFCL-v4, tau^2-Bench, VitaBench, and Fi
Key facts
- EnvACE is an agentic reinforcement learning method.
- It replaces external environment interaction with world rehearsal.
- The policy alternates between acting and rehearsing.
- It generates a tool call and then simulates the environment's response.
- Both roles are jointly optimized end-to-end using task-success rewards.
- The policy internalizes the relationship between actions and environment responses.
- It yields an agent world model that directly supports decision making.
- Evaluated on BFCL-v4, tau^2-Bench, VitaBench, and Fi.
Entities
—