PatchWorld: Gradient-Free Optimization of Executable World Models
PatchWorld, a novel framework, presents a method for optimizing executable world models in text-agent settings without relying on gradients. Unlike conventional models that view these settings as partially observable Markov decision processes (POMDPs) with assumed hidden states and transition dynamics, PatchWorld generates executable Python world models from offline trajectories through counterexample-guided code repair. This technique yields symbolic belief-state programs featuring action updates that can be examined, replayed, and locally modified, thus circumventing the need for black-box next-observation predictions. In seven AgentGym environments, PatchWorld-Simple attained the highest code-based planning score among those assessed, achieving a macro success rate of 76.4% in live one-step lookahead without making any LLM calls in the world-model prediction module.
Key facts
- PatchWorld is a gradient-free framework for executable world models.
- It uses counterexample-guided code repair to induce Python world models from offline trajectories.
- The framework produces symbolic belief-state programs with inspectable action updates.
- PatchWorld-Simple achieves 76.4% macro success in live one-step lookahead across seven AgentGym environments.
- No LLM calls are invoked inside the world-model prediction module.
- The approach avoids black-box next-observation prediction.
- Text-agent environments are typically modeled as POMDPs.
- The work is published on arXiv with ID 2605.30880.
Entities
Institutions
- arXiv
- AgentGym