LLM Code World Models Risk Missing Critical Events in Continuous Control, New Paper Shows
A study available on arXiv (identifier 2608.17956) investigates the safety of code world models derived from large language models (LLMs) in continuous control environments. An LLM generates a model that can execute actions based on the behavior of a system, which is then utilized by a classical planner. This model is deemed valid if it accurately reflects random transitions from the actual system. The authors discuss potential risks, indicating that for a critical event with a probability of r, the likelihood that N independent rollouts miss it is (1-r)^N. Their experiments reveal that an accepted mode-blind model can be manipulated, resulting in considerable regret. The findings underscore the necessity for strong verification methods in LLM-based robotics. The preprint has not undergone peer review.
Key facts
- The paper is posted on arXiv as preprint 2608.17956.
- It studies the Code World Model paradigm, where an LLM generates an executable world model.
- A classical planner uses the model to search for action sequences.
- Acceptance of a model is based on its ability to reproduce sampled transitions.
- The probability that N independent gate rollouts all miss a critical event of probability r is (1-r)^N.
- Adding an independent acceptance sample increases the exponent of this probability.
- Tests on three hybrid instruments show the planner is trapped at the mode boundary, with regret nearly equal to the entire attainable return.
- A localization budget is proved at boundary points, but discontinuous reset modes are not subject to it.
Entities
—