LLMPhy Framework Integrates LLMs with Physics Engines for Physical Reasoning

Researchers have introduced LLMPhy, a black-box optimization framework that combines large language models (LLMs) with physics simulators to address parameter identification in physical reasoning. The framework bridges textbook knowledge from LLMs with world models in physics engines, enabling digital twin construction via latent parameter estimation. LLMPhy decomposes this into continuous parameter estimation and discrete scene layout estimation, iteratively prompting the LLM to generate candidate solutions. The approach targets real-world applications like collision avoidance and robotic manipulation, where identifying parameters such as mass and friction is crucial. The paper is available on arXiv under identifier 2411.08027.