Model Discovery Agent: LLM-Bayesian Framework for Data-Efficient Causal Discovery

I came across a new preprint on arXiv (2608.09696) that introduces something called the Model Discovery Agent (MDA). This innovative framework combines large language models (LLMs) with Bayesian inference to uncover world models with minimal changes. The MDA suggests possible structures using an LLM and applies sequential Monte Carlo (SMC) for analyzing parameters and structures. It also uses simulation-based inference (SBI) for complex likelihoods and value-of-information (VoI) to plan experiments. Operating in an M-open environment, the MDA can recognize when the true model doesn’t fit the current hypotheses and adapts accordingly. This approach addresses data efficiency challenges in learning causal models, which are crucial for predicting untested interventions.