Foundation Model Agents Achieve Stable Cooperation in Social Dilemmas, Defying Classical Game Theory

A new research paper on arXiv (2608.03958) presents a game-theoretic analysis of foundation model agents, revealing that they consistently converge to stable cooperation in stylized social dilemmas, contrary to classical game theory predictions of mutual defection. The study introduces the concept of the 'embedded Bayesian agent' to model these AI agents, which jointly predict their own future actions alongside external observations, challenging the 'decoupled agency' assumption of classical game theory. The findings have significant implications for the integration of autonomous AI agents into social and economic systems, suggesting potential pathways for ensuring safety and cooperation. The research was announced as a new submission on arXiv, with the abstract highlighting the striking divergence between optimal planning in foundation models and classical rational behavior. The paper's theoretical framework could inform the design of AI systems that promote cooperative outcomes in real-world applications.