LLM Agents Struggle in Sequential Games; Memory Framework Proposed

A recent paper on arXiv (2608.03420) examines how large language models (LLMs) perform in sequential decision-making scenarios, focusing on fully-observable two-player zero-sum games. The findings reveal that LLMs tend to underperform in straightforward games like tic-tac-toe and Connect Four, frequently losing to opponents utilizing Monte Carlo Tree Search (MCTS). Notably, modifications that change the game's surface appearance without affecting the game tree do not significantly impact performance, indicating that the performance gap is not merely a result of recalling memorized strategies. To tackle this issue, the authors propose an agentic framework augmented with an experience memory tailored for sequential contexts, addressing typical challenges like credit assignment. The abstract highlights the framework's potential to enhance LLM performance in these tasks, contributing to a deeper understanding of LLM limitations in dynamic settings.