Centralized vs Independent Q-Learning in Tabular Multi-Agent Gridworld

A recent study available on arXiv (2601.17454) explores coordination mechanisms in multi-agent reinforcement learning through a fully tabular 8x8 predator-prey gridworld, incorporating speed and stamina limitations. This research evaluates Independent Q-Learning (IQL) and Centralized Q-Learning (CQL) over three kinematic scenarios across 10 seeds. Findings indicate that fully independent learning (IQL-IQL) consistently results in shorter episodes and greater predator returns than the fully centralized approach (CQL-CQL) in all scenarios and seeds (Wilcoxon p = 0.00195, Cliff's delta = 1.0). Additionally, asymmetric pairings (IQL-CQL) exhibit persistent coordination failures throughout the 40,000 episode training budget. A best-response assessment against static IQL-IQL predators further validates the benefits of independent learning, challenging the perceived advantages of centralized value learning by separating coordination structure from function approximation and partial observability.