KL Divergence-Based Gating for Multi-Agent RL Communication

A new preprint on arXiv (2608.14559) presents an innovative approach to communication in multi-agent reinforcement learning that moves away from traditional binary methods. Instead of constant communication or using inconsistent REINFORCE gradients, agents only share information when the KL divergence between their belief distributions exceeds a set limit. Each agent maintains a belief about the hidden state of the world, calculated through a softmax function on its LSTM hidden state. This technique was evaluated using the Predator-Prey benchmark from IC3Net in two different environment sizes, with five seeds for each, and also on MPE simple_spread, comparing results against IC3Net, CommNet, and a baseline. The aim is to foster more reliable and understandable communication.