ICRL4AHT Benchmark Tests In-Context Reinforcement Learning for Ad-Hoc Teamwork

A new benchmark named ICRL4AHT has been developed by researchers, based on a high-throughput JAX implementation of Overcooked-V2, to assess In-Context Reinforcement Learning (ICRL) within Ad-Hoc Teamwork (AHT) contexts. This benchmark features a varied suite of teammates that includes both RL and heuristic strategies, allowing for controlled training and testing variations. It also offers a reproducible pipeline for generating teammates, collecting learning histories, constructing datasets, and conducting online evaluations across multiple episodes. The study tested Algorithm Distillation (AD) and Decision-Pretrained Transformer (DPT) over millions of transitions. Findings indicate significant drawbacks: unlike their performance in single-agent settings, these methods struggle to coordinate with unfamiliar partners, underscoring the difficulties of implementing ICRL in AHT.