Cross-Benchmark Generalization in Long-Horizon Agents

A new study on arXiv (2608.00181v1) explores cross-benchmark generalization in long-horizon reinforcement learning (RL) agents. The researchers argue that in controlled settings, policies can exploit specific patterns—like tool schemas and task templates—to gain rewards without truly developing transferable skills. They emphasize that the agent's behavior post-training is crucial and advocate for cross-benchmark transfer as an important measure. To demonstrate this, they applied a mixture-of-experts model (Qwen3.5-122B-A10B) to 363 long-horizon Model Context Protocol tasks across 27 categories, using a two-step SFT-then-RL method. Although the initial choices were influenced by Toolathlon performance, external benchmarks didn’t affect training or rewards. The model showed better performance on an unnamed benchmark, suggesting that cross-benchmark transfer might indicate actual skill development. This study sheds light on RL agents' generalization and highlights the importance of assessments beyond just in-distribution tests.