AgentStream: Evaluating Self-Evolving LLM Agents in Streaming Tasks

A novel framework named AgentStream has been launched to assess self-evolving large language model (LLM) agents within authentic streaming task settings. In contrast to earlier research that mainly relied on independent evaluations, AgentStream arranges agentic benchmarks into a customizable task stream and implements three testing scenarios—Isolated, Sequential, and Interleaved—that progressively alter the stream's scope and domain composition. This framework evaluates five representative self-evolving techniques across three leading foundation models, aiming to clarify the impact of model capability, method architecture, and streaming conditions on performance. This study fills a critical gap in comprehending how self-evolving agents adjust to varied and intricate task streams, essential for real-world applications. The paper can be found on arXiv with the identifier 2608.00155.