MemAgent: RL-Based Memory Agent for Long-Context LLMs

MemAgent, a groundbreaking agent workflow, has been unveiled by researchers, transforming the processing of long-context large language models (LLMs) through segment-based text reading and a memory overwrite strategy. This system enhances the DAPO algorithm, facilitating training through independent-context multi-conversation generation. MemAgent showcases exceptional long-context performance, effectively extrapolating from an 8K context trained on 32K text to tackle a 3.5M question-answering task with less than 5% performance loss, and achieving over 95% on the 512K RULER test. This innovation directly targets long-text tasks in an end-to-end manner, addressing the complexities of managing infinitely lengthy documents while maintaining linear complexity and preventing performance decline during extrapolation.