Raven: A New Linear-Time Sequence Model with Sparse Memory Routing

Researchers introduce Raven, a linear-time sequence model that improves long-context recall by maintaining a fixed set of memory slots and updating only a selected subset via learned, input-dependent routing. This approach mitigates interference from dense state updates in state-space models (SSMs) and linear Transformers, while avoiding the hard eviction of sliding-window attention (SWA). Raven achieves high recall by preserving long-range content without the computational cost of full attention. The model is detailed in a preprint on arXiv (2607.25357).