Stem: A New Sparsity Module for Efficient Causal Attention in LLMs

A recent study published on arXiv (2603.06274) presents Stem, a modular sparsity solution aimed at mitigating the quadratic computational demands of self-attention in Large Language Models (LLMs) during the pre-filling phase. The researchers contend that current sparse techniques, which generally implement a uniform top-k selection across all token positions in a layer, overlook the cumulative dependencies of token information found in causal architectures. To address this, Stem introduces a Token Position-Decay strategy, which employs position-specific top-k selections within each layer to maintain initial tokens for recursive dependencies, alongside a method to retain tokens rich in information. The full paper can be accessed at https://arxiv.org/abs/2603.06274.