LongCat Sparse Attention: A New Hardware-Algorithm Co-Design for Efficient Long-Context Modeling

A recent study presents LongCat Sparse Attention (LSA), a framework co-designed for hardware and algorithms aimed at overcoming system-level limitations in DeepSeek Sparse Attention (DSA). While DSA facilitates efficient long-context modeling through its Lightning Indexer, its practical use is hindered by the costly O(L^2) scoring overhead and inefficient, non-continuous memory access patterns. LSA integrates three synergistic approaches: Streaming-Aware Indexing, which organizes scattered KV entries into hardware-friendly contiguous layouts for optimized HBM access; Cross-Layer Indexing, which reduces indexing costs by reusing results across layers with the aid of cross-layer distillation; and Hierarchical Indexing, implementing a coarse-to-fine scoring method to narrow down the candidate set for queries. The research, relevant to AI technology, is accessible on arXiv under ID 2608.01662, with the abstract recently announced.