Local Structure-Aware Self-Attention Breaks Bottlenecks in Transformer SNNs
A new paper on arXiv presents LSFormer, a Spiking Neural Network inspired by Transformer architecture. It addresses two major issues in existing models: the failure of max pooling to preserve key features and the high quadratic complexity of global self-attention. To improve both efficiency and the way features are represented, LSFormer uses Spiking Response Pooling (SPooling) along with Local Structure-Aware Spiking Self-Attention (LS-SSA). This study aims to blend the sparse, energy-efficient benefits of Spiking Neural Networks with the Transformer framework.
Key facts
- arXiv paper 2605.13887 proposes LSFormer.
- LSFormer addresses max pooling limitations in Transformer SNNs.
- LSFormer introduces Spiking Response Pooling (SPooling).
- LSFormer introduces Local Structure-Aware Spiking Self-Attention (LS-SSA).
- Global self-attention in existing Transformer SNNs has quadratic complexity.
- LSFormer aims to reduce computational redundancy.
- The paper is classified as a cross submission on arXiv.
- LSFormer is a novel Transformer-based Spiking Neural Network.
Entities
Institutions
- arXiv