Cluster Attention Boosts Graph Neural Networks
A new method called cluster attention (CLATT) has been proposed to enhance graph machine learning models. The approach, detailed in a paper on arXiv (2604.07492), addresses the limited receptive field of Message Passing Neural Networks (MPNNs) by dividing graph nodes into clusters using community detection algorithms. Each node then attends to all other nodes within its cluster, providing a large receptive field while preserving graph-structure-based inductive biases. The authors demonstrate that augmenting MPNNs or Graph Transformers with CLATT significantly improves their performance. The paper was announced as a replace-cross type on arXiv.
Key facts
- CLATT divides graph nodes into clusters using off-the-shelf graph community detection algorithms.
- Each node attends to all other nodes in each cluster.
- CLATT provides large receptive fields while maintaining graph-structure-based inductive biases.
- Augmenting Message Passing Neural Networks or Graph Transformers with CLATT significantly improves performance.
- The paper is available on arXiv with identifier 2604.07492.
- The announcement type is replace-cross.
- Message Passing Neural Networks are currently the most popular approach to graph machine learning tasks.
- Graph Transformers with global attention lack graph-structure-based inductive biases.
Entities
Institutions
- arXiv