Link Prediction Models Learn Trivial Heuristics Due to Batch Normalization
A new study reveals that popular link prediction models in Graph Neural Networks (GNNs) can learn a trivial mini-batch dependent heuristic, enabled by batch-normalization layers, to solve edge classification tasks. This finding challenges the assumption that GNNs trained for link prediction learn a representation consistent with node classification. When correcting for this heuristic, the network's representation aligns better with node-class relevant features, suggesting improved learning of underlying graph properties. The research indicates that standard link prediction training may overestimate models' ability to learn generalized representations.
Key facts
- Prior work on node classification showed GNNs can learn transferable representations across graphs.
- For a fixed graph, GNNs trained for link prediction were expected to learn a representation consistent with node classification.
- The study found that popular link prediction models can learn a trivial mini-batch dependent heuristic.
- Batch-normalization layers enable this heuristic for edge classification tasks.
- Correcting for the heuristic increases alignment of network representation with node-class relevant features.
- The findings suggest standard link prediction training may overestimate link predictors' ability to learn generalized representations.
- The research is published on arXiv with ID 2604.25978.
- The study focuses on Graph Neural Networks.
Entities
Institutions
- arXiv