Study Challenges Graph Neural Network Efficacy for Bitcoin Fraud Detection Under Temporal Shifts

A new preprint on arXiv, arXiv:2604.19514v1, reassesses how Graph Neural Networks (GNNs) such as GCN, GraphSAGE, GAT, and EvolveGCN perform on the Elliptic Bitcoin Dataset for detecting fraud. The findings indicate that when applying a strictly inductive evaluation without leakage, the assumption that GNNs outperform traditional feature-based methods is incorrect. A Random Forest model utilizing raw features achieved an F1 score of 0.821, exceeding all GNNs, with GraphSAGE only reaching 0.689 +/- 0.017. A controlled experiment identified a 39.5-point F1 difference due to training exposure to test-period adjacency, revealing evaluation issues. Additionally, edge-shuffle tests demonstrated that randomly connected graphs outperformed the actual transaction graph, suggesting potential misinterpretations of dataset topology in dynamic settings. Hybrid models that merge GNN embeddings with raw features yielded minimal improvements and fell short of feature-only performance. This research highlights the necessity for thorough evaluation in machine learning for financial fraud detection, especially in the rapidly changing realm of cryptocurrency transactions.