Simple Graph Heuristic Outperforms Generative Recommenders on Benchmarks

A recent study published on arXiv (2605.07125) indicates that numerous benchmarks for sequential recommendation can be solved through shortcuts, indicating that they do not necessitate the sophisticated abilities of contemporary generative recommenders. The researchers created a deliberately straightforward graph heuristic that utilizes only the last one or two items a user engaged with to gather candidates from a few-hop item-transition graph, ranking them based on item-feature similarity. Remarkably, without a sequence encoder, generative objective, or training, this heuristic either matches or surpasses many modern benchmarks. It demonstrates relative NDCG@10 enhancements of 38.10% and 44.18% on the Amazon Review Sports and CDs datasets compared to the leading competing baseline, suggesting that current evaluation methods might exaggerate the efficacy of intricate generative models.