New survey maps attention-based graph neural networks evolution
Three developmental stages from graph recurrent attention to transformers systematically reviewed.
A team of researchers (Chengcheng Sun, Chenhao Li, Xiang Lin, Tianji Zheng, Fanrong Meng, Xiaobin Rui, Zhixiao Wang) has published a comprehensive survey on attention-based graph neural networks (GNNs) in the journal Artificial Intelligence Review. The paper addresses the rapid evolution of attention mechanisms applied to GNNs, which now dominate tasks in social networks, molecular biology, and recommendation systems. The authors propose a novel two-level taxonomy: the upper level identifies three developmental stages—graph recurrent attention networks, graph attention networks (GAT), and graph transformers—while the lower level categorizes architectural variants within each stage.
The survey reviews over 100 models, providing a detailed comparison table covering advantages, disadvantages, and key applications. It also highlights open challenges such as handling dynamic graphs, improving scalability to billion-node graphs, and enhancing interpretability—crucial for fields like drug discovery. This work fills a gap in the literature by systematically organizing the fast-paced advances in attention-based GNNs. Researchers can access a curated list of relevant papers via an open GitHub repository linked in the paper.
- Proposes a two-level taxonomy: three historical stages (recurrent, plain attention, transformers) and architectural subtypes.
- Includes a comprehensive model comparison table covering 100+ attention-based GNN methods.
- Identifies open issues: scalability, dynamic graphs, interpretability, and handling heterophily.
Why It Matters
Structures the chaotic GNN landscape, helping researchers quickly navigate models and identify future directions.