GNN Fraud Detection Model Trails SOTA with 0.87 AUC on IEEE Dataset

Researchers working on an explainable fraud detection GNN model have run into a performance wall. Using the widely-adopted IEEE CIS Fraud Detection Dataset—which already comes with substantial feature engineering—they constructed a heterogeneous graph embedding transaction attributes like device, transaction ID, and amount as nodes connected to transaction nodes. After training three popular GNN architectures (GCN, GraphSAGE, and GAT), all models performed similarly, yielding an average AUC of 0.87, PR-AUC of 0.52, recall at 5% of 0.57, and precision at 5% of 0.37. These numbers fall significantly short of state-of-the-art results reported in recent fraud detection literature, prompting the team to seek community input on what might be going wrong.

Potential pitfalls could include insufficient or incorrect graph construction—perhaps node/edge definitions don't capture the nuanced fraud patterns present in the data. The team may also be using a suboptimal training setup, such as improper handling of class imbalance (fraud datasets typically have less than 5% positive samples). Another possibility is that their feature engineering, while using the dataset's pre-built features, may miss domain-specific transformations or temporal dynamics that SOTA models exploit. The fact that all three GNN variants perform nearly identically suggests the bottleneck lies in the graph structure or data preprocessing rather than the model architecture itself. Addressing these issues could help close the gap and make the GNN a viable tool for explainable fraud detection in production environments.