What Do Temporal Graph Learning Models Learn?

A new research paper titled "What Do Temporal Graph Learning Models Learn?" by Abigail J. Hayes, Tobias Schumacher, and Markus Strohmaier provides a critical, systematic evaluation of the capabilities of temporal graph learning models. The work addresses growing concerns about the reliability of benchmark results in the field, where simple heuristics have been surprisingly competitive. The researchers methodically tested eight prominent models on their ability to learn and reproduce eight fundamental characteristics of temporal graphs. These characteristics spanned three categories: structural (like graph density), temporal (like recency of interactions), and edge-formation mechanisms (like homophily, where similar nodes connect).

Using both controlled synthetic datasets and real-world data, the analysis revealed a mixed and often limited picture of model performance. While models successfully captured some characteristics, they consistently failed to reproduce others, highlighting significant gaps in what these supposedly advanced systems actually learn from data. The findings challenge the assumption that strong benchmark performance equates to a deep, general understanding of temporal graph dynamics. The authors conclude that their results expose important limitations in current evaluation protocols and advocate for more interpretability-driven assessments in graph learning research to ensure models are learning meaningful patterns rather than exploiting dataset artifacts.