GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems

As Large Language Models (LLMs) are increasingly integrated into Multi-Agent Systems (MAS) for collaborative problem-solving, their attack surfaces expand—exposing them to vulnerabilities like prompt injection and compromised inter-agent communication. Researchers Pablo Mateo-Torrejón and Alfonso Sánchez-Macián have introduced GAMMAF (Graph-based Anomaly Monitoring for LLM Multi-Agent systems Framework), an open-source benchmarking platform designed to standardize evaluation of graph-based anomaly detection methods. GAMMAF operates through two interdependent pipelines: a Training Data Generation stage that simulates multi-agent debates across varied network topologies to capture interactions as robust attributed graphs, and a Defense System Benchmarking stage that actively evaluates defense models by dynamically isolating flagged adversarial nodes during live inference rounds.

In rigorous evaluations using established defense baselines XG-Guard and BlindGuard across knowledge tasks like MMLU-Pro and GSM8K, GAMMAF demonstrated high utility, topological scalability, and execution efficiency. Notably, the research revealed that equipping an LLM-MAS with effective attack remediation not only restores system integrity but also substantially reduces operational costs by enabling early consensus and cutting off the extensive token generation typical of adversarial agents. This framework addresses a critical gap in the field, providing a reproducible environment for training and benchmarking defense models, which is essential for secure deployment of multi-agent AI systems.