New explainable agentic system catches conversational scams with 97.8% accuracy

Conversational scams that unfold over weeks or months—gradually building trust before asking for money or sensitive information—are a growing threat. Traditional scam detectors focus on isolated messages and miss these long-term patterns. A new paper by Adnan, Manjunath, and Khare tackles this with an explainable agentic system that maintains a summary-based memory of the entire conversation history. The system uses multiple agents (e.g., a summarizer, a detector, and an explainer) to flag suspicious interactions, providing transparency into why a conversation is flagged.

The system was evaluated on two datasets: the public LoveFraud02 corpus (83 conversations) and a new public benchmark, ConScamBench-278, covering eight scam types (e.g., romance, tech support, phishing). On isolated messages, it achieves 100% phishing recall. On long conversations, it identifies all 83 scams in LoveFraud02 and reaches 97.8% accuracy (95% CI [95.4, 99.0]) on ConScamBench-278. Two user studies (N=100 and N=45) revealed that participants often feel uncertain about suspicious conversations. After using the system, self-reported trust, confidence, and perceived need for AI scam detection all increased significantly (p < 0.001). The system also received a System Usability Scale score of 74.7, above the standard usability benchmark.