Multimodal NLP pipeline detects insurance fraud from call transcripts

A team led by Muhammad Shakeel Akram has published a preprint on arXiv detailing a novel multimodal hybrid pipeline for insurance fraud detection called 'Dialogue to Detection.' The system targets the critical First Notice of Loss (FNOL) stage, where fraud indicators often emerge in initial phone calls. Unlike existing text-only approaches, this framework synthesizes realistic agent-customer dialogue transcripts and two-speaker audio recordings to simulate FNOL conditions. It then processes the audio through automatic speech recognition (ASR) and speaker diarization, while downstream modules extract named entities, apply regular expressions, and leverage an LLM with retrieval-augmented generation (RAG) to compare claims against known fraud patterns. Speaker embeddings capture voice characteristics to detect cross-case voice repetition across different claims.

The integrated risk score combines these multimodal signals to flag narrative reuse, structural inconsistencies, and repeated voices while carefully managing false positives. The authors validated the dataset and each component, demonstrating stability and transfer potential to real-world claims. The approach is reproducible and addresses a major gap in fraud detection research by moving beyond private, text-only datasets. This baseline could help insurers reduce losses estimated in the billions annually and improve trust among legitimate policyholders. The paper is available on arXiv under arXiv:2606.28002.