New AI framework MAR-12 detects harmful memes with 80% accuracy

Researchers from multiple institutions have developed MAR-12, a novel framework that tackles the complex challenge of detecting harmful humor in internet memes. Traditional multimodal classifiers often struggle with the nuanced interplay of visual cues, text, and cultural context, especially when humor and hate coexist. MAR-12 leverages Vision Language Models (VLMs) to interpret each meme through 12 distinct perspectives derived from established humor and hate theories. It then employs a role-aware soft-gated attention mechanism to dynamically weight each perspective's contribution, followed by a prototype-based classifier for final prediction. The system also synthesizes explanations using perspective-specific reasoning and learned attention weights, ensuring transparent and context-grounded justifications.

Tested on the PrideMM and Memotion datasets, MAR-12 achieved up to 80.3% accuracy for humor detection and 75.9% for hate detection, outperforming current state-of-the-art models. Both human evaluators and GPT-4-based assessments confirmed that MAR-12 produces coherent and persuasive explanations, particularly for memes where humorous and harmful cues are interwoven. Accepted for publication at AAAI-ICWSM 2027, this work represents a significant step toward explainable AI for content moderation, offering a structured reasoning framework that balances accuracy with interpretability.