TANDEM boosts hate speech detection by 30% with temporal awareness
New framework identifies hate speech targets with timestamps, improving F1 by 30%.
Researchers from the University of Surrey have developed TANDEM, a novel framework for detecting hate speech in long-form multimodal content (audio, video, text). Unlike black-box classifiers, TANDEM transforms detection into a structured reasoning problem by using a tandem reinforcement learning strategy where vision-language and audio-language models iteratively refine each other's predictions. This self-constrained cross-modal context stabilizes reasoning over extended sequences without dense frame-level supervision.
On the HateMM benchmark, TANDEM achieves 0.73 F1 in target identification, a 30% improvement over prior state-of-the-art, while maintaining precise temporal grounding (e.g., identifying exact timestamps of harmful content). The paper, accepted to AAAI-ICWSM 2027, notes that binary detection is robust but distinguishing between offensive and hateful content remains challenging due to label ambiguity. TANDEM's interpretable alignment offers a blueprint for transparent, actionable online safety tools.
- TANDEM uses a tandem reinforcement learning approach where vision-language and audio-language models optimize each other via self-constrained cross-modal context.
- Achieves 0.73 F1 on HateMM benchmark, a 30% improvement over prior state-of-the-art, with precise temporal grounding.
- Accepted to AAAI-ICWSM 2027; the framework enables human-in-the-loop moderation by providing granular evidence like timestamps and target identities.
Why It Matters
TANDEM makes hate speech moderation interpretable and actionable, enabling platforms to identify harmful content with precise timing and context.