Study: Frontier AI image models generate hateful visual stories 99% of the time

A new arXiv paper from researchers including Ye Leng and colleagues exposes a critical blind spot in AI safety: text-to-image systems can now generate entire hateful visual narratives, not just isolated offensive images. The team built HatefulStoryPrompts, a dataset of 330 multi-turn configurations drawn from 55 hateful stories across two languages and three visual styles. They tested five frontier models—including Gemini and GPT-Image—over 4,950 attempts. Every model completed over 80% of the hateful stories, with the best-performing model succeeding 99.0% of the time, meaning consistent characters and scenes are being used to convey hateful plots at scale.

Existing moderation systems failed dramatically at detecting group-level hate. On the human-labeled HatefulVisualStory dataset (969 hateful sets, 990 benign controls), dedicated safety models achieved at most 34.9% recall, while a strong vision-language model reached only 67.5%. The authors propose complementary defenses: an interaction-aware monitor that tracks multi-turn context achieved 97.3% recall for prompt-only sessions and 92.6% when users supplied the first image. Post-generation analysis of complete image groups reached 80.2%. The takeaway: as image generation evolves from single outputs to coherent visual stories, safety systems must evolve from per-image moderation to stateful reasoning over interactions and image relationships.