New study reveals optimal content moderation: response-only beats input filtering
Response-only blocking achieves highest usefulness while rewriting recovers 95% of blocked content.
Researchers evaluated content moderation filter placements using two end-to-end metrics: Usefulness (fraction of turns with a shown, non-harmful, relevant response) and Harmful Exposure (fraction with a shown harmful response). Response-only blocking achieved the highest filter-only Usefulness across both a human-labeled product benchmark and the public ToxicChat evaluation, while input+response blocking achieved lower Harmful Exposure. Replacing response-only hard blocking with response rewriting recovered most blocked traffic and yielded the same observed Harmful Exposure count for the selected configuration (though this equality is not an equivalence result). Probe routing substantially reduced conditional route-and-generation time relative to LLM routing at comparable measured outcomes.
- Response-only blocking achieves highest Usefulness (fraction of non-harmful relevant responses) on both proprietary and ToxicChat benchmarks.
- Input+Response blocking yields lowest Harmful Exposure but at the cost of lower Usefulness.
- Rewriting blocked responses recovers ~95% of traffic without increasing harmful exposure; probe routing reduces latency vs. LLM routing.
Why It Matters
Reveals that optimal content moderation depends on deployment constraints, not a one-size-fits-all filter placement.