DExperts achieves 100% explicit toxicity safety but 10x slower, study finds

Researchers Mokshit Surana, Archit Rathod, and Akshaj Satishkumar conducted a rigorous replication study of DExperts (Decoding-time Experts), a popular inference-time technique for reducing toxic outputs from large language models. The study was structured in three phases: first, establishing baseline toxicity using the RealToxicityPrompts dataset on standard GPT-2 models; second, implementing DExperts to mitigate explicit toxicity; third, stress-testing against implicit hate speech with the adversarial ToxiGen dataset.

Results show DExperts achieves near-perfect safety (100%) on explicit toxicity benchmarks, confirming its effectiveness against straightforward harmful content. However, when faced with adversarial, implicit hate speech, safety rates dropped to 98.5%, revealing brittleness. More critically, DExperts introduces a significant latency penalty—from 0.2 seconds to 2.0 seconds per generation (~10x slower)—making it impractical for real-time applications. The authors emphasize the need for more sophisticated approaches that generalize across diverse hate speech patterns without prohibitive computational costs.