Rulemapping boosts LLM hate speech precision from 34% to 86%
New neuro-symbolic system cuts LLM false alarms by 2.5x on German hate speech law.
Rulemapping is a neuro-symbolic architecture that combines the flexibility of large language models (LLMs) with the auditability of symbolic legal reasoning. The system uses visual logic trees to operationalize classic legal syllogisms (if-then rules based on statute elements). In this study, focused on online hate speech classification under §130(1) of the German Criminal Code, Rulemapping acts as a 'scaffold' that guides LLM outputs while preventing 'scope drift'—the tendency of unconstrained models to classify morally offensive but legally permissible speech as illegal.
The results are striking: across four LLM families, Rulemapping maintained high recall (0.82–0.89) while achieving precision of 0.80–0.86—a 2x to 2.5x improvement over the 0.34–0.49 precision of standard zero-shot prompting. The paper, accepted at ICAIL 2026, shows that expert-authored symbolic scaffolds can enable robust legal automation that meets regulatory requirements for traceable, verifiable decision-making. This approach is particularly relevant for high-volume content moderation and mass administrative proceedings where operators must assess thousands of daily cases under strict legal standards.
- Rulemapping achieves 80-86% precision on German hate speech classification, vs 34-49% for unconstrained LLMs
- Recall remains high at 82-89% across diverse language models while preventing 'scope drift'
- Uses visual logic-tree scaffolds based on classic legal syllogisms for verifiable automation
Why It Matters
A practical blueprint for legally compliant AI content moderation—transparent, precise, and court-defensible.