Swiss Supreme Court Tests Heretic to Fix Over-Alignment in LLMs

The Swiss Federal Supreme Court is taking a novel approach to a common frustration: LLMs that refuse perfectly legitimate requests. Instead of banning such models, the court is evaluating Heretic, a method or model designed to mitigate over-alignment—the tendency of AI systems to overly restrict outputs even for lawful queries. A paper published by the court's researchers, 'Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts,' investigates techniques like abliteration, which removes refusal behaviors, and specifically tests Heretic in Section 5.2. The results are favorable, suggesting Heretic could help the court deploy LLMs for tasks like analyzing case law across German, French, Italian, and Romansh without incurring false refusals.

This move is significant because it shows a high-stakes legal body adopting advanced AI customization rather than defaulting to safer but more limited models. The court's paper highlights that over-alignment is especially problematic in multilingual legal contexts where nuanced requests may be incorrectly flagged as sensitive. By evaluating Heretic, the court aims to balance safety with utility—ensuring LLMs assist in criminal law proceedings without hindering legitimate access to legal information. This could set a precedent for other judicial systems grappling with AI's refusal behaviors, proving that targeted fine-tuning (including abliteration) can be responsibly applied in professional environments.