LoMC: New Technique Suppresses Refusals in MoE Models Without Sacrificing Performance

A team of researchers (Yan Hong et al.) has introduced LoMC (Localized Multidirectional Correction), a method to suppress refusal behaviors in routed Mixture-of-Experts (MoE) and hybrid-MoE foundation models without degrading general performance. Existing approaches either apply broad direction-based edits that perturb general-purpose computation or rely on expert-only edits that lack capacity to correct heterogeneous refusal representations. LoMC bridges this gap with a support-then-correction paradigm: it first identifies a compact set of model components (the edit support), then aggregates prototype correction directions into layer-wise vectors, and finally applies rank-one corrections exclusively within that selected support. This structural gating constraint increases correction capacity without expanding the intervention scope.

In experiments on text-only and multimodal safety benchmarks across four routed backbones, LoMC substantially improved non-refusal target-response behavior while preserving general capability under a compact intervention footprint. The technique is validated on both standard MoE models and hybrid architectures, demonstrating its versatility. By enabling precise suppression of unwanted refusals—such as overly cautious safety filters that decline valid requests—LoMC offers a scalable approach for fine-tuning model behavior post-training. This work has implications for AI safety and alignment, providing a method to reduce false positives in refusal mechanisms without sacrificing model utility.