Microsoft researchers crack AI guardrails with a single prompt

Microsoft researchers revealed a critical vulnerability called GRP-Obliteration that can dismantle LLM safety guardrails. Using Group Relative Policy Optimization (GRPO) in reverse, a separate 'judge' model rewards harmful outputs. Over repeated iterations, the aligned model abandons its safety training. Alarmingly, researchers noted a single unlabeled harmful prompt could be enough to meaningfully shift a model's safety behavior without harming its utility, reframing safety as a lifecycle problem.