Google DeepMind finds SFT, not RL, drives Gemini safety properties

In a new research update from Google DeepMind's Language Model Interpretability team, researchers report a counterintuitive finding: most safety-relevant properties in Gemini models appear to originate from the combination of pretraining and supervised fine-tuning (SFT), not from later stages like reinforcement learning (RL). The team performed SFT using the Gemini mixture on pretraining-only versions of Gemini 3.1 Pro and Gemini 3 Flash, then compared these post-SFT models to the production versions across multiple safety benchmarks. The main result is that the SFT-only models and production models are remarkably similar across evaluations, with error bars overlapping at 95% confidence intervals. This was unexpected, as the team initially assumed RL post-training played a larger role.

The benchmarks included ODCV (from arxiv.org/abs/2512.20798), alignment evals (a version of Petri modified to be single-prompt with alignment dilemma problems), safety evals measuring over-refusal and unsafe response rates, a reward hacking environment where models optimize against a timing script, and analysis of 50k anonymized user prompts from AI Studio. The similarity persisted even on reward hacking and user log evaluations. The team cautions that this may be specific to Gemini and could change with future versions, but the finding suggests SFT is a high-leverage intervention point for model safety. They plan to invest more in SFT-based safety techniques going forward.