How will we do SFT on models with opaque reasoning?
Future AI models may develop alien internal languages, making current safety training techniques like SFT impossible.
In a concerning analysis published February 2026, AI safety researchers warn that future frontier models may develop 'opaque reasoning'—internal processes that don't translate to human-interpretable language like current Chain-of-Thought (CoT). This shift could render current training methods like supervised fine-tuning (SFT) ineffective, as SFT relies on human-readable reasoning traces to teach models desired behaviors. The post speculates that opaque reasoning might emerge through RL training drift or architectural changes, creating models whose internal 'neuralese' language humans can't understand or edit. This represents a critical safety challenge: if labs can't directly train reasoning patterns, they may lose control over increasingly capable AI systems.
- Future AI models may develop 'opaque reasoning' in alien internal languages instead of human-readable Chain-of-Thought
- Current safety training methods like SFT rely on interpretable reasoning traces that wouldn't exist with opaque reasoning
- This creates a dangerous asymmetry where capabilities could advance faster than our ability to control them
Why It Matters
If AI reasoning becomes uninterpretable, we could lose our primary method for aligning and controlling increasingly powerful AI systems.