Amazon's SSFT and GFPO boost LLM reasoning with diverse traces

Amazon researchers have unveiled a new post-training approach to enhance LLM reasoning by training on multiple, diverse solution paths. Their methods—set-supervised fine-tuning (SSFT) and global forking policy optimization (GFPO)—address the problem of mode collapse that plagues standard supervised fine-tuning (SFT). By introducing global forking tokens (like <think1> through <think6>), each token prompts a distinct reasoning strategy, allowing a single model to generate varied, high-quality traces for the same problem. SSFT treats reasoning as a set of complete solution paths, using bipartite matching to assign traces to tokens, while GFPO uses reinforcement learning to select the best reasoning mode for each input.

The combined approach leads to 5-7% improvements in single-shot accuracy on standard benchmarks, including AIME 2025 and LiveCodeBench. The researchers gathered diverse reasoning traces from multiple teacher models, sampling strategies, and heterogeneous sources. This work was presented at ICLR 2026 and opens the door for more robust, multi-strategy reasoning in LLMs without sacrificing performance on individual tasks.