StepFun releases SFT dataset used to train Step 3.5 Flash

AI research company StepFun has taken a significant step towards transparency by publicly releasing the Supervised Fine-Tuning (SFT) dataset used to train its high-performance Step 3.5 Flash model. The dataset, a crucial component in the model's development, comprises 1.8 million meticulously curated instruction-response pairs. This collection is split between 1.2 million English examples and 600,000 Chinese examples, highlighting the model's bilingual training foundation. The release provides an unprecedented look at the high-quality data that shaped Step 3.5 Flash's renowned reasoning and instruction-following abilities, which have made it a competitive model in benchmarks against offerings from larger labs.

By open-sourcing this dataset, StepFun is contributing a valuable resource to the broader AI research and developer community. Access to such a large-scale, production-grade SFT dataset is rare, as companies often treat their training data as proprietary IP. This move allows other researchers and engineers to study the data composition, understand the types of instructions and responses that lead to strong model performance, and potentially use it to fine-tune their own models. It represents a shift towards more open and collaborative development in a field often characterized by closed training processes, enabling faster iteration and innovation across the ecosystem.