HSS-Synth generates 237K training samples, pushes Qwen3-8B to SOTA

High-quality data is critical for large language models, but humanities and social sciences (HSS) data is scarce and costly, and its open-ended nature makes synthetic generation difficult. Moving beyond fragmented, capability-centric attempts, Ru Peng and colleagues built HSS-Synth, a subject-centric pipeline spanning 14 mainstream HSS fields. It has three stages: (1) filtering and refining web corpora into clean seed documents, evaluated by a judge model; (2) backtranslating those documents into diverse yet faithful instructions using "requirements + persona" prompts, followed by a strict Q&A alignment check; and (3) using teacher-forced answering, which feeds seed documents back into the model during response generation to anchor semantics, reduce hallucinations, and preserve tone and integrity. This process yields 237,000 high-quality instruction-tuning samples.

In extensive benchmarks, fine-tuning Qwen3-8B-Base on HSS-Synth set a new state-of-the-art across 16 benchmarks, outperforming 14 leading baselines and approaching the official Qwen3-8B model. Human preference and knowledge capability both improved without performance seesaws, and the authors demonstrated robustness and transferability across settings. The work shows that synthetic data can effectively cover free-form disciplines like ethics, history, and sociology, not just closed tasks like code generation or math. With code and data publicly available, HSS-Synth offers a practical way for researchers and enterprises to build specialized humanities-focused LLMs without massive proprietary datasets, reducing cost and data scarcity for these often-overlooked domains.