Qwen-3-Swallow-8B Japanese reasoning model fails to beat English baselines

Reasoning Language Models (RLMs) have achieved peak performance when reasoning in English, largely because English dominates their training data. This creates a barrier for non-English users who want interpretable reasoning traces in their native language. Researchers led by Yuu Jinnai explored whether a model could be trained to reason in Japanese without sacrificing accuracy. They used Qwen-3-Swallow-8B, a Japanese-continually-pretrained variant of Qwen-3-8B, and fine-tuned it with Group Relative Policy Optimization (GRPO) to produce reasoning chains in Japanese.

The results were mixed: while the Japanese-reasoning model successfully generated reasoning traces in Japanese, its performance on coding, math, and science benchmarks was at best comparable to strong English-reasoning baselines—not a clear improvement. More concerningly, on Japanese cultural benchmarks, the model actually underperformed relative to baseline models. This suggests that simply switching the reasoning language does not automatically unlock cultural competence, and that further work is needed to align reasoning and cultural knowledge in non-English RLMs.