StepFun's Step 3.7 Flash model performs better with Claude Code than Hermes
New benchmarks show Claude Code beating Hermes on StepFun's latest flash model.
Deep Dive
According to StepFun's blog, their Step 3.7 Flash model performed better when run with CC than with Hermes. The Reddit poster speculates older Hermes versions may have been used. Separately, on artificialanalysis.ai, a comparison of OpenCode and ClaudeCode showed OpenCode performing better on certain tests.
Key Points
- Step 3.7 Flash model shows better performance with Claude Code (CC) than Hermes, though older Hermes versions may have been used.
- OpenCode outperformed ClaudeCode on certain benchmarks from artificialanalysis.ai.
- The results highlight the importance of version parity and the growing competition in AI-powered coding assistants.
Why It Matters
Developers must track model-assistant pairings carefully as open-source alternatives like OpenCode challenge proprietary tools.