Qwen3.5 27B vs Devstral Small 2 - Next.js & Solidity (Hardhat)

A developer's viral informal benchmark test has sparked discussion by comparing Alibaba's open-source Qwen3.5 models (27B and 35B parameters) against Mistral's popular Devstral Small 2 for real-world coding tasks. The test focused on agentic challenges within a production repository using Next.js and Solidity with Hardhat. The reviewer noted that while both the Qwen3.5 27B and Devstral Small 2 performed nearly identically in execution and token efficiency for repo work, the Qwen model produced more extensive documentation, especially at a Q6 quantization level. The larger Qwen3.5 35B model demonstrated significant refactoring capability but was noted to potentially over-engineer solutions without proper guidance.

The technical setup was rigorous, using a custom-built llama.cpp with CUDA support on an RTX 5090 GPU and a Ryzen 9 9950X CPU. The benchmark creator ran 78 specific agentic challenges to settle the comparison, finding the choice difficult. The results align with third-party analysis from Artificial Analysis, which places the Qwen3.5 27B's performance suspiciously close to models like Claude 3.5 Sonnet and DeepSeek V3.2. This test highlights the rapid closing of the performance gap between leading open-source coding models, giving developers more viable, cost-effective alternatives for complex software engineering tasks like planning, coding, and refactoring.