Tested how OpenCode Works with SelfHosted LLMS: Qwen 3.5 & 3.6, Gemma 4, Nemotron 3, GLM-4.7 Flash...

An independent developer has conducted extensive benchmarking of self-hosted large language models using OpenCode, an AI-powered coding assistant. The tests evaluated seven models including Qwen 3.5, Qwen 3.6, Gemma 4, Nemotron 3, and GLM-4.7 Flash on two distinct coding tasks: creating a simple IndexNow CLI tool in Golang and developing a complex website migration map following SiteStructure strategy. All models were tested with context windows ranging from 25,000 to 50,000 tokens on an RTX 4080 GPU with 16GB VRAM using llama-server with default parameters.

The results revealed clear performance leaders among open-source models. Qwen 3.5 27B demonstrated strong performance that matched its hardware requirements well, while the newer Gemma 4 26B showed particularly promising results worth further exploration. Both models performed comparably to cloud-hosted alternatives available through OpenCode Zen for the tested tasks. The testing also provided valuable speed benchmarks, showing significant variation in inference times between different models under identical hardware conditions.

Detailed analysis of each model's behavior revealed specific strengths and weaknesses in code generation quality, with some models excelling at structured tasks while others struggled with complex architectural decisions. The comprehensive comparison provides practical data for developers deciding which self-hosted LLM to deploy for local coding assistance, balancing performance, hardware requirements, and task-specific capabilities.