Running Qwen 3.5 0.8B locally in the browser on WebGPU w/ Transformers.js

Alibaba's Qwen AI team has launched its Qwen 3.5 Small model family, a series of compact, multimodal AI models specifically engineered for on-device and edge computing applications. Available in four parameter sizes—0.8B, 2B, 4B, and 9B—these models represent a strategic push toward efficient, locally-runnable AI that doesn't require constant cloud connectivity. In a significant technical demonstration, a developer has successfully implemented the smallest 0.8B parameter variant to run inference directly within a web browser. This is achieved by leveraging two key web technologies: WebGPU, which provides low-level access to a device's graphics hardware for parallel computation, and Transformers.js, a JavaScript library that ports popular AI model architectures to the browser environment.

The demo, hosted on Hugging Face Spaces, showcases a fully client-side multimodal application where the model processes inputs locally. The primary performance constraint is currently the vision encoder component, which handles image understanding. This breakthrough demonstrates a clear path toward privacy-preserving AI applications, as data never leaves the user's device, and reduces latency and server costs. It signals a near-future where lightweight but capable AI features can be embedded directly into websites and web apps, from intelligent document parsing to real-time image captioning, without the need for backend API calls. The availability of the full model family on Hugging Face allows developers to experiment with the size/performance trade-off for their specific use cases.