Huawei's openPangu 2.0 hits 512K context, 28:1 sparsity

Huawei unveiled openPangu 2.0 at its developer conference on June 12, marking a major step in its AI strategy. The model family includes a Pro variant (505B total parameters, 18B activated) and a Flash variant (92B total, 6B activated), both built on a novel DSA+SWA independent layered hybrid architecture for ultra-sparse attention. This design achieves a record sparsity ratio of 28:1, meaning only a tiny fraction of parameters are active per inference. According to Huawei, this enables 2x single-card user throughput versus comparable open-source models on Ascend hardware, with a 30% improvement in hyper-node training efficiency and 50% higher throughput for 512K long-sequence training. Training consistency exceeds 99%, and the model uses a high-precision architecture (mHC, Muon, ModAttn) for more precise compute allocation.

Huawei also addressed why the Pro model has such a large total parameter count: Richard Yu explained that the company allocated significant Ascend compute resources to support other Chinese enterprises, leaving limited capacity for its own models. The focus is on latency and throughput gains rather than raw parameter count. Starting June 30, Huawei will progressively open-source core components including model weights, architecture details, inference code, and—new this version—pre-training code, post-training code, and training operators. This move aims to empower developers in the HarmonyOS ecosystem and compete with other open-source LLMs like Llama and Qwen. The two versions target different deployment scenarios: the Pro variant for high-performance server-side tasks, and the Flash variant for edge or latency-sensitive applications.