Huawei Ascend 910 field study reveals 8 critical flaws for large-model inference

A new field study published on arXiv (arXiv:2607.08215) by Zheng Yu provides a rare, detailed look at the real-world challenges of deploying large-model inference on non-GPU AI accelerators. Using a 16-device Huawei Ascend 910 system with CANN and vLLM-Ascend, the study ran two demanding workloads: an LLM-as-a-judge safety pipeline based on the W8A8 MoE (Mixture of Experts) model DeepSeek-V4-Flash, and a multimodal medical vision-language benchmark using DeepSeek-V4-Flash-Vision for MMMU and MMMU-Pro. The results are sobering: making these workloads reliable required 12 source-level patches to the vendor inference plugin, disabling several high-throughput features to avoid numerical errors, and adding operational safeguards for recurring device-level failures.

The study systematically categorizes the platform’s limitations into eight areas: incomplete operator and feature support, fragile parallelism, numerical faults in low-level kernels, immature graph compilation, unstable advanced features, limited scalability, weak observability, and ecosystem fragmentation. For each category, the author presents specific symptoms, evidence, and likely causes. Despite extensive workarounds, the system still struggled with concurrency and batch performance. The paper concludes that while non-GPU accelerators are increasingly promoted as alternatives to CUDA-based GPUs, the engineering cost—in patches, feature sacrifices, and operational overhead—remains poorly documented and unexpectedly high. This serves as a cautionary reference for any team evaluating similar hardware for large-model inference.