Huawei Chips Just Got 20% Faster at Running AI Chatbots

When you type a question into an AI chatbot, the reply doesn't appear all at once. The AI writes it one word at a time, and each word requires a fresh round of math. That word-by-word grind is often the slowest part of the whole experience. A pair of researchers looked at the chips that power this, specifically Huawei's Ascend chips, and noticed something wasteful: the chip has two main workers, and during this word-by-word math, one of them sat almost completely idle.

Their fix is a scheduling trick. Instead of letting the busy worker (the part built for heavy matrix math) handle everything, they rewrote the software so the idle worker picks up part of the load using simpler instructions. Both workers now compute at the same time rather than taking turns. The result: about 20% faster processing for a single word of AI output, measured on DeepSeek-V3's attention mechanism, the component that lets a model track context in a conversation.

Why does a 20% gain matter? In AI, speed translates directly into money. Faster chips mean fewer of them are needed to serve the same number of users, which lowers the cost of running chatbots, translation tools, and coding assistants. It also means snappier replies for you, especially on phones and servers built on Huawei hardware, which is widely used in China and increasingly elsewhere.

The catch is narrow. This is a software optimization for one specific chip family and one specific type of math — it won't magically speed up every AI task or every brand of hardware. It also came out of academic research, so it needs to be adopted by real products before you notice anything. Think of it as a well-engineered tune-up, not a new engine.