Arm and PyTorch Team Up to Bring AI to Tiny Devices

Microcontrollers are tiny computers that power billions of everyday devices, from your fitness tracker to your smart thermostat. They're cheap and use very little power, but they have limited memory and processing power. Until now, running AI on these devices was hard because developers had to use different tools than they use for bigger AI projects. That's changing thanks to a new collaboration between Arm, which designs the chips, and PyTorch, a popular AI software framework.

The new tool, called ExecuTorch, lets developers take AI models built with PyTorch and run them on Arm's smallest chips, called Cortex-M. This means a developer can train an AI model on a powerful computer and then easily put it onto a tiny device without rewriting everything. The AI can then do things like recognize voices, detect motion, or monitor sensors right on the device, without sending data to the cloud. This is faster, more private, and saves battery life.

ExecuTorch works by converting the AI model into a form that the tiny chip can understand, using optimized code from Arm's CMSIS-NN library. It also supports chips that have a special AI accelerator, like Arm's Ethos-U NPU, so parts of the AI can run even faster. The system automatically decides which parts of the AI run on the regular processor and which run on the accelerator, making the most of the hardware.

For you, this means future gadgets could be smarter and more responsive. Imagine a smart watch that can detect if you fall without needing your phone, or a home security camera that recognizes familiar faces without sending video to the cloud. It also means better privacy, since your data stays on the device. And because these chips are so low-power, your devices will last longer between charges. The era of tiny AI is just beginning, and it's coming to a device near you.