UH-NAS uses LLMs to co-design neural networks for exotic hardware

Tyler King and Timothee Leleu have introduced UH-NAS (Unconventional Hardware Neural Architecture Search), a framework that leverages large language models (LLMs) to navigate the complex trade-offs of deploying neural networks on unconventional hardware. Traditional NAS methods are tied to single hardware families (e.g., digital GPUs), making cross-platform comparisons nearly impossible. UH-NAS treats hardware as a swappable backend with per-platform energy models, physical constraints, and non-ideality simulators—allowing the same search algorithm to fairly compare optical MZI, analog, or digital backends.

Tested on optical Mach–Zehnder interferometer (MZI) hardware, UH-NAS discovered architectures that were more diverse and robust against physical non-idealities than those from conventional NAS baselines. It also outperformed earlier LLM-to-NAS approaches, demonstrating the power of using LLMs as evolutionary operators in conjunction with hardware-specific constraints. The authors highlight that system prompts play a key role in guiding the LLM's search, and that architecture-hardware co-design is essential as computing platforms diverge from traditional silicon.