hls4ml brings Spiking Neural Networks to FPGAs with 34μs latency
New extension deploys PyTorch SNNs on FPGAs for real-time inference at 34μs.
Spiking Neural Networks (SNNs) have long been the domain of asynchronous neuromorphic chips, but a new arXiv paper by Barry Dillon changes the game for conventional synchronous FPGAs. The work extends hls4ml—a popular open-source toolkit for deploying machine learning models on FPGAs—to support clock-driven SNN inference. Using a quantized dense SNN trained in PyTorch on the Heidelberg Spiking Digits dataset, the system achieves inference latencies of roughly 34 microseconds. The entire pipeline, from training to synthesis, is validated through software comparisons, HLS C simulation, and Vivado synthesis reports.
This integration is a significant step for real-time scientific inference systems that rely on FPGAs rather than dedicated neuromorphic processors. By enabling streamlined optimization, synthesis, and deployment of SNNs, hls4ml now bridges the gap between conventional FPGA workflows and the temporal processing advantages of SNNs. The result is a practical path to low-latency, event-driven AI on widely available hardware—opening up new possibilities for edge computing, particle physics triggers, and autonomous systems.
- Enables clock-driven SNN deployment on FPGAs using PyTorch-trained models via hls4ml.
- Achieves 34μs inference latency on the Heidelberg Spiking Digits dataset with a quantized dense SNN.
- Validated through software reference comparisons, HLS C simulation, and Vivado synthesis reports.
Why It Matters
Brings neuromorphic computing to mainstream FPGA workflows for ultra-low-latency, real-time AI at the edge.