Privacy-preserving fall detection at the edge using Sony IMX636 event-based vision sensor and Intel Loihi 2 neuromorphic processor

A collaborative research team has published a paper detailing a novel system for privacy-preserving fall detection, a critical challenge in elderly care. The system directly addresses the need for robust, real-time perception under strict hardware and privacy constraints by performing all inference at the edge. It does this by integrating Sony's IMX636 event-based vision sensor—which only outputs pixel changes (events) rather than full video frames—with Intel's Loihi 2 neuromorphic processor, a chip designed to mimic the brain's efficient, sparse processing. A custom FPGA-based interface bridges these two specialized components, creating a smart security camera proof-of-concept that processes data asynchronously and only when movement occurs.

The team explored the design space of sparse neural networks deployable on a single Loihi 2 chip, analyzing trade-offs between accuracy and computational cost. Their most efficient model, a Leaky Integrate-and-Fire (LIF) Spiking Neural Network (SNN) with graded spikes, achieved a 55x sparsity in synaptic operations for a 58% F1 score. For higher accuracy, they combined an MCUNet feature extractor with a patched inference technique and an S4D state space model, reaching the project's highest F1 score of 84%. Critically, this high-performance configuration runs with a 2x synaptic operations sparsity and a total system power draw of just 90 milliwatts on the Loihi 2 processor. This breakthrough demonstrates the practical potential of co-designing neuromorphic sensing and processing for edge AI applications where ultra-low latency, minimal energy consumption, and data privacy are non-negotiable requirements.