What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model
A new study maps how individual neurons in audio models respond to speech, music, and pitch.
Researchers from Japan's National Institute of Informatics and Tokyo Metropolitan University published "What Do Neurons Listen To?" on arXiv. Using mechanistic interpretability, they analyzed a self-supervised learning (SSL) audio model and discovered class-specific neurons that respond to shared acoustic features like speech attributes and musical pitch. These neurons provide broad coverage across tasks and directly impact classification performance, offering the first systematic neuron-level view of how audio AI models internally represent sound.
Why It Matters
This work advances AI interpretability, helping engineers build more reliable and transparent audio recognition systems.