Beyond Deep Learning: Speech Segmentation and Phone Classification with Neural Assemblies

A team of researchers has published a paper proposing a radical departure from standard deep learning for speech processing. Their framework, based on Assembly Calculus (AC), models sparse, competing groups of neurons (assemblies) that learn via biologically plausible Hebbian plasticity rules, unlike the data-hungry backpropagation used in models like Whisper or Wav2Vec2. The system works directly on raw speech by converting it into spike patterns and organizing assemblies across hierarchical timescales. Crucially, for the core tasks of finding sound/word boundaries and classifying those sounds, the model requires no traditional weight training.

Applied to benchmark tasks, the AC-based system demonstrated surprising capability. It detected phone boundaries with an F1 score of 0.69 and word boundaries with a score of 0.61 purely from its initial architecture and update rules. For classification, it achieved 47.5% accuracy on phone recognition and 45.1% on command recognition. While these numbers don't yet surpass state-of-the-art deep learning models, they prove the concept's viability. The work, submitted to Interspeech 2026, suggests a future where more efficient, brain-inspired systems could complement or even replace aspects of current speech AI, reducing reliance on massive datasets and energy-intensive training.