DendriCL: Single-layer spiking network matches Transformer-level in-context learning
A new spiking neural network uses dendritic dynamics to outperform deep Transformers on complex ICL benchmarks.
A new paper from Juwei Shen, Yujie Wu, and Changwen Chen introduces DendriCL, a single-layer spiking neural network (SNN) that achieves in-context learning (ICL) without attention mechanisms, deep layers, or inference-time plasticity. The key insight: the subthreshold dynamics of a single dendritic compartment inherently implement an online Widrow-Hoff LMS algorithm. By treating the dendrite as the computational substrate rather than a passive conduit, DendriCL collapses the architectural depth required for general-purpose ICL into a single layer.
On the Garg-2022 ICL benchmark, DendriCL uniquely remains stable at super-dimensional tasks where dense Transformers exhibit grokking-style instability and fail at moderate task dimensions. A linear probe recovers the reference online-LMS trajectory from the apical membrane with R²=0.93, proving the algorithm is structurally embedded in the dynamics. This work shows that ICL requires neither attention, depth, nor inference-time plasticity — a single compartment with online-LMS dynamics suffices, offering a path toward far more energy-efficient and biologically plausible AI systems.
- DendriCL uses a single-layer spiking neural network with dendritic dynamics to achieve in-context learning, bypassing the need for multi-layer Transformers.
- On Garg-2022 ICL benchmark, DendriCL remains stable at super-dimensional tasks where dense Transformers exhibit grokking-style instability and fail.
- The apical membrane dynamics exactly match online Widrow-Hoff LMS (R²=0.93), showing the algorithm is structurally embedded, not implicitly learned.
Why It Matters
This work could drastically reduce AI energy consumption by replacing deep attention layers with a single, biologically plausible spiking neuron.