Spike-HTR's spiking transformer hits 3.5% error on IAM with just 2 timesteps

Spike-HTR is a new hybrid spiking neural network from Xiubo Liang, Jinxing Han, Yuke Li, Haoqi Zhu, Yu Zhao, and Hongzhi Wang, designed to address computational imbalances in handwritten text recognition (HTR). Traditional HTR wastes compute on background pixels and blank-dominated sequence positions. The authors' solution marries spiking neural networks (SNNs) with a transformer-based decoder, using a two-step strategy: a custom encoder named InkCoder converts a static handwriting image into a coarse-to-fine spiking input stream over timesteps, so early timesteps capture broad stroke regions and later ones sharpen details. Simultaneously, a CTC-guided length reducer prunes long blank stretches before deep sequence mixing, preserving only probable characters and uncertain positions.

With the aggressive setting of T=2 (two spiking steps), Spike-HTR trains on target data only and decodes without any language models or lexicons. The results are competitive: character error rates (CER) of 3.5/5.4 on IAM, 2.3/2.5 on LAM, and 4.2/3.9 on READ2016 (validation/test). This demonstrates that biologically inspired spiking computation—often seen as too costly or inaccurate for static image tasks—can be made both efficient and accurate for HTR by explicitly controlling spiking steps and sequence length. The authors have open-sourced the code, making the approach immediately reproducible. For practitioners, this could lead to lower-energy handwriting recognition on edge devices.