New AI model beats phoneme alignment benchmarks with soft dynamic programming

Forced alignment—mapping phonemes to audio timestamps—has lagged behind ASR, still relying on traditional HMM-GMM systems. To bridge this, researchers from (presumably) Bar-Ilan University introduce a fully differentiable neural approach. The model consists of an encoder with two complementary branches: one verifies phoneme identity, the other detects phoneme boundaries. This design explicitly separates the two tasks, improving robustness. The decoder is a trainable module based on differentiable soft dynamic programming, allowing the entire system to be optimized end-to-end via a novel contrastive loss that encourages clear separation between steady-state phoneme regions and transitions.

The results are striking: the method beats current state-of-the-art on hand-annotated English benchmarks, achieves strong word-level generalization, and even generalizes to languages not seen during training. This end-to-end trainability and cross-lingual performance could make forced alignment more accessible and accurate for speech recognition, linguistic analysis, and voice cloning—all without needing language-specific HMM engineering.