Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field

A team of researchers including Ozge Mercanoglu Sincan, Jian He Low, Sobhan Asasi, and Richard Bowden has published a comprehensive study challenging the reported progress in Sign Language Translation (SLT) AI models. Their paper, 'Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field,' re-implements key contributions from recent literature in a unified codebase to eliminate variables like different backbones, training optimizations, and metric calculations. The study, published in Computer Vision and Image Understanding (vol. 261, p.104498, 2025), reveals that many claimed performance improvements often diminish when models are evaluated under consistent, standardized conditions.

By ensuring fair comparison through standardized preprocessing, video encoders, and training setups across all methods, the researchers found that implementation details and evaluation methodologies play a more significant role in determining results than previously acknowledged. This suggests that some reported advances in gloss-free SLT—which converts visual sign language videos directly to spoken language text without intermediate gloss representations—may be overstated. The team has made their complete codebase publicly available to support transparency and reproducibility in SLT research, addressing concerns about inconsistent benchmarking in the field.