HATL: Hierarchical Adaptive-Transfer Learning Framework for Sign Language Machine Translation

Researchers Nada Shahin and Leila Ismail have introduced HATL (Hierarchical Adaptive-Transfer Learning), a novel framework that addresses critical limitations in sign language machine translation. Traditional approaches suffer from scarce datasets, limited signer diversity, and large domain gaps between sign motion patterns and pretrained representations. HATL overcomes these challenges through progressive, dynamic unfreezing of neural network layers based on training performance behavior, combined with layer-wise learning rate decay and stability mechanisms.

The framework was evaluated on Sign2Text and Sign2Gloss2Text translation tasks using a pretrained ST-GCN++ backbone for feature extraction and both Transformer and adaptive transformer (ADAT) models for translation. Testing across three diverse datasets—RWTH-PHOENIXWeather-2014 (PHOENIX14T), Isharah, and MedASL—demonstrated HATL's robust multilingual generalization capabilities. The ADAT model achieved particularly impressive results with BLEU-4 score improvements of 15.0% on PHOENIX14T and Isharah datasets and a remarkable 37.6% improvement on MedASL.

This breakthrough represents a significant advancement in adaptive transfer learning methodology, moving beyond static approaches that often lead to overfitting. By preserving generic representations while effectively adapting to specific sign language characteristics, HATL establishes a new benchmark for accuracy in sign language translation systems. The framework's consistent performance across different models and tasks suggests broad applicability beyond the specific implementations tested.