Text2Sign runs sign-language video generation on a single L4 GPU

Text2Sign, developed by Ruize Xia, is a diffusion-based baseline for generating sign-language video from text, designed to run entirely on a single NVIDIA L4 GPU. The model combines a frozen vision-language text encoder with a 3D encoder-decoder backbone, employing factorized spatiotemporal attention to cut the computational cost of full-video attention. Xia compares convolution-only and transformer-style backbones, as well as frozen versus task-specific text encoders, and finds that frozen conditioning improves short-budget validation loss but has limited prompt-specific separation. The system produces 32-frame, 64×64 pixel clips at 2.54 frames per second (12.6 seconds per clip) with peak inference memory of only 3.12 GB.

On a compact evaluation slice of How2Sign, the best checkpoint achieves an SSIM of 0.2403 ± 0.0238, a PSNR of 15.11 ± 0.42 dB, and remarkable temporal consistency of 1.0000 (perfect). A longer-run checkpoint reaches a validation loss of 0.00999. However, the system is explicitly positioned as a research baseline rather than a production tool—it is restricted to low-resolution, short clips and has not undergone expert linguistic evaluation. The code is open-sourced, making it a practical starting point for academic labs exploring text-to-sign generation without expensive multi-GPU clusters.