LocalSubs: On-Device Subtitle Translation Beats Google Translate 59.2% Win Rate

A new research paper from Tsz-To Wong introduces LocalSubs, a workload-driven optimization for on-device real-time subtitle translation, specifically targeting English-to-Traditional Chinese for Taiwan. The system overcomes the limitations of standard language model optimizations (designed for long-context or high-throughput serving) by focusing on short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints. Starting from the LMT-60-0.6B model, preliminary profiling revealed that vocabulary projection becomes a dominant decode-time cost after GGUF quantization reduces Transformer block overhead. To address this, the author replaced the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer, migrated the embedding space, and adapted the model through embedding calibration followed by full supervised fine-tuning.

On a fixed 500-example subset of the OpenSubtitles2024 test set, LocalSubs achieved a 59.2% tie-excluded win rate against Google Translate under GPT-4o pairwise judging. Performance is strongest on short cues and declines as cue length increases. Preliminary Apple M2 Metal measurements on the 64k-vocabulary model showed a 1.63× speedup over the 151k-vocabulary profiling baseline, though the raw benchmark configuration is incomplete, making the latency result preliminary. This work demonstrates that domain-specific vocabulary reduction and fine-tuning can yield significant quality and speed gains for on-device translation tasks, while maintaining user privacy by keeping all computation local.