Pixel-TTS turns text into images for more robust speech synthesis

Traditional TTS systems treat each character as an independent token, requiring embedding expansion when adding new languages or scripts. This limits cross-lingual generalization and struggles with rare or unseen characters. Pixel-TTS, developed by researchers at the Indian Institute of Science (IISc) Bangalore and MBZUAI, takes a radically different approach: it renders input text as a 2D image and passes it through a convolutional layer to generate embeddings. By grounding text in its visual form, structurally similar characters with different Unicode encodings produce similar embeddings, enabling the model to exploit visual cues for language understanding.

In experiments, Pixel-TTS matches or exceeds the performance of conventional token-based TTS baselines while converging faster. It also demonstrates robust zero-shot generalization to unseen characters and orthographic variations across languages, a key advantage for multilingual systems. The 5-page paper (arXiv:2606.14750) presents results on multiple TTS benchmarks, showing that visual text grounding eliminates the need for embedding matrix expansion during fine-tuning. This work opens the door to truly language-agnostic speech synthesis where any script can be processed without retraining the embedding layer.