LuxSQA uses TTS to train spoken QA for low-resource Luxembourgish

Spoken Question Answering (SQA) has largely focused on high-resource languages like English, leaving low-resource languages like Luxembourgish—spoken by only about 400,000 people—without viable speech-LLM solutions. A team of researchers from FBK and other institutions tackled this by asking: can text-to-speech (TTS) generate task-specific training data for Luxembourgish SQA without requiring expensive human-recorded QA (question-answer) corpora? They started from existing text-based QA resources, translated questions into Luxembourgish, and synthesized spoken questions using three different TTS systems: Meta's MMS-TTS, Alibaba's Qwen3-TTS, and the OmniVoice family. They built two training datasets: a single-source corpus of 48,000 questions and a multi-source mix of approximately 230,000 questions. Their architecture is a parameter-efficient SLAM-style model: a frozen Whisper encoder processes audio, a learned projector connects to frozen multilingual LLM backends, and LoRA adapters fine-tune the LLM for the SQA task.

The evaluation on the LLAMA-LB-Test benchmark—using two conditions with real Luxembourgish speakers—revealed two key findings. First, multi-source synthetic training (combining all TTS systems) consistently outperformed single-source setups, and voice-design strategies (varying speakers and prosody) further boosted performance. Second, and more surprisingly, no-reference TTS quality scores (how 'natural' the synthetic speech sounds) did ​not monotonically predict downstream QA accuracy. This means that synthetic speech must be evaluated as task-specific training data, not merely as natural-sounding audio. The work demonstrates that TTS-augmented data can effectively bootstrap SQA for extremely low-resource languages, opening a path to voice-based AI assistants for languages that lack large recorded speech corpora.