HT-Moonstone SLM beats 7x larger models in Singapore tasks

Adapting spoken language models (SLMs) to sensitive, domain-specific applications remains challenging when original training data is inaccessible and the use case demands multilingual, spoken-query interaction. In a new paper, researcher Ng Jia Sheng Jason tackles this problem by fine-tuning an open-source SLM for the Singaporean Home Team context. The approach combines LoRA (Low-Rank Adaptation) to efficiently update model weights, a surrogate text-QA dataset that guards against catastrophic forgetting, and a multi-task objective that adapts the CoBa reweighting scheme to speech tasks. To support training and evaluation, the author also built HTD-multilingual-QA, a dataset of 504,853 samples spanning Singapore's four official languages (English, Mandarin, Malay, Tamil) in both text and spoken form.

The resulting model, HT-Moonstone (5B parameters), achieves impressive results across five speech tasks including speech question answering, accent recognition, and gender recognition. It matches or outperforms SLMs up to 7x larger on most benchmarks, and attains the best performance among all evaluated models for accent and gender recognition. Notably, it loses less than 2% of its original speech QA ability—demonstrating strong resistance to catastrophic forgetting. This work shows that efficient, data-efficient adaptation of SLMs to multilingual and domain-specific contexts is feasible without sacrificing general capabilities, opening the door for similar deployments in other sensitive or resource-constrained settings.