From Hallucination to Articulation: Language Model-Driven Losses for Ultra Low-Bitrate Neural Speech Coding

Researchers have developed a new method to improve the quality of heavily compressed speech. When audio is compressed too much, AI decoders often invent incorrect sounds, a problem called 'phoneme hallucinations.' The new technique uses language models to guide the decoder, comparing the output to inferred or known text. This significantly improves the clarity and meaning of the speech while keeping file sizes extremely small, outperforming previous methods.