FullDiT music AI beats commercial systems on 15 of 18 metrics

A team of 16 researchers from academia and industry (led by Yunjia Li) published a new music generation framework called FullDiT, detailed in arXiv paper 2608.08787. FullDiT is a conditional Diffusion Transformer (DiT) that addresses a critical flaw in hybrid music generators: the acoustic renderer is trained on clean codec tokens but deployed on imperfect predictions from an autoregressive language model, causing codec-interface exposure bias. Instead of treating rendering as simple reconstruction, FullDiT performs full-context generation from imperfect discrete tokens, fusing eight frame-aligned RVQ streams with independently encoded captions and lyrics, and applying non-causal self-attention over the complete acoustic latent sequence.

Two key innovations drive its gains. First, Error-Matched Distractor Conditioning (EMDC) matches per-codebook replacement rates to teacher-forced top-1 error rates during training, sampling near-miss tokens from cosine-KNN neighborhoods without changing the acoustic target. Second, four-way classifier-free guidance (4-CFG) independently scales codec, lyric, and caption guidance at inference. Ablations show EMDC improves ViSQOL by 0.77 under synthetic corruption, and full-song context plus renderer-side text conditioning add further gains. The complete system outperforms five commercial systems on 15 of 18 automatic metrics and ranks among the top three on the Artificial Analysis Music with Vocals Leaderboard. A demo page is available, though no commercial release has been announced yet.