Diff-Symbo generates long-duration symbolic music from text with latent diffusion

Diff-Symbo, developed by researchers led by Zhiwei Lin, is a new text-controlled symbolic music generation system that leverages a latent diffusion model (LDM). Unlike audio-based diffusion models that generate raw waveforms, symbolic music represents notes, chords, and rhythms as discrete symbols (like MIDI), making composition more editable and controllable. The key innovation is combining an autoregressive approach with LDM to generate long-duration music without losing structural coherence. A dedicated music information encoder extracts effective control representations from text, reducing training overhead while improving controllability.

To overcome the lack of text-symbolic music datasets, the team used large language models to create a comprehensive dataset of 19,345 text templates. In experiments, Diff-Symbo surpassed strong baselines including GPT-4, MuseCoco, and the Multitrack Music Transformer (MMT) across text controllability, generation duration, and musical quality. The paper, available on arXiv, suggests that latent diffusion is a promising path for composing full-length, diverse music from simple text descriptions, making it valuable for both amateurs and professional composers.