SongFormer: Scaling Music Structure Analysis with Heterogeneous Supervision

A research team led by Chunbo Hao has introduced SongFormer, a novel AI framework designed to overcome the long-standing data bottleneck in music structure analysis (MSA). MSA, the task of identifying sections like verses and choruses in songs, is crucial for music understanding and controllable generation but has been hampered by small, inconsistent training datasets. SongFormer tackles this by learning from 'heterogeneous supervision'—it can fuse multiple, imperfect sources of structural labels that may be partial, noisy, or use different labeling schemas. A key innovation is its use of a learned source embedding, which allows the model to understand and reconcile these conflicting annotations during training.

To support this scalable approach, the team created and open-sourced two major resources: SongFormDB, the largest MSA corpus to date with over 14,000 songs spanning diverse languages and genres, and SongFormBench, a curated 300-song benchmark verified by experts. On this new benchmark, SongFormer achieved a new state-of-the-art in strict boundary detection (measured by HR.5F) and the highest functional label accuracy. Notably, it surpassed not only established MSA baselines but also the general-purpose multimodal model Gemini 2.5 Pro on these specific music analysis tasks, all while maintaining computational efficiency. The entire project—including the model code, SongFormDB dataset, and SongFormBench—has been made publicly available, providing a significant boost to research in computational musicology and AI-powered music tools.