Researchers use LLaMa-3 and Qwen2.5 to boost music recommendations by 95%

Traditional music recommendation systems treat songs as opaque tokens, relying on collaborative filtering and ignoring the actual content. A new paper from researchers at Adobe Research and academic institutions tackles this by introducing a multimodal framework that jointly models audio, lyrics, and user engagement signals within an LLM-based sequential reasoning architecture. The team enriched the LastFM-1K dataset with three complementary inputs: audio and lyric embeddings from pretrained models, LLM-generated semantic metadata using the MGPHot annotation schema (powered by LLaMa-2-13B, Qwen2.5-7B-Instruct, and LLaMa-3-70B in zero-shot and fine-tuned settings), and listening completion ratios. They extended the E4SRec framework, testing multiple item ID encoder backbones including SASRec, BERT4Rec, and GRU4Rec, and integrated LLMs for both feature generation and reasoning.

The experimental results are striking: integrating content-based features improved over ID-only baselines by up to 95% in Recall and 79% in NDCG. However, the team also found that naive multimodal fusion does not always yield additive gains, highlighting the challenges of cross-modal integration. The work demonstrates that combining semantic, acoustic, and behavioral signals can significantly outperform traditional collaborative filtering, especially in cold-start and sparse interaction scenarios. As a contribution to the field, the authors are releasing a large-scale multimodal benchmark for music recommendation, which includes the enriched dataset and evaluation protocols. This research points toward a future where streaming services can recommend songs based on what they actually sound and feel like, not just what similar users listened to.