CustomDance turns your music into AI-choreographed 3D dances

Researchers from the University of Texas at Dallas and UT Dallas introduced **CustomDance**, a novel AI-assisted choreography system that transforms raw music and text prompts into 3D dance animations with interactive control. Unlike prior approaches that generate statistically plausible but uninspired motion, CustomDance mimics expert choreographers by breaking the process into three stages: a multimodal LLM (MLLM) first analyzes the music and high-level text prompt to extract key temporal anchors and creative cues.

Next, a multimodal retriever surfaces high-quality motion clips from a dance library, giving users concrete options aligned with local music and text. Finally, a music-conditioned diffusion in-painter seamlessly connects selected motion clips, enabling iterative, user-guided refinement supported by real-time motion dynamics visualization. The system demonstrates superior performance over competitive baselines in both quantitative evaluations and user studies, highlighting its creative utility and empowering potential for dancers, game developers, and animators.