Open Source

audio.cpp's ACE-Step generates 10-min music in 60 seconds

New C++/GGML framework beats Python by 3x on music generation tasks.

Deep Dive

The audio.cpp project has evolved from a simple TTS-in-C++ library into a comprehensive native audio framework. The latest batch adds ACE-Step 1.5 Turbo/Base, HeartMuLa, Stable Audio 3 (Small Music/SFX and Medium), Mel-Band RoFormer, and HTDemucs for source separation. This pushes framework coverage to 75% (21 of 28 models). Developers can now generate music, create sound effects, separate stems, and perform speech/voice tasks — all through a unified C++/GGML pipeline without Python dependencies.

Performance benchmarks show ACE-Step Turbo generating 600 seconds of music in 60.16 seconds wall time (RTF 0.100, 9.97x real-time), compared to Python's 88.52 seconds (RTF 0.148, 6.78x real-time). However, not all models are faster — HTDemucs currently runs slower than its Python counterpart, and Stable Audio performance on warm runs is mixed. The author acknowledges this and prioritizes getting end-to-end paths into the shared framework first, with backend-specific optimizations to follow. A mem_saver mode reduces resident VRAM for server-style usage without significant speed loss.

Key Points
  • ACE-Step Turbo generates 10 minutes of music in 60 seconds (9.97x real-time), outperforming Python's 6.78x speedup.
  • HeartMuLa now supports up to 10-minute audio generation, removing its previous short generation limit.
  • Framework coverage reaches 75% (21 of 28 models), including music/SFX generation, source separation, and voice tasks.

Why It Matters

Brings state-of-the-art audio AI to native C++/GGML, enabling local, fast, and dependency-free audio pipelines for developers.

📬 Get the top 10 AI stories daily