Audio & Speech

New AI Edits Music from Simple Text Commands

New AI Edits Music from Simple Text Commands

⚡You could soon edit songs as easily as editing a document.

Deep Dive

Researchers present ProCraft, a lightweight 0.5B-parameter autoregressive codec language model that edits 30-second music excerpts end-to-end from free-form instructions, including compound requests. It supports seven operations across eight instruments, from track removal to accompaniment generation.

To preserve content outside the edit, it aligns intermediate representations with pretrained music embeddings and pitch and instrument targets, and uses gated cross-attention in every decoder layer to keep generation tied to the instruction. Trained with On-the-fly Editing-pair Synthesis, which generates paired instructions and audio combining track-level operations with effect changes, ProCraft performs competitively against task-specific systems on single-task editing and outperforms four free-text editors on compound instructions for track removal, extraction, and denoising on real (MoisesDB) and synthetic (Slakh) recordings.

In a 16-listener user study, it was rated highest on removal and denoising and on par with a 16-times larger model on extraction and instrument transfer. Ablations show representation alignment and cross-attention reduce leakage from unwanted tracks and raise editing success rates, especially on out-of-domain data.

Key Points
  • ProCraft lets you edit music by typing instructions like 'remove the vocals' or 'add a guitar'.
  • It can handle multiple edits at once, saving time compared to doing them one by one.
  • The AI is small and efficient, but only works on 30-second clips for now.

Why It Matters

This could make music editing accessible to anyone, saving time and money on production.

🔒 FREE VIP ACCESS

Read the full breakdown & get daily AI executive intelligence

Join like-minded builders, researchers, and AI professionals who read The Neural Feed daily. 100% free. No spam.

🔔 BREAKING RELEASE RADAR INSTANT ALERTS • 1-CLICK PAUSE

Get instant breaking AI release alerts by topic

Choose the AI labs, models, or chip tracks you follow. Receive fast alerts only when something major ships — plus our free daily morning briefing.

Includes full article unlock • Only major releases • Unsubscribe anytime
🎁 FREE EXECUTIVE DOWNLOAD ISSUE #21

Download This Week's 8-Page Executive AI Briefing (PDF)

Our curated executive intelligence packet: in-depth breakdowns of frontier models, market movements, plus full-color AI comics.

  • ✓ 8-page formatted executive PDF briefing
  • ✓ Frontier AI models & technical breakthroughs
  • ✓ Full-color satirical AI comic strips
Instant 1-click download • 100% free • Unsubscribe anytime
📬 Get the top 10 AI stories daily