New AI learns piano style from audio, generates stylized covers
This ISMIR 2026 model transfers swing or classical style from raw audio to MIDI piano arrangements.
A cross-modal framework learns implicit music styles from raw audio and applies them to symbolic music generation for piano arrangements. Inspired by BLIP-2, it uses a Querying Transformer to extract style representations from a large, pre-trained audio language model, then conditions a symbolic language model to generate piano performances. Training happens in two stages: contrastive learning aligns auditory style with symbolic expression, followed by generative modeling for arrangement. The model takes both a lead sheet and a reference audio example as inputs, enabling controllable and stylistically faithful output. Experiments report substantial improvements in style-aware alignment and music quality.
- Uses a BLIP-2-inspired Q-Former to bridge audio style and symbolic music representations
- Two-stage training: contrastive alignment followed by generative modeling for arrangement
- Applies to piano covers, style transfer, and audio-to-MIDI retrieval with improved style-aware quality
Why It Matters
Musicians can generate piano covers in any style from a reference audio clip, streamlining creative production workflows.