EG-VAE unifies guitar tone transfer and removal with one VAE model

EG-VAE, developed by Yen-Tung Yeh, Yun-Ning Hung, and Yi-Hsuan Yang, tackles two related guitar processing tasks in a single framework: electric guitar tone transfer (EGTT) and tone removal (EGTR). EGTT replaces a recording's tone with that of a reference amp or pedal setup, while EGTR recovers the clean direct-input (DI) signal from a wet, effects-heavy recording. Previous systems handled these separately, often with subpar results. EG-VAE uses a variational autoencoder to split a wet recording into frame-level content (what's played) and a global tone representation (how it's shaped). A novel tone masking objective enforces this disentanglement during training, so at inference the model can remove the tone entirely or recombine content with a new reference tone.

To handle tones never seen in training, the framework adds a second training stage that smooths the tone space using variational sampling and audio-effects augmentation. This lets EG-VAE generalize beyond fixed amp settings, making it more practical for real-world pedalboards and studio chains. The researchers report that EG-VAE outperforms task-specific baselines in both objective metrics and subjective listening tests. For musicians and producers, this means one model could replace dedicated reamping hardware and manual DI re-recording — either matching a signature guitar tone instantly or stripping away effects to recover a clean track for future processing. Demos are available, and the paper is on arXiv under the audio and speech processing category.