AudioX: A Unified Framework for Anything-to-Audio Generation

A research team from institutions including HKUST and Microsoft Research Asia has introduced AudioX, a groundbreaking unified framework for 'anything-to-audio' generation. The core innovation is a Multimodal Adaptive Fusion module, a technical design that enables the model to effectively integrate and align diverse input signals—whether text descriptions, video frames, or other audio clips. This solves a key challenge in the field: creating a single, cohesive model capable of understanding and processing varied multimodal conditions. To train this ambitious model, the team constructed IF-caps, a massive, high-quality dataset of over 7 million samples curated through a structured annotation pipeline, providing the comprehensive supervision needed for such a complex task.

Benchmark results show AudioX outperforming current state-of-the-art methods across a wide range of audio generation tasks. It shows particularly strong results in text-to-audio and text-to-music generation, indicating its robust ability to follow complex, natural language instructions. The model's 'unified' nature means a single system can replace multiple specialized models, handling prompts from "generate thunder sounds for this storm video" to "create a jazz track with a walking bassline." The research has been accepted for presentation at ICLR 2026, a top machine learning conference, and the team plans to release the code and the IF-caps dataset, which could accelerate future research in multimodal AI and generative audio.