!! Audio on !! Audioreactive experiments with ComfyUI and TouchDesigner
A VJ's workflow combines TouchDesigner's 3D audio visuals with ComfyUI's AI rendering for live shows.
A visual artist (VJ) has created a viral demonstration of a hybrid workflow that merges the real-time 3D capabilities of TouchDesigner with the generative AI power of ComfyUI. The goal was to build a pipeline for creating unique, audio-reactive visual assets for live performances in software like Resolume Arena and Mad Mapper. The artist reversed the typical direction of asset creation, using TouchDesigner not just as an output tool but as a generator of raw material for AI processing.
The workflow is a two-step, node-based process. First, the artist uses the free software TouchDesigner to create 3D scenes that react in real-time to audio input, a common technique for digital art installations and live visuals. These generated frames are then sent to ComfyUI, the popular Stable Diffusion workflow manager. There, they are processed through a specialized, paid workflow called the "AI Render Engine" by creator Mickmumpitz, which applies AI stylization and enhancement to the 3D renders. The final output is a video where AI-transformed visuals are perfectly synced to music, creating a new genre of audio-reactive AI art suitable for professional VJing.
This experiment highlights the growing trend of artists using multiple specialized tools in tandem, treating AI not as a standalone creator but as a powerful filter or stylizer within a larger, controlled pipeline. It demonstrates a practical application where the generative randomness of AI is directed by the precise, real-time parameters of a dedicated visual tool, opening new creative possibilities for live performance.
- Uses TouchDesigner to generate base 3D visuals that react to audio input in real-time.
- Processes those visuals through ComfyUI using Mickmumpitz's specialized AI Render Engine workflow for stylization.
- Creates a final pipeline for producing AI-enhanced, audio-synced assets for professional VJ software like Resolume.
Why It Matters
It provides a practical blueprint for artists to integrate AI generation into real-time, performance-driven visual workflows.