InsertFuse unifies multi-category image insertion with expert distillation

InsertFuse, developed by Guangzhao Li and nine colleagues, tackles the long-standing challenge of multi-category reference-guided image insertion — placing an object from a reference image into a new scene with proper geometry, lighting, and context. The framework's core innovation is decoupling category-specific expertise learning from cross-category consolidation. Instead of jointly training a single model on all categories (which causes interference), InsertFuse first trains specialized experts for each insertion category. It then introduces Insertion On-Policy Distillation (IOPD), which queries the matched expert at states visited by the student model, effectively preserving category-specific behavior while merging capabilities into one unified network.

To strengthen spatial control, the team proposes Token-Aligned Geometry Conditioning (TAGC), which maps mask-derived geometric cues directly to the visual token grid, and Region-Balanced Flow Matching, which separately normalizes prediction errors inside and outside the insertion region — preventing background-dominated, scale-dependent supervision. A Reference CFG mechanism further isolates and amplifies the visual reference's guidance under fixed scene and geometry conditions, with IOPD transferring that enhanced supervision into the final student. On the public AnyInsertion benchmark and their own multi-category test set, InsertFuse achieves state-of-the-art results on most metrics, demonstrating robust reference fidelity and generation quality across a wide range of insertion categories.