AnimeAdapter lets you generate consistent anime characters from one image

Yixuan Han's AnimeAdapter introduces a compact, modular appearance adapter for Stable Diffusion that enables fine-grained, consistent zero-shot anime character generation. Instead of relying on per-subject fine-tuning or large vision-language models, it injects visual features from a single reference image directly into the diffusion process. The key innovation is semantic-selective local attention, built on CLIP’s emergent local spatialization, which lets the model focus on specific character parts (e.g., hair, eyes, outfit) while ignoring background. To further separate appearance from spatial layout, the adapter is trained with pose-aware conditioning, allowing pose changes without breaking character identity. The result is a pretrained adapter that works out-of-the-box with any Stable Diffusion workflow—no extra training at deployment time.

The paper also presents a high-quality anime character dataset derived from curated Danbooru prompts, designed to support consistent character generation tasks. AnimeAdapter excels in practical editing scenarios such as changing expressions, outfits, or camera angles while preserving fine details like accessories and color schemes. Compared to existing methods like DreamBooth or LoRA, it requires zero per-subject training and maintains consistency across diverse generations. All code, model weights, and the dataset are promised for public release upon acceptance. This makes AnimeAdapter a practical tool for animators, game artists, and content creators who need rapid, consistent character generation without heavy compute or dataset curation.