Segment Anything (SAM) ControlNet for Z-Image
A new open-source model lets users guide AI image generation with precise segmentation masks, trained on 200K images.
Developer NeuralVFX has open-sourced a specialized AI image generation tool called a Segment Anything (SAM) ControlNet for the Z-Image model. This tool acts as a powerful steering mechanism, allowing creators to input a segmentation mask—a map that outlines specific objects or regions—which the AI then uses as a strict guide for composing the final image. This means users can dictate exactly where elements like a person, a car, or a building should appear, bringing a new level of precision to the often-unpredictable process of AI image synthesis.
The model was trained at a resolution of 1024x1024 pixels using a dataset of 200,000 images sourced from the large-scale LAION-2B collection. While the developer notes this dataset size is on the smaller side for ControlNet training, the model reportedly maintains strong adherence to the provided control image. For best results, NeuralVFX recommends users scale their input control images to at least 1500 pixels. The release is fully operational, complete with ready-to-use code for the Hugging Face Diffusers library and a pre-configured model patch and workflow for the popular ComfyUI visual programming interface.
- Enables precise compositional control in Z-Image by using SAM-generated segmentation masks as a guide.
- Trained on 200,000 images from the LAION-2B dataset at a base resolution of 1024x1024.
- Includes production-ready code for Hugging Face Diffusers and a ComfyUI workflow for immediate use.
Why It Matters
This gives artists and designers predictable, fine-grained control over AI image composition, moving from pure prompt generation to guided, editable creation.