SAM 3 fine-tuned on synthetic tomatoes boosts greenhouse segmentation

A new study tackles one of the biggest bottlenecks in agricultural AI: the lack of annotated training data for complex greenhouse environments. Researchers from KAUST, McGill University, and other institutions developed a sim-to-real framework that combines procedural synthetic data generation with fine-tuning of the Segment Anything Model 3 (SAM 3). They created a detailed procedural model of a commercial cherry tomato greenhouse, generating a large-scale synthetic dataset under diverse viewpoints, lighting conditions, and plant morphologies. Fine-tuning SAM 3 on this synthetic data specialized its text-conditioned segmentation behavior for greenhouse crop organs (e.g., stems, leaves, fruits) while retaining the general visual prior that enables zero-shot transfer to real-world scenarios.

The results are striking: when evaluated on multiple real-world greenhouse datasets, the fine-tuned SAM 3 outperforms the base model in both segmentation performance and model confidence. This approach reduces the need for costly manual annotation and demonstrates that synthetic data can effectively bridge the sim-to-real gap in agriculture. To accelerate research, the team has publicly released the procedural model, the generated synthetic dataset, and the fine-tuned SAM 3 weights. This open-source contribution could lower the barrier for AI-driven crop monitoring, disease detection, and phenotyping in controlled environment agriculture.