A Comparative Study of Transformer and Convolutional Models for Crop Segmentation from Satellite Image Time Series
New research benchmarks six models on Sentinel-2 time series data across two regions.
A team led by Mattia Gatti and colleagues from the University of Insubria and CNR-IREA published a corrected comparative study (arXiv:2412.01944v2) evaluating six deep learning architectures for crop segmentation from Sentinel-2 satellite image time series (SITS). The models included three convolutional neural networks (3D U-Net, 3D FPN, 3D DeepLabv3) and three transformers (Swin UNETR, TSViT, VistaFormer), each with different strategies for capturing spatial and temporal dependencies in multispectral data. Experiments were conducted on the Munich (Germany) and Lombardia (Italy) datasets, which provide diverse agricultural landscapes.
The results reveal that TSViT, a transformer that explicitly models temporal dynamics through a time-series-specific attention mechanism, achieved the highest overall segmentation accuracy, slightly outperforming the strong CNN baseline 3D U-Net. VistaFormer proved most efficient in terms of computational cost while maintaining competitive performance, making it a strong candidate for large-scale or real-time applications. Notably, Swin UNETR—which treats time as an additional spatial dimension in a 3D volume—failed to surpass CNN baselines after an evaluation pipeline error was corrected in this revised version. The study underscores that temporal modeling is critical for SITS tasks: transformers designed for sequential time series data (like TSViT) outperform both CNNs and transformers that treat time spatially.
- TSViT transformer achieved best overall segmentation accuracy on Munich and Lombardia Sentinel-2 datasets, slightly beating 3D U-Net.
- VistaFormer offered the best efficiency-performance trade-off, suitable for large-scale agricultural monitoring.
- Revised analysis found Swin UNETR did not outperform CNN baselines, highlighting the importance of explicit temporal modeling.
Why It Matters
Better crop mapping from satellite data can improve agricultural monitoring, yield prediction, and food security analysis for governments and farmers.