SLED: Distillation-based location encoder runs at 128 batch, beats CLIP-style
Current location encoders need 16K-32K batch sizes; SLED needs just 128.
Location encoders compress Earth Observation data into embeddings, but existing CLIP-style methods require massive batch sizes (16K–32K) to avoid false negatives and struggle to incorporate new sensor modalities. SLED (Scalable Location Encoder via Distillation) flips this by using geospatial location as a binding modality, distilling representations from any data source into a lightweight, modular encoder. This eliminates the need for spatiotemporal coregistration and drops the batch size to as low as 128—a dramatic reduction in compute and runtime.
The researchers pretrained unimodal and multimodal SLED models on Sentinel-1 radar, Sentinel-2 multispectral, and Landsat imagery. Across 19 human-centric benchmark tasks, SLED kept pace with or outperformed existing approaches, including those trained with far larger resources. This makes scalable geospatial pretraining accessible to smaller teams and applications, from disaster mapping to urban planning, by significantly lowering the cost of building location-aware AI systems.
- SLED uses distillation with geospatial location as the binding signal, avoiding expensive CLIP-style contrastive learning and false negative issues.
- Works with batch size 128 vs 16K-32K for existing encoders, cutting compute and runtime dramatically.
- Pretrained on Sentinel-1, Sentinel-2, and Landsat; matches or exceeds state-of-the-art on 19 benchmark tasks.
Why It Matters
SLED makes geospatial AI training 100x cheaper and more flexible, unlocking scalable Earth observation models for more teams.