---
title: OlmoEarth Platform Overview
url: https://www.emergentmind.com/topics/olmoearth-platform
type: topic
---

# OlmoEarth Platform Overview

The OlmoEarth Platform is a large-scale, open, end-to-end ecosystem for training, evaluating, and deploying multimodal Earth-observation foundation models. Centered around an encoder–decoder Vision Transformer architecture trained using a self-supervised “Latent MIM Lite” protocol, OlmoEarth is engineered for efficient and robust modeling of spatiotemporal, highly multimodal geospatial data. The platform integrates modular subsystems for data management, human-in-the-loop annotation, model pretraining and fine-tuning, and real-time/large-area inference, targeting a spectrum of use cases from land-cover mapping to hazard detection—especially in the environmental and humanitarian sectors [2511.13655, 2605.20804].

## 1. Platform Architecture and System Components

OlmoEarth is architected as a modular platform encompassing four key subsystems [2605.20804, §4.1; 2511.13655, §1]:

- **Data Ingestion and Preprocessing:** The ingestion pipeline supports three primary remote-sensing modalities (Sentinel-1 SAR, Sentinel-2 optical, Landsat-8 multispectral) and six auxiliary map products (WorldCover, Cropland Data Layer, SRTM elevation, canopy-height, WorldCereal, OpenStreetMap). Data is structured as tiles (e.g., 256×256 px or 2.56×2.56 km) linked to metadata (timestamp, geolocation) and managed in a global catalog with H3 indexing. Preprocessing includes radiometric calibration, cloud masking, co-registration, and time-series stacking.

- **Patch Embedding and Tokenization:** Preprocessed image cubes are partitioned into fixed-size patches and projected into a D-dimensional token space. Notably, v1.1 collapses each modality into a singular “bandset” per timestep (versus three previously for Sentinel-2), applying a learnable linear embedding (nn.Linear) instead of convolutional projections, resulting in a 1.23× throughput increase with identical weight mathematics (Appendix A, [2605.20804]).

- **Model Training Engine:** OlmoEarth employs an encoder–decoder ViT trained via masked image modeling with a LatentMIM Lite protocol, using random or structured temporal/bandset masking, patch-level and instance-level (InfoNCE) contrastive losses, and negative sampling.

- **Inference and Fine-Tuning Suite:** Pretrained checkpoints (Nano, Tiny, Base) are served via a uniform inference engine supporting zero-shot embedding extraction (kNN, linear probing) and full fine-tuning. Downstream workflows—classification, segmentation, detection—are orchestrated via PyTorch scripts utilizing AdamW/LLRD optimizers, with fine-tuning recipes such as frozen-backbone then full adaptation [2605.20804, §4.1; 2511.13655, §5].

## 2. Foundation Model Principles and Innovations

The core OlmoEarth foundation model is a spatiotemporal encoder–decoder Vision Transformer with extensible modality and temporal embeddings [2511.13655, §2]:

- **Patch Embedding (“FlexiViT”):** Supports variable patch sizes (1–8 px), adding 2D sin-cos positional, sinusoidal temporal, and learnable modality embeddings.

- **Masking Strategy:** Each input sample is subdivided into “bandsets” (e.g., Sentinel-2 10/20/60 m bands), and each is flagged as Not Selected, Encode-Only, Decode-Only, or Encode+Decode. Map modalities (e.g., OSM, WorldCover) are always Decode-Only, forcing nontrivial cross-modal reconstruction.

- **Latent MIM Lite Loss:** Rather than a learned target encoder, OlmoEarth uses a fixed, randomly initialized linear projection per modality to generate target representations, mitigating collapse and simplifying the loss computation:
  $$
  z = W_{\text{rand}} \cdot \text{patch\_pixels}
  $$
  The objective combines modality patch discrimination (local contrastive) and instance-level InfoNCE (global contrastive):
  $$
  L_{\text{total}} = L_{\text{patch}} + \lambda L_{\text{instance}}
  $$
  with λ=0.1 (v1), λ=0.05 (v1.1).

- **v1.1 Model Refinements:** Major advances in v1.1 include (1) single bandset grouping with random band dropout and nonlinear P×P×C→P×P×H→D projection (restoring spectral interaction and boosting kNN performance on benchmarks such as m-Eurosat), (2) structured temporal masking (all map tokens unmasked; temporal masking probability $p_t = 0.5$, uniformly improving mIoU), (3) refined loss with negative sample thresholding (cosine ≥ 0.999) for decode-only maps, halved instance loss weight, and larger micro-batches—improving training stability and reducing MACs by 2.9× (Sentinel-2), training GPU hours by 1.7× [2605.20804, §4.2].

## 3. Data, Labeling, and Workflow Pipelines

The data lifecycle in OlmoEarth is governed by scalable, reproducible protocols [2511.13655, §§3–4]:

- **Sampling and Cataloging:** Up to 12 modalities per tile, with systematic sampling across 120 OSM feature categories and ~285,000 global tile samples. Each tile maintains a full metadata entry (coords, time span, modality presence).

- **Batching:** Pretraining draws random tiles, time windows (2016–2024), patch sizes (1–8), crops, and sequence lengths (3–12 timesteps). All batch parameters are defined in open YAML config files [2605.20804].

- **Labeling and Annotation:** Integrated web UI (“Helios”) supports point/polygon labels, multi-round review, audit logs, and active learning loops (model-inference proposals) with dataset versioning and export (TFRecords, COCO, GeoJSON).

- **Fine-Tuning Pipelines:** Recipe involves freezing the encoder for 20% of epochs (with only decoders trained), followed by joint training. Decoders (MLP, U-Net, Faster-RCNN) are auto-selected for task type. AdamW optimizers, LR plateaus, and LLRD are supported; fine-tuning distributed over partner research datasets and tasks [2511.13655, §5; 2605.20804, §4.4].

## 4. Inference, APIs, and Integration

OlmoEarth’s deployment infrastructure is designed for low-latency real-time inference and bulk mapping at continental scale [2511.13655, §6]:

- **API Endpoints:** REST and gRPC serve /embed (tile embeddings), /infer/classify, /infer/segment, and /infer/detect with per-task outputs (class probabilities, segmentation masks, detection boxes).

- **Performance:** Single-tile inference latency is ≈100 ms (96×96×12 time-series on A100), with throughput up to 1,000 tiles/s; asynchronous mapping covers 10,000 km² in ≈20 min.

- **Integration:** Python SDK enables one-line execution for all core tasks and GIS plugins overlay maps in QGIS/ArcGIS. A CLI toolchain supports local testing and bulk cloud jobs.

## 5. Evaluation, Benchmarks, and Environmental Impact

Evaluation covers embedding-based tasks, full fine-tuning, and environmental footprints [2511.13655, §7; 2605.20804, §4.3]:

- **Embedding Evaluation:** On 13 public benchmarks (m-BigEarthNet, m-So2Sat, m-Eurosat, PASTIS, MADOS), OlmoEarth Base achieves 67.8% average kNN/LP accuracy; Tiny 63.4%, Nano 61.6%. OlmoEarth outperforms 12 other models on 15/24 tasks.

- **Fine-Tuning:** Across 10 research and 19 partner tasks, OlmoEarth is top-performing on 5/10 research and 14/19 partner leaderboards. Representative results: m-so2sat (kNN 67.7% vs. next-best 60.5%), PASTIS mIoU (50.6 vs. 44.7), AWF (87.3% vs. prior 78.0%), Nandi crop-type (82.2% vs. prior 70.6%).

- **Efficiency Gains (v1.1):** MACs per example (Base): 2.74×10¹¹ (v1) → 9.46×10¹⁰ (v1.1); 2.9× reduction. Training GPU hours (Base): 2,989 → 1,763 (1.7× reduction), enabling broader accessibility.

- **Environmental Metrics:** Total training: ~4.3 MWh (1.7 tCO₂e, 6.7 kL water); fine-tuning: ~1.2 MWh (0.5 tCO₂e, 1.8 kL water) [2511.13655, §7.3; 2605.20804].

## 6. Applications and Case Studies

OlmoEarth’s universal representations are suited to a diverse range of scientific, conservation, and operational use cases [2511.13655, §8; 2605.20804, §4.5]:

- **Land Cover / Crop Classification:** Models (zero-shot or fine-tuned) support maps at global (m-BigEarthNet, m-Eurosat), regional (So2Sat, CropHarvest, BreizhCrops), and high-resolution (AWF, Nandi) scales.

- **Temporal and Change Analysis:** Temporal masking in pretraining enhances deforestation detection and live fuel moisture estimation (Forest Loss Driver, Globe-LFMC).

- **Hazard Monitoring:** SAR-optical fusion learns robust flood detection (Sen1Floods11, Rapid Inundation).

- **Infrastructure & Marine Mapping:** Vessel, platform, solar-farm detection and regression using multimodal imagery and map fusion.

- **Partner Deployments:** Mangrove Watch (98.1% F1, up from ~95.3%), AWF (86.0% vs. 78.0%), Nandi crop-types (82.2% with field surveys), marine infrastructure (F1=89.3%), solar-farm segmentation (mIoU=86.7%).

- **NGO / Non-Profit Integration:** Data-to-insights cycles require no local GPU infrastructure; annotation to map publishing typically completes in under 24 hours [2511.13655, §9].

## 7. Open Source and Reproducibility

All code, model weights, and pretraining manifests for OlmoEarth are released under the Allen Institute OlmoEarth license [2605.20804, §4.4; 2511.13655, abstract].

| Component           | Repository Path        | Description                                                      |
|---------------------|-----------------------|------------------------------------------------------------------|
| Data loaders        | src/data              | Multi-modal ingestion, cloud masking, time-series assembly       |
| Model definition    | src/models            | ViT backbone, nonlinear projection, masking scheduler            |
| Loss functions      | src/loss              | Patch discrimination, InfoNCE instance, negative thresholding    |
| Training scripts    | train_pretrain.py     | AdamW, LR scheduler, mixed-precision                             |
| Fine-tuning scripts | src/finetune          | kNN, linear probe, full fine-tune orchestration                  |
| Config files        | configs/*.yaml        | Hyperparameters, batch sizes, dropout rates, negative thresholds |

All platform improvements (e.g., single bandset grouping, two-stage nonlinear projection, random band dropout, updated masking/loss regimes) are fully reproducible via public scripts and configuration templates [2605.20804].

Source: https://www.emergentmind.com/topics/olmoearth-platform