---
title: Earth Observation Foundation Models
url: https://www.emergentmind.com/topics/earth-observation-foundation-models-eofms
type: topic
---

# Earth Observation Foundation Models

Searching arXiv for recent papers on Earth observation foundation models to ground the article.
{"queries":[{"query":"Earth Observation Foundation Models remote sensing foundation models arXiv 2025 2026","limit":10},{"query":"site:arxiv.org Earth Observation Foundation Models benchmark robustness multimodal 2026","limit":10}]}
Earth Observation Foundation Models (EOFMs) are large-scale pretrained models for Earth observation data that learn general-purpose representations from massive archives of satellite imagery and are subsequently adapted to downstream tasks with limited labeled data. In the literature, EOFMs are described both as self-supervised vision models trained on large satellite archives and as multimodal models that align one or more remote-sensing modalities into a shared feature space, sometimes including language. Their scope now spans classification, segmentation, detection, regression, retrieval, change detection, captioning, and visual question answering, but the defining constraint is domain specificity: EOFMs must operate over multispectral and hyperspectral bands, SAR backscatter, thermal and elevation layers, heterogeneous ground sampling distances, irregular revisit intervals, and sensor metadata rather than over natural-image RGB alone [2604.04306] [2503.06312] [2410.16602] [2607.07758].

## 1. Definition, scope, and problem setting

An EOFM is a large-scale pretrained model designed specifically for Earth observation data. Depending on the formulation, it may ingest a single remote-sensing modality, a multitemporal image stack, or a multimodal bundle containing optical, multispectral, hyperspectral, SAR, infrared, elevation, or other derived geospatial layers. A common definition in the recent literature is that such a model is pretrained on massive unlabeled or weakly labeled EO archives, learns transferable representations, and is then adapted by linear probing, fine-tuning, or parameter-efficient tuning to a wide variety of downstream tasks [2503.06312] [2505.24528] [2607.07758].

The need for EOFMs arises from the structure of EO data itself. Remote-sensing archives are large, geographically broad, and increasingly multimodal, while labeled data are often sparse and expensive to collect. At the same time, EO signals differ substantially from natural-domain inputs: multispectral and hyperspectral sensors produce 3–200+ bands; SAR introduces active-sensor backscatter and speckle; revisit intervals range from hours to weeks; spatial resolution ranges from sub-meter to tens of meters; and auxiliary context such as geolocation, acquisition date, patch area, and sensor identity often matters materially for transfer [2410.16602] [2503.11849] [2607.07758].

Benchmark studies frame EOFMs as principally label-efficient. In one benchmark, with only 100 training images per region, a geo-aware U-Net fine-tuned from the foundation model achieved an mIoU of approximately 0.55 versus approximately 0.46 for a fully supervised U-Net, while image-level land-cover classification rose from 0.55 to approximately 0.64 and building-density regression reduced MSE by 86% relative to a scratch-trained U-Net [2406.18295]. PhilEO Bench reports the same general pattern: pretrained models such as Prithvi, SatMAE, SeCo, MAE-ViT, and a geo-aware U-Net consistently outperform randomly initialized baselines in few-shot regimes, even though classical encoder–decoder designs can still retain advantages on dense prediction tasks that require fine local detail [2401.04464].

## 2. Pretraining objectives and architectural families

The dominant pretraining families in EOFMs are masked image modeling, contrastive learning, hybrid reconstruction-plus-alignment objectives, and vision–language alignment. Masked autoencoding is especially common. A representative objective is

$$
L_{MAE} = \frac{1}{|M|}\sum_{i\in M}\|x_i - \hat{x}_i\|^2,
$$

where $M$ indexes the masked patch locations. This formulation appears in HighFM, Prithvi-style models, SatMAE-style models, and multiple benchmark summaries [2604.04306] [2401.04464].

Contrastive objectives are equally prominent, particularly for multimodal alignment and vision–language training. A standard formulation used in benchmark summaries is InfoNCE,

$$
\mathcal{L}_{\mathrm{NCE}} = -\frac{1}{N}\sum_i \log \frac{\exp(\langle v_i,t_i\rangle/\tau)}{\sum_j \exp(\langle v_i,t_j\rangle/\tau)},
$$

while TerraFM adopts a DINO-style student–teacher framework with local–global multi-crop learning and dual centering, and GeoLangBind alternates sigmoid-based contrastive learning with multimodal distillation [2505.16793] [2506.06281] [2503.06312].

Architecturally, Vision Transformers dominate. Prithvi, HighFM, DOFA, TerraFM, Copernicus-FM, and many benchmarked EOFMs use ViT backbones, while DEO employs a Swin-Transformer student and SpectralEarth-FM uses a hierarchical transformer derived from Hiera-style design [2506.20174] [2604.04306] [2602.19863] [2605.21075]. Yet EOFMs are not restricted to a single input strategy. Several recurrent architectural motifs have emerged.

**Dynamic spectral adaptation** is exemplified by DOFA and Copernicus-FM. DOFA uses a wavelength-conditioned dynamic hypernetwork to generate patch-embedding weights from central wavelengths, allowing a single Transformer to process Sentinel-1 SAR, Sentinel-2 multispectral, NAIP RGB, Gaofen, and EnMAP hyperspectral inputs, including sensors not seen during pretraining [2403.15356]. Copernicus-FM extends this idea to both spectral and non-spectral modalities, generating patch-embedding weights from spectral response metadata or variable-name embeddings and fusing Fourier-encoded metadata for geolocation, patch area, and time [2503.11849].

**Cross-modal fusion** appears in multiple forms. PyViT-FUSE uses multi-head cross-attention to fuse mixed-resolution band tokens at each spatial location, then processes the fused sequence through a pyramidal ViT stack [2504.18770]. TerraFM uses modality-specific patch embeddings for Sentinel-1, Sentinel-2 L1C, and Sentinel-2 L2A, followed by adaptive cross-attention fusion and a shared ViT encoder [2506.06281]. SpectralEarth-FM adds sensor-specific local hierarchical branches, projected-attention fusion across available sensors, and a shared hierarchical trunk to jointly handle HSI, MSI, SAR, and land-surface temperature [2605.21075].

**Objective alignment rather than reconstruction** is the core premise of DEO. Its dual-teacher framework distills from a multispectral teacher and a frozen optical vision foundation model teacher, with separate losses for multispectral global features, optical class-token features, and optical patch-token features. The model uses no pixel-reconstruction term, and the abstract reports average improvements of 3.64 percentage points in semantic segmentation, 1.2 in change detection, and 1.31 in classification tasks [2602.19863].

## 3. Data regimes, temporal structure, and multimodal coverage

EOFMs are increasingly differentiated by the breadth of their pretraining corpora. Some are sensor-specific and temporally dense, others geographically global and multimodal, and others explicitly aligned across surface, atmospheric, and language domains.

HighFM represents the temporally dense end of the spectrum. It is pretrained on approximately 2.23 TB of cleaned MSG/SEVIRI radiance imagery over the Mediterranean basin, using 11 spectral bands, 15-minute revisit, and non-overlapping $32\times32$ patches. Its central claim is that high temporal resolution matters operationally because wildfire fronts, convective storms, and flash floods can evolve on timescales of minutes to hours, and because a delay of even one polar-orbiting overpass can render alerts obsolete [2604.04306]. To exploit this regime, HighFM augments SatMAE with minute-resolution temporal encodings for year, day-of-year, hour, and minute, and its multi-timestep variant concatenates token streams from three acquisitions sampled randomly within the same hour [2604.04306].

At the opposite end are globally distributed, strongly multimodal corpora. Copernicus-Pretrain integrates 18.71 million images from Sentinel-1 GRD, Sentinel-2 TOA, Sentinel-3 OLCI, Sentinel-5P atmospheric gases, and Copernicus DEM over 312,567 cells, with a joint subset of approximately 220 K cells containing all eight modalities [2503.11849]. TerraFM pretrains on 1.53 million grid cells from Major-TOM, corresponding to 18.7 million tiles across Sentinel-1, Sentinel-2 L1C, and Sentinel-2 L2A after land-cover-aware stratification [2506.06281]. SpectralEarth-MM is explicitly designed to bring HSI into multisensor pretraining, pairing EnMAP, EMIT, and DESIS with Sentinel-2, Landsat-8/9 optical imagery, Landsat LST, and Sentinel-1 SAR over approximately 2 million globally distributed locations, 25 million georeferenced patches, and more than 40 TB of data [2605.21075].

Language-centered unification constitutes a separate axis. GeoLangBind-2M contains 2,050,983 image–text pairs spanning six EO modalities: RGB, SAR, MSI, HSI, elevation, and infrared. GeoLangBind aligns arbitrary EO channel sets into a shared language embedding space through a wavelength-aware dynamic encoder, Modality-aware Knowledge Agglomeration, and progressive multimodal weight merging. The model is explicitly positioned as enabling zero-shot classification, retrieval, caption-like description, and cross-modal integration across heterogeneous EO sensors [2503.06312].

A further expansion of the EOFM concept is the move beyond raster-only representations. A recent perspective argues that current EOFMs remain largely confined to raster modalities and underuse vector data such as OpenStreetMap, cadastral layers, parcels, and points of interest. That perspective proposes joint Spatial Representation Learning in a unified embedding space combining raster perception with vector-based reasoning, geometry, topology, and semantic relations [2606.02374]. This suggests that the EOFM problem is broadening from “multispectral image encoding” to “geospatial representation learning” in a stronger sense.

## 4. Adaptation, model composition, and evaluation protocols

Downstream use of EOFMs is organized around linear probing, full fine-tuning, and parameter-efficient methods. Benchmarking work repeatedly distinguishes frozen-backbone evaluation from full adaptation and shows that the choice materially changes conclusions [2401.04464] [2406.18295] [2607.07758].

Parameter-efficient adaptation has become a practical concern for large EO encoders. In flood segmentation, LoRA applied to the Clay EO-FM encoder updates only low-rank matrices in transformer attention projections while keeping the encoder weights frozen. With rank $r=256$, LoRA updates 26.74 M parameters, compared with approximately 124 M for full fine-tuning, raises F1 from 84.20 to 90.86, and raises IoU from 0.72 to 0.83, while full fine-tuning is reported as out-of-memory on an A100 40 GB GPU [2409.09907].

An alternative to training new monolithic models is composition of existing pretrained EOFMs. A feature-level ensemble study combines Prithvi, Hiera, and DOFA on GEO-Bench by learning convex weights over feature embeddings,

$$
F(x) = \sum_{i=1}^n w_i f_i(x), \qquad \sum_i w_i = 1,\; w_i \ge 0.
$$

The study reports that the ensemble outperformed its components on 10 of 11 tasks, and that a distilled student achieved approximately 98–99% of the ensemble’s accuracy or mIoU while reducing inference FLOPs by approximately 40% [2506.20174]. This suggests that EOFM deployment need not always mean selecting a single pretrained backbone.

Evaluation practice is now itself a substantive research topic. PhilEO Bench introduces a 400 GB Sentinel-2 dataset with building density estimation, road segmentation or density estimation, and land-cover classification under controlled n-shot splits [2401.04464]. GEO-Bench, as used in model-composition studies, covers 11 datasets spanning classification and segmentation across modalities and resolutions [2506.20174]. Copernicus-Bench organizes 15 downstream tasks hierarchically from cloud detection through surface applications to specialized atmospheric and flood tasks [2503.11849]. SustainFM assembles 16 tasks mapped to Sustainable Development Goals and argues that evaluation should include transferability, generalization, and energy efficiency rather than accuracy alone [2505.24528]. REOBench adds corruption robustness across six tasks and twelve corruption types, while recent review work emphasizes that inconsistent train/test splits, preprocessing, and reporting standards still prevent fair comparison [2505.16793] [2607.07758].

## 5. Representative systems and reported performance

Reported performance varies strongly with modality, benchmark, and adaptation protocol, but several systems illustrate the range of current EOFM capabilities.

| System | Pretraining regime | Representative reported outcome |
|---|---|---|
| HighFM_MT [2604.04306] | ~2.23 TB SEVIRI, 11 bands, 15-minute cadence | Cloud segmentation: balanced accuracy 0.831, cloud-IoU 0.737; active fire detection: balanced accuracy 0.925, fire-IoU 0.079 |
| Copernicus-FM [2503.11849] | 18.71 M images across Sentinel-1/2/3/5P and DEM | EuroSAT-S2 OA 97.9; LC100Seg-S3 mIoU 24.1; AQ-O3-S5P RMSE 789.4 |
| TerraFM-B [2506.06281] | 18.7 M tiles across S1, S2-L1C, S2-L2A | Cloud-S2 67.9% mIoU; BigEarthNet-S2 84.4% mAP; LCZ-S2 87.0% OA |
| SpectralEarth-FM-a [2605.21075] | ~2 M locations, 25 M patches, ≈40 TB | Average rank 1.4 over 10 hyperspectral tasks; average rank 3.43 on PANGAEA |
| GeoLangBind-L-384 [2503.06312] | 2,050,983 image–text pairs across six modalities | Zero-shot scene average 64.5; UCM-caption Recall@1 43.3 |

HighFM is particularly important because it shifts EOFM design toward geostationary, high-revisit data rather than high-resolution, low-revisit data. On cloud segmentation, HighFM_MT exceeds U-Net from scratch, ViT from scratch, ViT-ImageNet, Copernicus-FM, and Panopticon in both balanced accuracy and IoU; on active fire detection, it attains a balanced accuracy of 0.925, which the paper reports as +0.031 over Copernicus-FM [2604.04306].

SpectralEarth-FM demonstrates that hyperspectral information can be integrated into a general multimodal pretraining pipeline rather than remaining isolated in HSI-only models. On hyperspectral segmentation, SpectralEarth-FM-a attains state-of-the-art average rank 1.4 over 10 tasks, with examples such as EnMAP CDL 74.0 versus 70.9 for Spec.ViT-L and EuroCrops 63.5 versus 60.9 [2605.21075]. On standard EO tasks under the PANGAEA protocol, the same model achieves average rank 3.43 and reports mIoU values including 85.10 on HLS-Burns and 91.08 on Sen1Floods11 [2605.21075].

Copernicus-FM and TerraFM show the effectiveness of large unified pretraining on mainstream EO benchmarks. Copernicus-FM reports strong gains in Sentinel-1 classification, Sentinel-3 segmentation and classification, and atmospheric regression, including approximately 55% RMSE reduction on O$_3$ regression relative to the listed comparison [2503.11849]. TerraFM reports state-of-the-art results on GEO-Bench and Copernicus-Bench, with ablations attributing gains to modality augmentation, cross-attention fusion, and dual-centering regularization [2506.06281].

## 6. Limitations, controversies, and research directions

Despite rapid progress, the recent literature is explicit that no single EOFM is universally best. Benchmark and review papers repeatedly state that performance depends on the match between pretraining data, architecture, objective, downstream task, and deployment regime, and that inconsistent evaluation remains a major obstacle to reliable comparison [2505.24528] [2607.07758].

A central controversy concerns instrumentation bias. A dedicated study of Prithvi and DOFA shows that EOFM embedding spaces remain highly sensitive to sensor architecture. Using Landsat-8, Landsat-9, Sentinel-2, HLS, and Sentinel-1 over 600 geolocated Indiana samples, the study finds strong modality clustering in t-SNE, low neighborhood overlap across sensors, and very high modality-prediction accuracy from patch embeddings: a Random Forest reaches 90.7% for Prithvi and 88.7% for DOFA, while 5-NN reaches 85.6% and 91.1%, respectively [2510.03316]. This directly challenges the assumption that off-the-shelf EOFM embeddings are sensor-invariant.

Robustness under perturbation is another documented weakness. REOBench evaluates six tasks and twelve corruption types and reports that existing EOFMs suffer significant degradation, with drops ranging from less than 1% to over 20% depending on task, architecture, and corruption. MIM-based models are the least robust, especially under motion blur and noise, whereas vision–language models show enhanced robustness, particularly in multimodal tasks [2505.16793].

Trustworthiness studies extend this critique from corruption to distribution shift. SHRUG-FM augments a frozen geospatial foundation model with input-space OOD detection, embedding-space OOD detection, and predictive uncertainty. On burn-scar segmentation, the paper reports Pearson $r \approx -0.85$ between mean embedding-space NCDD and F1 across HydroATLAS deciles, and shows that discarding up to 30% of images with the highest uncertainty raises mean IoU from 0.85 to 0.91 on the retained set [2511.10370]. The same analysis links failures to low-elevation zones and large-river areas, indicating that errors are geographically structured rather than random [2511.10370].

Current future directions are correspondingly broad. Several papers advocate multimodal, resolution-agnostic pretraining across geostationary and polar-orbiting sensors, SAR, LiDAR, and hyperspectral data; physics-informed masking and reconstruction objectives; continual learning for seasonal and calibration shifts; and any-to-any sensor-conditioned architectures [2604.04306] [2607.07758]. GeoLangBind argues that language can serve as a “lingua franca” across EO modalities [2503.06312], while raster–vector work argues for unified spatial representation learning beyond pixels [2606.02374]. Taken together, these directions imply that the next generation of EOFMs will likely be judged not only by benchmark accuracy, but also by modality-aware transfer, robustness under corruption and shift, physical plausibility, uncertainty calibration, and operational usefulness in climate-sensitive decision settings [2505.24528] [2607.07758].

Source: https://www.emergentmind.com/topics/earth-observation-foundation-models-eofms