Earth Observation Foundation Models

• EOFMs are large-scale pretrained models designed specifically for Earth observation data, integrating multispectral, SAR, and temporal modalities.
• They enable label-efficient transfer learning for tasks such as classification, segmentation, detection, and change detection.
• Evaluation and adaptation methods like linear probing, fine-tuning, and parameter-efficient tuning demonstrate EOFMs' potential across different sensor types and applications.

Searching arXiv for papers on Earth observation foundation models to ground the article. {"queries":[{"query":"Earth Observation Foundation Models remote sensing foundation models arXiv 2025 2026","limit":10},{"query":"site:arxiv.org Earth Observation Foundation Models benchmark robustness multimodal 2026","limit":10}]} Earth Observation Foundation Models (EOFMs) are large-scale pretrained models for Earth observation data that learn general-purpose representations from massive archives of satellite imagery and are subsequently adapted to downstream tasks with limited labeled data. In the literature, EOFMs are described both as self-supervised vision models trained on large satellite archives and as multimodal models that align one or more remote-sensing modalities into a shared feature space, sometimes including language. Their scope now spans classification, segmentation, detection, regression, retrieval, change detection, captioning, and visual question answering, but the defining constraint is domain specificity: EOFMs must operate over multispectral and hyperspectral bands, SAR backscatter, thermal and elevation layers, heterogeneous ground sampling distances, irregular revisit intervals, and sensor metadata rather than over natural-image RGB alone (Girtsou et al., 5 Apr 2026, Xiong et al., 8 Mar 2025, Xiao et al., 2024, Imtiaz et al., 8 Jul 2026).

1. Definition, scope, and problem setting

An EOFM is a large-scale pretrained model designed specifically for Earth observation data. Depending on the formulation, it may ingest a single remote-sensing modality, a multitemporal image stack, or a multimodal bundle containing optical, multispectral, hyperspectral, SAR, infrared, elevation, or other derived geospatial layers. A common definition in the recent literature is that such a model is pretrained on massive unlabeled or weakly labeled EO archives, learns transferable representations, and is then adapted by linear probing, fine-tuning, or parameter-efficient tuning to a wide variety of downstream tasks (Xiong et al., 8 Mar 2025, Ghamisi et al., 30 May 2025, Imtiaz et al., 8 Jul 2026).

The need for EOFMs arises from the structure of EO data itself. Remote-sensing archives are large, geographically broad, and increasingly multimodal, while labeled data are often sparse and expensive to collect. At the same time, EO signals differ substantially from natural-domain inputs: multispectral and hyperspectral sensors produce 3–200+ bands; SAR introduces active-sensor backscatter and speckle; revisit intervals range from hours to weeks; spatial resolution ranges from sub-meter to tens of meters; and auxiliary context such as geolocation, acquisition date, patch area, and sensor identity often matters materially for transfer (Xiao et al., 2024, Wang et al., 14 Mar 2025, Imtiaz et al., 8 Jul 2026).

Benchmark studies frame EOFMs as principally label-efficient. In one benchmark, with only 100 training images per region, a geo-aware U-Net fine-tuned from the foundation model achieved an mIoU of approximately 0.55 versus approximately 0.46 for a fully supervised U-Net, while image-level land-cover classification rose from 0.55 to approximately 0.64 and building-density regression reduced MSE by 86% relative to a scratch-trained U-Net (Dionelis et al., 2024). PhilEO Bench reports the same general pattern: pretrained models such as Prithvi, SatMAE, SeCo, MAE-ViT, and a geo-aware U-Net consistently outperform randomly initialized baselines in few-shot regimes, even though classical encoder–decoder designs can still retain advantages on dense prediction tasks that require fine local detail (Fibaek et al., 2024).

2. Pretraining objectives and architectural families

The dominant pretraining families in EOFMs are masked image modeling, contrastive learning, hybrid reconstruction-plus-alignment objectives, and vision–language alignment. Masked autoencoding is especially common. A representative objective is

$$L_{MAE} = \frac{1}{|M|}\sum_{i\in M}\|x_i - \hat{x}_i\|^2,$$

where $M$ indexes the masked patch locations. This formulation appears in HighFM, Prithvi-style models, SatMAE-style models, and multiple benchmark summaries (Girtsou et al., 5 Apr 2026, Fibaek et al., 2024).

Contrastive objectives are equally prominent, particularly for multimodal alignment and vision–language training. A standard formulation used in benchmark summaries is InfoNCE,

$$\mathcal{L}_{\mathrm{NCE}} = -\frac{1}{N}\sum_i \log \frac{\exp(\langle v_i,t_i\rangle/\tau)}{\sum_j \exp(\langle v_i,t_j\rangle/\tau)},$$

while TerraFM adopts a DINO-style student–teacher framework with local–global multi-crop learning and dual centering, and GeoLangBind alternates sigmoid-based contrastive learning with multimodal distillation (Li et al., 22 May 2025, Danish et al., 6 Jun 2025, Xiong et al., 8 Mar 2025).

Architecturally, Vision Transformers dominate. Prithvi, HighFM, DOFA, TerraFM, Copernicus-FM, and many benchmarked EOFMs use ViT backbones, while DEO employs a Swin-Transformer student and SpectralEarth-FM uses a hierarchical transformer derived from Hiera-style design (Chuc, 25 Jun 2025, Girtsou et al., 5 Apr 2026, Wolf et al., 23 Feb 2026, Braham et al., 20 May 2026). Yet EOFMs are not restricted to a single input strategy. Several recurrent architectural motifs have emerged.

Dynamic spectral adaptation is exemplified by DOFA and Copernicus-FM. DOFA uses a wavelength-conditioned dynamic hypernetwork to generate patch-embedding weights from central wavelengths, allowing a single Transformer to process Sentinel-1 SAR, Sentinel-2 multispectral, NAIP RGB, Gaofen, and EnMAP hyperspectral inputs, including sensors not seen during pretraining (Xiong et al., 2024). Copernicus-FM extends this idea to both spectral and non-spectral modalities, generating patch-embedding weights from spectral response metadata or variable-name embeddings and fusing Fourier-encoded metadata for geolocation, patch area, and time (Wang et al., 14 Mar 2025).

Cross-modal fusion appears in multiple forms. PyViT-FUSE uses multi-head cross-attention to fuse mixed-resolution band tokens at each spatial location, then processes the fused sequence through a pyramidal ViT stack (Weber et al., 26 Apr 2025). TerraFM uses modality-specific patch embeddings for Sentinel-1, Sentinel-2 L1C, and Sentinel-2 L2A, followed by adaptive cross-attention fusion and a shared ViT encoder (Danish et al., 6 Jun 2025). SpectralEarth-FM adds sensor-specific local hierarchical branches, projected-attention fusion across available sensors, and a shared hierarchical trunk to jointly handle HSI, MSI, SAR, and land-surface temperature (Braham et al., 20 May 2026).

Objective alignment rather than reconstruction is the core premise of DEO. Its dual-teacher framework distills from a multispectral teacher and a frozen optical vision foundation model teacher, with separate losses for multispectral global features, optical class-token features, and optical patch-token features. The model uses no pixel-reconstruction term, and the abstract reports average improvements of 3.64 percentage points in semantic segmentation, 1.2 in change detection, and 1.31 in classification tasks (Wolf et al., 23 Feb 2026).

3. Data regimes, temporal structure, and multimodal coverage

EOFMs are increasingly differentiated by the breadth of their pretraining corpora. Some are sensor-specific and temporally dense, others geographically global and multimodal, and others explicitly aligned across surface, atmospheric, and language domains.

HighFM represents the temporally dense end of the spectrum. It is pretrained on approximately 2.23 TB of cleaned MSG/SEVIRI radiance imagery over the Mediterranean basin, using 11 spectral bands, 15-minute revisit, and non-overlapping $32\times32$ patches. Its central claim is that high temporal resolution matters operationally because wildfire fronts, convective storms, and flash floods can evolve on timescales of minutes to hours, and because a delay of even one polar-orbiting overpass can render alerts obsolete (Girtsou et al., 5 Apr 2026). To exploit this regime, HighFM augments SatMAE with minute-resolution temporal encodings for year, day-of-year, hour, and minute, and its multi-timestep variant concatenates token streams from three acquisitions sampled randomly within the same hour (Girtsou et al., 5 Apr 2026).

At the opposite end are globally distributed, strongly multimodal corpora. Copernicus-Pretrain integrates 18.71 million images from Sentinel-1 GRD, Sentinel-2 TOA, Sentinel-3 OLCI, Sentinel-5P atmospheric gases, and Copernicus DEM over 312,567 cells, with a joint subset of approximately 220 K cells containing all eight modalities (Wang et al., 14 Mar 2025). TerraFM pretrains on 1.53 million grid cells from Major-TOM, corresponding to 18.7 million tiles across Sentinel-1, Sentinel-2 L1C, and Sentinel-2 L2A after land-cover-aware stratification (Danish et al., 6 Jun 2025). SpectralEarth-MM is explicitly designed to bring HSI into multisensor pretraining, pairing EnMAP, EMIT, and DESIS with Sentinel-2, Landsat-8/9 optical imagery, Landsat LST, and Sentinel-1 SAR over approximately 2 million globally distributed locations, 25 million georeferenced patches, and more than 40 TB of data (Braham et al., 20 May 2026).

Language-centered unification constitutes a separate axis. GeoLangBind-2M contains 2,050,983 image–text pairs spanning six EO modalities: RGB, SAR, MSI, HSI, elevation, and infrared. GeoLangBind aligns arbitrary EO channel sets into a shared language embedding space through a wavelength-aware dynamic encoder, Modality-aware Knowledge Agglomeration, and progressive multimodal weight merging. The model is explicitly positioned as enabling zero-shot classification, retrieval, caption-like description, and cross-modal integration across heterogeneous EO sensors (Xiong et al., 8 Mar 2025).

A further expansion of the EOFM concept is the move beyond raster-only representations. A recent perspective argues that current EOFMs remain largely confined to raster modalities and underuse vector data such as OpenStreetMap, cadastral layers, parcels, and points of interest. That perspective proposes joint Spatial Representation Learning in a unified embedding space combining raster perception with vector-based reasoning, geometry, topology, and semantic relations (Knoblauch et al., 1 Jun 2026). This suggests that the EOFM problem is broadening from “multispectral image encoding” to “geospatial representation learning” in a stronger sense.

4. Adaptation, model composition, and evaluation protocols

Downstream use of EOFMs is organized around linear probing, full fine-tuning, and parameter-efficient methods. Benchmarking work repeatedly distinguishes frozen-backbone evaluation from full adaptation and shows that the choice materially changes conclusions (Fibaek et al., 2024, Dionelis et al., 2024, Imtiaz et al., 8 Jul 2026).

Parameter-efficient adaptation has become a practical concern for large EO encoders. In flood segmentation, LoRA applied to the Clay EO-FM encoder updates only low-rank matrices in transformer attention projections while keeping the encoder weights frozen. With rank $r=256$, LoRA updates 26.74 M parameters, compared with approximately 124 M for full fine-tuning, raises F1 from 84.20 to 90.86, and raises IoU from 0.72 to 0.83, while full fine-tuning is reported as out-of-memory on an A100 40 GB GPU (Selvam et al., 2024).

An alternative to training new monolithic models is composition of existing pretrained EOFMs. A feature-level ensemble study combines Prithvi, Hiera, and DOFA on GEO-Bench by learning convex weights over feature embeddings,

$$F(x) = \sum_{i=1}^n w_i f_i(x), \qquad \sum_i w_i = 1,\; w_i \ge 0.$$

The study reports that the ensemble outperformed its components on 10 of 11 tasks, and that a distilled student achieved approximately 98–99% of the ensemble’s accuracy or mIoU while reducing inference FLOPs by approximately 40% (Chuc, 25 Jun 2025). This suggests that EOFM deployment need not always mean selecting a single pretrained backbone.

Evaluation practice is now itself a substantive research topic. PhilEO Bench introduces a 400 GB Sentinel-2 dataset with building density estimation, road segmentation or density estimation, and land-cover classification under controlled n-shot splits (Fibaek et al., 2024). GEO-Bench, as used in model-composition studies, covers 11 datasets spanning classification and segmentation across modalities and resolutions (Chuc, 25 Jun 2025). Copernicus-Bench organizes 15 downstream tasks hierarchically from cloud detection through surface applications to specialized atmospheric and flood tasks (Wang et al., 14 Mar 2025). SustainFM assembles 16 tasks mapped to Sustainable Development Goals and argues that evaluation should include transferability, generalization, and energy efficiency rather than accuracy alone (Ghamisi et al., 30 May 2025). REOBench adds corruption robustness across six tasks and twelve corruption types, while recent review work emphasizes that inconsistent train/test splits, preprocessing, and reporting standards still prevent fair comparison (Li et al., 22 May 2025, Imtiaz et al., 8 Jul 2026).

5. Representative systems and reported performance

Reported performance varies strongly with modality, benchmark, and adaptation protocol, but several systems illustrate the range of current EOFM capabilities.

System	Pretraining regime	Representative reported outcome
HighFM_MT (Girtsou et al., 5 Apr 2026)	~2.23 TB SEVIRI, 11 bands, 15-minute cadence	Cloud segmentation: balanced accuracy 0.831, cloud-IoU 0.737; active fire detection: balanced accuracy 0.925, fire-IoU 0.079
Copernicus-FM (Wang et al., 14 Mar 2025)	18.71 M images across Sentinel-1/2/3/5P and DEM	EuroSAT-S2 OA 97.9; LC100Seg-S3 mIoU 24.1; AQ-O3-S5P RMSE 789.4
TerraFM-B (Danish et al., 6 Jun 2025)	18.7 M tiles across S1, S2-L1C, S2-L2A	Cloud-S2 67.9% mIoU; BigEarthNet-S2 84.4% mAP; LCZ-S2 87.0% OA
SpectralEarth-FM-a (Braham et al., 20 May 2026)	~2 M locations, 25 M patches, ≈40 TB	Average rank 1.4 over 10 hyperspectral tasks; average rank 3.43 on PANGAEA
GeoLangBind-L-384 (Xiong et al., 8 Mar 2025)	2,050,983 image–text pairs across six modalities	Zero-shot scene average 64.5; UCM-caption Recall@1 43.3

HighFM is particularly important because it shifts EOFM design toward geostationary, high-revisit data rather than high-resolution, low-revisit data. On cloud segmentation, HighFM_MT exceeds U-Net from scratch, ViT from scratch, ViT-ImageNet, Copernicus-FM, and Panopticon in both balanced accuracy and IoU; on active fire detection, it attains a balanced accuracy of 0.925, which the paper reports as +0.031 over Copernicus-FM (Girtsou et al., 5 Apr 2026).

SpectralEarth-FM demonstrates that hyperspectral information can be integrated into a general multimodal pretraining pipeline rather than remaining isolated in HSI-only models. On hyperspectral segmentation, SpectralEarth-FM-a attains state-of-the-art average rank 1.4 over 10 tasks, with examples such as EnMAP CDL 74.0 versus 70.9 for Spec.ViT-L and EuroCrops 63.5 versus 60.9 (Braham et al., 20 May 2026). On standard EO tasks under the PANGAEA protocol, the same model achieves average rank 3.43 and reports mIoU values including 85.10 on HLS-Burns and 91.08 on Sen1Floods11 (Braham et al., 20 May 2026).

Copernicus-FM and TerraFM show the effectiveness of large unified pretraining on mainstream EO benchmarks. Copernicus-FM reports strong gains in Sentinel-1 classification, Sentinel-3 segmentation and classification, and atmospheric regression, including approximately 55% RMSE reduction on O$_3$ regression relative to the listed comparison (Wang et al., 14 Mar 2025). TerraFM reports state-of-the-art results on GEO-Bench and Copernicus-Bench, with ablations attributing gains to modality augmentation, cross-attention fusion, and dual-centering regularization (Danish et al., 6 Jun 2025).

6. Limitations, controversies, and research directions

Despite rapid progress, the recent literature is explicit that no single EOFM is universally best. Benchmark and review papers repeatedly state that performance depends on the match between pretraining data, architecture, objective, downstream task, and deployment regime, and that inconsistent evaluation remains a major obstacle to reliable comparison (Ghamisi et al., 30 May 2025, Imtiaz et al., 8 Jul 2026).

A central controversy concerns instrumentation bias. A dedicated study of Prithvi and DOFA shows that EOFM embedding spaces remain highly sensitive to sensor architecture. Using Landsat-8, Landsat-9, Sentinel-2, HLS, and Sentinel-1 over 600 geolocated Indiana samples, the study finds strong modality clustering in t-SNE, low neighborhood overlap across sensors, and very high modality-prediction accuracy from patch embeddings: a Random Forest reaches 90.7% for Prithvi and 88.7% for DOFA, while 5-NN reaches 85.6% and 91.1%, respectively (Demilt et al., 1 Oct 2025). This directly challenges the assumption that off-the-shelf EOFM embeddings are sensor-invariant.

Robustness under perturbation is another documented weakness. REOBench evaluates six tasks and twelve corruption types and reports that existing EOFMs suffer significant degradation, with drops ranging from less than 1% to over 20% depending on task, architecture, and corruption. MIM-based models are the least robust, especially under motion blur and noise, whereas vision–LLMs show enhanced robustness, particularly in multimodal tasks (Li et al., 22 May 2025).

Trustworthiness studies extend this critique from corruption to distribution shift. SHRUG-FM augments a frozen geospatial foundation model with input-space OOD detection, embedding-space OOD detection, and predictive uncertainty. On burn-scar segmentation, the paper reports Pearson $r \approx -0.85$ between mean embedding-space NCDD and F1 across HydroATLAS deciles, and shows that discarding up to 30% of images with the highest uncertainty raises mean IoU from 0.85 to 0.91 on the retained set (Cohrs et al., 13 Nov 2025). The same analysis links failures to low-elevation zones and large-river areas, indicating that errors are geographically structured rather than random (Cohrs et al., 13 Nov 2025).

Current future directions are correspondingly broad. Several papers advocate multimodal, resolution-agnostic pretraining across geostationary and polar-orbiting sensors, SAR, LiDAR, and hyperspectral data; physics-informed masking and reconstruction objectives; continual learning for seasonal and calibration shifts; and any-to-any sensor-conditioned architectures (Girtsou et al., 5 Apr 2026, Imtiaz et al., 8 Jul 2026). GeoLangBind argues that language can serve as a “lingua franca” across EO modalities (Xiong et al., 8 Mar 2025), while raster–vector work argues for unified spatial representation learning beyond pixels (Knoblauch et al., 1 Jun 2026). Taken together, these directions imply that the next generation of EOFMs will likely be judged not only by benchmark accuracy, but also by modality-aware transfer, robustness under corruption and shift, physical plausibility, uncertainty calibration, and operational usefulness in climate-sensitive decision settings (Ghamisi et al., 30 May 2025, Imtiaz et al., 8 Jul 2026).