Foundation Models in Environmental Science

• Foundation models in environmental science are large-scale, pre-trained AI systems that unify diverse data modalities—from satellite imagery to simulations—for comprehensive environmental analysis.
• They leverage advanced architectures like CNNs, vision transformers, and hybrid models with self-supervised methods such as contrastive learning and masked autoencoding to enhance performance.
• These models achieve state-of-the-art results in tasks like land-cover classification, semantic segmentation, and change detection, enabling efficient adaptation and robust environmental monitoring.

Foundation models in environmental science are large-scale, pre-trained artificial intelligence systems designed to extract general-purpose knowledge from vast, heterogeneous environmental datasets. They enable unprecedented performance and transferability across key tasks in remote sensing, climate modeling, ecosystem assessment, and policy-relevant analytics. By leveraging architectures such as vision transformers, convolutional neural networks, and hybrid models, foundation models unify multi-modal data streams—satellite, in-situ, simulation, and text—allowing rapid fine-tuning and efficient adaptation for a diverse range of geoscientific tasks (Lu et al., 2024).

1. Model Architectures and Pretraining Paradigms

Foundation models (FMs) in environmental domains fall into distinct architectural families:

• Convolutional Neural Network (CNN) Backbones: Architectures like ResNet-34/50/101/152 employ stacked convolutional layers with residual connections, optimal for local feature extraction in high-resolution imagery. The canonical residual formulation is $$\mathbf{y} = \mathcal{F}(\mathbf{x}; W) + \mathbf{x}$$, facilitating gradient propagation in deep networks.
• Vision Transformer (ViT) Families: Examples include ViT-Base/Small and Swin-Transformers. Key mechanisms involve splitting images into non-overlapping patches, linear embedding, positional encoding, and processing via multi-head self-attention:

$$\mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left( \frac{Q K^T}{\sqrt{d_k}} \right) V,$$

capturing global spatial dependencies essential for wide-area land-cover or spatiotemporal pattern recognition.

• Hybrid CNN–Transformer Architectures: Models such as RingMo-lite and ConvNeXt + shallow ViT employ CNN layers for strong spatial priors combined with transformer modules for context reasoning, achieving improved balance between local detail and global context (Lu et al., 2024, Zhang et al., 2023).

Pretraining is predominantly self-supervised, capitalizing on massive unlabeled archives. Core methodologies include:

• Contrastive Learning (CL): Frameworks like SimCLR, MoCo, BYOL, and DINO operate by attracting augmented “positive” pairs and repelling “negatives” in the latent space. The InfoNCE loss is standard:

$$\mathcal{L}_{\mathrm{contrast}} = -\log \frac {\exp(\mathrm{sim}(h_i, h_j)/\tau)} { \sum_{k=1}^{K} \exp(\mathrm{sim}(h_i, h_k)/\tau) },$$

with similarity measured as cosine and $\tau$ a temperature scaling parameter.

• Masked Autoencoding (MAE): Models like SatMAE, Scale-MAE randomly mask high fractions of input patches and optimize an $\ell_2$ reconstruction loss over masked regions:

$$\mathcal{L}_{\mathrm{MAE}} = \left\| x_{\mathrm{masked}} - \hat{x}_{\mathrm{masked}} \right\|^2_2.$$

• Distillation/Teacher-Student Paradigms: Techniques such as CMID, GFM distill high-capacity teacher representations into compact, resource-efficient student models (Lu et al., 2024).

Fine-tuning strategies span:

• Full-parameter tuning: All weights updated on target datasets.
• Parameter-efficient adaptation: Adapters, LoRA, and bit-fit update limited trainable parameters for rapid transfer in data-scarce regimes.
• Linear Probing: Only shallow heads are updated atop frozen backbones, preserving generalized encodings (Lu et al., 2024, Dionelis et al., 2024).

2. Major Datasets and Data Modalities

Pretraining leverages large, domain-specific datasets to ensure scale, diversity, and sensor coverage:

Dataset	Characteristics	Modality
RSD46-WHU	117,000 RGB tiles, 0.5–2 m, 46 scene classes	RGB
fMoW	1M+ multispectral images, 207 countries, up to 8 bands, 0.3–1 m	Multispectral
DOTA	11,268 RGB tiles (0.3–5 m), 18 object categories	RGB
SEN12MS	541,986 Sentinel-1/2 pairs, 33 classes, 10–60 m	SAR/Multispectral
BigEarthNet	590,326 Sentinel-2 patches, 43 classes	Multispectral
SSL4EO-S12	3M multispectral+SAR (global, 10–60 m)	Multimodal

Each dataset enhances spectral, spatial, and geographic variability, enabling models to internalize diverse environmental structures and conditions (Lu et al., 2024).

3. Domain Applications and Quantitative Performance

Foundation models deliver new benchmarks in leading remote sensing and geoscientific tasks:

• Land-Cover & Scene Classification: On BigEarthNet, msGFM reaches 92.90% mAP; SkySense, DeCUR, and DINO-MC achieve 92.09%, 89.70%, and 88.75% mAP, respectively (Lu et al., 2024).
• Semantic Segmentation: ISPRS Potsdam tasks see SkySense attaining mF1 = 93.99%, CMID mIoU = 87.04%, and BFM OA = 91.82%.
• Object Detection: On DOTA, RVSA achieves 81.24% mAP; SMLFR: 79.33%. On DIOR, MTP attains AP50 = 78.00%.
• Change Detection: SkySense and GFM score above 59% F1 on OSCD (change maps), and over 92% F1 on LEVIR-CD for building change detection.
• Vegetation & Crop Monitoring: RSP delivers crop-yield proxy correlations >0.85; EarthPT enhances NDVI-stress detection accuracy by 10% over classical LSTMs.
• Flood & Wildfire Detection: USat's cross-sensor encoder boosts flood extent segmentation by +7% IoU compared to ResNet-50 UNet; OFA-Net for wildfire mapping achieves 88.4% overall accuracy.
• Climate Change Analysis: SkySense's multi-modal encoder reduces glacier-retreat mapping MSE by 15% compared to baselines.

These performance metrics rely on standard definitions, e.g., mean Intersection over Union (mIoU), mean Average Precision (mAP), and F1 for detection and segmentation (Lu et al., 2024).

4. Technical Challenges and Methodological Directions

Adoption and further development of FMs face critical challenges:

• Data Quality & Gaps: Cloud occlusion, sensor limitations, and misregistration degrade model robustness.
• Domain Shift: Seasonal, sensor, and geographic heterogeneity—models trained in Europe may not translate to tropics.
• Computational Cost: Training ViT-Large with MAE over millions of images requires multi-GPU clusters and extensive runtime.
• Label Scarcity: Many environmental challenges are low-label; robust few-shot or zero-shot generalization is essential (Lu et al., 2024).

Future research and best practices include:

• Domain-Adaptive Learning: SSL with adversarial augmentation, test-time adaptation, and cross-domain tuning.
• Multi-Modal Sensor Fusion: Integrating optical, SAR, and thermal data streams (e.g., CROMA, SkySense) for complete environmental context (Ghamisi et al., 30 May 2025).
• Parameter-Efficient Fine-Tuning: PEFT (e.g., UPetu) and edge deployment of distilled "Edge-FMs" (Lu et al., 2024).
• Scale-Aware Modeling: Multi-scale modeling as in Cross-Scale MAE captures both micro and macro phenomena.
• Specialized Benchmarks: Creation of new datasets for carbon stock, biodiversity, and pollution mapping (Lu et al., 2024).

5. Evaluation Metrics, Benchmarks, and Label Efficiency

Model assessment is standardized via:

• Accuracy, mIoU, F1, mAP, AP50 for classification, segmentation, and detection.
• Cross-Region Robustness: Variance in IoU (Δ_IoU) across geographic regions is a standard metric for generalization (Δ_IoU ≈ 0.03 for pre-trained FMs vs. 0.07 for baselines) (Dionelis et al., 2024).
• Label Efficiency: Foundation models outperform task-specific models by substantial margins with as little as 10–20% of labeled data per task, enabling high performance in data-scarce regions.
• Few-Shot Adaptation: Fine-tuning with 100–1,000 samples delivers competitive task accuracy and segmentation IoU (Dionelis et al., 2024).

Benchmarking initiatives like PhilEO, GEO-Bench, and SustainFM provide curated multi-task, multi-region testbeds for evaluating both absolute accuracy and generalization under real-world domain shifts (Ghamisi et al., 30 May 2025, Lacoste et al., 2023).

6. Practical Implications, Toolkits, and Operational Integration

Foundation models are increasingly accessible and practical:

• Pretrained Model Availability: Checkpoints released on HuggingFace and GitHub facilitate immediate integration and adaptation for specific environmental tasks and regions.
• Compute Scaling: For large ViT or CNN-based FMs, practitioners are advised to use cloud GPU/TPU resources; lightweight or pruned variants (RingMo-lite, UPetu) are recommended for edge/field deployment.
• Parameter-Efficient Workflow: LoRA, adapters, and bit-fit yield effective fine-tuning in low-label regimes.
• Integrated Pipelines: Time-series FMs (EarthPT, U-BARN) can be combined with change detection heads for continuous monitoring.
• Open-source Toolchains: Raster Vision, TorchEO, and Earth Observation Transformers support core operations from patch management to SSL data loading and fine-tuning.
• Interpretability and Uncertainty: Adoption of saliency mapping and dropout-based uncertainty quantification is advised to flag low-confidence predictions for policy and decision support (Lu et al., 2024).

7. Impact, Limitations, and Future Outlook

Foundation models have led to pronounced advances in remote sensing tasks vital to environmental science. By leveraging massive unlabeled datasets, innovative self-supervised learning frameworks, and advanced architectures, they now deliver state-of-the-art performance in land-use classification, segmentation, object detection, change mapping, and environmental monitoring (Lu et al., 2024).

Current limitations stem from data gaps (e.g., clouds, label paucity), model transferability across diverse environmental domains, computational demands for pretraining, and the need for interpretable, robust, and uncertainty-aware outputs, particularly in critical applications such as disaster response and climate-change analytics.

Key directions for further research include:

• Expanding cross-domain adaptation via domain-adaptive SSL and adversarial training.
• Deepening multi-modal fusion (optical, SAR, thermal, time-series) for comprehensive scene understanding.
• Scaling up parameter-efficient learning, enabling widespread deployment.
• Developing interpretable and uncertainty-quantified FMs for risk-sensitive decision contexts.
• Curating interdisciplinary benchmarks for emerging environmental analytics (carbon, biodiversity, pollution indices).

By following established best practices—adopting published checkpoint weights, modular tuning methods, and tailored RS toolkits—environmental scientists and practitioners can seamlessly integrate foundation models into advanced, operational environmental monitoring and analysis workflows (Lu et al., 2024).