OceanMAE: A Foundation Model for Ocean Remote Sensing

Abstract: Accurate ocean mapping is essential for applications such as bathymetry estimation, seabed characterization, marine litter detection, and ecosystem monitoring. However, ocean remote sensing (RS) remains constrained by limited labeled data and by the reduced transferability of models pre-trained mainly on land-dominated Earth observation imagery. In this paper, we propose OceanMAE, an ocean-specific masked autoencoder that extends standard MAE pre-training by integrating multispectral Sentinel-2 observations with physically meaningful ocean descriptors during self-supervised learning. By incorporating these auxiliary ocean features, OceanMAE is designed to learn more informative and ocean-aware latent representations from large- scale unlabeled data. To transfer these representations to downstream applications, we further employ a modified UNet-based framework for marine segmentation and bathymetry estimation. Pre-trained on the Hydro dataset, OceanMAE is evaluated on MADOS and MARIDA for marine pollutant and debris segmentation, and on MagicBathyNet for bathymetry regression. The experiments show that OceanMAE yields the strongest gains on marine segmentation, while bathymetry benefits are competitive and task-dependent. In addition, an ablation against a standard MAE on MARIDA indicates that incorporating auxiliary ocean descriptors during pre-training improves downstream segmentation quality. These findings highlight the value of physically informed and domain-aligned self-supervised pre- training for ocean RS. Code and weights are publicly available at https://git.tu-berlin.de/joanna.stamer/SSLORS2.

• The paper introduces OceanMAE, a model that integrates physical ocean descriptors with multispectral satellite imagery using a masked autoencoder framework.
• It demonstrates improved performance in segmentation and regression tasks by fusing transformer embeddings with a modified UNet architecture.
• Experiments reveal that ocean-specific pre-training yields higher accuracy and stability in pollutant detection and bathymetry estimation.

OceanMAE: A Foundation Model for Ocean Remote Sensing

Introduction

Ocean remote sensing (RS) supports critical tasks in marine monitoring, ranging from bathymetry estimation to pollutant detection and debris segmentation. However, the domain faces persistent challenges: limited annotated data, underrepresentation in general-purpose models pre-trained on terrestrial imagery, and dynamic spectral/physical attributes unique to oceanic scenes. Addressing these issues, the OceanMAE model introduces an ocean-centric masked autoencoder (MAE) framework that tightly integrates multispectral satellite observations and physically meaningful auxiliary descriptors, thereby tailoring foundation model (FM) design to the marine context.

OceanMAE Architecture

OceanMAE extends the classical MAE paradigm by fusing Sentinel-2 L2A ocean imagery with auxiliary ocean descriptors at the representation learning stage. The model preserves the core MAE mechanism—masked image modeling with high patch-wise masking rates—but augments it with external oceanographic features (e.g., bathymetry, chlorophyll-a concentration, Secchi depth), which are projected and concatenated with the [CLS] token to form a comprehensive multimodal latent. The decoder leverages this fused vector for image reconstruction. This architectural update aims to produce latent embeddings that are both visually and physically ocean-aware, improving transferability to downstream marine tasks.

Figure 1: Overview of the OceanMAE architecture, highlighting masked patch encoding and auxiliary oceanic variable integration at the representation learning stage.

Downstream Model Integration

For downstream segmentation and regression tasks, the OceanMAE encoder’s output is interfaced with a modified UNet family. Crucially, the architecture includes a parallel embedding stream: the OceanMAE latent $z$ is linearly projected, reshaped, and convolved to form a spatial tensor, which is fused at the UNet bottleneck with the hierarchical convolutional features. This integration provides structured local features from the CNN pipeline and broad contextual semantics from the transformer backbone, which is shown to be beneficial for both dense (segmentation) and sparse (regression) prediction paradigms.

Figure 2: Modified UNet architecture for downstream ocean tasks, illustrating parallel OceanMAE embedding fusion at the bottleneck for enhanced feature integration.

Embedding Initialization Strategies

Three initialization protocols for transferring OceanMAE representations downstream are rigorously ablated:

• Random: Embeddings are initialized from scratch and fixed.
• Frozen Embedding (FE): Pre-trained OceanMAE encoder features are fixed during UNet training.
• Fully Finetuned (FF): Pre-trained weights are updated during conjunction training with the downstream task.

This protocol analysis elucidates under which conditions SSL representations should be locked or further refined, revealing task-specific trends in optimal transfer.

Experimental Evaluation

OceanMAE is pre-trained on the Hydro dataset (100k unlabeled Sentinel-2 patches) and evaluated on MADOS (marine pollutant and sea-surface segmentation), MARIDA (marine debris segmentation), and MagicBathyNet (shallow water bathymetry regression).

Bathymetry Regression

OceanMAE embeddings provide consistent improvements in regression stability (lower RMSE/stddev) on MagicBathyNet compared to both random and generic self-supervised approaches, with the magnitude of gains being dataset-size and region-dependent. Notably, when labeled data is minimal (Agia Napa subset), the FF protocol achieves the lowest MAE and RMSE, emphasizing the utility of ocean-specific pre-training for scenarios of label scarcity.

Marine Debris and Pollutant Segmentation

Strong gains are shown for MARIDA: OceanMAE + FF increases Pixel Accuracy from 0.69 (UNet baseline) to 0.77 and Macro F1 from 0.69 to 0.72. On MADOS, OceanMAE + FE with Patch-16 outperforms all state-of-the-art models in mIoU (70.8 vs 64.3 for MariNeXt) and Macro F1 (81.3 vs 76.0 for MariNeXt), though MariNeXt retains a slight lead in raw pixelwise accuracy. Qualitative results further demonstrate that the pre-trained, frozen embedding strategy yields more precise, less noisy segmentation masks compared to conventional CNN baselines.

Figure 3: Qualitative comparison for pollutant and sea-surface segmentation, with OceanMAE pre-training (d) yielding denoised predictions and sharper boundaries versus UNet (c).

Ablation: Impact of Auxiliary Descriptors

A direct comparison between OceanMAE and standard MAE pre-training on MARIDA indicates that inclusion of physically meaningful features in the self-supervised phase boosts both pixel and macro-level metrics, though a standard MAE may produce higher mIoU. This metric dependency points to different learned invariances and class-specific calibration when augmenting patch-level reconstruction with auxiliary physical context.

Implications and Future Directions

OceanMAE demonstrates that domain-aligned self-supervised pre-training, particularly the injection of physical variables during representation learning, is crucial for effective feature transfer in ocean RS applications. The results indicate that state-of-the-art segmenters benefit substantially more from physically informed, ocean-centric embeddings than from models pre-trained on terrestrial-dominated corpora. The dual-pathway integration (UNet + MAE embedding) is shown to be particularly advantageous for segmentation tasks involving sparse or fine-grained water-surface phenomena.

Practically, this empowers marine monitoring initiatives to exploit large, unlabeled datasets and achieve precise pollutant, debris, and bathymetry mapping with reduced annotation costs. Theoretically, the approach sets a template for designing foundation models in underrepresented geophysical domains, advocating for explicit multimodal fusion at the self-supervised stage.

Looking forward, similar methodologies could extend to spatiotemporal modeling (multi-date Sentinel-2 or future hyperspectral payloads), integration with physical oceanographic simulation outputs, or cross-sensor transfer to SAR and thermal modalities. The modality-agnostic and descriptor-fusion principles established by OceanMAE could generalize to other geoscience foundation modeling initiatives.

Conclusion

OceanMAE substantiates the argument that foundation models for remote sensing must not only scale with data volume, but also rigorously encode physical priors and domain context to maximize downstream utility—especially in resource-scarce environments. The approach’s consistent improvements on multiple marine benchmarks with both frozen and finetuned transfer policies set the direction for future research in tailored, multimodal self-supervision for oceanic and wider geoscientific applications (2604.08171).