---
title: OceanForecastBench Global Forecasting Benchmark
url: https://www.emergentmind.com/topics/oceanforecastbench
type: topic
---

# OceanForecastBench Global Forecasting Benchmark

OceanForecastBench is an open, standardized benchmark for global, data-driven ocean forecasting that provides a unified training dataset, an independent observation-based evaluation set, a standard evaluation pipeline, and six baseline models for model development, evaluation, and comparison [2511.18732]. It is designed around the task of predicting key ocean variables such as temperature, salinity, and currents from a global three-dimensional ocean state plus sea-surface forcing, with forecasts evaluated at 1–10 day lead times against both reanalysis-derived targets and real-world observations. Its stated motivation is the absence of open-source, standardized benchmarks in this area, which had produced inconsistent data usage and evaluation methods and thereby hindered fair comparison and efficient model development [2511.18732].

## 1. Benchmark definition and design objectives

OceanForecastBench is framed as a benchmark dataset for data-driven global ocean forecasting whose three core contributions are: a high-quality global ocean reanalysis data over 28 years for model training, a high-reliability satellite and in-situ observations for model evaluation covering approximately 100 million locations in the global ocean, and an evaluation pipeline with six typical baseline models [2511.18732]. The benchmark is explicitly intended to standardize both data usage and evaluation procedure, addressing the problems that different groups had been using different reanalyses, domains, resolutions, auxiliary inputs, and observation protocols.

The benchmark’s problem definition is global and three-dimensional. At time \(t\), the input is a tensor
\[
X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,
\]
and the model predicts
\[
\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},
\]
for lead times \(\Delta t = 1,\ldots,k\), with \(k \le 10\) in the reported experiments and \(C_{out}=94\) because the two wind components are inputs only and not forecast variables [2511.18732].

A central design choice is the separation between training and final evaluation. Training and internal validation use standardized reanalysis-derived fields, whereas final evaluation uses independent observations from EN4, GDP, and CMEMS L3. This is presented as a correction to a common practice in which models are validated only against the same reanalysis used for training; the benchmark argues that such evaluation can hide biases and errors. In that sense, OceanForecastBench adopts an observation-anchored philosophy rather than a purely emulator-style one [2511.18732].

## 2. Standardized training dataset and state representation

The core training dataset is built primarily from the GLORYS12 “Global Ocean Physics Reanalysis,” supplemented with ERA5 surface winds and OSTIA SST [2511.18732]. GLORYS12 supplies sea surface height, temperature, salinity, eastward current velocity, northward current velocity, and mean dynamic topography. ERA5 provides 10 m zonal and meridional winds, and OSTIA provides daily SST. These heterogeneous sources are regridded and standardized into a single common tensor representation intended for deep learning workflows.

The common horizontal grid spans 80°S to 90°N and 180°W to 180°E, with grid size 121 \(\times\) 256 at 1.40625° \(\times\) 1.40625° resolution [2511.18732]. Vertically, GLORYS12’s 50 levels are downsampled to 23 fixed \(z\)-levels from 0.49 m to 643.57 m:
\[
\begin{aligned}
&0.49,\ 2.65,\ 5.08,\ 7.93,\ 11.41,\ 15.81,\ 21.60,\ 29.44,\ 40.34,\ 55.76,\ 77.85,\\
&92.32,\ 109.73,\ 130.67,\ 155.85,\ 186.13,\ 222.48,\ 266.04,\ 318.13,\ 380.21,\\
&453.94,\ 541.09,\ 643.57\ \text{m}.
\end{aligned}
\]

The resulting channel layout contains four sea-surface variables and four three-dimensional variables across 23 depth levels. The sea-surface variables are SST, SLA, \(u10\), and \(v10\). The three-dimensional variables are temperature, salinity, zonal current velocity, and meridional current velocity. Accordingly, the input channel count is
\[
C_{\text{in}} = 4 + 4 \times 23 = 96.
\]
The output excludes the two wind components and therefore has \(C_{out}=94\) channels [2511.18732].

Land–sea masking is handled by extracting an ocean–land mask from GLORYS12 and setting land points to \(0\) in the data arrays before model ingestion [2511.18732]. This preserves tensor regularity while keeping the representation compatible with standard CNN and transformer pipelines.

The temporal split is purely chronological. The training set covers 1993–2017, and the validation set covers 2018–2020 [2511.18732]. This temporal split mirrors the benchmark’s general emphasis on out-of-sample evaluation in time rather than arbitrary shuffling. A plausible implication is that OceanForecastBench is intended to measure operational generalization rather than interpolation across a stationary pooled dataset.

## 3. Observation-based evaluation corpus and collocation procedure

OceanForecastBench’s evaluation set is built from three observational sources: EN4 temperature and salinity profiles, GDP drifters for SST and 15 m surface currents, and CMEMS L3 altimetry for sea level anomaly [2511.18732]. The benchmark states that these sources together provide approximately 100 million locations in the global ocean and that only observations passing source quality flags are used.

The three observation streams serve distinct roles. EN4 provides temperature and salinity profiles, primarily from Argo floats plus historical casts, and is used as ground truth for subsurface \(T\) and \(S\). GDP drifters provide SST and 15 m eastward and northward velocity and are used for surface temperature and upper-ocean current verification. CMEMS L3 altimetry provides SLA along satellite tracks and is used for dynamic sea level verification [2511.18732]. The evaluation period uses 2022–2023 observations, which are outside the 1993–2020 training/validation interval.

A concise summary of the evaluation corpus is as follows.

| Source | Variables | Evaluation role |
|---|---|---|
| EN4.2.2 | Temperature, salinity | Subsurface \(T/S\) verification |
| GDP | SST, 15 m \(Uo/Vo\) | Surface temperature and current verification |
| CMEMS L3 altimetry | SLA | Dynamic sea level verification |

The core evaluation operation is a mapping
\[
\mathcal{M} : \hat{Y}_{fct}(t,\text{pos}_{grid}) \rightarrow \hat{Y}_{obs}(t,\text{pos}),
\]
which takes model forecasts on the regular benchmark grid and interpolates them to observation positions and depths [2511.18732]. This includes temporal alignment to the correct forecast lead time, spatial interpolation to observation longitude and latitude, and vertical interpolation from the 23 benchmark depth levels to observed depths for \(T\), \(S\), and currents. Observations outside the valid grid or over land are discarded.

This collocation-centered procedure is one of the benchmark’s defining features. Rather than evaluating only on the benchmark grid, it evaluates in observation space. That choice places OceanForecastBench closer to operational verification practice than to image-to-image forecasting benchmarks.

## 4. Forecasting tasks, metrics, and baseline systems

The benchmark task is to forecast the global three-dimensional ocean state and selected surface fields at lead times from 1 to 10 days [2511.18732]. The variables evaluated are temperature and salinity in three dimensions, SST, SLA, and 15 m zonal and meridional currents. Model outputs are global, but scores are also broken down by depth and by region.

OceanForecastBench defines four standard evaluation metrics. Let \(F^t_i\) be forecast values, \(O^t_i\) observations, \(C^t_i\) climatology, and \(N\) the number of evaluation points. Then:
\[
\text{RMSE} = \sqrt{\frac{1}{N}\sum_{i=1}^N (F^t_i - O^t_i)^2},
\]
\[
\text{Bias} = \frac{1}{N}\sum_{i=1}^N (F^t_i - O^t_i),
\]
\[
\text{ACC} = \frac{\sum_{i=1}^N (F^t_i - C^t_i)(O^t_i - C^t_i)}
{\sqrt{\sum_{i=1}^N (F^t_i - C^t_i)^2} \sqrt{\sum_{i=1}^N (O^t_i - C^t_i)^2}},
\]
\[
\text{CSS} = 1 - \frac{\text{RMSE}_{\text{Forecast}}}{\text{RMSE}_{\text{Climatology}}}.
\]
These metrics are aggregated globally and also resolved by region and depth [2511.18732].

The benchmark includes six baselines: one operational numerical model and five deep learning systems [2511.18732]. PSY4 is the numerical baseline, taken from Mercator Ocean’s operational “GLOBAL_ANALYSISFORECAST_PHY_001_024” product and downsampled to the benchmark grid. The deep learning baselines are ResNet, SwinTransformer, ClimaX, FourCastNet, and XiHe. Their reported parameter counts are approximately 51.4 million, 38.2 million, 65.7 million, 20.9 million, and 130 million, respectively [2511.18732].

This baseline suite places OceanForecastBench in a broader methodological landscape. It overlaps with the global forecast framing of FuXi-Ocean, which formulates a global 6-hourly, multi-depth, multivariate forecasting system and emphasizes standardized splits and skill curves [2506.03210], while also connecting to probabilistic and graph-based systems such as Njord, which is explicitly evaluated on OceanBench-style global observation tracks and ensemble metrics [2605.15470]. The benchmark therefore supports comparison across CNNs, vision transformers, Fourier-operator models, ocean-specific transformers, and numerical forecasts.

## 5. Reported performance patterns and benchmark findings

OceanForecastBench reports several consistent skill patterns across variables and lead times [2511.18732]. Error increases with lead time for all models: RMSE rises, while ACC and CSS decline. This is presented as the expected effect of accumulated forecast error in a chaotic ocean system.

For subsurface temperature and salinity, XiHe is reported as the strongest deep learning model, with the lowest RMSE and highest ACC, particularly at longer lead times [2511.18732]. PSY4 performs well at short lead times but degrades faster with lead time. For SST, PSY4 remains the strongest model, maintaining lower RMSE and higher ACC and CSS than the deep learning baselines even at 10 days. For SLA, PSY4 is best up to approximately 6 days, while XiHe overtakes it beyond approximately 7 days as PSY4’s RMSE grows faster [2511.18732].

For surface currents at 15 m, all models exhibit relatively low ACC, below approximately 0.5–0.6, and the benchmark attributes this to the scarcity and noisiness of velocity observations together with known deficits in modeling ocean currents [2511.18732]. Deep learning models often outperform PSY4 at longer lead times for these current variables, but performance remains weak overall.

The benchmark also reports structured spatial and vertical error behavior. All models show high RMSE in coastal regions and western boundary currents and low RMSE in open tropical oceans [2511.18732]. Vertically, errors peak around the thermocline, approximately 100–200 m, where vertical gradients are strong and model skill is reduced. XiHe is reported to have the best vertical performance across depths, with the lowest RMSE and highest ACC [2511.18732].

These findings address a common misconception that a single global score is sufficient to characterize ocean forecast quality. OceanForecastBench shows that relative model ranking depends strongly on variable, depth, lead time, and region. In particular, the benchmark reports that numerical systems remain very strong for surface-constrained variables such as SST and short-lead SLA, while deep learning systems can be stronger for some longer-lead and subsurface tasks [2511.18732].

## 6. Relation to adjacent benchmark efforts, limitations, and extensions

OceanForecastBench belongs to a broader progression from interpolation-oriented ocean benchmarks to forecast-oriented ones. OceanBench: The Sea Surface Height Edition standardized SSH interpolation pipelines, observation operators, spectral evaluation, and configurable data handling, but it was explicitly scoped to interpolation rather than forecasting [2309.15599]. OceanForecastBench extends the benchmark idea to global three-dimensional forecasting with observation-based verification [2511.18732]. A later OceanBench-related study on neural forecasting from sparse altimetry data further demonstrates how lead-time-aware, observation-collocated evaluation can be adapted to short-term SLA forecasting [2512.22152].

The benchmark’s own limitations are explicit. Its spatial resolution is 1.40625°, which is too coarse to resolve eddies and many coastal processes; its vertical extent reaches only the upper \(\sim 644\) m; it includes only physical ocean variables and selected surface forcings; and it supports deterministic forecasts only, without benchmark-native probabilistic evaluation [2511.18732]. Observation coverage is also uneven, especially for currents and for some periods of SST evaluation. These limitations matter when interpreting results: low-resolution global skill should not be conflated with eddy-resolving or coastal skill.

Several subsequent model papers suggest plausible benchmark extensions. SeaCast defines a high-resolution regional forecasting setup with explicit boundary forcing and graph-based handling of irregular coastlines [2410.11807]. FuXi-ONS and Njord introduce ensemble ocean forecasting with CRPS- and spread-skill-oriented evaluation, indicating a clear path toward probabilistic benchmark tracks [2603.19591]; [2605.15470]. Ocean-E2E shows how end-to-end neural assimilation and extreme-event metrics such as CSI can be incorporated for marine heatwave forecasting [2505.22071]. KFTD highlights continuous-time forecasting, arbitrary sub-step interpolation, and modular PDE-regularized losses in OceanForecastBench-style regional tasks [2606.17070].

Taken together, these developments suggest that OceanForecastBench functions both as a benchmark and as a reference interface. It standardizes a global, multivariate, observation-based evaluation problem, while also exposing the methodological gaps that later systems attempt to close: probabilistic forecasting, physical consistency metrics, higher resolution, regional nesting, and multi-timescale prediction.

Source: https://www.emergentmind.com/topics/oceanforecastbench