OceanForecastBench Global Forecasting Benchmark

• OceanForecastBench is a standardized, open benchmark for global ocean forecasting using reanalysis and satellite data to predict temperature, salinity, and currents.
• It integrates a 28-year training set with independent observational evaluations, ensuring fair model comparison across multiple lead times.
• The benchmark separates training and operational evaluation, employing six baseline models that span numerical and deep learning methods to highlight forecast skill variations.

OceanForecastBench is an open, standardized benchmark for global, data-driven ocean forecasting that provides a unified training dataset, an independent observation-based evaluation set, a standard evaluation pipeline, and six baseline models for model development, evaluation, and comparison (Jia et al., 24 Nov 2025). It is designed around the task of predicting key ocean variables such as temperature, salinity, and currents from a global three-dimensional ocean state plus sea-surface forcing, with forecasts evaluated at 1–10 day lead times against both reanalysis-derived targets and real-world observations. Its stated motivation is the absence of open-source, standardized benchmarks in this area, which had produced inconsistent data usage and evaluation methods and thereby hindered fair comparison and efficient model development (Jia et al., 24 Nov 2025).

1. Benchmark definition and design objectives

OceanForecastBench is framed as a benchmark dataset for data-driven global ocean forecasting whose three core contributions are: a high-quality global ocean reanalysis data over 28 years for model training, a high-reliability satellite and in-situ observations for model evaluation covering approximately 100 million locations in the global ocean, and an evaluation pipeline with six typical baseline models (Jia et al., 24 Nov 2025). The benchmark is explicitly intended to standardize both data usage and evaluation procedure, addressing the problems that different groups had been using different reanalyses, domains, resolutions, auxiliary inputs, and observation protocols.

The benchmark’s problem definition is global and three-dimensional. At time $t$, the input is a tensor

$$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$

and the model predicts

$$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$

for lead times $\Delta t = 1,\ldots,k$, with $k \le 10$ in the reported experiments and $C_{out}=94$ because the two wind components are inputs only and not forecast variables (Jia et al., 24 Nov 2025).

A central design choice is the separation between training and final evaluation. Training and internal validation use standardized reanalysis-derived fields, whereas final evaluation uses independent observations from EN4, GDP, and CMEMS L3. This is presented as a correction to a common practice in which models are validated only against the same reanalysis used for training; the benchmark argues that such evaluation can hide biases and errors. In that sense, OceanForecastBench adopts an observation-anchored philosophy rather than a purely emulator-style one (Jia et al., 24 Nov 2025).

2. Standardized training dataset and state representation

The core training dataset is built primarily from the GLORYS12 “Global Ocean Physics Reanalysis,” supplemented with ERA5 surface winds and OSTIA SST (Jia et al., 24 Nov 2025). GLORYS12 supplies sea surface height, temperature, salinity, eastward current velocity, northward current velocity, and mean dynamic topography. ERA5 provides 10 m zonal and meridional winds, and OSTIA provides daily SST. These heterogeneous sources are regridded and standardized into a single common tensor representation intended for deep learning workflows.

The common horizontal grid spans 80°S to 90°N and 180°W to 180°E, with grid size 121 $\times$ 256 at 1.40625° $\times$ 1.40625° resolution (Jia et al., 24 Nov 2025). Vertically, GLORYS12’s 50 levels are downsampled to 23 fixed $z$-levels from 0.49 m to 643.57 m: $$\begin{aligned} &0.49,\ 2.65,\ 5.08,\ 7.93,\ 11.41,\ 15.81,\ 21.60,\ 29.44,\ 40.34,\ 55.76,\ 77.85,\ &92.32,\ 109.73,\ 130.67,\ 155.85,\ 186.13,\ 222.48,\ 266.04,\ 318.13,\ 380.21,\ &453.94,\ 541.09,\ 643.57\ \text{m}. \end{aligned}$$

The resulting channel layout contains four sea-surface variables and four three-dimensional variables across 23 depth levels. The sea-surface variables are SST, SLA, $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$0, and $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$1. The three-dimensional variables are temperature, salinity, zonal current velocity, and meridional current velocity. Accordingly, the input channel count is

$$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$2

The output excludes the two wind components and therefore has $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$3 channels (Jia et al., 24 Nov 2025).

Land–sea masking is handled by extracting an ocean–land mask from GLORYS12 and setting land points to $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$4 in the data arrays before model ingestion (Jia et al., 24 Nov 2025). This preserves tensor regularity while keeping the representation compatible with standard CNN and transformer pipelines.

The temporal split is purely chronological. The training set covers 1993–2017, and the validation set covers 2018–2020 (Jia et al., 24 Nov 2025). This temporal split mirrors the benchmark’s general emphasis on out-of-sample evaluation in time rather than arbitrary shuffling. A plausible implication is that OceanForecastBench is intended to measure operational generalization rather than interpolation across a stationary pooled dataset.

3. Observation-based evaluation corpus and collocation procedure

OceanForecastBench’s evaluation set is built from three observational sources: EN4 temperature and salinity profiles, GDP drifters for SST and 15 m surface currents, and CMEMS L3 altimetry for sea level anomaly (Jia et al., 24 Nov 2025). The benchmark states that these sources together provide approximately 100 million locations in the global ocean and that only observations passing source quality flags are used.

The three observation streams serve distinct roles. EN4 provides temperature and salinity profiles, primarily from Argo floats plus historical casts, and is used as ground truth for subsurface $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$5 and $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$6. GDP drifters provide SST and 15 m eastward and northward velocity and are used for surface temperature and upper-ocean current verification. CMEMS L3 altimetry provides SLA along satellite tracks and is used for dynamic sea level verification (Jia et al., 24 Nov 2025). The evaluation period uses 2022–2023 observations, which are outside the 1993–2020 training/validation interval.

A concise summary of the evaluation corpus is as follows.

Source	Variables	Evaluation role
EN4.2.2	Temperature, salinity	Subsurface $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$7 verification
GDP	SST, 15 m $$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$8	Surface temperature and current verification
CMEMS L3 altimetry	SLA	Dynamic sea level verification

The core evaluation operation is a mapping

$$X^t \in \mathbb{R}^{C_{in} \times H \times W},\quad C_{in}=96,\ H=121,\ W=256,$$9

which takes model forecasts on the regular benchmark grid and interpolates them to observation positions and depths (Jia et al., 24 Nov 2025). This includes temporal alignment to the correct forecast lead time, spatial interpolation to observation longitude and latitude, and vertical interpolation from the 23 benchmark depth levels to observed depths for $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$0, $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$1, and currents. Observations outside the valid grid or over land are discarded.

This collocation-centered procedure is one of the benchmark’s defining features. Rather than evaluating only on the benchmark grid, it evaluates in observation space. That choice places OceanForecastBench closer to operational verification practice than to image-to-image forecasting benchmarks.

4. Forecasting tasks, metrics, and baseline systems

The benchmark task is to forecast the global three-dimensional ocean state and selected surface fields at lead times from 1 to 10 days (Jia et al., 24 Nov 2025). The variables evaluated are temperature and salinity in three dimensions, SST, SLA, and 15 m zonal and meridional currents. Model outputs are global, but scores are also broken down by depth and by region.

OceanForecastBench defines four standard evaluation metrics. Let $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$2 be forecast values, $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$3 observations, $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$4 climatology, and $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$5 the number of evaluation points. Then: $$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$6

$$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$7

$$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$8

$$\hat{X}^{t+1:t+k} = (\hat{X}^{t+1}, \ldots, \hat{X}^{t+k}),\quad \hat{X}^{t+\Delta t} \in \mathbb{R}^{C_{out} \times H \times W},$$9

These metrics are aggregated globally and also resolved by region and depth (Jia et al., 24 Nov 2025).

The benchmark includes six baselines: one operational numerical model and five deep learning systems (Jia et al., 24 Nov 2025). PSY4 is the numerical baseline, taken from Mercator Ocean’s operational “GLOBAL_ANALYSISFORECAST_PHY_001_024” product and downsampled to the benchmark grid. The deep learning baselines are ResNet, SwinTransformer, ClimaX, FourCastNet, and XiHe. Their reported parameter counts are approximately 51.4 million, 38.2 million, 65.7 million, 20.9 million, and 130 million, respectively (Jia et al., 24 Nov 2025).

This baseline suite places OceanForecastBench in a broader methodological landscape. It overlaps with the global forecast framing of FuXi-Ocean, which formulates a global 6-hourly, multi-depth, multivariate forecasting system and emphasizes standardized splits and skill curves (Huang et al., 3 Jun 2025), while also connecting to probabilistic and graph-based systems such as Njord, which is explicitly evaluated on OceanBench-style global observation tracks and ensemble metrics (Holmberg et al., 14 May 2026). The benchmark therefore supports comparison across CNNs, vision transformers, Fourier-operator models, ocean-specific transformers, and numerical forecasts.

5. Reported performance patterns and benchmark findings

OceanForecastBench reports several consistent skill patterns across variables and lead times (Jia et al., 24 Nov 2025). Error increases with lead time for all models: RMSE rises, while ACC and CSS decline. This is presented as the expected effect of accumulated forecast error in a chaotic ocean system.

For subsurface temperature and salinity, XiHe is reported as the strongest deep learning model, with the lowest RMSE and highest ACC, particularly at longer lead times (Jia et al., 24 Nov 2025). PSY4 performs well at short lead times but degrades faster with lead time. For SST, PSY4 remains the strongest model, maintaining lower RMSE and higher ACC and CSS than the deep learning baselines even at 10 days. For SLA, PSY4 is best up to approximately 6 days, while XiHe overtakes it beyond approximately 7 days as PSY4’s RMSE grows faster (Jia et al., 24 Nov 2025).

For surface currents at 15 m, all models exhibit relatively low ACC, below approximately 0.5–0.6, and the benchmark attributes this to the scarcity and noisiness of velocity observations together with known deficits in modeling ocean currents (Jia et al., 24 Nov 2025). Deep learning models often outperform PSY4 at longer lead times for these current variables, but performance remains weak overall.

The benchmark also reports structured spatial and vertical error behavior. All models show high RMSE in coastal regions and western boundary currents and low RMSE in open tropical oceans (Jia et al., 24 Nov 2025). Vertically, errors peak around the thermocline, approximately 100–200 m, where vertical gradients are strong and model skill is reduced. XiHe is reported to have the best vertical performance across depths, with the lowest RMSE and highest ACC (Jia et al., 24 Nov 2025).

These findings address a common misconception that a single global score is sufficient to characterize ocean forecast quality. OceanForecastBench shows that relative model ranking depends strongly on variable, depth, lead time, and region. In particular, the benchmark reports that numerical systems remain very strong for surface-constrained variables such as SST and short-lead SLA, while deep learning systems can be stronger for some longer-lead and subsurface tasks (Jia et al., 24 Nov 2025).

6. Relation to adjacent benchmark efforts, limitations, and extensions

OceanForecastBench belongs to a broader progression from interpolation-oriented ocean benchmarks to forecast-oriented ones. OceanBench: The Sea Surface Height Edition standardized SSH interpolation pipelines, observation operators, spectral evaluation, and configurable data handling, but it was explicitly scoped to interpolation rather than forecasting (Johnson et al., 2023). OceanForecastBench extends the benchmark idea to global three-dimensional forecasting with observation-based verification (Jia et al., 24 Nov 2025). A later OceanBench-related study on neural forecasting from sparse altimetry data further demonstrates how lead-time-aware, observation-collocated evaluation can be adapted to short-term SLA forecasting (Botvynko et al., 15 Dec 2025).

The benchmark’s own limitations are explicit. Its spatial resolution is 1.40625°, which is too coarse to resolve eddies and many coastal processes; its vertical extent reaches only the upper $\Delta t = 1,\ldots,k$0 m; it includes only physical ocean variables and selected surface forcings; and it supports deterministic forecasts only, without benchmark-native probabilistic evaluation (Jia et al., 24 Nov 2025). Observation coverage is also uneven, especially for currents and for some periods of SST evaluation. These limitations matter when interpreting results: low-resolution global skill should not be conflated with eddy-resolving or coastal skill.

Several subsequent model papers suggest plausible benchmark extensions. SeaCast defines a high-resolution regional forecasting setup with explicit boundary forcing and graph-based handling of irregular coastlines (Holmberg et al., 2024). FuXi-ONS and Njord introduce ensemble ocean forecasting with CRPS- and spread-skill-oriented evaluation, indicating a clear path toward probabilistic benchmark tracks (Huang et al., 20 Mar 2026); (Holmberg et al., 14 May 2026). Ocean-E2E shows how end-to-end neural assimilation and extreme-event metrics such as CSI can be incorporated for marine heatwave forecasting (Shu et al., 28 May 2025). KFTD highlights continuous-time forecasting, arbitrary sub-step interpolation, and modular PDE-regularized losses in OceanForecastBench-style regional tasks (Chen et al., 6 Jun 2026).

Taken together, these developments suggest that OceanForecastBench functions both as a benchmark and as a reference interface. It standardizes a global, multivariate, observation-based evaluation problem, while also exposing the methodological gaps that later systems attempt to close: probabilistic forecasting, physical consistency metrics, higher resolution, regional nesting, and multi-timescale prediction.