CIENS Dataset: Station-Centric Weather Benchmark

• CIENS is a station-centric meteorological benchmark combining convection-permitting ensemble forecasts with matched synoptic observations from 170 German stations.
• It features 20-member ensemble forecasts for 55 variables across multiple DWD model generations, supporting detailed probabilistic forecast verification and calibration.
• The dataset incorporates spatial predictors and non-stationary elements, providing a robust testbed for evaluating innovative post-processing and machine-learning methods.

Searching arXiv for the CIENS dataset paper and closely related benchmark context. CIENS is a station-centric meteorological benchmark dataset that combines operational convection-permitting ensemble numerical weather prediction forecasts from the German Weather Service with matched synoptic-station observations in Germany. The acronym denotes “Operational convection-permitting COSMO/ICON ensemble predictions at observation sites,” and the dataset is designed for ensemble post-processing, forecast verification, and machine-learning-based probabilistic forecasting. Its distinguishing characteristics are the pairing of forecasts and observations at station locations, the inclusion of 20-member ensemble forecasts for 55 meteorological variables, and a long temporal span from 8 December 2010 to 30 June 2023 that crosses multiple major operational model upgrades, thereby making non-stationarity a first-order property of the benchmark rather than an incidental complication (Lerch et al., 5 Aug 2025).

1. Definition and benchmark role

CIENS is an openly available dataset of operational convection-permitting ensemble forecasts mapped to observation sites, augmented with matched station observations at 170 synoptic stations in Germany. It contains forecasts from three successive regional ensemble systems of the German Weather Service—COSMO-DE-EPS, COSMO-D2-EPS, and ICON-D2-EPS—each represented as a 20-member ensemble, with two daily initializations at 00 and 12 UTC and hourly lead times from 0 to 21 hours (Lerch et al., 5 Aug 2025).

The dataset occupies a specific position within the ecosystem of weather-and-climate benchmarks. In contrast to global gridded datasets such as WeatherBench and WeatherBench 2, CIENS is station-centric and convection-permitting rather than global and grid-native. In contrast to EUPPBench, it focuses on operational regional forecasts at convection-permitting resolution and emphasizes a long, continuous operational archive rather than a shorter archive extended by reforecasts. This makes CIENS particularly suitable for research on probabilistic forecast calibration, station-level verification, feature-rich post-processing, and temporal robustness under operational model changes (Lerch et al., 5 Aug 2025).

A central implication is methodological rather than merely archival: CIENS is not only a repository of forecast–observation pairs, but a testbed for studying how post-processing schemes respond when the underlying forecasting system changes over time. The long record spanning multiple model generations suggests that any method evaluated on CIENS must confront regime shifts, varying forecast skill, and evolving predictor–target relationships rather than assuming a single stationary data-generating process.

2. Forecast systems, temporal extent, and observational backbone

CIENS spans the period from 8 December 2010 through 30 June 2023. Over this interval, it incorporates forecasts from three operational DWD convection-permitting ensemble systems. COSMO-DE-EPS is included from 8 Dec 2010 00 UTC to 15 May 2018 12 UTC, COSMO-D2-EPS from 15 May 2018 12 UTC onward, and ICON-D2-EPS from 10 Feb 2021 12 UTC onward. All are convection-permitting regional ensemble systems with horizontal resolution on the order of roughly 2–3 km, and all contribute 20 exchangeable ensemble members per run (Lerch et al., 5 Aug 2025).

The forecast archive is structured around two initialization times per day, 00 UTC and 12 UTC, with hourly lead times from 0 to 21 hours. This lead-time design makes the dataset particularly appropriate for short-range post-processing, nowcasting-adjacent verification, and diurnally resolved calibration studies, while making it less suitable for medium-range prediction research. That limitation is explicit in the dataset design: CIENS is a short-range operational benchmark rather than a multi-day forecast archive (Lerch et al., 5 Aug 2025).

The observational component consists of six variables at 170 synoptic stations in Germany:

Observed variable	Unit
air_temperature	K
air_pressure	Pa
precipitation_amount	kg m$^{-2}$
wind_speed	m s$^{-1}$
wind_speed_of_gust	m s$^{-1}$
wind_from_direction	degrees

These observations are extracted from WMO synoptic observations distributed via the GTS. CIENS performs extraction, conversion, and subsetting to NetCDF, while relying on DWD standard quality control in the original data stream rather than imposing a new QC framework of its own (Lerch et al., 5 Aug 2025).

The forecast side provides 55 meteorological variables at each station, including near-surface variables, upper-air variables on five pressure levels, and diagnostic fields relevant for convection and storms. Variables are represented either as instantaneous fields at forecast hour or as accumulations over the preceding hour, depending on their physical meaning. A plausible implication is that CIENS supports both direct univariate post-processing, such as for temperature or gusts, and richer multivariate or feature-based methods that exploit physical covariates beyond the target variable itself.

3. Station mapping, spatial predictors, and data representation

A defining design choice in CIENS is that gridded NWP output is transformed into station-level predictors before release. For each ensemble member, variable, lead time, and initialization, the “standard variable” forecast at a station is obtained from the nearest model grid point to that station’s location. If $X^{(k)}(i,j,\tau)$ denotes the value of variable $X$ for member $k$ at grid point $(i,j)$ and lead time $\tau$, and $(i^*,j^*)$ is the nearest grid point to station $s$, then the station forecast is

$^{-1}$0

This mapping is carried out independently for each member and variable (Lerch et al., 5 Aug 2025).

CIENS extends this pointwise representation with neighborhood-based spatial predictors. Around each station’s nearest grid point, the dataset constructs two neighborhoods, $^{-1}$1 and $^{-1}$2 grid points, and computes for selected variables the neighborhood mean and neighborhood standard deviation for each ensemble member and lead time. Formally, for a neighborhood $^{-1}$3 of size $^{-1}$4,

$^{-1}$5

and

$^{-1}$6

These derived quantities are stored as “spatial variables” and encode mesoscale context around a station location (Lerch et al., 5 Aug 2025).

This design is consequential. Many post-processing datasets provide either raw gridded fields or station-matched forecasts, but not both station-level values and precomputed neighborhood summaries. CIENS chooses the latter representation, which lowers the entry barrier for statistical and ML workflows while retaining some spatial context. A plausible implication is that the dataset is especially convenient for tabular, distributional-regression, and boosting-based methods, whereas it is less directly suited to CNN-style models that operate on full two-dimensional forecast fields.

Another cross-system consistency intervention concerns wind components: ICON-D2 stores truly geographic $^{-1}$7 and $^{-1}$8, whereas COSMO periods use rotated-grid components. CIENS rotates ICON-D2 wind components so that they are consistent with the COSMO periods across the full archive (Lerch et al., 5 Aug 2025). This is an important harmonization step for longitudinal studies, since otherwise apparent temporal changes in predictor behavior could partly reflect coordinate inconsistency rather than meteorological or model differences.

4. File organization, access structure, and preprocessing choices

All CIENS components are distributed in NetCDF format. The archive is partitioned into a parent entry and four sub-datasets: Run 00 UTC standard variables with observations, Run 00 UTC spatial variables, Run 12 UTC standard variables, and Run 12 UTC spatial variables. Observations are included only in the Run 00 UTC package to avoid duplication (Lerch et al., 5 Aug 2025).

The directory organization is year- and month-structured, with daily forecast files containing all ensemble members, all lead times, all stations, and all variables for the relevant run. Observations are stored in annual files. The resulting organization is analysis-ready in a precise sense: station IDs match across forecasts and observations, and valid times are obtained directly as initialization time plus lead time, so forecast–observation joining does not require bespoke regridding or temporal alignment logic (Lerch et al., 5 Aug 2025).

The dataset is preprocessed but not post-processed. Specifically, CIENS has already performed station mapping, spatial aggregation, wind-component harmonization across model families, and extraction of the six observational variables into NetCDF. However, it deliberately does not apply bias correction or calibration; forecasts remain raw ensemble output (Lerch et al., 5 Aug 2025). This distinction matters because it preserves CIENS as a benchmark for post-processing rather than a dataset of already corrected products.

The following table summarizes the main structural components.

Component	Contents	Format
Run 00 UTC standard package	Standard station forecasts + observations	NetCDF
Run 00 UTC spatial package	Neighborhood mean and sd predictors	NetCDF
Run 12 UTC standard package	Standard station forecasts	NetCDF
Run 12 UTC spatial package	Neighborhood mean and sd predictors	NetCDF

From a workflow perspective, this partitioning allows users to restrict downloads to the predictor families relevant to a given study. For example, classical EMOS baselines can be implemented from the standard package alone, whereas richer ML post-processing can add the spatial package without needing any regridding pipeline.

5. Verification framework and post-processing use case

CIENS is explicitly positioned as a benchmark for probabilistic forecast verification and post-processing. The paper uses proper scoring rules centered on the continuous ranked probability score. For predictive CDF $^{-1}$9 and observation $^{-1}$0,

$^{-1}$1

with equivalent representation

$^{-1}$2

for independent $^{-1}$3. The paper also uses the CRPS skill score relative to a reference forecast,

$^{-1}$4

as well as MAE, MSE, and prediction-interval width and coverage (Lerch et al., 5 Aug 2025).

The primary worked example in the CIENS paper is a wind-gust post-processing study at lead time 12 h using 00 UTC runs. The comparison includes the raw ensemble, classical EMOS with a truncated logistic predictive distribution, gradient-boosted EMOS using many predictors, and gradient-boosted EMOS extended with spatial predictors (Lerch et al., 5 Aug 2025). For wind gusts, the assumed predictive law is a logistic distribution truncated at zero,

$^{-1}$5

In classical EMOS, the location and scale are linked to the ensemble mean $^{-1}$6 and ensemble standard deviation $^{-1}$7:

$^{-1}$8

In gradient-boosted EMOS, this is generalized to a distributional regression using multiple standardized predictors,

$^{-1}$9

This use case is not merely illustrative; it demonstrates why CIENS was designed with many predictor variables and spatial summaries rather than only the target ensemble (Lerch et al., 5 Aug 2025).

The reported averaged scores over 169 stations show a consistent progression from raw forecasts to richer post-processing. Raw ensemble wind-gust forecasts yield CRPS $X^{(k)}(i,j,\tau)$0, MAE $X^{(k)}(i,j,\tau)$1, MSE $X^{(k)}(i,j,\tau)$2, and coverage $X^{(k)}(i,j,\tau)$3. EMOS improves CRPS to $X^{(k)}(i,j,\tau)$4 and coverage to approximately $X^{(k)}(i,j,\tau)$5. EMOS-GB further reduces CRPS to $X^{(k)}(i,j,\tau)$6, and EMOS-GB-SP, which adds spatial predictors, reduces CRPS to $X^{(k)}(i,j,\tau)$7 (Lerch et al., 5 Aug 2025). The paper interprets these increments as evidence that additional physical covariates and neighborhood information materially improve probabilistic forecasting skill.

This result is methodologically significant beyond wind gusts. It suggests that post-processing gains in CIENS arise not only from calibrating the target variable’s own ensemble statistics, but also from leveraging cross-variable and spatially contextual predictors. That is precisely the kind of regime in which modern ML and distributional-regression methods can outperform low-dimensional parametric baselines.

6. Non-stationarity, limitations, and research implications

The most distinctive scientific property of CIENS is that it is intentionally non-homogeneous in time. The archive spans the transitions from COSMO-DE-EPS to COSMO-D2-EPS and then to ICON-D2-EPS, together with smaller updates in physics, parameterizations, and data assimilation. The paper illustrates this by showing step changes in mean CRPS for 18-h wind-gust forecasts, smoothed with a 30-day running mean, aligned with operational model upgrades (Lerch et al., 5 Aug 2025).

This makes CIENS unusually valuable for research on adaptation under forecast-system drift. Classical post-processing frequently assumes that the relationship between ensemble predictors and observations is approximately stationary over the training period. CIENS violates that assumption by construction. As a result, it supports investigations into rolling-window training, time-varying parameters, regime-dependent models, transfer across model generations, and strategies that explicitly condition on system era or related metadata. This suggests that CIENS is as much a benchmark for robustness under operational change as it is a benchmark for raw forecast skill.

The dataset also has clear limitations. Its lead-time range ends at 21 h, so it does not address medium-range post-processing. Only six observation variables are included, deliberately prioritizing station coverage and low missingness over broader variable diversity. CIENS does not provide gridded forecast fields, only station-mapped values and neighborhood summaries, which limits direct use by spatial deep learning methods operating on two-dimensional NWP fields. Representativeness errors remain important, especially at high-altitude or coastal stations where model orography and resolved flows diverge from point observations. Finally, the long time span implies changes not only in the forecast system but potentially in station instrumentation and observing practice, requiring caution in long-horizon analyses (Lerch et al., 5 Aug 2025).

These caveats are not merely ancillary. They define the proper scope of CIENS. The dataset is best interpreted as a benchmark for short-range station-based probabilistic forecasting under realistic operational heterogeneity. It is not a universal weather dataset, nor a replacement for grid-native archives such as WeatherBench-like resources. Rather, it is specialized toward the class of problems in which forecast–observation pairing, local calibration, and operational drift matter more than direct access to full spatial fields.

7. Access, licensing, and relation to adjacent benchmarks

CIENS is hosted on KITOpen and distributed under a CC BY 4.0 license. The parent DOI is 10.35097/EOvvQEsgILoXpYTK, with separate DOIs for the four sub-datasets corresponding to 00 UTC and 12 UTC runs and to standard versus spatial variables. The accompanying GitHub repository, https://github.com/slerch/CIENS/, provides R code for reading the NetCDF files, matching forecasts to observations, constructing training and test datasets, fitting EMOS and EMOS-GB models via crch, and computing verification metrics with scoringRules (Lerch et al., 5 Aug 2025).

In the broader benchmark landscape, CIENS complements rather than duplicates other weather datasets. WeatherBench-style resources emphasize global gridded prediction and are well suited to large-scale data-driven forecasting, but they are not station-centric and are not tailored to convection-permitting operational ensembles. EUPPBench is closer in spirit as a forecast–observation benchmark for ensemble post-processing, yet differs in scale, forecast source, and temporal construction. CIENS adds a long operational archive of regional convection-permitting ensemble forecasts, mapped directly to stations and enriched with neighborhood predictors, thereby filling a niche at the intersection of operational NWP, probabilistic verification, and feature-rich ML post-processing (Lerch et al., 5 Aug 2025).

A plausible implication is that CIENS can serve as a canonical benchmark for a specific class of methods: those that ingest ensemble-member information, station identity, lead time, and physically meaningful covariates to produce calibrated predictive distributions at observation sites. It is also well suited to teaching and competition settings, because much of the technically difficult data engineering—forecast extraction, station matching, and predictor construction—has already been performed. The result is a dataset in which methodological comparisons can focus on modeling choices rather than preprocessing idiosyncrasies.

In that sense, CIENS represents a mature benchmark design: operationally realistic, technically structured, and sufficiently rich to support both classical statistical post-processing and modern machine-learning approaches, while making the challenge of non-stationarity central rather than avoidable.