ThousandWorlds Exoclimate Benchmark

• ThousandWorlds is a benchmark for exoclimate emulation, mapping eight planetary parameters to comprehensive 3D atmospheric fields.
• It integrates around 1,760 simulations from five global climate models, harmonizing diverse outputs via advanced regridding and interpolation.
• The framework supports nested tasks from single-simulator to multi-simulator regression with structured missingness, advancing uncertainty quantification.

Searching arXiv for the ThousandWorlds benchmark and closely related exoclimate-emulation context. ThousandWorlds is a benchmark dataset and evaluation suite for machine-learning emulation of the 3D climates of potentially habitable exoplanets. It was introduced as an “ML-ready benchmark for exoclimate emulation and for the broader regime of low-data, multi-simulator, parameter-to-field regression,” motivated by the fact that interpreting atmospheric biosignatures requires climate context while individual global climate model runs can require up to millions of core-hours and substantial domain expert time (Stevenson et al., 16 Jun 2026). The benchmark contains approximately 1800 simulations from five global climate models, mapping eight planetary parameters to 3D atmospheric fields including temperature, humidity, winds, clouds, and radiation, and it defines progressively harder tasks spanning single-simulator regression, multi-simulator regression, and multi-simulator regression with structured missingness (Stevenson et al., 16 Jun 2026).

1. Definition and scientific role

ThousandWorlds is centered on exoplanet climate emulation for tidally locked, ocean-covered rocky planets around cool stars. In this setting, the target is not weather forecasting or time stepping, but parameter-to-field regression: given planetary descriptors such as radius, gravity, rotation period, surface pressure, atmospheric composition, incident stellar flux, and stellar effective temperature, predict the equilibrium climate fields produced by a 3D GCM (Stevenson et al., 16 Jun 2026).

The scientific motivation is tied directly to biosignature interpretation. The same molecule may indicate life on one planet and abiotic chemistry on another; the benchmark therefore treats climate as an essential latent context for atmospheric interpretation. A plausible implication is that ThousandWorlds occupies a middle layer in an end-to-end astrobiological inference stack: between planetary bulk parameters and observational or retrieval-facing spectral interpretation. This suggests that the benchmark is not only a machine-learning resource, but also an attempt to formalize a scientifically consequential surrogate-modelling problem.

The benchmark was also framed as a response to the absence of a curated, multi-model exoclimate dataset analogous to established Earth-system ML resources. Its creators emphasize that exoplanet climate studies had been scattered across groups, with differing formats, vertical grids, and diagnostic sets, and that this heterogeneity had limited systematic emulator development (Stevenson et al., 16 Jun 2026).

2. Dataset design and planetary parameterization

The dataset contains 1,760 total GCM runs, summarized in the abstract as “approximately 1800 simulations,” drawn from five exoplanet climate models: UM, ExoCAM, ExoCAM-pre-2022, LFRic, and ExoPlaSim (Stevenson et al., 16 Jun 2026). Two of these, UM and post-2022 ExoCAM, serve as the target GCMs for evaluation, while the remaining models contribute auxiliary training data.

Each simulation corresponds to a tidally locked waterworld. The planets are parameterized by eight continuous inputs together with a discrete GCM label. The continuous inputs span the following ranges within the target physical domain (Stevenson et al., 16 Jun 2026):

Parameter	Range
Radius $R$ (Earth radii)	$[0.7,\,1.4]$
Surface gravity $g$ (m s$^{-2}$)	$[6.0,\,16.0]$
Rotation period $P_{\text{rot}}$ (days)	$[0.1,\,1000.0]$
Surface pressure $P_0$ (bar)	$[0.5,\,5]$
CO$_2$ volume fraction (%)	$[0.7,\,1.4]$0
CH$[0.7,\,1.4]$1 volume fraction (%)	$[0.7,\,1.4]$2
Incident stellar flux $[0.7,\,1.4]$3 (W m$[0.7,\,1.4]$4)	$[0.7,\,1.4]$5
Stellar effective temperature $[0.7,\,1.4]$6 (K)	$[0.7,\,1.4]$7

The benchmark merges simulations from prior literature with 397 bespoke runs chosen to fill gaps in the eight-dimensional parameter space. Those bespoke runs are selected using a weighted coverage design,

$[0.7,\,1.4]$8

where $[0.7,\,1.4]$9 denotes the existing design and $g$0 downweights implausible regions, including very dense small planets and inconsistent rotation periods (Stevenson et al., 16 Jun 2026). The weighting decomposes into a prior on bulk density given radius and a prior on rotation period inferred from stellar properties and habitable-zone orbital period. This suggests that ThousandWorlds was designed not merely as a space-filling benchmark, but as a hybrid of statistical coverage and astrophysical plausibility.

The target of prediction is the time-mean equilibrium climate. The authors standardize all simulations to a common $g$1 latitude–longitude grid and, for 3D variables, 10 pressure levels, yielding about $g$2 scalar outputs per simulation (Stevenson et al., 16 Jun 2026). Output variables include temperature, specific humidity, zonal and meridional wind, cloud fraction, surface temperature, outgoing longwave radiation, and absorbed shortwave radiation, for a total of 53 fields in the full dataset.

3. Preprocessing, harmonization, and missingness

A major contribution of ThousandWorlds is homogenization across heterogeneous exoplanet GCM outputs. Different source models use different horizontal grids, different vertical coordinates, and different diagnostic subsets. The benchmark regrids all simulations horizontally by bilinear interpolation to a T21 Gaussian grid and interpolates vertically in log-pressure to 10 relative isobars defined by

$g$3

with $g$4 and $g$5 (Stevenson et al., 16 Jun 2026).

The preprocessing is variable-specific. Rotation period and surface pressure are log-transformed. CO$g$6 and CH$g$7 volume fractions are transformed using

$g$8

with species-specific pivots $g$9 for CO$^{-2}$0 and $^{-2}$1 for CH$^{-2}$2, chosen so that the transformation behaves roughly logarithmically at climatically significant abundances (Stevenson et al., 16 Jun 2026). Humidity is log-transformed. Cloud fraction is mapped through a smoothed logit,

$^{-2}$3

with predictions later mapped back and clamped to $^{-2}$4 (Stevenson et al., 16 Jun 2026). Absorbed shortwave radiation and outgoing longwave radiation are normalized by each planet’s incident stellar flux.

Because the forcing is equatorially symmetric, the released data imposes symmetry on long-time means: symmetric fields are averaged across hemispheres, while the meridional wind is antisymmetrized (Stevenson et al., 16 Jun 2026). This reduces residual numerical asymmetries not directly tied to the eight planetary inputs.

The benchmark explicitly represents structured missingness. Some fields or pressure levels are absent because different GCMs output different diagnostics, because native vertical extent differs, or because isobaric remapping interacts irregularly with model tops and bottoms. Partial availability across the horizontal grid is treated as fully unobserved in order to avoid interpolation bias (Stevenson et al., 16 Jun 2026). This design decision is central to the hardest benchmark subset and makes ThousandWorlds relevant beyond exoclimate, as a test case for missing-not-at-random multi-output regression.

4. Multi-model architecture and benchmark tasks

ThousandWorlds is deliberately multi-simulator. The five source models span distinct dynamical cores, radiative transfer treatments, and physics packages. UM and ExoCAM are treated as the two “target” models; ExoCAM-pre-2022, LFRic, and ExoPlaSim are auxiliary models used for training but not primary evaluation (Stevenson et al., 16 Jun 2026). The benchmark interprets the disagreement between target GCMs as epistemic uncertainty associated with structural differences in frontier climate models.

Three nested subsets define progressively harder tasks (Stevenson et al., 16 Jun 2026):

Subset	Core setting	Train/test
Single-complete	UM only, 48 fields, no missingness	206 train, 50 test
Multi-complete	All 5 GCMs, 48 common fields, no missingness	1,538 train, 90 test
Multi-partial	All 5 GCMs, full 53 fields, structured missingness	1,626 train, 100 test

In Single-complete, the task is pure parameter-to-field regression within a single simulator. In Multi-complete, the model must exploit multi-GCM training data while conditioning on the GCM label and avoiding confusion from systematic inter-model biases. In Multi-partial, the model must additionally cope with structured missing outputs. Train/test splitting is designed to avoid leakage: no planet appearing in test with one GCM appears in train with another GCM, and auxiliary-GCM runs corresponding to test planets are also removed from training (Stevenson et al., 16 Jun 2026).

This task hierarchy encodes a specific methodological claim: realistic scientific emulation rarely involves a single simulator with complete observations. A plausible implication is that ThousandWorlds functions as a benchmark for transfer, heterogeneity management, and incomplete supervision as much as for climate emulation per se.

5. Evaluation protocols and baseline methods

The benchmark defines two complementary evaluation protocols. The primary deterministic metric is the area-weighted RMSE,

$^{-2}$5

where $^{-2}$6 encodes Gaussian latitude weights (Stevenson et al., 16 Jun 2026). For probabilistic methods, the principal score is the energy score,

$^{-2}$7

supplemented by metrics such as anomaly correlation coefficient and spread–skill ratio (Stevenson et al., 16 Jun 2026).

The first protocol is standard ranking on the full test set. The second, more distinctive protocol evaluates performance relative to GCM disagreement using planets simulated by both UM and ExoCAM. For RMSE, the benchmark defines

$^{-2}$8

where $^{-2}$9 is the target GCM output and $[6.0,\,16.0]$0 is the other target GCM’s output for the same planet (Stevenson et al., 16 Jun 2026). Values below 1 indicate that the emulator is, on average, closer to the target GCM than the other frontier GCM is.

Seven baselines are reported (Stevenson et al., 16 Jun 2026). These span simple methods, deep learning, and Gaussian-process approaches:

• Train-mean: predicts the area-weighted training mean per field.
• k-nearest neighbours: averages outputs of neighbouring training planets in standardized input space.
• Coord-MLP: pointwise coordinate-conditioned MLP.
• Coord-DeepONet: operator-learning model with branch and trunk networks.
• PCA-MLP: spectral PCA plus an MLP on latent scores.
• PPCA-ICM: probabilistic PCA plus a multi-task GP with an intrinsic coregionalization model across GCMs.
• GPLFR: Gaussian Process Latent Factor Regression, an end-to-end GP latent factor model.

The benchmark reports that GP-based methods perform best overall, with GPLFR almost always giving the lowest RMSE and PPCA-ICM usually second, while PCA-MLP is the strongest deep-learning baseline (Stevenson et al., 16 Jun 2026). kNN is described as surprisingly competitive for clouds, winds, and absorbed shortwave radiation, and Coord-MLP underperforms, especially on absorbed shortwave radiation. The authors also report that removing auxiliary GCMs hurts GP methods more than deep networks, suggesting that GP models exploit cross-GCM structure more effectively (Stevenson et al., 16 Jun 2026).

6. Interpretation, significance, and limitations

The principal empirical conclusion is that ThousandWorlds exposes a regime in which off-the-shelf deep learning does not yet dominate. GP-based methods are strongest despite the high-dimensional outputs, implying that inductive bias, probabilistic structure, and uncertainty handling are especially valuable in low-data, multi-simulator, parameter-to-field regression (Stevenson et al., 16 Jun 2026). This suggests that the benchmark is not merely a new dataset for existing pipelines, but an argument about where current scientific ML methods succeed and fail.

The shared-planets protocol is particularly significant because it rescales emulator error by inter-GCM disagreement. The paper reports that most methods achieve relative RMSE below 1 for most variables and that GPLFR attains a geometric mean relative RMSE of 0.616 on Multi-partial, while relative energy scores are even better for the strongest GP models (Stevenson et al., 16 Jun 2026). The interpretation offered is pragmatic: if frontier GCMs themselves disagree at a certain level, an emulator whose error is smaller than that disagreement may already be useful for parameter sweeps, uncertainty propagation, and retrieval coupling.

The benchmark also states clear limitations. Its planetary scope is restricted to tidally locked waterworlds, excluding continents, obliquity, eccentricity, and asynchronous rotation (Stevenson et al., 16 Jun 2026). The multi-model ensemble includes only five GCMs, with two related pairs and one lower-fidelity model dominating the auxiliary set. Hidden configuration differences within a nominal GCM lineage, such as cloud schemes or ocean salinity assumptions, are not encoded in the eight inputs and therefore appear as structured noise. Test sets are relatively small, and all emulators inherit the biases of the source GCMs they approximate (Stevenson et al., 16 Jun 2026).

These limitations matter for interpretation. ThousandWorlds does not provide validated planetary truth; it provides surrogates for a particular ensemble of high-end numerical models. A plausible implication is that its most reliable use cases are comparative and exploratory: sensitivity analysis, approximate uncertainty propagation, benchmarked emulator development, and coupling to retrieval frameworks. The benchmark is less suited to unsupported extrapolation into qualitatively different planetary regimes.

7. Broader meanings of “ThousandWorlds”

Although the benchmark itself is a 2026 scientific-ML resource, the phrase “ThousandWorlds” resonates with older themes in physics and astronomy. In Everettian discussions of pure wave mechanics, branch structure and “typical worlds” are invoked to explain why observers in a deterministic multiverse would nonetheless see Born-rule frequencies; Barrett’s analysis emphasizes that such accounts require auxiliary assumptions about branch structure, measures, and the bridge from typicality to probability (Barrett, 2019). A plausible implication is that the benchmark’s name evokes plurality and typicality without importing Everettian ontology into climate science.

In exoplanet instrumentation, “A Thousand Earths” proposed the Nautilus concept, an array of 35 identical 8.5 m telescopes using MODE lenses to characterize roughly 1,000 transiting habitable-zone Earth-sized planets via transit spectroscopy, explicitly framing the problem as a shift from a small “hero sample” to a population-level biosignature mission (Apai et al., 2019). ThousandWorlds occupies a complementary computational role: it is not an observing architecture, but it addresses the interpretive bottleneck that such a large observational sample would create.

In planetary composition studies, Gaidos argued that most nearby rocky planets should emerge from oxygen-rich dust with near-solar $[6.0,\,16.0]$1, that intrinsically carbon-rich systems with $[6.0,\,16.0]$2 are extraordinarily rare, and that most “little worlds” in the solar neighborhood should be silicate-metal planets with variation driven more by redox state and volatile history than by exotic bulk chemistry (Gaidos, 2015). This provides useful astrophysical context for ThousandWorlds: the benchmark samples a rich climatic parameter space, but not an unconstrained planetary chemistry space.

Taken together, these neighboring literatures highlight distinct meanings of plurality. In Everettian foundations, many worlds motivate questions about typicality and probability (Barrett, 2019). In telescope architecture, a thousand worlds motivate large-sample observational design (Apai et al., 2019). In exoplanet geochemistry, many little worlds are chemically diverse but bounded by Galactic chemical evolution and dust-controlled planet formation (Gaidos, 2015). In ThousandWorlds proper, the plurality is computational and comparative: many simulated climates, many models, and many planetary parameter combinations, organized as a benchmark for learning scientifically meaningful emulators (Stevenson et al., 16 Jun 2026).