CES: A Bayesian Inversion Framework

• CES is a three-stage framework for Bayesian inversion that uses ensemble calibration, GP emulation, and MCMC sampling to quantify uncertainty with fewer expensive forward evaluations.
• It leverages ensemble-based methods to concentrate parameter estimates in high-posterior regions, ensuring that the surrogate model is trained where accurate inference is most critical.
• CES has been applied successfully in climate model calibration and multiscale systems, providing significant computational speedups compared to direct MCMC while maintaining reliable posterior approximations.

Searching arXiv for CES and related calibrate-emulate-sample papers. Calibrate–Emulate–Sample (CES) is a three-stage framework for Bayesian inversion and Bayesian calibration in settings where the forward or parameter-to-data map is expensive to evaluate, derivatives or adjoints are unavailable, and forward evaluations may be noisy. In its canonical formulation, CES first uses an ensemble-based inverse method to move parameter ensembles toward regions of high posterior probability, then fits a surrogate model—typically a Gaussian process (GP)—to the resulting simulator evaluations, and finally samples an approximate Bayesian posterior with MCMC using the surrogate in place of the original simulator (Cleary et al., 2020). The framework was introduced as a derivative-free strategy for approximate Bayesian inference requiring only a small number of forward evaluations and has since been used in climate-model calibration and experimental design, where it enables uncertainty quantification with approximately $\mathcal{O}(10^2)$ simulator evaluations rather than the $\mathcal{O}(10^5)$ evaluations often associated with direct MCMC on the full model (Cleary et al., 2020, Dunbar et al., 2020, Dunbar et al., 2022).

1. Bayesian inverse-problem setting

CES is formulated for inverse problems of the form

$y = G(\theta) + \eta,$

where $\theta \in \mathbb{R}^p$ denotes unknown parameters, $y \in \mathbb{R}^d$ observed data, $G:\mathbb{R}^p \to \mathbb{R}^d$ the forward or parameter-to-data map, and $\eta$ observational noise (Cleary et al., 2020). In the simplest Bayesian setting described in the original CES formulation, both the noise and the prior are Gaussian, leading to a posterior density

$$\pi^y(\theta) \propto \exp\bigl(-\Phi_R(\theta)\bigr),$$

with negative log-posterior

$$\Phi_R(\theta)=\frac{1}{2}\|y-G(\theta)\|^2_{y}+\frac{1}{2}\|\theta\|^2.$$

The norm is defined by $\|v\|_A := \|A^{-1/2}v\|$, so the data misfit is weighted by the observation covariance (Cleary et al., 2020).

The motivating regime is one in which evaluating $\mathcal{O}(10^5)$0 is very expensive, gradients or derivative-adjoints are unavailable, and in some applications only noisy evaluations of $\mathcal{O}(10^5)$1 are accessible (Cleary et al., 2020). Standard MCMC remains the reference mechanism for principled posterior sampling and uncertainty quantification, but its evaluation cost can be prohibitive for large PDE and climate models. CES addresses this by separating the inferential task into a low-budget calibration stage, an emulator construction stage, and a posterior sampling stage carried out on the emulator rather than on the original simulator (Cleary et al., 2020, Dunbar et al., 2020).

In climate applications, the same logic is expressed in terms of time-averaged statistics. For finite averaging window $\mathcal{O}(10^5)$2, the forward map is written as $\mathcal{O}(10^5)$3, while the target statistical map is $\mathcal{O}(10^5)$4; internal variability is modeled through a central-limit-theorem approximation

$\mathcal{O}(10^5)$5

which makes direct likelihood-based MCMC both noisy and expensive (Dunbar et al., 2020). This use case exemplifies the class of problems for which CES was designed.

2. Three-stage architecture

CES consists of three coupled stages: calibration, emulation, and sampling (Cleary et al., 2020). The stages are conceptually distinct but computationally interdependent, because the simulator evaluations generated during calibration are reused as training data for the emulator, and the calibrated ensemble also supplies an initial state and scale information for MCMC in the sampling stage (Cleary et al., 2020).

The original methodological statement is that CES first uses an ensemble Kalman method to calibrate parameters and locate the high-posterior region cheaply, second uses those forward evaluations to emulate the map with a surrogate model, and third uses the surrogate inside MCMC to sample an approximate Bayesian posterior at low cost (Cleary et al., 2020). In climate-model applications, this same structure was implemented with ensemble Kalman inversion (EKI) for calibration, GP emulation of time-averaged model outputs, and MCMC on the emulator to approximate the posterior over uncertain convective parameters (Dunbar et al., 2020). In the subsequent experimental-design work, CES became the computational engine for repeated posterior estimation over many candidate design locations and seasons (Dunbar et al., 2022).

The practical significance of this decomposition is that the expensive simulator is used only in the early phase, often on the order of $\mathcal{O}(10^5)$6 evaluations, whereas the posterior-sampling phase is shifted to a cheap surrogate (Cleary et al., 2020, Dunbar et al., 2022). A central claim in the original paper is that ensemble Kalman sampling (EKS) solves, cheaply and derivative-free, the design problem of where to place points in parameter space to train the emulator efficiently for Bayesian inversion (Cleary et al., 2020). This distinguishes CES from workflows based solely on space-filling designs or from methods that attempt direct MCMC on the full simulator.

A recurrent misconception is that CES is merely a surrogate-modeling pipeline. In the cited formulations, the method is more specific: the calibration stage is not only a preliminary optimizer but also an adaptive design mechanism targeted at the main support of the posterior, and the sampling stage is intended to recover approximate Bayesian uncertainty quantification rather than only point estimates (Cleary et al., 2020, Dunbar et al., 2020).

3. Calibration stage: ensemble-based localization of posterior mass

In the original formulation, the calibration stage uses ensemble Kalman sampling (EKS), a stochastic interacting-particle dynamics for ensemble members $\mathcal{O}(10^5)$7:

$\mathcal{O}(10^5)$8

with

$\mathcal{O}(10^5)$9

and

$y = G(\theta) + \eta,$0

The first term drives data fit, the $y = G(\theta) + \eta,$1 term incorporates the prior, and the Brownian term makes the ensemble behave like a sampler rather than a pure optimizer (Cleary et al., 2020).

The paper discretizes this with a linearly implicit split-step scheme:

$y = G(\theta) + \eta,$2

$y = G(\theta) + \eta,$3

This stage is derivative-free and uses only forward evaluations of the simulator (Cleary et al., 2020).

In the climate-model variant, the calibration stage is implemented instead with EKI. The finite-time data model is

$y = G(\theta) + \eta,$4

with calibration misfit

$y = G(\theta) + \eta,$5

The EKI update is

$y = G(\theta) + \eta,$6

and is described as derivative-free, ensemble-based, robust to noisy forward evaluations, and computationally cheap relative to brute-force Bayesian sampling (Dunbar et al., 2020).

The principal role of the calibration stage across these CES variants is to concentrate parameter ensembles in regions of high posterior probability or near the best-fitting parameter region (Cleary et al., 2020, Dunbar et al., 2022). That property is methodologically crucial because the emulator is subsequently trained where approximation quality matters most for inference. At the same time, the climate-model paper explicitly notes an important limitation: EKI is an optimization method and its collapsing ensemble spread is not a reliable posterior uncertainty estimate (Dunbar et al., 2020). CES therefore does not equate ensemble spread in the calibration phase with posterior uncertainty; instead, it reserves uncertainty quantification for the final sampling phase.

4. Emulation stage: surrogate construction in posterior-relevant regions

After calibration, CES constructs a surrogate for the parameter-to-data map from the expensive forward evaluations already obtained. In the original paper, the calibration output provides training data

$y = G(\theta) + \eta,$7

with $y = G(\theta) + \eta,$8 in general, and often $y = G(\theta) + \eta,$9 by using the final EKS ensemble as the design set (Cleary et al., 2020). The stated rationale is that the EKS output is concentrated in regions of high posterior mass, so the emulator is trained precisely where accurate approximation matters most for Bayesian inference (Cleary et al., 2020).

The default emulator in CES is a Gaussian process, often fitted independently to each output component $\theta \in \mathbb{R}^p$0, $\theta \in \mathbb{R}^p$1, using a squared-exponential kernel with a nugget:

$\theta \in \mathbb{R}^p$2

Here $\theta \in \mathbb{R}^p$3 is the amplitude, $\theta \in \mathbb{R}^p$4 encodes lengthscales, and $\theta \in \mathbb{R}^p$5 represents observation or evaluation noise in the simulator outputs (Cleary et al., 2020). The emulator is represented as

$\theta \in \mathbb{R}^p$6

so the forward map is replaced by a probabilistic surrogate with predictive mean $\theta \in \mathbb{R}^p$7 and covariance $\theta \in \mathbb{R}^p$8 (Cleary et al., 2020).

The original CES paper assigns two functions to the emulator: cheap evaluation inside MCMC and noise reduction or smoothing when the original simulator is noisy, as in finite-time averages in chaotic systems (Cleary et al., 2020). In climate CES, the emulator is trained on finite-time outputs $\theta \in \mathbb{R}^p$9 but is intended to approximate the infinite-time or statistical map $y \in \mathbb{R}^d$0 (Dunbar et al., 2020, Dunbar et al., 2022). This shift from noisy finite-time evaluations to a smooth GP surrogate is one reason the final MCMC stage is more stable than direct sampling on the simulator.

Several papers emphasize decorrelation transforms before GP fitting. The original CES paper discusses diagonalizing the observation covariance or using PCA/SVD on the EKS-generated training outputs so that independent GP emulators can be trained in transformed coordinates (Cleary et al., 2020). The idealized GCM implementation diagonalizes $y \in \mathbb{R}^d$1 through

$y \in \mathbb{R}^d$2

and then trains scalar-valued GPs on the decorrelated components (Dunbar et al., 2020). The experimental-design paper likewise uses an SVD of the internal-variability covariance,

$y \in \mathbb{R}^d$3

and builds the emulator in a transformed coordinate system (Dunbar et al., 2022).

A plausible implication is that CES is not tied to a particular GP architecture so much as to a specific local-design principle: surrogate accuracy is prioritized in the posterior-relevant region rather than globally over the prior domain. That interpretation is explicit in the claim that EKS provides a cheap solution to the experimental-design problem of emulator training (Cleary et al., 2020).

5. Sampling stage: approximate posterior inference on the surrogate

Once the forward map is replaced by the emulator, CES samples from an approximate posterior in which $y \in \mathbb{R}^d$4 is replaced by $y \in \mathbb{R}^d$5:

$y \in \mathbb{R}^d$6

Depending on the treatment of emulator and observation uncertainty, the approximate negative log-likelihood takes several forms (Cleary et al., 2020). If emulator uncertainty is ignored,

$y \in \mathbb{R}^d$7

If simulator uncertainty is retained but observation noise is neglected,

$y \in \mathbb{R}^d$8

If both are included,

$y \in \mathbb{R}^d$9

The resulting approximate posterior is

$G:\mathbb{R}^p \to \mathbb{R}^d$0

MCMC is then run on this surrogate posterior rather than on the original model (Cleary et al., 2020).

In the original CES implementation, a random-walk Metropolis method is initialized at the final EKS ensemble mean,

$G:\mathbb{R}^p \to \mathbb{R}^d$1

with Gaussian proposal covariance taken from the final EKS ensemble covariance:

$G:\mathbb{R}^p \to \mathbb{R}^d$2

The acceptance probability is

$G:\mathbb{R}^p \to \mathbb{R}^d$3

Thus the calibration stage informs not only emulator training but also MCMC initialization and proposal scale (Cleary et al., 2020).

In the climate-model formulation, the posterior in decorrelated space is written as

$G:\mathbb{R}^p \to \mathbb{R}^d$4

with emulator-based likelihood

$G:\mathbb{R}^p \to \mathbb{R}^d$5

and, for Gaussian prior $G:\mathbb{R}^p \to \mathbb{R}^d$6,

$G:\mathbb{R}^p \to \mathbb{R}^d$7

The log-determinant term is retained because the covariance depends on $G:\mathbb{R}^p \to \mathbb{R}^d$8 (Dunbar et al., 2020). This is a more explicit treatment of emulator uncertainty than is often used in simpler surrogate-based workflows.

A common misunderstanding is that the sampling stage merely produces draws from the GP emulator. In CES, the target is an approximate Bayesian posterior over the parameters, and the emulator serves only as a substitute for the expensive forward map inside the likelihood (Cleary et al., 2020, Dunbar et al., 2020).

6. Applications, numerical studies, and extensions

The original CES paper validates the framework on a linear inverse problem, a Darcy flow inverse problem, and Lorenz ’63 and Lorenz ’96 systems (Cleary et al., 2020). In the linear inverse problem, the exact posterior is known, EKS calibrates correctly, GP emulation becomes accurate as the training set grows, and GP-based MCMC recovers the posterior well using $G:\mathbb{R}^p \to \mathbb{R}^d$9 design points (Cleary et al., 2020). In the Darcy flow inverse problem, CES with $\eta$0 closely matches gold-standard MCMC, while with $\eta$1 some systematic deviation appears although posterior mass still captures the truth (Cleary et al., 2020). In Lorenz ’63 and Lorenz ’96, time-averaged observations generate noisy forward evaluations; the GP stage smooths the misfit and improves sampling, and for Lorenz ’96 the method is particularly useful because gold-standard MCMC is too expensive or rejects too often (Cleary et al., 2020).

In climate science, CES was developed into a practical framework for calibrating an idealized GCM with internal variability noise (Dunbar et al., 2020). The uncertain parameters were the convective relaxation timescale $\eta$2 and the relative humidity parameter RH, with priors

$\eta$3

implemented through unconstrained transformed coordinates (Dunbar et al., 2020). Synthetic data were generated in a perfect-model setting from 30-day zonal means of free-tropospheric relative humidity, precipitation rate, and extreme precipitation frequency, all at 32 latitudes (Dunbar et al., 2020). The paper reports that the CES approach generates parameter distributions that approximate the Bayesian posteriors at a fraction of the usual computational cost, with about 600 short GCM runs and an estimated factor 1000 speedup relative to direct sampling on the GCM (Dunbar et al., 2020).

CES was subsequently embedded in an ensemble-based experimental-design algorithm for climate-model parameterizations (Dunbar et al., 2022). In that setting, CES is run repeatedly over candidate design points $\eta$4 to estimate the posterior

$\eta$5

compute the utility

$\eta$6

and select the maximizing design

$\eta$7

The paper interprets this as information gain through reduction in posterior uncertainty volume and reports that the largest information gain typically, but not always, results from regions near the intertropical convergence zone (Dunbar et al., 2022). This work extends CES from a calibration methodology to an engine for Bayesian optimal data targeting.

A broader extension is suggested by a 2025 comparison study of surrogate-based Bayesian calibration methods on the Lorenz ’96 multiscale system (Holthuijzen et al., 18 Aug 2025). There CES is described as a sequential Bayesian calibration framework using EKS, GP emulation, and MCMC, and is compared with History Matching (HM), Bayesian Optimal Experimental Design (BOED), and Goal-Oriented BOED (GBOED). The study finds that CES offers excellent performance but at high computational expense, while GBOED achieves comparable accuracy using fewer model evaluations (Holthuijzen et al., 18 Aug 2025). This suggests that later work increasingly treats CES as a baseline architecture against which more explicitly design-optimized strategies are compared.

7. Advantages, limitations, and related interpretations

The main advantages attributed to CES in the original and subsequent papers are consistent. It is derivative-free, requires only a small number of expensive forward solves relative to direct MCMC, provides a good design for emulator training because the calibration stage targets high-posterior regions, supports approximate Bayesian uncertainty quantification through posterior sampling, and is robust to noisy forward models because the GP emulator smooths simulator noise (Cleary et al., 2020). In climate applications, CES is also presented as robust to internal climate variability and capable of converting parameter uncertainty into predictive uncertainty through posterior propagation (Dunbar et al., 2020).

Its limitations are equally explicit. The calibration ensemble spread from EKI should not be interpreted as posterior uncertainty, because EKI is fundamentally an optimization method whose ensemble collapses toward consensus (Dunbar et al., 2020). The GP emulator can become the main scalability bottleneck in higher-dimensional parameter spaces (Dunbar et al., 2020). The method works best when the EKS or EKI stage succeeds in locating the region that carries most posterior mass; if calibration misses relevant regions, the local emulator may be inadequate there (Cleary et al., 2020, Dunbar et al., 2020). The 2025 comparison study further indicates that CES can be computationally expensive, and on the Lorenz ’96 multiscale benchmark EKS alone can outperform CES for that specific testbed (Holthuijzen et al., 18 Aug 2025).

Several adjacent uses of the acronym “CES” are not the same method. The 2017 paper on bilateral multifactor CES general equilibrium concerns constant elasticity of substitution calibration and uses “CES” in the sense of CES aggregators, not Calibrate–Emulate–Sample (Kim et al., 2017). By contrast, the 2023 paper on Bayesian neural networks adopts a “Calibration-Emulation-Sampling” strategy for posterior approximation in BNNs, but its calibration stage uses early-stopped SGHMC, its emulator is a neural network approximating the parameter-to-output or likelihood map, and its sampling stage uses pCN rather than the original EKS–GP–MCMC structure (Moslemi et al., 2023). This suggests that “CES” has broadened from a specific ensemble-Kalman-based surrogate pipeline into a more general design pattern for expensive Bayesian inference, though the canonical meaning remains the three-stage framework introduced for Bayesian inversion in 2020 (Cleary et al., 2020).

A plausible implication of the later literature is that CES occupies a middle position between purely optimization-based calibration and fully design-theoretic experimental design. It retains posterior sampling and uncertainty quantification, unlike pure calibration or history-matching workflows, but it avoids the cost of direct MCMC on the simulator by concentrating computational effort into a posterior-relevant surrogate. That balance explains its continuing use as both a practical inference method and a reference point for more specialized calibration and experimental-design schemes (Cleary et al., 2020, Dunbar et al., 2022, Holthuijzen et al., 18 Aug 2025).