Optimal Experimental Design (OED)

• Optimal Experimental Design is a systematic process for choosing experimental conditions to maximize data informativeness and reduce parameter uncertainty.
• Modern OED integrates classical criteria with Bayesian and information-theoretic approaches, employing computational strategies like Monte Carlo and surrogate models.
• OED is pivotal in fields such as engineering and physical sciences, addressing challenges in high-dimensional, nonlinear, and adaptive experiment planning.

Optimal Experimental Design (OED) formalizes the process of selecting experimental conditions—such as sensor placements, measurement times, or experimental controls—in order to maximize the informativeness of the resulting data about unknown parameters or model predictions. Rooted in both classical and Bayesian statistics, OED provides rigorous criteria—often expressed in terms of uncertainty reduction or information gain—for quantifying experiment value, and computational methodologies for finding optimal designs. Contemporary OED encompasses inverse problems, high-dimensional and non-Gaussian models, PDE-constrained models, and sequential (adaptive) experiment planning. The following sections synthesize recent advances, core methodologies, and unresolved challenges in OED from theoretical and applied perspectives.

1. Foundations and Evolution of OED

The origins of OED are in the theory of experimental design for linear regression and inference tasks, where the objective is often to select a set of discrete experiments or sensor placements to maximize the precision of parameter estimates. The early canon is defined by the so-called alphabetic optimality criteria, including:

• A-optimality: Minimize the trace of the (generalized) posterior covariance—i.e., the average variance of the estimated parameters.
• D-optimality: Minimize the log-determinant of the posterior covariance or, equivalently, maximize the determinant of the Fisher information matrix (FIM).
• E-optimality: Minimize the largest eigenvalue of the posterior covariance.

These and related criteria remain central in modern OED for linear problems, but the field has expanded rapidly. Bayesian OED extends these ideas, assigning a prior to unknown parameters and accounting for model and data uncertainty. In nonlinear or non-Gaussian models, information-theoretic criteria (primarily the expected information gain based on Kullback–Leibler (KL) divergence) are now widely used as design benchmarks (Huan et al., 23 Jul 2024). The general OED problem in the Bayesian context is to select design variables $\xi$ that maximize a utility functional $U(\xi)$, typically taking the form: $$U(\xi) = \mathbb{E}_{Y,\theta}\left[\log\frac{p(\theta\mid y,\xi)}{p(\theta)}\right]$$ which represents the expected reduction in uncertainty about $\theta$ provided by the experiment $\xi$. The flexibility of the Bayesian and decision-theoretic approaches enables OED to handle complex physical, biological, and engineering systems.

2. Design Criteria and Goal-Oriented Extensions

Modern OED formulations employ a rich taxonomy of criteria tuned to linear/nonlinear, Gaussian/non-Gaussian, and goal-oriented applications:

• Classical Linear Criteria:
  • For linear-Gaussian models, the OED problem is typically encoded in terms of the Fisher information matrix $F(\xi) = G(\xi)^\top \Gamma_{Y|\xi}^{-1} G(\xi)$, with design functionals such as:
  • $\text{A-opt:}~ \mathrm{Tr}(F^{-1}(\xi)),$
  • $\text{D-opt:}~ \log\det(F^{-1}(\xi)).$
• Information-Based Bayesian Criteria:
  • Expected Information Gain (EIG): The mutual information between observable $Y$ and unknown parameter $\theta$, $U(\xi)=\mathcal{I}(Y;\theta|\xi)$, remains the foundational utility in Bayesian OED.
• Goal-Oriented Design:
  • For predictions of particular quantities (QoI), such as functionals of the parameter field or physical observables, OED can be formulated to maximize the information gain about these predictions, $U(\xi) = \mathcal{I}(Y;Z|\xi)$, where $Z = \Psi(\theta)$ (Zhong et al., 26 Mar 2024, Wu et al., 2021).
• Alternative Metrics:
  • Wasserstein-based design criteria have been introduced to quantify the expected "transport cost" needed to move the prior to the posterior, yielding geometry-based utilities that are robust in high- or infinite-dimensional settings (Helin et al., 14 Apr 2025).
  • Measure-theoretic geometric functionals (expected scaling effect and expected skewness effect) designed for data-consistent inversion leverage singular value decompositions of Jacobians to quantify how much uncertainty is "pulled back" into the parameter space (Butler et al., 13 Jun 2025).

The choice of criterion is dictated by the statistical structure and inferential objective of the problem. Bayesian and information-based criteria dominate in nonlinear and high-dimensional problems; goal-oriented criteria are essential for predictive design; geometry-based functionals are prominent in measure-theoretic and data-consistent approaches.

3. Computational Strategies for Utility Evaluation

The evaluation of OED utilities is computationally demanding, particularly in nonlinear, high-dimensional, or implicit models. Several algorithmic classes have emerged:

• Nested Monte Carlo (NMC) Estimation: Used for expected information gain and related utilities, NMC involves sampling from the prior, simulating possible data, and integrating over both parameter and data spaces. The estimator has cubic mean-square error convergence ($O(W^{-2/3})$ in total computational effort), but is often computationally prohibitive in large-scale PDE-based problems (Huan et al., 23 Jul 2024).
• Variance Reduction and Multi-fidelity Methods: To address computational cost, multi-fidelity estimators under the approximate control variate (ACV) framework combine high- and low-fidelity models, yielding unbiased estimators that can reduce variance by one or two orders of magnitude (Coons et al., 18 Jan 2025).
• Dimension Reduction and Surrogates: For models with expensive forward operators (e.g., neural ODEs, PDEs), dimension reduction via principal components, truncated SVD, sensitivity lumping, or goal-oriented low-rank projections is employed to represent the essential space for uncertainty reduction (Plate et al., 13 Aug 2024, Wu et al., 2021). Surrogates (e.g., Gaussian process regression) are used in settings like MOCU to amortize or proxy expensive cost computations (DeGennaro et al., 2020).
• Density Estimation and Variational Inference: For non-Gaussian posteriors and predictive QoIs, kernel density estimation and normalizing flows are applied to estimate densities and variational lower bounds to mutual information (Orozco et al., 28 Feb 2024, Huan et al., 23 Jul 2024).
• Randomized Trace Estimation: For high-dimensional operators (posterior covariances, Hessians), randomized estimators based on Monte Carlo probing avoid explicit operator construction and scale independently of state space dimensions (Alexanderian et al., 2014).

These computational strategies are often combined with model-specific optimizations (adjoint equations, parallel evaluation, surrogate-assisted design), with design selection relying heavily on the tractable approximation of gradients and utility evaluations under resource constraints.

4. Design Optimization Algorithms

Once a utility or risk criterion is computed, optimization over the design space remains a core challenge:

• Discrete and Binary Selection: For combinatorial sensor placement and experiment selection, relaxation-based (continuous weights with sparsity penalties), stochastic (Bernoulli policy gradient, REINFORCE), and greedy algorithms dominate (Alexanderian et al., 2014, Attia et al., 2021, Alexanderian et al., 2022). Submodularity properties (or near-submodularity) enable approximation guarantees for greedy schemes (Huan et al., 23 Jul 2024).
• Continuous Design Spaces: For problems where experimental settings form a continuum (e.g., spatial/temporal design), stochastic gradient, Wasserstein gradient flow, and Bayesian optimization are applied (Jin et al., 15 Jan 2024, Zhong et al., 26 Mar 2024). Wasserstein gradient flows for OED, for example, lift the design optimization to probability measure spaces, enabling continuous design selection via simulated particle transport.
• Bilevel and Sequential Formulations: OED is inherently a bilevel problem in inverse problems and constrained state estimation, with the experimental design impacting the inner inverse problem (and vice versa). This necessitates advanced bilevel programming and, for empirical Bayes risk settings, parallelization over training examples (Ruthotto et al., 2017, Alexanderian et al., 2022).
• Adaptive and Sequential Design: Sequential OED (sOED) formalizes the planning of experiment batches or adaptive, feedback-driven experiment sequences using Markov decision process (MDP) or reinforcement learning frameworks, including actor-critic policy gradients and approximate dynamic programming (Huan et al., 23 Jul 2024). Sequential approaches are non-myopic and adapt to accumulating experimental outcomes.

The practical choice of optimization technique is dictated by combinatorial scale, design space structure, availability of gradients, and computational cost of utility evaluations.

5. Recent Developments and Extensions

OED has seen rapid progress in high-dimensional and complex systems, with several notable trends:

• Integration with Hybrid and Neural Models: For universal differential equations (UDEs) that combine mechanistic and neural components, OED methods must address overparameterization and regularization of the FIM through SVD or sensitivity lumping, yielding regular and computationally tractable design optimization (Plate et al., 13 Aug 2024).
• Handling Correlated Observational Errors: In large-scale PDE-constrained inverse problems, new formulations based on Hadamard product (pointwise weighting) of observation covariances account for spatial and temporal measurement correlations, promoting distributed non-redundant sensor placements (Attia et al., 2020).
• Multi-fidelity and Surrogate-driven Design: Integration of multi-fidelity models via control variate estimators and surrogate models yields order-of-magnitude efficiency improvements while maintaining unbiasedness of key estimators (Coons et al., 18 Jan 2025).
• Goal-Oriented and Predictive Design: Emphasis on experiment designs that minimize prediction uncertainty in user-defined quantities of interest, distinct from purely parameter-focused OED, has led to new computational frameworks based on MCMC, density estimation, and nested/goal-oriented utilities (Wu et al., 2021, Zhong et al., 26 Mar 2024).
• Non-traditional Design Criteria: The emergence of Wasserstein distance-based utilities provides alternatives to KL-based information gain; geometry-based design criteria are exploited in measure-theoretic, data-consistent inversion (Helin et al., 14 Apr 2025, Butler et al., 13 Jun 2025).

These developments reflect increasing flexibility and scalability relative to classical OED, with special attention to uncertainty quantification, computational resource constraints, and interpretable design choices.

6. Open Challenges and Future Directions

Several fundamental challenges remain in OED:

• Model Misspecification: All OED decisions rely on correct specification of forward and uncertainty models; misspecification can lead to severely suboptimal or misleading designs. Robust OED methods, including ambiguity set formulations and distributionally robust optimization, represent an active area (Huan et al., 23 Jul 2024).
• Risk-Sensitive Design: While most criteria maximize expected utility, many applications require explicit control of tail risks or worst-case uncertainties (e.g., CVaR-based design), for which analytical and computational tools are lacking (Huan et al., 23 Jul 2024).
• State Representation in Sequential Design: Efficient (minimal yet sufficient) representations of the posterior or belief state in sOED are needed to overcome dimensionality bottlenecks in long experiment sequences.
• Theory of Neural and Surrogate-Based OED: With increased reliance on surrogates, neural density approximators, and reinforcement learning methods for sOED, theoretical analyses of approximation error, convergence, and design reliability are necessary to ensure robust practice (Orozco et al., 28 Feb 2024).
• Scalability in Infinite-Dimensional/Nonlinear Regimes: While scalable randomized and low-rank strategies exist for some PDE-based problems (Alexanderian et al., 2014, Alexanderian et al., 2022), unifying scalable OED across all nonlinear and measure-theoretic formulations remains an important goal.

Continued development of computationally efficient, interpretable, and risk-aware OED frameworks—especially for high-dimensional, nonlinear, and data-integrated systems—remains essential for advancing scientific discovery, engineering design, and decision-making.