Optimal experimental design: Formulations and computations (2407.16212v1)

Abstract: Questions of `how best to acquire data' are essential to modeling and prediction in the natural and social sciences, engineering applications, and beyond. Optimal experimental design (OED) formalizes these questions and creates computational methods to answer them. This article presents a systematic survey of modern OED, from its foundations in classical design theory to current research involving OED for complex models. We begin by reviewing criteria used to formulate an OED problem and thus to encode the goal of performing an experiment. We emphasize the flexibility of the Bayesian and decision-theoretic approach, which encompasses information-based criteria that are well-suited to nonlinear and non-Gaussian statistical models. We then discuss methods for estimating or bounding the values of these design criteria; this endeavor can be quite challenging due to strong nonlinearities, high parameter dimension, large per-sample costs, or settings where the model is implicit. A complementary set of computational issues involves optimization methods used to find a design; we discuss such methods in the discrete (combinatorial) setting of observation selection and in settings where an exact design can be continuously parameterized. Finally we present emerging methods for sequential OED that build non-myopic design policies, rather than explicit designs; these methods naturally adapt to the outcomes of past experiments in proposing new experiments, while seeking coordination among all experiments to be performed. Throughout, we highlight important open questions and challenges.

• The paper develops novel formulations for optimal experimental design that leverage the Fisher information matrix to maximize parameter estimation accuracy.
• It details computational techniques such as nested Monte Carlo methods, density approximations, and dimension reduction to efficiently address nonlinear design challenges.
• The study emphasizes Bayesian approaches and sequential design strategies to integrate prior knowledge and adaptively optimize data collection.

An Overview of "Optimal Experimental Design: Formulations and Computations"

The paper "Optimal Experimental Design: Formulations and Computations" constitutes a thorough examination of contemporary optimal experimental design (OED). Developed by Xun Huan, Jayanth Jagalur, and Youssef Marzouk, the paper explores the robust frameworks used to optimize the process of experiment and observation design, crucial for data acquisition in various scientific and engineering disciplines.

Foundational Concepts and Key Objectives

At its core, OED formalizes how to optimally gather data to guide decision-making and model development. This paper presents OED's evolution from classical experimental design principles into its modern implementations, noting its applicability to nonlinear and non-Gaussian statistical models.

The principal goal of OED is to establish an optimal set of design criteria, which are scalar functionals typically derived from the Fisher information matrix of the model. These criteria assist in choosing the best design according to various objectives such as D-optimality, which seeks to maximize the determinant of the Fisher information matrix, enhancing the precision of parameter estimation across possible designs.

Nonlinear Design Challenges and Bayesian Approaches

A significant portion of the paper deals with the complexities of nonlinear design where parameter dependencies introduce significant computational challenges. The authors emphasize Bayesian approaches due to their flexibility in expressing prior knowledge and managing nonlinearity through its integration with decision theory. Through this integration, OED criteria can be reformulated in terms of expected utilities which can involve information-theoretic measures such as expected Kullback–Leibler divergence.

Computational Strategies

The computational section lays out various strategies for estimating and optimizing these design criteria:

• Nested Monte Carlo Methods: This method provides a framework for calculating complex integrals necessary for estimating design criteria, albeit with issues related to bias at finite sample sizes.
• Density Approximations and Variational Bounds: The use of density approximations permits more computationally efficient strategies by turning density estimation into an optimization problem, facilitating the construction of variational bounds for information measures.
• Dimension Reduction Techniques: Dimension reduction, particularly significant in high-dimensional Bayesian inverse problems, leverages the intrinsic structure within the model to reduce the computational burden, applying techniques like truncated eigenvalue decomposition of the Fisher information matrix.

Optimization Techniques

This essay provides a comprehensive view of various optimization methods applicable in OED, including:

• Combinatorial Algorithms: Particularly suitable for linear models, these methods focus on iteratively improving the design within a discrete set of configurations.
• Continuous Optimization: Explores the use of continuous parameter spaces typical in nonlinear models, incorporating both derivative-based and derivative-free optimization techniques.
• Sequential Design Approaches: The sOED (Sequential Optimal Experimental Design) frameworks explore adaptive strategies that exploit the results of past experiments to inform the choice of future design points, matching policies with practice through frameworks like Markov decision processes.

Implications and Future Research Directions

The implications of the research presented extend across a spectrum of scientific applications, offering refined methods for improving the efficiency and effectiveness of data-gathering endeavors. The work draws attention to unresolved challenges, such as handling model misspecification, incorporating risk measures in design criteria, and advancing computational methods to handle the increasing scale and complexity of emerging applications.

The authors effectively map out a future research landscape, in which efforts should aim to integrate robust uncertainty quantification, develop enhanced sequential design strategies, and further refine the sensitivity of designs to model parameters. The paper provides an essential reference for researchers focused on the cutting edge of experimental design in science and engineering, helping bridge theoretical advancements with practical applications.