An adaptive discretization algorithm for locally optimal experimental design with constraints

Abstract: We develop a novel iterative algorithm for locally optimal experimental design under constraints, like budget or performance constraints. It is an adaptive discretization algorithm. In every iteration, a discretized version of the constrained-design problem is solved and then the discretization is adaptively refined by adding an approximate violator of a suitable sufficient $\eps$-optimality condition for the current design. We prove that with $\eps = 0$, our algorithm converges to an optimal design and that with $\eps > 0$, our algorithm finitely terminates at an $\eps$-optimal design. Compared to the existing algorithms on constrained experimental design, our algorithm comes with considerably less computational effort because the nonlinear subproblems in our algorithm have a smaller dimension and have to be solved only approximately and only in selected iterations (typically the last few). Additionally, our algorithm covers a considerably larger class of constraints. We demonstrate the good convergence properties of the algorithm on experimental design problems from chemical engineering that feature time and yield constraints.

• The paper presents an adaptive discretization algorithm that iteratively refines the design space to solve constrained locally optimal experimental design problems.
• Methodology involves solving finite-dimensional approximations, evaluating sensitivity functions, and adding critical design points to ensure convergence.
• Numerical experiments validate rapid convergence, robustness under various constraints, and significant computational efficiency improvements.

Adaptive Discretization Algorithm for Constrained Locally Optimal Experimental Design

Problem Formulation and Theoretical Framework

The paper presents a rigorous treatment of constrained locally optimal experimental design (C-OED). The problem is formalized as the minimization of a convex design criterion $\Psi_0$ over the space of probability measures $\Xi(X)$ on a compact design space $X$, subject to convex inequality and affine equality constraints $\Psi_i$:

$$\min_{\xi\in\Xi(X)} \Psi_0(\xi) \quad \text{s.t.} \quad \Psi_i(\xi)\le 0,\, i\in I_{\text{ineq}};\quad \Psi_i(\xi)=0,\, i\in I_{\text{eq}}$$

Such formulations accommodate practical restrictions like budget and time while seeking maximal informativeness for model parameter estimation. Convexity and directional differentiability assumptions ensure theoretical tractability and establish the existence of optimal solutions, often with finitely many support points due to Carathéodory-type results.

The paper characterizes optimality by saddle points of the Lagrangian $L$ that incorporates constraint multipliers:

$$L(\xi, \lambda) = \Psi_0(\xi) + \sum_{i\in I} \lambda_i \Psi_i(\xi)$$

Directional derivatives of $L$ are represented via sensitivity functions $\psi^L(\xi, \lambda, x)$, which serve as the integral kernels for the design measure variations. Sufficient and necessary optimality conditions are formulated in terms of lower bounds on these sensitivity functions, with complementarity for active constraints. Under standard regularity conditions (strict feasibility and non-one-sidedness of constraint images), every optimal design generates a Lagrange multiplier yielding a saddle point.

Adaptive Discretization Algorithm

To address the infinite-dimensionality of OED problems with constraints, the authors introduce an adaptive discretization algorithm. The main principle is iterative refinement of the discretization of $X$ guided by violations of optimality conditions, as follows:

1. Design Problem Discretization: At each iteration $\Xi(X)$0, solve the constrained OED problem on a finite discretization $\Xi(X)$1, yielding a saddle point $\Xi(X)$2.
2. Sensitivity Evaluation: Compute the sensitivity function $\Xi(X)$3 over $\Xi(X)$4.
3. Violator Search: If there exists $\Xi(X)$5 such that $\Xi(X)$6 (for chosen tolerance $\Xi(X)$7), $\Xi(X)$8 is added to the discretization: $\Xi(X)$9.
4. Approximate Solutions: When no violator is found by local search, a global approximate minimizer (possibly via enumeration or optimization) is sought; if its value satisfies the optimality bound, terminate; otherwise, add it to $X$0.
5. Termination and Guarantees: Under mild regularity conditions, the algorithm converges to an optimal (or $X$1-optimal) design in finitely many steps.

This approach leverages computational tractability by focusing nonlinear subproblems exclusively on sensitivity minimization, yielding far fewer and lower-dimensional nonlinear solves compared to prior vertex-direction or descent methods.

Figure 1: Sensitivity function $X$2 for the Lagrange function of the unconstrained OED problem, showing near vanishing at terminal design.

Figure 2: Sensitivity function $X$3 for constrained OED, illustrating adaptation in discretization and active constraint effects across iterations.

Comparison with Prior Algorithms

A detailed comparative analysis establishes substantial advantages of the presented algorithm over extant methods due to:

• General Constraint Handling: Supports convex, affine, and discontinuous constraints, broadening applicability, unlike prior work limited to continuous, affine cases.
• Computational Efficiency: Nonlinear search is typically limited to small subsets and solved approximately; dimension scales with $X$4, not the number of constraints times $X$5.
• Convergence: Each iteration entails full re-optimization, leading to rapid progress and often linear convergence; avoids oscillatory or zig-zag behavior typical in classical algorithms.
• Finite Termination: Guaranteed for positive tolerance $X$6; prior methods either lack termination criteria or require exact nonlinear solves every iteration.

A thorough examination of the literature (Cook & Fedorov, Gaivoronski, Molchanov & Zuyev) confirms that the adaptive discretization method significantly reduces computational burden and enhances generality, while matching or surpassing convergence properties.

Numerical Experiments and Applications

The efficacy is demonstrated on several realistic design problems from chemical engineering, including exponential growth models and reaction kinetics:

• Exponential Growth Model: The unconstrained D-optimal design (Example 1) terminates in two iterations, reproducing known analytical results with two support points; constrained versions adapt discretization dynamically, handling affine and nonlinear constraints, converging in fewer iterations, and with support well below theoretical bounds.
• Reaction Kinetics Model: The algorithm accommodates constraints on product yield, time, and purity, efficiently computes optimal designs on large candidate sets (up to ~2 million points), and maintains feasibility throughout.

Extensive reporting of criterion values, constraint satisfaction, approximation error bounds ($X$7), iteration counts, and support cardinalities confirm both computational efficiency and correctness.

Practical and Theoretical Implications

This work substantively advances the methodology for practical OED under constraints. The adaptive discretization scheme enables scalable solution of large realistic design spaces, while guaranteeing $X$8-optimality and feasibility. The theoretical analysis provides robust convergence and finite termination guarantees under minimal regularity, with transparent verification.

Practically, the algorithm is suited for industrial settings with complex experimental restrictions. Theoretically, it generalizes previous convex programming results by relaxing continuity and affinity requirements. Moreover, the approach is modular, allowing integration with modern optimization solvers and local/global search heuristics, opening avenues for mixed-integer and non-convex extension.

Future developments in AI-assisted experiment planning may exploit such algorithms for automated design with real-time constraint adaptation, or for robust design in the presence of uncertainty or stochastic effects.

Conclusion

The adaptive discretization algorithm presented unifies theoretical optimality, computational efficiency, and practical feasibility for constrained locally optimal experimental design. Its deployment yields fast, certified, and versatile solutions across diverse problem classes, representing a substantial contribution to both experimental design theory and applied optimization in engineering domains.