---
title: Approximate Coordinate Exchange (ACE) Algorithm
url: https://www.emergentmind.com/topics/approximate-coordinate-exchange-ace-algorithm
type: topic
---

# Approximate Coordinate Exchange (ACE) Algorithm

The Approximate Coordinate Exchange (ACE) algorithm is a general-purpose stochastic optimization scheme for constructing Bayesian optimal experimental designs when the expected utility function is analytically intractable and the design space is high-dimensional. ACE applies coordinate-exchange methodology in conjunction with Gaussian process (GP) regression to sequentially optimize each element (“coordinate”) of the design, iteratively building surrogate models and leveraging conditional expected utility evaluations via Monte Carlo approximation. The methodology is suitable for a wide variety of nonlinear and generalized linear models, utility functions, and experimental settings—substantially extending the practical scope of fully Bayesian design [1501.00264] [1705.08096].

## 1. Bayesian Optimal Design and Intractability

The general Bayesian design problem seeks a design $d$ (e.g., a collection of settings for experimental runs) that maximizes the expected utility,
\[
U(d) = \int_{\mathbf y} \int_\theta u(d, \theta, \mathbf y) \, \pi(\theta, \mathbf y|d) \, d\theta \, d\mathbf y,
\]
where $u$ is a user-specified utility function, $\theta$ are model parameters, $\mathbf y$ are potential future data, and $\pi(\theta, \mathbf y|d)$ is the joint prior predictive for $(\theta, \mathbf y)$ under design $d$ [1705.08096]. In practical applications, $U(d)$ is typically analytically intractable due to the complexity or dimensionality of the integrals, stochastic utility structure, and possibly the need for nested Monte Carlo for information-type utilities. The design space $\mathcal{D}$ is often high-dimensional; for $n$ runs with $k$ variables, $d$ comprises $n \cdot k$ continuous elements.

## 2. Structure of the ACE Algorithm

ACE reduces the curse of dimensionality by decomposing $U(d)$ maximization into a cyclic sequence of one-dimensional optimization subproblems, each corresponding to a single coordinate $d_c$ of the design vector. The algorithm’s key components are:

- **Monte Carlo evaluation**: At each coordinate update, $U(d)$ is approximated at a small grid of values along the chosen dimension by averaging $u(d_c,\theta_b, y_b)$ over $B$ prior-predictive samples $(\theta_b, y_b)$.
- **Gaussian process surrogate modeling**: The noisy conditional evaluations are modeled as a realization of a one-dimensional GP with mean function $m(\cdot)$ and covariance $k(\cdot,\cdot)$. The surrogate $\hat U_c$ is then optimized on a dense grid to propose a new coordinate value.
- **Stochastic acceptance**: The move is accepted by comparing the current and candidate utilities through a two-sample t-test, controlling for Monte Carlo error.
- **Iterative cycling and restarts**: The process is repeated for all coordinates over $N_1$ cycles, with optional random restarts to mitigate local mode trapping. A point-exchange phase may follow to consolidate duplicate or close runs [1501.00264] [1705.08096].

## 3. Mathematical Details and Pseudocode

The algorithm can be formally structured as:

**Phase I: Coordinate Exchange**
- For $c = 1,\ldots,p$ (with $p = n \cdot k$), hold all coordinates except $c$ fixed.
- For a set $\{\xi^q\} \subset \mathcal{D}_c$ ($q=1,\ldots,Q$):

   1. Compute $\tilde U(d^{(q)}) = \frac{1}{B} \sum_{b=1}^B u(d^{(q)}, \theta_b, y_b)$, where $d^{(q)}$ replaces coordinate $c$ with $\xi^q$.

   2. Fit a GP to $(\xi^q, \tilde U(d^{(q)}))$ data.

   3. Maximize the surrogate $\hat U_c(\xi)$ on a fine grid to yield $\xi^*$.

   4. Propose $d^{\text{cand}}$ with $c$ replaced by $\xi^*$.

   5. Compute acceptance probability $p^*$ via a two-sample t-test between MC samples for $d^{\text{cand}}$ and $d^{\text{curr}}$. Accept with probability $p^*$.

The process repeats for a user-specified number of cycles and independent starts. A consolidation phase (Phase II) further improves final design structure via point exchanges [1501.00264] [1705.08096].

| Phase           | Description                                      | Output                                   |
|-----------------|--------------------------------------------------|------------------------------------------|
| Coordinate      | Sequential 1-D surrogate optimization            | Updated design vector                    |
| Point-exchange  | Merges/consolidates near-duplicate runs          | Replicated/improved design               |

## 4. Gaussian Process Surrogates in ACE

At each coordinate update, the conditional expected utilities $\tilde U$ are standardized:

\[
z_j = \frac{\tilde U(\delta^j) - \hat \mu}{\hat \sigma},
\quad \hat \mu = \frac{1}{m}\sum_{j=1}^m \tilde U(\delta^j), \quad \hat \sigma^2 = \frac{1}{m-1} \sum_{j=1}^m (\tilde U(\delta^j)-\hat\mu)^2
\]

A GP prior $z(\cdot)\sim \mathcal{GP}(0, R)$, typically with squared-exponential covariance,
\[
R(\delta, \delta') = \exp\{-\rho(\delta - \delta')^2\} + \eta \mathbb{I}(\delta = \delta'),
\]
is fit to these points. The emulator’s posterior predictive mean is
\[
\hat U_c(\xi) = \hat\mu + \hat\sigma\,\mathbf a(\xi)^T A^{-1} \mathbf z,
\]
with $A$ and $\mathbf a$ derived from the covariance function and chosen grid [1501.00264]. Surrogate prediction is computationally efficient: inversion of an $m \times m$ matrix per update, with $m$ typically 10–30.

## 5. Computational Properties and Practical Implementation

ACE’s computational costs are dominated by Monte Carlo utility evaluations, required for surrogate fitting and acceptance steps. At each coordinate update:

- $m$ conditional utilities approximated, each requiring $B$ likelihood/utility calls.
- GP fit per coordinate costs $O(m^3)$ due to matrix inversion.
- For $q$ design parameters, $N_1$ cycles, cost scales as $O(N_1 q m B)$ [1501.00264].

Practical guidelines:

- $m \approx 20$, $B \approx 10^3$–$2 \times 10^4$.
- $N_1 \approx 20$, $N_2 \approx 100$ (for optional consolidation).
- $M \approx 20$ random restarts advised to avoid local optima.
- Convergence is monitored via trace plots of $\tilde U$, with stabilization indicating sufficient cycling.

The cyclic coordinate-exchange, constrained to one-dimensional surrogates, enables ACE to scale to design spaces previously inaccessible to direct global search or brute-force Monte Carlo [1501.00264] [1705.08096].

## 6. Illustrative Applications

The methodology has been demonstrated on a range of experiments, including:

- **Pharmacokinetic compartment models:** ACE produced unrestricted designs with 3–5% higher expected utility than Beta-DRS and pseudo-Bayesian $D$-optimal competitors.
- **Mixed models (hierarchical logistic regression):** Fully-Bayesian SIG and NSEL designs found by ACE outperform pseudo-Bayesian $D/A$-optimal alternatives by 20–25% for small sample sizes $n$; differences diminish for larger $n$ as expected.
- **Model uncertainty (binomial regression for beetle mortality):** Model-averaged NSEL designs concentrated runs near the posterior LD50, quickly reducing posterior variance [1501.00264].

The ACE method is implemented in the R package acebayes, which provides end-user tools and tutorials for Bayesian design in generalized linear and nonlinear settings [1705.08096].

## 7. Algorithmic Significance and Scope

ACE, by combining stochastic coordinate-wise optimization with adaptive one-dimensional GP emulation, enables computation of fully Bayesian designs for experiments with large numbers of variables, runs, and randomization restrictions. It avoids reliance on asymptotic approximations to posterior distributions or expected utilities, and is applicable to arbitrary user-specified utility functions and statistical models. Its flexibility, scalability, and practical effectiveness position it as the most general solution to high-dimensional Bayesian design-of-experiments problems to date [1501.00264] [1705.08096].

Source: https://www.emergentmind.com/topics/approximate-coordinate-exchange-ace-algorithm