---
title: Parallel Expected Improvement (q-EI)
url: https://www.emergentmind.com/topics/parallel-expected-improvement-q-ei
type: topic
---

# Parallel Expected Improvement (q-EI)

Parallel Expected Improvement (q-EI) is a Bayesian optimization acquisition criterion for selecting a batch of $q$ input locations to evaluate expensive black-box objectives in parallel, under a Gaussian process (GP) surrogate model. It quantifies the expected gain, relative to the current incumbent, from simultaneously evaluating several candidate points, thereby enabling efficient use of parallel computational resources in global optimization. Unlike sequential Expected Improvement (EI), the q-EI criterion internalizes dependencies and interaction effects among all batch points. Mathematical, computational, and numerical properties of q-EI have been extensively developed, with significant focus on analytic expansions, stochastic gradient estimators, and numerically robust reformulations for high-dimensional and large-batch regimes [1503.05509] [1602.05149] [2010.10698] [2310.20708].

## 1. Formal Definition and Analytic Structure

Given a set of $n$ completed evaluations $\{(x_i, y_i)\}_{i=1}^n$ and the best observed value $f^* = \min_{i} y_i$ (for minimization), the batch q-EI for new points $X = (x^{(1)}, \dots, x^{(q)})$ is defined as:
\[
\mathrm{EI}_q(X) = \mathbb{E}\bigl[ \max\{0, f^* - \min_{i=1}^q f(x^{(i)}) \} \bigr]
\]
where the joint predictive distribution under the GP posterior is multivariate normal,
\[
f(X) \sim \mathcal{N}_q\left( \mu(X), \Sigma(X) \right)
\]
with $\mu(X) \in \mathbb{R}^q$ and $\Sigma(X) \in \mathbb{R}^{q \times q}$. This leads to an integral form:
\[
\mathrm{EI}_q(X) = \int_{\mathbb{R}^q} (f^* - \min_i z_i)_+ \;\phi_q(z; \mu(X), \Sigma(X)) dz
\]
where $\phi_q$ denotes the $q$-variate normal density.

For practical computation, this can be decomposed as a sum of $q$ lower-dimensional quadrature integrals:
\[
\mathrm{EI}_q(X) = \sum_{j=1}^q \int_{-\infty}^{f^*} (f^* - z_j) \:\varphi(z_j; \mu_j, \Sigma_{jj}) \: \Phi_{q-1}(a_j(z_j)) dz_j
\]
where $\varphi(\cdot)$ is the univariate normal density, $\Phi_{q-1}$ is the $(q-1)$-dimensional normal CDF evaluated at a shifted and scaled argument $a_j(z_j)$, and each term corresponds to the event that the $j$th coordinate yields the minimum [2010.10698].

An alternative (maximization) form, as applied in [1503.05509] and [1602.05149], conditions on the event that each batch point attains the maximum, resulting in analytic expansions via Tallis's formula and its extensions, where all terms are explicit functions of GP posterior moments and multivariate normal probabilities.

## 2. Computational Complexity and Scaling Properties

The direct computation of q-EI involves:
- Multivariate normal CDF evaluations ($\Phi_{q-1}$ and $\Phi_{q}$) scaling as $O(q^3)$ per call using, e.g., Genz's adaptive integration.
- Analytic gradients with respect to all $qd$ batch-location variables, incurring $O(q^4)$ CDF calls for each batch evaluation if all terms are kept exact [1503.05509].
- Curse of dimensionality: Both the number of batch variables ($q$) and input dimension ($d$) contribute multiplicatively to the size of the optimization landscape; practical exact evaluation is infeasible for $q \gtrsim 10$ or $d \gg 10$ [2010.10698].

Finite-difference gradients require $O(dq^3)$ CDF calls, making analytic or stochastic gradients necessary for efficiency [1503.05509] [1602.05149].

## 3. Gradient Estimation and Optimization Techniques

Closed-form analytic gradients have been derived for q-EI, leveraging Gaussian identities, conditional laws, and matrix algebra. For the analytic gradient, all derivatives of predictive means, covariances, univariate and multivariate normal functions are tractable (see explicit expressions in [1503.05509]).

As $q$ increases, Monte Carlo approaches become dominant:
- Infinitesimal Perturbation Analysis (IPA) constructs an unbiased stochastic gradient estimator by differentiating through the batch maximum's realization, yielding
  \[
  \nabla_X \mathrm{EI}_q(X) \approx \frac{1}{M} \sum_{m=1}^M \nabla_X h(X, Z^{(m)})
  \]
  where $h$ is the sample-wise batch improvement and $Z^{(m)}$ are i.i.d. draws from $\mathcal{N}(0, I_q)$ [1602.05149].
- Projected stochastic gradient ascent (SGA) with Polyak–Ruppert averaging is used for local maximization over candidate batches, converging under standard stochastic approximation conditions [1602.05149].

Importance-resampling (SIR) approaches reduce the burden by generating a candidate pool (via Sobol or Latin Hypercube sequences), weighting by classical EI, and drawing batch points in a way that heuristically targets high q-EI values, reducing per-stage CPU time to $O(mn)$ for $m$ candidates [2010.10698].

## 4. Numerical Pathologies and Robust Reformulations

Canonical q-EI suffers from vanishing acquisition values and gradients in moderate-to-large data regimes:
- As BO proceeds, posterior means rarely exceed the incumbent, causing improvement terms to be numerically zero for most batch configurations.
- Analytic q-EI gradients decay rapidly in floating-point arithmetic for negative scaled improvement, often dropping below machine precision, especially in high dimensions or large batches.
- Monte Carlo max operations have zero gradient almost everywhere due to the non-differentiable nature of the maximum [2310.20708].

The LogEI-q reformulation addresses these pathologies:
- Applies a log-transformation to EI (and its batch generalization) for numerical stability, replacing the hard positive-part and max operations with softplus and smooth maximums (e.g., $τ \log\sum_j e^{a_{ij}/τ}$).
- Ensures strong, non-vanishing gradients even in regions where canonical q-EI stalls.
- Has the same or approximately identical optima as canonical q-EI by monotonicity of log, and smoothing errors are controllable by temperature parameters ($τ_0$, $τ_{max}$) [2310.20708].

## 5. Algorithmic Implementations and Empirical Performance

Closed-form q-EI maximization involves multistart BFGS/quasi-Newton in $qd$ dimensions, with analytic gradients enabling tractable optimization for $q \leq 10$ [1503.05509]. For larger $q$ or higher $d$, stochastic gradient-based maximization or SIR-based batch assembly becomes essential:
- MC-based stochastic optimizers (as in MOE) draw many MC samples to estimate q-EI and its gradient, using restarts and step-size schedules [1602.05149].
- LogEI-q can be optimized directly using autodiff-enabled libraries with MC sampling, gradient smoothing, and robust handling of constraints and noise [2310.20708].
- Accelerated EGO via SIR randomly samples well-spread candidate batches, weighs by one-point EI, and assembles diverse, high-EI batches with low clustering and competitive empirical regret, reducing computational cost and iteration count drastically for $q$ up to 15 and $d$ up to 12 [2010.10698].

Empirically:
- Analytic q-EI outperforms heuristic BUCB in one-step and worst-case regret but is more computationally expensive; SIR and MC-based methods recover similar or better performance at a fraction of the wall-clock cost [1503.05509] [2010.10698].
- LogEI-q achieves 2–5x lower regret than classic MC q-EI for $q=4,8,16$ in high-dimensional settings and avoids zero-gradient stagnation [2310.20708].

## 6. Practical Considerations and Recommendations

- For small $q$ ($\leq4$), both analytic and MC approaches are practical, but IPA-based MC gradient ascent is easily parallelized and extensible [1602.05149].
- For moderate to large $q$ ($q\geq5$), MC-based or SIR-based batch construction, with robustified acquisition (LogEI-q), should be preferred.
- Numerical stability is critical: Use log-transformed EI and smooth max/plus operators, tune temperature parameters to maintain gradient flow, and employ multiple restarts with space-filling initializations [2310.20708].
- Implementation in open-source frameworks such as MOE supports scalable, GPU-accelerated MC q-EI [1602.05149].
- For constrained, noisy, or multi-objective settings, the same log-smoothing principles extend to robust acquisition construction, avoiding degeneracy in derivative-free optimization [2310.20708].

## 7. Comparative Summary of Techniques

| Method                  | Batch Size/Dim Scale      | Complexity/Cost        | Empirical Robustness           |
|-------------------------|--------------------------|-----------------------|-------------------------------|
| Analytic q-EI           | $q \leq 10$, $d \leq 20$ | $O(q^3)$/$O(q^4)$ grad| High regret perf, costly      |
| Stochastic Grad. MC q-EI| $q$ large, any $d$       | $O(q^2 M)$, scalable  | Similar regret, much faster   |
| SIR (Accelerated EGO)   | $q$ large, moderate $d$  | $O(mn)$ per stage     | Heuristic, near-optimal regret|
| LogEI-q (MC)            | Any $q$, high $d$         | $O(Nq)$, stable       | State-of-art, robust gradients|

Analytic approaches are preferred for small $q$, while MC- and SIR-based strategies scale efficiently to large batch sizes and higher dimensions. LogEI-q is recommended for robust, numerically stable acquisition maximization, especially in high-dimensional or constrained batch selection settings [1503.05509] [1602.05149] [2010.10698] [2310.20708].

Source: https://www.emergentmind.com/topics/parallel-expected-improvement-q-ei