B3O: Scalable Boltzmann Batch Bayesian Optimization

Abstract: Modern engineering workflows increasingly rely on massive parallel simulation, driving the need for scalable, large-batch Bayesian Optimization (BO). Existing batch BO methods, however, incur large computational cost or rely on approximations that erode batch diversity. We propose B3O (Boltzmann Batch Bayesian Optimization), a framework that reframes batch generation as a pure sampling problem: drawing samples directly from the Boltzmann distribution defined by the acquisition function avoids the bottlenecks of existing large-batch methods. Theoretically, we prove that queries sampled from this distribution incur only negligible additional regret. Empirically, B3O outperforms existing batch BO methods on standard synthetic benchmarks and adapts robustly across complex applied tasks, including multi-objective electrode design and mixed-variable race car configuration.

• The paper introduces B3O, a scalable framework that reframes batch Bayesian optimization as i.i.d. Boltzmann sampling over acquisition functions.
• It details the methodology with a theoretical regret analysis, demonstrating linear computational scaling and enhanced candidate diversity.
• Empirical evaluations on synthetic benchmarks and real-world tasks validate B3O’s effectiveness in multi-objective and mixed-variable optimization.

Scalable Boltzmann Batch Bayesian Optimization (B3O)

Introduction

The paper "B3O: Scalable Boltzmann Batch Bayesian Optimization" (2606.30228) presents a unified and scalable framework for batch Bayesian optimization (BO), termed B3O, which reframes batch candidate generation as i.i.d. sampling from a Boltzmann distribution over the acquisition landscape. Unlike existing batch BO methods, B3O simultaneously achieves computational tractability for large batch sizes, diversity preservation without explicit repulsion, and broad applicability across continuous, mixed-variable, and multi-objective scenarios. The authors provide a theoretical regret analysis and demonstrate strong empirical results in large-batch, multi-objective, and heterogeneous optimization environments.

Context and Motivation

Batch BO is fundamental for optimization tasks involving expensive black-box objectives with high-throughput or parallel evaluation, such as simulation-driven engineering design. Traditional strategies do not scale well: multi-point acquisition functions like $q$-EI and $q$-UCB incur $\mathcal{O}(B^3)$ cost and become intractable in high dimensions or large batches, while greedy and repulsion-based heuristics (e.g., local penalization) degrade in exploration or diversity. Scalable Thompson Sampling (TS) approaches circumvent joint optimization but suffer from hard diversity limitations due to variance starvation in spectral approximate GPs, especially in high-parallelism regimes and in non-stationary models.

B3O builds on Boltzmann sampling by constructing the batch via direct sampling from a density proportional to the exponentiated point-wise acquisition function, resulting in scalable and highly parallelizable batch construction with inherent diversity.

Methodology

Boltzmann Sampling for Batch Generation

B3O recasts batch candidate selection as direct i.i.d. sampling from the Boltzmann (Gibbs) distribution over the chosen acquisition function $\alpha_t(\mathbf{x})$,

$$p_t(\mathbf{x}) = \frac{\exp(\lambda_t \alpha_t(\mathbf{x}))}{Z_t(\lambda_t)}$$

where $\lambda_t$ is the inverse temperature controlling the trade-off between exploration and exploitation. A low $\lambda_t$ yields broad, exploration-heavy batches, while high $\lambda_t$ leads to greedy exploitation.

This approach differs fundamentally from trajectory-based (e.g., TS) and repulsion-based strategies. B3O does not depend on sampling or optimizing joint posteriors, nor does it require hand-tuned diversity penalties or variance-corrected surrogates.

Figure 1: Comparison of batch construction by TS (trajectory optimization) versus B3O (Boltzmann sampling) in 1D, demonstrating higher diversity and effective coverage by B3O.

B3O's sampler-agnostic design allows for seamless substitution of the acquisition function (e.g., switching from UCB to LogEI or EHVI for multi-objective BO) and domain handling (e.g., using different samplers for mixed or discrete spaces).

Theoretical Regret Analysis

The authors derive a finite-time cumulative regret bound for B3O under an increasing inverse temperature schedule, particularly with UCB-type acquisitions. Under regularity conditions and assuming exact Boltzmann sampling, they show that the cumulative regret

$$R(T, B; f) = \mathbb{E}\left[\sum_{t=1}^T\sum_{b=1}^B f(\mathbf{x}^*)-f(\mathbf{x}_{t,b})\right]$$

is upper bounded by

$O(\sqrt{T} + \sqrt{C_1 T \beta_T \gamma_T})$

where $q$0 and $q$1 are standard information-theoretic terms from GP bandit literature. This matches the batch GP-UCB rate up to a negligible additive term from the Boltzmann sampling relaxation. The analysis explicitly motivates an annealing schedule for $q$2 but empirically constant $q$3 often suffices.

Sampling and Diversity

The core technical contribution is demonstrating that, for large batch sizes, Boltzmann sampling delivers diverse and well-distributed candidate sets with computational scaling $q$4. The paper explores the impact of $q$5 on the diversity–quality trade-off:

Figure 2: Effect of inverse temperature $q$6 on the diversity and mode-seeking behavior of Boltzmann batch sampling from a 2D UCB acquisition.

Diversity metrics measured as average pairwise distance between batch points confirm substantially higher and more sustained diversity for B3O compared to trajectory-based TS, especially as the optimization progresses.

Empirical Evaluation

Synthetic Benchmark Performance

B3O is evaluated on classic noisy, multimodal synthetic benchmarks: Shekel-4D, Ackley-5D, and Hartmann-6D. Baselines include sequential EI, batch TS, random search, and small-batch algorithms.

Figure 3: Simple regret performance on Shekel, Ackley, and Hartmann functions, highlighting B3O's superior final regret and robustness to $q$7 scheduling.

B3O outperforms TS and other parallel baselines across all evaluated functions, with the LogEI variant (constant $q$8) consistently achieving the lowest regret in higher-dimension and more multimodal landscapes. Notably, B3O's scheduled UCB variant is most effective on Shekel for escaping deep local minima, indicating flexibility in acquisition strategy and $q$9 schedule.

Small-Batch Comparison

Even at small batch sizes ($\mathcal{O}(B^3)$0), where traditional batched BO methods are more competitive, B3O is at least as effective or better than $\mathcal{O}(B^3)$1-EI and local penalization on challenging benchmarks.

Figure 4: Small-batch (B=5) comparison, confirming B3O's robustness in both scheduled and constant temperature regimes relative to strong baselines.

Real-World Applications

Multi-Objective Battery Electrode Design

In a high-dimensional battery design task (9D, multi-objective), B3O is directly applied via an EHVI acquisition. B3O (with $\mathcal{O}(B^3)$2 function evaluations) matches sequential EHVI's performance with equivalent total cost, and the constant-temperature B3O yields Pareto fronts closer in diversity and coverage to an NSGA-II benchmark.

Figure 5: (Left) Pareto front comparison for battery electrode optimization; (Right) hypervolume difference convergence, demonstrating sample efficiency and diversity advantages for B3O.

Mixed-Variable Formula E Configuration

On a real-world mixed-variable (continuous-discrete) race car configuration problem, B3O's sampler-agnostic property enables optimization via a simple Metropolis-Hastings kernel without BO loop changes. B3O delivers the best objectives per iteration and per evaluation, compared to scalable TS and a sequential mixed-variable EI baseline, regardless of $\mathcal{O}(B^3)$3 settings.

Figure 6: Best objective trace for mixed-variable Formula E race car optimization, showing rapid convergence of B3O with both LogEI and UCB acquisitions.

Diversity Analysis

Diversity ablations confirm that B3O maintains substantially higher batch diversity throughout the optimization on all benchmarks, a sharp contrast to TS where diversity collapses early due to variance starvation effects.

Figure 7: Batch diversity traces for Shekel, Ackley, and Hartmann; B3O strategies consistently maintain higher mean pairwise batch distances.

Practical and Theoretical Implications

B3O provides a robust, scalable framework for large-batch BO in settings where exploration–exploitation calibration, candidate diversity, and parallel evaluation dominate computational cost. Its sampler- and acquisition-agnostic nature allows for immediate deployment in both continuous and hybrid-domain applications, including constrained and multi-objective settings. Theoretical guarantees confirm that the regret cost of relaxing maximization to Boltzmann sampling is negligible for practical purposes.

The framework motivates further research on adaptive temperature schedules, specialized samplers for higher-dimensional and structured spaces, and extensions to asynchronous parallel BO. More generally, B3O establishes Boltzmann sampling as a foundation for modern, scalable BO pipelines where diversity and computational tractability must be balanced.

Conclusion

B3O makes a strong case for reframing batched BO as a sampling problem, attaining linear batch-scaling, consistent batch diversity, and robust final regret. It requires only minor modifications to existing BO pipelines, is broadly compatible with arbitrary surrogates and acquisitions, and demonstrates empirical and theoretical competitiveness on both classic and modern, heterogeneous optimization tasks. This framework is well suited as a foundation for scalable, parallel Bayesian optimization under computational and experimental constraints.