Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimization

Abstract: In Bayesian optimization, Thompson sampling selects the evaluation point by sampling from the posterior distribution over the objective function maximizer. Because this sampling problem is intractable for Gaussian process (GP) surrogates, the posterior distribution is typically restricted to fixed discretizations (i.e., candidate points) that become exponentially sparse as dimensionality increases. While previous works aim to increase candidate point density through scalable GP approximations, our orthogonal approach increases density by adaptively reducing the search space during sampling. Specifically, we introduce Adaptive Candidate Thompson Sampling (ACTS), which generates candidate points in subspaces guided by the gradient of a surrogate model sample. ACTS is a simple drop-in replacement for existing TS methods -- including those that use trust regions or other local approximations -- producing better samples of maxima and improved optimization across synthetic and real-world benchmarks.

• The paper introduces ACTS, which adapts candidate sets using GP gradients to enhance TS for high-dimensional BO.
• It demonstrates theoretical consistency, accelerated convergence, and effective batch diversity across robotics, hyperparameter tuning, and molecular design tasks.
• Experiments show ACTS outperforms state-of-the-art methods by dramatically reducing search volume while preserving global exploration guarantees.

Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimization: A Technical Essay

Introduction

This paper introduces Adaptive Candidate Thompson Sampling (ACTS), an algorithm for high-dimensional Bayesian optimization (BO) that fundamentally revises candidate point selection in Thompson Sampling (TS). BO is a sampling-efficient surrogate modeling-based optimization paradigm widely used in hyperparameter tuning, scientific discovery, and robotics. TS is frequently chosen for its randomized policy—balancing exploration and exploitation and enjoying global convergence guarantees. However, canonical TS with a Gaussian process (GP) surrogate is intractable in high dimensions due to the exponential combinatorics of candidate point discretization, severely restricting the practical density of candidate sets.

ACTS addresses this issue by adapting the candidate set to the gradient of a posterior sample, thus increasing the density of samples in subregions likely to contain a (local) maximum of the sampled function. By adaptively restricting the search volume rather than using global, fixed-discretization or axis-aligned subspace perturbations (e.g., RAASP), ACTS achieves more effective posterior maximization without resorting to strong local search heuristics that can bias the optimization.

Background and Related Work

High-dimensional BO remains challenging largely due to the curse of dimensionality in GP surrogate modeling. Trust region methods like TuRBO and axis-aligned subspace strategies such as RAASP, or progressive latent subspace methods like BAxUS, mitigate this curse by reducing the effective search space. In TS, the traditional approach samples from the GP posterior at a fixed, globally- or locally-defined set of candidate points. The computational budget caps these sets severely: with GP posteriors on $M$ points, time and memory scale as $O(M^3)$. As dimensions increase, the volumetric density of candidates falls off exponentially, limiting practical search effectiveness.

Recently, some works have explored direct optimization of GP samples via pathwise conditioning [Wilson et al., 2020, 2021] or MCMC over the maximizer's location, but these approaches suffer from approximation issues or slow mixing in high dimensions. No prior method exploits the fact that the candidate set itself can adaptively depend on the sampled function, which is the key innovation of ACTS.

ACTS: Method and Theoretical Properties

ACTS leverages the joint Gaussian structure of GPs for both function values and their gradients, sampling first the gradient at the incumbent. The method then defines a low-volume, axis-aligned cone in the direction of this (random) gradient, generating candidate points densely within this cone (or its intersection with any imposed trust region). Subsequent sampling of the function over this adaptive set yields, by construction, an exact realization of the GP posterior on the chosen finite set—importantly, unifying candidate point construction and posterior sample path maximization.

This approach has the following technical characteristics:

• Sample-Path-Dependent Candidate Set: Whereas candidate sets are traditionally independent of the sample path, here the set is constructed on-the-fly based on the sampled gradient, focusing density where high-value maxima are likely.
• Exactness and Theoretical Consistency: The paper proves that, under mild regularity conditions, the global consistency of TS is preserved. Despite the locally focused candidate generation, non-collapsed posterior covariance ensures that, with probability one, ACTS eventually samples near the true global maximizer as the query budget grows, maintaining the exploration property of classical TS.
• Compatibility and Complexity: ACTS is a drop-in replacement for any candidate-based TS policy (including trust-region methods and sparse subspace approaches). Computationally, the marginal cost over standard candidate-based TS is negligible and dominated by the matrix operations for GP inference.

Experimental Results

Empirical evaluation is extensive, covering both medium and high-dimensional BO tasks across robotics, control, hyperparameter tuning (up to 1000D), and molecular design (latent spaces >250D). ACTS is compared to major baselines: RAASP, Cylindrical TS, PATHWISE TS, TuRBO, and state-of-the-art methods like SAASBO, BAxUS, and LogEI. The results demonstrate:

• Consistent Top-Ranked Optimization: On nearly all benchmarks, with or without trust region augmentation, ACTS attains the highest or statistically indistinguishable best objective values, often significantly outperforming pure RAASP or Cylindrical TS policies. For example, on MOPTA08 (124D), SVM (388D), and Median Molecules tasks, ACTS statistically dominates other TS-based and non-TS competitors.
• Accelerated Convergence: ACTS typically achieves high-quality solutions in fewer iterations, attributed directly to higher-fidelity discretization of the GP sample maxima.
• Batch Optimization: When extended to batched (parallel) settings, ACTS's use of independent gradient samples ensures batch diversity, which remains effective as batch sizes increase ($q=10,50,100$).
• Search Space Analysis: Quantitative analysis confirms that by shrinking the candidate region volume by several orders of magnitude, ACTS increases the effective candidate density dramatically (e.g., reducing each dimension by $>50\%$ on average in 60D tasks), yet maintains sufficient exploration.

Ablation and Analytical Studies

The authors conduct a thorough ablation study:

• Base Policy Sensitivity: Integrating ACTS with naive Sobol sampling uplifted its performance substantially, demonstrating that the adaptive region, not the sophistication of the underlying candidate generation, drives improvements.
• Search Space Geometry: A stricter, 1D line search along the sampled gradient, though still competitive, underperformed the axis-aligned cone—highlighting the benefit of moderate subspace broadening.
• Gradient Masking: Perturbing more dimensions aligned with larger (normalized) elements of the sampled gradient provided marginal gains, with "softmax" and "top-k" schemes showing some sensitivity but generally less impact than the global ACTS strategy.
• Locality and Exploration: Metrics such as mean query distance and TSP-tour-length demonstrate that ACTS's search is not more local than baseline methods (indeed, PATHWISE often explores more), allaying concerns about over-exploitation due to the local candidate cone.

Limitations and Future Research

ACTS assumes differentiable kernels (e.g., RBF, Matérn) and is thus best suited for objectives with non-trivial smoothness. In settings with very low intrinsic lengthscales (i.e., highly non-smooth objectives), relying on gradient-informed cones could be sub-optimal, although extensions involving autoregressive candidate set constructions and adaptive cone geometry (multi-stage conditioning) are conceivable. Such extensions could bridge towards gradient-descent-like local maximizers while maintaining global consistency (theoretically, at the cost of additional computation).

Integrating ACTS more tightly with latent variable models for structure-exploiting BO, and investigating its statistical behavior under manifold constraints, remain open questions.

Conclusion

ACTS provides a rigorously justified, practically efficient solution for candidate-based TS in high-dimensional Bayesian optimization. Its core mechanism—leveraging sample-path-dependent candidate sets via posterior sample gradients—achieves denser, better-targeted discretization of the search space, consistently yielding higher-value acquisitions across standard and challenging tasks. This strategic adaptation of candidate density has substantial implications for scalable BO in settings where query budgets, model complexity, and domain size are all pressing constraints. Future developments will likely extend ACTS principles to more complex model classes, manifold-constrained objectives, and fully Bayesian surrogate frameworks.

---

Reference:

"Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimization" (2604.08891)