Gaussian Process Thompson Sampling

• Gaussian Process Thompson Sampling (GP-TS) is a Bayesian optimization technique that models unknown functions with a Gaussian process and selects points by sampling from its posterior.
• It uses candidate-based approximations and maximizer sampling to efficiently balance exploration and exploitation in continuous and high-dimensional domains.
• Recent extensions include multiobjective optimization, scalable implementations, and adaptive prior selection, with theoretical analyses addressing regret bounds and convergence.

to=arxiv_search.search 天天彩票与你同行json {"query":"Gaussian Process Thompson Sampling Bayesian optimization regret bounds high-dimensional Thompson sampling adaptive candidate points trajectory GP-TS", "max_results": 10} to=arxiv_search.search 体育彩票天天json {"query":"\"Gaussian Process Thompson Sampling\" Thompson Sampling GP arXiv", "max_results": 15} to=arxiv_search.search 大发时时彩开奖json {"query":"(Fadikar et al., 20 Oct 2025, Takeno et al., 10 Mar 2026, Fan et al., 10 Apr 2026) GP-TS", "max_results": 10} Gaussian Process Thompson Sampling (GP-TS) is a Bayesian optimization policy in which an unknown objective function is modeled by a Gaussian process posterior and each decision is obtained by sampling a function from that posterior and optimizing the sample. In its canonical form, GP-TS is a continuous-armed bandit or black-box optimization method: with data $\mathcal{D}_{t-1}$, one draws $g_t \sim p(f\mid \mathcal{D}_{t-1})$ and selects $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$. In continuous domains, this can also be interpreted as sampling from the posterior distribution of the maximizer, even though that maximizer distribution is analytically intractable. The resulting framework has become a standard reference point for posterior-sampling methods in Bayesian optimization and has been extended to high-dimensional search, stochastic simulators, multiobjective optimization, prior selection, and reinforcement learning (Takeno et al., 10 Mar 2026, Bijl et al., 2016).

1. Formal definition and posterior construction

In the standard Bayesian optimization setting, the domain is either finite or a compact subset $\mathcal{X}\subset\mathbb{R}^d$, the latent objective is modeled as $f\sim\mathcal{GP}(0,k)$, and observations satisfy $y_t=f(x_t)+\epsilon_t$ with $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$. Given $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$, the posterior remains a Gaussian process with mean and covariance

$$\mu_t(x)=k_t(x)^\top (K_t+\sigma^2 I_t)^{-1}y_t, \qquad \sigma_t^2(x)=k(x,x)-k_t(x)^\top (K_t+\sigma^2 I_t)^{-1}k_t(x),$$

where $k_t(x)=(k(x,x_1),\dots,k(x,x_t))^\top$ and $g_t \sim p(f\mid \mathcal{D}_{t-1})$0 is the kernel matrix. GP-TS then samples a posterior function and queries its maximizer; in minimization problems the same rule is applied to $g_t \sim p(f\mid \mathcal{D}_{t-1})$1 instead of $g_t \sim p(f\mid \mathcal{D}_{t-1})$2 (Takeno et al., 10 Mar 2026).

A practical implementation typically replaces infinite-dimensional function sampling by finite-dimensional Gaussian sampling on a candidate set. If $g_t \sim p(f\mid \mathcal{D}_{t-1})$3 is a finite candidate set, then under the GP posterior

$g_t \sim p(f\mid \mathcal{D}_{t-1})$4

and a sample can be written as $g_t \sim p(f\mid \mathcal{D}_{t-1})$5 with $g_t \sim p(f\mid \mathcal{D}_{t-1})$6 and $g_t \sim p(f\mid \mathcal{D}_{t-1})$7. The decision is then the candidate with largest sampled value. This finite-set view is the basis of many practical GP-TS systems, even when the underlying problem is continuous (Fan et al., 10 Apr 2026).

2. Maximizer sampling and the decision rule

The conceptual object behind GP-TS is the posterior law of the optimizer,

$g_t \sim p(f\mid \mathcal{D}_{t-1})$8

In finite-armed settings, sampling the optimizer is straightforward: sample rewards and take an argmax. In continuous domains, however, $g_t \sim p(f\mid \mathcal{D}_{t-1})$9 is an infinite-dimensional random object and $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$0 has no analytic form. One way to implement GP-TS is therefore to approximate this maximizer distribution directly rather than optimize an analytic acquisition such as EI or UCB. A sequential Monte Carlo–style approximation, termed Monte Carlo Maximum Distribution, represents $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$1 by weighted particles, updates them through pairwise GP-sampled competitions, and then samples the next query from the resulting particle approximation (Bijl et al., 2016).

The same maximizer-sampling perspective also explains why discretization quality is central. In candidate-based GP-TS, the algorithm does not optimize the continuous posterior sample; it optimizes a restriction of that sample to a finite candidate set. In low dimensions, Sobol or uniform candidate sets can approximate the true maximizer well, but in high dimensions a fixed budget of candidates becomes exponentially sparse, and even large candidate sets may miss regions where a posterior sample is high. This observation motivates later work that modifies candidate generation rather than the GP posterior itself (Fan et al., 10 Apr 2026).

A separate line of work focuses on optimizing the sampled function more accurately once it has been drawn. For separable kernels, a posterior sample can be written in a decoupled form $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$2 using Matheron’s rule, where $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$3 is a prior sample and $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$4 is a smooth data-adjustment term. With squared exponential kernels and separable structure, the prior sample admits a spectral representation whose univariate derivatives can be globally root-found, yielding starting points for multistart optimization of the posterior sample. This reframes GP-TS not as a purely statistical problem but also as a global optimization problem over sampled GP paths (Adebiyi et al., 2024).

3. Regret, concentration, and theoretical limits

The modern regret theory of GP-TS is organized around cumulative regret

$x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$5

and the kernel-dependent maximum information gain

$x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$6

For common kernels, $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$7 has standard growth rates: $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$8 for linear kernels, $x_t=\arg\max_{x\in\mathcal{X}} g_t(x)$9 for squared exponential kernels, and a slower polynomial rate for Matérn kernels. Under the assumption that the objective is itself a GP sample path, GP-TS samples $\mathcal{X}\subset\mathbb{R}^d$0 and plays $\mathcal{X}\subset\mathbb{R}^d$1; this is the reference algorithm studied in current analyses (Takeno et al., 10 Mar 2026).

Several sharp theoretical distinctions between GP-TS and GP-UCB are now explicit. A lower bound shows that GP-TS cannot in general admit high-probability regret bounds with logarithmic dependence on $\mathcal{X}\subset\mathbb{R}^d$2; there exist GP instances for which $\mathcal{X}\subset\mathbb{R}^d$3, implying polynomial $\mathcal{X}\subset\mathbb{R}^d$4 tails. At the same time, GP-TS does satisfy improved upper bounds: the second moment obeys $\mathcal{X}\subset\mathbb{R}^d$5, which yields $\mathcal{X}\subset\mathbb{R}^d$6 with high probability, and the first expected lenient regret bounds for GP bandits are polylogarithmic in $\mathcal{X}\subset\mathbb{R}^d$7 for squared exponential and Matérn kernels. Under additional regularity assumptions, the time-horizon dependence can match the best known GP-UCB rates: $\mathcal{X}\subset\mathbb{R}^d$8 for squared exponential kernels and $\mathcal{X}\subset\mathbb{R}^d$9 for Matérn kernels with $f\sim\mathcal{GP}(0,k)$0, although the dependence on $f\sim\mathcal{GP}(0,k)$1 remains fundamentally weaker than for GP-UCB (Takeno et al., 10 Mar 2026).

A complementary frequentist analysis interprets variance inflation in GP-TS as Thompson sampling from a fractional posterior with tempering parameter $f\sim\mathcal{GP}(0,k)$2. In that framework, regret obeys a kernel-agnostic template

$f\sim\mathcal{GP}(0,k)$3

where $f\sim\mathcal{GP}(0,k)$4 is a posterior contraction rate. This yields $f\sim\mathcal{GP}(0,k)$5 expected regret for squared exponential kernels, and explicit sublinear bounds for Matérn and rational quadratic kernels. The same analysis gives a statistical interpretation to variance inflation: the inflation factor corresponds to $f\sim\mathcal{GP}(0,k)$6 and should scale with posterior contraction rather than be treated as an ad hoc device (Roy et al., 16 Feb 2026).

A distinct theorem addresses convergence of the optimizer itself rather than regret. Under regularity assumptions on the Mercer decomposition and a $f\sim\mathcal{GP}(0,k)$7-greedy exploration schedule, the probability that the selected point remains outside an $f\sim\mathcal{GP}(0,k)$8-ball around the unique maximizer decays as

$f\sim\mathcal{GP}(0,k)$9

where $y_t=f(x_t)+\epsilon_t$0 is the function gap outside that ball. This makes the convergence rate adaptive to the sharpness of the objective near its maximizer (Basu et al., 2017).

4. Computational strategies and scalable implementations

The main computational bottleneck in GP-TS is not the posterior mean but the combination of posterior sampling and optimization of the sampled function. In candidate-based implementations, finite-dimensional sampling on $y_t=f(x_t)+\epsilon_t$1 candidates requires a factorization of an $y_t=f(x_t)+\epsilon_t$2 covariance matrix, which is $y_t=f(x_t)+\epsilon_t$3 without approximation. In high-dimensional spaces, the resulting candidate sets are extremely sparse, and increasing $y_t=f(x_t)+\epsilon_t$4 alone does not overcome the curse of dimensionality. Adaptive Candidate Thompson Sampling addresses this by first sampling the posterior gradient at an incumbent point, then restricting candidate generation to a gradient-aligned cone, and finally sampling function values on those adaptive candidates conditional on the sampled gradient. Because the candidates are generated after partially sampling the GP, yet the conditional Gaussian law is used exactly, the method remains exact at the finite-dimensional GP level while substantially increasing candidate density in promising regions (Fan et al., 10 Apr 2026).

Another route is to improve inner-loop optimization of sampled paths rather than candidate geometry. Rootfinding-based GP-TS exploits separable kernels and spectral representations to identify all local optima of a prior sample through univariate global rootfinding, then uses those prior optima and data locations as starting points for gradient-based optimization of a posterior sample represented in decoupled form. Empirically this improves both optimization of the sampled acquisition and the downstream BO performance of GP-TS relative to random multistart and genetic algorithms (Adebiyi et al., 2024).

Scalability with respect to the evaluation budget can be obtained by replacing exact GP inference with sparse variational GP models and decoupled sampling. In exact GP-TS, posterior updates over $y_t=f(x_t)+\epsilon_t$5 observations scale as $y_t=f(x_t)+\epsilon_t$6, and sampling on a discretization of size $y_t=f(x_t)+\epsilon_t$7 can scale as $y_t=f(x_t)+\epsilon_t$8. Sparse GP-TS uses inducing points or inducing features together with a sampled function of the form “finite-feature prior term + sparse variational update term,” which can be evaluated on $y_t=f(x_t)+\epsilon_t$9 candidates in $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$0. The resulting regret remains of the same order as exact GP-TS, $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$1, under appropriate growth of the inducing set and feature truncation (Vakili et al., 2020).

The sample-then-optimize paradigm extends this computational logic to neural surrogates while preserving a GP interpretation. In STO-BNTS and STO-BNTS-Linear, training a neural network or a linearized NTK model with a quadratic loss and $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$2-regularization yields a function that, in the infinite-width limit, is equivalently sampled from the GP posterior with the neural tangent kernel. The method avoids inverting a parameter-space matrix and admits batch regret bounds while remaining a GP-TS procedure at the kernel level (Dai et al., 2022).

5. Structured extensions of the GP-TS paradigm

GP-TS has been generalized well beyond single-objective deterministic BO. In stochastic simulators, one may index each run not only by continuous parameters $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$3 but also by a random seed $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$4, treat $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$5 as the GP input, and optimize a scalar discrepancy $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$6 between simulated and observed trajectories. The resulting Common Random Number Gaussian Process uses a separable covariance

$\epsilon_t\sim\mathcal{N}(0,\sigma^2)$7

with a constant seed correlation $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$8 when no seed structure is known. Thompson sampling then operates over parameter–seed pairs, and an adaptive grid can be refined by a surrogate-based likelihood over trajectories together with Metropolis–Hastings densification. This converts standard GP-TS into a trajectory-oriented algorithm for identifying data-consistent stochastic realizations rather than only good parameter averages (Fadikar et al., 20 Oct 2025).

In multiobjective optimization, the Thompson idea is lifted from “probability of being globally optimal” to “probability of being Pareto optimal.” qPOTS samples one posterior function per objective, computes the Pareto set of the sampled objectives, and chooses evaluation points from that set by a maximin distance rule. For $\epsilon_t\sim\mathcal{N}(0,\sigma^2)$9 objective, the method reduces to ordinary GP-TS; for $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$0, it becomes a batch multiobjective extension of posterior sampling that replaces hard acquisition-function optimization with evolutionary search over GP sample paths (Renganathan et al., 2023).

Unknown prior structure can itself be folded into GP-TS. EGP-TS models the surrogate posterior as a mixture over $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$1 GP models with different kernels or hyperparameters, updates the model weights online via marginal likelihood, samples a model index from the weight vector, and then samples a function from that model’s posterior. The resulting Bayesian regret scales as $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$2 under the paper’s information-gain assumptions. A related line treats the prior family as a discrete latent variable to be learned during bandit interaction: PE-GP-TS uses prior elimination and attains regret in terms of the worst-case information gain $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$3, whereas HP-GP-TS samples priors from a hyperposterior and achieves Bayesian regret in terms of the average information gain $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$4 (Lu et al., 2022, Sandberg et al., 3 Feb 2025).

Posterior sampling with GP models also extends naturally to temporally extended decision problems. RL-GPS places a joint multi-output GP prior over rewards and transitions in an episodic finite-horizon MDP, samples a complete model at the start of each episode, computes the optimal policy of the sampled MDP by backward induction, and executes that policy for the episode. The regret bound is $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$5, where $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$6 is a multi-output information-gain term. When $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$7, this reduces to the GP bandit setting, so model-based posterior sampling in RL can be viewed as a structured extension of GP-TS (Bayrooti et al., 23 Oct 2025).

6. Limitations, misconceptions, and open problems

A common misconception is that GP-TS and GP-UCB are interchangeable apart from randomization. They are not. GP-TS can match the best known dependence on the horizon $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$8 for smooth kernels, but its high-probability tail behavior is strictly weaker: logarithmic dependence on $\mathcal{D}_t=\{(x_i,y_i)\}_{i=1}^t$9 is impossible in general, whereas GP-UCB retains such dependence. Conversely, GP-TS often requires no separate confidence-width tuning parameter, which partly explains its practical appeal despite weaker $$\mu_t(x)=k_t(x)^\top (K_t+\sigma^2 I_t)^{-1}y_t, \qquad \sigma_t^2(x)=k(x,x)-k_t(x)^\top (K_t+\sigma^2 I_t)^{-1}k_t(x),$$0-dependence (Takeno et al., 10 Mar 2026).

Another misconception is that fixed discretization is intrinsic to GP-TS. It is only one implementation device. The literature includes direct maximizer-distribution approximations, adaptive candidate sets, rootfinding-based inner-loop optimization, sparse variational samplers, and posterior samples induced by neural tangent kernels. This suggests that much of GP-TS design is really about how one represents and optimizes sampled functions, not just how one specifies the GP posterior (Bijl et al., 2016, Fan et al., 10 Apr 2026).

The choice of prior remains a central unresolved issue. Classical theory usually assumes that the GP prior is known, but practical workflows often rely on marginal-likelihood hyperparameter selection without regret guarantees. Recent prior-selection algorithms address this by either eliminating candidate priors or maintaining a hyperposterior over them, which indicates that prior uncertainty should be treated as part of the sequential decision problem rather than an offline nuisance parameter (Sandberg et al., 3 Feb 2025).

Open problems remain substantial. Current theory leaves the optimal dependence on $$\mu_t(x)=k_t(x)^\top (K_t+\sigma^2 I_t)^{-1}y_t, \qquad \sigma_t^2(x)=k(x,x)-k_t(x)^\top (K_t+\sigma^2 I_t)^{-1}k_t(x),$$1 unresolved for GP-TS, does not fully characterize continuous-domain lower bounds, and still requires strong smoothness assumptions for improved Matérn results. The Bayesian setting for variance-inflated TS, extensions to other randomized BO algorithms, and refined analyses for multi-fidelity, multi-objective, constrained, and parallel variants are all explicitly identified as open directions. In practice, exact posterior sampling is also frequently replaced by approximations, so closing the gap between theoretical GP-TS and large-scale implementations remains a central research theme (Takeno et al., 10 Mar 2026).