Time-Varying Bayesian Optimization

• Time-Varying Bayesian Optimization is the sequential optimization of dynamic, noisy black-box functions where the objective continuously changes over time.
• It employs spatio-temporal kernels and adaptive mechanisms, such as sliding windows and reset schemes, to handle stale data and track the moving optimum.
• Key advances include dynamic regret analysis, latent space representations, and safety-constrained formulations applicable in various fields like controller tuning and molecular design.

Time-Varying Bayesian Optimization (TVBO) is the sequential optimization of an expensive, noisy black-box objective whose value changes with time, so the optimization target is a moving maximizer rather than a single static optimum. In GP-based formulations, the unknown response is written either as $f:\mathcal S\times\mathcal T\to\mathbb R$ with observations $y_i=f(\mathbf x_i,t_i)+\epsilon$, or as a round-indexed sequence $f_t$ with $y_t=f_t(x_t)+\varepsilon_t$; performance is then measured against the instantaneous optimizer, typically through dynamic regret, rather than against a fixed incumbent (Bardou et al., 19 May 2025). Across the literature, temporal variation is modeled through spatio-temporal kernels, bounded variation in RKHS norm, reset or sliding-window mechanisms, elapsed-time-aware acquisitions, and, in structured domains, time-aware latent representations (Zhou et al., 2021).

1. Formal models and performance criteria

A standard TVBO formulation places a GP prior directly on a spatio-temporal objective,

$f \sim \mathcal{GP}(0,k),$

with observations

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$

and instantaneous regret

$$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$

aggregated into

$R_n=\sum_{i=1}^n r_i.$

This is the core dynamic benchmark in TVBO, and the no-regret property is usually expressed as $\lim_{n\to\infty} R_n/n=0$ (Bardou et al., 19 May 2025).

A distinct but closely related frequentist formulation assumes that each time slice lies in a common RKHS and that nonstationarity is controlled by a variation budget,

$$\sum_{t=1}^{T-1}\|f_{t+1}-f_t\|_{\mathcal H}\le P_T.$$

Under this model, dynamic regret remains the central criterion,

$y_i=f(\mathbf x_i,t_i)+\epsilon$0

but temporal change is quantified geometrically in the same RKHS that defines smoothness (Zhou et al., 2021).

TVBO also admits a continuous elapsed-time interpretation. When evaluating an action takes non-constant time $y_i=f(\mathbf x_i,t_i)+\epsilon$1, the relevant timestamp is not the round index but the cumulative elapsed time

$y_i=f(\mathbf x_i,t_i)+\epsilon$2

and the observation model becomes

$y_i=f(\mathbf x_i,t_i)+\epsilon$3

In that setting, regret is defined at the actual completion times,

$y_i=f(\mathbf x_i,t_i)+\epsilon$4

so the difficulty of TVBO depends not only on how fast the function changes but also on how rapidly feedback arrives (Imamura et al., 2020).

These formulations share the same operational problem: old data become stale. The mathematical differences lie in how staleness is encoded—explicit time kernels, RKHS drift budgets, or elapsed-time indexing—and in which comparator is deemed appropriate. In dynamic molecular design, for example, current-time rescoring leads to the current-objective best-so-far

$y_i=f(\mathbf x_i,t_i)+\epsilon$5

rather than the usual static best-so-far summary (Vu et al., 1 Mar 2026).

2. Spatio-temporal surrogates and stale-data handling

Most TVBO methods rely on a GP surrogate over joint spatial and temporal inputs. A common construction is the separable spatio-temporal kernel

$y_i=f(\mathbf x_i,t_i)+\epsilon$6

or its latent-space analogue

$y_i=f(\mathbf x_i,t_i)+\epsilon$7

This gives posterior predictions whose influence decays both with spatial distance and temporal separation, and underlies classical TV-GP-UCB, latent-space extensions, and safe spatio-temporal BO (Bardou et al., 19 May 2025).

Mechanism	Representative formulation	Representative work
Sliding window	Use only the most recent $y_i=f(\mathbf x_i,t_i)+\epsilon$8 observations in the surrogate	(Zhou et al., 2021)
Periodic reset	Restart GP-UCB every $y_i=f(\mathbf x_i,t_i)+\epsilon$9 rounds	(Zhou et al., 2021)
Event-triggered reset	Treat the problem as static until residuals violate a GP confidence bound	(Brunzema et al., 2022)
Uncertainty injection	Preserve posterior mean while increasing uncertainty over time via a Wiener-process temporal kernel	(Brunzema et al., 2022)
Response-time-aware memory control	Choose a maximal dataset size from the response-time function $f_t$0 and delete observations by minimal Wasserstein impact	(Bardou et al., 31 Jan 2025)
Stale-data refresh with side queries	Reinterpret old data as heteroscedastic observations of the current function and selectively refresh them	(Mauduit et al., 24 Oct 2025)

Restart and sliding-window GP-UCB are the canonical forgetting schemes in the RKHS variation-budget setting. R-GP-UCB discards all previous data every $f_t$1 rounds, whereas SW-GP-UCB keeps only the last $f_t$2 samples; both retain the standard acquisition rule

$f_t$3

but differ in how much stale information is retained (Zhou et al., 2021).

Event-triggered TVBO replaces fixed schedules with online inconsistency tests. ET-GP-UCB keeps a static GP-UCB model within each block and resets only when the newest observation violates a probabilistic uniform error bound derived from GP regression: $f_t$4 This makes the method adaptive to realized temporal changes without requiring the exact drift-rate hyperparameter $f_t$5 used by TV-GP-UCB and related Markovian formulations (Brunzema et al., 2022).

Controller-tuning work has argued that some TVBO problems are not well described by mean-reverting temporal priors. UI-TVBO therefore replaces “Back-2-Prior” forgetting with uncertainty injection via the Wiener-process temporal kernel

$f_t$6

so that in the absence of new data the posterior mean is retained while posterior variance grows linearly with elapsed time: $f_t$7 This is tailored to incremental and lasting changes such as wear and tear in closed-loop control systems (Brunzema et al., 2022).

A separate practical line makes stale-data management depend explicitly on computational latency. BOLT models the optimizer’s response time $f_t$8, recommends the maximal dataset size

$f_t$9

and removes the observation whose deletion minimally perturbs the posterior in integrated $y_t=f_t(x_t)+\varepsilon_t$0-Wasserstein distance. A plausible implication is that, in TVBO, memory size is jointly a statistical and systems-level design parameter rather than a purely inferential choice (Bardou et al., 31 Jan 2025).

3. Regret theory, impossibility results, and conditions for no-regret

Frequentist TVBO under RKHS variation budgets admits dynamic-regret guarantees with explicit drift–estimation tradeoffs. For R-GP-UCB and SW-GP-UCB, the leading terms are

$y_t=f_t(x_t)+\varepsilon_t$1

and

$y_t=f_t(x_t)+\varepsilon_t$2

respectively, which makes the choice of reset period $y_t=f_t(x_t)+\varepsilon_t$3 or window size $y_t=f_t(x_t)+\varepsilon_t$4 a direct bias–variance compromise between adaptation to drift and statistical efficiency (Zhou et al., 2021).

When elapsed evaluation time is non-constant, the theory changes qualitatively. Continuous Time-Varying GP-UCB (CTV) scores candidates at their future completion time, either using known duration $y_t=f_t(x_t)+\varepsilon_t$5,

$y_t=f_t(x_t)+\varepsilon_t$6

or by integrating over a learned delay distribution. Its regret analysis introduces the evaluation time uniformity

$y_t=f_t(x_t)+\varepsilon_t$7

showing that hardness depends on the geometry of the elapsed-time sequence, not just on temporal smoothness $y_t=f_t(x_t)+\varepsilon_t$8 or the number of evaluations (Imamura et al., 2020).

Recent asymptotic theory identifies a stronger structural distinction: the spectral class of the temporal kernel. Under separable kernels with fixed-frequency sampling $y_t=f_t(x_t)+\varepsilon_t$9, if the temporal kernel $f \sim \mathcal{GP}(0,k),$0 is broadband or band-limited, then

$f \sim \mathcal{GP}(0,k),$1

whereas if $f \sim \mathcal{GP}(0,k),$2 is almost-periodic or low-rank, GP-UCB can satisfy

$f \sim \mathcal{GP}(0,k),$3

In this analysis, no-regret is possible only when temporal variation has discrete spectral support; continuous temporal spectra induce irreducible temporal novelty and persistent regret (Bardou et al., 19 May 2025).

A different impossibility result arises once response time is modeled explicitly. For separable kernels with noncompact temporal spectral support and positive observation cost $f \sim \mathcal{GP}(0,k),$4, any TVBO algorithm incurs

$f \sim \mathcal{GP}(0,k),$5

and, under the paper’s assumptions,

$f \sim \mathcal{GP}(0,k),$6

This result motivates bounded-memory designs such as BOLT and formalizes the practical claim that, in TVBO, using more data can eventually be harmful because it slows decision making while the world continues to drift (Bardou et al., 31 Jan 2025).

No-regret can nevertheless be recovered in a different regime by augmenting bandit feedback with refreshed side information. SparQ-GP-UCB models stale observations as heteroscedastic proxies for the current objective,

$f \sim \mathcal{GP}(0,k),$7

discards highly stale points, and refreshes a sparse subset through $f \sim \mathcal{GP}(0,k),$8 expert queries per round. Under a bounded per-step drift assumption and a squared exponential kernel, it achieves

$f \sim \mathcal{GP}(0,k),$9

which shows that minimal auxiliary feedback can restore dynamic no-regret in a regime where pure bandit TVBO cannot (Mauduit et al., 24 Oct 2025).

4. Structured domains, latent representations, and induced objectives

TVBO increasingly extends beyond direct optimization in Euclidean action spaces. In latent-space Bayesian optimization (LSBO), the objective over structured objects is pulled back through a decoder $$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$0 to a latent objective

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$1

Time-Aware Latent-space Bayesian Optimization (TALBO) argues that, in structured domains, temporal drift should affect not only the surrogate $$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$2 but also the latent search geometry itself. Its central device is dual temporal modeling: a spatio-temporal GP surrogate over $$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$3 together with a time-conditioned GP-prior latent generative model in which time is one of the covariates of the latent code,

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$4

The surrogate uses

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$5

while the representation update re-embeds all evaluated objects under the time-aware latent map (Vu et al., 1 Mar 2026).

This design is motivated by a specifically TVBO observation: if the decoder geometry remains fixed while the objective drifts, then latent neighborhoods may become misaligned with current preferences. TALBO evaluates this claim on drifting multi-property molecular objectives of the form

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$6

with smoothly varying scalarization weights generated by GP-softmax processes. It reports higher current-objective best-so-far $$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$7, lower cumulative regret, and better average rank over time than strong LSBO baselines, while remaining competitive under time-invariant objectives (Vu et al., 1 Mar 2026).

A related but distinct extension appears in time-varying bilevel optimization. There, the time-varying object is the lower-level response map

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$8

which induces a changing upper-level objective

$$y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),$$9

W-SparQ-BL models $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$0 with coordinate-wise GPs, uses windows $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$1 to cap staleness, performs sparse refresh queries at window boundaries, and selects actions by optimistic propagation of response uncertainty through the known upper-level objective,

$$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$2

Under its assumptions, the method attains sublinear dynamic regret in both stationary and time-varying settings (Mauduit et al., 20 May 2026).

These developments broaden the meaning of TVBO. The time-varying quantity need not be a directly observed scalar reward surface; it may be a latent representation, a decoder-induced search geometry, or an implicit objective determined by another adaptive subsystem. A plausible implication is that future TVBO theory will increasingly need to reason about how temporal variation propagates through learned structure rather than only through scalar function values.

5. Safety, constraints, and contextual nonstationarity

One branch of the literature addresses time-varying safety constraints directly. TVSafeOpt considers

$$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$3

models reward and constraints jointly through a spatio-temporal GP $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$4, and combines posterior confidence intervals with explicit temporal Lipschitz margins. Its safe set update,

$$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$5

allows the safe region to shrink over time rather than expand monotonically. With the prescribed $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$6, the method guarantees with probability at least $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$7 that all points in the maintained safe set are truly safe; when the problem becomes stationary, it also recovers a SafeOpt-style near-optimality result over a reachable safe region (Li et al., 2024).

A different line treats apparent nonstationarity as observed exogenous context rather than latent temporal drift. VACBO studies objectives and constraints of the form $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$8 and $$r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),$$9, where the ambient/context variable $R_n=\sum_{i=1}^n r_i.$0 is observed before the decision is made. It is therefore best interpreted as contextual constrained BO for time-varying environments, not as canonical TVBO with a temporal kernel. Its distinctive contribution is a budgeted-violation mechanism,

$R_n=\sum_{i=1}^n r_i.$1

together with the contextual acquisition CPEI and adaptive per-step violation budgets $R_n=\sum_{i=1}^n r_i.$2 (Xu et al., 2023).

Primal-Dual Contextual Bayesian Optimization (PDCBO) pushes this contextual view further by targeting dynamic contextual regret against the moving comparator $R_n=\sum_{i=1}^n r_i.$3,

$R_n=\sum_{i=1}^n r_i.$4

while enforcing time-average constraints through a dual update,

$R_n=\sum_{i=1}^n r_i.$5

Under its assumptions, it yields

$R_n=\sum_{i=1}^n r_i.$6

The important conceptual distinction is explicit in the paper: time variation is induced by observed contexts $R_n=\sum_{i=1}^n r_i.$7, not by an explicit GP prior over time itself (Xu et al., 2023).

For TVBO, these works delineate two nonstationary regimes. In one, time is a latent or direct input, and temporal smoothness or spectral structure governs the inference problem. In the other, variation is explained by observable side information and the GP is stationary on an augmented input $R_n=\sum_{i=1}^n r_i.$8. The distinction matters for both modeling and guarantees.

6. Applications, empirical patterns, and practical design tensions

TVBO has been applied to controller tuning, beamline drift correction, sensor selection, industrial energy optimization, compressor operation, molecular design, and bilevel game-theoretic problems. In online controller tuning, UI-TVBO is motivated by lasting changes in plant dynamics and convexity in controller parameters; on time-varying LQR tuning with changing friction, it reports lower cumulative regret and fewer unstable controller evaluations than TV-GP-UCB, and convexity-constrained variants further reduce over-exploration (Brunzema et al., 2022).

Event-triggered resets have shown strong empirical behavior when change rates are unknown or non-constant. ET-GP-UCB outperforms competing GP-UCB variants on synthetic TVBO tasks, a temperature sensor benchmark, and a changing cart-pole policy-search problem, with particular robustness under misspecified $R_n=\sum_{i=1}^n r_i.$9 and under abrupt changes, where fixed-rate temporal models degrade more severely (Brunzema et al., 2022).

In complex optical systems, TVBO has been implemented in a deliberately simple sliding-window form rather than through an explicit temporal kernel. For X-ray beam alignment in the six-crystal, twelve-dimensional HXRSND system, the method uses a Matern-kernel GP, UCB with $\lim_{n\to\infty} R_n/n=0$0, and a fixed window $\lim_{n\to\infty} R_n/n=0$1; under linear drift, discontinuous drift, and a throughput-constrained setting, the reported outcome is that about $\lim_{n\to\infty} R_n/n=0$2 or more of TVBO samples keep beam position error below $\lim_{n\to\infty} R_n/n=0$3 across 10 numerical experiments, whereas retaining the initial optimum degrades substantially (Mishra et al., 6 Sep 2025).

The empirical literature also makes the practical limits of TVBO explicit. Smooth-drift models can be effective, but abrupt or highly nonstationary changes motivate reset or refresh mechanisms; rescoring old candidates under the current objective is useful in controlled benchmarks but may be infeasible in real deployments; and hyperparameter misspecification, especially in temporal components, can dominate behavior. Several papers therefore emphasize bounded memory, conservative noise modeling, or robustness to unknown change rates as first-class design choices rather than implementation details (Bardou et al., 31 Jan 2025).

Across these variants, a common theme is that TVBO is not only a problem of exploration under uncertainty; it is a problem of exploration under uncertainty while the inferential target moves, the relevance of historical data decays, and the optimizer’s own computational latency can feed back into performance. The modern literature correspondingly treats temporal modeling, stale-data handling, structure exploitation, and systems constraints as coupled components of a single sequential design problem.