---
title: Time-Varying Bayesian Optimization
url: https://www.emergentmind.com/topics/time-varying-bayesian-optimization-tvbo
type: topic
---

# Time-Varying Bayesian Optimization

Time-Varying Bayesian Optimization (TVBO) is the sequential optimization of an expensive, noisy black-box objective whose value changes with time, so the optimization target is a moving maximizer rather than a single static optimum. In GP-based formulations, the unknown response is written either as \(f:\mathcal S\times\mathcal T\to\mathbb R\) with observations \(y_i=f(\mathbf x_i,t_i)+\epsilon\), or as a round-indexed sequence \(f_t\) with \(y_t=f_t(x_t)+\varepsilon_t\); performance is then measured against the instantaneous optimizer, typically through dynamic regret, rather than against a fixed incumbent [2505.13012]. Across the literature, temporal variation is modeled through spatio-temporal kernels, bounded variation in RKHS norm, reset or sliding-window mechanisms, elapsed-time-aware acquisitions, and, in structured domains, time-aware latent representations [2102.06296].

## 1. Formal models and performance criteria

A standard TVBO formulation places a GP prior directly on a spatio-temporal objective,
\[
f \sim \mathcal{GP}(0,k),
\]
with observations
\[
y_i = f(\bm x_i,t_i)+\epsilon,\qquad \epsilon \sim \mathcal N(0,\sigma_0^2),
\]
and instantaneous regret
\[
r_i = f(\bm x_i^*,t_i)-f(\bm x_i,t_i), \qquad \bm x_i^*=\argmax_{\bm x\in\mathcal S} f(\bm x,t_i),
\]
aggregated into
\[
R_n=\sum_{i=1}^n r_i.
\]
This is the core dynamic benchmark in TVBO, and the no-regret property is usually expressed as \(\lim_{n\to\infty} R_n/n=0\) [2505.13012].

A distinct but closely related frequentist formulation assumes that each time slice lies in a common RKHS and that nonstationarity is controlled by a variation budget,
\[
\sum_{t=1}^{T-1}\|f_{t+1}-f_t\|_{\mathcal H}\le P_T.
\]
Under this model, dynamic regret remains the central criterion,
\[
\mathcal R(T)=\sum_{t=1}^T \bigl(f_t(x_t^*)-f_t(x_t)\bigr),
\]
but temporal change is quantified geometrically in the same RKHS that defines smoothness [2102.06296].

TVBO also admits a continuous elapsed-time interpretation. When evaluating an action takes non-constant time \(t_n\), the relevant timestamp is not the round index but the cumulative elapsed time
\[
\tau_n=\sum_{i=1}^n t_i,
\]
and the observation model becomes
\[
y_n=f(x_n,\tau_n)+z_n,\qquad z_n\overset{i.i.d.}{\sim}\mathcal N(0,\sigma^2).
\]
In that setting, regret is defined at the actual completion times,
\[
r_n=\max_{x\in D}f(x,\tau_n)-f(x_n,\tau_n),
\]
so the difficulty of TVBO depends not only on how fast the function changes but also on how rapidly feedback arrives [2003.04691].

These formulations share the same operational problem: old data become stale. The mathematical differences lie in how staleness is encoded—explicit time kernels, RKHS drift budgets, or elapsed-time indexing—and in which comparator is deemed appropriate. In dynamic molecular design, for example, current-time rescoring leads to the current-objective best-so-far
\[
\tilde f_t=\max_{i\le t} f_t(x_i),
\]
rather than the usual static best-so-far summary [2603.00935].

## 2. Spatio-temporal surrogates and stale-data handling

Most TVBO methods rely on a GP surrogate over joint spatial and temporal inputs. A common construction is the separable spatio-temporal kernel
\[
k((\bm x,t),(\bm x',t'))=k_S(\bm x,\bm x')\,k_T(t,t'),
\]
or its latent-space analogue
\[
k((z,t),(z',t'))=k_z(z,z')\,k_t(t,t').
\]
This gives posterior predictions whose influence decays both with spatial distance and temporal separation, and underlies classical TV-GP-UCB, latent-space extensions, and safe spatio-temporal BO [2505.13012].

| Mechanism | Representative formulation | Representative work |
|---|---|---|
| **Sliding window** | Use only the most recent \(w\) observations in the surrogate | [2102.06296] |
| **Periodic reset** | Restart GP-UCB every \(H\) rounds | [2102.06296] |
| **Event-triggered reset** | Treat the problem as static until residuals violate a GP confidence bound | [2208.10790] |
| **Uncertainty injection** | Preserve posterior mean while increasing uncertainty over time via a Wiener-process temporal kernel | [2207.11120] |
| **Response-time-aware memory control** | Choose a maximal dataset size from the response-time function \(R(n)\) and delete observations by minimal Wasserstein impact | [2501.18963] |
| **Stale-data refresh with side queries** | Reinterpret old data as heteroscedastic observations of the current function and selectively refresh them | [2510.21274] |

Restart and sliding-window GP-UCB are the canonical forgetting schemes in the RKHS variation-budget setting. R-GP-UCB discards all previous data every \(H\) rounds, whereas SW-GP-UCB keeps only the last \(w\) samples; both retain the standard acquisition rule
\[
x_t=\arg\max_{x\in\mathcal X}\mu_{t-1}(x)+\beta_t\sigma_{t-1}(x),
\]
but differ in how much stale information is retained [2102.06296].

Event-triggered TVBO replaces fixed schedules with online inconsistency tests. ET-GP-UCB keeps a static GP-UCB model within each block and resets only when the newest observation violates a probabilistic uniform error bound derived from GP regression:
\[
|y_t-\mu_{\mathcal D_t}(x_t)|>\sqrt{\rho_{t'}\,\sigma_{\mathcal D_t}(x_t)+\bar w_{t'}}.
\]
This makes the method adaptive to realized temporal changes without requiring the exact drift-rate hyperparameter \(\varepsilon\) used by TV-GP-UCB and related Markovian formulations [2208.10790].

Controller-tuning work has argued that some TVBO problems are not well described by mean-reverting temporal priors. UI-TVBO therefore replaces “Back-2-Prior” forgetting with uncertainty injection via the Wiener-process temporal kernel
\[
k_{T,wp}(t,t')=\sigma_w^2(\min(t,t')-c_0),
\]
so that in the absence of new data the posterior mean is retained while posterior variance grows linearly with elapsed time:
\[
\mathrm{cov}[(\mathbf{x},t),(\mathbf{x}',t')]=\sigma_k^2\sigma_w^2(t'-t).
\]
This is tailored to incremental and lasting changes such as wear and tear in closed-loop control systems [2207.11120].

A separate practical line makes stale-data management depend explicitly on computational latency. BOLT models the optimizer’s response time \(R(n)\), recommends the maximal dataset size
\[
n^*=\arg\max_{n\in\mathbb N}\sum_{i=1}^n k_T^2(iR(n)),
\]
and removes the observation whose deletion minimally perturbs the posterior in integrated \(2\)-Wasserstein distance. A plausible implication is that, in TVBO, memory size is jointly a statistical and systems-level design parameter rather than a purely inferential choice [2501.18963].

## 3. Regret theory, impossibility results, and conditions for no-regret

Frequentist TVBO under RKHS variation budgets admits dynamic-regret guarantees with explicit drift–estimation tradeoffs. For R-GP-UCB and SW-GP-UCB, the leading terms are
\[
\mathcal{R}(T)=O\!\left(\sqrt{\gamma_H}H^{3/2}P_T+\beta(\delta)T\sqrt{\frac{\gamma_H}{H}}\right)
\]
and
\[
\mathcal{R}(T)=O\!\left(\sqrt{\gamma_w}w^{3/2}P_T+\beta_T(\delta)T\sqrt{\frac{\gamma_w}{w}}\right),
\]
respectively, which makes the choice of reset period \(H\) or window size \(w\) a direct bias–variance compromise between adaptation to drift and statistical efficiency [2102.06296].

When elapsed evaluation time is non-constant, the theory changes qualitatively. Continuous Time-Varying GP-UCB (CTV) scores candidates at their future completion time, either using known duration \(t(x)\),
\[
x_{n+1}=\arg\max_{x\in D}\left[\mu_n(x,\tau_n+t(x))+\beta_n\sigma_n(x,\tau_n+t(x))\right],
\]
or by integrating over a learned delay distribution. Its regret analysis introduces the evaluation time uniformity
\[
C_{\epsilon,T'}=\sum_{\tau_j\in T'}\sum_{\tau_k\in T'}\min\!\left(\frac{1}{\epsilon^2},(\tau_j-\tau_k)^2\right),
\]
showing that hardness depends on the geometry of the elapsed-time sequence, not just on temporal smoothness \(\epsilon\) or the number of evaluations [2003.04691].

Recent asymptotic theory identifies a stronger structural distinction: the spectral class of the temporal kernel. Under separable kernels with fixed-frequency sampling \(t_i=i\Delta\), if the temporal kernel \(k_T\) is broadband or band-limited, then
\[
\mathbb E[R_n]\in \Theta(n),
\]
whereas if \(k_T\) is almost-periodic or low-rank, GP-UCB can satisfy
\[
\lim_{n\to\infty}\frac{R_n}{n}=0.
\]
In this analysis, no-regret is possible only when temporal variation has discrete spectral support; continuous temporal spectra induce irreducible temporal novelty and persistent regret [2505.13012].

A different impossibility result arises once response time is modeled explicitly. For separable kernels with noncompact temporal spectral support and positive observation cost \(c\), any TVBO algorithm incurs
\[
\mathbb E[R_T]\in \Omega(\epsilon_c T),
\]
and, under the paper’s assumptions,
\[
\mathbb E[R_T]\in \Theta(T).
\]
This result motivates bounded-memory designs such as BOLT and formalizes the practical claim that, in TVBO, using more data can eventually be harmful because it slows decision making while the world continues to drift [2501.18963].

No-regret can nevertheless be recovered in a different regime by augmenting bandit feedback with refreshed side information. SparQ-GP-UCB models stale observations as heteroscedastic proxies for the current objective,
\[
y_t=f_T(x_t)+\epsilon_{t,T},\qquad \epsilon_{t,T}\sim \mathcal N\!\bigl(0,\sigma^2([T-t]^2+1)\bigr),
\]
discards highly stale points, and refreshes a sparse subset through \(\mathcal O(\log^d t)\) expert queries per round. Under a bounded per-step drift assumption and a squared exponential kernel, it achieves
\[
R_T=\tilde{\mathcal O}(\sqrt T),
\]
which shows that minimal auxiliary feedback can restore dynamic no-regret in a regime where pure bandit TVBO cannot [2510.21274].

## 4. Structured domains, latent representations, and induced objectives

TVBO increasingly extends beyond direct optimization in Euclidean action spaces. In latent-space Bayesian optimization (LSBO), the objective over structured objects is pulled back through a decoder \(\Gamma\) to a latent objective
\[
g(z,t):=f_t(\Gamma(z)).
\]
Time-Aware Latent-space Bayesian Optimization (TALBO) argues that, in structured domains, temporal drift should affect not only the surrogate \(g(z,t)\) but also the latent search geometry itself. Its central device is dual temporal modeling: a spatio-temporal GP surrogate over \((z,t)\) together with a time-conditioned GP-prior latent generative model in which time is one of the covariates of the latent code,
\[
z(\mathbf c;\boldsymbol A)=\sum_{r=1}^R \boldsymbol A^{(r)}\boldsymbol\phi^{(r)}(c^{(r)}).
\]
The surrogate uses
\[
k((z,t),(z',t'))=k_z(z,z')\,k_t(t,t'),
\]
while the representation update re-embeds all evaluated objects under the time-aware latent map [2603.00935].

This design is motivated by a specifically TVBO observation: if the decoder geometry remains fixed while the objective drifts, then latent neighborhoods may become misaligned with current preferences. TALBO evaluates this claim on drifting multi-property molecular objectives of the form
\[
f_t(\mathbf x)=\sum_{k=1}^K \alpha_{t,k}\,s_k(\mathbf x),
\]
with smoothly varying scalarization weights generated by GP-softmax processes. It reports higher current-objective best-so-far \(\tilde f_t\), lower cumulative regret, and better average rank over time than strong LSBO baselines, while remaining competitive under time-invariant objectives [2603.00935].

A related but distinct extension appears in time-varying bilevel optimization. There, the time-varying object is the lower-level response map
\[
\tilde g_t(x)=\argmin_{y\in\mathcal Y} g_t(x,y),
\]
which induces a changing upper-level objective
\[
h_t(x)=f\bigl(x,\tilde g_t(x)\bigr).
\]
W-SparQ-BL models \(\tilde g_t\) with coordinate-wise GPs, uses windows \(\mathcal W_j\) to cap staleness, performs sparse refresh queries at window boundaries, and selects actions by optimistic propagation of response uncertainty through the known upper-level objective,
\[
x_{t+1}=\argmax_{x\in\mathcal X}\ \max_{y\in \tilde{\mathcal U}_t(x)} f(x,y).
\]
Under its assumptions, the method attains sublinear dynamic regret in both stationary and time-varying settings [2605.20962].

These developments broaden the meaning of TVBO. The time-varying quantity need not be a directly observed scalar reward surface; it may be a latent representation, a decoder-induced search geometry, or an implicit objective determined by another adaptive subsystem. A plausible implication is that future TVBO theory will increasingly need to reason about how temporal variation propagates through learned structure rather than only through scalar function values.

## 5. Safety, constraints, and contextual nonstationarity

One branch of the literature addresses time-varying safety constraints directly. TVSafeOpt considers
\[
\max_{\mathbf x\in\mathcal X} f(\mathbf x,t)\quad \text{subject to}\quad c_i(\mathbf x,t)\ge 0,\ i\in\mathcal I_c,
\]
models reward and constraints jointly through a spatio-temporal GP \(h(\mathbf x,t,i)\), and combines posterior confidence intervals with explicit temporal Lipschitz margins. Its safe set update,
\[
S_k = \bigcap_{i \in \mathcal{I}_c} \bigcup_{\mathbf{x} \in S_{k-1}}
\left\{ \mathbf{x}^{\prime} \in \mathcal{X} \mid l_k(\mathbf{x},i) - L_{\mathbf{x}} d(\mathbf{x}, \mathbf{x}^{\prime}) - L(t) \ge 0 \right\},
\]
allows the safe region to shrink over time rather than expand monotonically. With the prescribed \(\beta_k\), the method guarantees with probability at least \(1-\delta\) that all points in the maintained safe set are truly safe; when the problem becomes stationary, it also recovers a SafeOpt-style near-optimality result over a reachable safe region [2409.18000].

A different line treats apparent nonstationarity as observed exogenous context rather than latent temporal drift. VACBO studies objectives and constraints of the form \(\ell(\theta,z_t)\) and \(g_i(\theta,z_t)\), where the ambient/context variable \(z_t\) is observed before the decision is made. It is therefore best interpreted as contextual constrained BO for time-varying environments, not as canonical TVBO with a temporal kernel. Its distinctive contribution is a budgeted-violation mechanism,
\[
\sum_{t=1}^{T}\bar c_i(\theta_t,z_t)\le B_i,
\]
together with the contextual acquisition CPEI and adaptive per-step violation budgets \(B_{i,t}\) [2301.12099].

Primal-Dual Contextual Bayesian Optimization (PDCBO) pushes this contextual view further by targeting dynamic contextual regret against the moving comparator \(\theta^*(z_t)\),
\[
\mathcal R_T=\sum_{t=1}^T\Big(f(\theta_t,z_t)-f(\theta^*(z_t),z_t)\Big),
\]
while enforcing time-average constraints through a dual update,
\[
\lambda_{t+1} = \big[ \lambda_t+\underline g_t(\theta_t,z_t)+\epsilon e \big]^+.
\]
Under its assumptions, it yields
\[
\mathcal R_T = \mathcal O\!\left( \sum_{i=0}^N \gamma_{i,T}\sqrt{T} \right), \qquad \mathcal V_T=0.
\]
The important conceptual distinction is explicit in the paper: time variation is induced by observed contexts \(z_t\), not by an explicit GP prior over time itself [2304.06104].

For TVBO, these works delineate two nonstationary regimes. In one, time is a latent or direct input, and temporal smoothness or spectral structure governs the inference problem. In the other, variation is explained by observable side information and the GP is stationary on an augmented input \((x,z)\). The distinction matters for both modeling and guarantees.

## 6. Applications, empirical patterns, and practical design tensions

TVBO has been applied to controller tuning, beamline drift correction, sensor selection, industrial energy optimization, compressor operation, molecular design, and bilevel game-theoretic problems. In online controller tuning, UI-TVBO is motivated by lasting changes in plant dynamics and convexity in controller parameters; on time-varying LQR tuning with changing friction, it reports lower cumulative regret and fewer unstable controller evaluations than TV-GP-UCB, and convexity-constrained variants further reduce over-exploration [2207.11120].

Event-triggered resets have shown strong empirical behavior when change rates are unknown or non-constant. ET-GP-UCB outperforms competing GP-UCB variants on synthetic TVBO tasks, a temperature sensor benchmark, and a changing cart-pole policy-search problem, with particular robustness under misspecified \(\varepsilon\) and under abrupt changes, where fixed-rate temporal models degrade more severely [2208.10790].

In complex optical systems, TVBO has been implemented in a deliberately simple sliding-window form rather than through an explicit temporal kernel. For X-ray beam alignment in the six-crystal, twelve-dimensional HXRSND system, the method uses a Matern-kernel GP, UCB with \(\beta=0.1\), and a fixed window \(w=40\); under linear drift, discontinuous drift, and a throughput-constrained setting, the reported outcome is that about \(80\%\) or more of TVBO samples keep beam position error below \(1~\mu\text{m}\) across 10 numerical experiments, whereas retaining the initial optimum degrades substantially [2509.05658].

The empirical literature also makes the practical limits of TVBO explicit. Smooth-drift models can be effective, but abrupt or highly nonstationary changes motivate reset or refresh mechanisms; rescoring old candidates under the current objective is useful in controlled benchmarks but may be infeasible in real deployments; and hyperparameter misspecification, especially in temporal components, can dominate behavior. Several papers therefore emphasize bounded memory, conservative noise modeling, or robustness to unknown change rates as first-class design choices rather than implementation details [2501.18963].

Across these variants, a common theme is that TVBO is not only a problem of exploration under uncertainty; it is a problem of exploration under uncertainty while the inferential target moves, the relevance of historical data decays, and the optimizer’s own computational latency can feed back into performance. The modern literature correspondingly treats temporal modeling, stale-data handling, structure exploitation, and systems constraints as coupled components of a single sequential design problem.

Source: https://www.emergentmind.com/topics/time-varying-bayesian-optimization-tvbo