---
title: 'GKB-UCB: Unified Kernelized Bandits'
url: https://www.emergentmind.com/topics/gkb-ucb
type: topic
---

# GKB-UCB: Unified Kernelized Bandits

Generalized Kernelized Bandits Upper Confidence Bound (GKB-UCB) is an algorithmic framework for regret minimization in the setting where a learner optimizes an unknown reward function $f^*$ belonging to a Reproducing Kernel Hilbert Space (RKHS) and observes outcomes corrupted by exponential family (EF) noise with a mean response $\mu(f^*)$, a general nonlinear link. This model strictly generalizes both the kernelized bandits (KBs) and generalized linear bandits (GLBs), providing a unified analytical and algorithmic treatment. GKB-UCB employs a novel self-normalized Bernstein-type, dimension-free concentration inequality to derive sharp high-probability regret bounds, overcoming obstacles posed by the heteroscedastic, non-Gaussian, and potentially infinite-dimensional settings [2508.01681].

## 1. Problem Formulation

The decision domain is a subset $\mathcal{X}\subseteq\mathbb{R}^d$ (possibly infinite). At each round $t=1,\dots,T$, the learner selects $x_t\in\mathcal{X}$ and observes stochastic feedback $y_t$ drawn according to an exponential family model,
\[
y_t \sim p(y \mid x_t; f^*) = \exp\left(\frac{y f^*(x_t) - m(f^*(x_t))}{g} + h(y)\right),
\]
where $m$ is a convex log-partition function, $g>0$ a known scale, and $h$ a normalization function. The reward mean and variance at $x_t$ satisfy
\[
\mathbb{E}[y_t \mid x_t] = \mu(f^*(x_t)), \quad \mathrm{Var}[y_t \mid x_t] = g\dot{\mu}(f^*(x_t)), \quad \mu = m'.
\]
Key assumptions include bounded RKHS norm of $f^*$, bounded kernel $k(x, x)$, bounded noise $|y_t-\mu(f^*(x_t))|$, and self-concordance of $\mu$. The regret is measured by
\[
R_T = \sum_{t=1}^T \left(\mu(f^*(x^*)) - \mu(f^*(x_t))\right),
\]
with $x^* \in \arg\max_{x\in\mathcal{X}} f^*(x)$.

A central parameter is
\[
\kappa_* = \frac{1}{\dot{\mu}(f^*(x^*))},
\]
quantifying the maximum curvature (slackness) of the reward mean function in the domain.

## 2. Self-Normalized Bernstein-Type, Dimension-Free Inequality

The exponential family structure induces noise with variance dependent on $\dot{\mu}(f^*)$. Standard self-normalized concentration results either do not leverage the variance structure (Hoeffding-type, thus overly pessimistic) or become dimension-dependent (ill-suited for infinite-dimensional RKHS).

GKB-UCB establishes a Freedman-style martingale concentration that is both variance-adaptive (Bernstein-like) and dimension-free, using a stitching argument across variance levels. For a martingale sequence with absolute value bounded by $R$ and conditional variance $v_s$, the result guarantees, with high probability over all $t$,
\[
\left\| S_t \right\|_{H_t^{-1}} \leq \big(\sqrt{73\log\det(\lambda^{-1}H_t)}+\sqrt{3}\big) \sqrt{\log\frac{\pi^2(\rho+1)^2}{3\delta}} + \frac{3 R K}{\sqrt{\lambda}} \log\frac{\pi^2(\rho+1)^2}{3\delta},
\]
where $S_t = \sum_{s=1}^{t-1} \epsilon_s \phi(x_s)$ with $\epsilon_s=y_s-\mu(f^*(x_s))$, $H_t$ is a data-dependent weighted norm matrix, and $\rho$ is logarithmic in $T$ and kernel/scale parameters. Critically, the bound is independent of ambient or feature-space dimension.

## 3. Algorithmic Structure

At round $t$, GKB-UCB performs:

(a) **Maximum Likelihood Estimation:** Compute
\[
\hat f_t = \arg\min_{f\in\mathcal{H}} \sum_{s=1}^{t-1} \frac{-y_s f(x_s) + m(f(x_s))}{g} + \frac{\lambda}{2}\|f\|^2.
\]
By the representer theorem, this is finite-dimensional over $t-1$ parameters.

(b) **Statistical Confidence Set Construction:** For any $f\in\mathcal{H}$ (or the associated coefficient vector $\alpha$),
\[
\mathcal{C}_t(\delta) = \left\{ f:\, \|g_t(f) - g_t(\hat f_t)\|_{H_t(f)^{-1}} \le B_t(\delta; f) \right\},
\]
with
\[
g_t(f) = \sum_{s < t} \frac{\mu(f(x_s))}{g}\phi(x_s) + \lambda \alpha,
\]
and $B_t(\delta; f)$ given by the novel concentration bound.

(c) **Optimistic Arm Selection:** Play
\[
x_t \in \arg\max_{x\in\mathcal{X},\,f\in\mathcal{C}_t(\delta)} \mu(f(x)).
\]
Since $\mu$ is nondecreasing, this is equivalent to maximizing $f(x)$ over the confidence set.

Practical implementations employ finite-dimensional approximations for $\mathcal{C}_t$ and loss-based formulations (Appendix A in [2508.01681]).

## 4. Regret Analysis and Theoretical Guarantees

The regret decomposition leverages the confidence that $f^*$ resides in $\mathcal{C}_t(\delta)$ for all $t$. The analysis yields:
\[
R_T = O\big( (1+R_s B K) \beta_T \sqrt{\gamma_T}\sqrt{T/\kappa_*} \big) + o(\sqrt{T}),
\]
where $\beta_T$ collects supremal bounds for the confidence width, $\gamma_T$ is the maximal information gain, and $o(\sqrt T)$ is a negligible second-order term arising from self-concordance. This matches, up to logarithmic and multiplicative factors, the minimax rates for both KB and GLB regimes. The analysis requires precise control of the elliptical potential via the new Bernstein-type bound, ensuring optimal dependence in $T$, $\gamma_T$, and $\kappa_*$.

## 5. Unification of Kernelized and Generalized Linear Bandits

GKB-UCB recovers and unifies prior literature:
- **Kernelized Bandits:** With $\mu=I$, $m(u)=u^2/2$, $g=1$, the confidence ellipsoid shrinks to the classical Gaussian process bandit structure; the regret bound specializes to $\widetilde{O}(\gamma_T \sqrt T)$.
- **Generalized Linear Bandits:** With linear kernel and finite $d$, $\gamma_T = O(d \log T)$, and the regret matches $\widetilde{O}(\sqrt{dT/\kappa_*})$ as in established GLB results.

GKB-UCB thus subsumes both classes, providing a unified approach for nonparametric, nonlinear, and heteroscedastic bandit models.

## 6. Implementation Considerations

At each iteration, GKB-UCB requires solving:
1. A finite-dimensional maximum likelihood problem in the coefficient vector $\alpha \in \mathbb{R}^{t-1}$;
2. A convex (typically quadratic) program to select the optimistic arm, subject to a loss-based confidence region.

Key hyperparameters are the regularization $\lambda$ and Freedman-stitching parameters $\eta, v_0$, with typical choices $\lambda=1$, $\eta=e$, $v_0=1$.

The resulting computational complexity per round remains manageable for moderate $T$ but scales with the number of past actions.

## 7. Significance and Scope

The introduction of GKB-UCB and its associated concentration theory closes an open problem regarding dimension-free, variance-aware self-normalized inequalities in RKHS-valued, EF-noise bandit processes. The method offers a robust, theoretically sound, and practically relevant solution for bandit learning under highly flexible reward models, smoothly interpolating between classical linear/GLB and nonparametric/kernelized scenarios [2508.01681].

Source: https://www.emergentmind.com/topics/gkb-ucb