Bayesian Cumulative Regret Bounds

• Bayesian cumulative regret bounds are measures that quantify the total discrepancy between a learner’s sequential performance and the optimal loss achieved in hindsight.
• They inform the selection of hierarchical priors by showing that heavy-tailed distributions, like the t-distribution, can restrict regret growth from quadratic to logarithmic for large optimal parameters.
• These bounds bridge online learning and PAC-Bayesian theory, offering reliable generalization guarantees and enabling efficient statistical strength sharing across related tasks.

Bayesian cumulative regret bounds are a central tool for understanding the theoretical and practical performance of Bayesian learning and optimization algorithms, particularly in sequential prediction, bandit, and reinforcement learning settings. The notion of "cumulative regret" quantifies, over a sequence of T rounds, the aggregate discrepancy between the incurred loss (or negative reward) of a Bayesian learner and the best possible loss achievable by the optimal model in hindsight. In the Bayesian framework, these bounds not only offer guarantees about the efficiency of online or batch learners, but also guide the choice of hierarchical priors, hyperparameters, and structural modeling strategies, connecting risk, robustness, and statistical strength sharing.

1. Hierarchical Bayesian Learners and Regret/Risk Bounds

Hierarchical Bayesian models introduce multi-level prior structures to encode robustness and share statistical strength. A key insight is that hierarchical priors—for example, a Gaussian prior with an inverse gamma hyperprior on its variance—result in heavy-tailed marginal distributions, such as the multivariate t-distribution. This yields regret bounds exhibiting different growth rates depending on the magnitude of the optimal parameter $\theta^*$:

• For small $\|\theta^*\|$ (relative to the scale $\nu \sigma^2$ in the t-prior), the cumulative regret is quadratic, akin to the Gaussian case.
• For large $\|\theta^*\|$, the regret increases only logarithmically with $\|\theta^*\|$, sharply limiting penalty for "unexpectedly" large optimal parameters.

Formally, the regret bound for the multivariate t-prior ($\nu$ degrees of freedom, scale $\sigma^2$) is:

$$R_{\mathrm{Bayes}}^{(\mathrm{mvt})}(Z, \theta^*) \leq \frac{\nu+n}{2} \ln\!\left(1 + \frac{\|\theta^*\|^2}{\nu \sigma^2}\right) + \frac{n}{2} \ln\!\left( \frac{(\nu+1)(\nu+n)}{\nu^2 + [Tc(\nu+1)\sigma^2]/(\nu n)} \right)$$

This formalizes the robustness property—heavy-tailed priors insulate against large, rare parameters (Huggins et al., 2015).

2. From Regret to Generalization: PAC-Bayesian Connections

Regret bounds developed for online Bayesian learning under log-loss are transferable to the batch (statistical) setting via PAC-Bayesian analysis. The central result establishes that if the Bayesian regret is small relative to an empirical risk minimizer, the generalization error (expected risk on unseen data) is controlled. Specifically, for a Gibbs predictor $P_T$,

$$\big| \mathcal{L}(P_T) - \hat{\mathcal{L}}(P_T, Z_T) \big| \leq T^{-1/2} \sqrt{\kappa}\, \sqrt{ B(\hat{\theta}) + C(T) + \ln\!\left(\frac{\kappa'}{\delta}\right) }$$

where $B(\hat{\theta})$ measures complexity via the KL divergence from a test distribution centered around $\hat{\theta}$, $C(T)$ is sublinear in $T$, and $\kappa,\kappa'$ are constants determined by the loss and prior. This bridges online cumulative regret and generalization across arbitrary bounded loss functions (Huggins et al., 2015).

3. Statistical Strength Sharing in Hierarchical Models

Sharing statistical strength via hierarchical models reduces sample complexity, especially when data are spread across related but diverse tasks.

• Single Level (Tasks): For $K$ tasks, each with parameter vector $\theta^{(k)}$, a hierarchical Gaussian prior links them through a shared mean

$$\mu_j \sim \mathcal{N}(0, \sigma_0^2),\quad \theta_j^{(k)} \mid \mu_j \sim \mathcal{N}(\mu_j, \sigma^2)$$

Integrating $\mu_j$ yields a joint Gaussian with inter-task correlation $\rho = \sigma_0^2 / (\sigma_0^2 + \sigma^2)$. The regret for this structure penalizes both the norms of $\theta^{*(k)}$ and their pairwise differences, rewarding parameter proximity and penalizing divergence, thus encapsulating shared information and heterogeneity (Huggins et al., 2015).

• Two-Level (Superclasses): For grouped classes (image categories, for instance), hyperparameters are nested, providing maximum benefit when true parameters within a "superclass" are close. Regret bounds here justify hierarchical groupings in settings such as image classification, where "borrowing strength" leads to superior performance for similar categories and potentially inferior for outlier groups.

4. Practical Hyperparameter Selection and Robustness

Selecting hyperparameters to optimize regret is problem dependent:

• Degrees of freedom $\nu$ (t-distribution): Small $\nu$ provides robustness with minimal sensitivity to large $\|\theta^*\|$, logarithmically controlling regret; setting $\nu$ proportional to the dimension (e.g., $\nu = Cn$) balances performance for small optimal parameters while maintaining tail robustness.
• Sharing strength ($\rho$): In hierarchical Gaussian models, a higher $\rho$ (i.e., larger $\sigma_0^2$) aggressively penalizes task divergence, optimal when true task parameters are expected to be close.
• These settings are not merely hyperparameters but encode strong modeling assumptions about the likely similarity and scale of true parameters—thus practitioner domain knowledge critically informs model configuration.

5. Feature Selection Priors and High-Dimensional Regret Mitigation

Feature selection in high-dimensional spaces benefits from tailored Bayesian priors:

• Bayesian Lasso: Imposes an $\ell_1$-type penalty via Laplace priors, but for general linear models the regret scales as $\Theta(n)$, which is suboptimal when only a small subset of features is active (Huggins et al., 2015).
• Spike-and-Slab Priors: Introduce sparsity explicitly, with latent inclusion indicators,

$$z_i \sim \mathrm{Ber}(p),\quad \theta_i | z_i \sim z_i \delta_0 + (1-z_i)\mathcal{N}(0,\sigma^2)$$

The corresponding regret bound is:

$$R_{\mathrm{Bayes}}^{(\mathrm{SS})}(Z, \theta^*) \leq \frac{\|\theta^*\|^2}{2\sigma^2} + m\ln\!\frac{1}{1-p} + (n-m)\ln\!\frac{1}{p} + \frac{m}{2} \ln\!\left(1+\frac{Tc\sigma^2}{m}\right)$$

with $m = \|\theta^*\|_0$, the number of active features. Crucially, by choosing $p = q^{1/n}$ for fixed $0$n$, preserving linearity in $m$—a sharp contrast to uninformative Gaussian priors. This provides a justified, theory-backed method for prior construction and hyperparameter scaling in large $n$ settings (Huggins et al., 2015).

Application Context	Prior Structure	Effect on Regret Upper Bound
Robust inference	t-distribution	Logarithmic for large $\|\theta^*\|$
Multi-task/statistical share	Hierarchical Gaussian	Penalizes inter-task distance; reduces sample need
Feature selection	Spike-and-slab	Regret scales with $\|\theta^*\|_0$ and $\log n$

6. Summary and Implications

The interplay between hierarchical prior design and Bayesian cumulative regret bounds yields deep practical implications:

• Robustness: Heavy tails (t-priors) guard against large but rare optimal values, ensuring that regret does not grow uncontrollably for “outlier” scenarios.
• Transfer/multi-task learning: Hierarchical Gaussian models rigorously justify, and quantify, the transfer of statistical strength between related tasks, reducing effective data needs.
• Feature selection/high dimensions: Properly scaled spike-and-slab priors avoid the curse of dimensionality, maintaining manageable regret rates by tying the prior inclusion probability to dimensionality.
• Generalization guarantees: PAC-Bayesian conversions extend online regret bounds to expected risk, supporting confidence in Bayesian predictors even beyond log-loss.

The analytical methods—especially the use of second-order Taylor expansions, covering number arguments, and precise KL-divergence calculations—are practically translatable: practitioners can compute, interpret, and trade off the “cost” of prior choices, statistical strength sharing, and robustness in their models. In turn, these insights inform the principled design of Bayesian learning algorithms for contemporary challenges in robust inference, multi-task transfer, and large-scale feature selection (Huggins et al., 2015).