---
title: Parameterized Bandit Algorithms Overview
url: https://www.emergentmind.com/topics/parameterized-families-of-bandit-algorithms
type: topic
---

# Parameterized Bandit Algorithms Overview

A parameterized family of bandit algorithms consists of a collection of bandit strategies, each indexed by a set of tunable parameters—such as priors, thresholds, regularization strengths, or exploration-exploitation trade-off coefficients—that systematically control their behavior and adaptivity. This rich structure enables the formulation of bandit models that interpolate between classic algorithms, achieve statistical adaptivity to unknown problem structure, and facilitate meta-learning or data-driven hyperparameter tuning. Such parameterizations are central to both the analysis and deployment of modern multi-armed, linear, structured, and nonparametric bandit approaches.

## 1. Canonical Examples of Parameterized Bandit Families

Parameterized bandit families appear universally across stochastic, adversarial, and structured bandit problems. Seminal archetypes include:

- **Tsallis-entropy regularization and FTPL families:** The Tsallis-entropy parameter $\alpha\in(0,1)$ interpolates between EXP3 (as $\alpha\to1$) and a minimax-optimal mirror descent at $\alpha=1/2$, controlling the exploration-exploitation bias in adversarial bandit learning. Similarly, perturbation-based (Follow-the-Perturbed-Leader, FTPL) algorithms form a parameterized family where the hazard rate and moment structure of the perturbation distribution (e.g., Gumbel, Pareto, Weibull, Frechet) govern the regret [1512.04152].

- **Thompson Sampling and Bayesian priors:** The choice of prior (e.g., Gaussian, Beta, Jeffreys, conjugate within exponential families) indexes a continuum of TS algorithms. For one-dimensional exponential families, the use of the Jeffreys prior yields a parameter-free, asymptotically optimal family that achieves the Lai–Robbins regret lower bound [1307.3400].

- **Stochastic linear bandits (ROFUL family):** The ROFUL framework is parameterized by the confidence-radius parameter $\rho$, the optimism-in-expectation parameter $p$, and the sieving rate $\alpha$ in sieved-greedy (SG) variants, smoothly spanning optimistic (OFUL), randomized (Thompson) and greedy behaviors. Regret bounds are explicit functions of these parameters [2002.05152].

- **Computation-regret tradeoff families:** Recent Thompson Sampling variants such as TS-MA-$\alpha$ and TS-TD-$\alpha$ use a parameter $\alpha\in[0,1]$ to mediate the trade-off between computational efficiency (i.e., number of posterior samples or resamplings) and regret, enabling smooth interpolation between full TS (optimal regret, maximal sampling) and highly efficient heuristics (slightly worse regret, polylog sampling) [2405.01010].

- **Regionally, structurally, or finitely parameterized bandits:** In regional bandit algorithms (UCB-g, SW-UCB-g), the control parameter is the sliding window or group-level confidence schedule; in finitely parameterized multi-armed bandits (FP-UCB), the structure of the finite parameter set directly determines arm-elimination protocols and their regret scaling [2003.01328, 1802.07917].

## 2. Theoretical Frameworks and Regret Analysis

Parameterized families are analyzed by making regret bounds explicit functions of the parameters—often exposing precise exploration-exploitation-computation trade-offs, optimality criteria, or tuning guidelines.

- **Asymptotic optimality and one-parameter exponential families:** In the exponential family bandit setting, the TS family parameterized by prior achieves regret
  $$
  R(T) \leq \sum_{a:\Delta_a>0} \frac{\Delta_a}{\mathrm{KL}(\theta_a,\theta^*)} \log T + o(\log T),
  $$
  where KL is the Bregman divergence of the cumulant, and the prior affects only lower order terms if chosen as Jeffreys [1307.3400].

- **Family-level regret bounds:** In ROFUL, general regret is
  $$
  \mathrm{Reg}(T) \leq 2\sqrt{(2K/p + D)T},
  $$
  where $K$ is the “uncertainty complexity” and $p$ reflects the optimism-in-expectation parameter. OFUL ($p=1$), TS, and sieved greedy ($p=\alpha^2$) are all specializations [2002.05152].

- **Computation-adaptivity in TS-MA-$\alpha$ / TS-TD-$\alpha$:** Regret scales as $O\left( K \ln^{1+\alpha} T/\Delta \right)$ under the parameter $\alpha$. Lower $\alpha$ improves regret but increases sampling complexity; higher $\alpha$ decreases total computation but yields an extra logarithmic term [2405.01010].

- **Finitely parameterized FP-UCB:** When the “confusion set” is empty, regret is $O(1)$ independent of $T$; otherwise, regret grows logarithmically, but with smaller constants than classical UCB due to parameter structure [2003.01328].

- **General recipe for randomized bandit families:** Parameterization appears not only in the choice of divergence $D_\pi$ (KL, empirical divergence, moment-based duals) but in the sampling rule, and in verifying that the two-sided bounds on arm sampling probabilities are satisfied. Asymptotically optimal regret is recovered across parametric, bounded, or heavy-tailed families; for example, for exponential families with the canonical KL divergence, TS and MED match the Lai–Robbins lower bound [2303.06058].

## 3. Representative Table of Parameterized Bandit Families

| Family/Algorithm          | Parameter(s)   | Regret/Optimality Properties                                |
|--------------------------|---------------|-------------------------------------------------------------|
| Tsallis-Exp family       | $\alpha$      | $O(\sqrt{TN})$ for $\alpha=1/2$; interpolates EXP3/$\alpha=1$ [1512.04152]      |
| FTPL family              | Perturbation dist. (hazard) | $O(\sqrt{T N \log N})$ for bounded hazard, e.g. Gumbel [1512.04152]      |
| TS (exp. families)       | Prior (e.g. Jeffreys)   | Asymptotic optimality: $R(T) = O(\log T)$, matches lower bound [1307.3400] |
| ROFUL (linear bandit)    | $\rho$, $p$, $\alpha$  | $O\left( d/\alpha \sqrt{T\log T} \right)$ for sieved greedy [2002.05152]       |
| FP-UCB                   | Structure of $\Theta$  | $O(1)$ when confusion set empty; $O(\log T)$ otherwise [2003.01328]       |
| TS-MA-$\alpha$/TS-TD-$\alpha$ | $\alpha\in[0,1]$    | $O\left(K \ln^{1+\alpha}T/\Delta \right)$; sample efficiency tunable [2405.01010] |

This table demonstrates explicit trade-offs governed by parameters, with regret or sample complexity analytically characterizable as a function of the family index.

## 4. Rationale for Parameterization: Flexibility, Adaptivity, Meta-Learning

Parameterized bandit families enable systematic exploration of algorithmic behavior and statistical adaptivity. Key advantages include:

- **Problem-structure exploitation:** Families such as FP-UCB, regional bandits, and parametric/PGLB bandits adapt to finite or grouped reward models, reducing regret relative to structure-agnostic baselines [2003.01328, 1802.07917].

- **Computational tractability:** Computation-regret trade-off families (e.g., TS-MA-$\alpha$) allow practitioners to set $\alpha$ to optimize between statistical efficiency and sample budget, crucial for very large-scale applications [2405.01010].

- **Meta-learning and data-driven selection:** The sample complexity of optimizing over a parameterized family (hyperparameter transfer) is precisely quantifiable in improving bandits, allowing for data-driven tuning without performance loss [2511.10619].

- **Analytic unification:** A multitude of classical and modern strategies (UCB, KL-UCB, TS, MED, h-NPTS, RB-SDA) can be unified as parameter choices in abstract frameworks (e.g., divergence-based, randomized, or regularization-based families) [2303.06058, 2010.14323].

## 5. Parameterized Algorithm Design Principles

Designing effective parameterized bandit families involves the following key principles:

- **Explicit parameter indexing:** Specify continuous or discrete parameters (e.g., prior variance, entropy order, confidence width, threshold power, batch size) and analyze their effect on exploration probability, statistical estimation, and regret bounds.

- **Ensuring adaptivity:** Analytical guarantees must cover both worst-case and instance-adaptive regimes, often requiring uniform finite-time concentration and carefully chosen confidence/threshold schedules or priors [1307.3400, 2405.01010].

- **Universality vs. specialization:** Some families (e.g., RB-SDA) are distribution-free and require no parameter tuning, while others (e.g., TS with prior, FTPL with hazard) deliver optimality only for suitable parameter choices, emphasizing the need for hyperparameter robustness [2010.14323, 1512.04152].

- **Efficient meta-optimization:** The hyperparameter transfer complexity is $O((H/\epsilon)^2 \log(kT))$ for piecewise-constant loss in improving bandits, enabling practical model selection over rich families [2511.10619].

## 6. Recent Advances and Open Problems

Contemporary work continues to generalize and sharpen the role of parameterized families:

- **Non-parametric and heavy-tailed families:** TS-like and divergence-based families parameterize over moment constraints, with concrete minimax-optimal bounds for $h$-moment-limited distributions or nonparametric settings [2303.06058].

- **Functional parameterizations:** Algorithms can be indexed by functionals or norm-approximators (gradient-stable surrogates), capturing entire classes of performance metrics and enabling unified bandit/online-load-balancing analyses for exotic norms [2210.13755].

- **Best-of-both-worlds data-dependent families:** Hybrid parameterized designs (e.g., Hybrid$_{α,B}$) are capable of best-arm identification when problem structure permits and automatically revert to worst-case guarantees otherwise, driven by explicit parameterized switching rules [2511.10619].

- **Online semi-bandit/algorithm-selection problems:** When the loss function itself is parameterized (hyperparameter optimization), the semi-bandit optimization framework yields instance-optimal regret in parameter space via continuous Exp3-SET, exploiting dispersion and Lipschitz-continuity [1904.09014].

Potential open directions include: adaptive parameter selection under nonstationarity, robustification to model misspecification, and scalable meta-optimization in high-dimensional or combinatorial parameter spaces.

## 7. Summary and Impact

Parameterized families of bandit algorithms provide a mechanism to encode, analyze, and computationally tune exploration strategies, adaptivity, and exploitation across a vast array of bandit settings. They unify classic and modern methods, enable meta-learning and structural exploitation, and admit explicit regret and (sometimes) sample complexity guarantees as functions of the family index. The theoretical characterization of such families, the methods for hyperparameter transfer, and the development of distribution-free or instance-adaptive versions are among the most significant advances in the last decade of bandit research [1307.3400, 1512.04152, 2002.05152, 2003.01328, 2210.13755, 2405.01010, 2511.10619].

Source: https://www.emergentmind.com/topics/parameterized-families-of-bandit-algorithms