Dynamic Generalized Linear Bandits

• Nonstationary GLBs are models that extend generalized linear bandits by incorporating time-varying parameters into nonlinear reward structures for dynamic decision-making.
• They use regime-based techniques such as breakpoint, variation-budget, and path-length models combined with forgetting strategies like sliding windows and discounting.
• Recent approaches emphasize computational efficiency with one-pass online updates, balancing statistical regret and scalability in adversarial or drifting contexts.

Nonstationary generalized linear bandits (GLBs) are contextual bandit models in which the conditional mean reward is governed by a generalized linear model while the unknown parameter changes over time. In the canonical formulation, at round $t$ the learner observes an action set in $\mathbb R^d$, chooses an action, and receives a stochastic reward with mean $\mu(\langle x_t,\theta_t^\star\rangle)$, where $\mu$ is a known inverse link and $\theta_t^\star$ is an unknown time-varying parameter. The topic sits at the intersection of generalized linear bandits, nonstationary contextual bandits, and adaptive online estimation. Its central technical difficulty is that nonstationarity must be handled simultaneously with the nonlinear score geometry of GLMs, so methods that work for stationary GLBs or nonstationary linear bandits do not automatically extend. The modern literature organizes the problem primarily around three nonstationarity models—breakpoint-based, variation-budget, and path-length—and around three main methodological families: forgetting-based likelihood methods, projection-based optimism under drift, and one-pass online updates (Russac et al., 2020, Russac et al., 2020, Faury et al., 2021, Lee et al., 25 May 2026).

1. Core model, notation, and regret criteria

In nonstationary GLBs, the expected reward is modeled as

$$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$

or, in canonical exponential-family form,

$$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$

The action or context set may vary with time, and several works allow it to be arbitrary or even adversarially chosen. Standard boundedness assumptions include $\|\theta_t^\star\|_2\le S$, bounded action norms, and bounded rewards; the admissible parameter domain is typically

$\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$

For GLM analysis, a central pair of constants is

$$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$

evaluated over the bounded feasible domain. The constant $\mathbb R^d$0 is the local curvature or nondegeneracy parameter, while $\mathbb R^d$1 measures how nonlinear the model is over the admissible region (Faury et al., 2021).

Performance is measured by dynamic regret or dynamic pseudo-regret against the instantaneous oracle. A standard form is

$\mathbb R^d$2

or equivalently against $\mathbb R^d$3. This is not static regret: the comparator is allowed to change with time because both the environment and the optimal action may change (Faury et al., 2021).

The literature uses several distinct nonstationarity measures. In breakpoint-based models, the parameter sequence is piecewise constant and complexity is measured by the number of changepoints $\mathbb R^d$4. In drifting models, complexity is measured by a variation budget or path length such as

$\mathbb R^d$5

These regimes are analytically different. Breakpoint models target abrupt switches; variation-budget and path-length models target cumulative drift, allowing many small changes (Russac et al., 2020, Faury et al., 2021, Lee et al., 25 May 2026).

2. Abruptly changing environments and forgetting-based likelihood methods

The first systematic treatment of nonstationary GLBs with abrupt changes introduced two UCB-type algorithms based on forgetting old data: a sliding-window penalized MLE and a discounted penalized MLE. In that setting, the parameter sequence is piecewise constant, nonstationarity is measured by the number of breakpoints $\mathbb R^d$6, and the learner is evaluated by dynamic regret. The analysis covers general, possibly adversarially chosen action sets rather than stochastic contexts (Russac et al., 2020).

The sliding-window estimator retains only the most recent $\mathbb R^d$7 observations: $\mathbb R^d$8 while the discounted estimator uses exponentially decaying weights: $\mathbb R^d$9 Both methods construct optimism bonuses from prediction error bounds rather than from a direct Euclidean ellipsoid around the estimator. Because the unconstrained penalized MLE need not satisfy $\mu(\langle x_t,\theta_t^\star\rangle)$0, both algorithms project it back into the admissible ball through a geometry induced by the nonlinear score map $\mu(\langle x_t,\theta_t^\star\rangle)$1 (Russac et al., 2020).

The corresponding dynamic regret guarantees are of order

$\mu(\langle x_t,\theta_t^\star\rangle)$2

under appropriate tuning of $\mu(\langle x_t,\theta_t^\star\rangle)$3 or $\mu(\langle x_t,\theta_t^\star\rangle)$4. In particular, for sliding windows, choosing

$\mu(\langle x_t,\theta_t^\star\rangle)$5

yields that rate, and for discounting, choosing

$\mu(\langle x_t,\theta_t^\star\rangle)$6

gives the same order (Russac et al., 2020). These results exhibit the standard forgetting tradeoff: larger memory improves statistical efficiency on stationary stretches but worsens adaptation after a change because stale data remain influential.

A later line sharpened this abrupt-change analysis using self-concordance and projection-free confidence construction. In self-concordant GLBs, which include logistic and Poisson regression, forgetting can be implemented through either exponential discounting or sliding windows, but the confidence analysis exploits local curvature along the segment joining $\mu(\langle x_t,\theta_t^\star\rangle)$7 and $\mu(\langle x_t,\theta_t^\star\rangle)$8 instead of relying only on the global lower bound $\mu(\langle x_t,\theta_t^\star\rangle)$9. The resulting discounted algorithm $\mu$0 uses the regularized weighted MLE directly, without the earlier non-convex projection step and without a burn-in phase (Russac et al., 2020).

Under abruptly changing environments with at most $\mu$1 switches, the self-concordant analysis yields gap-independent dynamic regret

$\mu$2

and, under a uniform minimum-gap assumption $\mu$3,

$\mu$4

The stated improvement over prior nonlinear GLB analyses is that the dependence on the GLM curvature worsens less severely: the analysis improves from $\mu$5-type dependence to $\mu$6 or $\mu$7, depending on the regime (Russac et al., 2020).

3. Parameter drift, variation budgets, and the projection problem

A distinct strand studies GLBs under parameter drift rather than only abrupt switches. In this model, the time-varying parameter obeys a variation budget

$\mu$8

and the learner may know only an upper bound $\mu$9. The main conceptual result in this line is that nonstationary GLBs are not a routine extension of nonstationary linear bandits, because the nonlinear link function creates a delicate issue in confidence construction and linearization (Faury et al., 2021).

The core critique is directed at naive LB-to-GLB extensions. In GLBs, confidence analysis linearizes the reward around an estimator, and this requires the interpolation path between the true parameter and the estimator to stay inside the admissible region $\theta_t^\star$0, where derivative bounds such as

$\theta_t^\star$1

hold. Earlier arguments effectively behaved as if the unconstrained MLE $\theta_t^\star$2 remained in $\theta_t^\star$3, but there is no proof of this, and in fact it is typically false. Even in the stationary case, existing deviation bounds suggest that $\theta_t^\star$4 can be outside $\theta_t^\star$5, and under nonstationarity it can drift even farther. This matters because the lower bound

$\theta_t^\star$6

only holds when the relevant $\theta_t^\star$7 lies in $\theta_t^\star$8 (Faury et al., 2021).

The issue is especially sharp in logistic models. Since regret scales linearly in

$\theta_t^\star$9

forcing $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$0 into an enlarged admissible ball can make $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$1 explode exponentially. For logistic $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$2, the paper notes $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$3, and inflating the radius by $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$4 adds a factor $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$5. This is why the stationary projection step of Filippi et al. remains structurally important, but the stationary projection is insufficient under drift because there are now two distinct errors: stochastic learning error and deterministic tracking bias, and they are controlled in different metrics (Faury et al., 2021).

To address this, the paper introduces a noiseless discounted estimator $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$6 to split the error into

$$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$7

corresponding respectively to learning and tracking. A high-probability confidence region controls $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$8 around $$\mathbb E[r_{t+1}\mid \mathcal F_t,x_t]=\mu(\langle x_t,\theta_t^\star\rangle),$$9 in a $$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$0-metric, while the tracking term is controlled in a $$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$1-metric. The resulting algorithm, BVD-GLM-UCB, introduces a generalized projection center $$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$2 that shifts the confidence set until it intersects $$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$3, minimizing the shift in the tracking-relevant metric. This is the key adaptation of the stationary GLM-UCB projection to the nonstationary setting (Faury et al., 2021).

Under an orthogonal arm-set geometry, choosing

$$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$4

gives the high-probability bound

$$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$5

Without that geometric assumption, choosing

$$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$6

yields

$$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$7

The orthogonal-arm result matches the known nonstationary linear-bandit minimax rate up to logarithms under that favorable geometry, while the general-geometry result leaves a gap, mirroring a similar gap already present in corrected nonstationary linear-bandit analyses (Faury et al., 2021).

4. Confidence construction, curvature control, and recurring technical themes

The central analytical challenge in nonstationary GLBs is that likelihood curvature is nonlinear and time-varying, while stale data induce bias that does not appear in stationary GLBs. Across the literature, confidence statements are therefore built from nonlinear score maps, weighted Hessian-like matrices, and action-dependent prediction inequalities rather than from a single static Euclidean ellipsoid (Russac et al., 2020, Russac et al., 2020).

In abrupt-change forgetting methods, the design matrix under discounting has the form

$$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$8

and a squared-weight analogue $$d \mathbb P_\theta(r\mid a)=\exp\!\big(r\, a^\top \theta - b(a^\top \theta)+c(r)\big)\,d\nu(r), \qquad \mu=\dot b.$$9 appears because weighted martingale concentration involves squared weights. For sliding windows, the corresponding matrix uses only the most recent $\|\theta_t^\star\|_2\le S$0 actions. The prediction error is then bounded by a confidence width proportional to $\|\theta_t^\star\|_2\le S$1, with an additional stale-data bias term in the discounted case (Russac et al., 2020, Russac et al., 2020).

The self-concordant analysis adds a structural inequality of the form

$\|\theta_t^\star\|_2\le S$2

or, in the bounded-reward exponential-family setting, a self-concordance-type bound induced by bounded support. This enables local curvature comparisons along the segment joining $\|\theta_t^\star\|_2\le S$3 and $\|\theta_t^\star\|_2\le S$4, replacing the crude use of a worst-case global lower bound. A practical consequence is that the algorithm can use the regularized MLE directly, even when $\|\theta_t^\star\|_2\le S$5, because the prediction-confidence argument no longer depends on an explicit non-convex projection back onto the parameter ball (Russac et al., 2020).

In variation-budget models, the technical novelty is the explicit separation of learning and tracking. The learning confidence set controls stochastic estimation error through a discounted self-normalized concentration inequality, while a separate tracking lemma bounds the deterministic drift bias caused by the mismatch between current and historical parameters. The generalized projection lemma ensures that the chosen center for admissible prediction does not worsen the tracking term. This decomposition is what earlier flawed analyses omitted when they adapted linear-bandit methods too literally (Faury et al., 2021).

A common misconception is that nonlinear GLB structure only changes constants relative to linear bandits. The literature shows the opposite. The derivative lower bound $\|\theta_t^\star\|_2\le S$6, the admissible-domain requirement for linearization, the distinction between $\|\theta_t^\star\|_2\le S$7, $\|\theta_t^\star\|_2\le S$8, and $\|\theta_t^\star\|_2\le S$9 metrics, and the possibility that $\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$0 all create GLB-specific obstacles. This suggests that nonstationary GLBs are best viewed as a separate technical class rather than as a direct corollary of nonstationary linear-bandit theory (Faury et al., 2021).

5. One-pass online methods and computationally time-independent updates

Most early nonstationary GLB algorithms are MLE-based: they solve a weighted or truncated likelihood problem at every round and often revisit past observations repeatedly. Even when statistically efficient, such methods have computation and memory costs that grow with time, window length, or restart block size. A more recent direction replaces repeated weighted MLE with discounted online mirror descent, yielding a one-pass algorithm called DOMD-GLB (Lee et al., 25 May 2026).

The method maintains a discounted Hessian-like matrix

$\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$1

and a pre-update matrix

$\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$2

Using the local quadratic surrogate of the negative log-likelihood, the iterate is updated by

$\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$3

The action rule is optimistic: $\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$4 The confidence set takes the form

$\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$5

where $\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$6 is a discounted variation term capturing nonstationarity (Lee et al., 25 May 2026).

The algorithm’s main computational claim is that per-round memory is $\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$7 and per-round computation is $\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$8, both independent of the time index $\Theta=\{\theta\in\mathbb R^d:\|\theta\|_2\le S\}.$9. In the paper’s terminology, this is the first nonstationary GLB algorithm with per-round computation and memory costs independent of time. The practical significance is that the update depends only on the current estimate, the maintained curvature matrix, and the current observation; no past samples need to be stored or revisited (Lee et al., 25 May 2026).

For drifting environments measured by path length $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$0, the paper proves

$$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$1

and for piecewise-stationary environments measured by $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$2,

$$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$3

up to suppressed $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$4-dependent factors (Lee et al., 25 May 2026). In piecewise-stationary environments this is slightly weaker than the strongest discounted MLE-based projection method because of an extra curvature dependence, but the computational gains are substantial. This suggests a new design axis in nonstationary GLBs: jointly statistical and computational efficiency, rather than regret alone.

6. Related paradigms, applications, and unresolved directions

The dominant formal nonstationary GLB literature models reward through an explicit known link function and a time-varying parameter, but related work also explores history-dependent representation learning. A notable example combines recurrent neural networks with a linear posterior-sampling head to address nonstationary contextual bandits. That method learns a representation from raw interaction history and then applies linear Thompson sampling in the learned feature space. It explicitly does not formulate a generalized linear bandit with a known inverse link or exponential-family reward model, so it is better understood as a related but distinct approach rather than as a formal nonstationary GLB algorithm (Ramesh et al., 2020).

Its relevance is conceptual rather than definitional. The recurrent method attacks nonstationarity by learning a sufficient state-like embedding of history so that the reward becomes approximately stationary and approximately linear in the learned representation. This is well suited to periodicity, latent state, action-dependent future structure, and partial observability. The accompanying theory is a regret bound for linear posterior sampling with measurement error, not for GLMs or for time-varying GLM parameters. A useful implication is that representation error can induce linear regret components if it does not decay sufficiently fast, which cautions against treating learned history features as if they automatically solved the exploration problem (Ramesh et al., 2020).

Several open distinctions remain central in the formal GLB literature. First, abrupt-change and variation-budget models are not interchangeable: $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$5 counts switches, whereas $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$6 or $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$7 measures cumulative drift. Second, the best tuning of sliding-window length or discount factor usually depends on the unknown nonstationarity level, though bandit-over-bandit style procedures have been proposed for variation-budget drift (Faury et al., 2021). Third, self-concordant projection-free analyses are specific to self-concordant GLMs, while the projection-based drift analysis covers a broader GLB setting but at higher algorithmic complexity (Russac et al., 2020, Faury et al., 2021).

Empirically, the main application domains are those where linear rewards are misspecified or poorly calibrated: logistic bandits for clicks, likes, or diagnoses; Poisson models for count data; and other canonical exponential-family settings. The experimental literature repeatedly reports that nonstationary GLB methods outperform stationary GLB baselines after switches, and that GLB methods outperform linear-bandit baselines when the reward nonlinearity is genuine. It also reports that $$k_\mu=\sup \dot\mu(\cdot),\qquad c_\mu=\inf \dot\mu(\cdot),$$8 can occur frequently in practice, which directly supports the projection and projection-free debates in the theory (Russac et al., 2020, Russac et al., 2020, Faury et al., 2021).

Taken together, the field has converged on a precise picture. Nonstationary generalized linear bandits are defined by dynamic oracle regret, nonlinear reward geometry, and explicitly modeled temporal change in the parameter. Their theory now includes high-probability dynamic regret guarantees under abrupt switches, variation budgets, and path-length drift; projection-based and projection-free confidence mechanisms; and both MLE-based and one-pass online algorithms. The remaining frontier is not whether nonstationary GLBs can be analyzed, but how to unify adaptivity, optimal curvature dependence, computational scalability, and richer forms of temporal structure within a single framework (Faury et al., 2021, Lee et al., 25 May 2026).