DOMD-GLB: Efficient Nonstationary GL Bandits

• DOMD-GLB is an algorithmic framework for nonstationary generalized linear bandits that models rewards with a nonlinear link and time-varying parameters.
• It employs a one-pass, exponentially discounted online mirror descent estimator offering O(1) per-round computation and memory, independent of time.
• The method achieves dynamic regret guarantees by integrating drift measures (P_T) and change points (Γ_T) with UCB-based action selection and confidence geometry.

Searching arXiv for DOMD-GLB and closely related nonstationary generalized linear bandit work to ground the article. DOMD-GLB is an algorithmic framework for nonstationary generalized linear bandits in which the expected reward is modeled through a nonlinear link function with an unknown time-varying parameter. It utilizes discounted online mirror descent for parameter estimation and is designed to handle both drifting environments, quantified by a path length $P_T$, and piecewise-stationary environments, quantified by a number of change points $\Gamma_T$. The central feature of the method is a one-pass, exponentially discounted second-order update that yields per-round computation and memory costs independent of time, while retaining nontrivial dynamic regret guarantees (Lee et al., 25 May 2026).

1. Problem setting and objective

In the formulation of DOMD-GLB, rounds are indexed by $t=1,\dots,T$. At each round, an action set $\mathcal X_t\subseteq \mathbb R^d$ is observed and an action $x_t\in\mathcal X_t$ is selected. Each arm is represented by a feature vector $x\in\mathbb R^d$ satisfying $\|x\|_2\le 1$, and the unknown parameter evolves over time as $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$ (Lee et al., 25 May 2026).

The reward model is a generalized linear model. The link function $\mu:\mathbb R\to\mathbb R$ is assumed known, twice differentiable, and strictly increasing, with derivatives bounded as

$$c_\mu\le \mu'(\cdot)\le k_\mu \quad \text{on } [-S,S].$$

Given the chosen action $\Gamma_T$0, the reward $\Gamma_T$1 is drawn from an exponential family with natural parameter $\Gamma_T$2, so that

$\Gamma_T$3

The summary also states that bounded reward implies that $\Gamma_T$4 is self-concordant (Lee et al., 25 May 2026).

Two distinct measures of nonstationarity are used. In drifting environments, variation is quantified by

$\Gamma_T$5

In piecewise-stationary environments, it is quantified by

$\Gamma_T$6

The optimization target is the dynamic pseudo-regret

$\Gamma_T$7

where

$\Gamma_T$8

Because the link is strictly increasing, the benchmark is defined by the same linear score used inside the generalized linear model (Lee et al., 25 May 2026).

This framework encompasses linear, Bernoulli, and binomial rewards. A plausible implication is that DOMD-GLB is intended as a computationally lightweight alternative to nonstationary GLB methods built around repeated maximum-likelihood recomputation.

2. Discounted online mirror descent estimator

The estimator in DOMD-GLB is a one-pass quadratic surrogate update with exponential forgetting. At round $\Gamma_T$9, the negative log-likelihood loss is

$t=1,\dots,T$0

with derivatives

$t=1,\dots,T$1

A quadratic surrogate is formed around the current estimate $t=1,\dots,T$2:

$t=1,\dots,T$3

This local model is the basis for the online update (Lee et al., 25 May 2026).

Discounted curvature information is accumulated through

$t=1,\dots,T$4

where $t=1,\dots,T$5 is the forgetting factor. The corresponding pre-update matrix is

$t=1,\dots,T$6

Thus newer curvature has weight approximately $t=1,\dots,T$7, while older curvature decays geometrically as a function of age (Lee et al., 25 May 2026).

The mirror map is chosen as

$t=1,\dots,T$8

with Bregman divergence

$t=1,\dots,T$9

Fixing step size $\mathcal X_t\subseteq \mathbb R^d$0, the update is

$\mathcal X_t\subseteq \mathbb R^d$1

Equivalently, after dropping $\mathcal X_t\subseteq \mathbb R^d$2-independent constants,

$\mathcal X_t\subseteq \mathbb R^d$3

The state is then advanced by

$\mathcal X_t\subseteq \mathbb R^d$4

The resulting procedure is fully online: it retains only the current parameter and discounted curvature state, rather than the full observation history (Lee et al., 25 May 2026).

3. Action selection and confidence geometry

DOMD-GLB combines the estimator with a UCB rule. At round $\mathcal X_t\subseteq \mathbb R^d$5 the selected action satisfies

$\mathcal X_t\subseteq \mathbb R^d$6

The exploration term is expressed in the inverse discounted curvature metric, so uncertainty is coupled directly to the same matrix that drives the DOMD update (Lee et al., 25 May 2026).

The analysis is carried out on a high-probability event under which the true parameter belongs to the confidence set

$\mathcal X_t\subseteq \mathbb R^d$7

Here

$\mathcal X_t\subseteq \mathbb R^d$8

is the statistical radius,

$\mathcal X_t\subseteq \mathbb R^d$9

and the drift contribution is encoded by

$x_t\in\mathcal X_t$0

This decomposition separates estimation error caused by stochastic sampling from error induced by temporal parameter movement (Lee et al., 25 May 2026).

A common misconception is that nonstationary GLB confidence sets must be built by explicitly revisiting a window of past observations at every round. DOMD-GLB does not do so: the confidence geometry is propagated through the discounted matrix $x_t\in\mathcal X_t$1 and the drift kernel $x_t\in\mathcal X_t$2, both of which are updated online.

4. Dynamic regret guarantees

For drifting environments, the analysis combines the UCB rule, Lipschitzness of $x_t\in\mathcal X_t$3, a discounted elliptical-potential bound, and a bound on the cumulative drift kernels. The summary gives

$x_t\in\mathcal X_t$4

together with

$x_t\in\mathcal X_t$5

These ingredients imply that for any $x_t\in\mathcal X_t$6, with probability $x_t\in\mathcal X_t$7,

$x_t\in\mathcal X_t$8

When $x_t\in\mathcal X_t$9 is tuned as

$x\in\mathbb R^d$0

the analysis yields three regimes; in the intermediate $x\in\mathbb R^d$1 regime,

$x\in\mathbb R^d$2

one obtains

$x\in\mathbb R^d$3

(Lee et al., 25 May 2026).

For piecewise-stationary environments, the argument splits rounds into “good” rounds, with no change in the last $x\in\mathbb R^d$4 steps, and “bad” rounds. The number of bad rounds is bounded by $x\in\mathbb R^d$5, each contributing at most $x\in\mathbb R^d$6. On good rounds, the drift kernel is controlled by

$x\in\mathbb R^d$7

and the path length is bounded by

$x\in\mathbb R^d$8

Choosing

$x\in\mathbb R^d$9

forces

$\|x\|_2\le 1$0

With

$\|x\|_2\le 1$1

the overall regret bound becomes

$\|x\|_2\le 1$2

(Lee et al., 25 May 2026).

These guarantees are dynamic rather than static: the benchmark changes with the underlying parameter sequence. This suggests that the central analytical problem is not only concentration but also how forgetting interacts with cumulative temporal variation.

5. Computational profile and comparison with prior approaches

The computational profile of DOMD-GLB is explicitly one of its main contributions. The algorithm stores only $\|x\|_2\le 1$3 and $\|x\|_2\le 1$4, with $\|x\|_2\le 1$5 a $\|x\|_2\le 1$6 matrix. Each round computes $\|x\|_2\le 1$7, $\|x\|_2\le 1$8, and solves a $\|x\|_2\le 1$9-dimensional convex quadratic over $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$0. The stated costs are memory $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$1 and time $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$2 per round, both independent of $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$3 (Lee et al., 25 May 2026).

By contrast, the summary states that sliding-window or weighted MLE methods must revisit past data, leading to $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$4 or $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$5 memory and growing computation, and that some rely on nonconvex projection steps without polynomial-time guarantees. The paper therefore positions DOMD-GLB as a fully online alternative to history-dependent estimation procedures (Lee et al., 25 May 2026).

The abstract states the same point in asymptotic language: DOMD-GLB incurs only $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$6 computation and memory costs per round, where the constant depends on $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$7 but not on time. The authors further state that, to the best of their knowledge, it is the first algorithm for nonstationary GLBs with per-round computation and memory costs independent of time (Lee et al., 25 May 2026).

6. Assumptions, limitations, and prospective extensions

The assumptions emphasized in the summary are bounded features and rewards, together with upper and lower bounds on $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$8, which provide local curvature control through self-concordance. These conditions are structurally important because the algorithm relies on second-order surrogates and discounted Hessian accumulation rather than repeated exact likelihood optimization (Lee et al., 25 May 2026).

A stated limitation appears in the piecewise-stationary guarantee: the regret bound incurs an extra $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$9 factor compared to the best MLE methods. Reducing that factor while remaining fully online is identified as an open problem (Lee et al., 25 May 2026). This is the main tradeoff explicitly recorded in the summary: improved online efficiency is accompanied by a gap, at least in the current analysis, relative to the strongest MLE-based dependence on curvature.

The same summary identifies several extensions. The DOMD framework may be applied to other nonstationary parametric bandit models, including heavy-tailed and MNL settings, and to reinforcement-learning settings with time-varying dynamics (Lee et al., 25 May 2026). A plausible implication is that the discounted mirror-descent mechanism is being presented not merely as a specialized estimator for one GLB model, but as a template for online second-order adaptation under controlled nonstationarity.