---
title: 'DOMD-GLB: Efficient Nonstationary GL Bandits'
url: https://www.emergentmind.com/topics/domd-glb
type: topic
---

# DOMD-GLB: Efficient Nonstationary GL Bandits

Searching arXiv for DOMD-GLB and closely related nonstationary generalized linear bandit work to ground the article.
DOMD-GLB is an algorithmic framework for nonstationary generalized linear bandits in which the expected reward is modeled through a nonlinear link function with an unknown time-varying parameter. It utilizes discounted online mirror descent for parameter estimation and is designed to handle both drifting environments, quantified by a path length $P_T$, and piecewise-stationary environments, quantified by a number of change points $\Gamma_T$. The central feature of the method is a one-pass, exponentially discounted second-order update that yields per-round computation and memory costs independent of time, while retaining nontrivial dynamic regret guarantees [2605.25590].

## 1. Problem setting and objective

In the formulation of DOMD-GLB, rounds are indexed by $t=1,\dots,T$. At each round, an action set $\mathcal X_t\subseteq \mathbb R^d$ is observed and an action $x_t\in\mathcal X_t$ is selected. Each arm is represented by a feature vector $x\in\mathbb R^d$ satisfying $\|x\|_2\le 1$, and the unknown parameter evolves over time as $\theta_t^*\in \Theta=\{\theta:\|\theta\|_2\le S\}$ [2605.25590].

The reward model is a generalized linear model. The link function $\mu:\mathbb R\to\mathbb R$ is assumed known, twice differentiable, and strictly increasing, with derivatives bounded as
$$
c_\mu\le \mu'(\cdot)\le k_\mu \quad \text{on } [-S,S].
$$
Given the chosen action $x_t$, the reward $r_t$ is drawn from an exponential family with natural parameter $x_t^\top \theta_t^*$, so that
$$
\mathbb E[r_t\mid x_t]=\mu(x_t^\top \theta_t^*).
$$
The summary also states that bounded reward implies that $\mu$ is self-concordant [2605.25590].

Two distinct measures of nonstationarity are used. In drifting environments, variation is quantified by
$$
P_T=\sum_{t=1}^{T-1}\|\theta_{t+1}^*-\theta_t^*\|_2.
$$
In piecewise-stationary environments, it is quantified by
$$
\Gamma_T=\sum_{t=1}^{T-1}\mathbf 1\{\theta_{t+1}^*\neq \theta_t^*\}.
$$
The optimization target is the dynamic pseudo-regret
$$
R_T=\sum_{t=1}^T\bigl[\mu(x_t^{*\top}\theta_t^*)-\mu(x_t^\top \theta_t^*)\bigr],
$$
where
$$
x_t^*\in \arg\max_{x\in\mathcal X_t} x^\top \theta_t^*.
$$
Because the link is strictly increasing, the benchmark is defined by the same linear score used inside the generalized linear model [2605.25590].

This framework encompasses linear, Bernoulli, and binomial rewards. A plausible implication is that DOMD-GLB is intended as a computationally lightweight alternative to nonstationary GLB methods built around repeated maximum-likelihood recomputation.

## 2. Discounted online mirror descent estimator

The estimator in DOMD-GLB is a one-pass quadratic surrogate update with exponential forgetting. At round $t$, the negative log-likelihood loss is
$$
\ell_t(\theta)=\frac{m(x_t^\top \theta)-r_t\,x_t^\top \theta}{g(\tau)},
$$
with derivatives
$$
\ell_t'=\frac{\mu(\cdot)-r_t}{g(\tau)}, \qquad \ell_t''=\frac{\mu'}{g(\tau)}.
$$
A quadratic surrogate is formed around the current estimate $\theta_t$:
$$
\tilde \ell_t(\theta)=\ell_t(\theta_t)+\langle \nabla \ell_t(\theta_t),\theta-\theta_t\rangle+\frac12\|\theta-\theta_t\|_{\nabla^2 \ell_t(\theta_t)}^2.
$$
This local model is the basis for the online update [2605.25590].

Discounted curvature information is accumulated through
$$
H_t=\lambda I_d+\sum_{s=1}^{t-1}\gamma^{t-1-s}\nabla^2 \ell_s(\theta_{s+1}),
$$
where $\gamma\in(0,1)$ is the forgetting factor. The corresponding pre-update matrix is
$$
A_t=\gamma H_t+(1-\gamma)\lambda I_d
=\lambda I_d+\sum_{s=1}^{t-1}\gamma^{t-s}\nabla^2 \ell_s(\theta_{s+1}).
$$
Thus newer curvature has weight approximately $1$, while older curvature decays geometrically as a function of age [2605.25590].

The mirror map is chosen as
$$
\Psi(\theta)=\frac12\theta^\top A_t\theta,
$$
with Bregman divergence
$$
D_{\Psi}(\theta,\theta_t)=\frac12\|\theta-\theta_t\|_{A_t}^2.
$$
Fixing step size $\eta=1+RS$, the update is
$$
\theta_{t+1}\in\arg\min_{\theta\in\Theta}
\left\{
\tilde \ell_t(\theta)+\frac{1}{2\eta}\|\theta-\theta_t\|_{A_t}^2
\right\}.
$$
Equivalently, after dropping $t$-independent constants,
$$
\theta_{t+1}\in\arg\min_{\theta\in\Theta}
\Bigl\{
\langle \nabla \ell_t(\theta_t),\theta-\theta_t\rangle
+\frac12\|\theta-\theta_t\|_{\nabla^2 \ell_t(\theta_t)+\eta^{-1}A_t}^2
\Bigr\}.
$$
The state is then advanced by
$$
H_{t+1}=A_t+\nabla^2 \ell_t(\theta_{t+1}).
$$
The resulting procedure is fully online: it retains only the current parameter and discounted curvature state, rather than the full observation history [2605.25590].

## 3. Action selection and confidence geometry

DOMD-GLB combines the estimator with a UCB rule. At round $t$ the selected action satisfies
$$
x_t\in\arg\max_{x\in\mathcal X_t}\bigl[x^\top \theta_t+\beta_t\|x\|_{H_t^{-1}}\bigr].
$$
The exploration term is expressed in the inverse discounted curvature metric, so uncertainty is coupled directly to the same matrix that drives the DOMD update [2605.25590].

The analysis is carried out on a high-probability event under which the true parameter belongs to the confidence set
$$
C_t(\delta)=\{\theta:\|\theta-\theta_t\|_{H_t}\le \beta_t+\iota\,\mathcal V_t\}.
$$
Here
$$
\beta_t^2\approx 4\lambda S^2+O(d\log t)
$$
is the statistical radius,
$$
\iota=\sqrt{2\eta\,k_\mu/g(\tau)},
$$
and the drift contribution is encoded by
$$
\mathcal V_t=\sum_{p=1}^{t-1}\gamma^{(t-1-p)/2}\sqrt{\frac{1-\gamma^p}{1-\gamma}}\,
\|\theta_{p+1}^*-\theta_p^*\|.
$$
This decomposition separates estimation error caused by stochastic sampling from error induced by temporal parameter movement [2605.25590].

A common misconception is that nonstationary GLB confidence sets must be built by explicitly revisiting a window of past observations at every round. DOMD-GLB does not do so: the confidence geometry is propagated through the discounted matrix $H_t$ and the drift kernel $\mathcal V_t$, both of which are updated online.

## 4. Dynamic regret guarantees

For drifting environments, the analysis combines the UCB rule, Lipschitzness of $\mu$, a discounted elliptical-potential bound, and a bound on the cumulative drift kernels. The summary gives
$$
\sum \|x_t\|_{H_t^{-1}}^2
=O\!\left(\frac{1}{c_\mu}\left[d\log(1+\cdots)+T\log(1/\gamma)\right]\right),
$$
together with
$$
\sum_t \mathcal V_t = O\!\left(\frac{P_T}{(1-\gamma)^{3/2}}\right).
$$
These ingredients imply that for any $\gamma\in[1/T,1)$, with probability $1-\delta$,
$$
R_T=\tilde O\!\left(
\frac{k_\mu}{\sqrt{c_\mu}}
\bigl(d^{1/2}T^{1/2}+dT\sqrt{1-\gamma}\bigr)
+
\frac{k_\mu^{3/2}}{\sqrt{c_\mu}}
\frac{P_T}{(1-\gamma)^{3/2}}
\right).
$$
When $\gamma$ is tuned as
$$
\gamma\approx 1-\min\left\{1-\frac1T,\sqrt{\frac{\sqrt{k_\mu}P_T}{dT}}\right\},
$$
the analysis yields three regimes; in the intermediate $P_T$ regime,
$$
\frac{d}{\sqrt{k_\mu}\,T}\le P_T\le \frac{dT}{\sqrt{k_\mu}},
$$
one obtains
$$
R_T=\tilde O\!\left(c_\mu^{-1/2}d^{3/4}P_T^{1/4}T^{3/4}\right)
$$
[2605.25590].

For piecewise-stationary environments, the argument splits rounds into “good” rounds, with no change in the last $D$ steps, and “bad” rounds. The number of bad rounds is bounded by $\Gamma_T D$, each contributing at most $R$. On good rounds, the drift kernel is controlled by
$$
\sum_t \mathcal V_t\le O\!\left(\gamma^{D/2}\frac{P_T}{(1-\gamma)^{3/2}}\right),
$$
and the path length is bounded by
$$
P_T\le 2S\,\Gamma_T.
$$
Choosing
$$
D\approx \frac{2}{\log(1/\gamma)}\log\!\left[\frac{\text{const}\cdot T}{1-\gamma}\right]
$$
forces
$$
\gamma^{D/2}\approx \frac{1-\gamma}{\text{const}\cdot T}.
$$
With
$$
\gamma\approx 1-\min\left\{1-\frac1T,\left(\frac{\Gamma_T\sqrt{c_\mu}}{k_\mu dT}\right)^{2/3}\right\},
$$
the overall regret bound becomes
$$
R_T=\tilde O\!\left(c_\mu^{-1/3}d^{2/3}\Gamma_T^{1/3}T^{2/3}\right)
$$
[2605.25590].

These guarantees are dynamic rather than static: the benchmark changes with the underlying parameter sequence. This suggests that the central analytical problem is not only concentration but also how forgetting interacts with cumulative temporal variation.

## 5. Computational profile and comparison with prior approaches

The computational profile of DOMD-GLB is explicitly one of its main contributions. The algorithm stores only $\theta_t$ and $H_t$, with $H_t$ a $d\times d$ matrix. Each round computes $\nabla \ell_t$, $\nabla^2 \ell_t$, and solves a $d$-dimensional convex quadratic over $\Theta$. The stated costs are memory $O(d^2)$ and time $O(d^3)$ per round, both independent of $t$ [2605.25590].

By contrast, the summary states that sliding-window or weighted MLE methods must revisit past data, leading to $O(t)$ or $O(\text{window-size})$ memory and growing computation, and that some rely on nonconvex projection steps without polynomial-time guarantees. The paper therefore positions DOMD-GLB as a fully online alternative to history-dependent estimation procedures [2605.25590].

The abstract states the same point in asymptotic language: DOMD-GLB incurs only $O(1)$ computation and memory costs per round, where the constant depends on $d$ but not on time. The authors further state that, to the best of their knowledge, it is the first algorithm for nonstationary GLBs with per-round computation and memory costs independent of time [2605.25590].

## 6. Assumptions, limitations, and prospective extensions

The assumptions emphasized in the summary are bounded features and rewards, together with upper and lower bounds on $\mu'$, which provide local curvature control through self-concordance. These conditions are structurally important because the algorithm relies on second-order surrogates and discounted Hessian accumulation rather than repeated exact likelihood optimization [2605.25590].

A stated limitation appears in the piecewise-stationary guarantee: the regret bound incurs an extra $1/c_\mu$ factor compared to the best MLE methods. Reducing that factor while remaining fully online is identified as an open problem [2605.25590]. This is the main tradeoff explicitly recorded in the summary: improved online efficiency is accompanied by a gap, at least in the current analysis, relative to the strongest MLE-based dependence on curvature.

The same summary identifies several extensions. The DOMD framework may be applied to other nonstationary parametric bandit models, including heavy-tailed and MNL settings, and to reinforcement-learning settings with time-varying dynamics [2605.25590]. A plausible implication is that the discounted mirror-descent mechanism is being presented not merely as a specialized estimator for one GLB model, but as a template for online second-order adaptation under controlled nonstationarity.

Source: https://www.emergentmind.com/topics/domd-glb