Dri-MED: Drift-Adaptive Conservative Bandit

• Dri-MED is a drift-adaptive, baseline-conservative bandit algorithm that manages non-stationary, heteroskedastic noise through personalized contextual recommendations.
• It combines discounted ridge regression, MED-style randomized allocation, and a Lagrangian safety mechanism to ensure rigorous, controlled exploration amidst drifting contexts.
• Empirical evaluations show Dri-MED achieves lower cumulative regret and near-zero violation rates compared to strong baselines in diverse drift and noise settings.

Dri-MED, short for Drift-adaptive Minimum Empirical Divergence, is an algorithm for conservative sequential experimentation in a linear contextual stochastic multi-armed bandit setting with multiple users, personalized preference vectors, and drifting context distributions. In the formulation introduced in “Bandits for Efficient Experimentation: Adapting to Control Group, Preferences, and Context Drifts” (Das et al., 8 Jun 2026), the learner must make recommendations over repeated episodes while remaining competitive with a known baseline policy $\pi_0$. The method combines a reduction to a stationary-mean linear bandit with heteroskedastic, non-stationary noise, a discounted weighted estimator, MED-style randomized allocation, and a Lagrangian safety mechanism. Its stated applications include agriculture, clinical trials, recommendation, and other forms of sequential experimentation in which heterogeneity, drift, and control-group conservativeness must be handled jointly (Das et al., 8 Jun 2026).

1. Problem formulation

Dri-MED is defined over episodes $\ell \in [L]$, with $H$ users in each episode. For each user $h$ in episode $\ell$, the learner selects an arm $A_{h,\ell}\in\mathcal A$, observes an $M$-dimensional feedback vector $Y_{h,\ell}$, and obtains scalar reward through the user-specific preference vector $p_h \in \mathbb R^M$:

$r_{h,\ell}(a) \defn Y_{h,\ell}^\top p_h.$

Contexts are sampled from an episode-specific distribution $\ell \in [L]$0, so drift is modeled through changing context distributions rather than fixed covariates. The paper emphasizes that users have heterogeneous scalarizations, and that practitioners often require recommendations not to fall substantially below a control-group or baseline policy $\ell \in [L]$1 (Das et al., 8 Jun 2026).

The conservative requirement is encoded through a per-round comparison with the baseline mean reward. Violations are counted by

$\ell \in [L]$2

where $\ell \in [L]$3 is the conservativeness tolerance. Regret is the cumulative gap to the optimal arm for each user and episode:

$\ell \in [L]$4

This makes Dri-MED a baseline-constrained experimentation algorithm rather than a purely reward-maximizing bandit policy.

2. Reduction to a heteroskedastic linear bandit

The paper’s key modeling step is a reduction under two assumptions. First, conditional on arm and context,

$\ell \in [L]$5

with context affecting the covariance but not the mean:

$\ell \in [L]$6

Second, the mean is linear:

$\ell \in [L]$7

where $\ell \in [L]$8 is unknown and $\ell \in [L]$9 is known (Das et al., 8 Jun 2026).

This implies that the scalar reward mean for user $H$0 is

$H$1

The effective scalar variance is

$H$2

and Dri-MED defines normalized features

$H$3

After rescaling, the learner faces

$H$4

The important implication is that the mean parameter $H$5 remains stationary, while drift enters through time-varying heteroskedasticity and user-specific effective features. This is one of the central cautionary points in the paper: Dri-MED does not address arbitrary mean drift. Its theoretical treatment depends on the context-independent mean assumption, so the non-stationarity it adapts to is variance and normalization drift induced by changing context distributions (Das et al., 8 Jun 2026).

3. Algorithmic design

Dri-MED is explicitly described as being inspired by the linear version of the MED strategy, but adapted to heteroskedastic non-stationary noise. Its first stage constructs user- and episode-specific normalized features and then estimates $H$6 with discounted heteroskedastic ridge regression:

$H$7

with discounted Gram matrix

$H$8

Discounting is the mechanism by which old data are downweighted when context distributions drift (Das et al., 8 Jun 2026).

The second stage makes the baseline conservative through optimism. Dri-MED estimates the current baseline value and inflates it to an optimistic upper confidence estimate:

$H$9

then defines the threshold

$h$0

Because the baseline is estimated optimistically, the resulting safety threshold is stringent, making the algorithm cautious about declaring an arm acceptable.

Within each episode and user, Dri-MED computes an empirical best arm, empirical gaps, and lower confidence bounds:

$h$1

Its MED-style score is then exponentially penalized when the arm’s lower confidence bound falls below the conservative threshold:

$h$2

This Lagrangian factor is the main safety modification. Dri-MED then combines approximate $h$3-optimal design, a point mass on the empirical best arm, and uniform exploration:

$h$4

A saturation test forces additional exploration for geometrically under-sampled arms, and a dual variable is updated after each decision:

$h$5

with $h$6 (Das et al., 8 Jun 2026).

4. Regret, safety, and theoretical guarantees

The analysis decomposes regret into safe and violating components,

$h$7

and introduces a constraint-aware gap through

$h$8

The paper’s main theorem states that, under Assumptions 1–3,

$h$9

where $\ell$0 is a variance-aware multiplicative term derived from heteroskedasticity, and $\ell$1 is a discounted cumulative variance term. The abstract summarizes this more compactly as instance-dependent regret of order

$\ell$2

with $\ell$3 (Das et al., 8 Jun 2026).

The constraint guarantee is separate. Dri-MED is shown to satisfy

$\ell$4

This is not a zero-violation guarantee. A common misunderstanding would be to read Dri-MED as exact safe control; the formal result is instead that expected violations scale only logarithmically-hidden linearly in the feature dimension. The safety argument depends on confidence ellipsoids for discounted regression, MED-style saturation control, and lower bounds on violating-round gaps relative to the baseline.

5. Empirical evaluation

The reported experiments are synthetic. The main setup uses $\ell$5 users, $\ell$6 arms, $\ell$7 episodes, $\ell$8, $\ell$9, and 128 random seeds. User preferences are drawn from a Dirichlet distribution and fixed across episodes. The observation model is

$A_{h,\ell}\in\mathcal A$0

so the mean is fixed while the noise scale changes with context. The baseline policy is chosen as the median-performance arm per user. The study evaluates no drift, gradual drift, periodic drift, and abrupt drift, with Dri-MED hyperparameters including $A_{h,\ell}\in\mathcal A$1, $A_{h,\ell}\in\mathcal A$2, $A_{h,\ell}\in\mathcal A$3, $A_{h,\ell}\in\mathcal A$4, and $A_{h,\ell}\in\mathcal A$5 (Das et al., 8 Jun 2026).

The empirical findings are consistent across drift regimes. Dri-MED and Dri-IMED achieve substantially lower cumulative regret than OFUL, LinMED, LinIMED, and LinTS. OFUL is reported as the strongest baseline, but still about an order of magnitude worse. The paper further states that Dri-MED rapidly drives the true violation rate close to zero after initial exploration, whereas the baselines maintain persistent violations. In arm-allocation plots, Dri-MED concentrates nearly all mass on the oracle-optimal arm for each user, while LinMED spreads mass across suboptimal arms. The reported interpretation is that ignoring drift, heteroskedasticity, or personalized preferences leads to materially worse behavior under the same experimentation budget (Das et al., 8 Jun 2026).

6. Scope, interpretation, and limitations

Dri-MED is best understood as a baseline-conservative experimentation algorithm rather than a general non-stationary contextual bandit in the strongest possible sense. Its theory relies crucially on the mean-stationarity assumption

$A_{h,\ell}\in\mathcal A$6

so if contexts alter the mean as well as the covariance, the paper’s reduction no longer applies. This suggests that Dri-MED is most appropriate when drift manifests as changes in user mix, context distribution, or noise scale rather than structural reward-function drift (Das et al., 8 Jun 2026).

Several practical constraints follow directly from the formulation. First, one needs access to, or a reasonable estimate of, the heteroskedastic variances $A_{h,\ell}\in\mathcal A$7, because both features and rewards are normalized by them. Second, Dri-MED is more computationally involved than optimism-based baselines such as OFUL because it requires confidence computations and an ApproxDesign call. Third, its conservative mechanism is relative to a known baseline policy $A_{h,\ell}\in\mathcal A$8, which is appropriate in settings such as clinical trials or recommendation systems with existing control policies, but less natural when no baseline is available. The paper also notes an implementation-level virtue of randomized allocation: explicit design probabilities are useful for downstream offline evaluation via inverse propensity scoring.

A further interpretive point concerns the name. In the supplied arXiv literature, Dri-MED refers explicitly to the bandit algorithm above, not to a medical diagnosis system or medication recommender. Its connection to clinical domains arises through experimentation with control groups and personalized outcomes, not through medical reasoning architecture. That distinction matters because the algorithm’s contribution is methodological: it unifies personalized scalarization, context-distribution drift, and conservative baseline constraints inside a MED-style linear bandit framework (Das et al., 8 Jun 2026).