Neural Dueling Bandits Overview

• Neural dueling bandits are advanced contextual methods that use neural networks to model latent utilities from binary pairwise feedback.
• They replace scalar rewards with pairwise comparisons modeled by the Bradley–Terry–Luce approach, enabling exploration based on utility differences.
• Recent extensions incorporate active query design, multi-user clustering, and variance-aware exploration to enhance efficiency and address delayed feedback.

Searching arXiv for papers on neural dueling bandits and closely related variants. Neural dueling bandits are contextual dueling bandit methods in which the latent reward or utility function is modeled by a neural network and learned from pairwise preference feedback rather than scalar rewards. In the standard formulation, at each round the learner observes a context-dependent candidate set, selects two arms, and receives only a binary outcome indicating which arm is preferred; the preference probability is typically modeled by Bradley–Terry–Luce (BTL), so comparisons depend on a latent utility difference rather than an absolute score. The line of work represented by "Neural Dueling Bandits: Preference-Based Optimization with Human Feedback" introduced the first neural algorithms for this setting with regret guarantees (Verma et al., 2024). Subsequent work extended the framework to online user clustering (Wang et al., 4 Feb 2025), active human-feedback collection (Verma et al., 16 Apr 2025), variance-aware shallow exploration (Oh et al., 2 Jun 2025), and stochastic delayed feedback (Wang et al., 26 May 2026), while the broader dueling-bandit literature supplies the underlying winner notions, regret definitions, and structural assumptions (Bengs et al., 2018).

1. Formal setting and preference model

A contextual dueling bandit differs from a standard contextual bandit in that the learner selects two arms and observes only a relative comparison. In the formulation used by the foundational neural paper, at round $t$ the environment generates a context $c_t$, the learner forms a candidate set $X_t$, selects two context-arm vectors $x_{t,1},x_{t,2}\in X_t$, and observes a binary preference label $y_t\in\{0,1\}$ (Verma et al., 2024). The learner does not observe absolute rewards.

The dominant feedback model is Bradley–Terry–Luce. In the neural dueling formulation, the probability that the first selected arm is preferred over the second is

$$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$

equivalently

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$

This formulation appears throughout the literature on neural dueling bandits and related extensions (Verma et al., 2024, Wang et al., 4 Feb 2025, Verma et al., 16 Apr 2025, Wang et al., 26 May 2026).

The latent target is usually utility-maximization. If

$x_t^\star=\arg\max_{x\in X_t} f(x),$

then one standard average instantaneous regret is

$$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$

and one standard weak instantaneous regret is

$$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$

with cumulative regret

$c_t$0

The multi-user clustering work instead writes the standard dueling regret as

$c_t$1

which is the same utility-loss principle expressed in a collaborative setting (Wang et al., 4 Feb 2025).

Within the broader dueling-bandit literature, this latent-utility perspective is one structured alternative to an arbitrary preference matrix. The survey literature emphasizes that a neural method based on a scalar score $c_t$2 most naturally aligns with utility-based winner notions and Condorcet-style targets, whereas arbitrary pairwise models may require Copeland, Borda, or von Neumann notions instead (Bengs et al., 2018). This suggests that most neural dueling-bandit papers deliberately impose latent-utility structure in order to retain tractable exploration and regret analysis.

2. Neural reward modeling and NTK-based confidence

The defining step in neural dueling bandits is to replace the linear reward assumption with a neural estimator. The foundational formulation uses a fully connected ReLU network of depth $c_t$3, hidden width $c_t$4, and parameters $c_t$5: $c_t$6 with gradient feature

$c_t$7

The loss is a regularized pairwise logistic objective,

$c_t$8

which is maximum likelihood under BTL with regularization around initialization (Verma et al., 2024).

The analysis is carried out in the neural tangent kernel regime. The key feature representation is the initialization gradient $c_t$9, and the uncertainty geometry is built on pairwise gradient differences. In the foundational paper, the design matrix is

$X_t$0

where $X_t$1 denotes the intended gradient-difference feature

$X_t$2

The pairwise uncertainty is

$X_t$3

The effective complexity measure is

$X_t$4

where $X_t$5 aggregates pairwise gradient-difference features over all rounds (Verma et al., 2024).

The central confidence statement is a bound on reward differences rather than rewards themselves: $X_t$6 with

$X_t$7

and linearization error

$X_t$8

This difference-space confidence ellipsoid is the basic technical object that enables UCB and TS exploration under pairwise preference feedback (Verma et al., 2024).

Later papers retain this architecture. The clustering algorithm CONDB also uses a fully connected ReLU network, shared initialization $X_t$9, and an NTK feature map

$x_{t,1},x_{t,2}\in X_t$0

with a cluster-level regularized logistic pairwise objective and the same form of uncertainty on pairwise differences (Wang et al., 4 Feb 2025). Neural-ADB uses the same NTK-style gradient features for active context selection and arm selection (Verma et al., 16 Apr 2025). NDB-DF again relies on initialization-time gradient differences

$x_{t,1},x_{t,2}\in X_t$1

to define the delayed-feedback confidence geometry (Wang et al., 26 May 2026). A broad methodological continuity therefore runs through the literature: nonlinear modeling is supplied by the network, while uncertainty is controlled through linearization around initialization.

3. Core algorithms: UCB and Thompson sampling for pairwise feedback

The first neural dueling-bandit algorithms are NDB-UCB and NDB-TS (Verma et al., 2024). Both use the same asymmetric duel-construction pattern. First, the learner trains the neural reward model on past preference observations. Then it chooses the first arm greedily: $x_{t,1},x_{t,2}\in X_t$2 For UCB, the second arm is chosen optimistically: $x_{t,1},x_{t,2}\in X_t$3 For Thompson sampling, the learner samples

$x_{t,1},x_{t,2}\in X_t$4

and chooses

$x_{t,1},x_{t,2}\in X_t$5

In both cases, the exploration target is uncertainty in a utility difference relative to the greedy anchor arm, not uncertainty in a single arm’s score (Verma et al., 2024).

This basic pattern recurs in later work. In CONDB, the first arm is again

$x_{t,1},x_{t,2}\in X_t$6

and the second arm is

$x_{t,1},x_{t,2}\in X_t$7

where the predictor and information matrix are built from cluster-pooled comparison data (Wang et al., 4 Feb 2025). Neural-ADB adopts the same first-arm greedy rule and then offers both UCB and TS rules for the second arm, but embedded in an active query-design procedure that also chooses the context (Verma et al., 16 Apr 2025). NDB-DF preserves the same greedy-first, optimistic-second structure, but trains with delayed feedback and IPW-corrected loss (Wang et al., 26 May 2026).

A different architectural compromise appears in the variance-aware literature. "Neural Variance-aware Dueling Bandits with Deep Representation and Shallow Exploration" proposes NVLDB, which uses a deep neural representation for the utility model

$x_{t,1},x_{t,2}\in X_t$8

but constructs the Gram matrix only in the final linear layer: $x_{t,1},x_{t,2}\in X_t$9 The loss weights are variance-aware,

$y_t\in\{0,1\}$0

and the paper provides multiple duel-selection strategies: UCB-ASYM, UCB-OSYM, UCB-CSYM, and their TS analogues (Oh et al., 2 Jun 2025). The paper explicitly characterizes this as "deep representation and shallow exploration," meaning that representation learning is deep but exploration is carried out only in the last-layer feature space. This separates it from earlier NDB-style methods that form the Gram matrix from gradients of all network parameters (Oh et al., 2 Jun 2025).

4. Main extensions of the framework

One major extension is collaborative multi-user learning. "Online Clustering of Dueling Bandits" introduces the first clustering of dueling bandit algorithms and proposes CONDB, a neural method for nonlinear shared-preference structure across users (Wang et al., 4 Feb 2025). The setting assumes $y_t\in\{0,1\}$1 users partitioned into $y_t\in\{0,1\}$2 latent clusters, with all users in the same cluster sharing a common reward function. CONDB maintains a graph over users, pools pairwise observations within estimated clusters, retrains a cluster-level neural model from pooled data, and updates the graph by comparing user-specific neural parameter estimates. The deletion rule removes an edge when the user-specific models are statistically distinguishable: $y_t\in\{0,1\}$3 The core significance is collaborative sample sharing when each user is data-starved (Wang et al., 4 Feb 2025).

A second extension is active query design. "Active Human Feedback Collection via Neural Contextual Dueling Bandits" frames the problem as choosing not only which arms to compare but also which context to query (Verma et al., 16 Apr 2025). Neural-ADB maintains an NTK-style Gram matrix

$y_t\in\{0,1\}$4

with

$y_t\in\{0,1\}$5

selects the context by maximizing pairwise uncertainty,

$y_t\in\{0,1\}$6

then applies neural UCB or neural TS for the second arm (Verma et al., 16 Apr 2025). The performance target is not only cumulative regret but the worst sub-optimality gap of the final learned policy

$y_t\in\{0,1\}$7

This shifts the emphasis from online action quality to active preference-dataset construction (Verma et al., 16 Apr 2025).

A third extension addresses delayed and partially missing preference labels. "Linear and Neural Dueling Bandits with Delayed Feedback" studies stochastic delays $y_t\in\{0,1\}$8, a hard observation window $y_t\in\{0,1\}$9, and observation probability

$$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$0

(Wang et al., 26 May 2026). The key claim is that naive training on arrived comparisons is biased in a dueling bandit because the estimator is a logistic MLE without closed form. NDB-DF corrects this using an inverse-probability-weighted loss inserted directly into the cross-entropy objective: $$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$1 where

$$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$2

The first arm remains greedy, the second is optimistic under an NTK-style uncertainty bonus, and the confidence bound acquires explicit dependence on $$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$3 and $$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$4 (Wang et al., 26 May 2026).

A fourth extension is computationally motivated. NVLDB introduces variance-aware weighting and last-layer exploration in order to reduce the cost of full-parameter uncertainty tracking (Oh et al., 2 Jun 2025). The paper states that this yields much lower computational cost than prior neural dueling bandits and reports a substantial runtime difference over 2000 rounds between NVLDB-UCB-ASYM and NDB (Oh et al., 2 Jun 2025). A plausible implication is that shallow-exploration variants may be the most practical route when full NTK-feature covariance becomes the computational bottleneck.

5. Theoretical guarantees and complexity measures

The foundational regret guarantee for neural dueling bandits is sublinear in horizon under wide-network NTK assumptions. For NDB-UCB and NDB-TS, the regret order is

$$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$5

where $$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$6 is the effective dimension built from pairwise gradient-difference features (Verma et al., 2024). The paper emphasizes that $$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$7 is larger than in standard neural contextual bandits because comparisons are statistically weaker than direct reward feedback (Verma et al., 2024).

CONDB modifies this rate by separating cluster-identification cost from post-clustering learning cost. Its theorem gives an expected regret of the form

$$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$8

which the paper interprets as a clustering cost plus a post-clustering learning cost that depends on the number of clusters $$\mathbb{P}(y_t=1\mid x_{t,1},x_{t,2})=\mu\!\big(f(x_{t,1})-f(x_{t,2})\big), \qquad \mu(z)=\frac{1}{1+e^{-z}},$$9, not the number of users $$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$0 (Wang et al., 4 Feb 2025). This is the central collaborative benefit claimed by the clustering framework.

Neural-ADB changes the target metric from online regret to final-policy quality. For both UCB and TS variants, the worst contextwise policy gap decreases at rate

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$1

with the theorem stated through a Mahalanobis-norm bound and the pairwise confidence interval

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$2

for UCB, and the corresponding log-factor version for TS (Verma et al., 16 Apr 2025). The paper also includes cumulative regret corollaries of order

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$3

but explicitly treats them as secondary to the active-learning guarantee (Verma et al., 16 Apr 2025).

NDB-DF incorporates delay parameters into the confidence radius and regret. Its neural theorem states

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$4

simplified in the paper as

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$5

(Wang et al., 26 May 2026). This makes the degradation from delayed or censored comparisons explicit.

NVLDB departs from the NTK-all-parameters template and proves a variance-aware regret bound in terms of contextual dimension $$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$6 and the variance of comparisons: $$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$7 and summarizes the dominant rate as

$$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$8

for sufficiently wide networks (Oh et al., 2 Jun 2025). This is a distinct theoretical emphasis: the bound is variance-adaptive, and the extra finite-width term is interpreted as bias induced by shallow exploration.

Across these papers, several assumptions recur: a logistic or otherwise smooth monotone link; a derivative lower bound such as $$\mathbb{P}(x_{t,1}\succ x_{t,2}) = \frac{\exp f(x_{t,1})}{\exp f(x_{t,1})+\exp f(x_{t,2})}.$$9; bounded rewards or utilities; sufficiently wide networks; and NTK nondegeneracy conditions (Verma et al., 2024, Wang et al., 4 Feb 2025, Verma et al., 16 Apr 2025, Wang et al., 26 May 2026, Oh et al., 2 Jun 2025). A common misconception is that neural dueling-bandit theory analyzes arbitrary deep learning dynamics. The published theory instead overwhelmingly relies on wide-network linearization around initialization and confidence sets in gradient-feature space.

6. Empirical evidence, interpretation, and limitations

The empirical evidence consistently tests nonlinear reward functions, since the stated purpose of neural dueling bandits is to move beyond linear utility models. The foundational NDB paper evaluates synthetic reward functions such as

$x_t^\star=\arg\max_{x\in X_t} f(x),$0

using a network with 2 hidden layers, width $x_t^\star=\arg\max_{x\in X_t} f(x),$1, $x_t^\star=\arg\max_{x\in X_t} f(x),$2, $x_t^\star=\arg\max_{x\in X_t} f(x),$3, default $x_t^\star=\arg\max_{x\in X_t} f(x),$4, $x_t^\star=\arg\max_{x\in X_t} f(x),$5, and fixed exploration parameter $x_t^\star=\arg\max_{x\in X_t} f(x),$6 (Verma et al., 2024). It reports that NDB-UCB and NDB-TS outperform linear baselines on nonlinear tasks, while regret increases with the number of arms $x_t^\star=\arg\max_{x\in X_t} f(x),$7 and feature dimension $x_t^\star=\arg\max_{x\in X_t} f(x),$8 (Verma et al., 2024).

CONDB reports synthetic and MovieLens experiments with $x_t^\star=\arg\max_{x\in X_t} f(x),$9 users, $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$0 arms per round, feature dimension $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$1, and experiments with $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$2 and $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$3 clusters, over three independent trials reported as mean $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$4 standard error (Wang et al., 4 Feb 2025). The major nonlinear synthetic reward is

$$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$5

The paper states that CONDB achieves significantly smaller cumulative regret than NDB_IND on both synthetic and MovieLens experiments, and that performance improves when there are fewer clusters (Wang et al., 4 Feb 2025). This empirically supports the collaborative premise that pooling pairwise comparison data across similar users is beneficial when cluster sharing is real.

Neural-ADB reports only synthetic experiments. Features are sampled as

$$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$6

with nonlinear latent rewards

$$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$7

using a neural network with 2 hidden layers, width $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$8, $$r_t^a \doteq f(x_t^\star)-\frac{f(x_{t,1})+f(x_{t,2})}{2},$$9, $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$0, $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$1, $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$2, fixed $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$3, and retraining every 20 rounds for 50 gradient steps (Verma et al., 16 Apr 2025). The paper reports worst suboptimality gap, MAE / average suboptimality gap, and average regret, and states that Neural-ADB (UCB) outperforms baselines on most nonlinear synthetic functions in worst-gap and performs best across all tested synthetic functions in MAE and average regret (Verma et al., 16 Apr 2025).

NDB-DF reports both nonlinear synthetic environments and a real application in automated prompt optimization for LLMs. The nonlinear synthetic rewards are quadratic and cubic: $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$4 The real-world setup uses 29 instruction-induction tasks from InstructZero, candidate pool size $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$5, and reward defined as validation accuracy on $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$6 held-out examples (Wang et al., 26 May 2026). The paper reports that NDB-DF converges faster and achieves lower final regret than NDB-Ignore and NDB-Heuristic, and significantly outperforms both baselines in prompt optimization (Wang et al., 26 May 2026).

NVLDB evaluates on synthetic Square, Cosine, and Quadratic utilities, UCI-derived tasks Statlog, Magic, and Covertype, and compares against VALDB, ColSTIM, NDB-UCB, and NDB-TS (Oh et al., 2 Jun 2025). The main synthetic setting uses $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$7, $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$8, $$r_t^w \doteq f(x_t^\star)-\max\{f(x_{t,1}),f(x_{t,2})\},$$9, and 20 random seeds. The reported implementation uses a fully connected ReLU network, $c_t$00 hidden layers, width $c_t$01, $c_t$02, confidence coefficient fixed to $c_t$03, $c_t$04 gradient steps per round, $c_t$05, Adam optimizer, and learning rate $c_t$06 (Oh et al., 2 Jun 2025). The paper states that the proposed methods, especially Neural-$c_t$07-aware-UCB and Neural-$c_t$08-aware-TS, consistently outperform baselines on synthetic nonlinear tasks and UCI-derived tasks, and that over 2000 rounds NVLDB-UCB-ASYM requires $c_t$09 minutes while NDB requires $c_t$10 minutes (Oh et al., 2 Jun 2025).

Several limitations are repeated across the literature. The theory commonly assumes sufficiently wide networks, NTK nondegeneracy, bounded utility, and a lower-bounded link derivative $c_t$11, which may be restrictive if reward differences saturate the logistic link (Verma et al., 2024, Verma et al., 16 Apr 2025, Wang et al., 26 May 2026, Oh et al., 2 Jun 2025). The foundational NDB paper validates only on synthetic data and explicitly notes practical deviations from theory, such as using $c_t$12 rather than $c_t$13 in experiments (Verma et al., 2024). CONDB does not provide detailed computational complexity analysis, retraining schedule, minibatch strategy, or memory-buffer implementation details in the provided text (Wang et al., 4 Feb 2025). Neural-ADB does not validate on real human-comparison data despite its RLHF motivation (Verma et al., 16 Apr 2025). NDB-DF assumes known $c_t$14 and a hard delay threshold $c_t$15 (Wang et al., 26 May 2026).

Within the broader dueling-bandit literature, these limitations reflect a more general distinction between utility-based and arbitrary-preference formulations. The survey emphasizes that best-arm and regret definitions are less straightforward in pairwise settings because preferences may be cyclic, and winner notions such as Condorcet, Copeland, Borda, and von Neumann can diverge (Bengs et al., 2018). Neural dueling bandits mostly avoid this issue by assuming a latent scalar utility and BTL-type comparisons. This suggests that the current literature is best understood as neural latent-utility dueling bandits rather than neural solutions to general preference tournaments.

In aggregate, the field defines a coherent research program. The foundational NDB work establishes the NTK-based template for learning nonlinear utility from pairwise feedback (Verma et al., 2024). CONDB shows that the same machinery can support online user clustering and collaborative regret reductions (Wang et al., 4 Feb 2025). Neural-ADB adapts it to active human-feedback collection and final-policy guarantees (Verma et al., 16 Apr 2025). NDB-DF addresses stochastic delayed feedback via inverse-probability-weighted logistic objectives (Wang et al., 26 May 2026). NVLDB trades full-parameter exploration for last-layer uncertainty and variance-aware weighting in order to improve computational efficiency (Oh et al., 2 Jun 2025). A plausible implication is that future work will continue to balance three competing objectives already visible in these papers: nonlinear expressivity, statistically principled exploration in pairwise-difference space, and computational tractability.