UCB-NOM: Nominal & Nonparametric UCB Methods

• UCB-NOM is an informal descriptor for UCB-style methods that incorporate nominal-model awareness or nonparametric optimism, as seen in ARBMLE and BaNk-UCB.
• It distinguishes approaches where optimism is applied via model selection with a nominal anchor versus direct action indexing using local regression and confidence bonuses.
• Understanding UCB-NOM aids in clarifying the shared UCB spirit—optimism in the face of uncertainty—across adaptive control and nonparametric contextual bandit algorithms.

UCB-NOM is not an algorithm name that appears explicitly in the cited arXiv literature. Several nearby papers state that they do not mention a method called “UCB-NOM,” while nonetheless providing constructions that are conceptually adjacent to such a label. A plausible reading is that the term denotes a UCB-style method with an additional qualifier such as a nominal-model anchor or a nonparametric optimistic model. Under that reading, the closest explicit exemplars are the Augmented RBMLE-UCB formulation for adaptive linear-quadratic control and the BaNk-UCB method for batched nonparametric contextual bandits (Mete et al., 2022, Arya, 15 May 2025).

1. Terminological status and nearest explicit uses

The cited papers repeatedly clarify that they do not introduce a method literally called “UCB-NOM.” This is stated for the adaptive-control construction ARBMLE (Mete et al., 2022), the nonparametric contextual-bandit method BaNk-UCB (Arya, 15 May 2025), the divergence-based stochastic-bandit method Hellinger-UCB (Yang et al., 2024), the best-arm-identification method TTUCB (Jourdan et al., 2022), the PSR method PSR-UCB (Huang et al., 2023), and the replicable bandit constructions RepUCB and RepLinUCB (Deb et al., 21 Apr 2026).

This suggests that “UCB-NOM” is best treated as an informal or context-dependent label rather than a canonical algorithm title. In the supplied literature, two interpretations are especially close. One is a nominal-model-aware optimistic selector, where the learner stays near a nominal fit while optimizing an optimistic criterion under a confidence constraint; this is the role played by ARBMLE in adaptive control (Mete et al., 2022). The other is a nonparametric optimistic method, where optimism is built from local geometry and explicit confidence bonuses rather than from a finite-dimensional parametric model; this is the role played by BaNk-UCB in batched contextual bandits (Arya, 15 May 2025).

Interpretation of “UCB-NOM”	Closest explicit construction	Signature rule
Nominal-model-aware optimism	ARBMLE	$$\theta_t \in \arg\min_{\theta\in\mathcal S\cap\mathcal C_t(\delta)} \big(V_t(\theta)+\alpha(t)J^\star(\theta)\big)$$
Nonparametric optimistic method	BaNk-UCB	$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$
Heuristic optimism from learned uncertainty	Q-ensemble UCB	$$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$

2. Shared UCB structure across nearby interpretations

Despite their domain differences, the nearby constructions share the classical UCB principle of optimism in the face of uncertainty. What changes is the object to which optimism is applied.

In nonparametric contextual bandits, BaNk-UCB constructs an arm index from three terms: a local $k$-NN regression estimate, a stochastic confidence bonus, and a Lipschitz bias correction,

$$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$

with action rule

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t).$

The bonus is explicit, and the construction is tied to local geometry through the $k$-NN radius (Arya, 15 May 2025).

In adaptive LQ control, ARBMLE does not use a “mean plus bonus” index. Instead, it combines a nominal-data-fit term with an optimism bias toward low-cost models, while constraining the parameter to lie in a high-probability confidence ellipsoid:

$$\theta_t \in \arg\min_{\theta\in\mathcal S\cap\mathcal C_t(\delta)} \Big(V_t(\theta)+\alpha(t)J^\star(\theta)\Big).$$

Here, optimism is implemented in model selection rather than directly in arm indexing (Mete et al., 2022).

In PSR learning, optimism appears as an explicit trajectory bonus that upper bounds total variation error between the estimated and true model:

$\hat b^k(\tau_H) = \min\left\{ \alpha \sqrt{ \sum_{h=0}^{H-1} \left\| \bar{\hat\psi}^k(\tau_h) \right\|^2_{(\hat U_h^k)^{-1} } ,\,1 \right\}.$

The planning step then maximizes the optimistic value under the learned PSR model (Huang et al., 2023).

A plausible implication is that any use of “UCB-NOM” should be interpreted through three questions: what the nominal object is, what quantity represents uncertainty, and whether optimism is applied in action selection, model selection, or planning.

3. The nominal-model interpretation: ARBMLE in adaptive LQ control

The sharpest “nominal” interpretation in the cited literature comes from adaptive linear-quadratic control. The underlying system is

$x_{t+1}=A x_t+B u_t+w_{t+1},$

or equivalently

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$0

The control objective is regret relative to the optimal average-cost controller for the true parameter,

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$1

with

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$2

The known-model solution is certainty-equivalent LQG, based on the Riccati equation and feedback law $a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$3 (Mete et al., 2022).

The paper’s conceptual point is that RBMLE and OFULQ/UCB are two optimistic model-selection principles in different forms. OFULQ uses a constrained optimistic rule,

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$4

whereas RBMLE uses an unconstrained penalty form,

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$5

ARBMLE combines both:

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$6

with

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$7

The confidence region is the ellipsoid

$a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$8

centered at the regularized least-squares estimate (Mete et al., 2022).

This is the clearest supplied example of a confidence-constrained nominal optimistic estimator. The paper explicitly distinguishes it from OFULQ by stating that OFULQ “treats all models within the UCB-ball equally and only assesses them by their cost,” whereas ARBMLE “prefers models that are closer to the Least Squares Estimate.” It also notes empirically that $a_t=\arg\max_{a\in\mathcal A}\hat f_{a,t}(X_t)$9 tends to lie on the boundary of the confidence ball while $$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$0 is often in the interior (Mete et al., 2022).

The theoretical guarantee is a high-probability regret bound of order

$$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$1

which the paper presents as matching the best known order for OFULQ. A common misconception is that this establishes the same result for the original unconstrained RBMLE; the paper states explicitly that the proof is for the augmented version, and that the confidence-set constraint is what enables the analysis (Mete et al., 2022).

4. The nonparametric optimistic interpretation: BaNk-UCB

The strongest “nonparametric optimistic method” interpretation in the cited literature is BaNk-UCB, developed for batched nonparametric contextual bandits. The setting is

$$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$2

where contexts are i.i.d., the arm-specific mean reward functions are unknown, and no linear or generalized linear structure is assumed. Instead, the paper assumes Lipschitz smoothness,

$$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$3

and a margin condition,

$$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$4

Feedback is batched, so the policy within a batch can depend only on data revealed at the end of the previous batch (Arya, 15 May 2025).

For a context $$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$5 and arm $$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$6, the method forms a nearest-neighbor set $$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$7 among previously observed contexts where arm $$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$8 was played, and defines the local radius

$$a_t \in \arg\max_a \{\tilde{\mu}(s_t,a)+\lambda \tilde{\sigma}(s_t,a)\}$$9

The neighborhood size is adaptive and chosen through a bias–variance condition. If the nearest observed context for arm $k$0 is too far from $k$1, the arm’s UCB is set to $k$2, forcing exploration. When estimation is feasible, the index is

$k$3

with confidence bonus

$k$4

The action rule is the optimistic one

$k$5

The index therefore combines local regression, a stochastic confidence term, and a deterministic geometric bias correction (Arya, 15 May 2025).

The analysis proceeds through a uniform concentration event under which the estimate is optimistic and controlled:

$k$6

The method is then shown to satisfy a regret bound that the paper describes as minimax-optimal up to logs under the proposed batch schedule, with matching lower bound up to logarithmic factors (Arya, 15 May 2025).

This construction is a direct fit for an interpretation of “UCB-NOM” as nonparametric optimism. It is fully nonparametric, the uncertainty bonus is explicit rather than heuristic, and the theoretical analysis is carried out in terms of smoothness, margin, and local sample complexity. At the same time, the paper notes an important qualification: although the algorithm is described as adapting to local geometry, the formal regret rate is parameterized by the ambient dimension $k$7 rather than an intrinsic dimension (Arya, 15 May 2025).

5. Adjacent UCB variants that should not be conflated with UCB-NOM

Several additional UCB constructions in the cited literature are closely related in spirit but materially different in mechanism.

In deep reinforcement learning, “UCB Exploration via Q-Ensembles” uses a multi-head Q-network and defines the optimistic action rule

$k$8

where $k$9 and $$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$0 are the empirical mean and standard deviation across Q-heads. The paper is explicit that this “confidence” term is heuristic rather than statistically calibrated: there is no explicit count term, no formal confidence coverage guarantee, and no per-state-action visitation accounting. It also states that, if one is comparing the method to something called “UCB-NOM,” the key distinction is that confidence is constructed from cross-head dispersion of Q-values and used only at decision time (Chen et al., 2017).

In stochastic bandits, Hellinger-UCB is a divergence-based index policy rather than a nominal-model or nonparametric local-regression method. Its arm index is

$$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$1

built from a squared Hellinger ball in a one-parameter exponential family. The paper explicitly states that it does not introduce UCB-NOM and does not discuss normalized objectives, nonparametric modeling, or novelty/outlier modeling under that name (Yang et al., 2024).

In best-arm identification, TTUCB is a Top Two construction rather than a nominal or nonparametric UCB. Its leader is

$$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$2

its challenger is chosen by a transportation-cost criterion, and its main contribution is the first non-asymptotic upper bound on the expected sample complexity of a Top Two algorithm. The paper states directly that the method is called TTUCB, not UCB-NOM (Jourdan et al., 2022).

In large-scale pure exploration beyond sub-Gaussianity, the meta-UCB framework studies indices of the form

$$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$3

with a recommendation rule based on the largest sample size. The paper does not introduce UCB-NOM, but it is directly relevant if the label is intended to mean UCB under bounded-$$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$4th-moment or other non-sub-Gaussian assumptions (Li et al., 27 Nov 2025).

In predictive-state-representation learning, PSR-UCB uses a trajectory bonus based on normalized predictive features,

$$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$5

and proves that this bonus upper bounds total variation distance between the estimated and true model. The paper again states that the exact term UCB-NOM does not appear (Huang et al., 2023).

In replicable bandits, RepUCB and RepLinUCB are optimistic algorithms designed under a replicability constraint rather than under nominal anchoring or nonparametric geometry. The defining idea is to stabilize UCB updates through batching and replicable estimators, not to introduce a new “NOM” principle (Deb et al., 21 Apr 2026).

6. Distinguishing criteria, misconceptions, and scope

The supplied literature distinguishes nearby “UCB-NOM”-like ideas along three main axes.

The first axis is the source of uncertainty. In ARBMLE, uncertainty is represented by a confidence ellipsoid around a regularized least-squares estimate (Mete et al., 2022). In BaNk-UCB, uncertainty is local and nonparametric, encoded through $$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$6-NN sample size and neighborhood radius (Arya, 15 May 2025). In Q-ensemble UCB, uncertainty is ensemble disagreement (Chen et al., 2017). In Hellinger-UCB, it is a divergence ball in model space (Yang et al., 2024). In PSR-UCB, it is an elliptical bonus over predictive features that upper bounds total variation error (Huang et al., 2023).

The second axis is the role of the nominal estimate. ARBMLE explicitly includes a nominal-data-fit term $$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$7 inside the optimistic objective, which is why the paper describes it as a confidence-constrained nominal optimistic estimator (Mete et al., 2022). By contrast, BaNk-UCB, Hellinger-UCB, and Q-ensemble UCB do not optimize a separate nominal-anchor term; they use optimistic indices directly in action choice (Arya, 15 May 2025, Yang et al., 2024, Chen et al., 2017).

The third axis is the location of optimism in the algorithmic pipeline. In Q-ensemble UCB, optimism acts only at decision time and is not inserted into the Bellman target or learned reward for the core UCB method (Chen et al., 2017). In BaNk-UCB, optimism is the arm index itself (Arya, 15 May 2025). In ARBMLE, optimism enters model selection through a cost-biased objective under confidence constraints (Mete et al., 2022). In PSR-UCB, optimism enters planning through a trajectory bonus (Huang et al., 2023).

A recurring misconception is that every UCB quantity is a genuine confidence bound. The Q-ensemble RL paper explicitly rejects that interpretation for its own bonus: the term $$\hat f_{a,t}(x) = \frac{1}{k_{a,t}(x)}\sum_{s\in N_{t_{m-1},k_{a,t}(x)}(x,a)} Y_s \;+\; \xi_{a,t}(x) \;+\; L d_{a,t}(x),$$8 is not a true confidence interval in the statistical sense (Chen et al., 2017). Another misconception is that all UCB variants are fundamentally parametric. BaNk-UCB is fully nonparametric and uses local geometry rather than linear or generalized linear structure (Arya, 15 May 2025). A third misconception is that nominal anchoring is universal across optimism-based methods; in the supplied literature, that feature is specific to the ARBMLE-style adaptive-control formulation rather than to UCB methods in general (Mete et al., 2022).

Taken together, the cited arXiv literature supports a narrow conclusion. “UCB-NOM” is not an established standalone method name in these papers. The label is most coherently understood through context: if it refers to nominal-model-aware optimism, the closest explicit construction is ARBMLE; if it refers to a nonparametric optimistic method, the closest explicit construction is BaNk-UCB; and if it is used more loosely, it risks conflating materially different UCB mechanisms whose uncertainty sources, update locations, and theoretical guarantees are not interchangeable (Mete et al., 2022, Arya, 15 May 2025).