Net-SGD-UCB: Decentralized Contextual Bandit

• The paper presents Net-SGD-UCB, a decentralized algorithm that leverages online momentum SGD to estimate both shared and node-specific parameters efficiently.
• It decomposes contextual information into common and local components and uses an adaptive weight matrix to share only necessary information across nodes.
• Empirical and theoretical results demonstrate that Net-SGD-UCB scales sublinearly with network size and maintains robust regret guarantees in high-dimensional, noisy environments.

Searching arXiv for Net-SGD-UCB and closely related neural/UCB bandit references. Net-SGD-UCB is a decentralized, network-aware contextual linear bandit algorithm for multi-agent settings in which reward structure contains both a shared component and node-specific heterogeneity. It is introduced together with NetLinUCB in "Decentralized Contextual Bandits with Network Adaptivity" (Deng et al., 19 Aug 2025). The method is designed for synchronous learning over a network of $N$ nodes that face the same arm set but observe locally varying contexts and rewards. Its defining combination is a shared/local parameter decomposition, adaptive cross-node information sharing through a dynamically updated weight matrix, online momentum SGD for parameter estimation, a diagonal gradient-based uncertainty matrix, and a UCB action rule. In the formulation of the paper, this construction aims to preserve the statistical benefit of sharing common structure while avoiding the matrix inversions and communication burden of full centralization, especially in high-dimensional, streaming, and high-variance regimes (Deng et al., 19 Aug 2025).

1. Problem formulation

The algorithm is posed in a decentralized contextual linear bandit with $N$ nodes operating over $T$ rounds. At each round $t$, node $i \in [N]$ observes a context vector $\mathbf{x}_{i,t}$, chooses an arm $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$, receives the reward of the chosen arm, and updates its local state (Deng et al., 19 Aug 2025).

A central modeling device is the decomposition of context into common and node-specific coordinates: $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$ For each arm $a^{(k)}$, node $i$ has parameter

$N$0

where $N$1 is shared across all nodes and $N$2 is local (Deng et al., 19 Aug 2025).

The reward model is linear: $N$3 with $N$4. The paper also states that, in the SGD-UCB analysis, the noise is treated as sub-Gaussian with variance $N$5 (Deng et al., 19 Aug 2025). The expected reward therefore satisfies

$N$6

The network cumulative regret is defined by

$N$7

where

$N$8

The boundedness assumptions are

$N$9

for all $T$0 (Deng et al., 19 Aug 2025).

This formulation sits between two extremes identified in the paper: disjoint local learners that relearn shared structure independently, and fully centralized shared learners that pool all information but pay higher computational and communication costs. Net-SGD-UCB is the first-order alternative within that design space (Deng et al., 19 Aug 2025).

2. Shared and local structure over an adaptive network

Net-SGD-UCB assumes an initially fully connected communication network, justified in the paper by $T$1, so that every pair of nodes shares at least some common feature structure (Deng et al., 19 Aug 2025). Effective communication, however, is not fixed; it is governed by an adaptive arm-specific weight matrix

$T$2

where $T$3 measures the directional influence from node $T$4 to node $T$5 for arm $T$6 (Deng et al., 19 Aug 2025).

The matrix is initialized as

$T$7

Its update uses two signals. The first is arm-selection similarity,

$T$8

where $T$9 is the number of times node $t$0 selected arm $t$1. The second is common-context cosine similarity,

$t$2

The paper states that these are combined and normalized to form $t$3, after which exponential smoothing is applied: $t$4 The exact closed-form combination rule before normalization is not specified beyond the pseudocode-level description (Deng et al., 19 Aug 2025).

The consequence of this structure is that only the common component is communicated and aggregated. After local updates, node $t$5 forms its shared estimate for arm $t$6 as

$t$7

while the node-specific component remains local: $t$8 The paper explicitly states that Net-SGD-UCB communicates only shared parameter estimates, not raw data and not full local statistics (Deng et al., 19 Aug 2025).

This decomposition operationalizes a hybrid statistical assumption: all nodes benefit from a common arm-dependent signal, but each node retains personalized coordinates. A plausible implication is that the method is intended for environments in which full pooling would oversmooth node-specific effects, while purely local learning would be redundant on the common subspace.

3. SGD-based estimation and uncertainty construction

The defining algorithmic departure from matrix-based UCB methods is that Net-SGD-UCB replaces ridge-regression estimation with online stochastic gradient descent with momentum (Deng et al., 19 Aug 2025). For node $t$9, arm $i \in [N]$0, and round $i \in [N]$1, the paper uses the squared-error loss

$i \in [N]$2

with linear prediction

$i \in [N]$3

The gradient is

$i \in [N]$4

The momentum update is

$i \in [N]$5

followed by the parameter update

$i \in [N]$6

where $i \in [N]$7 is the momentum parameter and $i \in [N]$8 is the learning rate (Deng et al., 19 Aug 2025). The update is online, one observation at a time; the paper does not mention minibatching.

Uncertainty is tracked through a gradient-based matrix $i \in [N]$9. The paper first defines

$\mathbf{x}_{i,t}$0

but then states that only diagonal entries are kept to reduce complexity (Deng et al., 19 Aug 2025). The online EMA update for coordinate $\mathbf{x}_{i,t}$1 is

$\mathbf{x}_{i,t}$2

or, in algorithm form,

$\mathbf{x}_{i,t}$3

with $\mathbf{x}_{i,t}$4 (Deng et al., 19 Aug 2025).

The diagonal accumulator is partitioned into shared and local blocks. The shared block $\mathbf{x}_{i,t}$5 participates in cross-node uncertainty aggregation, whereas the local block $\mathbf{x}_{i,t}$6 remains node-specific. This is the uncertainty analogue of the shared/local parameter split.

4. UCB index and decision rule

For node $\mathbf{x}_{i,t}$7, arm $\mathbf{x}_{i,t}$8, and round $\mathbf{x}_{i,t}$9, the Net-SGD-UCB index is

$a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$0

The selected arm is

$a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$1

(Deng et al., 19 Aug 2025).

The first term is the predicted reward using the aggregated common estimate and the local estimate. The second is the optimism bonus. The common-component contribution uses squared weights $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$2, reflecting the variance-style aggregation stated in the paper, while the local-component contribution depends only on node $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$3's own uncertainty state (Deng et al., 19 Aug 2025).

This index mirrors the structure of NetLinUCB but substitutes gradient-based diagonal uncertainty for ridge covariance. NetLinUCB uses

$a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$4

whereas Net-SGD-UCB replaces $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$5 by $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$6 and closed-form ridge updates by first-order SGD (Deng et al., 19 Aug 2025).

A concise operational summary of one round is therefore: update $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$7; aggregate the shared estimate; compute per-arm UCB scores; choose the maximizing arm; observe the reward; update gradient, momentum, parameter, and diagonal uncertainty accumulator. This yields a fully decentralized procedure in which nodes share only summaries of the common component (Deng et al., 19 Aug 2025).

5. Regret guarantees and scaling properties

The paper gives both single-node and network-level guarantees. At the level of uncertainty accumulation, Proposition 6 states

$a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$8

while also noting that the notation is imprecise because the left-hand side is per-time and the right-hand side is cumulative; the intended object in the proof and subsequent use is $a_{i,t} \in \mathcal{A}=\{a^{(1)},\dots,a^{(K)}\}$9 (Deng et al., 19 Aug 2025).

With probability at least $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$0, Proposition 7 gives the single-node regret bound

$$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$1

with $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$2 (Deng et al., 19 Aug 2025). The dependence is explicit in dimension $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$3, horizon $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$4, noise $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$5, momentum $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$6, and EMA smoothing $$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$7.

Theorem 4 decomposes the network-wide regret into shared and node-specific terms. For the shared component,

$$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$8

and for the node-specific component,

$$\mathbf{x}_{i,t} = [\mathbf{x}_{i,c,t},\ \mathbf{x}_{i,s,t}], \qquad \mathbf{x}_{i,c,t} \in \mathbb{R}^{d_c}, \qquad \mathbf{x}_{i,s,t} \in \mathbb{R}^{d_{i,s}}, \qquad d_i=d_c+d_{i,s}.$$9

The total regret is the sum of these two terms (Deng et al., 19 Aug 2025).

The scaling claim emphasized in the paper is that learning complexity associated with the shared structure is reduced from $a^{(k)}$0 to sublinear $a^{(k)}$1. Concretely, the paper contrasts a disjoint baseline with shared-component cost

$a^{(k)}$2

against Net-SGD-UCB's shared-component term

$a^{(k)}$3

(Deng et al., 19 Aug 2025). This is the main statistical argument for adaptive information sharing without full centralization.

The computational and communication profile is likewise central to the method's identity:

Quantity	NetLinUCB	Net-SGD-UCB
Computation	$a^{(k)}$4	$a^{(k)}$5
Memory	$a^{(k)}$6	$a^{(k)}$7
Communication	$a^{(k)}$8	$a^{(k)}$9

These complexity statements explain why the paper presents Net-SGD-UCB as the scalable first-order option, especially when $i$0 is large or matrix inversion is undesirable (Deng et al., 19 Aug 2025).

6. Empirical behavior, interpretation, and relation to adjacent UCB methods

The empirical evaluation in (Deng et al., 19 Aug 2025) is conducted on synthetic networked pricing simulations. The main metrics are average per-round regret over time,

$i$1

and average per-round per-node regret,

$i$2

The baselines are Disjoint LinUCB, Shared LinUCB, NetLinUCB, and Net-SGD-UCB (Deng et al., 19 Aug 2025).

Several findings are reported. First, the per-round regret $i$3 decreases toward zero, which the paper interprets as sublinear regret. Second, both network-aware methods, including Net-SGD-UCB, achieve per-node regret comparable to Shared LinUCB, supporting sublinear scaling in network size $i$4. Third, supplementary Figure 1(b) introduces a large contextual outlier, and Net-SGD-UCB remains robust due to adaptive variance tracking. Fourth, supplementary Figure 1(c) shows that when the number of actions increases, Net-SGD-UCB performs best. The paper repeatedly emphasizes that Net-SGD-UCB is stronger in noisier settings, whereas NetLinUCB is stronger when reward gaps are small and heterogeneity is fine-grained (Deng et al., 19 Aug 2025).

Supplementary Table 2 reports reduction in confidence radius relative to Disjoint LinUCB. For Net-SGD-UCB the reported reductions are $i$5 for $i$6, $i$7 for $i$8, and $i$9 for $N$00. For NetLinUCB the corresponding values are $N$01, $N$02, and $N$03 (Deng et al., 19 Aug 2025). The paper interprets these numbers as showing that NetLinUCB benefits more from denser connectivity, while Net-SGD-UCB is more stable and consistent across connectivity levels.

Within the broader UCB literature, Net-SGD-UCB is structurally closer to first-order, linear, decentralized optimism than to overparameterized neural contextual bandits. "Neural Contextual Bandits with UCB-based Exploration" (Zhou et al., 2019) uses a deep neural predictor and a confidence term built from dynamic gradient features $N$04, whereas Net-SGD-UCB uses a contextual linear reward model with a diagonal uncertainty matrix $N$05 built from past squared gradients (Zhou et al., 2019). "Neural Bandit with Arm Group Graph" (Qi et al., 2022) extends neural-gradient UCB to graph-structured arm groups, but there the graph indexes correlations among arm groups, not communication among decentralized agents, and the reward model includes a GNN representation stage before the UCB score is formed (Qi et al., 2022). In that sense, Net-SGD-UCB belongs to a different branch of the literature: networked contextual linear bandits with adaptive inter-agent sharing.

The paper also makes clear what Net-SGD-UCB does not eliminate. Communication still scales as

$N$06

per round; the exact rule that combines arm-count similarity and context similarity into $N$07 is not fully specified beyond pseudocode; and several hyperparameters require tuning, namely $N$08, $N$09, $N$10, $N$11, and $N$12 (Deng et al., 19 Aug 2025). The theoretical presentation is also described in the source material as informal in some places, especially around cumulative versus per-step notation. These caveats delimit the method's present scope while clarifying its role: a scalable decentralized UCB algorithm that preserves shared/local structure and replaces matrix-based confidence estimation with momentum-SGD updates and diagonal EMA uncertainty (Deng et al., 19 Aug 2025).