---
title: 'Gluon-MVR: Momentum Variance Reduction'
url: https://www.emergentmind.com/topics/gluon-mvr
type: topic
---

# Gluon-MVR: Momentum Variance Reduction

Searching arXiv for recent papers on Gluon-MVR, Gluon, Muon, and related optimizer variants.
Gluon-MVR denotes a family of momentum variance reduction methods built on the Gluon optimization framework, a layer-wise non-Euclidean linear minimization oracle (LMO) scheme that captures Muon, Scion, and related trust-region-style optimizers as special cases. In the non-convex stochastic setting, Gluon-MVR replaces vanilla momentum with control-variate updates based on same-sample gradient differences at consecutive iterates, while retaining Gluon’s layer-wise norm geometry and LMO step. In distributed and federated settings, the same principle is combined with SARAH-style recursive estimators, unbiased or contraction compressors, and error feedback, yielding communication-efficient variants with comparable communication complexity under weaker conditions when \(L_i^1 \neq 0\) [2512.16598][2604.10689].

## 1. Gluon framework and optimization setting

Gluon-MVR inherits its ambient problem from Gluon. The objective is stochastic optimization over a finite-dimensional vector space,
\[
\min_{X \in \mathcal{S}} \left\{ f(X) := \mathbb{E}_{\xi \sim \mathcal{D}}\big[f_\xi(X)\big] \right\},
\]
or, in the federated formulation,
\[
\min_{X \in \mathcal{S}} \; f(X) := \frac{1}{n}\sum_{\tau=1}^n f^\tau(X), \quad 
f^\tau(X):= \mathbb{E}_{\xi_\tau \sim D_\tau}\big[f_{\xi_\tau}(X)\big].
\]
The parameter is partitioned layer-wise as \(X=[X_1,\ldots,X_p]\), with \(X_i \in \mathcal{S}_i\). This block structure is central: Gluon does not assume a single Euclidean geometry, but instead equips each layer with its own norm \(\|\cdot\|_{(i)}\) and dual norm \(\|\cdot\|_{(i)\star}\) [2512.16598].

The defining regularity condition is the layer-wise \((L_i^0,L_i^1)\)-smoothness assumption,
\[
\|\nabla_i f(X) - \nabla_i f(Y)\|_{(i)\star}
\leq \left(L^0_i + L^1_i \|\nabla_i f(X)\|_{(i)\star}\right) \|X_i - Y_i\|_{(i)}.
\]
This formulation is more general than standard global \(L\)-smoothness and is meant to reflect the empirical observation that curvature and smoothness vary by layer and may scale with the local gradient norm. The analysis is supplemented by unbiased stochastic gradients with bounded variance, a norm-equivalence bound \(\|Z\|_{(i)\star} \le \rho \|Z\|_2\), and a Hessian-variance condition controlled by \(\delta_i\) [2512.16598].

The update itself is expressed through a trust-region LMO. For each block \(i\), Gluon defines
\[
\mathcal{B}_i^k := \left\{X_i \in \mathcal{S}_i : \|X_i - X_i^k\|_{(i)} \leq t_i \eta \right\},
\]
and computes
\[
X_i^{k+1} = \operatorname*{arg\,min}_{X_i \in \mathcal{B}_i^k}\, \langle M_i^k, X_i \rangle_{(i)}.
\]
Here \(M_i^k\) is the momentum-like search direction. By choosing the block partition and the norms appropriately, Gluon recovers Muon and Scion. Muon corresponds to spectral-norm trust-region LMOs on hidden-layer weight matrices, whereas Scion applies the spectral-norm LMO to all layers [2512.16598].

## 2. Momentum variance reduction and the three Gluon-MVR variants

The defining modification in Gluon-MVR is the replacement of vanilla momentum by same-sample gradient-difference corrections. In vanilla Gluon, the momentum is typically
\[
M_i^k = \beta M_i^{k-1} + \nabla_i f_{\xi^k}(X^k).
\]
Gluon-MVR inserts a STORM-style control variate that evaluates gradients at both \(X^k\) and \(X^{k-1}\) using the same sample \(\xi^k\). This reduces variance without introducing bias and does not require large batches or checkpoint gradients [2512.16598].

The first variant, Gluon-MVR-1, is the most direct substitution:
\[
M_i^k = \nabla_i f_{\xi^k}(X^k) + \beta\Big(M_i^{k-1} - \nabla_i f_{\xi^k}(X^{k-1})\Big).
\]
Its structure is minimal: the previous momentum is corrected by subtracting the stale stochastic gradient on the current sample and adding the current one. In the single-block spectral-norm case, this coincides with Muon-MVR. The paper also gives a decreasing-stepsize version with
\[
\beta^k = 1 - (k+1)^{-2/3}, \qquad t_i^k = t_i (k+1)^{-2/3},
\]
which is used to obtain theory-guided \( \tilde{\mathcal O}(K^{-1/3}) \) guarantees [2512.16598].

The second variant, Gluon-MVR-2, introduces an auxiliary recursive estimator \(g_i^k\),
\[
g_i^k = \nabla_i f_{\xi^k}(X^k) + (1-q)\Big(g_i^{k-1} - \nabla_i f_{\xi^k}(X^{k-1})\Big),
\]
followed by a smoothed momentum
\[
M_i^k = \beta M_i^{k-1} + (1-\beta) g_i^k.
\]
The parameter \(q \in (0,1]\) controls the strength of the variance-reduced correction. Smaller \(q\) increases the recursive contribution, whereas larger \(q\) makes the estimator more local [2512.16598].

The third variant, Gluon-MVR-3, combines the recursive estimator of Gluon-MVR-2 with an additional gradient-difference correction,
\[
M_i^k = \beta M_i^{k-1} + (1-\beta) g_i^k + \beta\Big(\nabla_i f_{\xi^k}(X^k) - \nabla_i f_{\xi^k}(X^{k-1})\Big).
\]
This formulation interpolates between the first two designs: it retains a recursively denoised estimator \(g_i^k\) while also injecting the explicit same-sample difference term into the momentum state [2512.16598].

Across all three variants, the LMO step remains unchanged. The algorithmic identity of Gluon-MVR is therefore not a new search geometry, but a new momentum estimator inside the pre-existing non-Euclidean trust-region geometry of Gluon.

## 3. Convergence theory and rate improvements

The principal theoretical claim associated with Gluon-MVR is that MVR improves the constant-stepsize non-convex rate from \(\mathcal O(K^{-1/4})\) for vanilla Gluon to \(\mathcal O(K^{-1/3})\) for all three MVR variants. The paper presents this as the optimal rate for single-sample stochastic non-convex optimization, citing lower-bound consistency. The gain is traced to tighter control of
\[
\mathbb{E}\big[\|M_i^k - \nabla_i f(X^k)\|_{(i)\star}\big],
\]
obtained by exploiting same-sample gradient differences, Hessian variance bounds, and the layer-wise smoothness structure [2512.16598].

For Gluon-MVR-1, the representative non-convex guarantee is stated in terms of
\[
\min_{k=0,\ldots,K-1}\sum_{i=1}^p t_i \,\mathbb{E}\big[\|\nabla_i f(X^k)\|_{(i)\star}\big].
\]
When \(L_i^1=0\), choosing
\[
\eta = K^{-2/3}, \qquad \alpha := 1-\beta = K^{-2/3}
\]
yields
\[
\min_{k}\sum_i t_i \,\mathbb{E}\big[\|\nabla_i f(X^k)\|_{(i)\star}\big] \le \mathcal{O}(K^{-1/3}).
\]
When \(L_i^1 \neq 0\), the same rate is recovered essentially under
\[
\eta \le \min_i \frac{1}{L_i^1 t_i},
\]
up to a small additive term of order \(\max_i L_i^1 t_i/K\). Gluon-MVR-2 and Gluon-MVR-3 admit analogous \(\mathcal O(K^{-1/3})\) statements under parameter choices such as \(\eta=q=K^{-2/3}\) and \(\alpha \ge K^{-1/3}\) for Gluon-MVR-2, and \(\eta=\alpha=K^{-2/3}\) with \(q \ge K^{-2/3}\) for Gluon-MVR-3 [2512.16598].

The star-convex case is treated separately through Muon-MVR. There the trust-region step is written as
\[
X^{k+1} = \arg\min_{X\in\mathcal{B}^k}\, \langle M^k, X-X^k \rangle,
\quad \mathcal{B}^k = \{X : \|X - X^k\| \le \eta\},
\]
with MVR update
\[
M^{k+1} = (1-\alpha)\Big(M^k - \nabla f_{\xi^{k+1}}(X^k)\Big) + \nabla f_{\xi^{k+1}}(X^{k+1}).
\]
Under star-convexity and Non-Euclidean smoothness, the resulting rate improves from the previously known \(\tilde{\mathcal O}(K^{-1/3})\) for Muon to \(\tilde{\mathcal O}(K^{-1/2})\) for Muon-MVR when \(\eta = \beta D\) and \(\alpha = \beta = \frac{2 \ln K}{K}\) [2512.16598].

A common misconception is that Gluon-MVR is a single optimizer with a single proof template. The published analysis instead treats three non-convex MVR insertions, plus a star-convex Muon-MVR specialization, all sharing the same LMO geometry but differing in how variance-reduced momentum is constructed.

## 4. Communication-efficient federated variants

A second line of work extends Gluon-MVR to communication-efficient federated learning. Here the objective is
\[
f(X)=\frac{1}{n}\sum_{\tau=1}^n f^\tau(X),
\]
with \(n\) workers, local data distributions \(D_\tau\), and server-side aggregation. The key design is a SARAH/PAGE-style recursive estimator \(g_i^k\) that alternates between occasional uncompressed refreshes and frequent compressed gradient-difference steps. This is explicitly motivated by the fact that gradient differences are smaller and easier to compress than full gradients [2604.10689].

Two compressor classes are considered. An unbiased compressor \(Q_i\) satisfies
\[
\mathbb{E}[Q_i(X)] = X, \qquad \mathbb{E}\big[\|Q_i(X)\|_2^2\big] \le (\omega+1)\|X\|_2^2,
\]
while a contraction compressor \(C_i\) satisfies
\[
\mathbb{E}\big[\|X - C_i(X)\|_2^2\big] \le (1-\delta)\|X\|_2^2.
\]
Contraction compressors are paired with error feedback, with local memory \(e_i^{\tau,k}\) compensating the compression bias [2604.10689].

In the unbiased compressed Gluon with MVR, the server-side recursive estimator is
\[
g_i^k =
\begin{cases}
\frac{1}{n}\sum_{\tau=1}^n g_i^{\tau,k}, & u^k=1,\\
g_i^{k-1} + y_i^k, & u^k=0,
\end{cases}
\]
where \(u^k\) is the Bernoulli refresh flag and \(y_i^k\) is the averaged compressed gradient difference. The MVR momentum takes the form
\[
M_i^k = \beta M_i^{k-1} + (1-\beta) g_i^k + \beta\, y_i^k.
\]
The same momentum law is used in the contraction-compressor setting after the error-feedback correction [2604.10689].

The theory preserves the overall Gluon-MVR logic while adding communication terms. For unbiased compressors, the rate corollary yields
\[
\min_k \sum_i t_i \,\mathbb{E}[\|\nabla_i f(X^k)\|_{(i)\star}]
= \mathcal{O}\Big(\frac{\sqrt{\Delta^0 \,\mathcal{L}_2}}{\sqrt{K}}\Big)
+ \mathcal{O}\Big(\frac{\sum_i t_i \rho \sigma}{\sqrt{Bn}}\Big),
\]
where \(\mathcal{L}_2\) includes the layer-wise smoothness constants and the compressor-dependent term
\[
\frac{\sqrt{1-q}+\sqrt{q}}{\sqrt{qn}}
\sum_i \rho t_i^2 \sqrt{(\omega+1)\delta_i^2 + \omega L_i^2}.
\]
For contraction compressors with error feedback and MVR, a similar bound holds with \(\mathcal{L}_4\) replacing \(\mathcal{L}_2\). In both cases, the notable point is structural rather than merely asymptotic: when \(L_i^1 \neq 0\), MVR requires only
\[
\eta \le \min_i \frac{1}{L_i^1 t_i},
\]
whereas the non-MVR compressed Gluon analysis requires the stricter condition
\[
\eta/\alpha \le \min_i \frac{1}{5 L_i^1 t_i}.
\]
This weaker condition is one of the central reasons MVR is introduced in the federated setting [2604.10689].

The communication model is explicit. Expected per-step uplink cost is
\[
\Theta\Big(n\big(1 + qd + \tfrac{(1-q)d}{\omega+1}\big)\Big)
\]
for unbiased compression and
\[
\Theta\Big(n\big(1 + qd + (1-q)\delta d\big)\Big)
\]
for contraction compression. For \(\epsilon\)-precision, the total expected communication is reported as
\[
\Theta\left(nd + \frac{1}{c^2 \epsilon^4 \sqrt{n}} + \frac{\sqrt{n}\,d}{\epsilon^2}\right)
\]
for compressed Gluon with unbiased compressors, and
\[
\Theta\left(\frac{1}{c^2 \epsilon^4} + \frac{n\,d}{\epsilon^2}\right)
\]
for compressed Gluon with error feedback and contraction compressors. These dominant terms improve on the stated EF21-Muon stochastic-gradient communication baseline of
\[
\Theta\left(\frac{n d}{\epsilon^3}\right)
\]
[2604.10689].

## 5. Empirical behavior and implementation

The non-federated Gluon-MVR experiments are conducted on NanoGPT with approximately \(124\)M parameters, using FineWeb-10B, a single NVIDIA A100 GPU, \(5{,}000\) steps, and global batch sizes \(B \in \{128,256,512\}\). Validation loss is the reported metric. The strongest large-batch result is for Gluon-MVR-2 at \(B=512\): baseline Gluon reaches \(3.832\), whereas a tuned Gluon-MVR-2 run with \(\beta=0.2\) and \(q=0.7\) reaches \(3.682\). Gluon-MVR-1 is nearly neutral at \(B=512\), with \(3.829\) versus the same \(3.832\) baseline, but becomes more helpful at smaller batch size, improving from \(4.256\) to \(4.107\) at \(B=128\). Gluon-MVR-3 also improves over baseline, with a best reported run of \(3.742\), though it is described as generally slightly inferior to Gluon-MVR-2 and superior to Gluon-MVR-1 [2512.16598].

These experiments support a practical division of labor among the variants. Gluon-MVR-2 is empirically strongest in the large-batch regime, Gluon-MVR-1 is more useful in smaller-batch settings, and Gluon-MVR-3 is intermediate. The recommended theory-guided schedule for provable \( \mathcal O(K^{-1/3}) \) behavior is
\[
\beta^k = 1-(k+1)^{-2/3}, \qquad t_i^k = t_i (k+1)^{-2/3},
\]
whereas constant stepsizes are used for most practical tuning. The main overhead is computational rather than conceptual: MVR requires evaluating both \(\nabla f_{\xi^k}(X^k)\) and \(\nabla f_{\xi^k}(X^{k-1})\), which is roughly a second backward pass per step, while the spectral-norm LMO is typically approximated efficiently by a few power iterations [2512.16598].

The federated experiments use two tasks. The first is logistic regression on the a5a dataset with cross-entropy, weight decay \(1\mathrm e{-4}\), batch size \(64\), and \(n=4\) clients. The second is a CNN on CIFAR-10 with spectral norm LMOs for hidden layers and \(\infty\)-norm for the head, batch size \(16\), and \(n=4\). Compressors include RandK% and TopK%. Across both tasks, Compressed Gluon and Compressed Gluon with error feedback achieve approximately \(60\)–\(65\%\) reduction in communication cost, measured in uplink bytes, relative to tuned baselines at comparable loss. The ablations report that smaller \(q\) slows per-iteration convergence but improves cost-versus-loss trade-offs under aggressive compression, larger \(B\) accelerates convergence of the variance-reduced methods, \(\beta \approx 0.99\) consistently improves performance, and TopK% does not show a clear advantage over RandK% [2604.10689].

From an implementation standpoint, the published guidance is specific. For compressed unbiased variants, the experiments treat \(q \approx 0.05\)–\(0.01\) as typical, while the theory suggests \(q \approx c^2 n \epsilon^2\). For compression, an effective setting is \(K\%=1\%\) for unbiased sparsification, with \((\omega+1)\approx d/r\), and \(\delta=K/d\) for contraction compressors used with error feedback. This suggests that Gluon-MVR is not only a rate-improvement device but also a practical stabilizer when the optimization geometry is non-Euclidean and communication is constrained [2604.10689].

## 6. Relations, scope, and terminological ambiguity

Within optimization, Gluon-MVR is best understood as a family rather than a single named algorithm. It is the MVR augmentation of Gluon, and therefore of any optimizer representable through Gluon’s layer-wise LMO geometry. In this sense it recovers Muon-MVR in the single-block spectral-norm case and Scion-MVR when the same spectral-norm LMO is applied layer-wise. The unifying idea is not a specific architecture or dataset, but the insertion of same-sample variance reduction into momentum under layer-wise non-Euclidean trust regions [2512.16598].

A separate source of confusion is terminological. In the supplied QCD-oriented synthesis, “Gluon-MVR” is also used as a shorthand for a multi-vertex renormalization/modeling framework centered on Landau-gauge four-gluon and ghost–gluon vertices. In that usage, the framework is anchored by a nearly flat four-gluon dressing \(F^{(0)}\), an IR-enhanced \(F^{(2)}\), a suppressed \(F^{(1)}\), and a soft-gluon ghost–gluon form factor \(H_1(k^2,k^2,0)\) that remains close to its tree-level value, supporting \(Z_1=1\) in Taylor kinematics [2505.23476]. Related Landau-gauge work on the gluon propagator and three-gluon vertex with dynamical quarks formulates mass–vertex relations through the decomposition
\[
\Delta^{-1}(q^2)=q^2 J(q^2)+m^2(q^2),
\]
with STIs separating the \(J\)-driven no-pole part of the vertex from the pole part driven by \(m^2\) [1912.12086].

The two usages are unrelated in technical content. One belongs to stochastic non-Euclidean optimization and federated learning; the other belongs to nonperturbative Yang–Mills theory. The overlap is lexical rather than conceptual. A plausible implication is that the term “Gluon-MVR” should always be interpreted in context: in optimizer literature it denotes Momentum Variance Reduction for Gluon, whereas in the QCD synthesis it denotes a multi-vertex renormalization framework.

Taken in its optimization sense, Gluon-MVR occupies a specific place in the recent theory of LMO-based training methods. It preserves Gluon’s layer-wise geometry, sharpens its non-convex theory from \(\mathcal O(K^{-1/4})\) to \(\mathcal O(K^{-1/3})\), improves the star-convex Muon rate to \(\tilde{\mathcal O}(K^{-1/2})\), and extends naturally to compressed federated training with explicit communication gains and weaker \(L_i^1 \neq 0\) conditions. Its significance lies in showing that the practical advantages associated with Muon-type non-Euclidean updates can be made compatible with modern variance-reduction methodology rather than treated as a separate optimization paradigm [2512.16598][2604.10689].

Source: https://www.emergentmind.com/topics/gluon-mvr