Gluon-MVR: Momentum Variance Reduction
- Gluon-MVR is a momentum variance reduction technique that augments the Gluon framework with same-sample gradient-difference corrections to efficiently reduce variance.
- It improves non-convex convergence rates from O(K⁻¹/4) to O(K⁻¹/3) (and O(K⁻¹/2) for star-convex cases) by leveraging layer-wise norms and trust-region LMOs.
- The method extends to federated settings by integrating SARAH-style recursive estimators and compression techniques, enhancing communication efficiency.
Searching arXiv for papers on Gluon-MVR, Gluon, Muon, and related optimizer variants. Gluon-MVR denotes a family of momentum variance reduction methods built on the Gluon optimization framework, a layer-wise non-Euclidean linear minimization oracle (LMO) scheme that captures Muon, Scion, and related trust-region-style optimizers as special cases. In the non-convex stochastic setting, Gluon-MVR replaces vanilla momentum with control-variate updates based on same-sample gradient differences at consecutive iterates, while retaining Gluon’s layer-wise norm geometry and LMO step. In distributed and federated settings, the same principle is combined with SARAH-style recursive estimators, unbiased or contraction compressors, and error feedback, yielding communication-efficient variants with comparable communication complexity under weaker conditions when (Qian et al., 18 Dec 2025, Qian et al., 12 Apr 2026).
1. Gluon framework and optimization setting
Gluon-MVR inherits its ambient problem from Gluon. The objective is stochastic optimization over a finite-dimensional vector space,
or, in the federated formulation,
The parameter is partitioned layer-wise as , with . This block structure is central: Gluon does not assume a single Euclidean geometry, but instead equips each layer with its own norm and dual norm (Qian et al., 18 Dec 2025).
The defining regularity condition is the layer-wise -smoothness assumption,
This formulation is more general than standard global -smoothness and is meant to reflect the empirical observation that curvature and smoothness vary by layer and may scale with the local gradient norm. The analysis is supplemented by unbiased stochastic gradients with bounded variance, a norm-equivalence bound 0, and a Hessian-variance condition controlled by 1 (Qian et al., 18 Dec 2025).
The update itself is expressed through a trust-region LMO. For each block 2, Gluon defines
3
and computes
4
Here 5 is the momentum-like search direction. By choosing the block partition and the norms appropriately, Gluon recovers Muon and Scion. Muon corresponds to spectral-norm trust-region LMOs on hidden-layer weight matrices, whereas Scion applies the spectral-norm LMO to all layers (Qian et al., 18 Dec 2025).
2. Momentum variance reduction and the three Gluon-MVR variants
The defining modification in Gluon-MVR is the replacement of vanilla momentum by same-sample gradient-difference corrections. In vanilla Gluon, the momentum is typically
6
Gluon-MVR inserts a STORM-style control variate that evaluates gradients at both 7 and 8 using the same sample 9. This reduces variance without introducing bias and does not require large batches or checkpoint gradients (Qian et al., 18 Dec 2025).
The first variant, Gluon-MVR-1, is the most direct substitution: 0 Its structure is minimal: the previous momentum is corrected by subtracting the stale stochastic gradient on the current sample and adding the current one. In the single-block spectral-norm case, this coincides with Muon-MVR. The paper also gives a decreasing-stepsize version with
1
which is used to obtain theory-guided 2 guarantees (Qian et al., 18 Dec 2025).
The second variant, Gluon-MVR-2, introduces an auxiliary recursive estimator 3,
4
followed by a smoothed momentum
5
The parameter 6 controls the strength of the variance-reduced correction. Smaller 7 increases the recursive contribution, whereas larger 8 makes the estimator more local (Qian et al., 18 Dec 2025).
The third variant, Gluon-MVR-3, combines the recursive estimator of Gluon-MVR-2 with an additional gradient-difference correction,
9
This formulation interpolates between the first two designs: it retains a recursively denoised estimator 0 while also injecting the explicit same-sample difference term into the momentum state (Qian et al., 18 Dec 2025).
Across all three variants, the LMO step remains unchanged. The algorithmic identity of Gluon-MVR is therefore not a new search geometry, but a new momentum estimator inside the pre-existing non-Euclidean trust-region geometry of Gluon.
3. Convergence theory and rate improvements
The principal theoretical claim associated with Gluon-MVR is that MVR improves the constant-stepsize non-convex rate from 1 for vanilla Gluon to 2 for all three MVR variants. The paper presents this as the optimal rate for single-sample stochastic non-convex optimization, citing lower-bound consistency. The gain is traced to tighter control of
3
obtained by exploiting same-sample gradient differences, Hessian variance bounds, and the layer-wise smoothness structure (Qian et al., 18 Dec 2025).
For Gluon-MVR-1, the representative non-convex guarantee is stated in terms of
4
When 5, choosing
6
yields
7
When 8, the same rate is recovered essentially under
9
up to a small additive term of order 0. Gluon-MVR-2 and Gluon-MVR-3 admit analogous 1 statements under parameter choices such as 2 and 3 for Gluon-MVR-2, and 4 with 5 for Gluon-MVR-3 (Qian et al., 18 Dec 2025).
The star-convex case is treated separately through Muon-MVR. There the trust-region step is written as
6
with MVR update
7
Under star-convexity and Non-Euclidean smoothness, the resulting rate improves from the previously known 8 for Muon to 9 for Muon-MVR when 0 and 1 (Qian et al., 18 Dec 2025).
A common misconception is that Gluon-MVR is a single optimizer with a single proof template. The published analysis instead treats three non-convex MVR insertions, plus a star-convex Muon-MVR specialization, all sharing the same LMO geometry but differing in how variance-reduced momentum is constructed.
4. Communication-efficient federated variants
A second line of work extends Gluon-MVR to communication-efficient federated learning. Here the objective is
2
with 3 workers, local data distributions 4, and server-side aggregation. The key design is a SARAH/PAGE-style recursive estimator 5 that alternates between occasional uncompressed refreshes and frequent compressed gradient-difference steps. This is explicitly motivated by the fact that gradient differences are smaller and easier to compress than full gradients (Qian et al., 12 Apr 2026).
Two compressor classes are considered. An unbiased compressor 6 satisfies
7
while a contraction compressor 8 satisfies
9
Contraction compressors are paired with error feedback, with local memory 0 compensating the compression bias (Qian et al., 12 Apr 2026).
In the unbiased compressed Gluon with MVR, the server-side recursive estimator is
1
where 2 is the Bernoulli refresh flag and 3 is the averaged compressed gradient difference. The MVR momentum takes the form
4
The same momentum law is used in the contraction-compressor setting after the error-feedback correction (Qian et al., 12 Apr 2026).
The theory preserves the overall Gluon-MVR logic while adding communication terms. For unbiased compressors, the rate corollary yields
5
where 6 includes the layer-wise smoothness constants and the compressor-dependent term
7
For contraction compressors with error feedback and MVR, a similar bound holds with 8 replacing 9. In both cases, the notable point is structural rather than merely asymptotic: when 0, MVR requires only
1
whereas the non-MVR compressed Gluon analysis requires the stricter condition
2
This weaker condition is one of the central reasons MVR is introduced in the federated setting (Qian et al., 12 Apr 2026).
The communication model is explicit. Expected per-step uplink cost is
3
for unbiased compression and
4
for contraction compression. For 5-precision, the total expected communication is reported as
6
for compressed Gluon with unbiased compressors, and
7
for compressed Gluon with error feedback and contraction compressors. These dominant terms improve on the stated EF21-Muon stochastic-gradient communication baseline of
8
5. Empirical behavior and implementation
The non-federated Gluon-MVR experiments are conducted on NanoGPT with approximately 9M parameters, using FineWeb-10B, a single NVIDIA A100 GPU, 0 steps, and global batch sizes 1. Validation loss is the reported metric. The strongest large-batch result is for Gluon-MVR-2 at 2: baseline Gluon reaches 3, whereas a tuned Gluon-MVR-2 run with 4 and 5 reaches 6. Gluon-MVR-1 is nearly neutral at 7, with 8 versus the same 9 baseline, but becomes more helpful at smaller batch size, improving from 0 to 1 at 2. Gluon-MVR-3 also improves over baseline, with a best reported run of 3, though it is described as generally slightly inferior to Gluon-MVR-2 and superior to Gluon-MVR-1 (Qian et al., 18 Dec 2025).
These experiments support a practical division of labor among the variants. Gluon-MVR-2 is empirically strongest in the large-batch regime, Gluon-MVR-1 is more useful in smaller-batch settings, and Gluon-MVR-3 is intermediate. The recommended theory-guided schedule for provable 4 behavior is
5
whereas constant stepsizes are used for most practical tuning. The main overhead is computational rather than conceptual: MVR requires evaluating both 6 and 7, which is roughly a second backward pass per step, while the spectral-norm LMO is typically approximated efficiently by a few power iterations (Qian et al., 18 Dec 2025).
The federated experiments use two tasks. The first is logistic regression on the a5a dataset with cross-entropy, weight decay 8, batch size 9, and 00 clients. The second is a CNN on CIFAR-10 with spectral norm LMOs for hidden layers and 01-norm for the head, batch size 02, and 03. Compressors include RandK% and TopK%. Across both tasks, Compressed Gluon and Compressed Gluon with error feedback achieve approximately 04–05 reduction in communication cost, measured in uplink bytes, relative to tuned baselines at comparable loss. The ablations report that smaller 06 slows per-iteration convergence but improves cost-versus-loss trade-offs under aggressive compression, larger 07 accelerates convergence of the variance-reduced methods, 08 consistently improves performance, and TopK% does not show a clear advantage over RandK% (Qian et al., 12 Apr 2026).
From an implementation standpoint, the published guidance is specific. For compressed unbiased variants, the experiments treat 09–10 as typical, while the theory suggests 11. For compression, an effective setting is 12 for unbiased sparsification, with 13, and 14 for contraction compressors used with error feedback. This suggests that Gluon-MVR is not only a rate-improvement device but also a practical stabilizer when the optimization geometry is non-Euclidean and communication is constrained (Qian et al., 12 Apr 2026).
6. Relations, scope, and terminological ambiguity
Within optimization, Gluon-MVR is best understood as a family rather than a single named algorithm. It is the MVR augmentation of Gluon, and therefore of any optimizer representable through Gluon’s layer-wise LMO geometry. In this sense it recovers Muon-MVR in the single-block spectral-norm case and Scion-MVR when the same spectral-norm LMO is applied layer-wise. The unifying idea is not a specific architecture or dataset, but the insertion of same-sample variance reduction into momentum under layer-wise non-Euclidean trust regions (Qian et al., 18 Dec 2025).
A separate source of confusion is terminological. In the supplied QCD-oriented synthesis, “Gluon-MVR” is also used as a shorthand for a multi-vertex renormalization/modeling framework centered on Landau-gauge four-gluon and ghost–gluon vertices. In that usage, the framework is anchored by a nearly flat four-gluon dressing 15, an IR-enhanced 16, a suppressed 17, and a soft-gluon ghost–gluon form factor 18 that remains close to its tree-level value, supporting 19 in Taylor kinematics (Brito et al., 29 May 2025). Related Landau-gauge work on the gluon propagator and three-gluon vertex with dynamical quarks formulates mass–vertex relations through the decomposition
20
with STIs separating the 21-driven no-pole part of the vertex from the pole part driven by 22 (Aguilar et al., 2019).
The two usages are unrelated in technical content. One belongs to stochastic non-Euclidean optimization and federated learning; the other belongs to nonperturbative Yang–Mills theory. The overlap is lexical rather than conceptual. A plausible implication is that the term “Gluon-MVR” should always be interpreted in context: in optimizer literature it denotes Momentum Variance Reduction for Gluon, whereas in the QCD synthesis it denotes a multi-vertex renormalization framework.
Taken in its optimization sense, Gluon-MVR occupies a specific place in the recent theory of LMO-based training methods. It preserves Gluon’s layer-wise geometry, sharpens its non-convex theory from 23 to 24, improves the star-convex Muon rate to 25, and extends naturally to compressed federated training with explicit communication gains and weaker 26 conditions. Its significance lies in showing that the practical advantages associated with Muon-type non-Euclidean updates can be made compatible with modern variance-reduction methodology rather than treated as a separate optimization paradigm (Qian et al., 18 Dec 2025, Qian et al., 12 Apr 2026).