HTMuon: Heavy-Tailed Matrix Optimization & Physics

• The paper introduces HTMuon as a matrix-based optimizer that applies a power transform to singular values, preserving geometry while inducing heavy-tailed weight spectra.
• HTMuon achieves significant model improvements by down-weighting noise-dominated directions, demonstrated by perplexity reductions and accuracy gains on LLM and vision benchmarks.
• HTMuon also denotes high multiplicity triggers in CMS and precision studies of the Higgs–muon coupling, highlighting its dual role in advanced machine learning and high-energy physics.

The term HTMuon appears in multiple research contexts. In machine learning, it denotes a matrix-valued optimizer introduced as “HTMuon: Improving Muon via Heavy-Tailed Spectral Correction,” where it is presented as a drop-in replacement and improvement over Muon for LLM pretraining and other large deep networks (Pang et al., 10 Mar 2026). In CMS Run 3 trigger language, the same term corresponds to the High Multiplicity Trigger (HMT) based on cathode strip chamber hit multiplicities in the muon system (Abbott et al., 25 Jun 2026). A separate high-energy physics usage applies “HTMuon” to the Higgs–muon coupling at a high-energy muon collider, especially in the context of multi-boson production and precision tests of $y_\mu$ (Reuter, 2023). Across these usages, the term consistently refers to muon-related high-throughput or high-sensitivity structures, but the underlying objects, methods, and scientific goals are distinct.

1. HTMuon as a matrix-valued optimizer

In the optimization literature, HTMuon is a matrix-based optimizer that operates on each parameter matrix $W \in \mathbb{R}^{m\times n}$ rather than on flattened vectors. Its design goal is to preserve Muon’s geometry-aware handling of parameter interdependencies while replacing Muon’s orthogonalized, spectrally flat update rule with one whose singular values are explicitly heavy-tailed (Pang et al., 10 Mar 2026).

The motivation is tied to Heavy-Tailed Self-Regularization (HT-SR) theory. In that framework, well-trained deep networks exhibit heavy-tailed empirical spectral densities of weight correlation matrices, and smaller power-law exponents $\alpha$ correlate with better model quality and generalization. The HTMuon paper argues that Muon’s practice of setting all singular values of the update to $1$ suppresses the emergence of heavy-tailed weight spectra and over-emphasizes training along noise-dominated directions. The paper further reports that exact-SVD Muon, whose update spectrum is exactly flat, is consistently outperformed by Newton–Schulz Muon on LLaMA-60M and 135M pretraining on C4, with the latter exhibiting nontrivial spectral deviations that effectively down-weight noise-dominated directions (Pang et al., 10 Mar 2026).

Within this formulation, HTMuon interpolates between two limiting cases. When the singular-value transform exponent is $p=0$, the method reduces to Muon, whose update is purely orthogonal. When $p=1$, it recovers SGDM on matrices. For $p\in(0,1)$, it remains matrix-based and geometry-aware, but produces an update spectrum that is heavier-tailed than Muon’s and empirically induces heavier-tailed weight spectra. The default choice reported in the paper is $p=0.125$, selected through ablations on LLaMA, OpenWebText, and vision models (Pang et al., 10 Mar 2026).

2. Algorithmic structure and heavy-tailed spectral correction

HTMuon retains Muon’s momentum-and-SVD structure, but replaces spectral flattening with a power transform. If the momentum matrix is

$$M_t=\beta M_{t-1}+(1-\beta)G_t,\qquad G_t=\nabla_W L(W_t),$$

and its singular value decomposition is

$M_t=U_t\Sigma_tV_t^\top,$

then HTMuon defines the update matrix as

$W \in \mathbb{R}^{m\times n}$0

The weight update is

$W \in \mathbb{R}^{m\times n}$1

with shape scaling

$W \in \mathbb{R}^{m\times n}$2

This is the central algorithmic difference from Muon, which replaces $W \in \mathbb{R}^{m\times n}$3 by the identity and therefore erases all singular-value variation (Pang et al., 10 Mar 2026).

The power transform $W \in \mathbb{R}^{m\times n}$4 preserves the singular vectors $W \in \mathbb{R}^{m\times n}$5 and $W \in \mathbb{R}^{m\times n}$6 while compressing singular values nonlinearly. In the formulation given in the paper, large singular values remain relatively large, small singular values are shrunk more strongly, and the relative ordering is preserved. The intended consequence is that the optimizer still exploits matrix geometry but no longer assigns equal magnitude to all spectral directions, including directions dominated by noise (Pang et al., 10 Mar 2026).

The paper also studies an ablation called HTMuon_HT, in which the singular values are replaced by a fixed heavy-tailed sequence $W \in \mathbb{R}^{m\times n}$7 rather than by the data-dependent spectrum of $W \in \mathbb{R}^{m\times n}$8. That variant still improves over Muon, which the authors interpret as evidence that heavy-tail injection itself is beneficial. However, the full HTMuon method performs better, indicating that preserving and reshaping the data-dependent singular spectrum is stronger than imposing a fixed heavy-tailed template (Pang et al., 10 Mar 2026).

A related spectral argument is given through a toy model. If the singular values of $W \in \mathbb{R}^{m\times n}$9 follow a power law $\alpha$0, then the empirical spectral density of the correlation matrix of $\alpha$1 obeys a power law with exponent

$\alpha$2

This formalizes the optimizer’s role as a spectral correction mechanism rather than a mere orthogonalizer (Pang et al., 10 Mar 2026).

3. Geometric interpretation and theoretical analysis

HTMuon is theoretically characterized as a steepest-descent method in a non-Euclidean matrix geometry. The paper considers the constrained first-order problem

$\alpha$3

with $\alpha$4 a Schatten-$\alpha$5 norm. If $\alpha$6, the maximizer is

$\alpha$7

where $\alpha$8 is Hölder-conjugate to $\alpha$9. Identifying the momentum with the negative gradient gives HTMuon’s update form

$1$0

when $1$1. In this sense, HTMuon is steepest descent under a Schatten-$1$2 norm constraint, while Muon is the special case associated with a Schatten-$1$3 constraint and a spectrally flat update (Pang et al., 10 Mar 2026).

The same paper gives convergence analysis in smooth non-convex settings under three assumptions: Frobenius-smoothness, unbiased stochastic gradients with bounded variance, and bounded singular values of the momentum. With an adaptive step size

$1$4

the average stationarity measure in a suitable norm obeys an $1$5-type bound, and with $1$6 and $1$7 the sample complexity is

$1$8

for reaching an $1$9-stationary point. The paper emphasizes that this matches the asymptotic sample-complexity upper bound of Muon and SGDM while changing the update geometry (Pang et al., 10 Mar 2026).

A later analysis of Schatten-$p=0$0 geometries places HTMuon-like methods within the broader SODA framework and argues that optimizer choice is regime dependent. Even when the objective is smooth in the Schatten-$p=0$1 geometry, smaller Schatten-$p=0$2 geometries can be optimal in the low-dimensional regime, which the paper states includes Chinchilla scaling. The same analysis explains why Muon-like methods do not require warmup, why they naturally favor large batches, and provides a batch-size scaling rule for arbitrary $p=0$3 (Pethick, 13 Jun 2026). This suggests that HTMuon’s finite-$p=0$4-style spectral shaping is not merely an empirical modification of Muon, but part of a broader family of non-Euclidean training geometries.

4. Empirical performance and implementation

The empirical evaluation in the HTMuon paper spans LLM pretraining, image classification, and downstream commonsense reasoning. On LLaMA pretraining on C4, the reported reduction in perplexity relative to Muon reaches up to $p=0$5, which is the headline quantitative example given in the abstract (Pang et al., 10 Mar 2026).

Benchmark	Baseline	HTMuon
LLaMA-60M on C4 (PPL)	Muon 28.80	27.88
LLaMA-135M on C4 (PPL)	Muon 22.23	21.25
GPT-2 small on OpenWebText (PPL)	Muon 22.46	22.20
ResNet-50 on CIFAR-100 (top-1)	Muon 79.85%	80.16%
ViT-tiny on ImageNet-1K (top-1)	Muon 71.02%	71.16%
LLaMA-1B, 7-task average accuracy	Muon 40.03	41.08

These results are accompanied by comparisons to Adam, AdamW, COSMOS, NorMuon, AdaMuon, Cautious, MARS, SOAP, GaLore, Sophia, and other baselines. HTMuon is also presented as a plug-in on top of Muon variants; for example, HTMuon+NorMuon improves over NorMuon on LLaMA-60M and LLaMA-135M, and the paper reports that HTMuon consistently improves over each Muon variant studied (Pang et al., 10 Mar 2026).

The main practical cost is computational overhead. Exact HTMuon requires an SVD per parameter matrix per update, which is significantly more expensive than Muon’s usual Newton–Schulz approximation. The paper therefore introduces two acceleration strategies: interval updates, where HTMuon is applied every $p=0$6 steps and Muon is used otherwise, and HTMuon_NS, which approximates the spectral power transform without an exact SVD. On $p=0$7A6000, the reported per-step wall times for LLaMA-60M are $p=0$8 s for Muon, $p=0$9 s for HTMuon with interval $p=1$0, $p=1$1 s for HTMuon_NS with interval $p=1$2, and $p=1$3 s for HTMuon_NS with interval $p=1$4 (Pang et al., 10 Mar 2026).

The paper recommends $p=1$5 as a robust default, retains the same learning-rate, weight-decay, and momentum interfaces as Muon, and reports released code at https://github.com/TDCSZ327/HTmuon (Pang et al., 10 Mar 2026).

5. Other scientific uses of “HTMuon”

Outside optimization, HTMuon is also used in detector and collider physics. In CMS Run 3 trigger language, it refers to the High Multiplicity Trigger (HMT) for long-lived particle searches in the endcap muon system. At Level-1, the trigger counts anode and cathode hits in CSC chambers per bunch crossing, requires hits in at least $p=1$6 of $p=1$7 layers, and classifies chambers into No Shower, Loose, Nominal, and Tight categories using a 2-bit encoding. At the HLT level, CSC rechits are clustered in $p=1$8-$p=1$9 with the Cambridge–Aachen algorithm and radius

$p\in(0,1)$0

with a cluster defined by at least $p\in(0,1)$1 CSC rechits and a timing selection

$p\in(0,1)$2

The operational HLT paths are One-Nominal and Two-Loose, and the combined HLT HMT rate is around $p\in(0,1)$3 Hz at pileup $p\in(0,1)$4, representing $p\in(0,1)$5 of the total HLT output. For benchmark LLP models, the trigger improves signal efficiency by more than a factor of 30 relative to the Run-2 baseline based largely on $p\in(0,1)$6 triggers (Abbott et al., 25 Jun 2026).

A different high-energy physics usage appears in work on the Higgs–muon coupling at a high-energy muon collider. There, “HTMuon” designates precision studies of $p\in(0,1)$7 using multi-boson processes such as

$p\in(0,1)$8

The central mechanism is a “subtle cancellation” between amplitudes involving the muon Yukawa vertex and longitudinal gauge-boson production. Under the luminosity scaling

$p\in(0,1)$9

the proceedings article reports a $p=0.125$0 sensitivity to deviations of about

$p=0.125$1

at $p=0.125$2 TeV and about

$p=0.125$3

at $p=0.125$4 TeV, with sign sensitivity unavailable at the HL-LHC. The same work emphasizes the necessity of full NLO electroweak corrections, which are large and negative for multi-boson final states at multi-TeV energies (Reuter, 2023).

These usages are unrelated at the methodological level. One concerns optimizer geometry in deep learning, one concerns CSC-hit-based LLP triggering in CMS, and one concerns precision Higgs–muon coupling studies at a future collider.

6. Limitations, open questions, and broader significance

The optimization literature identifies several limitations of HTMuon. The reported experiments extend to 1B parameters, but systematic evaluation at GPT-3 scale or on much larger corpora remains open. Exact SVD-based HTMuon is substantially more expensive per step than Muon_NS, and although HTMuon_NS and interval updates narrow the gap to roughly the $p=0.125$5–$p=0.125$6 range on the tested models, overhead remains a central practical constraint. The convergence analysis also relies on assumptions such as bounded singular values of the momentum, and the link between heavy-tailed spectral correction and generalization is described as largely empirical and HT-SR-motivated rather than fully formalized (Pang et al., 10 Mar 2026).

Related work on Schatten-$p=0.125$7 methods extends the list of open problems. It highlights the need for adaptive $p=0.125$8 policies, sharper characterizations of effective dimension in deep networks, and extensions from convex or smooth analyses to more realistic non-convex settings. It also suggests that the appropriate geometry depends on training regime: Muon-like Schatten-$p=0.125$9 methods are favored in high-dimensional, low-budget regimes, whereas finite-$$M_t=\beta M_{t-1}+(1-\beta)G_t,\qquad G_t=\nabla_W L(W_t),$$0 methods such as HTMuon or Soft-Muon are theoretically more attractive in low-dimensional, large-budget settings, including Chinchilla-style scaling (Pethick, 13 Jun 2026).

Taken together, these works place HTMuon at an intersection of optimizer design, matrix geometry, and heavy-tailed spectral analysis. In its machine-learning sense, HTMuon is a concrete attempt to encode heavy-tailed spectral inductive bias directly into the update rule rather than treating heavy-tailed structure only as a post hoc diagnostic. In its collider and detector senses, the same label designates high-sensitivity muon-sector methods that exploit multiplicity or interference structures rather than conventional observables. The shared abbreviation therefore spans several advanced arXiv subcultures, but in each case it marks an effort to extract more information from muon-related systems by changing the geometry of the measurement or update itself.