MagRan-DM: Diffusion Models & Dark Matter

• MagRan-DM is a multifaceted term covering a dynamic sparse training algorithm for diffusion models, gamma-ray dark matter search strategies, and dark matter-photon coupling theories.
• The sparse training approach employs magnitude-based pruning and random regrowth to optimize network efficiency and maintain sample quality in generative models.
• In astrophysics and particle physics, MagRan-DM underpins innovative gamma-ray searches for branon dark matter and UV-complete models using dipole and Rayleigh operators.

MagRan-DM is an acronym appearing in three distinct contexts in recent high-impact research: (1) as a magnitude-based random-growth dynamic sparse training strategy for generative diffusion models in deep learning (Oliveira et al., 30 Apr 2025); (2) as a search campaign for branon dark matter annihilation signatures using the MAGIC gamma-ray telescopes (Miener et al., 2022); and (3) as shorthand for Magnetic Inelastic/Rayleigh Dark Matter, a theoretical ultraviolet (UV) completion producing dipole and Rayleigh dark matter-photon couplings (Weiner et al., 2012). Each incarnation embodies a unique methodology, scientific motivation, and technical framework. The following presents an authoritative synthesis, emphasizing the dynamic sparse training algorithm for diffusion models, then contextualizing the astrophysical and particle physics usages.

1. Definition(s) and Overview

The term “MagRan-DM” originally denotes a magnitude-based random-growth dynamic sparse training algorithm for deep generative diffusion models: an approach that aims to sparsify neural networks during training, rather than post hoc, by iteratively pruning the lowest-magnitude weights and regrowing new connections randomly at constant sparsity. This enhances both computational and memory efficiency, without degrading—and sometimes improving—sample quality relative to dense baselines (Oliveira et al., 30 Apr 2025).

Separately, “MagRan-DM” also names:

• A multi-target search for branon dark matter annihilation in dwarf spheroidal galaxies with MAGIC (Major Atmospheric Gamma Imaging Cherenkov) telescopes, which achieved the leading gamma-ray bounds on TeV-mass branon WIMPs (Miener et al., 2022).
• Theoretical models of dark matter interacting via electromagnetic dipole (magnetic inelastic) and Rayleigh operators generated by heavy-charged messengers, forming a calculable UV-complete framework relevant for indirect detection signatures such as gamma-ray lines (Weiner et al., 2012).

2. Algorithmic Framework: Magnitude-Based Random-Growth Dynamic Sparse Training

MagRan-DM (in deep learning) is a class of dynamic sparse training (DST) algorithms, modifying the trainable connectivity graph of neural network weights during training. Contrasting with static sparsity methods (where the sparsity pattern is fixed at initialization), DST algorithms periodically reallocate connections to promote efficient representation learning under stringent sparsity constraints.

Key Steps

1. Initialize masks for each layer to achieve a global target sparsity $S$, using Erdős–Rényi-Kernel (ERK) or Erdős–Rényi (ER) sampling.
2. During training, every $\Delta T_e$ optimizer steps:
   • Prune the $p$ fraction of currently nonzero weights with lowest absolute value (“magnitude-based pruning”).
   • Regrow, at random, $p$ fraction of new connections among the zeroed-out positions (“random growth”).
   • The global sparsity and per-layer allocation are maintained constant.
3. Network updates (forward and backward) happen solely over the supported (“live”) weights.

This regime does not require evaluation of gradients for dormant connections (in contrast to RigL-DM, which does), resulting in reduced memory overhead and computational simplicity (Oliveira et al., 30 Apr 2025).

Mathematical Formalism

If $W^{(l)}\in\mathbb{R}^{d_l\times d_{l-1}}$ is a weight matrix, the active parameters at step $t$ are $\theta^{(l)}_t = W^{(l)} \odot M^{(l)}_t$, where $M^{(l)}_t\in\{0,1\}^{d_l\times d_{l-1}}$ is the binary mask. At DST update steps:

• Pruned mask $$M^{(l)}_{\text{prune}} = M^{(l)}_t \odot [|\theta^{(l)}_t|\geq\tau]$$ where $\tau$ is the $(1-p)$ quantile.
• Regrowth mask: randomly activate $p\|M^{(l)}_t\|_1$ new positions among currently masked weights.
• New mask $$M^{(l)}_{t+} = M^{(l)}_{\text{prune}} + M^{(l)}_{\text{grow}}$$.

The loss function for Denoising Diffusion Probabilistic Models (DDPM) is preserved:

$$\mathcal{L}_{\mathrm{DIF}} = \mathbb{E}_{t, x_0, \epsilon} \left[ \|\epsilon - \epsilon_\theta(x_t, t)\|^2 \right]$$

3. Empirical Evaluation in Diffusion Models

Extensive ablation and benchmarking across latent diffusion and GRU-based sequential diffusion (ChiroDiff) architectures demonstrates:

• 25–50% sparsity ($S=0.25$–$0.5$) generally preserves or improves sample quality (e.g., FID).
• At high sparsity ($S\geq 0.75$), MagRan-DM with conservative regrowth ($p=0.05$) stabilizes sparse exploration and maintains performance.
• On QuickDraw, MagRan-DM at $S=0.90$ matches dense FID with only 10% of parameters/FLOPs; on LSUN-Bedrooms and Imagenette, $S=0.25$–$0.50$ yields lower FIDs than dense or static sparse baselines.
• Compared with alternative DST algorithms (RigL), MagRan-DM demonstrates competitive or superior results at lower computational cost (Oliveira et al., 30 Apr 2025).

Selected Results: Latent Diffusion

Dataset	Approach	FID (↓)	Params	Train/Test FLOPs
CelebA-HQ	Dense	32.74 ± 3.68	1.00×	1.00×/1.00×
	MagRan-DM	32.83 ± 1.68 ($S=0.5$)	0.50×	0.67×/0.67×
Bedrooms	Dense	31.09 ± 12.42	1.00×	1.00×/1.00×
	MagRan-DM	28.20 ± 7.64 ($S=0.25$)	0.75×	0.91×/0.91×

At high sparsity, careful tuning of $p$ (prune-regrow ratio) is required to sustain accuracy.

4. Physical Modeling: Astrophysical and Particle Physics Usages

The term "MagRan-DM" also designates:

(A) MAGIC Branon Dark Matter Search

A coordinated gamma-ray search for TeV-scale branon dark matter annihilation in four dwarf spheroidal galaxies (dSphs) using the MAGIC telescopes. The analysis combines:

• Observation of Segue 1, Ursa Major II, Draco, and Coma Berenices for a total of 354 hours.
• Joint-likelihood statistical combination over all targets, incorporating astrophysical J-factors and background modeling.
• Photon flux prediction:

$$\Phi(\Delta\Omega,E) = \frac{1}{4\pi} \frac{\langle\sigma v\rangle}{2m_\chi^2} J(\Delta\Omega) \frac{dN_\mathrm{BDM}}{dE}$$

• 95% C.L. upper limit on $\langle\sigma v\rangle$ for $m_\chi \approx 0.7\;\mathrm{TeV}$: $4.9\times10^{-24}\;\mathrm{cm}^3\,\mathrm{s}^{-1}$, surpassing previous gamma-ray, positron, and collider constraints (Miener et al., 2022).

(B) Magnetic Inelastic and Rayleigh Dark Matter

Theoretical extensions (“Magnetic inelastic DM/RayDM”) involve:

• Dimension-5 dipole (“magnetic inelastic DM”) and dimension-7 Rayleigh operators coupling dark matter to SM photons/gauge bosons, generated via loops of electroweak-charged messengers.
• Pseudo-Dirac DM with small Majorana splitting: freeze-out via dipolar interactions, indirect detection via Rayleigh operators.
• Cross-sections receive significant form-factor corrections, enhancing gamma-ray line signals without excessive continuum emission.
• Realizations require moderately strong Yukawa couplings and messenger masses in the $150$–$300\;\mathrm{GeV}$ range (Weiner et al., 2012).

5. Practical Recommendations and Trade-Offs (Sparse Diffusion)

Practical deployment of MagRan-DM in deep generative models yields:

• Compelling sparsity “sweet spots” at $S=0.25$–$0.5$: parameter and FLOPs reductions of 25–50% with no FID penalty.
• For $S\geq0.75$, set $p=0.05$ to prevent topology collapse.
• Use ERK mask allocation for convolutional layers, ER for linear.
• Moderate sparsity often acts as effective implicit regularization, matching or exceeding dense network performance without sacrificing computational benefits (Oliveira et al., 30 Apr 2025).

6. Scientific Significance and Prospects

MagRan-DM exemplifies the evolution of sparse learning methods capable of “sparse-to-sparse” training in domains previously dominated by resource-intensive dense models. Its extension to high-sparsity training in diffusion models marks a paradigm shift in efficient generative modeling. In gamma-ray astrophysics, the MagRan-DM search methodology sets new exclusion milestones for branon WIMP models, motivating next-generation telescopes and multi-instrument campaigns. In particle physics, UV-complete MagRan-DM frameworks clarify the dual role of magnetic and Rayleigh operators in explaining gamma-ray lines and thermal relic abundance, informing both collider and indirect detection strategies.

Ongoing directions include public release of code and models for MagRan-DM sparse diffusion, prospective CTA/SKA branon searches, and systematic mapping of parameter space in dipole/Rayleigh dark matter frameworks.