Masked Diffusion Generative Recommendation

• The paper demonstrates that MDGR overcomes limitations of autoregressive models by leveraging parallel masked diffusion and adaptive tokenization, leading to improved recall and efficiency.
• It introduces a reverse denoising mechanism with bidirectional Transformers and curriculum masking to better capture multi-dimensional dependencies.
• Empirical results on Amazon and industrial datasets show significant gains in recall, CTR, and GMV while reducing computational complexity.

Masked Diffusion Generative Recommendation (MDGR) refers to a class of generative recommender systems that leverage masked diffusion-based architectures—either over discrete code tokens (“semantic IDs”) or in the continuous latent space—to produce top-quality item predictions. These frameworks depart from the traditional left-to-right autoregressive paradigm by employing parallel diffusion, denoising, and multi-step refinement with adaptive masking. The aim is to jointly model the multi-dimensional, high-order dependencies in item attributes and user histories, achieving improved accuracy, generalization, and inference efficiency on both public and industrial-scale recommendation benchmarks (Mu et al., 27 Jan 2026).

1. Background and Motivation

Traditional Generative Recommendation (GR) models employ autoregressive decoding over item semantic IDs (SIDs), with each item quantized as a tuple of discrete tokens via a hierarchical codebook. The scoring function factorizes as $$p_\theta(c|s_u) = \prod_{\ell=1}^L p_\theta(c^\ell | c^{<\ell}, s_u)$$, enforcing a unidirectional, fixed order for code prediction. This approach introduces three major limitations:

• Limited Global Consistency: Dependencies across SID subspaces cannot be fully captured due to sequential factorization.
• Inflexible Attribute Attention: The fixed decoding order assumes uniform user attention to item attributes, neglecting per-user or per-context priorities.
• Suboptimal Inference Throughput: Sequential left-to-right decoding requires $O(L^3)$ complexity (e.g., with beam search), impeding real-time deployment in large-scale settings.

To address these structural and efficiency bottlenecks, Masked Diffusion Generative Recommendation replaces unidirectional AR decoding with an order-agnostic, parallel masked diffusion process and bidirectional denoising transformers (Mu et al., 27 Jan 2026, Liu et al., 21 Oct 2025, Shi et al., 9 Nov 2025, Shah et al., 28 Nov 2025).

2. Parallel Codebook and Semantic ID Tokenization

MDGR frameworks consistently adopt parallel (independent) codebook tokenization for item representation, in contrast to earlier residual/hierarchical quantization:

• Item Embedding: Each item $i$ is mapped to a continuous feature vector $e_i \in \mathbb{R}^d$ (from a pretrained encoder such as Qwen3-8B).
• Parallel Projection: $e_i$ is projected through $L$ independent subspaces $e_i^\ell = f^\ell(e_i)$, with $d = \sum_\ell d_\ell$.
• Parallel Quantization: For each subspace, a codebook $C_\ell \in \mathbb{R}^{|V_\ell|\times d_\ell}$ is used to quantize via $c_i^\ell = \arg\min_j \|e_i^\ell - C_\ell[j]\|^2$, forming the SID $O(L^3)$0.
• Token Independence: Each SID position is independently quantized, allowing for bidirectional attention and parallel updating during both training and inference (Mu et al., 27 Jan 2026, Shi et al., 9 Nov 2025).

This parallel codebook design removes residual hierarchy, ensuring order-agnostic modeling and compatibility with bidirectional transformers and masked denoising diffusion.

3. Masked Diffusion-based Training

MDGR models the generation of target SIDs as a masked diffusion—Markovian corruption followed by parallel denoising:

3.1. Forward Diffusion (Noising)

• Discrete Diffusion: For a clean SID $O(L^3)$1, at each diffusion timestep $O(L^3)$2, positions are independently masked as $O(L^3)$3 with probability $O(L^3)$4, and left unmasked otherwise.
• Curriculum Masking (Temporal Dimension): The number of masked SID positions $O(L^3)$5 is drawn from a schedule $O(L^3)$6 governed by a “difficulty” scalar $O(L^3)$7, calibrated by training progression. Masking shifts from easier (fewer tokens) to harder (more tokens) over time.
• History-Aware Position Masking (Sample Dimension): For each SID position $O(L^3)$8, the masking probability is adaptively set by the frequency with which $O(L^3)$9 appears in user history $i$0, promoting harder, less-frequent positions (Mu et al., 27 Jan 2026).

3.2. Reverse Denoising

• Decoder: A $i$1-layer bidirectional Transformer (hidden size $i$2, $i$3 heads) receives the partially masked SID, user history embeddings, and a difficulty embedding $i$4. The model predicts all masked tokens in parallel.
• Loss Function: Training minimizes cross-entropy over only the masked positions, averaged over curriculum schedule and masking patterns:

$i$5

(Mu et al., 27 Jan 2026).

Similar masked diffusion schemes are found in DiffGRM (“on-policy coherent noising” focusing on hard digits), MADRec (unconditional random masking), and LLaDA-Rec (interleaved user-history- and next-item-level masking) (Liu et al., 21 Oct 2025, Shah et al., 28 Nov 2025, Shi et al., 9 Nov 2025).

4. Inference and Parallel Decoding Strategies

MDGR decouples inference from left-to-right order by employing a two-stage parallel decoding algorithm:

• Warm-up Phase: For $i$6 steps, fill one SID position per step to reduce uncertainty.
• Parallel Expansion Phase: Thereafter, at each step, fill $i$7 positions by confidence (top unfilled positions by predicted token probability).
• Beam Search Over Parallel Paths: For each active beam, all combinations of top-$i$8 candidate tokens at the selected positions are enumerated, retaining the highest-scoring global beams. This process continues until all SID positions are filled or the maximum step count is reached.
• Complexity: The process reduces computational cost from $i$9 (AR beam search) to $e_i \in \mathbb{R}^d$0 (MDGR), with $e_i \in \mathbb{R}^d$1 for practical settings (e.g., $e_i \in \mathbb{R}^d$2 for $e_i \in \mathbb{R}^d$3) (Mu et al., 27 Jan 2026).

Adaptive-order and confidence-guided decoding ensures high-throughput generation without loss in top-K accuracy. Analogous approaches appear in CPD (DiffGRM), uncertainty-based and greedy scheduling (MADRec), and dynamic beam search (LLaDA-Rec) (Liu et al., 21 Oct 2025, Shah et al., 28 Nov 2025, Shi et al., 9 Nov 2025).

5. Implementation and Hyperparameters

A concrete realization of MDGR on public and industrial-scale datasets is configured as follows:

• Encoder: Pretrained Qwen3-8B for item content ($e_i \in \mathbb{R}^d$4-dim vector), $e_i \in \mathbb{R}^d$5 subspaces of $e_i \in \mathbb{R}^d$6 dims each, $e_i \in \mathbb{R}^d$7 per codebook, yielding $e_i \in \mathbb{R}^d$8-token SIDs.
• Training Regimen: AdamW optimizer with learning rate warmup then cosine decay ($e_i \in \mathbb{R}^d$9e-$e_i$0 base LR); batch size $e_i$1 (across $e_i$2 GPUs); $e_i$3 for curriculum; $e_i$4K steps.
• Inference: Beam width $e_i$5; $e_i$6; $e_i$7; max steps $e_i$8.
• Decoder: $e_i$9 bidirectional layers, $L$0 hidden units, $L$1 attention heads; difficulty embedding table of size $L$2 (Mu et al., 27 Jan 2026).
• Production: GPU inference and codebook lookup enable throughput $L$32.5 QPS/user, matching or exceeding AR baselines.

6. Empirical Performance and Ablations

MDGR has been evaluated on Amazon Electronics, Amazon Books, and a large-scale industrial advertising dataset (see table below):

Dataset	Best Baseline (Recall@10)	MDGR (Recall@10)	Relative Lift
Amazon Electronics	0.0544	0.0583	+7.17%
Amazon Books	0.0763	0.0826	+8.26%
Industry	0.2018	0.2210	+9.54%

Ablation studies demonstrate that both curriculum masking and history-aware position selection contribute meaningfully; omitting either degrades Recall@5/NDCG@5 by 1-3%. Uniform masking is consistently inferior.

Decoding latency is tunable: full parallel decoding ($L$4) increases QPS by $L$5 compared to AR, with only marginal accuracy loss ($L$6 vs $L$7) (Mu et al., 27 Jan 2026).

Online A/B testing on an advertisement recommendation platform with 18M users and 25M items yields:

• $L$8 advertising revenue ($L$9)
• $e_i^\ell = f^\ell(e_i)$0 gross merchandise volume (GMV)
• $e_i^\ell = f^\ell(e_i)$1 click-through rate (CTR)

These results confirm the real-world value and robustness of masked diffusion-based generative approaches.

7. Related Work and Positioning

Masked Diffusion Generative Recommendation aligns with a growing body of research pursuing parallel, order-agnostic generation and improved semantic modeling:

• DiffGRM leverages parallel semantic encoding and confidence-prioritized masking to densify supervision and facilitate top-K diverse generation (Liu et al., 21 Oct 2025).
• MADRec employs random masking and diffusion-style denoising for robust learning and substantially faster inference, especially under data constraints (Shah et al., 28 Nov 2025).
• LLaDA-Rec incorporates both user-history-level and next-item-level masking in discrete diffusion, combining them with an adaptive-order beam search for synthesis (Shi et al., 9 Nov 2025).
• DeftRec studies diffusion-based generation in continuous latent space, avoiding information loss due to discretization and enabling rich, direct retrieval (Qu et al., 16 Apr 2025).

All methods emphasize bidirectional attention, parallel recovery of masked positions, adaptive difficulty-aware training, and flexible, rapid beam-search inference.

• "Masked Diffusion Generative Recommendation" (Mu et al., 27 Jan 2026)
• "DiffGRM: Diffusion-based Generative Recommendation Model" (Liu et al., 21 Oct 2025)
• "Masked Diffusion for Generative Recommendation" (Shah et al., 28 Nov 2025)
• "LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation" (Shi et al., 9 Nov 2025)
• "Generative Recommendation with Continuous-Token Diffusion" (Qu et al., 16 Apr 2025)