Hierarchical Quantization with Domain-Adaptive Sparse Routing for Generative Cross-Domain Recommendation

Abstract: Generative Recommendation (GenRec) represents a promising paradigm that achieves remarkable empirical success by encoding items as compact Semantic IDs (SIDs) and modeling user behavior via next-token prediction across diverse recommendation scenarios. Extending this paradigm to cross-domain recommendation is challenging because a unified model must accommodate heterogeneous item semantics and behavioral patterns across domains. Existing methods commonly rely on globally shared representations or lightweight domain adaptation, which may provide insufficient capacity for modeling heterogeneous patterns at different semantic granularities. To address these challenges, we propose HD-Rec, a unified generative framework for cross-domain recommendation. HD-Rec employs a hierarchical domain-aware quantizer that constructs semantic identifiers using globally shared coarse-level codebooks and adaptively routed fine-level codebooks. It further introduces a domain-adaptive sparse mixture-of-experts module that combines a continuously activated shared expert with a dynamically selected specialized expert. To improve the coherence of multi-token item representations, we develop a cross-granularity routing consistency objective that regularizes token-level routing decisions toward their item-level consensus. Experiments on three public cross-domain recommendation benchmarks show that HD-Rec consistently improves over competitive sequential, generative, and cross-domain recommendation baselines.

• The paper introduces HD-Rec, a unified generative recommender that combines domain-aware hierarchical quantization, domain-adaptive sparse experts, and cross-granularity routing consistency for heterogeneous domains.
• The model achieves the best HR@10 and NDCG@10 results across three benchmark dataset pairs, improving relative HR@10 by up to 17.6% on sparse domains while adding only 2.0–2.5% inference overhead.
• The paper shows that routed fine-level codebooks and coherent item-level expert routing drive the gains, with CRCL reducing intra-item routing variance by roughly sixfold and improving multi-token Semantic ID consistency.

HD-Rec (2608.06997) is a unified generative framework for cross-domain sequential recommendation (CDSR) that addresses a structural shortcoming of existing generative recommenders (GenRec) when applied across heterogeneous domains: globally shared semantic tokenization and dense Transformer backbones provide insufficient input-dependent capacity to capture domain-specific item semantics. The framework, developed by authors from City University of Hong Kong and Kuaishou Technology, couples three components—a hierarchical domain-aware quantizer (HDQ), a domain-adaptive sparse mixture-of-experts (DAS MoE), and a cross-granularity routing consistency loss (CRCL)—into a single next-token-prediction model over Semantic IDs (SIDs).

Motivation and problem setting

The paper targets CDSR without overlapping entities, where a unified model must predict next items in a target domain conditioned on a user's multi-domain behavior history, optimizing $\log P_\theta(v_t \mid H_u^{<t}, d_t)$. Prior approaches either rely on shared entities, PLM-derived content features, or lightweight adaptation (prompts, adapters, LoRA). The authors motivate their design with a t-SNE analysis of learned embeddings on Amazon sub-categories, showing that item embeddings exhibit both cross-domain overlap at coarse semantic levels and domain-dependent geometric structure at fine granularity. They are explicit that this visualization is qualitative motivation rather than a quantitative assessment of representation disentanglement.

Hierarchical domain-aware quantizer

HDQ extends RQ-VAE-style residual quantization with an asymmetric structure. The first $L-1$ quantization layers use globally shared codebooks to capture coarse, cross-domain semantics via nearest-neighbor lookup with residual propagation. The final layer instead offers $K$ candidate codebooks, selected per item by a lightweight MLP router trained with the Gumbel-Softmax trick and a straight-through estimator for discrete top-1 selection. Each routed codebook receives a unique offset so codewords occupy disjoint regions of the SID vocabulary. Routing collapse is prevented with a load-balancing regularizer penalizing deviation of mean routing probabilities from uniformity, and codebook/commitment losses follow the standard vector-quantization formulation with stop-gradients. Notably, codebooks are assigned by learned routing rather than to predefined domains, avoiding manual domain assignment.

Domain-adaptive sparse MoE and routing consistency

At the backbone level, each Transformer FFN is replaced by a shared-plus-routed MoE: an always-active shared expert processes every token, while a gating network deterministically selects one of $K$ specialized experts per token, preserving sparse conditional computation.

Because SIDs are multi-token, unconstrained token-level routing can assign the tokens of one item to divergent experts. CRCL addresses this with a soft KL-divergence penalty between each token's routing distribution and an item-level consensus distribution obtained by mean-pooling the item's token embeddings through the same gate. The total objective combines next-token prediction, the consistency loss ($\lambda_1 = 0.01$), and MoE load balancing.

Empirical results

Experiments use three dataset pairs—Clothing-Sports and Electronics-Phones from Amazon, Books-Movies from Douban—with leave-last-out evaluation on HR@10 and NDCG@10, against single-domain sequential baselines (BERT4Rec, SASRec, STOSA), generative baselines (VQ-Rec, TIGER, HSTU), and CDSR baselines (C2DSR, TriCDR, LLM4CDSR, GenCDR). HD-Rec achieves the best result on every domain and metric, with statistically significant improvements ($p \leq 0.05$). The strongest gains are on sparse domains: 17.6% relative HR@10 improvement over the strongest baseline (GenCDR) on Sports, 16.3% on Electronics, and 9.9% on Phones, while gains on Clothing (4.9%) and the denser Douban domains (0.4–3.6%) are smaller. This pattern supports the claim that adaptive fine-level capacity matters most where interaction data is scarce, though the near-parity with GenCDR on Douban Movies suggests the advantage of hierarchical routing diminishes when domains are dense and semantics are already well aligned.

Ablations confirm each component contributes: removing HDQ, DAS MoE, or CRCL each degrades performance on Clothing-Sports. Tokenizer design analysis is particularly informative—replacing routed codebooks with all-shared codebooks, moving them to the first layer, using random routing, or simply expanding a single final-layer codebook all underperform the full design. The last comparison is important: it indicates the gain stems from the routing structure itself, not merely increased final-layer vocabulary capacity.

Hyper-parameter analysis of $\lambda_1$ shows an inverted-U trend peaking at 0.01 (relative gains of 9.4–15.0% over $\lambda_1 = 0$), with excessive consistency regularization constraining token-level routing flexibility and hurting performance. Efficiency testing under a stress setting (beam size 200) shows only 2.0–2.5% inference overhead relative to TIGER (106.2 ms vs. 103.6 ms per instance on Clothing). A newly proposed Routing Variance (RV) metric shows CRCL reduces intra-item routing variance by roughly 6× (e.g., from $2.63 \times 10^{-4}$ to $0.429 \times 10^{-4}$ on Clothing), directly validating the consistency mechanism.

Limitations and open questions

The paper concedes several limitations. All experiments are restricted to two-domain settings, leaving the scalability and stability of the routed codebooks and experts under many heterogeneous domains untested. Performance depends on content-derived item embeddings, so noisy or incomplete metadata may degrade the tokenizer. Evaluation is purely offline on static benchmarks; behavior under dynamic catalogs, shifting domain distributions, and online feedback loops is unaddressed. The authors also note that the ablation results support the hierarchical design but do not by themselves establish representation disentanglement, and they identify direct measurement of representation specialization as an open question.

Conclusion

HD-Rec demonstrates that injecting input-dependent capacity at both the tokenization and backbone levels—via routed fine-level codebooks and shared-plus-specialized sparse experts—yields consistent improvements for unified generative cross-domain recommendation, with the largest gains on sparse domains and negligible inference cost. The CRCL objective offers a simple, effective mechanism for enforcing multi-token coherence in MoE-based generative recommenders, a concern that generalizes beyond the cross-domain setting. The principal open questions are whether the hierarchical routing structure scales beyond pairwise domain configurations and whether the learned specialization corresponds to measurable semantic disentanglement.