--- title: Generative Recommendation Systems url: https://www.emergentmind.com/topics/generative-recommendation-systems-grss type: topic --- # Generative Recommendation Systems Generative Recommendation Systems (GRSs) reconceptualize recommendation as a conditional generation problem, leveraging generative models—particularly large language models (LLMs), large recommendation models, and diffusion models—to generate recommended items or content directly rather than ranking a set of candidates. This paradigm employs unified Transformer-based architectures and follows explicit scaling laws, supporting end-to-end modeling, multimodal integration, and reasoning capabilities that go beyond the scope of classic discriminative recommenders [2510.27157][2502.13783]. ## 1. Generative Paradigm versus Traditional Approaches Traditional recommendation systems typically estimate user–item affinity using discriminative scoring functions and select top-ranked items from a fixed candidate set. In contrast, GRSs model the conditional probability of recommendation sequences or outputs, generating identifiers or content by sampling or beam-searching over the vast generative space: \[ P(y\mid x)\quad\text{(generative)}, \] \[ f(u,i)\quad\text{(traditional discriminative scoring)}. \] GRSs do not assume a closed candidate set and can produce novel or highly personalized recommendations by exploiting model-internal knowledge and learned world semantics [2510.27157]. This shift enables supporting conversational, creative, and explainable tasks difficult for discriminative pipelines. ## 2. Data Foundations and Tokenization The transition to generative modeling places novel requirements on data representation, particularly item and user tokenization. In GRSs, item identifiers (tokens) must encode semantic and collaborative signals, enabling both content understanding and behavior modeling. Approaches include: - **ID-based tokenization**: Uniquely indexes items without semantics; fails under cold start [2405.07314]. - **Text-based tokenization**: Uses item titles/descriptions; contains rich semantics but incurs length bias and lacks collaborative differentiation [2405.07314]. - **Codebook-based tokenization**: Employs vector-quantized (e.g. RQ-VAE) representations to discretize item embeddings into short semantic ID sequences, supporting hierarchical, content-aware, and collaborative-aware tokenization [2509.25522][2511.15122][2405.07314]. The LETTER tokenizer [2405.07314] exemplifies a learnable codebook system, integrating semantic regularization, collaborative contrastive loss, and diversity regularization to enable robust, generation-friendly identifiers. Modern GRSs also address code-assignment bias, cross-modal (multimodal) tokenization [2511.15122][2503.23333], and collision mitigation. ## 3. Generative Architectures and Training Methodologies GRSs predominantly utilize large Transformer architectures, including both decoder-only (causal language models) and encoder–decoder models. Systemically, these models ingest user histories and context—expressed via sequences of item or action tokens—and decode target item sequences or attributes in an autoregressive fashion [2507.06507][2511.15122][2509.25522]. Feature highlights include: - **Next-token prediction objectives**: Maximize the conditional log-likelihood of target item codes, with extensions to page-wise or session-wise grouping for denser supervision and reduced label ambiguity [2604.14878][2508.11977]. - **Preference optimization**: Reinforcement learning methods (e.g., GRPO-SR [2604.14878], PARS and MSRA [2508.15308], DPO, listwise direct preference optimization [2602.08575]) align generative policies with nuanced multi-level feedback and business metrics. - **Reflection and correction**: Structured decoding strategies with reflection–correction loops (e.g., GRC [2602.23639]) counteract exposure bias and enable trajectory repair. - **Multimodal modeling**: Cross-modal quantization (MACRec [2511.15122]), contrastive alignment, and late/early fusion [2503.23333], underpin multi-view sequence generation for diverse item content. Training leverages supervised fine-tuning (MLE), contrastive/InfoNCE objectives for negative mining, reinforcement signals for value alignment, and flow-matching via generative flow networks (GFlowGR [2506.16114]) to mitigate exposure bias by exploring plausible positives never seen in logs. ## 4. Retrieval, Inference, and System Engineering GRS inference faces operational and system-level challenges arising from the generative search space and the need for scalable deployment: - **Efficient retrieval**: Hybrid architectures (e.g., RankGR [2602.08575]) decompose retrieval into initial assessment (coarse scoring via next-token prediction, possibly listwise), followed by refined scoring through deep candidate–context interaction. - **Constrained decoding**: Decoding is often restricted to valid codepaths by Trie-based approaches to ensure only real-world items are generated [2405.07314]. - **Optimization on hardware**: Systems such as TurboGR [2605.13433] address "jagged" data structures, dynamic load-balancing, high-throughput negative sampling, and NPU/GPU utilization, supporting model/distributed training at 0.2B+ parameters with near-linear scalability. - **Cold-start adaptation**: Model editing approaches (GenRecEdit [2603.14259]) patch next-token generation for cold items by position-wise editing in Transformer FFNs, circumventing costly retraining. Baseline industrial deployments (e.g., TencentGR-10M [2604.04976], JD App [2604.14878], Taobao [2508.11977][2602.08575]) employ scalable inference through approximate nearest neighbor (ANN) vector search, hierarchical sparse parallelism, and asynchronous communication, sustaining real-time throughput at ~10,000 QPS. ## 5. Reasoning, Multimodality, and Task Diversity GRSs have extended the expressivity of recommendation toward reasoning, explainability, and task generality: - **Reasoning architectures**: REG4Rec [2508.15308] introduces MoE-based parallel quantization, diversified reasoning path exploration, and consistency-oriented self-reflection for high-confidence, robust recommendations. - **Multimodal fusion**: Strong empirical evidence (e.g., MGR-LF++ [2503.23333], MACRec [2511.15122]) shows >20% improvement when leveraging cross-modal tokenization and alignment, using contrastive objectives and special modality-marking tokens to preserve separability during generation. - **Task diversity**: GRSs encompass slates, ranked lists, conversational dialogs, and even creative item or image generation (GEMRec [2308.02205]). The two-stage prompt-model retrieval and generated-item ranking enables personalization amid “infinite” generative possibilities. Evaluation is multi-faceted, encompassing recall, NDCG, diversity, hallucination rates, and preference alignment, as well as online metrics such as click-through and conversion. ## 6. Scaling Laws, Model Bottlenecks, and Foundation Models Empirical studies [2502.13783][2509.25522][2510.27157] elucidate the scaling behavior of GRSs: - **Scaling laws**: Performance (cross-entropy loss, recall@K) improves sublinearly with log(model capacity) and log(training data), e.g., \[ L(N) \simeq L_\infty + a N^{-\alpha} \] with typical exponents $\alpha\in[0.05,0.1]$, and recall@K rising logarithmically. - **SID bottleneck**: SID-based GR architectures saturate early in scaling due to limited code capacity for semantic information, regardless of encoder or quantizer size [2509.25522]. - **LLM-based GR**: Direct fine-tuning of large decoder-only language models to generate item textual identifiers (“LLM-as-RS”) surpasses SID-based scaling limits, capturing both content understanding and collaborative filtering without explicit tokenization, but at increased inference cost [2509.25522][2507.06507]. - **Hybrid and foundation directions**: Ongoing work explores learnable tokenizers (LETTER [2405.07314]), joint code-continuous representations, and large unified backbones for multi-task, multi-modal recommendation [2510.27157]. ## 7. Challenges, Practical Considerations, and Future Directions Current research surfaces several open challenges: | Challenge | Context | Example Approaches | |--------------------------|------------------------------------------------------------------------|---------------------------------| | Data quality & diversity | Need for scalable, high-quality logs and multi-modal coverage | Data distillation, augmentation | | Robustness & fairness | Bias, cold start, and adversarial perturbations | Model editing, fairness-aware | | Computation efficiency | Training/inference at hundred-billion parameter scale | TurboGR, MoE, quantization | | Evaluation methodology | Lack of large, realistic, multi-turn and multi-modal benchmarks | TencentGR datasets | Scaling, continual adaptation, interpretability of generation, and robust human-in-the-loop alignment remain at the research frontier [2510.27157][2507.06507][2502.13783]. Practical deployments (JD App [2604.14878], Tencent Ads [2604.04976], Taobao [2508.11977][2602.08575]) and shared open-source benchmarks have established public testbeds for continued advances, while the best practice is to combine LLM-powered reasoning, learnable tokenization, business-specific reward modeling, and hardware-software co-design. --- **Key references**: - [2510.27157] A Survey on Generative Recommendation: Data, Model, and Tasks - [2502.13783] Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation - [2509.25522] Understanding Generative Recommendation with Semantic IDs from a Model-scaling View - [2405.07314] Learnable Item Tokenization for Generative Recommendation - [2511.15122] Multi-Aspect Cross-modal Quantization for Generative Recommendation - [2604.04976] Tencent Advertising Algorithm Challenge 2025: All-Modality Generative Recommendation - [2508.15308] REG4Rec: Reasoning-Enhanced Generative Model for Large-Scale Recommendation Systems - [2604.14878] GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation - [2605.13433] TurboGR: An Accelerated Training System for Large-Scale Generative Recommendation Source: https://www.emergentmind.com/topics/generative-recommendation-systems-grss