Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation

Abstract: Generative recommendation is an emerging paradigm that has shown promise in industrial recommendation systems, aiming to predict users' next interactions from their historical behaviors. At the core of generative recommendation lies item tokenization, which bridges item semantics and recommendation models. However, existing methods often struggle to effectively organize and inject complex user-behavioral and item-semantic contexts into recommendation models simultaneously. On the one hand, existing graph-based integration methods, such as graph serialization and graph neural networks, either suffer from scalability issues or exploit only local graph information. On the other hand, existing semantic tokenization methods typically rely on heuristics and lack explicit supervision signals, which may lead to inaccurate or suboptimal semantic representations. To address these limitations in user interest context modeling, we propose G2Rec, a scalable framework that unifies holistic graph-based user co-engagement modeling with semantic tokenization for industrial-scale generative recommendation. Overall, G2Rec enables recommendation models to capture holistic and semantically grounded user interest prototypes without requiring ground-truth user interests, thereby providing more comprehensive and accurate modeling of user behavior contexts in industrial sequential recommendation. Online deployment across product surfaces and extensive experiments on public datasets demonstrate the superiority of G2Rec over existing methods.

• The paper introduces G2Rec, a novel approach that integrates sparsified co-engagement graphs, soft clustering, and tokenization to model dynamic user interests in generative recommendation.
• It employs scalable graph sparsification and differentiable soft clustering to preserve key Laplacian spectral properties and achieves up to 14.9% higher NDCG@5 compared to baselines.
• Empirical evaluations confirm its industrial feasibility with negligible computational overhead and measurable improvements in user engagement across diverse datasets.

Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation

Generative Recommendation and Context Integration

The paper "Structuring and Tokenizing Distributed User Interest Context for Generative Recommendation" (2606.20554) introduces G2Rec, an industrial-scale generative recommendation architecture that unifies holistic graph-based user co-engagement modeling with semantic tokenization. Generative recommendation leverages autoregressive language modeling to sequentially predict user interactions conditioned on historical behavior—critical in domains such as e-commerce, streaming, and online advertising.

Traditional approaches fail to effectively structure and inject user-behavioral and item-semantic contexts into generative models. Graph-based integration (user-item bipartite or item co-engagement graphs) suffers from scalability bottlenecks or only captures local context. Semantic tokenization, often reliant on heuristic objectives without reliable supervision, produces suboptimal semantic item representations. G2Rec systematically addresses these deficiencies by combining sparsified item co-engagement graph construction, scalable soft clustering for distributed interest profiling, and interest profile tokenization.

Sparse Item Co-Engagement Graph Construction

G2Rec models user behavior via an item-item co-engagement graph, where each edge represents two items jointly engaged by a user. This paradigm captures nontrivial behavioral relations, such as accessory purchases conditioned on primary items, transcending pure item similarity. To tackle scalability for industry-scale deployment, G2Rec applies theoretical graph sparsification: sampling $\mathcal{O}(M \log M)$ edges, with $M$ denoting total user-item interactions—orders of magnitude fewer than quadratic in $M$ for the full graph. A theoretical guarantee is established: Laplacian spectral properties, underpinning structural graph information, are preserved with high probability under this sampling regime.

Figure 1: Visualization of sparse co-engagement contexts on the Beauty dataset, highlighting preserved relational structure after graph sparsification.

Scalable Soft Graph Clustering for Distributed Interest Profiling

G2Rec employs a scalable soft clustering algorithm to decompose the co-engagement graph into distributed item interest prototypes. Unlike classic hard clustering (Louvain, Leiden), soft clustering assigns continuous membership distributions $p_i$ for each item across $C$ prototypes—allowing multi-faceted semantic representation. The clustering objective extends modularity to the differentiable setting, and is efficiently computable in $\mathcal{O}(\rho M \log M)$ time (with $\rho$ denoting sparsity). The soft modularity yields interpretable clusters, encoding complex user interests and capturing nuanced transitions in user behavior sequences.

Ablations confirm the efficacy of soft clustering: modularity scores are consistently superior to hard clustering across benchmarks.

Figure 2: Ablation analysis of the loss weight $\lambda$ for $\mathcal{L}^t_\text{profile}$, confirming optimal tokenization for interest profiling.

Interest Profile Tokenization and Generative Modeling

G2Rec introduces tokenization of item interest profiles: for each user interaction sequence, the model alternates between item embeddings and their associated interest profile tokens (weighted averages over prototype embeddings). The generative sequential recommender is trained to predict the next item conditioned on this enriched sequence, jointly optimizing cross-entropy losses for item prediction and soft interest profile classification. This tokenization exposes underlying behavioral transitions to the autoregressive model, enabling fine-grained modeling of user intent.

The approach obviates the need for explicit ground-truth user interests, leveraging graph-structured behavioral context to derive semantically grounded interest prototypes.

Empirical Results and Numerical Evaluation

Comprehensive experiments across public datasets (Beauty, Sports, Toys, Yelp) and large-scale online A/B tests validate G2Rec's superiority. Quantitative metrics (Recall@k, NDCG, MRR) demonstrate significant improvement over classic, sequential, tokenization-based, and graph-enhanced baselines. For example, G2Rec achieves up to 14.9% higher NDCG@5 on Sports vs the strongest baseline. Average rank consistently favors G2Rec across all metrics/datasets.

Moreover, efficiency evaluations reveal negligible computational overhead—training time increases by only 0.043 s per batch, and inference by 0.0027 s per batch—making G2Rec practical for industrial deployment.

Figure 1: Dataset-level recall curves show G2Rec consistently outperforms established baselines under all cutoffs.

Industrial Deployment and Scalability

G2Rec is deployed at scale across several Meta product surfaces, serving billions of monthly active users. The clustering and profiling are executed offline, ensuring real-time feasibility for user response. Real-world A/B tests show measurable lifts in user engagement, content diversity, and serving efficiency: increases in metrics such as total time-spent, likes, and shares (up to 0.19%) validate G2Rec's impact on user experience.

Implications and Future Directions

G2Rec advances generative sequential recommendation by structurally organizing distributed user interest context and semantic item representation via scalable graph modeling and tokenization. The implications are twofold: (1) Practical—enabling robust, scalable deployment in industrial recommender systems; (2) Theoretical—offering guarantees for information preservation via graph sparsification and modular soft clustering.

Future work may include refinements in prototype extraction (e.g., using richer clustering objectives), incorporation of multimodal side information (content, social signals), and adaptive online modeling to capture temporal interest drift.

Conclusion

G2Rec provides a rigorous and scalable methodology for integrating complex relational user-item contexts within generative recommendation frameworks. By leveraging graph sparsification, differentiable modularity-based soft clustering, and semantic tokenization of item interest profiles, it significantly advances the state-of-the-art in industrial recommendation—delivering strong empirical results and real-world impact on user engagement and serving efficiency. Future developments may focus on deeper semantic modeling and multimodal integration to further enhance generative recommendation capabilities.