Generative Archetype-Grounded Item Representations for Sequential Recommendation

Abstract: Sequential recommendation aims to predict users' next interaction with items by analyzing their historical behavior. However, the limited quality of item representations remains a critical bottleneck. While pre-trained LLMs can provide rich semantic representations, existing approaches only rely on static encoding of fixed attributes, overlooking the crucial role of target audiences in defining item identity. Moreover, the semantic space struggles to reflect actual user behavior, resulting in a significant gap between semantic representations and behavioral patterns. To address these limitations, we propose GenAIR, a general framework that empowers sequential recommendation with Generative Archetype-grounded Item Representations. Specifically, we first leverage an LLM to analyze item metadata and infer textual description of the Archetype, which represents the conceptual profile of the item's ideal target audience. We then extract the corresponding embeddings in a single forward pass. Further, to ground these generative archetypes in real-world behavior, we introduce a behavioral calibration objective, which explicitly incorporates behavioral signals from actual interactions. This objective adjusts the structure of the embedding space to reflect empirical patterns. GenAIR enables seamless integration with most existing models while maintaining high efficiency. Comprehensive experiments conducted on three real-world datasets demonstrate that GenAIR significantly improves the performance of various sequential recommendation models and consistently outperforms state-of-the-art baseline approaches. Implementation codes are available at https://github.com/AI-Santiago/GenAIR.

• The paper introduces GenAIR, which uses LLMs to generate latent archetypes from item metadata and improve sequential recommendation quality.
• It employs a three-stage process—archetype generation, semantic embedding extraction, and behavioral calibration—that significantly enhances metrics like HR@10 and NDCG@10.
• The approach effectively addresses data sparsity and long-tail item challenges while maintaining efficiency through offline archetype caching.

Generative Archetype-Grounded Item Representations for Sequential Recommendation

Introduction and Motivation

The effectiveness of sequential recommendation models is often constrained by the ability to construct high-quality item representations that generalize well, particularly given data sparsity and the skewed distribution of real-world user interactions. While recent approaches have begun integrating LLMs to encode item textual metadata, existing paradigms suffer from several fundamental limitations: reliance on static attribute encodings, a sharp semantic-behavioral embedding mismatch, and underutilization of LLMs’ generative and reasoning capabilities.

The paper "Generative Archetype-Grounded Item Representations for Sequential Recommendation" (2606.11023) introduces GenAIR, a general framework that directly addresses these challenges by harnessing the generative capacity of LLMs to instantiate latent user archetypes per item and calibrates these embeddings with real user behavioral signals. This approach aligns the semantic richness from LLMs with empirical user interaction patterns, thereby enhancing recommendation accuracy and robustness across diverse settings.

Figure 1: A comparison of item representation strategies, highlighting how GenAIR grounds semantic representations via a behavioral calibration projector $p_\theta$.

GenAIR Framework Architecture

GenAIR is decomposed into three core algorithmic stages: (1) Archetype Generation using generative LLM inference, (2) Construction of archetype-grounded semantic embeddings, and (3) Behavioral Calibration to bridge semantic and behavioral preference signals.

1. Archetype Generation: For each item, metadata attributes such as name, brand, and descriptive text are serialized into a designated prompt template. An LLM generates a textual description of an "archetype," abstracting the prototypical user profile that the item is marketed toward—an approach rooted in the STP (Segmentation, Targeting, Positioning) marketing principle. This generative step leverages the LLM’s reasoning capacity to bridge static item attributes and anticipated audience segmentation.
2. Semantic Embedding Extraction: Both the serialized item metadata (prefill context) and the generated archetype text (decoding context) are encoded in a single forward LLM pass, sourcing hidden states from the final Transformer layer. Dimensionality reduction via PCA (maintaining 95% variance) and concatenation of prefill and decoding embeddings yield a semantic representation that encapsulates both factual and inferred user-group features. These embeddings are then mapped into the sequential model’s latent space through a trainable MLP-based projector, $p_\theta$.

   Figure 2: Overview of GenAIR, detailing sequence modeling, generative archetype production, and behavioral calibration.

3. Behavioral Calibration: To reconcile the gap between semantic similarity and actual co-interaction patterns, GenAIR introduces a behavioral calibration loss, $\mathcal{L}_\mathrm{cal}$. This loss incorporates pairwise co-occurrence statistics—captured via log-normalized association scores—between items within the same user context. The objective enforces differential repulsion in the embedding space: items frequently co-engaged by users are mapped closer, while others retain geometric separation. The calibration kernel parameterizes this interaction to avoid uninformative uniformity assumptions imposed by naive diversity-based objectives.

Empirical Evaluation and Analysis

Extensive experiments on three real-world datasets (Yelp, Amazon Beauty, Amazon Fashion) benchmark GenAIR against both classic and recent state-of-the-art baselines, including traditional collaborative filtering models and a range of LLM-based embedding and alignment strategies (e.g., LLM-ESR, LLMEmb, AlphaFuse).

Key Numerical Results:

• Across HR@10 and NDCG@10, GenAIR demonstrably outperforms all baselines across all backbone models (GRU4Rec, Bert4Rec, SASRec), with statistically significant improvement margins (often exceeding 2-7%) over the strongest LLM-based alternatives.
• Ablation studies show that omitting either the prefill or decoding embeddings, or removing behavioral calibration, consistently degrades performance, underscoring the necessity of each GenAIR component.

Figure 3: t-SNE visualization of GenAIR representation space for Amazon Fashion and Yelp datasets. Prefill and decoding embeddings demonstrate natural separation and diversity across items.

Figure 4: Sensitivity analysis on the behavioral calibration loss coefficient $\alpha$ shows optimal performance at intermediate values, validating the need for carefully balanced calibration.

Additional analysis visualizes the learned semantic spaces and demonstrates that GenAIR’s archetype-enhanced embeddings prevent feature collapse, especially for long-tail (infrequently interacted) items, resulting in broader and more informative group distributions.

Figure 5: PCA visualization of group-specific item embeddings and analysis of matrix entropy, demonstrating superior information retention and diversity with GenAIR’s representations across head and long-tail item strata.

Computational Efficiency and Practical Considerations

GenAIR maintains inference efficiency on par with conventional embedding models. All archetype representations are generated and cached offline, so LLM inference is not required at serving time. Only a minor parameter overhead is introduced in the embedding projector compared to lightweight baselines, and the approach avoids the substantial compute cost of end-to-end LLM fine-tuning and heavy post-hoc sequence modeling.

Implications and Future Directions

Practical Implications:

• GenAIR’s modular structure enables seamless deployment as a drop-in replacement for item representation layers in existing sequential recommendation pipelines.
• The behavioral calibration mechanism makes GenAIR robust to interaction sparsity and alleviates long-tail item cold start by leveraging both world knowledge and empirical feedback.

Theoretical Implications:

• The generative archetype approach formalizes a bridge between marketing-driven audience abstraction and large-scale behavioral modeling—a conceptual advancement over static attribute or meta-data-only representations.
• The calibration loss introduces a behavior-informed geometric regularization, suggesting broader applicability to other tasks requiring alignment between structured priors and empirical interaction graphs.

Future Directions:

• Integrating user-side generative archetypes or adapting the approach for cold-start users is a promising extension.
• As LLMs continue to improve contextual comprehension and domain adaptation, leveraging more advanced prompting and multi-modal item descriptors may further increase representation quality.
• Applying GenAIR-style behavioral calibration to other foundation model embedding scenarios (e.g., for multi-modal, graph, or knowledge-augmented recommendation) represents a natural generalization pathway.

Conclusion

GenAIR represents an important advance in sequential recommender systems, systematically combining LLM-based generative reasoning for audience archetype construction with empirical behavioral calibration. By producing archetype-grounded embeddings that reflect both semantic richness and behavioral reality, GenAIR consistently improves recommendation accuracy, robustness for the long tail, and practical deployment efficiency. This work lays a foundation for further research at the intersection of foundation models, behavioral regularization, and real-world recommender system design.