Multi-Interest Aggregator in Recommendation Systems

• Multi-Interest Aggregator is a framework that decomposes user history into distinct interest vectors, enabling nuanced personalization and diversity.
• It uses techniques like dynamic routing, self-attention, and clustering to extract fine-grained behavioral patterns from user interactions.
• Industrial implementations (e.g., Alibaba, Pinterest) demonstrate that these methods enhance recommendation accuracy, diversity, and scalability at billion-scale.

A multi-interest aggregator is a class of neural network modules and algorithmic frameworks used in modern recommendation systems to extract, represent, and strategically fuse multiple facets of user preferences from behavioral sequences. Unlike traditional single-embedding approaches, which collapse all of a user’s history into a single latent vector, multi-interest aggregators are explicitly designed to model diversity, dynamics, and the fine-grained structure within a user’s profile—enabling improved accuracy, recommendation diversity, and personalization in large-scale environments.

1. Conceptual Foundations and Motivations

Multi-interest aggregation emerges from the empirical observation that real users engage with multiple, often heterogeneous, item categories and patterns over time. Single-vector user representations tend to average out distinct interests, thus missing nuanced or niche preferences, suppressing diversity, and reducing explainability (Li et al., 18 Jun 2025). A multi-interest aggregator decomposes a user’s history into several latent vectors (often called “interest vectors” or “capsules”), each capturing a unique behavioral pattern or topical affinity. The multi-interest paradigm is theoretically inspired by capsule networks (dynamic routing) and attention mechanisms, which assign weights or routing coefficients to different input elements to construct distinct, complementary representations (Cen et al., 2020, Hao et al., 2021).

2. Core Architectures and Extraction Strategies

The extraction of multi-interest representations typically follows one of several key methodologies:

• Dynamic Routing: An iterative process (as in Capsule Networks) that assigns behavioral embeddings to a set of interest capsules based on coupling coefficients, followed by a squashing function to normalize each vector (Cen et al., 2020, Hao et al., 2021). The internal routing optimizes

$$\hat{e}_{j|i} = W_{ij} e_i; \qquad v_j = \text{squash}(s_j) = \frac{||s_j||^2}{1 + ||s_j||^2} \frac{s_j}{||s_j||}$$

• Self-attention or Multi-head Attention: Item embeddings from a user’s historical sequence are aggregated using an attention matrix that is either fixed-size (producing $K$ interest vectors) or adaptively learned (Cen et al., 2020, Tian et al., 2022). The extraction is typically

$$A = \mathrm{softmax}(W_2^\top \tanh(W_1 H)), \quad V_u = H A$$

where $H$ is the item embedding matrix and $A$ contains attention weights for each interest.

• Clustering and Differentiable Clustering Modules: Behavioral item vectors are grouped into “interest clusters” either with hard assignments (Fan et al., 29 Jun 2025) or via learned differentiable routing (Fan et al., 29 Jun 2025). The association between clusters and interests is data-driven and may use routing coefficients or winner-take-all strategies.
• Graph-based or Multi-Grained Models: User behavior sequences are represented as graphs (nodes = items) with edge weights indicating affinity. Graph convolutional layers propagate multi-level signals, followed by capsule modules to decompose multi-scale representations (Tian et al., 2022).

Modern systems frequently stack or hybridize these techniques to leverage both sequential correlations (short-term signals) and multiscale behavioral affinity (long-term or cross-domain signals) (Yuan et al., 2022, Xu et al., 16 Oct 2025).

3. Aggregation Mechanisms and Controllability

After multi-interest vectors are extracted, an aggregation mechanism is required to combine, weight, or select among candidate retrieval sets for final recommendation output. The aggregation strategies can be organized as:

• Accuracy-maximizing selection: For each candidate item, the maximum similarity across all interest vectors is used:

$f(u, i) = \max_k (e_i^\top v_u^{(k)})$

providing maximal relevance (Cen et al., 2020).

• Controllable Trade-off between Accuracy and Diversity: An explicit diversity term is introduced, such as

$$Q(u, S) = \sum_{i \in S} f(u, i) + \lambda \sum_{i,j \in S} g(i, j)$$

with $g(i, j)$ measuring diversity (e.g., category disparity), and $\lambda$ tuning the accuracy/diversity balance. Greedy or combinatorial optimization is used to construct the candidate set (Cen et al., 2020).

• Attention-based Aggregation: Aggregation weights across interest vectors are computed via context-dependent attention:

$w_i = \frac{\exp(s_i)}{\sum_{j=1}^K \exp(s_j)}$

where $s_i = q^\top v_i$ reflects contextual alignment (query $q$ may encode session context or temporal focus) (Shi et al., 2022, Dong et al., 9 Jan 2024).

• Prompt-based Conditioning: Recent work introduces learnable prompt embeddings prepended to user interactions, priming extractor and aggregator for distinct objectives and improving the capacity to disentangle, weight, or fuse interests according to specific aggregation needs (Dong et al., 9 Jan 2024).
• Dictionary/Quantization-based Retrieval: Interest quantization discretizes the user-interest space, enforcing structural separation (Voronoi partitioning) and enabling lookup-based retrieval over a shared dictionary, with additional latent interest generation modules to model evolving or latent interests (Wu et al., 16 Oct 2025).

4. Promoting Diversity and Avoiding Interest Collapse

A central challenge is “interest collapse,” where multiple user interest vectors converge to similar points, thus losing their ability to cover diverse behavioral modes. To counteract this, various regularization strategies are employed:

• Orthogonality/Contrastive Regularization: Pairwise penalization for non-orthogonality,

$$\mathcal{L}_{\mathrm{orth}} = \sum_{p \neq q} (v_p^\top v_q)^2$$

or InfoNCE-style contrastive terms to push apart different interest vectors (Hao et al., 2021, Li et al., 18 Jun 2025).

• Clustering-based Separation: Additional clustering losses ensure intra-interest compactness and inter-interest separation in latent space (Hao et al., 2021).
• User Representation Repel Loss: In multi-tower architectures, a repel term in the training loss explicitly pushes user representations away from each other unless they correspond to the same interest, implemented with triplet-based losses (Xiong et al., 8 Mar 2024).
• Quantization/Dictionary Partitioning: Vector-quantized codebooks guarantee minimal pairwise distance between distinct interests (a strict partition via discrete code indices), as detailed in the theoretical results of (Wu et al., 16 Oct 2025).

These methods are backed by empirical findings showing that controlling for redundancy and promoting orthogonality directly improves both accuracy and diversity, and can also boost generalization across domains and distributions (Hao et al., 2021, Liu et al., 2022).

5. Industrial Deployments and Scaling Properties

Multi-interest aggregators have moved from academic research into production systems:

• Alibaba Cloud and Taobao: ComiRec and variants have been deployed at billion-scale, where each user’s multi-interest vectors are used to retrieve candidate items via approximate nearest neighbor search; a controllable aggregation policy ensures a balance between accuracy and diversity on industrial data (Cen et al., 2020, Jiang et al., 2022).
• Pinterest Home Feed: A dual-pathway (implicit via differentiable clustering, explicit via condition/topic) multi-embedding retrieval engine aggregates diverse user interests for candidate recall. Merged candidates from both implicit and explicit pathways demonstrably improve recall, long-tail coverage, and engagement (Fan et al., 29 Jun 2025).
• Douyin Recommendation: The Trinity framework statically “remembers” long-term user interests, long-tail preferences, and multi-interest signals via hierarchical clustering/statistical histograms, ensuring that short-term bias and “interest amnesia” do not erode diversity in production (Yan et al., 5 Feb 2024).
• Xiaohongshu (Rednote): Deployment of GemiRec—a generative quantized aggregator—supports strict interest separation and latent interest evolution, yielding measurable business gains for click-through and engagement metrics at scale (Wu et al., 16 Oct 2025).

Key factors in industrial adoption include architecture modularity (for plug-and-play replacement with two-tower systems), real-time candidate retrieval cost (latency per user request), and the ability to cache, quantize, and efficiently update user representations as new behavior data arrives.

6. Evaluation, Empirical Results, and Trade-offs

Empirical evidence across public and proprietary datasets indicates that multi-interest aggregators demonstrate consistent improvements over single-embedding and single-tower baselines:

• Quantitative Gains: Aggregators like ComiRec, HCN, and GemiRec report superior values in Recall, NDCG, Hit Rate, GMV uplift, and engagement metrics (Cen et al., 2020, Yuan et al., 2022, Wu et al., 16 Oct 2025, Fan et al., 29 Jun 2025).
• Diversity vs. Accuracy: Aggregation strategies including controllable factors ($\lambda$) allow precise tuning of the diversity-accuracy trade-off (increased diversity can cause a marginal drop in recall but substantially increases coverage and user satisfaction) (Cen et al., 2020, Dong et al., 9 Jan 2024).
• Stability and OOD Robustness: Methods such as DESMIL use dependence measures (HSIC) and sample weighting to reduce spurious dependencies among captured interests, leading to improved out-of-distribution stability (Liu et al., 2022).
• Efficiency and Scalability: Aggregators support billion-scale serving via modular design, quantization-based caching, and separation of interest extraction from aggregation. Methods like Trinity and GemiRec report negligible online latency overhead under production loads (Yan et al., 5 Feb 2024, Wu et al., 16 Oct 2025).
• Fairness and Personalization: Architectures employing “virtual” or adaptive interest embeddings help mitigate the unfairness where users with wide-ranging interests receive lower-quality recommendations, achieving better trade-offs between fairness and recommendation utility (Zhao et al., 21 Feb 2024).

7. Challenges, Limitations, and Research Directions

While multi-interest aggregators have demonstrated progress, several challenges persist:

• Adaptive Interest Number: Most methods fix the number $K$ of interest vectors; dynamic, user-adaptive determination remains a key research direction (Li et al., 18 Jun 2025).
• Data Sparsity and Overfitting: For users with sparse interactions, interest clustering or alignment with LLM-driven semantic clusters (as in (Wang et al., 15 Jul 2025)) is used to synthesize robust representations.
• Interest Amnesia and Over-specialization: Statistical aggregation over long windows (as in Trinity (Yan et al., 5 Feb 2024)) or blending prompt-based conditioning (Dong et al., 9 Jan 2024) helps resist short-term bias and over-specialization.
• Explainability: Mapping each retrieved item back to the responsible user interest vector or semantic cluster enhances explainability and transparency—but requires careful design of extraction and matching logic (Li et al., 18 Jun 2025, Wu et al., 16 Oct 2025).
• Integration with LLMs and Multimodal Signals: LLM-driven multi-interest frameworks align semantic, collaborative, and behavioral signals at both user and population levels (Wang et al., 15 Jul 2025, Xu et al., 16 Oct 2025). The challenge is balancing granularity, coverage, and efficiency.
• Cold-Start/Long-Tail Adaptation: Explicit modeling of domain-invariant versus domain-specific interests, as well as real-time statistical clustering, enables better handling of new users and long-tail content (Yan et al., 5 Feb 2024, Jiang et al., 2022).

Summary Table: Core Multi-Interest Aggregation Strategies

Architecture/Method	Extraction Principle	Aggregation/Control
Dynamic Routing (ComiRec)	Capsule network routing	Max/Greedy + Diversity
Self-Attn. (ComiRec-SA, MGNM)	Multi-head/self-attention	Max/Attention Weights
Prompt-based Aggregator	Learnable prompts	Weighted sum via prompts
Quantization (GemiRec)	Residual dictionary	Lookup/code-based merge
Clustering (Trinity, DCM)	K-means/learned clusters	Histogram-based recall
Cross-scenario Fusion (RED-Rec)	Multi-tower LLM+queries	Scenario-aware queries

This table summarizes major variants present in the literature, all of which align on the core objective: transforming multi-modal, large-scale behavioral histories into a set of diverse, expressive interest representations, and aggregating their outputs to achieve an optimal blend of accuracy, diversity, user coverage, and scalability.