Category-Aware Hierarchical Dual-Interest Learning

• The paper introduces a module that partitions ultra-long user histories into category-specific clusters to resolve computational complexity and semantic dilution.
• It employs a dual-interest design where Multi-head Self-Attention captures recent behaviors and Target-Attention conditions long-term interest extraction on short-term intent.
• Empirical results on the Taobao dataset show improved hit rates and online performance, demonstrating the method's viability for large-scale industrial recommendations.

Searching arXiv for the cited papers and closely related work to ground the article. arxiv_search.query({"search_query":"ti:\"User Long-Term Multi-Interest Retrieval Model for Recommendation\" OR (Meng et al., 14 Jul 2025)","max_results":5,"sort_by":"relevance"}) arXiv search: ULIM and related category-aware hierarchical recommendation papers. Category-Aware Hierarchical Dual-Interest Learning is a user representation module for retrieval-stage recommendation that organizes ultra-long behavior histories into category-conditioned interest clusters and jointly learns short-term and long-term user preferences. It is introduced as the training-side backbone of the User Long-term Multi-Interest Retrieval Model (ULIM), whose central objective is to make thousand-scale sequence modeling feasible in industrial retrieval, not only in ranking. The mechanism is category-aware because it partitions long histories by category, hierarchical because it models both token-level behaviors and cluster-level subsequences, and dual-interest because it learns a short-term interest embedding and a long-term interest embedding in parallel (Meng et al., 14 Jul 2025).

1. Problem setting and design rationale

The method is motivated by a gap between ranking and retrieval. In the formulation used by ULIM, ranking models can already exploit behavior histories with lengths in the thousands, whereas retrieval models usually remain limited to tens or at most hundreds of actions. Two retrieval-specific constraints are emphasized. First, retrieval must search over tens of millions of items under a strict real-time latency budget, so complex long-sequence interaction mechanisms are difficult to afford online. Second, conventional retrieval is typically framed as next-item prediction without target-aware cross-interactions, which prevents direct reuse of ranking-style hierarchical simplifications such as narrowing search by category (Meng et al., 14 Jul 2025).

Within that setting, Category-Aware Hierarchical Dual-Interest Learning addresses two modeling failures of a single monolithic user embedding. One is computational: a very long heterogeneous sequence is expensive to process in retrieval. The other is semantic: training one vector against diverse clicked items introduces gradient conflict and diluted semantics. The proposed answer is to partition the long sequence into category-aware subsequences, each treated as an interest cluster, and then learn long-term category-conditioned preference jointly with recent intent.

This decomposition is conceptually close to broader hierarchical preference models that separate fine-grained item-level preference from coarser category-level preference, although the framing differs. For example, HPM explicitly models low-level preference over item IDs and high-level preference over item categories with dual transformer streams and dual contrastive objectives, but it is presented as sequential recommendation rather than retrieval-stage long-history compression (Huang et al., 2024).

2. Category-aware partitioning and hierarchical structure

The inputs are a user’s behavior history together with item/category information. ULIM distinguishes a recent behavior sequence, used for short-term modeling and capped at up to 100 actions, from a long-term behavior sequence, which in experiments spans two years of user historical behavior and reaches thousand-scale length. Each behavior is associated with an item and a category. The outputs are a short-term interest vector $\mathbf{v}_u^{(s)}$, a long-term interest vector $\mathbf{v}_u^{(l)}$ for the active category-aware subsequence during training, and, at serving time, multiple long-term interest embeddings together with one short-term embedding (Meng et al., 14 Jul 2025).

The partitioning mechanism is termed Granularity-Aligned Category Clustering. The long sequence is partitioned into category-aware subsequences, and the category granularity is aligned with the ranking stage so that retrieval and ranking share a consistent feature space. Each resulting category-homogeneous subsequence is treated as an interest cluster. Operationally, the method groups actions according to item category, but the exact partition rule beyond category grouping is not specified.

The paper’s conceptual hierarchy has two levels. At the lower level are individual behavior tokens within a subsequence. At the upper level are subsequence representations, each corresponding to a category-specific cluster. The stated complexity reduction is

$O(L) \;\to\; O(L/N),$

where $L$ is the original sequence length and $N$ is the number of categories. In plain terms, the effective sequence length per cluster becomes much smaller after category-aware splitting.

The paper does not write a full formal sequence notation, but a faithful conceptual rendering is

$$\mathcal{S}_u = \{(i_1,c_1), (i_2,c_2), \dots, (i_L,c_L)\},$$

with category-aware subsequences

$\mathcal{S}_u^{(c)} = \{ i_t \mid c_t = c \}.$

This notation is implied by the description rather than explicitly printed in the paper. What is explicit is the interpretation: the interest cluster is the category-aware subsequence, and the long-term interest is the embedding extracted from that cluster.

3. Dual-interest encoders and cross-sequence conditioning

The short-term encoder processes the recent sequence of up to 100 actions with Multi-head Self-Attention (MHSA) followed by average pooling. The paper specifies the component order as MHSA $\rightarrow$ average pooling and uses the pooled result as the short-term interest embedding $\mathbf{v}_u^{(s)}$. The number of heads, number of layers, hidden dimension, positional encoding, and exact MHSA formulation are not specified (Meng et al., 14 Jul 2025).

The long-term encoder is category-conditioned. After extracting category-aware subsequences, it applies Target-Attention, using the short-term pooled representation as the query, to generate the long-term interest embedding. This means that long-term extraction is not independent of recent intent: recent behavior selects and summarizes relevant long-term category-specific history. The design therefore couples the two interest channels asymmetrically: the short-term branch is encoded directly from recent actions, while the long-term branch is conditioned on the short-term summary.

During training, each positive sample activates a single category-aware subsequence. For a target item, only the subsequence whose category matches that item is used to produce the long-term embedding. During serving, multiple subsequences can be processed in parallel, yielding $K$ long-term multi-interest embeddings corresponding to the top-$\mathbf{v}_u^{(l)}$0 predicted categories.

This encoder design captures two kinds of structure. First, it isolates intra-category preference continuity by keeping same-category behaviors together. Second, it lets recent intent determine which historical category cluster is relevant through Target-Attention. The paper does not describe an explicit module for direct dependency modeling among different category clusters, so cross-cluster interaction beyond this query-based selection is not specified.

4. Objective function, supervision, and retrieval alignment

The module is trained through a weighted combination of long-term and short-term retrieval objectives:

$\mathbf{v}_u^{(l)}$1

where $\mathbf{v}_u^{(l)}$2 is the loss for the long-term interest embedding and $\mathbf{v}_u^{(l)}$3 is the loss for the short-term interest embedding. The paper provides sampled-softmax forms for both branches, with user embeddings $\mathbf{v}_u^{(l)}$4 and $\mathbf{v}_u^{(l)}$5, item embedding $\mathbf{v}_u^{(l)}$6, training dataset $\mathbf{v}_u^{(l)}$7, and negative sample set $\mathbf{v}_u^{(l)}$8 (Meng et al., 14 Jul 2025).

The crucial supervision detail is not merely the weighted dual loss, but the redefinition of retrieval at the interest-cluster level. For a training sample whose positive item belongs to category $\mathbf{v}_u^{(l)}$9, the active long-term subsequence is the one for category $O(L) \;\to\; O(L/N),$0, and negatives are drawn from the same category subspace. Training data are pre-grouped into category-homogeneous batches, with negatives shared within each batch. The stated purpose is to “block category leakage” and prevent trivial classification, collapse, and inflated offline metrics.

This implies that the long-term branch is trained under category-consistent supervision rather than over the full heterogeneous candidate pool. The short-term branch is also optimized by sampled softmax, but whether its negatives are constrained in exactly the same manner is not specified. The next-category prediction task belongs to the serving-side module, not to Category-Aware Hierarchical Dual-Interest Learning itself.

Serving-time use is tightly coupled to ULIM’s second component, Pointer-Enhanced Cascaded Category-to-Item Retrieval. A Pointer-Generator Interest Network (PGIN) predicts the top-$O(L) \;\to\; O(L/N),$1 likely future categories; those categories determine which long-term interest embeddings are instantiated and where category-constrained ANN retrieval is performed. The representation module therefore prepares a category-conditioned long-term interest space, while the serving module makes that space operational under latency constraints (Meng et al., 14 Jul 2025).

5. Empirical evidence and operational significance

On the Taobao offline dataset, the full ULIM system outperforms adapted retrieval baselines. The reported values are: YouTube DNN Variant with HR@500 $O(L) \;\to\; O(L/N),$2, HR@1000 $O(L) \;\to\; O(L/N),$3, HR@2000 $O(L) \;\to\; O(L/N),$4; MIND Variant with HR@500 $O(L) \;\to\; O(L/N),$5, HR@1000 $O(L) \;\to\; O(L/N),$6, HR@2000 $O(L) \;\to\; O(L/N),$7; and ULIM with HR@500 $O(L) \;\to\; O(L/N),$8, HR@1000 $O(L) \;\to\; O(L/N),$9, HR@2000 $L$0 (Meng et al., 14 Jul 2025).

The ablations are especially informative for Category-Aware Hierarchical Dual-Interest Learning. ULIM-half-sequence, which uses half the long behavior sequence, drops to HR@500 $L$1, HR@1000 $L$2, and HR@2000 $L$3. ULIM-self-attention, which replaces Target-Attention with Self-Attention in the long sequence branch, obtains HR@500 $L$4, HR@1000 $L$5, and HR@2000 $L$6. These results support two points: very long histories matter, and conditioning long-term extraction on short-term intent matters.

The online evidence is reported for Taobaomiaosha, a mini-app of Taobao. ULIM brings 5.54% clicks, 11.01% orders, and 4.03% GMV lift. As an independent retrieval channel, it increases overall recommendation system response time by only about 15 ms in online A/B tests. Within the paper’s framing, this supports the claim that category-structured long-history modeling can be made compatible with industrial serving requirements.

The method’s practical significance lies in where it shifts complexity. Rather than performing expensive long-sequence interaction online over the entire candidate pool, it compresses and organizes history during representation learning so that serving can restrict search to predicted categories. The reported behavior therefore reflects not only a modeling gain but also a training-serving alignment strategy.

6. Relation to adjacent paradigms and interpretive boundaries

Category-Aware Hierarchical Dual-Interest Learning is related to, but distinct from, several nearby lines of work. Relative to classic multi-interest retrieval such as MIND, ULIM does not organize interests through capsule routing over the entire sequence; it uses category-aware partitioning as the organizing principle, which gives both semantic alignment and a serving-time indexing axis (Meng et al., 14 Jul 2025). Relative to long-sequence ranking models such as DIN, SIM, and ETA, it borrows the intuition that long histories and hierarchical simplification are useful, but adapts those ideas to retrieval by redefining the training objective and integrating category structure.

A close conceptual analogue is HPM, which explicitly learns a low-level item preference stream and a high-level category preference stream with a dual transformer, semantics-enhanced context embedding, and dual contrastive learning. That work is strongly category-aware and hierarchical, but its hierarchy is defined by item-level versus category-level preference dynamics in sequential recommendation rather than by category-conditioned long-history partitioning for retrieval (Huang et al., 2024).

Other recent work suggests broader extensions of the theme without constituting direct instances of the same method. HM4SR uses a two-level Mixture of Experts in multi-modal sequential recommendation, with sequence-level category prediction (CP) as auxiliary supervision. This suggests a possible connection between hierarchical interest decomposition and category-aware supervision, but category is auxiliary rather than a primary organizing mechanism there (Zhang et al., 24 Jan 2025). CADRL, by contrast, uses category-aware item encoding and a dual-agent coarse-to-fine reasoning process over knowledge graphs; it is category-aware and hierarchical, but it is better understood as structured path reasoning than as dual-interest user representation learning (Zheng et al., 2024).

Several interpretive boundaries are explicit. The exact category partition rule beyond grouping by category is not specified. The paper does not provide visualizations of learned interest vectors or attention maps. Embedding dimension, optimizer, category vocabulary size, MHSA depth, head count, batch size, and negative sample count are also not specified for the module itself. Accordingly, the method is best characterized not as a generic “multi-interest” mechanism, but as a retrieval-oriented, category-aligned hierarchical design in which recent intent queries category-specific long-term memory. Its defining contribution is the coupling of three elements: category-aware decomposition of long histories, explicit dual-interest learning, and a category-constrained retrieval objective that aligns training with serving (Meng et al., 14 Jul 2025).