Two-Tower Recommendation Systems

• Two-tower recommendation systems are architectures that employ separate neural towers to embed users and items into a shared latent space.
• They enable efficient approximate nearest neighbor retrieval by precomputing item embeddings and executing only the user tower at query time.
• Enhancements such as fine-grained interactions and contrastive learning boost performance while preserving low-latency, scalable retrieval.

A two-tower recommendation system is a neural architecture designed to efficiently score and retrieve items for a user at scale. It comprises two embedding subnetworks (“towers”): one encoding user features and the other encoding item features. The two towers independently map their respective inputs into a shared embedding space, enabling fast approximate nearest neighbor (ANN) retrieval using simple interaction functions, typically an inner product. Two-tower models are prevalent in large-scale retrieval, candidate matching, and pre-ranking stages due to their balance of computational efficiency, architectural modularity, and support for offline item embedding precomputation. However, their decoupling of user and item representation typically limits cross-feature interactions, which has motivated numerous hybrid and enhanced variants.

1. Canonical Two-Tower Framework: Structure, Inference, and Motivation

The canonical two-tower model consists of a user-tower encoding user side features and an item-tower encoding item features. Each tower is typically realized via an embedding-lookup table (for categorical inputs) followed by MLP, transformer, or recurrent blocks, resulting in a d-dimensional latent vector. The interaction function $s(u,i)=f_u(U)\cdot f_v(I)$ is usually a normalized or unnormalized inner product, corresponding to the cosine similarity or Euclidean dot product between embeddings (Wang et al., 2021).

At serving time, the item embeddings $\{v_j\}$ can be precomputed offline and stored in an efficient ANN index (e.g., Faiss IVF-PQ, HNSW) for sublinear retrieval. For a query user, the user-tower produces $u=f_u(U)$ online, and the top-K item candidates are selected via $u^\top v_j$. This architecture reduces online scoring to $O(N) + k O(M)$, where $N$ is a single forward pass for the user tower and $M$ is much smaller than $N$ (retrieved candidates per user), yielding sub-20 ms latencies for $k\sim 10^3$ in industrial systems (Li et al., 2022, Osowska-Kurczab et al., 19 Jul 2025).

This framework is favored when: (1) the item pool is very large ($10^5$–$\{v_j\}$0), (2) fast, scalable candidate matching is required, and (3) latency and memory constraints prohibit a full joint user–item model for each query. However, it introduces “late interaction” limitations, as user and item features interact only at the final score computation, omitting deep or pair-specific representation learning.

2. Architectural Extensions: Overcoming Decoupling and Enhancing Interaction

Numerous recent works address interaction limitations in two-tower systems by introducing lightweight, scalable feature-level or embedding-level cross-tower communication. Several paradigms have emerged:

• Fine-grained Early Interaction: Modules such as the FE-Block in IntTower perform multi-head, multi-layer projections of user and item sub-embeddings, computing maximum cosine similarities between projected user and item representations at each layer. By stacking several FE-Blocks, IntTower closes the effectiveness gap with ranking models while retaining two-tower efficiency (Li et al., 2022).
• Contrastive Regularization: Self-supervised losses (e.g., CIR in IntTower) are frequently incorporated, pulling positive pairs’ representations together and pushing negatives apart using InfoNCE or sampled softmax losses applied to tower embeddings (Li et al., 2022).
• Meta-Interaction and Attention Pooling: FIT introduces a Meta Query Module that enables learnable early interaction by augmenting user features with codes derived from a learnable item meta-matrix, and a Lightweight Similarity Scorer for late-stage expressiveness using multi-head matrix subspace reductions and row/column-wise nonlinear projections (Xiong et al., 16 Sep 2025).
• Diffusion and Generative Modules: T2Diff reconstructs a user’s next positive intent embedding via a diffusion model over temporal behavioral drift, which is then injected into a session-based mixed-attention module—explicitly combining generative future intent with session/history for cross-tower interaction (Wang et al., 28 Feb 2025).
• Contrastive and Conditional User–Condition Interactions: Conditional two-tower models extend the user tower input to include item-side condition embeddings (e.g., topics, brands), enabling conditional retrieval beyond user–item only matching. This is efficient to deploy, with minimal changes to standard two-tower infrastructure, and achieves strong gains in conditional feed contexts (Lin et al., 22 Aug 2025).
• Graph-Based Enrichment: Some architectures use GNN layers to augment two-tower encodings with collaborative signals from co-action graphs (CAGR) or by combining local pair-wise representation learning (for “familiar” items) with the standard two-tower backbone (for exploratory items), as in ContextGNN (Sun et al., 2024, Yuan et al., 2024).
• Cross-Batch Negative Sampling: To increase training negative diversity beyond batch-size limitations, CBNS caches recent item embeddings for reuse as negatives across batches. This technique exploits the stability of item embeddings after warm-up and significantly improves Recall and NDCG without extra compute or memory overhead (Wang et al., 2021).

These enhancements yield empirical gains of 3–15% AUC, 11–23% Recall@K, or higher NDCG compared to classic two-tower methods, while keeping serving latencies low (e.g., IntTower: $\{v_j\}$12ms additional inference cost; FIT: Taobao latency 26.9 ms vs. 25.4 ms for two-tower) (Li et al., 2022, Xiong et al., 16 Sep 2025).

3. Training Paradigms, Sampling, and Optimization

Standard two-tower models are typically trained with a sampled softmax, InfoNCE, or binary cross-entropy contrastive loss. In a mini-batch, positive user–item pairs are contrasted with a set of negative items, often sampled from the batch (“in-batch negatives”) or augmented with longer-history negatives (via CBNS).

• In-batch negatives: Scalability is limited by GPU memory and the batch size, which constrains negative diversity (Wang et al., 2021).
• Cross-batch negatives: CBNS decouples negative diversity from batch size with an external memory queue, reducing variance in gradient estimation and accelerating convergence (Wang et al., 2021).

Recent empirical and theoretical work demonstrates that:

• Increasing the effective number of negatives improves model NDCG and Recall by up to 12%, with negligible increase in wall-clock time (YouTube DNN, MIND, GRU4Rec) (Wang et al., 2021).
• Parameter-tying, meta-interaction, and denoising regularizers (CS3: CAS, CTS, and CMS modules) stabilize embeddings over time, especially under online data drift or high-frequency re-training (Wang et al., 21 Apr 2026).

Advanced architectures employ multi-objective or listwise losses, e.g., combining retrieval and ranking objectives with distillation or alignment penalties (LT-TTD), to mitigate propagation errors in multi-stage pipelines (Abraich, 7 May 2025).

4. Deployment and System Integration in Industrial Pipelines

Industrial-scale recommender systems operationalize two-tower architectures in multi-stage cascades: candidate retrieval $\{v_j\}$2 pre-ranking $\{v_j\}$3 full ranking $\{v_j\}$4 re-ranking. Pre-ranking tasks typically involve scoring $\{v_j\}$5 candidates per user within sub-20 ms, imposing strict efficiency constraints.

• Serving Infrastructure: Item embeddings are precomputed offline and indexed for ANN search, while only the user tower is executed per request. This enables real-time QPS of $\{v_j\}$610^3 and online A/B testing at scale (Allegro, Huawei Ads, Pinterest feeds) (Li et al., 2022, Osowska-Kurczab et al., 19 Jul 2025, Lin et al., 22 Aug 2025).
• Extensibility: Modular two-tower backbones allow plug-and-play integration with knowledge distillation modules, graph signal enrichment, cascade sharing, or LLM-based semantic token generation (e.g., TTDS) (Wang et al., 21 Apr 2026, Yin et al., 2024).
• Privacy and Federated Learning: The split two-tower (STTFedRec) paradigm offloads item models to a server, with only user embeddings computed on-device and similarity scores sent to the server. Secure aggregation (secret-sharing) and obfuscated item sampling provide privacy preservation with 18–50$\{v_j\}$7 cost and 2–49$\{v_j\}$8 bandwidth reduction over naïve federated implementations, with retained accuracy (Qin et al., 2022).
• Conditioned and Multi-objective Retrieval: Conditional towers (e.g., topic-conditioned user embeddings) allow rapid launch and adaptation to new verticals without new labeled data or large-index modifications (Lin et al., 22 Aug 2025).

5. Benchmarking, Theoretical Guarantees, and Application Contexts

Two-tower methods are benchmarked on a wide range of interactions, including public data (MovieLens, Amazon, TaobaoAd, Adressa) and industrial data (Allegro, NetEase Cloud Music, eBay, Pinterest). Key findings:

• Metrics: Improvements of 3–15% (AUC), 7–20% (Recall@K), and 12–23% (NDCG@K) have been reported for interaction-enhanced architectures versus classic two-tower baselines, sometimes matching or exceeding single-tower ranking models’ effectiveness (Li et al., 2022, Xiong et al., 16 Sep 2025, Wang et al., 28 Feb 2025).
• Theory: Unified optimization, e.g., LT-TTD, provides formal guarantees: the upper bound of irretrievable relevant items is reduced by a factor $\{v_j\}$9, with $u=f_u(U)$0 set by the distillation loss. The global listwise optimum is strictly better than disjoint stage-wise optima unless objectives are perfectly aligned (Abraich, 7 May 2025).
• Latency and Complexity: Two-tower baselines: $u=f_u(U)$1. Enhanced two-tower models with FE-Blocks or meta-interaction modules: $u=f_u(U)$2, with $u=f_u(U)$3, $u=f_u(U)$4. Storage remains $u=f_u(U)$5, with only small increases from attention/interaction modules (Li et al., 2022, Xiong et al., 16 Sep 2025, Wang et al., 21 Apr 2026).
• Online Uplifts: Revenue gains up to 8% (CS3 in advertising systems), increases in new or daily active users, and significant serving cost reductions compared to multi-stage or single-tower baselines have been demonstrated (Wang et al., 21 Apr 2026, Lin et al., 22 Aug 2025).

6. Limitations, Open Problems, and Prospects

Major limitations include:

• Pair-agnostic representations: Standard two-tower models cannot exploit fine-grained user–item co-occurrences or preference context, limiting expressiveness for repeat consumption or fine personalization (Yuan et al., 2024).
• Trade-off between efficiency and deep interaction: All-to-all or early interaction modules must retain ANN compatibility to avoid prohibitive online scoring costs (Su et al., 2023).
• Cold-start and long-tail user/item robustness: Augmenting the retrieval tower with meta-embeddings, generative interest reconstruction, or conditional context improves robustness at the price of marginal extra complexity (Feng et al., 2021, Yin et al., 2024).
• Unified vs. cascade pipelines: LT-TTD and ContextGNN exemplify hybrid designs that retain two-tower scalability for exploratory candidates but build in contextualized or pair-specific re-ranking for a subset (“familiar” items), overcoming upper limits in standard cascade recall (Abraich, 7 May 2025, Yuan et al., 2024).

Ongoing research aims to:

• Close the ranking–retrieval gap via universal approximators (e.g., FIT’s LSS), cross-tower synchronization/EMA regularization, and knowledge transfer from LLMs or LLM backbones (Xiong et al., 16 Sep 2025, Wang et al., 21 Apr 2026, Yin et al., 2024);
• Support dynamic multimodal and multi-interest retrieval (IP2, TTDS, CAGR) for news, multimedia, and e-commerce scenarios (Wu et al., 18 Jul 2025, Sun et al., 2024, Xiong et al., 16 Sep 2025);
• Maintain privacy, low bandwidth, and rapid online adaptation through split/federated architectures with secure aggregation (Qin et al., 2022).

7. Summary Table: Key Recent Two-Tower Advances

Model	Key Enhancement	Efficiency	Effectiveness Gain	Data/Deployment Context	Reference
IntTower	FE-Block + Light-SE + CIR	$u=f_u(U)$6	3–7% AUC rel.	Pre-ranking, ads, public data	(Li et al., 2022)
FIT	Meta Query + LSS	$u=f_u(U)$7	5–15% AUC rel.	Amazon, Taobao, MovieLens	(Xiong et al., 16 Sep 2025)
T2Diff	Diffusion + Mixed Attention	$u=f_u(U)$8	7–21% Recall/NDGC	Recommendation matching	(Wang et al., 28 Feb 2025)
CS3	CAS + CTS + CMS modules	$u=f_u(U)$9	3–6% AUC rel.; +8% revenue	Real-time ads, large-scale	(Wang et al., 21 Apr 2026)
CBNS	Cross-Batch Negatives	$u^\top v_j$0	2.5–12% Recall@K	YouTubeDNN, MIND, GRU4Rec	(Wang et al., 2021)
SparCode	Quantized cross-interaction	$u^\top v_j$1 per code	up to 100% Recall@50	Deezer, MovieLens, search	(Su et al., 2023)
LT-TTD	Unified dual-tower + transformer	$u^\top v_j$2	Theoretical optimum	Two-stage retrieval & ranking	(Abraich, 7 May 2025)
ContextGNN	Pairwise+two-tower fusion	$u^\top v_j$3	+20% MAP	Multi-interest recommendation	(Yuan et al., 2024)
STTFedRec	Split learning, privacy	$u^\top v_j$4 item comm.	$u^\top v_j$5equal FL/cent.	Privacy-preserving, federated	(Qin et al., 2022)

• (Wang et al., 2021, Li et al., 2022, Xiong et al., 16 Sep 2025, Wang et al., 28 Feb 2025, Abraich, 7 May 2025, Su et al., 2023, Wang et al., 21 Apr 2026, Lin et al., 22 Aug 2025, Sun et al., 2024, Wu et al., 18 Jul 2025, Yuan et al., 2024, Yin et al., 2024, Qin et al., 2022, Feng et al., 2021)