Personalized Experts: Tailored AI Models

• Personalized experts are specialized AI modules that adapt model behavior to individual user profiles and dynamic contexts using MoE frameworks.
• They employ modular expert networks with individualized gating mechanisms to integrate global and local signals, enhancing prediction accuracy and safeguarding privacy.
• Empirical results from e-commerce and federated learning show measurable improvements in key metrics such as AUC, NDCG, and communication efficiency over conventional models.

Personalized experts are model components, modules, or entire architectures tailored to the unique characteristics, history, or dynamic requirements of individual users, clients, or data segments. The personalized-experts paradigm is now a cornerstone methodology for personalization in deep learning, federated learning, recommender systems, and LLMs, underpinning significant advances in prediction accuracy, interpretability, and user-aligned adaptation. Implementations combine modular expert networks—often within Mixture-of-Experts (MoE) frameworks—with individualized gating or routing networks, and rely on fine-grained representations or personalization objectives to specialize behavior for each user or context.

1. Foundations and Theoretical Motivation

Personalized experts leverage the MoE formalism, in which $K$ expert sub-networks $\{f_i\}_{i=1}^K$ are combined through a gating function $g(\cdot)$ that assigns per-input weights, usually normalized via softmax: $p_i=\frac{\exp r_i}{\sum_j \exp r_j}$, where $r=g(x)$. The output is the mixture $y=\sum_{i=1}^K p_i\,f_i(x)$.

This framework achieves personalization via several distinct mechanisms:

• Parameter specialization: Each expert learns from (or is adapted to) a specific user, user cohort, or task subspace, enabling the model to capture individual or group-specific patterns not present in the global model alone.
• Gating flexibility: The gating network adapts the expert mixture as a function of user history, session context, or input features, yielding session- or user-specific predictions.
• Modularity and privacy: In federated or distributed settings, local experts are adapted and stored on-device, and only shared/global models are communicated, achieving privacy and personalization [(Zec et al., 2020, Yi et al., 2024), 2020.12.15589].
• Meta-weighted combinations: The effectiveness of combining population and personalized models using meta-learned weights is demonstrated in sequential domains (Rudovic et al., 2019), where personalized expert selection outperforms any single expert.

The mixture framework is mathematically underpinned by both probabilistic latent-class modeling (as in Gaussian Process experts (Alaa et al., 2016)) and sparse neural activation (as in MoE-layers in transformers and MLPs).

2. Algorithmic Realizations: Architectures and Gating Mechanisms

MoE Instantiation in Deep Learning and Recommendation

In attention-weighted MoE (AW-MoE), the input vector is projected to parallel input and gate networks. User behavior sequences are encoded separately for feature interaction (experts) and for fine-grained gating, including activation units that modulate expert weights per behavior element (Gong et al., 2023). The gating pathway computes user- and context-dependent softmax weights $\alpha_k(x)$, which select and combine expert outputs per user session.

In recommender systems, this architecture enables explicit learning of (a) how features interact differently per user (via expert specialization), and (b) which expert "blends" to activate for tail or new users (via the gating network and contrastive learning regularizer). The contrastive loss aligns the gating output between masked (sparse-history) and full (rich-history) users, thereby improving robustness under data sparsity.

Personalized Experts in Federated and Decentralized Learning

Personalized federated frameworks (e.g., pFedMoE, FedMoE, FLEx) construct local MoE modules by combining:

• A global generalist expert, shared and aggregated periodically via FedAvg or similar protocols;
• Client-local personalized experts, adapted solely on local data, often with heterogeneous architectures (e.g., model-heterogeneous FL);
• Local gating networks (unique per client) assigning mixture weights, frequently at the level of data-sample, enabling data-level or context-level personalization (Yi et al., 2024, Mei et al., 2024, Liu et al., 1 Jun 2025).

For example, pFedMoE employs a gating network $\mathcal{H}_k$ that fuses the global (homogeneous) and local (heterogeneous) expert feature extractors into a mixed representation, processed downstream by a personalized classifier head; only the global expert parameters are communicated to maintain privacy (Yi et al., 2024). FedMoE further allows each client to adopt a personalized sub-MoE—i.e., a selective subset of experts per layer, optimized via a knapsack-like search on empirical activation statistics, with modular aggregation to avoid parameter interference (Mei et al., 2024).

The gating networks may range from small MLPs (taking raw or feature-extracted input) to specialized architectures (attention, softmax-masked selectors), often trained jointly with local heads under cross-entropy or contrastive objectives.

Personalized Experts in LLMs and Personality Simulation

P-React extends the base LLM by injecting multiple LoRA (Low-Rank Adapter) experts within each transformer layer and learns personality-guided routers. For each trait (e.g., Big Five), a trait embedding is mapped via a linear router to N expert weights, which then mix the LoRA expert outputs per forward pass. A personality specialization loss (PSL) promotes orthogonality of the routing vectors, ensuring expert specialization across personality traits (Dan et al., 2024).

Personalization is performed by dynamically selecting expert mixtures based solely on a user's trait vector, allowing for robust personality-simulating behaviors without user-specific fine-tuning. Similar methodologies underpin topic- and persona-adaptive LLMs in other domains.

3. Practical Implementations and Empirical Insights

E-Commerce Personalization

AW-MoE (deployed at JD.com) demonstrated up to +2.58% AUC and significant NDCG improvement over deep FFN and standard MoE baselines; the largest effects accrue to (a) elderly users and (b) long-tail users with sparse history, especially when contrastive learning augmentation is used (Gong et al., 2023). In production, AW-MoE yielded +0.78% UCVR and +0.35% UCTR, substantiating nontrivial revenue lifts.

Federated Learning and Model Heterogeneity

pFedMoE and FedMoE architectures outperform best prior methods (e.g., FedProto, FedKD, SCAFFOLD, IFCA) by up to 2.8–22% on CIFAR-100 and realize comparable or superior computation-communication trade-offs. FLEx achieves up to 50× bandwidth reduction by pruning to one expert per client while preserving or improving personalization and general performance (Liu et al., 1 Jun 2025).

A critical empirical result is that introducing local expert adapters—