Dual-view Personalized MoE Architecture

• The paper introduces DPMoE, which leverages dual-view adaptation with personalized low-rank perturbations in a frozen Transformer backbone to improve task performance.
• It employs adaptive cross-view fusion by routing contextual, user, and interaction signals to selectively combine semantic and behavioral representations.
• Empirical results demonstrate significant gains in recommendation metrics and federated learning accuracy, validating the effectiveness of the dual-view expert design.

Searching arXiv for the cited DPMoE-related papers to ground the article in current records. Searching arXiv for "Dual-view Personalized Mixture-of-Experts" and closely related titles. Dual-view Personalized Mixture-of-Experts (DPMoE) denotes an architectural pattern in which two complementary views are processed through expert mechanisms and then fused by personalized routing. In the narrowest sense, the term appears as the core adaptation module inside L2Rec, where a frozen Transformer backbone receives view-specific, user-conditioned low-rank perturbations for a semantic view and a behavioral view, followed by adaptive cross-view fusion (Pan et al., 26 May 2026). In a broader and more interpretive sense, several adjacent lines of work instantiate DPMoE-like structure without using the exact name: some define the two views as global/generalized versus local/personalized representations in federated learning, while others organize dual expert systems across semantic and personalized computation stages, or across feature learning and downstream prediction (Yi et al., 2024, Liu et al., 30 May 2025, Lee et al., 29 Oct 2025). The common thread is that “dual-view” usually refers not to two modalities alone, but to two distinct sources of information, two computational perspectives, or two specialization roles, combined by a routing mechanism that is adaptive at the user, client, sample, or token level.

1. Conceptual definition and scope

In L2Rec, DPMoE is defined as a parameter-efficient adaptation mechanism for sequential personalized recommendation that unifies behavioral signals and semantic signals at the parameter level of a frozen LLM backbone (Pan et al., 26 May 2026). The semantic view is constructed from concatenated item descriptions,

$$\mathbf{E}_u^{S} = \text{Embed}([\mathbf{c}_1;\mathbf{c}_2; \ldots;\mathbf{c}_L]) \in \mathbb{R}^{T\times d},$$

whereas the behavioral view compresses each interacted item into a vector sequence,

$$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$

DPMoE then produces view-specific perturbations of the same frozen Transformer parameters, yielding

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$

This makes DPMoE a dual-view model in a literal sense: two views are preserved throughout the backbone as two personalized adaptation pathways rather than being forced into a single shared representation at the input level.

The broader literature uses the same idea under adjacent names. In pFedMoE, the two views are a global/generalized representation view from a shared homogeneous small feature extractor and a local/personalized representation view from a client-private heterogeneous backbone; the paper explicitly states that this is “very close to a ‘Dual-view Personalized MoE (DPMoE)’ interpretation” if “dual-view” is understood as “two representation sources / two expertise streams” rather than two modalities (Yi et al., 2024). In NextLocMoE, the architecture is described as a dual-level MoE, with one expert system modeling a location-semantic / functional view and another modeling a user-behavior / personalization view (Liu et al., 30 May 2025). In dual-MoE survival analysis, the two perspectives are a subgroup-aware patient representation view and a temporal hazard-evolution view (Lee et al., 29 Oct 2025). This suggests that DPMoE has become a functional design principle: one branch models common or semantic structure, the other models personalized or dynamically conditioned structure, and the final predictor is obtained by learned fusion rather than fixed interpolation.

A persistent misconception is that “dual-view” must imply multimodality. The available literature does not support such a restriction. Some of the clearest DPMoE-like systems are not multimodal at all: they instead derive the two views from federated provenance, temporal conditioning, or distinct computational levels (Yi et al., 2024, Lee et al., 29 Oct 2025). Conversely, multimodal systems may still fail to be DPMoE in a strict sense if they do not include personalized or adaptive expert routing.

2. Core architectural patterns

The most explicit DPMoE formulation appears in L2Rec. For each adapted Transformer parameter matrix $\mathbf{W}_j$, the view-specific perturbation is

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$

where $\mathcal{E}_{sh}$ are shared experts and $\mathcal{E}_v$ are view-specific experts (Pan et al., 26 May 2026). Shared experts are always active, while semantic-specific and behavioral-specific experts are sparsely selected. The semantic branch and behavioral branch therefore traverse the same frozen backbone, but under different low-rank expert mixtures. The model uses 9 experts total—1 shared expert and 8 view-specific experts—with Top-2 experts selected per view, LoRA rank $r=8$, scaling $\alpha=16$, and about 32M trainable parameters, or about 5% of the backbone (Pan et al., 26 May 2026).

The view-specific outputs are then merged by Adaptive Cross-view Fusion (ACF). After residual projections,

$\tilde{\mathbf{h}^v_u = \mathbf{h}^v_u + P_v(\mathbf{h}^v_u), \qquad v\in\{B,S\},$

a dynamic gate is computed as

$$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$0

and the fused user representation is

$$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$1

The same dual-view encoding and fusion are used on the item side. Architecturally, DPMoE is therefore not merely a routing module; it is a two-path adaptation-and-fusion system spanning parameter perturbation, representation formation, and final preference scoring (Pan et al., 26 May 2026).

Other papers instantiate the same pattern with different expert granularity. PFL-MoE is a two-expert predictive fusion framework in which a global/shared model and a personalized/local model are mixed by a client-specific gate: $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$2 with

$$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$3

In the freeze-base instantiation PFL-MF, the two experts are classifier heads sharing one extractor, and the improved PFL-MFE uses the shared feature $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$4 rather than raw input as gate input: $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$5 This is a minimal DPMoE-like design in which the two views are “global generalization” and “local specialization” (Guo et al., 2020).

pFedMoE moves the fusion point from output space to representation space. For each client $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$6, a shared homogeneous small feature extractor $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$7 produces a generalized representation, a local heterogeneous extractor $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$8 produces a personalized representation, and a local gate $$\mathbf{E}_{u}^B = P_{U}\big([\text{pool}(\text{Embed}(\mathbf{c}_i))]_{i=1}^{L}\big) \in \mathbb{R}^{L \times d}.$$9 yields instance-specific weights

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$0

The fused representation is

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$1

which is then passed to the local head. This is explicitly a dual-expert MoE over two representation sources (Yi et al., 2024).

3. Personalization and routing mechanisms

The defining property of DPMoE is not just duality, but personalized routing. In L2Rec, the router uses three signals for the view-specific experts: a context signal

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$2

a user signal

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$3

and an interaction signal

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$4

Each is mapped to expert scores by a two-layer MLP,

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$5

and the final routing weights are formed by

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$6

followed by Top-$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$7 sparsification (Pan et al., 26 May 2026). The stated design intuition is that context remains primary while user-specific information acts as a personalized correction.

The most common alternative is sample-wise gating between a generalized branch and a personalized branch. pFedMoE exemplifies this at the data level: the local gating network produces personalized weights for both experts on each data sample, and weight analyses show that distributions vary across clients, across classes, and across samples (Yi et al., 2024). This is why the paper characterizes the method as data-level personalization rather than merely client-level personalization.

NextLocMoE uses a more heterogeneous routing design. Its Location Semantics MoE routes current records to function experts using the current embedding and long-term history,

$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$8

then applies fixed top-$\widetilde{\mathbf{W}^S = \mathbf{W} + \Delta \mathbf{W}^S,\qquad \widetilde{\mathbf{W}^B = \mathbf{W} + \Delta \mathbf{W}^B.$9 routing to obtain a function-aware spatial representation. Its Personalized MoE, inserted into upper Transformer layers, forms per-expert inputs

$\mathbf{W}_j$0

computes user-expert probabilities

$\mathbf{W}_j$1

and selects experts until cumulative probability exceeds $\mathbf{W}_j$2, with $\mathbf{W}_j$3 (Liu et al., 30 May 2025). Personalization here is trajectory-conditioned and prior-guided rather than user-ID-based.

A stronger user-centric routing regime appears in sparse MoE reward modeling. The reward model uses expert heads $\mathbf{W}_j$4 and a router $\mathbf{W}_j$5, with final reward

$\mathbf{W}_j$6

Personalization occurs at test time by freezing experts and adapting only the router using a small adaptation set, so that individual preference profiles are expressed as different mixtures over shared latent preference components (Wang et al., 2 Jun 2026). This is not a dual-view design, but it establishes router-only personalization as a reusable DPMoE principle.

4. Learning objectives and regularization

DPMoE systems differ substantially in where they impose coupling and regularization. In L2Rec, the primary recommendation objective is a contrastive loss on fused representations,

$\mathbf{W}_j$7

supplemented by a Bidirectional Preference Contrastive loss,

$\mathbf{W}_j$8

and a load-balancing loss $\mathbf{W}_j$9, yielding

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$0

This explicitly addresses cross-view drift even though both views already inhabit the same frozen parameter manifold (Pan et al., 26 May 2026).

Other dual-expert systems use much simpler task losses. pFedMoE uses only cross-entropy on the fused representation and states that there is no explicit auxiliary regularization term; regularization is implicit in normalization layers, federated averaging of the shared extractor, and architecture choices (Yi et al., 2024). PFL-MoE similarly trains the gate by minimizing

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$1

after first adapting the personalized branch on local data (Guo et al., 2020).

When dual expert groups are meant to remain functionally distinct, explicit decorrelation losses become central. In OneTrackerV2’s Dual Mixture-of-Experts, T-MoE models spatio-temporal relations and M-MoE models multimodal knowledge; their outputs are regularized by an expert decoupling loss

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$2

and M-MoE routing is further structured by a router clustering loss

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$3

The total loss also includes a balance loss, though its explicit formula is not provided in the text (Hong et al., 5 May 2026). This suggests a general DPMoE design principle: if the two views are intended to specialize, explicit anti-collapse regularization may be required rather than assumed.

A different form of expert regularization appears in sparse MoE reward modeling, where local routing entropy,

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$4

global batch balance,

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$5

and expert diversity on pairwise reward differences,

$\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$6

are combined with the main reward-modeling loss (Wang et al., 2 Jun 2026). Although again not dual-view, these terms are directly applicable when a DPMoE must remain both sparse and interpretable.

5. Applications and empirical evidence

The most direct empirical validation of DPMoE comes from L2Rec. On four recommendation datasets, the reported best results are Scientific $\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$7, Instruments $\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$8, Arts $\Delta\mathbf{W}_j = \sum_{i \in \mathcal{E}_{sh} \mathbf{B}_{j,i} \mathbf{A}_{j,i} + \sum_{i \in \mathcal{E}_{v} g_{j,i}\, \mathbf{B}_{j,i} \mathbf{A}_{j,i}, \qquad v\in\{s,b\},$9, and Industrial $\mathcal{E}_{sh}$0, with relative NDCG@10 improvements over the strongest baseline of +8.02%, +3.87%, +5.27%, and +5.74% respectively (Pan et al., 26 May 2026). Ablations show that removing either view is damaging: for example, on Scientific, $\mathcal{E}_{sh}$1 without the semantic view and $\mathcal{E}_{sh}$2 without the behavioral view; removing personalized routing reduces performance to $\mathcal{E}_{sh}$3, and removing BPC reduces it to $\mathcal{E}_{sh}$4 (Pan et al., 26 May 2026). In a one-month online A/B test on a platform with about 1.5M DAU, L2Rec improved CTR by +9.24% and reply rate by +3.15% with $\mathcal{E}_{sh}$5 (Pan et al., 26 May 2026).

Federated interpretations of DPMoE also show strong evidence for global–local dual-view fusion. In heterogeneous settings, pFedMoE consistently achieves the best mean accuracy, with up to $\mathcal{E}_{sh}$6 improvement over the best baseline under a given setting and up to $\mathcal{E}_{sh}$7 improvement over the best same-category baseline (Yi et al., 2024). The paper also reports that compared to FedProto, 76% of clients on CIFAR-10 and 60% on CIFAR-100 achieve higher accuracy, which supports its claim of finer-grained personalization (Yi et al., 2024).

NextLocMoE provides perhaps the clearest evidence that semantic and personalized expert systems can be jointly necessary rather than substitutable. On Shanghai fully supervised, the full model reports 64.92 / 75.88 / 77.43, compared with 59.72 / 65.62 / 69.44 without the Location Semantics MoE and 33.68 / 55.21 / 63.19 without the Personalized MoE (Liu et al., 30 May 2025). In Shanghai $\mathcal{E}_{sh}$8 Kumamoto zero-shot transfer, the full model achieves 16.02 / 36.06 / 48.42, whereas removing the semantic MoE gives 5.519 / 17.90 / 28.24 and removing the personalized MoE gives 1.437 / 5.142 / 9.352 (Liu et al., 30 May 2025). These results support a dual-aspect reading of DPMoE in which semantic specialization and personalized behavioral specialization are complementary.

Evidence from neighboring application domains indicates that the same structural idea generalizes. In pFedMoAP, a local prompt, multiple downloaded non-local prompt experts, and a client-specific attention-based gate produce large gains over single-global-prompt approaches; for instance, on Flowers102 under pathological non-IID, pFedMoAP reports 98.41 versus 72.80 for PromptFL (Luo et al., 2024). In survival analysis, the dual-MoE design improves time-dependent C-index by up to 0.04 and outperforms either encoder MoE alone or hazard MoE alone in ablation (Lee et al., 29 Oct 2025). These results do not prove a single canonical DPMoE recipe, but they do show that dual expert specialization combined with adaptive fusion is effective across recommendation, federated learning, prompt learning, next-location prediction, and time-to-event modeling.

6. Variants, limitations, and boundaries of the concept

Despite the breadth of related work, DPMoE is not a single fixed architecture. The literature supports at least four interpretations. First, DPMoE can mean parallel dual-view parameter adaptation, as in L2Rec (Pan et al., 26 May 2026). Second, it can mean dual-expert global–local fusion, as in pFedMoE and PFL-MoE (Yi et al., 2024, Guo et al., 2020). Third, it can denote dual-level specialization across different points of the model pipeline, as in NextLocMoE and dual-MoE survival models (Liu et al., 30 May 2025, Lee et al., 29 Oct 2025). Fourth, it can denote dual-source collaborative versus local expert composition, as in pFedMoAP (Luo et al., 2024). A plausible implication is that the term is best treated as a family resemblance concept rather than a rigid taxonomy.

Several limitations recur. L2Rec, while the clearest named DPMoE, requires explicit dual-view input construction and does not provide the exact explicit formula of the load-balancing loss in the provided text (Pan et al., 26 May 2026). pFedMoE requires that the last-layer dimensions of the shared homogeneous small extractor and the heterogeneous large extractor be identical, so its heterogeneity support is mediated by an aligned feature interface (Yi et al., 2024). NextLocMoE does not specify exact $\mathcal{E}_{sh}$9, $\mathcal{E}_v$0, semantic top-$\mathcal{E}_v$1, TCN depth, or LoRA ranks in the provided details (Liu et al., 30 May 2025). pFedMoAP does not provide detailed ablations on the number of non-local experts $\mathcal{E}_v$2, despite depending on retrieved prompt experts (Luo et al., 2024). These omissions show that the dual-view idea is often architecturally compelling but incompletely standardized.

Another boundary concerns personalization itself. Some models use genuinely user-conditioned routing, as in L2Rec’s user, context, and interaction signals (Pan et al., 26 May 2026). Others use client-local parameters without explicit user embeddings, as in PFL-MoE or pFedMoE (Guo et al., 2020, Yi et al., 2024). Still others are only weakly “personalized” in a strict sense: MoDiffE, for example, is dual-view and MoE-based but its gates are bundle-conditioned rather than user-conditioned (Li et al., 8 May 2025). This suggests that the adjective “personalized” in DPMoE can refer to different granularities—user, client, sample, token, or instance—and these should not be conflated.

A further misconception is that more experts necessarily imply a better DPMoE. The evidence is mixed. In MvP for personality detection, performance improves as the number of views increases up to 6, then degrades (Zhu et al., 2024). In sparse MoE reward modeling, description fidelity improves and then peaks around $\mathcal{E}_v$3, while expert specialization decreases monotonically with larger $\mathcal{E}_v$4 (Wang et al., 2 Jun 2026). This suggests that DPMoE benefits not simply from capacity, but from expert semantics, routing quality, and regularization.

Taken together, the literature supports a concise characterization. DPMoE is a dual-branch or dual-system expert architecture in which two complementary views—most often semantic versus behavioral, global versus local, subgroup versus temporal, or collaborative versus private—are preserved as distinct computational sources and fused by personalized routing. In the strict terminological sense, L2Rec provides the clearest instantiation (Pan et al., 26 May 2026). In the broader methodological sense, adjacent work shows that the same design principle recurs whenever a model must retain both shared structure and individualized specialization without collapsing them into a single monolithic predictor (Yi et al., 2024, Liu et al., 30 May 2025, Lee et al., 29 Oct 2025).