---
title: Progressive MoE Adapter (PMOE)
url: https://www.emergentmind.com/topics/progressive-moe-adapter-pmoe
type: topic
---

# Progressive MoE Adapter (PMOE)

Searching arXiv for the cited PMoE/PMOE papers and closely related work.
Progressive MoE Adapter (PMOE) denotes a Mixture-of-Experts module introduced within the FakeSV-VLM framework for detecting fake news on short video platforms. In that usage, PMOE is a two-stage, hierarchical reasoning mechanism integrated into the early-to-mid layers of a VLM: detection experts first provide an initial analysis, and attribution experts then perform a more comprehensive diagnosis over learnable Artifact Tokens, after which a final decision is produced jointly with the base model decoder. The term is not unique across arXiv: closely related PMoE or pMoE formulations also appear in continual learning for LLMs, continual RANS turbulence modelling, and visual prompt tuning, but those instantiate different notions of progression, routing, and expert growth [2508.19639].

## 1. Definition, scope, and nomenclature

In FakeSV-VLM, PMOE is designed for a setting in which short-video news samples are categorized into four distinct scenarios: both video and text are real, both are fake, only the video is fake, or only the text is fake. The module accordingly combines a coarse binary detection stage with a fine-grained attribution stage. This organization reflects the paper’s claim that fake news videos often show inconsistency between modalities and that diagnosis benefits from progressively refining the decision from authenticity screening to manipulation attribution [2508.19639].

The term “progressive” has a specific meaning in this setting. It refers to staged refinement of the decision process rather than continual addition of experts over a task stream. A common source of confusion is that “PMoE” in the continual-learning literature instead denotes architectures that progressively add experts as new tasks or regimes arrive. In other words, the acronym is shared, but the operational semantics differ substantially across domains [2407.21571].

## 2. Placement within the VLM and end-to-end data flow

PMOE is inserted into the early-to-mid layers of the LLM component. The input pipeline first extracts visual features and textual embeddings,
$$
f_v = M(E_v(v_1 \ldots v_k)), \qquad f_t = T_k(t),
$$
then forms a fused representation
$$
f_c = [f_v \parallel f_t].
$$
A set of $q$ learnable Artifact Tokens $A \in \mathbb{R}^{q \times d}$ is prepended, and the concatenated sequence is processed by the first $l$ layers of the LLM:
$$
[f_c \parallel A] \leftarrow \mathrm{LLM}_{1:l}([f_c \parallel A]).
$$
Only after this contextualization does PMOE operate on the Artifact Tokens [2508.19639].

The module itself is organized into three successive components. The Detection MoE consumes the contextualized Artifact Tokens and performs binary routing between “real” and “fake” experts. The Attribution MoE receives the Detection MoE’s refined tokens and routes them among four experts corresponding to $\{r, fv, ft, fm\}$, namely both real, video-fake, text-fake, and full-fake. The Manipulation-Guided Artifact Perceiving (MGAP) block then pools the attribution output into final real/fake confidences. These refined tokens are concatenated back with $f_c$ and passed through the remaining decoder layers,
$$
P = \mathrm{LLM}_{l+1:L}([f_c \parallel \bar A^{attr}]),
$$
so PMOE acts as an intermediate expertized adapter rather than as a separate classifier head [2508.19639].

This architecture situates PMOE as a token-level intervention inside a frozen or adapted VLM stack, rather than as a full replacement of the base backbone. The paper’s pseudocode further states that training updates all parameters, including Artifact Tokens, expert MLPs, and LoRA adapters, which indicates that PMOE is intended to operate within a parameter-efficient adaptation regime [2508.19639].

## 3. Detection and attribution experts

The Detection MoE has two experts, $\mathrm{Exp}_r$ and $\mathrm{Exp}_f$, and the Attribution MoE has four experts, $\mathrm{Exp}_r$, $\mathrm{Exp}_{fv}$, $\mathrm{Exp}_{ft}$, and $\mathrm{Exp}_{fm}$. Both stages follow a sparse-gating MoE pattern. For each token $i$, PMOE first computes a self-attended token representation,
$$
\bar A = \mathrm{softmax}\!\Bigl(\frac{(AW_q)(AW_k)^\top}{\sqrt{d_k}}\Bigr)(AW_v),
$$
and then obtains token-wise routing probabilities by applying a linear layer and softmax over the expert set [2508.19639].

In the detection stage, the gating rule is
$$
p_i^r, p_i^f = \mathrm{softmax}(W_g \bar A_i), \qquad
z_i = \arg\max\{p_i^r, p_i^f\}.
$$
The routed token is then processed by the selected expert MLP with residual and LayerNorm:
$$
\bar A_i^{out} = \mathrm{LN}\bigl(\mathrm{Exp}_z(\mathrm{LN}(\bar A_i))\bigr) + \bar A_i.
$$
The stage-level authenticity scores are averages over tokens,
$$
p_r = \frac{1}{q}\sum_i p_i^r, \qquad
p_f = \frac{1}{q}\sum_i p_i^f.
$$
The attribution stage applies the same pattern to $\bar A^{det}$, but with four-way routing,
$$
p_i^{r}, p_i^{fv}, p_i^{ft}, p_i^{fm}
= \mathrm{softmax}(W_g' \bar A_i^{det}),
$$
followed by hard expert selection and expert-specific transformation [2508.19639].

The central representational idea is that PMOE does not route the whole multimodal sequence directly. It routes Artifact Tokens that have already absorbed multimodal context from the early LLM layers. This suggests a separation between general multimodal encoding in the backbone and manipulation-sensitive reasoning in the adapter. The paper explicitly characterizes the resulting workflow as a two-step human-style forensics process: coarse “real vs. fake” detection first, then diagnosis of which modality is falsified [2508.19639].

## 4. Objective functions and optimization behavior

PMOE introduces two task-specific losses in addition to the final decoding loss. The Detection MoE is supervised by the Authenticity Probability Guidance loss,
$$
L_{APG} = -[(1-y)\log p_r + y\log p_f],
$$
and MGAP is supervised by the Artifact Classification Loss,
$$
L_{ACL} = -[(1-y)\log p'_r + y\log p'_f].
$$
These are combined as
$$
L_{PMOE} = \frac{L_{APG} + L_{ACL}}{2},
$$
and the overall objective is
$$
L_{total} = L_{CE} + L_{PMOE} + L_{ADEC}.
$$
Thus, PMOE is trained not only through downstream classification but also through auxiliary supervision on intermediate authenticity and attribution signals [2508.19639].

MGAP itself converts the attribution-stage tokens into a final authenticity estimate through learned token weighting:
$$
w = \mathrm{softmax}(FC(\bar A^{attr})),
$$
followed by
$$
[p'_r, p'_f] = \mathrm{MLP}\Bigl(\sum_i w_i \bar A_i^{attr}\Bigr).
$$
This gives PMOE an internal supervision hierarchy: token-level routing, stage-level authenticity guidance, and final decision supervision all coexist in the same end-to-end objective [2508.19639].

The paper also reports an optional entropy regularizer on attribution gating in an ablation setting. Adding that entropy term slightly degrades performance. Within the paper’s framing, this indicates that forcing attribution routing toward a more regularized distribution is not beneficial in this configuration. A plausible implication is that the learned token routing already provides sufficient diversity without an extra balancing term [2508.19639].

## 5. Empirical results and ablation evidence

On the two benchmark datasets named in the paper, FakeSV and FakeTT, FakeSV-VLM with PMOE significantly outperforms current state-of-the-art models by +3.32% and +5.02%, respectively. The ablation sequence on FakeSV further isolates the contribution of PMOE’s internal stages [2508.19639].

| Configuration | FakeSV ACC |
|---|---:|
| Backbone only | 88.38% |
| + Detection MoE | 89.67% |
| + Attribution MoE | 89.85% |
| + MGAP | 90.04% |
| All PMOE components | 90.22% |

This progression shows that each stage contributes additional accuracy, with the full stack performing best. The paper summarizes these gains as evidence that the two-stage expert structure, dynamic routing, and progressive refinement are central to the model’s performance [2508.19639].

The ablations also examine placement and routing quality. Inserting PMOE at layer 12 yields the best performance, while placements such as 8 and 12 or 12 and 16 slightly hurt. Routing analysis reports that the Detection MoE matches final test accuracy with 89.30% ACC on FakeTT, and that the Attribution MoE achieves 82.0% ACC against manually annotated manipulation types. These results are presented as validation that the binary and fine-grained routing stages are not merely auxiliary, but correspond meaningfully to authenticity and manipulation structure in the data [2508.19639].

## 6. Relation to other PMoE variants and recurrent misconceptions

A recurrent misconception is to treat PMOE as a single canonical architecture. The arXiv record instead shows several distinct systems using the same or closely related acronym.

| Variant | Domain | Defining mechanism |
|---|---|---|
| PMoE [2407.21571] | Continual learning for LLMs | Asymmetric transformer with progressively added deep-layer LoRA experts |
| PMoE [2601.09305] | Continual RANS turbulence modelling | Autoencoder router with top-1 expert selection and regime-wise expansion |
| pMoE [2602.22938] | Visual adaptation | Expert-specific prompt tokens with learnable dispatcher |
| PMOE [2508.19639] | Fake short-video news detection | Two-stage Detection/Attribution MoE over Artifact Tokens |

In the continual-learning PMoE for LLMs, shallow layers maintain general knowledge while deep layers accumulate task-specific experts, and a router adjacent to the deep layers allocates new knowledge to the appropriate experts. There, progression means expert growth across tasks, and the architecture emphasizes catastrophic forgetting, backward transfer, and parameter-efficient LoRA adaptation [2407.21571].

In the RANS turbulence PMoE, the router is a bank of autoencoders whose reconstruction errors determine a top-1 expert choice, and a new router/expert pair is spawned when no existing component adequately represents an unseen flow regime. There, progression refers to regime-wise expansion under sparse activation, with the stated aim of avoiding catastrophic forgetting while preserving inference efficiency [2601.09305].

In pMoE for visual adaptation, the “experts” are diverse pre-trained vision encoders whose knowledge is integrated through expert-specific prompt tokens and a learnable dispatcher at multiple prompt layers. In that case, progression is layer-wise prompt fusion rather than staged diagnosis or continual expert spawning [2602.22938].

Against that background, the distinctive property of the FakeSV-VLM PMOE is that its progression is epistemic rather than temporal: the model first decides whether a sample is real or fake, then attributes the manipulation pattern, and only then feeds the refined representation back to the decoder. This distinguishes it from both continual-learning PMoE systems and prompt-level pMoE variants [2508.19639].

Source: https://www.emergentmind.com/topics/progressive-moe-adapter-pmoe