Progressive MoE Adapter (PMOE)

• The paper introduces PMOE as a two-stage expert system that first conducts binary detection of real versus fake content followed by detailed manipulation attribution.
• PMOE is integrated into visual language models using Artifact Tokens and sparse-routing mechanisms, ensuring parameter-efficient adaptation.
• Empirical tests on FakeSV and FakeTT datasets demonstrate that each PMOE component adds cumulative accuracy gains, outperforming backbone-only models by up to 5%.

Searching arXiv for the cited PMoE/PMOE papers and closely related work. Progressive MoE Adapter (PMOE) denotes a Mixture-of-Experts module introduced within the FakeSV-VLM framework for detecting fake news on short video platforms. In that usage, PMOE is a two-stage, hierarchical reasoning mechanism integrated into the early-to-mid layers of a VLM: detection experts first provide an initial analysis, and attribution experts then perform a more comprehensive diagnosis over learnable Artifact Tokens, after which a final decision is produced jointly with the base model decoder. The term is not unique across closely related PMoE or pMoE formulations also appear in continual learning for LLMs, continual RANS turbulence modelling, and visual prompt tuning, but those instantiate different notions of progression, routing, and expert growth (Wang et al., 27 Aug 2025).

1. Definition, scope, and nomenclature

In FakeSV-VLM, PMOE is designed for a setting in which short-video news samples are categorized into four distinct scenarios: both video and text are real, both are fake, only the video is fake, or only the text is fake. The module accordingly combines a coarse binary detection stage with a fine-grained attribution stage. This organization reflects the paper’s claim that fake news videos often show inconsistency between modalities and that diagnosis benefits from progressively refining the decision from authenticity screening to manipulation attribution (Wang et al., 27 Aug 2025).

The term “progressive” has a specific meaning in this setting. It refers to staged refinement of the decision process rather than continual addition of experts over a task stream. A common source of confusion is that “PMoE” in the continual-learning literature instead denotes architectures that progressively add experts as new tasks or regimes arrive. In other words, the acronym is shared, but the operational semantics differ substantially across domains (Jung et al., 2024).

2. Placement within the VLM and end-to-end data flow

PMOE is inserted into the early-to-mid layers of the LLM component. The input pipeline first extracts visual features and textual embeddings,

$f_v = M(E_v(v_1 \ldots v_k)), \qquad f_t = T_k(t),$

then forms a fused representation

$f_c = [f_v \parallel f_t].$

A set of $q$ learnable Artifact Tokens $A \in \mathbb{R}^{q \times d}$ is prepended, and the concatenated sequence is processed by the first $l$ layers of the LLM:

$$[f_c \parallel A] \leftarrow \mathrm{LLM}_{1:l}([f_c \parallel A]).$$

Only after this contextualization does PMOE operate on the Artifact Tokens (Wang et al., 27 Aug 2025).

The module itself is organized into three successive components. The Detection MoE consumes the contextualized Artifact Tokens and performs binary routing between “real” and “fake” experts. The Attribution MoE receives the Detection MoE’s refined tokens and routes them among four experts corresponding to $\{r, fv, ft, fm\}$, namely both real, video-fake, text-fake, and full-fake. The Manipulation-Guided Artifact Perceiving (MGAP) block then pools the attribution output into final real/fake confidences. These refined tokens are concatenated back with $f_c$ and passed through the remaining decoder layers,

$$P = \mathrm{LLM}_{l+1:L}([f_c \parallel \bar A^{attr}]),$$

so PMOE acts as an intermediate expertized adapter rather than as a separate classifier head (Wang et al., 27 Aug 2025).

This architecture situates PMOE as a token-level intervention inside a frozen or adapted VLM stack, rather than as a full replacement of the base backbone. The paper’s pseudocode further states that training updates all parameters, including Artifact Tokens, expert MLPs, and LoRA adapters, which indicates that PMOE is intended to operate within a parameter-efficient adaptation regime (Wang et al., 27 Aug 2025).

3. Detection and attribution experts

The Detection MoE has two experts, $\mathrm{Exp}_r$ and $f_c = [f_v \parallel f_t].$0, and the Attribution MoE has four experts, $f_c = [f_v \parallel f_t].$1, $f_c = [f_v \parallel f_t].$2, $f_c = [f_v \parallel f_t].$3, and $f_c = [f_v \parallel f_t].$4. Both stages follow a sparse-gating MoE pattern. For each token $f_c = [f_v \parallel f_t].$5, PMOE first computes a self-attended token representation,

$f_c = [f_v \parallel f_t].$6

and then obtains token-wise routing probabilities by applying a linear layer and softmax over the expert set (Wang et al., 27 Aug 2025).

In the detection stage, the gating rule is

$f_c = [f_v \parallel f_t].$7

The routed token is then processed by the selected expert MLP with residual and LayerNorm:

$f_c = [f_v \parallel f_t].$8

The stage-level authenticity scores are averages over tokens,

$f_c = [f_v \parallel f_t].$9

The attribution stage applies the same pattern to $q$0, but with four-way routing,

$q$1

followed by hard expert selection and expert-specific transformation (Wang et al., 27 Aug 2025).

The central representational idea is that PMOE does not route the whole multimodal sequence directly. It routes Artifact Tokens that have already absorbed multimodal context from the early LLM layers. This suggests a separation between general multimodal encoding in the backbone and manipulation-sensitive reasoning in the adapter. The paper explicitly characterizes the resulting workflow as a two-step human-style forensics process: coarse “real vs. fake” detection first, then diagnosis of which modality is falsified (Wang et al., 27 Aug 2025).

4. Objective functions and optimization behavior

PMOE introduces two task-specific losses in addition to the final decoding loss. The Detection MoE is supervised by the Authenticity Probability Guidance loss,

$q$2

and MGAP is supervised by the Artifact Classification Loss,

$q$3

These are combined as

$q$4

and the overall objective is

$q$5

Thus, PMOE is trained not only through downstream classification but also through auxiliary supervision on intermediate authenticity and attribution signals (Wang et al., 27 Aug 2025).

MGAP itself converts the attribution-stage tokens into a final authenticity estimate through learned token weighting:

$q$6

followed by

$q$7

This gives PMOE an internal supervision hierarchy: token-level routing, stage-level authenticity guidance, and final decision supervision all coexist in the same end-to-end objective (Wang et al., 27 Aug 2025).

The paper also reports an optional entropy regularizer on attribution gating in an ablation setting. Adding that entropy term slightly degrades performance. Within the paper’s framing, this indicates that forcing attribution routing toward a more regularized distribution is not beneficial in this configuration. A plausible implication is that the learned token routing already provides sufficient diversity without an extra balancing term (Wang et al., 27 Aug 2025).

5. Empirical results and ablation evidence

On the two benchmark datasets named in the paper, FakeSV and FakeTT, FakeSV-VLM with PMOE significantly outperforms current state-of-the-art models by +3.32% and +5.02%, respectively. The ablation sequence on FakeSV further isolates the contribution of PMOE’s internal stages (Wang et al., 27 Aug 2025).

Configuration	FakeSV ACC
Backbone only	88.38%
+ Detection MoE	89.67%
+ Attribution MoE	89.85%
+ MGAP	90.04%
All PMOE components	90.22%

This progression shows that each stage contributes additional accuracy, with the full stack performing best. The paper summarizes these gains as evidence that the two-stage expert structure, dynamic routing, and progressive refinement are central to the model’s performance (Wang et al., 27 Aug 2025).

The ablations also examine placement and routing quality. Inserting PMOE at layer 12 yields the best performance, while placements such as 8 and 12 or 12 and 16 slightly hurt. Routing analysis reports that the Detection MoE matches final test accuracy with 89.30% ACC on FakeTT, and that the Attribution MoE achieves 82.0% ACC against manually annotated manipulation types. These results are presented as validation that the binary and fine-grained routing stages are not merely auxiliary, but correspond meaningfully to authenticity and manipulation structure in the data (Wang et al., 27 Aug 2025).

6. Relation to other PMoE variants and recurrent misconceptions

A recurrent misconception is to treat PMOE as a single canonical architecture. The arXiv record instead shows several distinct systems using the same or closely related acronym.

Variant	Domain	Defining mechanism
PMoE (Jung et al., 2024)	Continual learning for LLMs	Asymmetric transformer with progressively added deep-layer LoRA experts
PMoE (Ji et al., 14 Jan 2026)	Continual RANS turbulence modelling	Autoencoder router with top-1 expert selection and regime-wise expansion
pMoE (Mo et al., 26 Feb 2026)	Visual adaptation	Expert-specific prompt tokens with learnable dispatcher
PMOE (Wang et al., 27 Aug 2025)	Fake short-video news detection	Two-stage Detection/Attribution MoE over Artifact Tokens

In the continual-learning PMoE for LLMs, shallow layers maintain general knowledge while deep layers accumulate task-specific experts, and a router adjacent to the deep layers allocates new knowledge to the appropriate experts. There, progression means expert growth across tasks, and the architecture emphasizes catastrophic forgetting, backward transfer, and parameter-efficient LoRA adaptation (Jung et al., 2024).

In the RANS turbulence PMoE, the router is a bank of autoencoders whose reconstruction errors determine a top-1 expert choice, and a new router/expert pair is spawned when no existing component adequately represents an unseen flow regime. There, progression refers to regime-wise expansion under sparse activation, with the stated aim of avoiding catastrophic forgetting while preserving inference efficiency (Ji et al., 14 Jan 2026).

In pMoE for visual adaptation, the “experts” are diverse pre-trained vision encoders whose knowledge is integrated through expert-specific prompt tokens and a learnable dispatcher at multiple prompt layers. In that case, progression is layer-wise prompt fusion rather than staged diagnosis or continual expert spawning (Mo et al., 26 Feb 2026).

Against that background, the distinctive property of the FakeSV-VLM PMOE is that its progression is epistemic rather than temporal: the model first decides whether a sample is real or fake, then attributes the manipulation pattern, and only then feeds the refined representation back to the decoder. This distinguishes it from both continual-learning PMoE systems and prompt-level pMoE variants (Wang et al., 27 Aug 2025).