---
title: 'FedMGP: Federated Multi-Group Paradigm'
url: https://www.emergentmind.com/topics/fedmgp
type: topic
---

# FedMGP: Federated Multi-Group Paradigm

Searching arXiv for papers explicitly using the term “FedMGP” and closely related variants to ground the article.
FedMGP denotes a small but heterogeneous cluster of federated-learning methods centered on selective knowledge sharing under client heterogeneity. In current arXiv usage, the term is used most explicitly for prompt-based personalization in vision-language and continual-learning settings, while a broader probabilistic usage connects it to federated multi-output Gaussian-process modeling [2511.00480][2407.00113][2407.16935]. This suggests that FedMGP is best understood as a family label organized around a common design principle: preserve local specificity, communicate only compact shared structure, and avoid raw-data centralization.

## 1. Terminological scope

In the arXiv record, “FedMGP” does not name a single universally standardized algorithm. Instead, it appears in at least three distinct but related contexts.

| Usage | Paper | Defining idea |
|---|---|---|
| Personalized federated prompt learning | "FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts" [2511.00480] | Multiple groups of paired textual and visual prompts |
| Personalized federated continual learning | "Personalized Federated Continual Learning via Multi-granularity Prompt" [2407.00113] | Coarse-grained global prompt and fine-grained local prompt |
| Federated multi-output Gaussian processes | "Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes" [2407.16935] | Federated LMC with spike-and-slab latent selection |

The first usage is the most literal: FedMGP is introduced as “a new paradigm for personalized federated prompt learning in vision-language models,” in which each client maintains “multiple groups of paired textual and visual prompts” and uses “a dynamic prompt aggregation strategy based on similarity-guided probabilistic sampling” [2511.00480]. The second usage is structurally related but targets Personalized Federated Continual Learning rather than vision-language prompt learning; there the core notion is a “multi-granularity prompt,” with explicit separation between shared and personalized knowledge [2407.00113]. The third usage is more semantic than titular: the Gaussian-process paper frames the topic as a federated multi-output Gaussian process framework, but the concrete proposed model is named FedLMC-SS rather than FedMGP [2407.16935].

A common misconception is that every paper near this acronym refers to the same method. The literature does not support that reading. The term is overloaded, and careful disambiguation is necessary.

## 2. FedMGP as multi-group text-visual prompt learning

In its most direct arXiv usage, FedMGP is a personalized federated prompt-learning method for vision-language models. The method equips “each client with multiple groups of paired textual and visual prompts,” with the stated aim of capturing “diverse, fine-grained semantic and instance-level cues” [2511.00480]. Rather than using a single prompt block, the method redistributes a fixed prompt budget across several groups.

The organizing mechanism is a diversity constraint over prompt groups. A “diversity loss” is introduced so that “each prompt group” specializes in “distinct and complementary semantic aspects,” and the groups “collectively cover a broader range of local characteristics” [2511.00480]. This makes the prompt decomposition explicitly semantic rather than merely architectural.

Communication is governed by a selective aggregation rule. FedMGP uses “a dynamic prompt aggregation strategy based on similarity-guided probabilistic sampling”: each client computes “the cosine similarity between its prompt groups and the global prompts from the previous round,” then “samples $s$ groups via a softmax-weighted distribution” [2511.00480]. The intended effect is to “preferentially aggregate semantically aligned knowledge while still enabling exploration of underrepresented patterns,” thereby balancing “the preservation of common knowledge with client-specific features” [2511.00480].

The paper also emphasizes communication efficiency. FedMGP “maintains parameter efficiency by redistributing a fixed prompt capacity across multiple groups,” and reports “state-of-the-art performance with the lowest communication parameters among all federated prompt learning methods” [2511.00480]. Its theoretical claim is that the dynamic aggregation strategy “promotes robust global representation learning by reinforcing shared semantics while suppressing client-specific noise,” while the empirical claim is consistent gains in “both personalization and domain generalization across diverse federated vision-language benchmarks” [2511.00480].

Within the broader prompt-learning literature, this positions FedMGP as a personalization-oriented alternative to single-prompt or uniformly averaged prompt-sharing schemes. A plausible implication is that it treats prompt groups as a structured communication primitive, not merely as trainable adapters.

## 3. FedMGP as multi-granularity prompt in personalized federated continual learning

A second established usage of the name is the PFCL method introduced in "Personalized Federated Continual Learning via Multi-granularity Prompt" [2407.00113]. Here FedMGP stands for **Federated Multi-Granularity Prompt** and is motivated by what the paper calls **Spatial-Temporal Catastrophic Forgetting (STCF)**. The central design is a two-level prompt space built on a pre-trained frozen ViT backbone: a **coarse-grained global prompt** for shared knowledge and a **fine-grained local prompt** for client-, task-, and class-specific knowledge [2407.00113].

The PFCL setting is formalized over clients
$$
\mathcal{A}=\{A_1, A_2, \ldots, A_a\}
$$
with per-client task streams
$$
\mathcal{T}_i=\{T_i^1, T_i^2, \ldots, T_i^{n_i}\}.
$$
The paper distinguishes server-side generalized knowledge fusion from client-side continual adaptation and personalization. To measure forgetting, it defines temporal knowledge retention
$$
KR_t=\frac{1}{a}\sum_{i=1}^a\frac{Acc(\theta^r_i;T^0_i)}{Acc(\theta^0_i;T^0_i)},
$$
and spatial knowledge retention
$$
KR_s=\frac{1}{a}\sum_{i=1}^a\frac{Acc(\theta^r_g;T^r_i)}{Acc(\theta^r_i;T^r_i)}.
$$
These metrics are intended to separate temporal forgetting on each client from degradation caused by federated aggregation [2407.00113].

The coarse-grained global prompt pool is written as
$$
\mathcal{P}_g=\{P_g^1, P_g^2, \ldots, P_g^M\},
$$
with prompt-augmented embedding
$$
E' = \left[ P_g^{s_1}, \ldots, P_g^{s_k} ; E\right], \quad 1 \leq N \leq M.
$$
Prompt selection is driven by key-query matching:
$$
K_g^{in} = \mathcal{V}(E),
$$
$$
\mathcal{K}^s_g = \underset{\mathcal{K}_g}{\operatorname{argmin} \sum_{i=1}^{N} \text{dis}(K_g^{in}, K_g^{i}).
$$
The corresponding optimization objective is
$$
\underset{H^i_g,\mathcal{P}_g,\mathcal{K}_g}{\operatorname{min} \mathcal{L}(\mathcal{V}^i_g(E'),y)+ \lambda_1 \sum_{\mathcal{K}^{s}_g} \operatorname{dis}(K_g^{in},K_g^{s_i}).
$$
This part is explicitly designed to capture “temporal-spatial invariant knowledge” and to support server aggregation without transmitting the full model [2407.00113].

The fine-grained local prompt is class-wise,
$$
\mathcal{P}_l=\{(K_l^1,P_l^1), (K_l^2,P_l^2), \ldots, (K_l^C,P_l^C)\},
$$
and is injected into the multi-head self-attention layers through Prefix Tuning. The paper writes
$$
\text{MSA}(h_q,h_k,h_v) = \operatorname{Concat}\left(\mathrm{h}_{1}, \ldots, \mathrm{h}_{\mathrm{z}\right) W^{O},
$$
with
$$
h_i = \operatorname{Attention}({h}_{Q} W_{i}^{Q}, {h}_{K} W_{i}^{K}, {h}_{V} W_{i}^{V}),
$$
and the modified attention as
$$
\operatorname{MSA}' = \operatorname{MSA}(h_Q,\left[ p_V ; h_K\right],\left[ p_V ; h_V\right]).
$$
The paper notes that this notation is slightly inconsistent as written, but the intended role is clear: local prompts alter attention computation to encode finer-grained local knowledge [2407.00113].

Aggregation does not use ordinary FedAvg over full parameters. Instead, the method introduces **Selective Prompt Fusion** with a server proxy dataset $\mathcal{D}_s$. For two prompt pools $\mathcal{P}_g^i$ and $\mathcal{P}_g^j$, the fusion loss is written as
$$
\mathcal{L}_{CE} = \underset{x_s \in \mathcal{D_s}}{\operatorname{MSE}(\mathcal{V}(E'_i),\mathcal{V}(E'_j)).
$$
Only coarse-grained global prompts are fused; local prompts remain private [2407.00113].

Experimentally, the method is evaluated on CIFAR-100 with 5 clients and 5 tasks per client. Under the asynchronous setting, average accuracies include **FedViT: 82.46**, **FedL2P: 90.11**, and **FedMGP: 90.56**; under the synchronous setting, **FedL2P: 81.89** and **FedMGP: 83.46** are reported [2407.00113]. The ablations indicate that removing local prompts causes temporal retention to drop dramatically, while removing global prompts reduces spatial transfer and personalization. This supports the paper’s core division of labor: coarse prompts for shareable knowledge, fine prompts for continual personalized adaptation.

## 4. FedMGP in federated multi-output Gaussian processes

In a broader probabilistic usage, FedMGP refers to federated multi-output Gaussian-process modeling rather than prompt learning. The most relevant arXiv paper is "Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes" [2407.16935]. There, the concrete model is **FedLMC-SS**, described as “a federated linear model of coregionalization (LMC) with spike-and-slab priors over latent-process coefficients” [2407.16935].

The underlying multi-output GP is written in LMC form as
$$
f_m(x_{m,n}) = \sum_{l=1}^L w_{m,l} u_l(x_{m,n}),
$$
where each latent process satisfies
$$
u_l(\cdot) \sim \mathcal{GP}(0, k_l(\cdot,\cdot;\theta_l)).
$$
Cross-output covariance is induced through shared latent functions:
$$
\mathrm{Cov}\!\left[f_m(x_{m,n}), f_{m'}(x_{m',n'})\right]
= \sum_{l=1}^{L} b_{m,m'}^l \, k_l(x_{m,n},x_{m',n'};\theta_l),
$$
with
$$
b_{m,m'}^l = w_{m,l} w_{m',l}.
$$
This is a transfer-learning construction in which units share latent structure but retain unit-specific loadings [2407.16935].

Automatic latent-process selection is introduced through spike-and-slab priors:
$$
p(w) = \prod_{m=1}^M \prod_{l=1}^L \left[ \pi \,\mathcal{N}(0,\sigma_w^2) + (1-\pi)\delta_0(w_{m,l}) \right].
$$
The model is reparameterized as
$$
w_{m,l} = \tilde w_{m,l} \alpha_l,
$$
with Bernoulli selectors $\alpha_l \in \{0,1\}$. This enables the model to “automatically select only needed latent processes by shrinking the coefficients of unnecessary ones to zero” [2407.16935].

Inference is variational. The ELBO is
$$
\log p(y\mid X,Z) \ge L_{\mathrm{ELBO}}(\eta),
$$
with client-decomposable objective
$$
L_{\mathrm{ELBO}}(\eta) = \sum_{m=1}^M V_m(\phi_m,\psi;\mathcal{D}_m).
$$
That decomposition makes federated optimization possible: clients optimize local terms using only local data, then the server aggregates shared parameters in FedAvg-style form,
$$
\psi^h = \sum_{m=1}^M \frac{N_m}{N}\,\psi_m^h.
$$
The paper also gives a local adaptation mechanism for a new unit by fixing the shared latent structure and optimizing only the new unit’s local objective [2407.16935].

Empirically, the paper reports simulation and case studies on Li-ion battery degradation and air temperature data. In a simulation with six true latent eigenfunctions and ten candidate latent functions, average MSEs include **LMC-SS: $0.0454 \pm 0.0170$** and **FedLMC-SS: $0.0942 \pm 0.0741$**, while FedLMC-SS selects on average about **6.7** latent functions [2407.16935]. This usage of FedMGP is therefore probabilistic, uncertainty-aware, and regression-oriented, in contrast to the prompt-based personalization methods.

## 5. Adjacent acronyms and frequent confusions

The literature around FedMGP is unusually prone to acronym collision. Several nearby methods are distinct and should not be conflated with FedMGP.

Most directly, **Fed-PMG** in "Federated Pseudo Modality Generation for Incomplete Multi-Modal MRI Reconstruction" is not FedMGP; it addresses missing-modality MRI reconstruction through pseudo modality generation in frequency space and reports that clustering reduces extra information-sharing cost by about **97.5%** relative to direct amplitude-spectrum sharing [2308.10910]. **FedGradMP** in "Federated Gradient Matching Pursuit" is a sparse optimization algorithm for federated learning, not a prompt method or Gaussian-process model [2302.10755]. **FedMGS** in "Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approach" handles modality imbalance in multimodal federated graph learning through latent semantic synthesis, not prompt personalization [2606.20382]. **FedMGDA+** in "Federated Learning Meets Multi-objective Optimization" is a Pareto-oriented multi-objective federated optimization method rather than a FedMGP variant [2006.11489].

Other confusions are conceptual rather than nominal. "Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management" proposes **F-MADRL**, a federated PPO-based controller for decentralized multi-microgrid energy management, which is relevant only if “FedMGP” is being used loosely to mean federated optimization for microgrids [2301.00641]. "Federated Martingale Posterior Samping" proposes **FMP**, a one-shot federated approximation to martingale-posterior inference, not FedMGP [2605.18554]. "Federated Gaussian Mixture Models" proposes **FedGenGMM**, again outside the prompt-learning and Gaussian-process usages of FedMGP [2506.01780].

The practical implication is straightforward: the acronym alone is insufficient for bibliographic precision. Title-level disambiguation is often necessary.

## 6. Recurring design principles and research significance

Despite the diversity of usages, the FedMGP family exhibits a stable set of architectural ideas. First, all variants split model state into **shareable global structure** and **private local structure**. In the multi-group text-visual prompt formulation, this appears as prompt groups selectively sampled for communication [2511.00480]. In PFCL, it appears as coarse-grained global prompts versus fine-grained local prompts [2407.00113]. In the Gaussian-process setting, it appears as global latent processes and local loading coefficients [2407.16935].

Second, communication is deliberately **structure-aware** rather than uniform. FedMGP for vision-language models uses similarity-guided probabilistic sampling over prompt groups [2511.00480]. The PFCL variant communicates only coarse prompts and keys, with reported transmitted size **84,480 parameters**, while total trainable parameters per client are **4,700,160** [2407.00113]. The Gaussian-process formulation aggregates only global variational parameters while retaining personalized parameters on-device [2407.16935]. This suggests that FedMGP methods generally reject indiscriminate full-parameter averaging in favor of semantically or statistically filtered communication.

Third, FedMGP methods are designed for **non-IID personalization** rather than purely global optimization. The multi-group text-visual prompt version targets both “personalization and domain generalization” [2511.00480]. The PFCL version explicitly addresses federated heterogeneity and sequential task arrival under STCF [2407.00113]. The Gaussian-process version supports “a new unit within our proposed federated framework” by reusing previously learned shared latent structure [2407.16935]. Across domains, the shared objective is not only privacy-preserving collaboration, but also client-specific adaptation without forfeiting cross-client transfer.

A plausible synthesis is that FedMGP names a methodological direction rather than a single closed-form recipe. Its characteristic move is to federate only the representation components most likely to encode common structure—prompt groups, coarse prompts, latent processes—while leaving task- or client-specific detail local. That principle links the otherwise disparate prompt-learning, continual-learning, and Gaussian-process branches of the literature.

Source: https://www.emergentmind.com/topics/fedmgp