---
title: 'FedPromo: Federated Prompt Learning'
url: https://www.emergentmind.com/topics/fedpromo
type: topic
---

# FedPromo: Federated Prompt Learning

Searching arXiv for “FedPromo” and closely related federated prompt-learning papers to ground the article.
FedPromo is not the title of a canonical arXiv method; the term most plausibly denotes **federated prompt learning**, especially **FedPrompt**, which combines prompt tuning with federated learning by freezing the backbone model and communicating only soft prompt parameters [2208.12268]. In the broader research lineage, the same design pattern has been extended from NLP with pre-trained language models to vision-language models, heterogeneous personalized federation, and continual web personalization. Across these variants, the central idea is stable: prompts become the principal trainable interface between private client data and a frozen or mostly frozen foundation model, so that communication, privacy exposure, and adaptation overhead are reduced while personalization remains possible.

## 1. Terminological scope and research lineage

Within the arXiv literature, “FedPromo” is best understood as an umbrella label for **federated prompt-based adaptation** rather than a single standardized algorithm. The clearest naming match is **“FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learning”** [2208.12268]. Later work generalizes the same paradigm in several directions: personalized multimodal prompt groups in **FedMGP** [2511.00480], global-local prompt cooperation with optimal transport in **FedOTP** [2403.00041], and dual-timescale prototype-aligned soft prompts for continual personalization in **ProtoFed-SP** [2603.27678].

| Method | Setting | Core mechanism |
|---|---|---|
| FedPrompt [2208.12268] | Federated NLP with PLMs | Model split aggregation over soft prompts only |
| FedOTP [2403.00041] | Federated VLMs under label and feature shift | Global and local prompts with unbalanced OT |
| FedMGP [2511.00480] | Personalized federated VLMs | Multi-group paired text-visual prompts with dynamic aggregation |
| ProtoFed-SP [2603.27678] | Continual web personalization | Dual-timescale soft prompts with prototype anchoring |

This progression suggests a consistent conceptual shift in federated learning. Instead of treating the entire model as the object of communication and optimization, prompt-learning methods treat a lightweight prompt subspace as the federated object. A plausible implication is that “FedPromo” functions less as a precise bibliographic identifier than as a shorthand for this prompt-centric federated design family.

## 2. Canonical formulation: FedPrompt

FedPrompt formalizes federated prompt tuning for PLMs such as **BERT, RoBERTa, and T5** by splitting the model into fixed PLM parameters \(F\) and trainable soft prompt parameters \(P\), then aggregating only \(P\) across clients [2208.12268]. The learning problem is written as
\[
\arg \min_{P} \mathcal{L}(P) = \sum_{k=1}^{K} \frac{n_k}{n} \mathcal{L}_k(P),
\]
with per-client loss
\[
\mathcal{L}_k(P) = \mathbf{E}_{(x^{(i)}, y^{(i)}) \in \mathcal{D}_k} \ell_k(f(x^{(i)}, \mathbf{p}, P), y^{(i)}).
\]
At round \(t\), the server distributes the fixed PLM and the current prompt, each selected client sets \(P_t^k=P_t\), performs local prompt-only optimization, and the server aggregates
\[
P_{t+1} \gets \sum_{k=1}^{\lceil C \cdot K \rceil} \frac{n_k}{N_t} P_t^k,
\qquad
N_t = \sum_{k=1}^{\lceil C \cdot K \rceil} n_k .
\]

The method’s defining property is communication efficiency. The paper states that split aggregation reduces communication cost to about **0.01% of the full PLM parameters**. On IID SST-2, the reported communicated parameter counts are **0.016M vs 109.530M** for BERT, **0.016M vs 124.714M** for RoBERTa, and **0.015M vs 222.919M** for T5, corresponding to **0.014%**, **0.013%**, and **0.007%**. The main experimental configuration uses **20** soft prompt tokens, **10 clients**, **20** communication rounds, and **1000** max local steps.

Accuracy remains close to full fine-tuning. On IID SST-2, FedPrompt reports **90.16% vs 91.02%** for BERT, **92.43% vs 93.57%** for RoBERTa, and **92.69% vs 93.79%** for T5. The method is evaluated on **SST-2, IMDB, OffensEval, Twitter, Enron, Lingspam, QNLI, and RTE**, under both IID and non-IID splits, with the paper noting that the IID vs non-IID drop is usually small, often **less than 3%**. A notable weak case is **RTE**, attributed to its **2240 training samples total** and the consequent scarcity per client after a 10-way split.

The original FedPrompt paper also frames privacy and robustness as part of the method’s scope. Only prompt vectors are transmitted, and the paper reports that ordinary backdoor poisoning is not sufficient to produce a high global attack success rate after aggregation. Local differential privacy can be added by clipping gradients and adding Laplace noise, although the reported SST-2 accuracy drops by about **5%**.

## 3. Personalization and multimodal generalization

As federated prompt learning moved from PLMs to VLMs, the main difficulty shifted from pure communication reduction to **heterogeneity-aware personalization**. FedOTP addresses clients with both **label shift** and **feature shift** by assigning each client a global prompt \(P_g\) and a local prompt \(P_{l,i}\), written as \(P_i=[P_g,P_{l,i}]\), and aligning them with patch-level visual features through **unbalanced optimal transport** [2403.00041]. For class \(k\), the cost matrix is
\[
C = 1 - G_m^\top H_k \in \mathbb{R}^{V \times 2},
\]
and the relaxed transport constraint
\[
T\mathbf{1}_2 \le \alpha,\qquad T^\top \mathbf{1}_V = \beta
\]
allows prompts to focus on core image regions rather than forcing all patches to be matched. The paper reports strong results under severe heterogeneity, including **92.73** on Food101, **87.67** on DTD, **97.02** on Caltech101, **96.23** on Flowers102, **98.82** on OxfordPets, **96.05** on CIFAR-10, **78.03** on CIFAR-100, **48.08** average on DomainNet, and **99.16** on Office-Caltech10 under feature shift only.

FedMGP extends personalized federated prompt learning for VLMs by learning **multiple groups of paired textual and visual prompts** per client rather than a single prompt representation [2511.00480]. Each group functions as a prompt expert. A **diversity loss** is introduced so that the groups specialize in distinct and complementary semantic aspects, and a **dynamic prompt aggregation** mechanism computes cosine similarities to prior global prompts, converts them into a softmax-weighted distribution, and samples \(s\) groups for aggregation. The stated goal is to balance **shared knowledge** and **client-specific knowledge** while maintaining parameter efficiency by redistributing a fixed prompt capacity across groups. The paper describes FedMGP as achieving **state-of-the-art performance with the lowest communication parameters among all federated prompt learning methods** on federated vision-language benchmarks including **Caltech101, OxfordPets, Flowers102, Food101, DTD, CIFAR-10, CIFAR-100, Stanford Cars, FGVC Aircraft, SUN397, and UCF101**.

Taken together, these methods redefine the meaning of federated prompt learning in multimodal settings. Prompt parameters are no longer merely lightweight substitutes for full fine-tuning; they become structured personalization carriers, with explicit mechanisms for expert specialization, global-local cooperation, and alignment under heterogeneity.

## 4. Continual personalization and prototype-anchored prompting

ProtoFed-SP transfers the federated prompt-learning paradigm to **continual web personalization**, where user interests drift over time and raw-data replay is undesirable [2603.27678]. The backbone recommender \(f_\theta\) is frozen, and each user maintains a slow long-term prompt \(p_u^{long}\) and a fast short-term prompt \(p_{u,t}^{short}\). The composed prompt is
\[
p_{u,t} = p^{long}_u + \alpha_{u,t} p^{short}_{u,t} + \sum_{k \in \mathrm{Top\text{-}M}(q_{u,t})} w_k(q_{u,t}) c_k .
\]
Here the short-term prompt is the “fast, sparse” component tracking session intent, while the long-term prompt is the “slow” stable preference memory. The term involving \(c_k\) injects a mixture of retrieved server-side prototypes, which act as a shared semantic prior.

The short-term prompt is updated by a sparse proximal step,
\[
p^{short}_{u,t} \leftarrow \mathrm{SoftThresh}\!\left(p^{short}_{u,t} - \eta_s \nabla_{p^{short}} \mathcal{L}_{rec}(u,t),\; \eta_s \lambda_p \right),
\]
and the session-adaptive factor \(\alpha_{u,t}=\sigma(a^\top \Delta_{u,t})\) increases plasticity when drift is strong. The long-term prompt is updated more slowly through an alignment-aware proximal problem, with the alignment loss including a nearest-anchor contraction term plus an InfoNCE-style sharpening term; a Wasserstein alternative is also given. On the server side, clients upload only compressed long-term prompt embeddings,
\[
z_u = \mathrm{compress}(\phi(p^{long}_u)) + \xi,
\qquad
\xi\sim \mathcal{N}(0,\sigma^2 I),
\]
and the server refreshes prototypes using **DP-FedKMeans**, a geometric median update, or a **2-Wasserstein barycenter** update, while enforcing a minimum separation constraint
\[
\min_{i\neq j}\|v_i-v_j\|_2 \ge \rho .
\]

Experimentally, ProtoFed-SP is evaluated on **Amazon-Books, Amazon-Electronics, MovieLens-20M, Yelp, RetailRocket, Gowalla, Taobao, and H&M**. It is reported as best on all eight datasets for both **NDCG@10** and **HR@10**, with average gains of **+2.9%** and **+2.0%** over the strongest baseline.

| Dataset | NDCG@10 | HR@10 |
|---|---:|---:|
| Amazon-Books | 0.126 | 0.248 |
| Amazon-Electronics | 0.142 | 0.271 |
| MovieLens-20M | 0.329 | 0.658 |
| Yelp | 0.119 | 0.233 |
| RetailRocket | 0.257 | 0.487 |
| Gowalla | 0.216 | 0.534 |
| Taobao | 0.231 | 0.461 |
| H&M | 0.284 | 0.512 |

The paper highlights **Amazon-Books** with about **+5.0% NDCG vs. INFER**, **H&M** with about **+2.5% vs. Dual-LoRA**, and **Taobao** with about **+2.2% vs. FedRAP**. In a concrete time-slice example on Amazon-Books at \(t=8\), alignment yields **AF about 0.009 versus about 0.024 without alignment**, while the full model reaches **Steps-to-95% about 170 versus about 270 when the short-term prompt is removed**. These results formalize a prompt-based solution to the stability-plasticity dilemma rather than merely a communication-saving trick.

## 5. Privacy, robustness, and efficiency characteristics

Across the federated prompt-learning family, three properties recur: **parameter efficiency**, **privacy-aware communication**, and **robustness under heterogeneity**. FedPrompt transmits only prompt parameters and therefore reduces the communication object from hundreds of millions of parameters to approximately **0.01%** of a PLM; the paper further reports that ordinary backdoor attack experiments yield low global ASR after aggregation [2208.12268]. FedMGP explicitly states that it maintains parameter efficiency by redistributing a fixed prompt capacity across multiple groups, and that it achieves the **lowest communication parameters among all federated prompt learning methods** while improving personalization and domain generalization [2511.00480].

ProtoFed-SP makes the privacy layer more explicit by applying differential privacy to uploaded compressed prompt embeddings rather than raw interactions or gradients [2603.27678]. The implementation uses \(\delta=10^{-5}\) and noise scales \(\sigma\in\{0.2,0.4,0.8\}\), with privacy reported via a moments accountant over rounds. The reported privacy-utility tradeoff is that **moderate DP retains most utility**, while at \(\sigma=0.8\) performance drops by around **1–2.5 points** but remains usable, with **no catastrophic forgetting**. The paper notes that **Taobao is the most sensitive** and **MovieLens-20M the most robust**.

A common misconception is that prompt-only federation is simply a weaker approximation to full fine-tuning. The empirical record is more specific. Prompt-only methods do impose capacity limits, but they also create a control surface for selective adaptation, local-global disentanglement, and privacy-aware aggregation. This suggests that the reduction in trainable dimensionality is not merely a systems concession; in several settings it is part of the learning bias that enables personalization under communication and privacy constraints.

## 6. Limitations, misconceptions, and adjacent methods

The literature also defines the boundaries of what “FedPromo” should not denote. **FedPop** is **Federated Population-based Hyperparameter Tuning**, an online evolutionary tuner for server-side and client-side hyperparameters in federated learning; it is not a prompt-learning method [2308.08634]. **FedProxy** is a federated LLM fine-tuning framework based on **Proxy SLMs** and **heterogeneity-aware fusion**; it addresses IP protection, client privacy, and non-IID interference through proxy-model compression and plug-in fusion rather than prompt tuning [2604.19015]. Similar naming prefixes therefore do not imply methodological equivalence.

Within prompt-based federation itself, limitations are explicit. FedPrompt identifies future directions in **personalization**, **stronger privacy mechanisms**, and **more robust FL-specific backdoor defenses**. FedOTP notes that **generalization to unseen novel clients** has not been fully studied. ProtoFed-SP states that it **relies on frozen backbones**, uses **simulated federated settings with accountant-based DP estimates**, and depends on **manual choices for prototype capacity, separation, and refresh rules**. FedMGP, while emphasizing performance and communication efficiency, introduces added design complexity through prompt grouping, diversity regularization, and similarity-guided aggregation.

The most precise encyclopedic interpretation is therefore narrow but layered: **FedPromo** most closely names the research thread initiated by FedPrompt, then expanded into personalized, multimodal, and continual variants. It is not a single settled algorithm, and it should not be conflated with other “Fed-” methods that target hyperparameter tuning, proxy-model fine-tuning, or unrelated federated objectives.

Source: https://www.emergentmind.com/topics/fedpromo