FedBPrompt: Federated Domain Generalization for ReID

• FedBPrompt is a federated domain generalization framework for person re-identification that uses body-distribution-aware prompts to guide Vision Transformer attention under decentralized, heterogeneous training data.
• It integrates holistic full-body prompts and body part alignment prompts to suppress background bias and ensure viewpoint-invariant part alignment without external body annotations.
• The prompt-based fine-tuning strategy updates only lightweight prompt parameters, reducing communication costs by over 99% while achieving significant gains on multiple ReID datasets.

Searching arXiv for the target paper and closely related context. Searching arXiv for "FedBPrompt federated domain generalization person re-identification". FedBPrompt is a federated domain generalization method for person re-identification that augments Vision Transformers with body-distribution-aware visual prompts in order to steer attention toward pedestrian-centric regions under decentralized, heterogeneous training data (Xu et al., 13 Mar 2026). It addresses the FedDG-ReID setting, in which multiple institutions collaboratively train without sharing raw data, while the resulting global model must generalize to unseen domains. The method combines a Body Distribution Aware Prompt Mechanism that organizes prompts into holistic and part-level components with a Prompt-based Fine-Tuning Strategy that freezes the ViT backbone and communicates only lightweight prompt parameters, thereby targeting both cross-client representation misalignment and federated communication cost.

1. Problem setting and motivation

FedDG-ReID seeks domain-invariant identity representations from decentralized data, with each source dataset treated as a separate client and privacy constraints preventing centralization (Xu et al., 13 Mar 2026). The task is difficult because person re-identification is open-set, client distributions are heterogeneous, and the final model is evaluated on unseen domains. In this regime, client drift and feature misalignment are amplified.

FedBPrompt is motivated by a specific limitation of ViT-based ReID models in federated domain generalization. Although ViT is a widely adopted backbone, its global self-attention can focus on dominant but irrelevant backgrounds such as shadows, walls, or road textures, and it is brittle under viewpoint and pose changes that alter the apparent arrangement of body parts. In decentralized ReID, these issues compound across clients because different institutions contribute data with different cameras, viewpoints, and environmental statistics.

The method adopts prompts as a parameter-efficient mechanism for injecting task priors into a frozen or partially frozen transformer. In FedBPrompt, the prior is explicitly body-distribution-aware: prompts are designed to bias attention toward pedestrian-centric tokens, encode full-body appearance, and impose coarse part-level structure without using external human parsing or pose estimation. This design suggests a prompt-based alternative to explicit auxiliary supervision for body structure, while retaining compatibility with standard federated training.

2. Body-distribution-aware prompt architecture

The model uses a ViT-B/16 backbone. In prompt-tuning mode, the backbone serves as a frozen feature extractor; in full-parameter training, it can be trainable (Xu et al., 13 Mar 2026). FedBPrompt introduces two prompt families within the Body Distribution Aware Prompt Mechanism (BAPM):

Component	Function
Holistic Full Body Prompts (HFBP)	Capture overall appearance and suppress background bias
Body Part Alignment Prompts (BPAP)	Model upper, middle, and lower regions for viewpoint-invariant part alignment
Prompt-based Fine-Tuning Strategy (PFTS)	Freezes the ViT backbone and updates only prompts

Prompt tokens are concatenated with the CLS token and image patch tokens at every transformer layer. If an image $I$ is divided into $N$ patches with embeddings $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$, the learnable prompt tokens are defined as holistic prompts $P_h \in \mathbb{R}^{m_h \times D}$ and part prompts $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$ for $k = 1, \dots, K$. The augmented input to the first layer is

$$X^{(0)} = [CLS; P_h; P_p^{(1)}; \dots; P_p^{(K)}; x_1; \dots; x_N] \in \mathbb{R}^{(1 + m_h + K \cdot m_p + N)\times D}.$$

FedBPrompt inserts prompts at every transformer layer $i$, each with its own prompt set $P_{i-1}$, and updates them through

$$[x_i, E_i] = L_i([x_{i-1}, P_{i-1}, E_{i-1}]), \qquad i = 1, \dots, L,$$

where $N$0 is the CLS-token output and $N$1 is the set of updated patch embeddings.

The prompt configuration is fixed and lightweight. The implementation uses 50 prompts per layer: 35 HFBP prompts and 15 BPAP prompts, with BPAP divided equally into 5 prompts for each of the upper, middle, and lower regions. The prompt dimension $N$2 matches the ViT embedding size, such as 768 for ViT-B/16.

3. Attention control and body distribution modeling

FedBPrompt modifies standard multi-head self-attention by inserting prompt tokens directly into the token sequence, so that prompts contribute to the queries, keys, and values (Xu et al., 13 Mar 2026). For an input sequence $N$3,

$N$4

where $N$5 is the head dimension. Because prompts are part of $N$6, they influence attention weights and information flow throughout the transformer.

BAPM further constrains attention with a structured additive mask $N$7 over the attention logits. The prompt set is partitioned into $N$8 for BPAP and $N$9 for HFBP. For an $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$0 patch grid with $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$1, the patch index sets are defined by row-major flattening as

$x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$2

At layer $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$3, BAPM constrains $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$4 to attend only to $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$5, $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$6 only to $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$7, and $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$8 only to $x = [x_1, \dots, x_N] \in \mathbb{R}^{N \times D}$9, while $P_h \in \mathbb{R}^{m_h \times D}$0 can attend to all image tokens. The masked attention becomes

$P_h \in \mathbb{R}^{m_h \times D}$1

with

$P_h \in \mathbb{R}^{m_h \times D}$2

Prompt-to-prompt communication is always allowed, so $P_h \in \mathbb{R}^{m_h \times D}$3 for all $P_h \in \mathbb{R}^{m_h \times D}$4. This permits HFBP and BPAP tokens to exchange information freely while maintaining region-specific interactions with image patches.

A central property of this design is that body distribution is implemented through spatial partitioning of patch indices rather than external parsing or pose cues. The upper, middle, and lower prompts provide coarse alignment fields, while the full-body prompts integrate these cues globally to suppress background distraction. No explicit part-alignment loss is required; alignment is induced by the attention constraints themselves.

4. Training objective and federated optimization

FedBPrompt uses standard ReID objectives for local optimization (Xu et al., 13 Mar 2026). A common choice is the combination of ID classification and metric learning:

$P_h \in \mathbb{R}^{m_h \times D}$5

$P_h \in \mathbb{R}^{m_h \times D}$6

and the overall objective is

$P_h \in \mathbb{R}^{m_h \times D}$7

Here $P_h \in \mathbb{R}^{m_h \times D}$8 is a distance such as Euclidean distance, $P_h \in \mathbb{R}^{m_h \times D}$9 is the margin, and $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$0 denote anchor, positive, and negative features. Regularization such as prompt weight decay can be used, but it is not essential in the formulation.

The Prompt-based Fine-Tuning Strategy (PFTS) makes the federated procedure parameter-efficient. A pre-trained, prompt-free ViT-based ReID model is loaded as the backbone $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$1 and frozen on all clients. Randomly initialized visual prompts $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$2 are implanted, and only prompt parameters are updated locally. Federated aggregation then applies weighted FedAvg to prompt parameters:

$P_p^{(k)} \in \mathbb{R}^{m_p \times D}$3

In the implementation, aggregation weights are proportional to dataset sizes $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$4.

The communication implications are explicit. Full-parameter training with ViT-B/16 involves approximately 86M parameters, whereas prompt-only tuning communicates about 0.46M parameters. The communication ratio is

$P_p^{(k)} \in \mathbb{R}^{m_p \times D}$5

which corresponds to more than 99% reduction. In FP32, this is approximately 344 MB per round for full-parameter training versus approximately 1.84 MB per round for prompt-only training. The paper reports that PFTS achieves notable gains within only a few aggregation rounds, which makes the approach practical in resource-constrained federated settings.

5. Federated protocol, datasets, and evaluation

The federated protocol treats each source dataset as one client and maintains the usual server-client cycle of local training, upload, weighted aggregation, and redistribution (Xu et al., 13 Mar 2026). Privacy is preserved in the sense that data never leaves the clients; only model parameters are communicated. Under PFTS, only $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$6 is transmitted, whereas full-parameter training communicates both backbone and prompt parameters.

Experiments use four person ReID datasets: CUHK02 (C2), CUHK03 (C3), Market-1501 (M), and MSMT17 (MS). Two protocols are defined. In protocol-1, a leave-one-out setting is used: three datasets form the source federation and the fourth unseen dataset is the target domain. Each run therefore uses $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$7 clients for training. Results are measured by mean Average Precision (mAP) and Rank-1 and averaged across transfer directions. In protocol-2, the federation is trained on C2, C3, and M, and the converged model is evaluated on each source-domain test set.

The optimizer, local epochs, and learning rates follow the SSCU training framework, and the backbone is ViT-B/16. The paper emphasizes that both BAPM and PFTS can be plugged into existing ViT-based FedDG-ReID frameworks with consistent gains, rather than requiring a separate training ecosystem.

The implementation procedure is straightforward. For each ViT layer, prompt sets $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$8 are created; prompts are concatenated with the CLS and image tokens; the attention mask $P_p^{(k)} \in \mathbb{R}^{m_p \times D}$9 is constructed so that BPAP interacts only with corresponding patch subsets while prompt-to-prompt attention remains unrestricted; masked self-attention and feed-forward blocks update the sequence; and federated rounds exchange either prompt parameters only or the full trainable model, depending on the training mode.

6. Experimental findings, interpretability, and limitations

FedBPrompt is evaluated as both a standalone design and a modular addition to existing methods (Xu et al., 13 Mar 2026). The reported improvements are universal across the tested baselines, which span federated optimizers such as FedProx, style-based domain generalization methods such as MixStyle and CrossStyle, federated ReID baselines such as FedPav and FedReID, and FedDG-ReID methods such as DACS and SSCU.

Under protocol-1, the most challenging target is $k = 1, \dots, K$0. In this case, SSCU+BAPM improves mAP by 3.4% and Rank-1 by 5.8%, while on the weaker FedProx baseline BAPM yields +13.9% mAP and +13.3% Rank-1 on MSMT17. Averaged across leave-one-out scenarios, full-parameter BAPM surpasses SSCU by 3.3% mAP and 4.9% Rank-1. Under protocol-2, BAPM also improves in-domain source performance rather than trading it off for cross-domain generalization. With FedProx, performance on M rises from 68.7/85.1 to 72.9/88.4 in mAP/Rank-1, on C2 from 79.9/79.3 to 83.1/83.3, and on C3 from 44.6/45.6 to 49.5/50.6.

Ablation studies show that holistic-only HFBP improves over the baseline, local-only BPAP also improves over the baseline, and the combined HFBP+BPAP configuration performs better than either component alone. This establishes the complementarity between suppressing background bias and enforcing part-level alignment.

Interpretability analyses are a prominent part of the evaluation. Under cropping, misalignment, and occlusion, BPAP attention maps localize the designated upper, middle, and lower body regions, while HFBP maintains global focus on the person and suppresses background. Quantitatively, Insertion AUC increases substantially. For SSCU, class-token maps rise from 0.6160 in the baseline to 0.7103 with visual prompts and to 0.7559 with full BAPM; for RISE, the corresponding values are 0.6516, 0.7494, and 0.7737. The paper also reports tighter intra-domain clusters and better inter-domain separation in t-SNE visualizations, particularly on MSMT17.

The main limitations are explicitly stated. The upper/middle/lower spatial partition is coarse and may be suboptimal under extreme poses or camera tilt. Extreme occlusions and severe domain shifts remain difficult. Future directions proposed in the paper include incorporating human parsing or pose if privacy constraints permit, designing occlusion-aware prompts or uncertainty-aware aggregation, and extending body-distribution-aware prompting beyond ViTs to CNN or hybrid backbones. FedBPrompt avoids auxiliary privacy complications by computing its masks locally and does not require external body annotations.

Code is available at the project repository, and the reported reproducibility configuration includes ViT-B/16, 50 prompts per layer, weighted FedAvg by dataset size, leave-one-out evaluation across C2, C3, M, and MS, source-domain evaluation on C2, C3, and M, and losses combining ID cross-entropy with triplet optimization. The paper’s overall result is a federated ReID framework in which body-distribution-aware prompting improves feature discrimination and cross-domain generalization while prompt-only federated fine-tuning reduces communicated parameters by more than 99% (Xu et al., 13 Mar 2026).