---
title: 'FedKLPR: Federated KL Pruning for Re-ID'
url: https://www.emergentmind.com/topics/fedklpr
type: topic
---

# FedKLPR: Federated KL Pruning for Re-ID

FedKLPR is a personalized federated learning framework for unsupervised person re-identification that addresses two coupled constraints in practical federated Re-ID deployments: statistical heterogeneity across clients and substantial communication overhead from repeated transmission of large backbones such as ResNet-50 and ResNet-34. Its design combines KL-based distributional regularization, pruning-aware aggregation, parameter-wise sparse fusion, and dynamic pruning control. In experiments on eight benchmark datasets, it reduces communication cost by **33\%-38\%** on **ResNet-50** and **20\%-40\%** on **ResNet-34**, while maintaining model accuracy within **1\% degradation** [2508.17431].

## 1. Problem setting and design objective

FedKLPR is formulated for **unsupervised person Re-ID** in a federated environment where each client corresponds to a different dataset and therefore a different domain. The target application is decentralized surveillance or public-safety infrastructure in which data remain local, but a shared model is still desirable. In this setting, each client exhibits a distinct combination of camera layout, background, illumination, identity population, and dataset scale. The resulting non-IID regime is described through **label skew**, **feature skew**, and **quantity skew**.

Two difficulties motivate the method. The first is **statistical heterogeneity**: local optimization on one client can induce divergent updates, unstable global convergence, model drift, and poor performance on smaller or more difficult datasets. The second is **communication overhead**: standard federated training transmits dense deep models in every round, which is costly for bandwidth-constrained edge deployments.

The personalization objective in FedKLPR is not post-hoc fine-tuning of a single shared model. Rather, personalization is built into training through two mechanisms: local models are regularized toward a global feature distribution using KL divergence, and each client maintains its own pruning behavior and sparsity pattern. The framework therefore seeks a balance between **distributional alignment** and **client-specific structural adaptation**.

## 2. Local learning: camera-aware training and KL-divergence regularization

FedKLPR builds on **FedCAPR**, using its **camera-aware proxy loss** as the primary local task loss. For client \(k\), DBSCAN clustering and camera grouping produce a pseudo-labeled set
\[
D_k' = \{(x_i, \hat{y}_i, \hat{z}_i)\}_{i=1}^{N'},
\]
where \(\hat{y}_i\) is a pseudo identity label and \(\hat{z}_i\) is a camera-aware proxy label. A memory bank \(M \in \mathbb{R}^{d \times Z}\) stores feature prototypes for each proxy and is updated by
\[
M[\hat{z}_i] \leftarrow M[\hat{z}_i] + (1 - \mu) f_k(x_i).
\]

The baseline local loss is
\[
\mathscr{L}_{\text{capr}} = \alpha \mathscr{L}_{\text{intra}} + \beta \mathscr{L}_{\text{inter}} + \gamma \mathscr{L}_{\text{ca}},
\]
where \(\mathscr{L}_{\text{intra}}\) emphasizes intra-camera discrimination, \(\mathscr{L}_{\text{inter}}\) aligns identities across cameras, and \(\mathscr{L}_{\text{ca}}\) mitigates cross-camera variance via camera-aware proxy selection. These terms are implemented as contrastive softmax losses over similarities \(S(m, x_i) = \exp(M[m]^\top f_k(x_i)/\tau)\).

FedKLPR adds **KL-Divergence Regularization Loss (KLL)** to this baseline. Let \(M_k^{t-1}\) denote the personalized model received at round \(t\), \(M_k^t\) the updated local model, and \(D_{\text{batch}}\) a validation mini-batch. The KL term is
\[
\mathscr{L}_{\text{KL}} =
D_{\mathrm{KL}}\!\left(
P_{M_k^t}(D_{\text{batch}})
\;\Vert\;
P_{M_k^{t-1}}(D_{\text{batch}})
\right).
\]
The local objective becomes
\[
\mathscr{L}_{\text{all}} = \mathscr{L}_{\text{capr}} + \delta\, \mathscr{L}_{\text{KL}},
\]
with \(\delta \approx 0.13\).

This regularizer constrains the current local feature distribution relative to the previous personalized or global model. In the paper’s formulation, this is the mechanism by which local training incorporates new client-specific information without drifting arbitrarily far from the shared representation. The intended effect is mitigation of heterogeneity-induced instability in a distribution space rather than in raw parameter space [2508.17431].

## 3. Pruning-aware federated aggregation

FedKLPR introduces four named components, each tied to a distinct failure mode of standard federated optimization.

| Component | Function | Core mechanism |
|---|---|---|
| KLL | Distributional alignment | KL regularization between current and previous personalized models |
| KLPWA | Server aggregation | Combines KL-based weights and pruning-ratio weights |
| SAS | Sparse model fusion | Excludes zero-valued weights from averaging |
| CRR | Dynamic compression control | Halts or rolls back pruning when accuracy conditions fail |

The server-side aggregation starts from
\[
w^{t+1} = M_g^t = \sum_{k=1}^{K} \alpha_k^t M_k^t,
\]
but the weights \(\alpha_k^t\) are not based on dataset size alone.

The first weighting term is the **KL-Divergence-aggregation Weight (KLAW)**. After local training, client \(k\) computes
\[
f_k^t =
D_{\mathrm{KL}}\!\left(
P_{M_k^t}(D_{\text{batch}})
\;\Vert\;
P_{M_k^{t-1}}(D_{\text{batch}})
\right),
\qquad
f^t = \sum_{k=1}^{K} f_k^t,
\]
and then
\[
\text{KLAW}_k = \frac{f_k^t}{f^t}.
\]
The interpretation given in the method description is that larger KL change indicates that the client has acquired more new information and should therefore have greater influence on the global update.

The second weighting term is the **Pruning-ratio-aggregation Weight (PRAW)**. If \(P_k \in [0,1]\) is the pruning ratio of client \(k\), then
\[
P^t = \sum_{k=1}^{K} P_k^2,
\qquad
\text{PRAW}_k = W_k = \frac{P_k^2}{P^t}.
\]
Squaring \(P_k\) emphasizes heavily pruned clients, which the method treats as more structurally aligned with the target sparse global model.

These are combined in **KL-Divergence-Prune Weighted Aggregation (KLPWA)**:
\[
M_g^t
=
\gamma \sum_{k=1}^{K} \frac{f_k^t}{f^t} M_k^t
+
\delta \sum_{k=1}^{K} W_k M_k^t,
\qquad
\gamma + \delta = 1.
\]
Thus,
\[
\alpha_k^t
=
\gamma \frac{f_k^t}{f^t}
+
\delta W_k.
\]

A second aggregation issue arises because different clients prune different coordinates. **Sparse Activation Skipping (SAS)** addresses this by excluding zero-valued parameters from averaging. For parameter index \(j\), let \(K_j = \{k \mid w_{k,j} \neq 0\}\). The aggregated parameter is
\[
w_{g,j}^t
=
\frac{
\sum_{k \in K_j} \alpha_k^t\, w_{k,j}
}{
\sum_{k \in K_j} \alpha_k^t
}.
\]
Zeros therefore do not dilute informative nonzero values. In the terminology of the method description, KLPWA determines which clients matter more, while SAS determines where a client is allowed to contribute [2508.17431].

## 4. Adaptive pruning and round structure

FedKLPR uses **unstructured weight pruning**. Sparsity is imposed at the scalar-parameter level rather than through channel or filter removal, and each client maintains its own pruning mask. The stated target is approximately **70\% sparsity**, with clients attempting to remove **up to 9\% of remaining parameters per round**.

Pruning is controlled by **Cross-Round Recovery (CRR)**, a two-stage verification mechanism based on local validation accuracy. Let \(Acc_{\text{pre}}^t\) denote the accuracy after local training but before pruning in round \(t\), and \(Acc_{\text{post}}^t\) the accuracy after pruning. Let \(Acc_{\text{th}}\) be a minimum accuracy threshold, \(\delta_{rd}\) a cross-round stability margin, and \(\delta_{ep}\) a maximum within-round accuracy drop.

In the first stage, pruning is allowed only if
\[
Acc_{\text{pre}}^t \ge Acc_{\text{th}}
\]
and
\[
|Acc_{\text{pre}}^t - Acc_{\text{best}}^{t-1}| \le \delta_{rd}.
\]
The first condition requires that the model be strong enough to prune; the second requires that its performance be sufficiently stable across rounds.

In the second stage, pruning is retained only if
\[
Acc_{\text{pre}}^t - Acc_{\text{post}}^t \le \delta_{ep}.
\]
If the drop is too large, pruning is rolled back or postponed.

The federated training protocol uses **20 global rounds** and **5 local epochs per round**, with **Adam** at learning rate \(3.5 \times 10^{-4}\). In each round, the server broadcasts the current global model, clients run local unsupervised Re-ID training with KLL, apply CRR-governed pruning, compute \(f_k^t\) and \(P_k\), and return sparse local models together with pruning masks and aggregation statistics. The server then applies KLPWA and SAS to form the next global model. The experiments use **ResNet-50** and **ResNet-34**, both pre-trained on ImageNet [2508.17431].

## 5. Empirical evaluation

The evaluation treats **eight datasets as eight federated clients**. The large-scale clients are **DukeMTMC-reID** with **702 train IDs** and **8 cameras**, and **Market-1501** with **751 train IDs** and **6 cameras**. The smaller two-camera clients are **iLIDS-VID**, **CUHK03**, **PRID2011**, **VIPeR**, **CUHK01**, and **3DPeS**. Performance is reported with **Rank-1** and **mAP**.

Against earlier federated Re-ID baselines such as **FedUReID**, **FedUCC**, **FedUCC+**, **FedUCA**, and **FedCAPR**, the non-pruning variant using **KLL + KLAW** already yields strong accuracy. Relative to older federated Re-ID baselines, the reported gains include **more than 38\% Rank-1 improvement on CUHK03**, as well as **+5.5\%** on **PRID2011** and **+9\%** on **CUHK01** over non-FedCAPR baselines. Relative to **FedCAPR**, the method is described as broadly comparable overall, with a notable improvement on **VIPeR**, where **Rank-1** changes from **63.0\%** to **69.6\%** and **mAP** from **70.3\%** to **77.3\%**.

The pruning results are the distinguishing empirical claim. For **ResNet-50**, FedKLPR reduces total communication cost over 20 rounds to roughly **2.16–2.46 GB**, compared with the **FedCAPR** baseline at approximately **3.42 GB**, corresponding to the stated **33–38\%** reduction. For **ResNet-34**, communication is reduced from approximately **3.09 GB** to **2.08–2.59 GB**, corresponding to **20–40\%** reduction. The pruning ratios typically lie around **61–68\% sparsity** for ResNet-50 and around **60–70\% sparsity** for ResNet-34.

Accuracy retention is reported as strong on large clients and often favorable on smaller ones. Relative to non-pruned **FedCAPR**, the pruned **ResNet-50** variant maintains **Rank-1** and **mAP** within **\(\pm 1\%\)** on **Duke**, **Market**, and **iLIDS**, while improving smaller datasets such as **CUHK03** from **68.43** to **70.57** in Rank-1, **PRID2011** from **82.00** to **84.00**, and **VIPeR** from **62.97** to **67.41**. For **ResNet-34**, the reported improvements over pruning baselines such as **SubFedAvg** and **FedDIP** include **iLIDS: 84.69\%**, **CUHK03: 62.93\%**, and **VIPeR: 60.76\%**.

The ablation study isolates the contribution of each component. **KLL + KLAW** improves accuracy even without pruning. Adding plain pruning alone causes substantial drops on small datasets. Adding **PRAW + CRR** or **SAS + CRR** mitigates those drops. The full combination **KLL + KLAW + P + PRAW + SAS + CRR** achieves about **70\% pruning ratio** on most clients and improves Rank-1 over unstructured pruning alone by **+6.1\% on iLIDS**, **+6.7\% on VIPeR**, and **+6.1\% on 3DPeS** [2508.17431].

## 6. Scope, nomenclature, and limitations

FedKLPR is specifically a method for **unsupervised person Re-ID** with **KL-based personalization** and **adaptive pruning**. It should not be conflated with several similarly named methods in other subfields. **FedKL** addresses federated reinforcement learning by penalizing KL divergence between local and global policies [2204.08125]. A hybrid federated **Kernel Regularized Least Squares** scheme is implemented through **FedCG** rather than a method formally named FedKLPR [2407.17228]. **KF-FedKL** denotes **Kalman Filter aided Federated Koopman Learning** for nonlinear system linearization under partial observations [2507.04808]. A separate source on medical federated learning explicitly states that the acronym **“FedKLPR” does not appear in the text** of the corresponding paper and is **almost certainly a typo or variant referring to FedKPer** [2605.00698].

Within the Re-ID literature, FedKLPR is presented as the first method to **explicitly integrate model pruning, KL regularization, and pruning-aware aggregation tailored to unsupervised Re-ID**. Its direct predecessor is **FedCAPR**, from which it inherits the camera-aware contrastive training pipeline. The method’s personalization is representation-oriented rather than meta-learning-based: each client keeps a sparse local model and memory bank, while KLL keeps feature distributions aligned.

The principal limitations stated for the method are also specific. The experiments assume that **all 8 clients participate in each round**; **partial participation** and **asynchronous FL** are not analyzed. The pruning is **unstructured**, so hardware gains may require structured sparsity or dedicated sparse kernels. The evaluation is restricted to **ResNet-50/34** and to the unsupervised Re-ID pipeline based on clustering and camera-aware proxies. Labeled settings and other tasks are not evaluated. The paper therefore supports a narrow but technically well-defined reading of FedKLPR: a communication-efficient personalized federated framework for unsupervised person re-identification, not a generic KL-regularized federated method across domains [2508.17431].

Source: https://www.emergentmind.com/topics/fedklpr