---
title: 'PM-SFL: Privacy-Preserving Split Federated Learning'
url: https://www.emergentmind.com/topics/pm-sfl
type: topic
---

# PM-SFL: Privacy-Preserving Split Federated Learning

Searching arXiv for PM-SFL and closely related split federated learning work to ground the article in current literature.
PM-SFL is a split federated learning (SFL) framework designed to be simultaneously privacy-preserving and heterogeneity-aware by combining probabilistic mask training, personalized mask learning, and layer-wise knowledge compensation [2509.14603]. In this formulation, SFL is treated as an alternative to conventional federated learning for settings where client-side computation must be reduced through model partitioning, but where the transmission of intermediate activations and updates introduces privacy exposure, especially to data reconstruction attacks [2509.14603]. PM-SFL addresses that exposure by introducing structured randomness through probabilistic masks rather than relying only on explicit noise injection, while also targeting two distinct forms of heterogeneity: non-i.i.d. data across clients and system heterogeneity arising from varying computation and communication resources [2509.14603].

## 1. Position within split federated learning

Split federated learning partitions a model between clients and a server so that resource-constrained clients execute only a prefix of the network, sending intermediate activations—often called “smashed data”—to the server for the remaining computation [2509.14603]. This design is motivated by the same general objective found in earlier SFL formulations: reducing client-side training burden while preserving the collaborative training structure of federated learning [2307.11532].

The privacy problem PM-SFL addresses is intrinsic to this architecture. Although SFL reduces local computation, exchanging intermediate activations and model updates exposes clients to reconstruction attacks that attempt to recover original inputs from intermediate representations [2509.14603]. This concern is consistent with prior security analyses showing that SFL is not inherently secure merely because the full model is not shared. In particular, model extraction attacks can exploit gradient information returned by the server, and these attacks can succeed under practical settings even when traditional prediction-query-based extraction is blocked [2303.08581]. PM-SFL is therefore best understood as a response to a mature line of SFL research in which efficiency gains are accompanied by a nontrivial attack surface.

A further point of context is heterogeneity. Earlier SFL work already emphasized the importance of adaptive splitting and resource allocation for heterogeneous clients. One example is model-splitting allowed federated edge learning, where cut-layer selection and parameter-server resource allocation are optimized to minimize training latency without loss of test accuracy [2307.11532]. PM-SFL inherits that concern but focuses on heterogeneity not only in resources but also in data distributions, making privacy, personalization, and adaptive participation coequal design goals [2509.14603].

## 2. Probabilistic mask training and structured randomness

The core mechanism in PM-SFL is probabilistic mask training, which applies probabilistic binary masks over local submodels to introduce structured randomness during training [2509.14603]. For each layer \(i\), the masked output is

\[
y_i = (W_i \odot M_i) x_i
\]

where \(W_i\) is the weight matrix, \(M_i \in \{0,1\}^d\) is a mask, and \(\odot\) denotes element-wise multiplication [2509.14603]. Each mask entry is sampled as

\[
M_{i,j} \sim \text{Bernoulli}(\sigma(s_{i,j}))
\]

with \(s_{i,j}\) a learnable score and \(\sigma(\cdot)\) the sigmoid activation [2509.14603].

The mask parameters are trained by backpropagation using a straight-through estimator, identified in the source as the straight-through estimator of Bengio et al. (2013) [2509.14603]. The update of the mask scores for client \(k\) is given by

\[
\mathbf{s}_{t, r+1}^{(k)} = \mathbf{s}_{t, r}^{(k)} - \frac{1}{|\mathcal{Q}_k|} \sum_{q \in \mathcal{Q}_k} (\nabla q)^T \cdot \left( \frac{\partial q}{\partial s} \right)
\]

with chain rule expansion

\[
\frac{\partial q}{\partial s} = \frac{\partial f_{\mathbf{w}_b \odot \mathbf{M}_k}(x)}{\partial \mathbf{w}_b \odot \mathbf{M}_k} \odot \mathbf{w}_b \odot \sigma(s) \odot \sigma'(s)
\]

[2509.14603].

The distinguishing claim is that this masking introduces structured randomness rather than unstructured additive perturbation [2509.14603]. Each client uses a distinctive random mask per round, and this obfuscates the mapping from activations to raw data in a trainable way [2509.14603]. The paper contrasts this with existing defenses using noise injection, which are characterized as often degrading model performance [2509.14603]. In that sense, PM-SFL belongs to a class of privacy mechanisms that alter the computational pathway itself, rather than merely perturbing outputs after the fact.

This also differentiates PM-SFL from classical SFL security countermeasures implied by earlier attack work. Prior analyses suggested limiting gradient precision, using activation obfuscation, or applying differential privacy to intermediate activations as possible mitigations [2303.08581]. PM-SFL operationalizes a specific obfuscation strategy via trainable probabilistic masking and supplements it with formal privacy analysis [2509.14603].

## 3. Personalization under data heterogeneity

PM-SFL treats data heterogeneity as a first-class design constraint. Instead of enforcing a globally uniform mask structure, each client learns its own mask scores \(\mathbf{s}^{(k)}\) according to its local data distribution during local training [2509.14603]. The resulting mask structures may therefore differ across clients, allowing the submodels to optimize their effective structure for non-i.i.d. local data [2509.14603].

This mechanism is described as personalized mask learning [2509.14603]. The personalization occurs at the mask or “subnetwork” level rather than through fully independent local model families [2509.14603]. The paper explicitly associates this with improved utility and privacy under heterogeneous client data, and states that it avoids the deterioration in model performance that can arise under homogeneous pruning or masking schemes [2509.14603].

The conceptual significance of this design becomes clearer when viewed against related heterogeneous SFL frameworks. In spiking federated learning, for example, SFedHIFI allows clients to deploy models of different widths through channel-wise matrix decomposition and then performs cross-scale aggregation using a fire-rate-based heterogeneous information fusion module [2603.14956]. PM-SFL addresses a different substrate—probabilistic masking in conventional SFL rather than width-adaptive spiking models—but both works share the premise that heterogeneity should be encoded into the model structure rather than treated as an exception [2509.14603; 2603.14956].

A plausible implication is that PM-SFL’s personalization mechanism serves a dual role: it is a privacy mechanism because it diversifies masked representations across clients and rounds, and it is a utility mechanism because it adapts effective subnetworks to local distributions. The source directly supports both aspects separately, even if it does not reduce them to a single formal principle [2509.14603].

## 4. System heterogeneity and layer-wise knowledge compensation

PM-SFL also addresses system heterogeneity, meaning variation in client computation and communication resources that necessitates adaptive model splitting [2509.14603]. In such settings, different clients may split the network at different layers, so the server receives information from client submodels of varying depth [2509.14603]. To compensate for the resulting inconsistency, PM-SFL introduces a layer-wise knowledge compensation mechanism at the server [2509.14603].

The mechanism is described as reconciling or distilling knowledge from client submodels created with masks applied at different layers [2509.14603]. The summary identifies several realizations of this idea: federated distillation, distillation-based aggregation, and integration of knowledge from all split depths to ensure stable and fair updates [2509.14603]. The stated purpose is to enable clients with limited capacities to participate effectively under adaptive model splitting [2509.14603].

This concern parallels earlier SFL systems research. The training-latency-minimization framework for model-splitting allowed federated edge learning explicitly optimizes cut-layer selection and server resource allocation under synchronized global updates, reflecting the same dependency of system performance on where the model is partitioned [2307.11532]. Likewise, SFL-LEO handles personalized split models in intermittent satellite-ground environments using layer alignment so that satellites with different split points can still contribute to aggregation [2504.13479]. PM-SFL’s layer-wise knowledge compensation is not the same procedure as those earlier alignment or optimization methods, but it occupies the same design space: maintaining coherent training when participants operate at unequal split depths [2509.14603; 2504.13479; 2307.11532].

The framework therefore extends the meaning of heterogeneity-aware SFL beyond mere resource scheduling. In PM-SFL, system heterogeneity alters the representational interface between client and server, and the compensation mechanism is intended to preserve both participation and model quality under that variation [2509.14603].

## 5. Privacy analysis and attack model

PM-SFL includes theoretical privacy analysis that addresses both reconstruction risk and differential privacy properties [2509.14603]. The first component is a data reconstruction bound for the case where an adversarial server attempts to reconstruct an input \(x\) from smashed representations and an imperfectly recovered mask [2509.14603]. For an MLP with final split output

\[
y_m = (W_m \odot M_m) x_m,
\]

the reconstruction error is bounded below by

\[
\|x - \hat{x}\| \geq \frac{\sigma_{\min}(W \odot (\hat{M}-M))}{\sigma_{\max}(W \odot \hat{M})} \|x\|
\]

where \(\sigma_{\min}\) and \(\sigma_{\max}\) are the minimum and maximum singular values [2509.14603]. The paper interprets privacy as depending on the discrepancy between the true and guessed masks together with the structure of the masked weights [2509.14603].

The second component concerns forward-pass privacy with differential privacy amplification. The masked output is defined as

\[
f_M(x) = (\mathbf{w}_b \odot M) \cdot x + z
\]

with \(z \sim Lap(\Delta f/\epsilon)\) and sensitivity

\[
\Delta f = \max_{x,x'} \| \mathbf{w}_b (x - x') \|
\]

[2509.14603]. If each mask coordinate is independently masked with probability at least \(c\), the effective privacy parameter becomes

\[
\epsilon_{amp} = \ln \left( (1 - c^d)e^{\epsilon} + c^d \right)
\]

[2509.14603]. The paper states that probabilistic masking amplifies privacy by subsampling, thereby reducing effective privacy leakage relative to standard differentially private noise injection [2509.14603].

The third component concerns backward-pass privacy. Two options are described. One adds Gaussian noise to local updates, requiring

\[
\sigma^2 \ge \frac{2 R \Gamma^2 \ln(1/\delta)}{\epsilon^2 |\mathcal{Q}_k|^2}
\]

to ensure \((\epsilon,\delta)\)-DP over \(R\) rounds [2509.14603]. The other adds noise before mask sampling and then samples Bernoulli masks, with guarantee

\[
\sigma^2 \ge \frac{2 (1-2c)^2 \ln(1.25/\delta)}{\epsilon^2}
\]

for the randomness in the mask distribution [2509.14603].

These guarantees should be interpreted against known SFL threats. Earlier work on model extraction attacks demonstrated that malicious clients can exploit gradient information from the server to reconstruct high-fidelity surrogate server-side models under practical settings [2303.08581]. PM-SFL’s privacy analysis is focused primarily on data reconstruction and differential privacy guarantees, not on the entire spectrum of intellectual-property attacks. This suggests that PM-SFL should not be read as eliminating every SFL threat model; rather, it formalizes protection against a major class of privacy risks associated with smashed representations and update channels [2509.14603; 2303.08581].

## 6. Empirical behavior, comparative performance, and relation to adjacent frameworks

The experimental evaluation of PM-SFL covers image classification tasks such as CIFAR-10 and EMNIST, as well as wireless sensing with mmWave radar data for activity recognition [2509.14603]. The reported metrics include privacy under data reconstruction attacks, test accuracy under varying heterogeneity conditions, communication efficiency, and robustness to privacy attacks [2509.14603]. Across these tasks, PM-SFL is reported to improve accuracy, communication efficiency, and robustness relative to vanilla SFL and noise-based defenses, especially under data and system heterogeneity [2509.14603].

The quantitative results explicitly stated in the source can be summarized as follows.

| Setting | Reported outcome | Source |
|---|---:|---|
| CIFAR-10 reconstruction | attacker MSE increased by 2–4× over SFL | [2509.14603] |
| EMNIST non-i.i.d. accuracy | vanilla SFL 88.4%, SFL + DP 77.6%, PM-SFL 87.9% | [2509.14603] |
| Communication efficiency | communication reduced by 20–35% in typical configurations | [2509.14603] |

The paper further gives a task-level comparison table in which CIFAR-10 accuracy is reported as approximately \(87.0\%\) for baseline SFL, \(80.1\%\) for SFL+DP, and \(87.2\%\) for PM-SFL, alongside a communication reduction of \(-30\%\) and a reconstruction MSE increase of \(+3.2\times\) [2509.14603]. For EMNIST, the corresponding accuracy values are \(88.4\%\), \(77.6\%\), and \(87.9\%\), with \(-22\%\) communication and \(+2.7\times\) reconstruction MSE [2509.14603]. For mmWave radar, the source states “see paper” for the baseline numbers but indicates similar gains [2509.14603]. Because the provided table marks some values as approximate or deferred to the paper, only the explicitly supplied numbers can be stated as concrete values here.

The evaluation positions PM-SFL against explicit-noise defenses and reports that sparse masks reduce both upstream and downstream communication per round due to sparse activations and masked gradients [2509.14603]. This communication effect is notable because earlier SFL literature often treated privacy and systems efficiency as competing objectives. By contrast, PM-SFL presents probabilistic masking as simultaneously improving privacy and reducing communication overhead [2509.14603].

In the broader ecosystem of SFL variants, PM-SFL sits alongside several specialized frameworks. SFL-LEO combines split learning and federated learning for highly dynamic LEO satellite-ground networks, using asynchronous training and layer alignment under intermittent connectivity [2504.13479]. SFedHIFI addresses heterogeneous spiking federated learning by enabling adaptive model widths and cross-scale fusion through fire-rate-based aggregation [2603.14956]. The training-latency-minimization framework for SFL focuses on cut-layer selection and parameter-server resource allocation to minimize synchronized round latency [2307.11532]. PM-SFL differs from all three by centering structured probabilistic masking as the organizing principle, but it shares with them the general trajectory of SFL research toward practical deployment under heterogeneous resource conditions [2509.14603].

A potential misconception is that PM-SFL is simply an SFL system with differential privacy added on top. The paper’s characterization is narrower and more specific: its primary privacy mechanism is probabilistic mask training that injects structured randomness without relying on explicit noise, while differential privacy appears in the theoretical analysis and in optional backward-pass mechanisms [2509.14603]. Another possible misconception is that privacy in SFL follows automatically from not sharing raw data. Prior attack literature contradicts that view, and PM-SFL’s very motivation presupposes that intermediate activations are themselves privacy-sensitive [2303.08581; 2509.14603].

Source: https://www.emergentmind.com/topics/pm-sfl