PM-SFL: Privacy-Preserving Split Federated Learning

• PM-SFL is a split federated learning framework that uses structured probabilistic mask training to protect intermediate activations and mitigate data reconstruction attacks.
• It employs personalized mask learning and layer-wise knowledge compensation to adapt to both non-i.i.d. data and system resource variability.
• Experimental evaluations on datasets like CIFAR-10 and EMNIST demonstrate improved accuracy, reduced communication overhead, and robust performance under heterogeneous conditions.

Searching arXiv for PM-SFL and closely related split federated learning work to ground the article in current literature. PM-SFL is a split federated learning (SFL) framework designed to be simultaneously privacy-preserving and heterogeneity-aware by combining probabilistic mask training, personalized mask learning, and layer-wise knowledge compensation (Wang et al., 18 Sep 2025). In this formulation, SFL is treated as an alternative to conventional federated learning for settings where client-side computation must be reduced through model partitioning, but where the transmission of intermediate activations and updates introduces privacy exposure, especially to data reconstruction attacks (Wang et al., 18 Sep 2025). PM-SFL addresses that exposure by introducing structured randomness through probabilistic masks rather than relying only on explicit noise injection, while also targeting two distinct forms of heterogeneity: non-i.i.d. data across clients and system heterogeneity arising from varying computation and communication resources (Wang et al., 18 Sep 2025).

1. Position within split federated learning

Split federated learning partitions a model between clients and a server so that resource-constrained clients execute only a prefix of the network, sending intermediate activations—often called “smashed data”—to the server for the remaining computation (Wang et al., 18 Sep 2025). This design is motivated by the same general objective found in earlier SFL formulations: reducing client-side training burden while preserving the collaborative training structure of federated learning (Wen et al., 2023).

The privacy problem PM-SFL addresses is intrinsic to this architecture. Although SFL reduces local computation, exchanging intermediate activations and model updates exposes clients to reconstruction attacks that attempt to recover original inputs from intermediate representations (Wang et al., 18 Sep 2025). This concern is consistent with prior security analyses showing that SFL is not inherently secure merely because the full model is not shared. In particular, model extraction attacks can exploit gradient information returned by the server, and these attacks can succeed under practical settings even when traditional prediction-query-based extraction is blocked (Li et al., 2023). PM-SFL is therefore best understood as a response to a mature line of SFL research in which efficiency gains are accompanied by a nontrivial attack surface.

A further point of context is heterogeneity. Earlier SFL work already emphasized the importance of adaptive splitting and resource allocation for heterogeneous clients. One example is model-splitting allowed federated edge learning, where cut-layer selection and parameter-server resource allocation are optimized to minimize training latency without loss of test accuracy (Wen et al., 2023). PM-SFL inherits that concern but focuses on heterogeneity not only in resources but also in data distributions, making privacy, personalization, and adaptive participation coequal design goals (Wang et al., 18 Sep 2025).

2. Probabilistic mask training and structured randomness

The core mechanism in PM-SFL is probabilistic mask training, which applies probabilistic binary masks over local submodels to introduce structured randomness during training (Wang et al., 18 Sep 2025). For each layer $i$, the masked output is

$y_i = (W_i \odot M_i) x_i$

where $W_i$ is the weight matrix, $M_i \in \{0,1\}^d$ is a mask, and $\odot$ denotes element-wise multiplication (Wang et al., 18 Sep 2025). Each mask entry is sampled as

$M_{i,j} \sim \text{Bernoulli}(\sigma(s_{i,j}))$

with $s_{i,j}$ a learnable score and $\sigma(\cdot)$ the sigmoid activation (Wang et al., 18 Sep 2025).

The mask parameters are trained by backpropagation using a straight-through estimator, identified in the source as the straight-through estimator of Bengio et al. (2013) (Wang et al., 18 Sep 2025). The update of the mask scores for client $k$ is given by

$$\mathbf{s}_{t, r+1}^{(k)} = \mathbf{s}_{t, r}^{(k)} - \frac{1}{|\mathcal{Q}_k|} \sum_{q \in \mathcal{Q}_k} (\nabla q)^T \cdot \left( \frac{\partial q}{\partial s} \right)$$

with chain rule expansion

$y_i = (W_i \odot M_i) x_i$0

(Wang et al., 18 Sep 2025).

The distinguishing claim is that this masking introduces structured randomness rather than unstructured additive perturbation (Wang et al., 18 Sep 2025). Each client uses a distinctive random mask per round, and this obfuscates the mapping from activations to raw data in a trainable way (Wang et al., 18 Sep 2025). The paper contrasts this with existing defenses using noise injection, which are characterized as often degrading model performance (Wang et al., 18 Sep 2025). In that sense, PM-SFL belongs to a class of privacy mechanisms that alter the computational pathway itself, rather than merely perturbing outputs after the fact.

This also differentiates PM-SFL from classical SFL security countermeasures implied by earlier attack work. Prior analyses suggested limiting gradient precision, using activation obfuscation, or applying differential privacy to intermediate activations as possible mitigations (Li et al., 2023). PM-SFL operationalizes a specific obfuscation strategy via trainable probabilistic masking and supplements it with formal privacy analysis (Wang et al., 18 Sep 2025).

3. Personalization under data heterogeneity

PM-SFL treats data heterogeneity as a first-class design constraint. Instead of enforcing a globally uniform mask structure, each client learns its own mask scores $y_i = (W_i \odot M_i) x_i$1 according to its local data distribution during local training (Wang et al., 18 Sep 2025). The resulting mask structures may therefore differ across clients, allowing the submodels to optimize their effective structure for non-i.i.d. local data (Wang et al., 18 Sep 2025).

This mechanism is described as personalized mask learning (Wang et al., 18 Sep 2025). The personalization occurs at the mask or “subnetwork” level rather than through fully independent local model families (Wang et al., 18 Sep 2025). The paper explicitly associates this with improved utility and privacy under heterogeneous client data, and states that it avoids the deterioration in model performance that can arise under homogeneous pruning or masking schemes (Wang et al., 18 Sep 2025).

The conceptual significance of this design becomes clearer when viewed against related heterogeneous SFL frameworks. In spiking federated learning, for example, SFedHIFI allows clients to deploy models of different widths through channel-wise matrix decomposition and then performs cross-scale aggregation using a fire-rate-based heterogeneous information fusion module (Tao et al., 16 Mar 2026). PM-SFL addresses a different substrate—probabilistic masking in conventional SFL rather than width-adaptive spiking models—but both works share the premise that heterogeneity should be encoded into the model structure rather than treated as an exception (Wang et al., 18 Sep 2025, Tao et al., 16 Mar 2026).

A plausible implication is that PM-SFL’s personalization mechanism serves a dual role: it is a privacy mechanism because it diversifies masked representations across clients and rounds, and it is a utility mechanism because it adapts effective subnetworks to local distributions. The source directly supports both aspects separately, even if it does not reduce them to a single formal principle (Wang et al., 18 Sep 2025).

4. System heterogeneity and layer-wise knowledge compensation

PM-SFL also addresses system heterogeneity, meaning variation in client computation and communication resources that necessitates adaptive model splitting (Wang et al., 18 Sep 2025). In such settings, different clients may split the network at different layers, so the server receives information from client submodels of varying depth (Wang et al., 18 Sep 2025). To compensate for the resulting inconsistency, PM-SFL introduces a layer-wise knowledge compensation mechanism at the server (Wang et al., 18 Sep 2025).

The mechanism is described as reconciling or distilling knowledge from client submodels created with masks applied at different layers (Wang et al., 18 Sep 2025). The summary identifies several realizations of this idea: federated distillation, distillation-based aggregation, and integration of knowledge from all split depths to ensure stable and fair updates (Wang et al., 18 Sep 2025). The stated purpose is to enable clients with limited capacities to participate effectively under adaptive model splitting (Wang et al., 18 Sep 2025).

This concern parallels earlier SFL systems research. The training-latency-minimization framework for model-splitting allowed federated edge learning explicitly optimizes cut-layer selection and server resource allocation under synchronized global updates, reflecting the same dependency of system performance on where the model is partitioned (Wen et al., 2023). Likewise, SFL-LEO handles personalized split models in intermittent satellite-ground environments using layer alignment so that satellites with different split points can still contribute to aggregation (Wu et al., 18 Apr 2025). PM-SFL’s layer-wise knowledge compensation is not the same procedure as those earlier alignment or optimization methods, but it occupies the same design space: maintaining coherent training when participants operate at unequal split depths (Wang et al., 18 Sep 2025, Wu et al., 18 Apr 2025, Wen et al., 2023).

The framework therefore extends the meaning of heterogeneity-aware SFL beyond mere resource scheduling. In PM-SFL, system heterogeneity alters the representational interface between client and server, and the compensation mechanism is intended to preserve both participation and model quality under that variation (Wang et al., 18 Sep 2025).

5. Privacy analysis and attack model

PM-SFL includes theoretical privacy analysis that addresses both reconstruction risk and differential privacy properties (Wang et al., 18 Sep 2025). The first component is a data reconstruction bound for the case where an adversarial server attempts to reconstruct an input $y_i = (W_i \odot M_i) x_i$2 from smashed representations and an imperfectly recovered mask (Wang et al., 18 Sep 2025). For an MLP with final split output

$y_i = (W_i \odot M_i) x_i$3

the reconstruction error is bounded below by

$y_i = (W_i \odot M_i) x_i$4

where $y_i = (W_i \odot M_i) x_i$5 and $y_i = (W_i \odot M_i) x_i$6 are the minimum and maximum singular values (Wang et al., 18 Sep 2025). The paper interprets privacy as depending on the discrepancy between the true and guessed masks together with the structure of the masked weights (Wang et al., 18 Sep 2025).

The second component concerns forward-pass privacy with differential privacy amplification. The masked output is defined as

$y_i = (W_i \odot M_i) x_i$7

with $y_i = (W_i \odot M_i) x_i$8 and sensitivity

$y_i = (W_i \odot M_i) x_i$9

(Wang et al., 18 Sep 2025). If each mask coordinate is independently masked with probability at least $W_i$0, the effective privacy parameter becomes

$W_i$1

(Wang et al., 18 Sep 2025). The paper states that probabilistic masking amplifies privacy by subsampling, thereby reducing effective privacy leakage relative to standard differentially private noise injection (Wang et al., 18 Sep 2025).

The third component concerns backward-pass privacy. Two options are described. One adds Gaussian noise to local updates, requiring

$W_i$2

to ensure $W_i$3-DP over $W_i$4 rounds (Wang et al., 18 Sep 2025). The other adds noise before mask sampling and then samples Bernoulli masks, with guarantee

$W_i$5

for the randomness in the mask distribution (Wang et al., 18 Sep 2025).

These guarantees should be interpreted against known SFL threats. Earlier work on model extraction attacks demonstrated that malicious clients can exploit gradient information from the server to reconstruct high-fidelity surrogate server-side models under practical settings (Li et al., 2023). PM-SFL’s privacy analysis is focused primarily on data reconstruction and differential privacy guarantees, not on the entire spectrum of intellectual-property attacks. This suggests that PM-SFL should not be read as eliminating every SFL threat model; rather, it formalizes protection against a major class of privacy risks associated with smashed representations and update channels (Wang et al., 18 Sep 2025, Li et al., 2023).

6. Empirical behavior, comparative performance, and relation to adjacent frameworks

The experimental evaluation of PM-SFL covers image classification tasks such as CIFAR-10 and EMNIST, as well as wireless sensing with mmWave radar data for activity recognition (Wang et al., 18 Sep 2025). The reported metrics include privacy under data reconstruction attacks, test accuracy under varying heterogeneity conditions, communication efficiency, and robustness to privacy attacks (Wang et al., 18 Sep 2025). Across these tasks, PM-SFL is reported to improve accuracy, communication efficiency, and robustness relative to vanilla SFL and noise-based defenses, especially under data and system heterogeneity (Wang et al., 18 Sep 2025).

The quantitative results explicitly stated in the source can be summarized as follows.

Setting	Reported outcome	Source
CIFAR-10 reconstruction	attacker MSE increased by 2–4× over SFL	(Wang et al., 18 Sep 2025)
EMNIST non-i.i.d. accuracy	vanilla SFL 88.4%, SFL + DP 77.6%, PM-SFL 87.9%	(Wang et al., 18 Sep 2025)
Communication efficiency	communication reduced by 20–35% in typical configurations	(Wang et al., 18 Sep 2025)

The paper further gives a task-level comparison table in which CIFAR-10 accuracy is reported as approximately $W_i$6 for baseline SFL, $W_i$7 for SFL+DP, and $W_i$8 for PM-SFL, alongside a communication reduction of $W_i$9 and a reconstruction MSE increase of $M_i \in \{0,1\}^d$0 (Wang et al., 18 Sep 2025). For EMNIST, the corresponding accuracy values are $M_i \in \{0,1\}^d$1, $M_i \in \{0,1\}^d$2, and $M_i \in \{0,1\}^d$3, with $M_i \in \{0,1\}^d$4 communication and $M_i \in \{0,1\}^d$5 reconstruction MSE (Wang et al., 18 Sep 2025). For mmWave radar, the source states “see paper” for the baseline numbers but indicates similar gains (Wang et al., 18 Sep 2025). Because the provided table marks some values as approximate or deferred to the paper, only the explicitly supplied numbers can be stated as concrete values here.

The evaluation positions PM-SFL against explicit-noise defenses and reports that sparse masks reduce both upstream and downstream communication per round due to sparse activations and masked gradients (Wang et al., 18 Sep 2025). This communication effect is notable because earlier SFL literature often treated privacy and systems efficiency as competing objectives. By contrast, PM-SFL presents probabilistic masking as simultaneously improving privacy and reducing communication overhead (Wang et al., 18 Sep 2025).

In the broader ecosystem of SFL variants, PM-SFL sits alongside several specialized frameworks. SFL-LEO combines split learning and federated learning for highly dynamic LEO satellite-ground networks, using asynchronous training and layer alignment under intermittent connectivity (Wu et al., 18 Apr 2025). SFedHIFI addresses heterogeneous spiking federated learning by enabling adaptive model widths and cross-scale fusion through fire-rate-based aggregation (Tao et al., 16 Mar 2026). The training-latency-minimization framework for SFL focuses on cut-layer selection and parameter-server resource allocation to minimize synchronized round latency (Wen et al., 2023). PM-SFL differs from all three by centering structured probabilistic masking as the organizing principle, but it shares with them the general trajectory of SFL research toward practical deployment under heterogeneous resource conditions (Wang et al., 18 Sep 2025).

A potential misconception is that PM-SFL is simply an SFL system with differential privacy added on top. The paper’s characterization is narrower and more specific: its primary privacy mechanism is probabilistic mask training that injects structured randomness without relying on explicit noise, while differential privacy appears in the theoretical analysis and in optional backward-pass mechanisms (Wang et al., 18 Sep 2025). Another possible misconception is that privacy in SFL follows automatically from not sharing raw data. Prior attack literature contradicts that view, and PM-SFL’s very motivation presupposes that intermediate activations are themselves privacy-sensitive (Li et al., 2023, Wang et al., 18 Sep 2025).