Privacy-Enhancing Split Learning

Updated 23 November 2025

Privacy-enhancing split learning is a framework that partitions deep models between clients and servers, ensuring raw data remains local while employing formal privacy guarantees.
It uses methods like noise injection, gradient clipping, and homomorphic encryption to prevent data reconstruction and label leakage during training.
Key challenges include balancing privacy and utility, managing computational overhead, and adapting protocols for diverse applications such as healthcare and IoT.

Privacy-enhancing split learning refers to a set of distributed machine learning frameworks and protocols in which the model is partitioned between client(s) and server(s) to ensure that raw data remains localized, intermediate representations are selectively revealed, and formal privacy guarantees are enforced through cryptographic, differential privacy, or information-theoretic techniques. This paradigm targets privacy protection in collaborative or resource-constrained scenarios, ranging from healthcare and IoT to large-scale time series applications.

1. Architectural Principles and Threat Model

Split learning (SL) divides deep models at a specified "cut layer" such that clients run the initial layers locally, transmitting only "smashed data" (intermediate activations) to the server, which completes model execution. In privacy-enhancing variants, mechanisms are explicitly designed to mitigate or eliminate leakage even from these intermediates.

The standard threat model assumes semi-honest (honest-but-curious) adversaries occupying the server role, sometimes clients, or eavesdroppers on communication links. Attacker goals include reconstructing private input data from activations, gradients, or model weights, inferring sensitive labels, or manipulating forward/backward passes (e.g., feature-space hijacking attacks).

Key privacy risks in classical SL protocols include:

Activation/Gradient inversion attacks: Model-inversion or gradient-based reconstruction of original input from activations or backward gradients.
Label leakage: Inferring user labels from gradients or activations in vertically partitioned (feature-split) data.
Feature-space hijacking attacks: Malicious servers injecting auxiliary decoders to recover client data via intentionally crafted gradients or architectures (Khan et al., 14 Apr 2024).

Privacy-enhancing SL approaches augment the basic workflow with cryptographic protection, differential privacy, noise regularization, adversarial/structural modeling of the embeddings, and protocols to prevent weight or feature leakage in multi-client settings.

2. Differential Privacy in Split Learning

Differential privacy (DP) is foundational for bounding worst-case information leakage in privacy-enhancing split learning. Key instantiations include:

Noise injection at the split layer: Clients (or their proxy gateways, e.g., grid stations in power network forecasting) inject Laplace or Gaussian noise (magnitude set by sensitivity analysis) into the smashed data prior to transmission, obtaining mechanism-level $(\epsilon,\delta)$ -DP (Iqbal et al., 3 Mar 2024, Pham et al., 2023).

$\widetilde{a} = a + \eta \quad \text{with} \quad \eta \sim \mathcal{N}(0, \sigma^2 I) \ \text{or} \ \mathrm{Lap}(0, s/\epsilon)$

DP for gradients and backpropagation: Clipping per-sample gradients and adding Gaussian noise before model parameter updates (as in SFL) ensures that the server learns differentially private representations (Thapa et al., 2020).
DP for vertically split data/labels: In vertically partitioned split learning (features held by $P_N$ , labels by $P_L$ ), gradient perturbation mechanisms such as TPSL add noise only in the "label-discriminant" direction for minimal utility loss and provable transcript-DP (Yang et al., 2022).
Utility–privacy trade-off: All studies report an inevitable degradation in utility as the noise scale increases, but the best trade-offs can be achieved by:
- Adding DP noise only at the deepest client layer (empirically and theoretically minimizing amplification) (Pham et al., 2023).
- Server-side "noise-review" data augmentation to prevent catastrophic forgetting when clients apply heterogeneous privacy budgets (Pham et al., 2023).
- Careful tuning of $(\epsilon, \delta)$ and leveraging advanced composition for iterative protocols (Iqbal et al., 3 Mar 2024, Thapa et al., 2020).

Empirical results consistently show that with moderate DP budgets ( $\epsilon \approx 2.5$ –$5.0$), the increase in mean absolute or squared error (MAE/MSE) and the drop in $R^2$ remain small ( $\sim$ 16% degradation or less) while reducing mutual information between data and shared activations to near-random levels (Iqbal et al., 3 Mar 2024).

Homomorphic Encryption (HE) and Function Secret Sharing (FSS) have been deployed to further guarantee that even the intermediate activations transmitted between client and server do not leak information:

Homomorphic Encryption (CKKS/FHE): Clients encrypt activation maps with CKKS (approximate-arithmetic HE), and the server performs linear model computation over ciphertexts. Only the client can decrypt the results, precluding any server-side inversion attack (Khan et al., 2023, Nguyen et al., 2023, Khan et al., 2023, Kanpak et al., 12 Jul 2024).

System Test Accuracy (Δ vs. Plaintext) Runtime Overhead Comm Overhead Key Guarantees

Plain SL Baseline 1× 1× Activation exposure

HE + SL (N=4096) -2.65% 3–10× 2–100× No activation leak

Homomorphic encryption prevents linkable inversion of activations at the server, with a trade-off between modulus size/ciphertext packing and efficiency; epoch times can be reduced $16\times$ – $64\times$ over previous HE-only training by packing activations and restricting ciphertext-level multiplications (Kanpak et al., 12 Jul 2024).
Function Secret Sharing (FSS): The client randomly masks activations and the server splits its computation into two non-colluding parties, each receiving a secret share of the masked activations and function. FSS ensures that no single party can reconstruct the original feature-space, neutralizing both visual invertibility and feature-space hijacking attacks (Khan et al., 14 Apr 2024).
Hybrid models: Protocols such as U-shaped SL with HE, and combined DP+HE, offer an additional defense layer. For instance, privacy preservation is achieved for both labels (never leaving the client) and features (encrypted in transit), with a measured accuracy drop of only ≈2–3% under optimal parameters (Khan et al., 2023, Khan et al., 2023).

System	Test Accuracy (Δ vs. Plaintext)	Runtime Overhead	Comm Overhead	Key Guarantees
Plain SL	Baseline	1×	1×	Activation exposure
HE + SL (N=4096)	-2.65%	3–10×	2–100×	No activation leak

4. Representation and Embedding Regularization

Critical for limiting the information content in transmitted activations are approaches that explicitly reduce correlation structure or intra-class separability in the client-side embeddings:

Potential Energy Loss (PELoss): An additional training loss that penalizes clustering of same-class embeddings at the split layer, thereby preventing clustering or fine-tuning attacks from reconstructing labels (Zheng et al., 2022). A moderate penalty ( $\lambda\approx 1$ ) lowers attack success rates to random levels with <2% accuracy loss.
Binarization and Leak Loss: Binarizing client-side weights and activations ("B-SL") induces inherent quantization noise, effectively satisfying $(\epsilon,\delta)$ -DP for $\delta=e^{-\epsilon/2}$ , while local "leak-loss" objectives directly minimize information recoverable from activation maps via structural (SSIM, distance correlation) metrics (Pham et al., 2022). Binarization incurs negligible accuracy drop on standard benchmarks and massively cuts client computation.
Information Bound via Fisher Information (ReFIL): Direct measurement and control of split activation privacy via the Fisher information trace (dFIL) yields privacy-utility guarantees: reconstruction mean-squared error is lower-bounded by the inverse of dFIL (Maeng et al., 2022). Compression layers and SNR-based regularization are used for fine-grained privacy–utility tuning.

5. Structural and Protocol Variants for Enforcing Privacy

Personalized Split Points and Bi-Level Optimization: In heterogeneous edge-device environments, each client autonomously selects its split point (using bi-level optimization to balance privacy leakage, energy, and personalized resource constraints), with Laplace noise added to activations proportional to the client's privacy-sensitivity coefficient (Fan et al., 23 Jul 2025). This sequential framework adapts to per-client requirements without requiring sharing of local information with the server, and achieves up to 59% energy savings and robust accuracy–FSIM trade-offs.
Parallelization, Caching, and Weight-Sharing Restrictions: Avoiding inter-client weight sharing further halves data leakage (as measured by SSIM) compared to classic SL/SFL, with only a ≈1% drop in accuracy for balanced or IID data. Parallel or cache-augmented variants maintain utility for late-arriving or non-IID clients (Pham et al., 2022).
Label Privacy and Transcripts: Differential privacy can be enforced not just on features but also on the transcript of all communicated messages, using perturbation in the label-sensitive gradient direction (Yang et al., 2022). DP protection at this level yields strong confidentiality under both black- and white-box label inference attacks.
Domain-Specific Optimizations: For graph-based split learning in satellite communication, dynamic topology-informed pruning and DP on raw graph data are combined to retain utility (accuracy ≈0.82–0.85) while reducing communication and sweeping 50% of FLOPs (Sun et al., 13 Sep 2024). In generative diffusion models, built-in Gaussian forward noise, an activation obfuscation function, and privacy-preserving prompt masking ensure both low FID and reconstruction resistance (Yao, 13 Sep 2024).

6. Empirical Measures and Privacy–Utility Trade-offs

All privacy-enhancing protocols explicitly measure and bound information leakage using:

Mutual Information Neural Estimation (MINE): Empirical upper bounds on $I(X;A)$ show that strong nonlinearity and DP noise reduce leakage to near the noise-only limit (Iqbal et al., 3 Mar 2024).
Membership Inference and Reconstruction Metrics: Attack success measured via SSIM, MSE, or accuracy drop under model-inversion/fine-tuning or k-means clustering (Zheng et al., 2022, Pham et al., 2022, Pham et al., 2023).
Formal privacy metrics: $(\epsilon,\delta)$ -DP guarantees tracked over SGD rounds via advanced composition; dFIL or mutual information-based lower bounds on adversarial recovery error.

A consistent empirical pattern emerges:

Reasonable privacy budgets ( $\epsilon\approx2$ –5) and properly structured splits incur utility penalties of <3–5%, often closer to 1%.
DP or cryptographic augmentation eliminates or radically attenuates both direct inversion and more sophisticated membership-inference and hijacking attacks.
Optimizations such as noise-positioning, adaptive noise assignment, regularization, and compressed representation control allow precise configuration of the privacy–utility–efficiency trade space.

7. Open Problems and Ongoing Challenges

Despite significant progress, open technical questions and limitations persist:

Communication and Computation Cost: Homomorphic encryption and multi-layer secret-sharing protocols remain orders-of-magnitude less efficient than unprotected SL, limiting scale in real-world deployments (Khan et al., 2023, Kanpak et al., 12 Jul 2024).
Backward Gradients & Active Adversaries: Most protocols protect only forward activations; encryption or DP of backward-pass gradients and defenses against active adversaries (beyond honest-but-curious) are ongoing research (Zheng et al., 2022, Khan et al., 2023).
Automated Cut-Layer Selection: General adaptive algorithms for automated optimal split selection across models and data domains remain to be fully developed (Li et al., 2023).
Tighter Utility Bounds: Analytical and empirical work to sharpen privacy–utility lower bounds (e.g., dFIL, mutual information) and understand the trade-off frontier for various data/modalities is ongoing (Maeng et al., 2022).
Scalability and Multi-Client Protocols: Efficient and robust privacy-enhancing split learning across many clients in asynchronous or dynamic environments continues to pose significant engineering and theoretical challenges (Pham et al., 2022, Fan et al., 23 Jul 2025).

Overall, privacy-enhancing split learning combines representation engineering, formal privacy modeling, and cryptographic innovations to enable distributed learning under strong adversarial models, with demonstrated empirical efficacy and a growing body of rigorous analysis (Iqbal et al., 3 Mar 2024, Pham et al., 2023, Zheng et al., 2022, Khan et al., 2023, Fan et al., 23 Jul 2025, Khan et al., 14 Apr 2024).