DP in Split Learning: Techniques & Trade-offs

• Differential Privacy in Split Learning is a framework that integrates rigorous DP techniques into split training, ensuring mathematically sound protection of sensitive data.
• It employs calibrated noise injection in activations, gradients, and weight updates, with sensitivity analysis to balance privacy guarantees and model utility.
• Empirical studies reveal that while lower privacy budgets lead to significant accuracy drops, relaxed DP parameters can maintain high performance with minimal degradation.

Differential privacy (DP) provides a rigorous mathematical framework for quantifying the privacy loss incurred when releasing information derived from sensitive datasets. In the context of split learning (SL)—a distributed training paradigm in which a model is partitioned across clients and servers—DP mechanisms are incorporated to mitigate the risk of private information leakage through shared activations, gradients, weights, or parameter updates. The interaction between split learning architectures and differential privacy has motivated a substantial research body investigating algorithmic design, formal guarantees, empirical behavior, and trade-offs.

1. Split Learning Architectures: Vulnerabilities and Privacy Risks

A canonical split learning pipeline divides a neural network at a designated “cut layer” into a client-side model and a server-side model. The client processes its private data up to the split and transmits only the resulting activations (“smashed data”); the server completes forward and backward passes, sending gradients at the cut back to the client for local parameter updates. Typical exchanges include: (i) smashed activations, (ii) cut-layer gradients, (iii) periodic weight updates, and (iv) auxiliary information depending on the setting (vertical FL, label-partitioned learning, multi-tier hierarchies).

Privacy risks in vanilla SL have been extensively documented. Without augmentation, both smashed activations and gradients have been shown to leak substantial information—allowing adversaries to reconstruct private inputs or labels via inversion and feature-space hijacking attacks (Gawron et al., 2022, Qiu et al., 2023, Yang et al., 2022). Quantitative analyses consistently indicate that the backward-phase gradients are particularly vulnerable to label leakage under vertical partitioning (Yang et al., 2022).

2. Formalization of Differential Privacy Mechanisms in Split Learning

The integration of DP into SL follows the canonical (ε,δ)-DP (or ε-DP for pure Laplace mechanisms, or Rényi DP variants) paradigm. Mechanisms typically fall into three categories:

• Activation perturbation: DP noise (Laplace, Gaussian, or more subtle randomized-response techniques) is added to smashed activations before transmission to the server (Abuadbba et al., 2020, Iqbal et al., 3 Mar 2024, Pham et al., 2022, Pham et al., 2023, Sun et al., 13 Sep 2024).
• Gradient perturbation: DP-SGD or DP-Adam optimizers inject calibrated Gaussian noise into local gradient computations, often after gradient clipping (Gawron et al., 2022, Qiu et al., 2023, Thapa et al., 2020).
• Weight update perturbation: DP mechanisms (typically Laplace) are applied to weight updates before aggregation, as in hierarchical or federated split settings (Quan et al., 16 Jan 2024, Li et al., 30 Sep 2025).

Sensitivity analysis is carried out per mechanism: for activations, the ℓ₁ or ℓ₂-sensitivity of the split-layer mapping; for gradients, clipping bounds set the maximum influence of any single client or record.

Representative mechanisms include:

Mechanism	Noise Distribution	Sensitivity	DP Type
Activation DP	Laplace or Gaussian	Δ = max‖f(x)−f(x′)‖（ℓ₁/ℓ₂)	(ε, δ)-DP
Gradient DP	Gaussian (after clipping)	C (clip norm)	(ε, δ)-DP
Weight update DP	Laplace	Δf = max‖w − w′‖₁	ε-LDP

Noisy releases are mathematically guaranteed to satisfy the DP definition: $$\Pr[M(D)\in S] \leq e^\epsilon \Pr[M(D')\in S] + \delta$$ for any pair of neighboring datasets D, D'.

3. DP Integration in Split Learning: Algorithmic Realizations and Architectural Variants

Hierarchical Split Federated Learning (HierSFL): A three-tier model with clients, mobile edge servers (MES), and a cloud server. Each client injects Laplace noise calibrated to the ℓ₁-sensitivity of its weight updates before aggregation at the MES. Periodic edge and cloud aggregations combine these noisy updates using FedAvg. Empirical studies on CIFAR-10 and MNIST demonstrate that as ε increases (i.e., as privacy is relaxed), accuracy improves and utility degradation due to DP becomes marginal beyond ε=5. No advanced composition or adaptive budgeting is detailed (Quan et al., 16 Jan 2024).

Binarized and Resource-Constrained SL: Models leveraging sign-binarized layers combine DP perturbations with binarization-based intrinsic privacy. Laplace noise is injected after binarization; randomized-response can be used for strong per-coordinate privacy guarantees with minimal accuracy degradation. Empirical results on MNIST suggest randomized-response methods yield less than 3% accuracy loss at ε ≈ 0.5 (Pham et al., 2022).

Federated Split Learning with DP: Hybrid approaches split models into local (private) and global (public) parts, perturb only the public parameter vectors with Gaussian noise, and exploit random client participation plus subsampling to strengthen privacy via statistical amplification. Rigorous privacy accounting (using the Moments Accountant and strong composition) allows substantially smaller ε for the same noise parameter, yielding >10% accuracy gain over naive DP-FedAvg under the same ε (Li et al., 30 Sep 2025).

Distributed Graph Learning: Input-level DP (on feature matrices and adjacency) is achieved by calibrating Gaussian noise to graph query sensitivity. The DP guarantee composes over training rounds to (T·ε, T·δ), and utility degrades gracefully even for large graphs at ε ≈ 0.2–1.0 (Sun et al., 13 Sep 2024).

Domain-Specific SL Frameworks: In smart grids and healthcare, DP noise is typically integrated at the Split-1 client activation, with both Laplace (ε-DP) and Gaussian ((ε, δ)-DP) mechanisms supported. Privacy-utility trade-off analysis is conducted via mutual information reduction and application-level metrics; for ε ≥ 5, utility is nearly unaffected, but for ε ≤ 2.5, forecasting accuracy drops 10–35% (Iqbal et al., 3 Mar 2024, Abuadbba et al., 2020).

4. Privacy-Utility Trade-Offs and Empirical Behavior

A recurring theme is the evaluation of accuracy versus privacy parameters ε (and δ where applicable). Across experiments:

• Utility degrades sharply for low ε: For example, in 1D CNN SL for ECG, ε=1 yields ~50% accuracy versus 98.9% for no-DP; at ε=10, accuracy drop is negligible (Abuadbba et al., 2020). In collaborative load forecasting, mutual information between input and smashed activations can be reduced by 30–50% at low ε with 10–35% MAE increase (Iqbal et al., 3 Mar 2024).
• Privacy protection is ineffective without sufficient noise: DP-SGD-style Gaussian noise on local gradients thwarts cut-layer reconstruction attacks with little utility loss at noise multipliers σ ≥ 0.01 (Qiu et al., 2023).
• Dimension reduction as auxiliary defense: Projecting smashed data to lower-dimensional space can further diminish attack success but may also impair accuracy if excessive (Gawron et al., 2022, Pham et al., 2023).

Advanced multi-client SL settings support heterogeneous ε_i budgets via server-side augmentation strategies, ensuring that clients with stricter privacy do not suffer catastrophic forgetting or utility collapse (Pham et al., 2023).

5. Attacks and Limitations: Adversarial Leakage and Insufficiency of DP

Studies demonstrate that naïve application of DP mechanisms (e.g., adding noise to gradients but not activations) can fail to protect against sophisticated adversaries:

• Feature Space Hijacking Attack (FSHA): A malicious server employing autoencoders and min–max discriminators can invert smashed activations, reconstructing raw inputs with low error even under strong DP (ε=0.5). DP on gradients does not obfuscate the activations themselves; thus FSHA can eventually succeed given sufficient iterations (Gawron et al., 2022).
• Label Leakage under Vertical Partitioning: Gradients returned by the label party leak label information unless transcript-level DP is enforced. Attacks exploiting gradient norms or directionality (Norm Attack, Spectral Attack, SDA) achieve near-perfect inference unless gradients themselves are perturbed using targeted Laplace noise in the “label difference” direction (Yang et al., 2022).

Consequently, effective SL privacy requires DP at the exact points of information exchange—ideally on activations and gradients—rather than solely on model updates or local optimizers.

6. Privacy Accounting, Composition, and Practical Implementations

Privacy loss accumulates additively in the naïve composition over multiple DP noise applications or throughout multiple SL rounds. Most contemporary works recommend either basic composition (ε_total = Σ_p ε_p) or use advanced approaches such as Moments Accountant or Rényi DP for tighter bounds in high-iteration regimes (Thapa et al., 2020, Qiu et al., 2023, Ndeko et al., 9 Nov 2024, Li et al., 30 Sep 2025). In federated or hierarchical-SL with multi-client participation and data subsampling, statistical amplification further reduces per-client and global privacy loss (Li et al., 30 Sep 2025).

Efficient privacy tracking and tuning are critical: clipping calibrates sensitivity, minibatch size b reduces per-example noise, and adaptive schedules may front-load higher noise early or adjust clipping dynamically to minimize overall accuracy loss (Thapa et al., 2020).

7. Research Directions and Open Challenges

Despite the progress, several challenges remain:

• Tight composition and budgeting: HierSFL and related works point out the need for adaptive privacy budgets and advanced accounting, particularly as the number of aggregation rounds increases (Quan et al., 16 Jan 2024).
• Sensitivity estimation in deep architectures: Accurate sensitivity computation for arbitrary deep networks remains open.
• Combining DP with cryptography: Adversarial leakage not mitigated by DP alone motivates hybrid approaches combining DP with encrypted channels, secure enclaves, or dynamic anomaly detection (Gawron et al., 2022).
• Task- and architecture-specific tuning: The locus of noise injection (input, mid-layer, or cut-layer), the choice of noise distribution, and optional dimension reduction yield different trade-offs for communication, utility, and privacy (Pham et al., 2023).

A plausible implication is that robust privacy in split learning will necessitate context-aware, hybrid strategies—encompassing DP mechanisms tailored to architectural cut-points, application of advanced privacy composition, data-dependent dimension reduction or quantization, and defense-in-depth via cryptographic safeguards.

---

References:

• "HierSFL: Local Differential Privacy-aided Split Federated Learning in Mobile Edge Computing" (Quan et al., 16 Jan 2024)
• "Feature Space Hijacking Attacks against Differentially Private Split Learning" (Gawron et al., 2022)
• "Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?" (Abuadbba et al., 2020)
• "Evaluating Privacy Leakage in Split Learning" (Qiu et al., 2023)
• "An Efficient Privacy-aware Split Learning Framework for Satellite Communications" (Sun et al., 13 Sep 2024)
• "Privacy-Preserving Collaborative Split Learning Framework for Smart Grid Load Forecasting" (Iqbal et al., 3 Mar 2024)
• "SplitFed: When Federated Learning Meets Split Learning" (Thapa et al., 2020)
• "Enhancing Accuracy-Privacy Trade-off in Differentially Private Split Learning" (Pham et al., 2023)
• "Differentially Private Label Protection in Split Learning" (Yang et al., 2022)
• "Binarizing Split Learning for Data Privacy Enhancement and Computation Reduction" (Pham et al., 2022)
• "Federated Split Learning for Human Activity Recognition with Differential Privacy" (Ndeko et al., 9 Nov 2024)
• "Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation" (Li et al., 30 Sep 2025)