---
title: 'Federated Unlearning: Methods & Trade-offs'
url: https://www.emergentmind.com/topics/federated-unlearning-ful
type: topic
---

# Federated Unlearning: Methods & Trade-offs

Federated Unlearning (FUL) is a set of formal methodologies and protocols for selectively removing the influence of specific data—such as clients, classes, or samples—from the parameters of models collaboratively trained under the federated learning paradigm. FUL aims to operationalize data-subject rights such as the right to be forgotten (RTBF) within distributed, privacy-sensitive settings, all while minimizing efficiency overhead and collateral loss in utility or privacy. FUL techniques span algorithmic, systems, and theoretical components, encompassing information-theoretic quantification, parameter and update manipulation, optimization formalisms, fairness and verification frameworks, and trade-offs driven by data and system heterogeneity.

## 1. Formalization and Privacy Foundations

The federated unlearning objective can be cast as reconstructing a global model as if some subset of local data (client, class, or sample) had never participated in training, without incurring the prohibitive costs of retraining from scratch. Precisely, let the original model parameters be $\theta^* = \mathrm{argmin}_\theta L(\theta; \cup_{i=1}^N D_i)$ and suppose a target data set $D_T$ (client, class, or sample) must be forgotten. The ideal unlearning solution is $\theta^- = \mathrm{argmin}_\theta L(\theta; \cup_{i=1}^N D_i \setminus D_T)$. FUL procedures define an operator $U$ such that $\theta_U = U(\theta^*, D_T)$ with $\Delta(\theta_U, \theta^-) \leq \epsilon$, for a chosen parameter-space or output-distribution distance $\Delta$ and tolerance $\epsilon$ [2508.19065], [2510.00833].

From an information-theoretic perspective, privacy leakage is quantified via the Fisher information about $D_T$ remaining in $\theta$, with removal policies targeting minimization of this information bound. Specifically, via the Cramér–Rao bound, FUL can target minimizing the Fisher information $I(1)$ of $D_T$ in $\theta$ to ensure that adversarial membership inference attack (MIA) success probability drops to random-guessing levels [2508.19065].

## 2. Mechanistic Approaches and Algorithmic Taxonomy

FUL algorithms are classified as:

- **Exact unlearning**: Full retraining on the pruned data set, serving as the gold standard but rarely practical due to computational cost [2510.00833], [2310.19218].
  
- **Approximate unlearning**: Efficient surrogates using:
  - **Update calibration and subtraction**: Methods such as FedEraser retain or reconstruct per-client update histories, then subtract or recalibrate to annul the forgotten client's effect, optionally with local recalibration [2012.13891], [2512.13381].
  - **Hessian and influence-based criteria**: Sensitivity to forgotten data is estimated using second-order (Hessian diagonal) information. The parameter’s target information score (TIS) is computed as $(h'_i / h_i)^2$ to rank and reset the most sensitive parameters, followed by only minimal retraining (TRIM) on reset parameters [2508.19065].
  - **Gradient ascent and trust region projection**: The target client's loss is maximized within an $\ell_2$ neighborhood of the global or neighborhood reference, then repaired by brief post-unlearning federated updates among remaining clients [2207.05521].
  - **Adapter and knowledge-overwriting**: Sparse adapters are attached to sensitive layers, learning to overwrite unwanted knowledge on only the remaining data, making unlearning reversible and preventing indiscriminate erasure [2502.20709], [2510.17895].
  - **Knowledge distillation**: The forgotten update is subtracted, and a student network is trained via distillation from the original (teacher) on unlabeled or public data to recover utility [2201.09441].

A detailed taxonomy with representative methods is tabulated below:

| Approach                      | Representative Methods   | Key Concept                                    |
|-------------------------------|-------------------------|------------------------------------------------|
| Exact retraining              | Baseline                | Full retrain on pruned data                    |
| Update calibration/subtraction| FedEraser, DPUL         | Update subtraction, calibration, VAE recovery  |
| Hessian/influence-based       | Hessian-TIS/Reset (2508.19065) | Hessian-based parameter sensitivity      |
| Adapter-based/overwriting     | FUSED, LLM Adapter      | Sparse, reversible adapters, knowledge overwrite|
| Knowledge distillation        | KD-based FU             | Server-only distillation restores accuracy      |
| Gradient ascent/projection    | PGD/Trust region        | PGD on target loss, constrained to trust region|

Each strategy varies with respect to utility retention, forgetting completeness (measured by attack success or MIA), computational/communication complexity, and data access/disclosure profile [2401.05146], [2310.19218].

## 3. Fairness, Efficiency, and Trade-Off Analysis

Fulfillment of RTBF in federated learning involves structural tensions among competing dimensions:

- **Utility retention** (on retained data): $\mathrm{NTA} = \mathrm{Acc}(\theta^u, D_{test})/\mathrm{Acc}(\theta^r, D_{test})$, where $\theta^r$ is the retrained reference [2508.19065].
- **Privacy/effectiveness** (on forgotten data): $\mathrm{NFS_{target/MIA}}$ quantified by matching target/per-attack accuracy drops to those of retraining [2508.19065].
- **Efficiency**: Recovery Time Ratio (RTR; ratio of retraining time to unlearning time) and Break Even Epochs (BEE; unlearning time per retrain epoch). Substantial acceleration (RTR up to $0.34$–$0.9$; BEE $7$–$113$) is reported for Hessian-based and adapter-based approaches [2508.19065], [2502.20709].
- **Fairness**: Two axes are formalized in recent work [2508.09866]:
  - **Performance fairness** $M_p$: equitable utility drop across clients of similar data to leaving clients;
  - **Efficiency fairness** $M_e$: unlearning time cost proportionate to each client’s categorical contribution.

Additionally, FUL under data heterogeneity is characterized by unavoidable trade-offs: improvements in verification (distance to retrain solution) generally worsen either global stability (performance on overall data) or fairness (equitability across remaining clients), with explicit lower bounds defined under standard smoothness and heterogeneity assumptions [2402.01276].

## 4. Verification, Security, and Regulatory Compliance

Verification is essential for trust and regulatory validity. The emerging framework (“veriFUL”) formalizes verification artifacts (cryptographic proofs, hardware-based attestations, blockchain logs, or empirical tests), metrics, and roles (target, co-verifier, aggregator, auditor) [2510.00833]. Principal requirements:

- **Completeness**: Indistinguishability from retraining, as measured by output distance or probe attacks.
- **Timeliness**: End-to-end delay must comply with regulatory standards (e.g., $t_{latency} \leq$ 1 month).
- **Correctness/Exclusivity**: Only the requested data are affected; collateral forgetting minimized.
- **Reversibility**: Ability to audit or, where designed, revert the unlearning.

Technically, proposals span from Chameleon hash-based blockchains for auditability [2401.15917], to functional encryption for hiding intent and enforcing update inclusion [2508.07873], to hierarchical sharding/clustering for scalable and privacy-tolerant unlearning under dynamic user participation [2508.09866], [2406.00966]. No single approach suffices; hybrid verification and audit-friendly code design are recommended.

## 5. Experimentation, Evaluation Protocols, and Benchmarks

FUL methodology evaluation integrates multiple axes—efficiency (RTR, comm. reduction), utility (Normalized Test Accuracy, class/test accuracy), privacy (attack success, MIA/BA on forgotten data), and fairness or stability metrics [2401.05146], [2508.19065], [2310.19218]. Standard recommended protocol [2401.05146], [2310.19218]:

- Measure all metrics on three models: original, retrained reference, and unlearned.
- Evaluate on diverse unlearning request types (client, class, sample) under IID and non-IID splits.
- Include marker-based privacy measures (e.g., MIA, backdoor attack).
- OpenFU and similar benchmarks support systematic comparison [2310.19218].

Empirical results confirm that modern methods (e.g., Hessian-TIS, adapter-based, DPUL) achieve NTA $>0.9$ (within 1–5% retraining gap) and leave residual attack rates (BA/MIA) near retrained or random baseline, with up to 12$\times$ speedup over retraining [2508.19065], [2512.13381], [2502.20709].

## 6. Challenges, Open Questions, and Future Work

Continued FUL development faces major challenges:

- **Standardization**: Need for shared data splits, metrics, and FUL-centric benchmarks.
- **Heterogeneity**: Non-IID data splits complicate both utility retention and privacy guarantees [2402.01276].
- **Scalability and dynamic participation**: Client churning, unlearning-request concurrency, and adversarial settings [2406.00966], [2508.09866].
- **Formal privacy and reversibility**: Information-theoretic or differential privacy bounds for unlearning remain open.
- **Incentive design**: Strategic incentives to prevent mass departures or low-quality participation are being formalized via Stackelberg games and federated Shapley valuation [2308.12502].
- **Regulatory compliance and auditability**: Frameworks for verifiable, provable unlearning (e.g. via veriFUL or blockchain-based protocols) are critical for practical deployment in regulated domains [2510.00833], [2401.15917], [2402.16294].

The trajectory for FUL features a progression toward efficient, robust, auditable, and regulation-compliant unlearning for increasingly heterogeneous and dynamic federated networks, especially in large-scale settings such as federated LLMs and critical applications [2510.17895], [2508.09866], [2510.00833].

Source: https://www.emergentmind.com/topics/federated-unlearning-ful