Certified Unlearning in Decentralized Federated Learning

• The paper introduces a certified unlearning framework that leverages Newton-style corrective updates to achieve (ε,δ)-indistinguishability between models with and without deleted client data.
• It employs second-order approximations and Fisher information to compute corrections efficiently without full retraining, reducing computational overhead.
• Privacy guarantees are enforced using calibrated Gaussian noise and network-wide propagation, achieving near-retraining accuracy while being ≈97% faster.

A certified unlearning framework for decentralized federated learning (DFL) formally guarantees that, after a client’s data or updates are deleted per a “right to be forgotten” request, the resulting model is provably (ε,δ)-indistinguishable from retraining the DFL system from scratch without the deleted data. Such certification must address the propagation of client influence through networked, peer-to-peer training—a scenario fundamentally more challenging than centralized or server-coordinated FL due to the fully decentralized communication topology and the mixing of information across clients.

1. DFL System Model and Influence Propagation

In decentralized federated learning, $N$ clients are nodes in an undirected communication graph $G=(V,E)$. Model parameters are stored locally at each node and updates are exchanged only with immediate neighbors. Training proceeds via decentralized SGD (DSGD):

• Each client $i$ with local dataset $S_i$ (size $n_i$) samples $\xi_{k,i} \sim \text{Uniform}(S_i)$ and computes stochastic gradient $g_{k,i} = \nabla F_i(x_{k,i}; \xi_{k,i})$.
• Local models are averaged according to a symmetric, doubly stochastic mixing matrix $Q$ (adapted to $G$), inducing information diffusion.
• Update: $$x_{k+1,i} = \sum_{j} Q_{ij} x_{k,j} - \gamma g_{k,i}$$.

After $K$ rounds, the aggregation of all local models $\bar{x} = \frac{1}{N}\sum_i x_{K,i}$ serves as the global model. Lemma 1 asserts that after sufficient iterations, every client’s information is mixed into all others with approximately equal weight: $\|Q^k - \frac{1}{N}11^\top\| \leq \rho^{k/2}$ for spectral parameter $\rho < 1$ (Wu et al., 10 Jan 2026).

2. Formal Definition of Certified Unlearning in DFL

Certified unlearning in DFL requires that, after removing a subset $U_c \subset S_c$ from client $c$, the output model $\mathcal{G}(U_c, \mathcal{A}(S), T)$ is statistically indistinguishable from the model retrained from scratch on $S \setminus U_c$. The certification is based on $(\epsilon, \delta)$-indistinguishability:

$$\forall W \subseteq X^N,\quad P[\mathcal{G}(U_c, \mathcal{A}(S), T) \in W] \leq e^\epsilon P[\mathcal{A}(S^{(-c)}) \in W] + \delta$$

where $\mathcal{A}$ is the DFL training operator and $T$ denotes relevant auxiliary state. This criterion directly generalizes the standard definitions used in (centralized) certified machine unlearning (Wu et al., 10 Jan 2026).

3. Newton-Style Corrective Updates and Fisher Approximation

The core unlearning mechanism computes a certified correction using a second-order (Newton-style) influence function that locally inverts the effect of the deleted samples:

• The exact retrained optimum $x'$ solves $\nabla \tilde{f}^{S\setminus U}(x') = 0$, where $\tilde{f}^{S\setminus U}$ is the new post-deletion empirical risk.
• To avoid full retraining, a Taylor expansion approximates $x'$ around $x_{K,c}$:

$$x_c^\Delta = \frac{1}{n_c - m_c} \hat{H}_c^{-1} \sum_{\xi \in U_c} \nabla F_c(x_{K,c}; \xi)$$

with Hessian estimate $$\hat{H}_c = \frac{1}{n_c - m_c} \sum_{\xi \in S_c \setminus U_c} \nabla^2 F_c(x_{K,c};\xi)$$.

• For scalability, $\hat{H}_c$ is approximated by the empirical Fisher information:

$$\Psi_c = \frac{1}{n_c - m_c} \sum_{\xi \in S_c \setminus U_c} \nabla F_c(x_{K,c}; \xi) \nabla F_c(x_{K,c}; \xi)^\top$$

which matches the Hessian at the empirical minimizer for log-likelihood losses and reduces storage from $O(d^2)$ to $O(d)$.

4. Privacy Guarantees via Gaussian Mechanism and Network Noise Propagation

To provide a formal $(\epsilon, \delta)$-certificate, the correction is perturbed using calibrated Gaussian noise (mirroring the approach of differential privacy for adjacent datasets):

• Sensitivity $\Delta F_c = O(m^2/n^2)$. For all clients, $\|\hat{x}_i - x'_{K,i}\| \leq \Delta F_c$.
• Each correction $x_c^\Delta$ is independently perturbed: $\nu_c \sim \mathcal{N}(0, \sigma_c^2 I_d)$ with

$$\sigma_c = \frac{\Delta F_c}{\epsilon} \sqrt{2\ln(1.25/\delta)}$$

• The noisy correction is broadcast across the network and each client updates:

$$\widetilde{x}_i = x_{K,i} + \frac{1}{N} (x_c^\Delta + \nu_c)$$

• One optional post-unlearning round of DSGD on retained data can be performed; by the DP post-processing theorem, the $(\epsilon, \delta)$ guarantee remains valid.

By Lemmas 2–3 and Theorem 1, this procedure yields a certified unlearning guarantee; i.e., the output distribution of the unlearning operation is close (in $(\epsilon, \delta)$-DP sense) to that of retraining after data deletion (Wu et al., 10 Jan 2026).

5. Algorithmic Workflow and Complexity

A typical certified unlearning episode in DFL comprises:

1. The client requesting deletion computes the correction $x_c^\Delta$ using the local Hessian/Fisher (on existing retained data).
2. The client adds Gaussian noise, sends the correction to neighbors.
3. The correction is disseminated via network flooding/gossip, ensuring each client receives the correction once.
4. All clients apply the update, and if desired, a single fine-tuning round.
5. No communication with the deleted client is required after the initial request.

The storage overhead is $O(d^2)$ per deletion for a full Hessian, or $O(d)$ using Fisher approximations. Communication cost is a single network-wide broadcast of the correction vector. Thus, compared to retraining (which requires $O(K)$ rounds), the certified unlearning protocol completes in $O(1)$ communication rounds plus local computation (Wu et al., 10 Jan 2026).

6. Theoretical Utility and Privacy Bounds

The certified unlearning protocol delivers explicit utility and privacy bounds:

• For all $i$, the post-unlearning model satisfies $\|x'_{K,i} - x_{K,i}\| \leq \frac{2Lm}{\lambda n}$.
• The Newton-based surrogate achieves error $$\|\hat{x}_i - x'_{K,i}\| \leq \frac{2ML^2 m^2}{\lambda^3 n^2}$$.
• The overall generalization bound for global minimizer $\tilde{x}$ is:

$$\mathbb{E}[f(\tilde{x}) - f(x^*)] \leq O\left(\frac{m L^2}{(n-m)\lambda} + \frac{\sqrt{d\ln(1/\delta)}\,M L^3 m^2}{\lambda^3 n^2 \epsilon}\right)$$

All bounds scale favorably when $m/n$ is small and the network mixing is rapid.

7. Empirical Validation and Network Scalability

The certified DFL unlearning framework is empirically validated on image (CIFAR-10/ResNet-18) and tabular (MNIST/logistic regression) benchmarks with both ring and Erdős–Rényi topologies and varying degrees of non-IIDness:

• Post-unlearning accuracy is within $1\%$–$2\%$ of retraining for all deletion modalities (sample-, class-, client-wise).
• Membership inference attack precision/recall drops to random guessing (50%), consistent with full deletion.
• Unlearning is ≈97% faster than retraining, with a single correction round replacing hundreds of retraining rounds in naive PDUDT approaches.

The protocol is robust to network structure, showcases scalable, efficient removal guarantees, and achieves formal $(\epsilon, \delta)$-unlearning certification (Wu et al., 10 Jan 2026).

---

The certified unlearning framework for DFL rigorously integrates influence quantification, second-order correction, scalable Hessian statistics, and formal privacy analysis adapted to decentralized architectures. It achieves provable guarantees under minimal network assumptions and demonstrates practical, efficient, and robust performance, confirming its suitability for RTBF compliance in peer-to-peer federated learning.