---
title: Transformation-guided Federated Unlearning (ToFU)
url: https://www.emergentmind.com/topics/transformation-guided-federated-unlearning-tofu
type: topic
---

# Transformation-guided Federated Unlearning (ToFU)

Transformation-guided Federated Unlearning (ToFU) is a learning-to-unlearn federated unlearning framework that incorporates transformations during the learning process so that a federated model memorizes less instance-specific information and can later forget more easily. Rather than treating unlearning as a purely post-hoc repair step, ToFU modifies local training through sample-dependent transformations, progressive training, and a transformation-invariant regularizer, after which a forgetting request is handled by a lightweight fine-tuning step on the retain set. The framework is presented as a plug-and-play enhancement for federated learning and is supported by a mutual-information analysis in which transformation composition monotonically reduces information about the forget set [2509.15861].

## 1. Federated unlearning setting and the ToFU design shift

Federated unlearning (FU) is the removal of a specific client’s contribution or a target subset’s influence from a federated model after federated training has already occurred, under the constraint that raw data remain decentralized and inaccessible to the server. In the standard FU formalization, an unlearning algorithm transforms a trained global model into an unlearned model that should be indistinguishable or approximately indistinguishable from a model retrained without the forgotten data [2401.05146]. The same survey also emphasizes that FU must balance forgetting quality, retained utility, efficiency, and privacy, and that naive retraining from scratch is often computationally expensive, communication-heavy, and impractical in federated settings [2401.05146].

ToFU is motivated by the claim that current FU methods are mostly post-hoc and therefore must remove information that has already been deeply memorized by neural networks. In this view, the central problem is not only how to erase information after training, but how to design federated learning systems so that they are inherently amenable to forgetting. The framework’s basic intuition is that if the model is trained to rely on transformation-invariant features rather than raw instance-specific features, then later deletion requests become easier because the learned representation contains less information tied to any single sample [2509.15861].

This design shift changes the locus of intervention. Instead of concentrating all forgetting effort in a specialized FU stage, ToFU front-loads part of that effort into the original learning process. A plausible implication is that ToFU belongs to the family of efficient approximation-to-retraining FU methods, but its distinctive feature is that the approximation is prepared during training rather than being recovered only after a deletion request.

## 2. Training-time mechanism and optimization pipeline

ToFU operates within the standard client-server federated loop. For each federated learning round, the server sends the global model to selected clients, each client performs local training, the server aggregates the client updates via weighted averaging, and when a forgetting request arrives, unlearning is simplified to a lightweight fine-tuning step on the retain set [2509.15861].

The local training procedure modifies each mini-batch in three ways. First, ToFU evaluates per-sample loss values using the latest global model \(f_\theta^{t-1}\). Second, it applies sample-dependent transformations. Third, it optimizes a loss that combines supervised task loss on transformed data with a transformation-invariant regularizer [2509.15861].

The progressive transformation schedule is written as
\[
M := \left\lceil \frac{t}{T} M \right\rfloor,
\]
where \(T\) is the total number of communication rounds, \(t\) is the current round, and \(M\) is the maximum transformation intensity. The stated intent is that transformation strength grows as training proceeds [2509.15861].

The sample-dependent assignment uses an inverse quantile value function,
\[
\digamma(\mathbf{x}) = \left(\frac{\sum_{j=1}^n \mathbf{1}_{x_j > x_i}}{n}\right)_{i=1}^n,
\]
and the number of transformations applied to sample \(i\) is
\[
A_i = \left\lceil M \cdot \digamma(\mathbf{x})_i \right\rceil.
\]
The reported interpretation is that samples with lower loss receive more transformations, while samples with higher loss receive fewer transformations [2509.15861].

The local objective is
\[
\ell(\theta; x, y) = -y\log(p_\theta(x^*)) + \gamma D_{KL}(f_\theta(x^*) \parallel f_\theta(x)),
\]
where \(x^*\) is the transformed sample, \(f_\theta(\cdot)\) denotes latent representations, and \(\gamma\) controls regularization strength. The default value is
\[
\gamma = 0.01.
\]
This regularizer is intended to make original and transformed samples representation-consistent, thereby biasing training toward transformation-invariant features rather than fragile sample-specific cues [2509.15861].

The implementation uses an ordered transformation list comprising geometric transforms, color transforms, noise/blur transforms, and information-degrading transforms. The algorithms explicitly listed are HorizontalFlip, VerticalFlip, ShiftScaleRotate, RandomBrightnessContrast, HueSaturationValue, RandomGamma, RGBShift, GaussianBlur, MotionBlur, Downscale, ToGray, ChannelShuffle, ColorJitter, Sharpen, Emboss, GaussNoise, RandomResizedCrop, and CoarseDropout. The appendix states that these transformations are implemented with Albumentations [2509.15861].

## 3. Mutual-information formulation and transformation composition theorem

The theoretical formulation casts unlearning in terms of mutual information between models with respect to a dataset \(D_S\):
\[
I(\theta, \theta'; D_S) = D_{KL}(p_{\theta,\theta'}(x,y) \| p_\theta(x|y)p_{\theta'}(y|x)).
\]
The forget and retain objectives are then expressed as minimizing information shared with the forget set while maximizing information shared with the retain set. The relaxed optimization problem is written as
\[
\min I(\theta, \theta'; D_{\text{forget}}) \text{ and } \max I(\theta, \theta'; D_{\text{retain}})
\]
and, more compactly,
\[
\theta^* = \min_{\theta' \in \Theta} \frac{I(\theta, \theta'; D_{\text{forget}})}{I(\theta, \theta'; D_{\text{retain}})}.
\]
The paper interprets \(I(\theta,\theta';D_{\text{forget}})=0\) as meaning that the unlearned model contains no shared information with the original model on the forget set [2509.15861].

A transformation is defined as \(\mathcal{T}_i : \Theta \rightarrow \Theta\), and a transformation of intensity \(m\) is the composition
\[
\mathcal{T}^{(m)} = \mathcal{T}_1 \circ \mathcal{T}_2 \circ \dots \circ \mathcal{T}_m.
\]
The central theorem states that for two transformations \(\mathcal{T}_1\) and \(\mathcal{T}_2\), and for \(D_{\text{forget}} = D'_{\text{forget}} \cup D''_{\text{forget}}\), the composed transformations satisfy inequalities such as
\[
I(\theta, \mathcal{T}'^{(2)}; D_{\text{forget}}) \leq I(\theta, \mathcal{T}_1; D_{\text{forget}})
\]
and
\[
I(\theta, \mathcal{T}'^{(2)}; D_{\text{forget}}) \leq I(\theta, \mathcal{T}_2; D_{\text{forget}}),
\]
with analogous bounds for the reversed composition \(\mathcal{T}''^{(2)}\) [2509.15861].

The proof uses the data processing inequality together with the Markov chains
\[
\theta \rightarrow \mathcal{T}_1(\theta) \rightarrow \mathcal{T}_2(\mathcal{T}_1(\theta)) = \mathcal{T}'^{(2)}(\theta)
\]
and
\[
\theta \rightarrow \mathcal{T}_2(\theta) \rightarrow \mathcal{T}_1(\mathcal{T}_2(\theta)) = \mathcal{T}''^{(2)}(\theta).
\]
The corollary extends this to arbitrary composition length:
\[
I(\theta, \mathcal{T}^{(m)}; D_{\text{forget}}) \leq I(\theta, \mathcal{T}^{(m-1)}; D_{\text{forget}}).
\]
In the paper’s interpretation, adding more transformations monotonically improves unlearning effectiveness [2509.15861].

This theorem provides the conceptual backbone for the framework. It links transformation composition to a formal contraction of information about the forget set, thereby justifying the training-time use of progressive and repeated transformations rather than treating them as conventional augmentation alone.

## 4. Unlearning phase and plug-and-play integration with existing FU methods

The unlearning phase in ToFU is intentionally simple. When a removal request occurs, the model is trained only on the retain set, for \(E_u\) epochs and with learning rate \(\eta_u\), using the update
\[
\theta = \theta - \eta_u \nabla_\theta \mathcal{L}(\theta).
\]
This is described as a lightweight fine-tuning operation rather than a separate, heavy unlearning procedure [2509.15861].

The framework is explicitly characterized as plug-and-play. It is not tied to a specific FU algorithm; rather, it modifies the training stage so that the resulting model is easier to unlearn by a downstream FU mechanism. The reported integrations are with FedEraser, FedPGD, and FedAda, with average improvements in overall FU performance of \(6.030 \pm 1.359\%\), \(11.007 \pm 1.902\%\), and \(4.733 \pm 1.346\%\), respectively [2509.15861].

This architectural role matters for how ToFU should be classified. It is neither merely a training heuristic nor a complete replacement for federated unlearning algorithms. Instead, it is a training-time enhancement that is supposed to reduce memorization and thereby simplify later forgetting. A plausible implication is that ToFU can be layered atop heterogeneous post-hoc FU mechanisms without requiring them to share the same deletion objective or optimization geometry.

## 5. Empirical evaluation, metrics, and ablations

The reported experiments use CIFAR-10, CIFAR-100, and MUFAC. Client data are sampled from a Dirichlet distribution,
\[
p_{k,c} \sim \mathfrak{D}(\kappa),
\]
with \(\kappa = 1.0\). For CIFAR-10 and CIFAR-100, forgetting samples are randomly selected from clients at rates of 10% for client 1, 30% for client 3, 60% for client 6, and 90% for client 9, while MUFAC uses the predefined forget set from the original benchmark [2509.15861].

The implementation details are fixed: ResNet-18, Schedule-free-AdamW, batch size 128, learning rate \(3\times10^{-4}\), local training for 10 epochs, 50 federated learning rounds, 10 clients, 20 unlearning rounds, and 5 local epochs in unlearning. The reported hardware and software are 2 NVIDIA RTX 4090 GPUs, AMD EPYC 9654P CPU, 192 GB RAM, PyTorch, and the Flower FL framework. In addition, 10% of each client dataset is reserved for validation [2509.15861].

The evaluation reports Test Accuracy, Retain Accuracy, Membership Inference Attack Efficacy, and an Overall metric. LiRA is used as the membership inference attack, and shadow models are built using global model parameters from the final 5 communication rounds of federated learning training [2509.15861].

The main overall results are summarized below.

| Dataset | ToFU Overall | Comparator Overalls |
|---|---:|---|
| CIFAR-10 | 0.6712 | FedEraser 0.6450; FedPGD 0.6565; FedAda 0.6466; Exact unlearning 0.6874 |
| CIFAR-100 | 0.5798 | FedEraser 0.5046; FedPGD 0.4520; FedAda 0.5117; Exact unlearning 0.5106 |
| MUFAC | 0.7657 | FedEraser 0.7027; FedPGD 0.6988; FedAda 0.6564; Exact unlearning 0.6473 |

Across these datasets, the paper states that ToFU outperforms the listed FU baselines on Overall score and is comparable to exact unlearning in behavior. On CIFAR-10, for example, ToFU reports Test 0.7943, Retain 0.8955, MIA 0.3239, and Overall 0.6712; on MUFAC, the reported Overall is 0.7657, above the reported exact-unlearning Overall of 0.6473 [2509.15861].

The privacy analysis includes the claim that ToFU narrows the loss distribution gap between forgotten and unseen samples, reduces statistical divergence, and approaches the gold-standard exact unlearning distributional behavior. A figure based on Kolmogorov–Smirnov statistics is reported as showing that ToFU’s output distributions on test versus forget sets are much closer than those of competing FU methods [2509.15861].

The runtime results report a mean running time of 771.2 seconds and a 24–28% reduction over FedPGD, FedAda, and FedEraser. The explanation given is that ToFU shifts work into the original training stage so that unlearning later becomes a lightweight fine-tuning procedure [2509.15861].

The ablation study identifies three core components: SDT, TIR, and PT. The full model achieves Test Acc 0.5032, Retain Acc 0.7802, MIA Eff. 0.4560, and Overall 0.5798. The appendix further reports that \(\gamma = 0\) gives reasonable performance, small positive \(\gamma\) gives slight improvement, negative \(\gamma\) causes catastrophic degradation, and larger positive \(\gamma\) still performs well but with diminishing returns. Training cost increases with more transformations: from 1277s with 0 transforms to 2027s with 8 transforms, corresponding to a worst-case training overhead of 58.7% and an average-case overhead of 29.6% for 4 transforms [2509.15861].

## 6. Related transformation-guided methods, benchmark ambiguity, and limitations

ToFU belongs to a broader transformation-guided current in federated unlearning, but it is not the only formulation of that idea. A closely related example is Federated Unlearning via Class-aware Representation Transformation (FUCRT), which treats class-level forgetting as moving the representations of the forgotten class into a suitable remaining-class region in feature space. FUCRT uses a transformation class selection strategy and dual class-aware contrastive learning, with a local objective
\[
\mathcal{L} = \mathcal{L}_{CE} + \lambda_1 \mathcal{L}_{local} + \lambda_2 \mathcal{L}_{global},
\]
and reports complete erasure of unlearning classes together with strong remaining-class utility in IID and Non-IID settings [2410.06848]. The comparison is informative because both methods replace purely destructive forgetting with representation- or invariance-oriented transformation, but ToFU does so at training time for general FU, whereas FUCRT is organized around class-aware representation migration.

A separate source of ambiguity is the term “TOFU” in large-language-model unlearning. In “Hierarchical Federated Unlearning for Large Language Models,” TOFU denotes the benchmark “Task of Fictitious Unlearning for LLMs” from Maini et al. (2024), while the proposed algorithm in that paper is FULM. That paper states explicitly that FULM is not the same as a method called “Transformation-guided Federated Unlearning,” and that its relation to TOFU is benchmark-level rather than algorithmic [2510.17895]. This distinction is important because the same acronym appears in two different roles across the literature.

The limitations reported for ToFU are concrete. First, it increases federated learning training time because transformed samples require extra computation. Second, too many or too strong transformations can remove task-relevant information and harm utility, which is why the framework relies on sample-dependent transformation intensity, progressive scheduling, and the invariant regularizer. Third, the MUFAC benchmark shows a small utility drop relative to vanilla FedAvg, with FedAvg at 0.8451 and FedAvg + ToFU at 0.8165; the paper attributes this to real facial images with lighting variation and related conditions being less amenable to the specific transformations used. Fourth, the method is not claimed to be universally optimal for every transformation family, and RandAugment is suggested as a way to improve generalization across transformation families [2509.15861].

The reported edge case in which no clients request unlearning further clarifies the trade-off. On CIFAR-10 and CIFAR-100, ToFU slightly improves or maintains performance versus FedAvg, whereas on MUFAC there is a mild degradation. This suggests that ToFU is not intrinsically an unlearning-only procedure; it is a modified training regime whose benefits depend on how well the chosen transformation family matches the data geometry [2509.15861].

Source: https://www.emergentmind.com/topics/transformation-guided-federated-unlearning-tofu