Transformation-guided Federated Unlearning (ToFU)

• The paper introduces ToFU, which incorporates sample-dependent transformations and a progressive schedule during training to make federated models inherently easier to unlearn.
• It shifts unlearning effort from a heavy post-hoc repair process to training time by optimizing for transformation-invariant features using a dedicated regularizer.
• Empirical results on datasets like CIFAR-10, CIFAR-100, and MUFAC show that ToFU achieves significant improvements in unlearning performance and privacy metrics while reducing computational overhead.

Transformation-guided Federated Unlearning (ToFU) is a learning-to-unlearn federated unlearning framework that incorporates transformations during the learning process so that a federated model memorizes less instance-specific information and can later forget more easily. Rather than treating unlearning as a purely post-hoc repair step, ToFU modifies local training through sample-dependent transformations, progressive training, and a transformation-invariant regularizer, after which a forgetting request is handled by a lightweight fine-tuning step on the retain set. The framework is presented as a plug-and-play enhancement for federated learning and is supported by a mutual-information analysis in which transformation composition monotonically reduces information about the forget set (Tran et al., 19 Sep 2025).

1. Federated unlearning setting and the ToFU design shift

Federated unlearning (FU) is the removal of a specific client’s contribution or a target subset’s influence from a federated model after federated training has already occurred, under the constraint that raw data remain decentralized and inaccessible to the server. In the standard FU formalization, an unlearning algorithm transforms a trained global model into an unlearned model that should be indistinguishable or approximately indistinguishable from a model retrained without the forgotten data (Romandini et al., 2024). The same survey also emphasizes that FU must balance forgetting quality, retained utility, efficiency, and privacy, and that naive retraining from scratch is often computationally expensive, communication-heavy, and impractical in federated settings (Romandini et al., 2024).

ToFU is motivated by the claim that current FU methods are mostly post-hoc and therefore must remove information that has already been deeply memorized by neural networks. In this view, the central problem is not only how to erase information after training, but how to design federated learning systems so that they are inherently amenable to forgetting. The framework’s basic intuition is that if the model is trained to rely on transformation-invariant features rather than raw instance-specific features, then later deletion requests become easier because the learned representation contains less information tied to any single sample (Tran et al., 19 Sep 2025).

This design shift changes the locus of intervention. Instead of concentrating all forgetting effort in a specialized FU stage, ToFU front-loads part of that effort into the original learning process. A plausible implication is that ToFU belongs to the family of efficient approximation-to-retraining FU methods, but its distinctive feature is that the approximation is prepared during training rather than being recovered only after a deletion request.

2. Training-time mechanism and optimization pipeline

ToFU operates within the standard client-server federated loop. For each federated learning round, the server sends the global model to selected clients, each client performs local training, the server aggregates the client updates via weighted averaging, and when a forgetting request arrives, unlearning is simplified to a lightweight fine-tuning step on the retain set (Tran et al., 19 Sep 2025).

The local training procedure modifies each mini-batch in three ways. First, ToFU evaluates per-sample loss values using the latest global model $f_\theta^{t-1}$. Second, it applies sample-dependent transformations. Third, it optimizes a loss that combines supervised task loss on transformed data with a transformation-invariant regularizer (Tran et al., 19 Sep 2025).

The progressive transformation schedule is written as

$M := \left\lceil \frac{t}{T} M \right\rfloor,$

where $T$ is the total number of communication rounds, $t$ is the current round, and $M$ is the maximum transformation intensity. The stated intent is that transformation strength grows as training proceeds (Tran et al., 19 Sep 2025).

The sample-dependent assignment uses an inverse quantile value function,

$$\digamma(\mathbf{x}) = \left(\frac{\sum_{j=1}^n \mathbf{1}_{x_j > x_i}}{n}\right)_{i=1}^n,$$

and the number of transformations applied to sample $i$ is

$$A_i = \left\lceil M \cdot \digamma(\mathbf{x})_i \right\rceil.$$

The reported interpretation is that samples with lower loss receive more transformations, while samples with higher loss receive fewer transformations (Tran et al., 19 Sep 2025).

The local objective is

$$\ell(\theta; x, y) = -y\log(p_\theta(x^*)) + \gamma D_{KL}(f_\theta(x^*) \parallel f_\theta(x)),$$

where $x^*$ is the transformed sample, $M := \left\lceil \frac{t}{T} M \right\rfloor,$0 denotes latent representations, and $M := \left\lceil \frac{t}{T} M \right\rfloor,$1 controls regularization strength. The default value is

$M := \left\lceil \frac{t}{T} M \right\rfloor,$2

This regularizer is intended to make original and transformed samples representation-consistent, thereby biasing training toward transformation-invariant features rather than fragile sample-specific cues (Tran et al., 19 Sep 2025).

The implementation uses an ordered transformation list comprising geometric transforms, color transforms, noise/blur transforms, and information-degrading transforms. The algorithms explicitly listed are HorizontalFlip, VerticalFlip, ShiftScaleRotate, RandomBrightnessContrast, HueSaturationValue, RandomGamma, RGBShift, GaussianBlur, MotionBlur, Downscale, ToGray, ChannelShuffle, ColorJitter, Sharpen, Emboss, GaussNoise, RandomResizedCrop, and CoarseDropout. The appendix states that these transformations are implemented with Albumentations (Tran et al., 19 Sep 2025).

3. Mutual-information formulation and transformation composition theorem

The theoretical formulation casts unlearning in terms of mutual information between models with respect to a dataset $M := \left\lceil \frac{t}{T} M \right\rfloor,$3: $M := \left\lceil \frac{t}{T} M \right\rfloor,$4 The forget and retain objectives are then expressed as minimizing information shared with the forget set while maximizing information shared with the retain set. The relaxed optimization problem is written as

$M := \left\lceil \frac{t}{T} M \right\rfloor,$5

and, more compactly,

$M := \left\lceil \frac{t}{T} M \right\rfloor,$6

The paper interprets $M := \left\lceil \frac{t}{T} M \right\rfloor,$7 as meaning that the unlearned model contains no shared information with the original model on the forget set (Tran et al., 19 Sep 2025).

A transformation is defined as $M := \left\lceil \frac{t}{T} M \right\rfloor,$8, and a transformation of intensity $M := \left\lceil \frac{t}{T} M \right\rfloor,$9 is the composition

$T$0

The central theorem states that for two transformations $T$1 and $T$2, and for $T$3, the composed transformations satisfy inequalities such as

$T$4

and

$T$5

with analogous bounds for the reversed composition $T$6 (Tran et al., 19 Sep 2025).

The proof uses the data processing inequality together with the Markov chains

$T$7

and

$T$8

The corollary extends this to arbitrary composition length: $T$9 In the paper’s interpretation, adding more transformations monotonically improves unlearning effectiveness (Tran et al., 19 Sep 2025).

This theorem provides the conceptual backbone for the framework. It links transformation composition to a formal contraction of information about the forget set, thereby justifying the training-time use of progressive and repeated transformations rather than treating them as conventional augmentation alone.

4. Unlearning phase and plug-and-play integration with existing FU methods

The unlearning phase in ToFU is intentionally simple. When a removal request occurs, the model is trained only on the retain set, for $t$0 epochs and with learning rate $t$1, using the update

$t$2

This is described as a lightweight fine-tuning operation rather than a separate, heavy unlearning procedure (Tran et al., 19 Sep 2025).

The framework is explicitly characterized as plug-and-play. It is not tied to a specific FU algorithm; rather, it modifies the training stage so that the resulting model is easier to unlearn by a downstream FU mechanism. The reported integrations are with FedEraser, FedPGD, and FedAda, with average improvements in overall FU performance of $t$3, $t$4, and $t$5, respectively (Tran et al., 19 Sep 2025).

This architectural role matters for how ToFU should be classified. It is neither merely a training heuristic nor a complete replacement for federated unlearning algorithms. Instead, it is a training-time enhancement that is supposed to reduce memorization and thereby simplify later forgetting. A plausible implication is that ToFU can be layered atop heterogeneous post-hoc FU mechanisms without requiring them to share the same deletion objective or optimization geometry.

5. Empirical evaluation, metrics, and ablations

The reported experiments use CIFAR-10, CIFAR-100, and MUFAC. Client data are sampled from a Dirichlet distribution,

$t$6

with $t$7. For CIFAR-10 and CIFAR-100, forgetting samples are randomly selected from clients at rates of 10% for client 1, 30% for client 3, 60% for client 6, and 90% for client 9, while MUFAC uses the predefined forget set from the original benchmark (Tran et al., 19 Sep 2025).

The implementation details are fixed: ResNet-18, Schedule-free-AdamW, batch size 128, learning rate $t$8, local training for 10 epochs, 50 federated learning rounds, 10 clients, 20 unlearning rounds, and 5 local epochs in unlearning. The reported hardware and software are 2 NVIDIA RTX 4090 GPUs, AMD EPYC 9654P CPU, 192 GB RAM, PyTorch, and the Flower FL framework. In addition, 10% of each client dataset is reserved for validation (Tran et al., 19 Sep 2025).

The evaluation reports Test Accuracy, Retain Accuracy, Membership Inference Attack Efficacy, and an Overall metric. LiRA is used as the membership inference attack, and shadow models are built using global model parameters from the final 5 communication rounds of federated learning training (Tran et al., 19 Sep 2025).

The main overall results are summarized below.

Dataset	ToFU Overall	Comparator Overalls
CIFAR-10	0.6712	FedEraser 0.6450; FedPGD 0.6565; FedAda 0.6466; Exact unlearning 0.6874
CIFAR-100	0.5798	FedEraser 0.5046; FedPGD 0.4520; FedAda 0.5117; Exact unlearning 0.5106
MUFAC	0.7657	FedEraser 0.7027; FedPGD 0.6988; FedAda 0.6564; Exact unlearning 0.6473

Across these datasets, the paper states that ToFU outperforms the listed FU baselines on Overall score and is comparable to exact unlearning in behavior. On CIFAR-10, for example, ToFU reports Test 0.7943, Retain 0.8955, MIA 0.3239, and Overall 0.6712; on MUFAC, the reported Overall is 0.7657, above the reported exact-unlearning Overall of 0.6473 (Tran et al., 19 Sep 2025).

The privacy analysis includes the claim that ToFU narrows the loss distribution gap between forgotten and unseen samples, reduces statistical divergence, and approaches the gold-standard exact unlearning distributional behavior. A figure based on Kolmogorov–Smirnov statistics is reported as showing that ToFU’s output distributions on test versus forget sets are much closer than those of competing FU methods (Tran et al., 19 Sep 2025).

The runtime results report a mean running time of 771.2 seconds and a 24–28% reduction over FedPGD, FedAda, and FedEraser. The explanation given is that ToFU shifts work into the original training stage so that unlearning later becomes a lightweight fine-tuning procedure (Tran et al., 19 Sep 2025).

The ablation study identifies three core components: SDT, TIR, and PT. The full model achieves Test Acc 0.5032, Retain Acc 0.7802, MIA Eff. 0.4560, and Overall 0.5798. The appendix further reports that $t$9 gives reasonable performance, small positive $M$0 gives slight improvement, negative $M$1 causes catastrophic degradation, and larger positive $M$2 still performs well but with diminishing returns. Training cost increases with more transformations: from 1277s with 0 transforms to 2027s with 8 transforms, corresponding to a worst-case training overhead of 58.7% and an average-case overhead of 29.6% for 4 transforms (Tran et al., 19 Sep 2025).

6. Related transformation-guided methods, benchmark ambiguity, and limitations

ToFU belongs to a broader transformation-guided current in federated unlearning, but it is not the only formulation of that idea. A closely related example is Federated Unlearning via Class-aware Representation Transformation (FUCRT), which treats class-level forgetting as moving the representations of the forgotten class into a suitable remaining-class region in feature space. FUCRT uses a transformation class selection strategy and dual class-aware contrastive learning, with a local objective

$M$3

and reports complete erasure of unlearning classes together with strong remaining-class utility in IID and Non-IID settings (Guo et al., 2024). The comparison is informative because both methods replace purely destructive forgetting with representation- or invariance-oriented transformation, but ToFU does so at training time for general FU, whereas FUCRT is organized around class-aware representation migration.

A separate source of ambiguity is the term “TOFU” in large-language-model unlearning. In “Hierarchical Federated Unlearning for LLMs,” TOFU denotes the benchmark “Task of Fictitious Unlearning for LLMs” from Maini et al. (2024), while the proposed algorithm in that paper is FULM. That paper states explicitly that FULM is not the same as a method called “Transformation-guided Federated Unlearning,” and that its relation to TOFU is benchmark-level rather than algorithmic (Zhong et al., 19 Oct 2025). This distinction is important because the same acronym appears in two different roles across the literature.

The limitations reported for ToFU are concrete. First, it increases federated learning training time because transformed samples require extra computation. Second, too many or too strong transformations can remove task-relevant information and harm utility, which is why the framework relies on sample-dependent transformation intensity, progressive scheduling, and the invariant regularizer. Third, the MUFAC benchmark shows a small utility drop relative to vanilla FedAvg, with FedAvg at 0.8451 and FedAvg + ToFU at 0.8165; the paper attributes this to real facial images with lighting variation and related conditions being less amenable to the specific transformations used. Fourth, the method is not claimed to be universally optimal for every transformation family, and RandAugment is suggested as a way to improve generalization across transformation families (Tran et al., 19 Sep 2025).

The reported edge case in which no clients request unlearning further clarifies the trade-off. On CIFAR-10 and CIFAR-100, ToFU slightly improves or maintains performance versus FedAvg, whereas on MUFAC there is a mild degradation. This suggests that ToFU is not intrinsically an unlearning-only procedure; it is a modified training regime whose benefits depend on how well the chosen transformation family matches the data geometry (Tran et al., 19 Sep 2025).