Reminiscence Attack (ReA) in Machine Unlearning

• Reminiscence Attack (ReA) is a privacy attack against approximate machine unlearning that exploits residual loss landscapes to infer forgotten data.
• The attack employs targeted fine-tuning to measure re-learning speed as a membership signal for both sample-wise and class-wise unlearning.
• Empirical findings show ReA achieves up to 1.90× higher accuracy than baselines, exposing vulnerabilities even when models appear retrained.

Reminiscence Attack (ReA) is a privacy attack against approximate machine unlearning (AMU) that exploits persistent residuals of unlearned data in a model’s internal loss landscape rather than relying only on output-space leakage. In the formulation introduced in "Reminiscence Attack on Residuals: Exploiting Approximate Machine Unlearning for Privacy" (Xiao et al., 28 Jul 2025), ReA converts those residuals into a membership signal by re-learning a candidate forgotten target in a controlled way and measuring how quickly the model re-aligns with it. The attack is defined for sample-wise membership inference, class-wise membership inference, and concept unlearning in generation models, and is motivated by the observation that AMU can appear successful under ordinary utility or output-distribution checks while still preserving latent traces of the supposedly removed data (Xiao et al., 28 Jul 2025).

1. Problem setting and threat model

ReA is situated in the machine unlearning setting where a trained model is modified so that specific data are removed to uphold the right to be forgotten. The attack specifically targets approximate machine unlearning rather than exact retraining. Exact unlearning would retrain from scratch on $\mathcal{D}_r = \mathcal{D}\setminus \mathcal{D}_u$, thereby eliminating all traces of $\mathcal{D}_u$. By contrast, AMU modifies an already-trained model to approximate that retrained model, and the paper argues that this approximation is often imperfect in privacy terms even when it appears adequate in utility terms (Xiao et al., 28 Jul 2025).

The adversary is assumed to know the task scope and to have either white-box access to the unlearned model $F(\cdot;\theta^u)$ or black-box API access, in which case the attacker may first extract a substitute model. The attack is evaluated in two primary privacy settings. In sample-wise membership inference, the goal is to determine whether a specific sample was among the unlearned points. In class-wise membership inference, the goal is to determine whether a whole class was unlearned. The paper further extends the setting to concept unlearning in generation models, where the target is an identity or concept rather than a class (Xiao et al., 28 Jul 2025).

A central claim is that approximate unlearning introduces an attack surface beyond conventional output-based leakage. The model may match a retrained model at the level of outputs or accuracy while still preserving internal traces of the forgotten data. This is described as pseudo-convergence: the model’s external behavior appears close to retrained, but the optimization landscape around the forgotten data still differs. The paper characterizes this as a shallow form of forgetting, especially near output logits or final layers, with deeper representational traces remaining (Xiao et al., 28 Jul 2025).

2. Residuals, pseudo-convergence, and the leakage signal

The paper defines a residual for an unlearned set $\mathcal{D}_u$ as

$$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$

where $\mathcal{D}_\text{non-tr}$ is the non-training counterpart to $\mathcal{D}_u$, and $d$ is a task-specific discrepancy measure (Xiao et al., 28 Jul 2025).

This formulation is used to distinguish ReA from prior attacks that mainly inspect logits, confidence values, or loss/probability behavior. The paper contrasts conventional output-based leakage with residual leakage in the loss landscape. In the latter case, the forgotten data still lies unusually close to favorable optimization basins, so targeted fine-tuning can reactivate it faster than non-members. The privacy signal exploited by ReA is therefore not simply whether the model currently outputs a revealing score, but whether it can be made to “resonate” with a candidate target more quickly than would be expected for non-training data (Xiao et al., 28 Jul 2025).

The paper reports strong empirical correlation between residual metrics and attack success. For class-wise cases, representation misalignment correlates with ReA accuracy with Pearson $=0.88$; for sample-wise cases, recovered unlearning accuracy $\Delta\mathrm{UA}$ correlates with ReA accuracy with Pearson $\mathcal{D}_u$0. These correlations are presented as evidence that ReA probes optimization geometry left behind by approximate unlearning rather than merely repackaging output-based membership inference (Xiao et al., 28 Jul 2025).

This also clarifies a common misconception: apparent agreement with retraining under ordinary utility metrics does not imply privacy equivalence to retraining. In the ReA threat model, privacy failure arises precisely when the model is sufficiently close in outputs to look unlearned, yet remains sufficiently close in representation space or loss-landscape geometry to support rapid re-learning of forgotten content (Xiao et al., 28 Jul 2025).

3. Attack construction and operational variants

The core mechanism of ReA is targeted fine-tuning. The paper refers to this process as reminiscence: reintroducing a candidate target into fine-tuning and measuring how quickly the model aligns with it. If the model converges unusually fast on that target, the attack treats the target as more likely to have been originally unlearned. Faster convergence is interpreted as evidence that the model remains close to a favorable basin associated with the forgotten data (Xiao et al., 28 Jul 2025).

In the class-wise variant, the attacker seeks to infer whether a candidate OOD class $\mathcal{D}_u$1 was unlearned. The attack fine-tunes the unlearned model on the candidate class and measures the number of iterations needed to reach a convergence threshold. This quantity is the resonance index:

• $\mathcal{D}_u$2 = number of iterations needed to reach a convergence threshold on $\mathcal{D}_u$3.

The corresponding confidence score is

$\mathcal{D}_u$4

where $\mathcal{D}_u$5 is the maximum number of iterations. Smaller $\mathcal{D}_u$6 therefore implies faster resonance and higher confidence that the class is unlearned. Because resonance can depend strongly on the learning rate, the paper stabilizes the attack by averaging over multiple learning rates:

$\mathcal{D}_u$7

The appendix gives the class-wise fine-tuning objective as

$\mathcal{D}_u$8

followed by SGD updates

$\mathcal{D}_u$9

The resonance index is recorded when class accuracy exceeds the convergence threshold, for example 75% (Xiao et al., 28 Jul 2025).

In the sample-wise variant, the attack proceeds in two phases. First is the reminiscence phase, in which the model is fine-tuned on data approximating the retained set. Second is the MIA phase, in which the attacker applies MIA-LiRA. If the retained set is unknown, the paper proposes constructing a pseudo-retain set by scoring samples using MIA-LiRA, selecting the top-$F(\cdot;\theta^u)$0 high-confidence positives as pseudo-retained data, and fine-tuning on them. The reminiscence objective is

$F(\cdot;\theta^u)$1

followed by

$F(\cdot;\theta^u)$2

After this tuning, MIA-LiRA is run again on the updated model. If the unlearned samples recover more strongly than test samples, membership confidence increases (Xiao et al., 28 Jul 2025).

The paper formalizes the output decisions as

$F(\cdot;\theta^u)$3

for sample-wise inference, and

$F(\cdot;\theta^u)$4

for class-wise inference (Xiao et al., 28 Jul 2025).

4. Empirical findings and attack regimes

The empirical evaluation covers 11 AMU methods plus its own defense, OUR, across classification and generation settings. The principal reported result is that ReA achieves up to $F(\cdot;\theta^u)$5 higher attack accuracy than prior attacks for class-wise membership inference and up to $F(\cdot;\theta^u)$6 higher attack accuracy for sample-wise membership inference. These gains are reported relative to existing baselines, especially MIA-LiRA and MIA-UP, and are attributed to ReA’s use of residuals in the optimization landscape rather than output-space leakage alone (Xiao et al., 28 Jul 2025).

For class-wise unlearning, the paper reports over 81% accuracy against many class-wise AMU methods and, in some settings, attacks near 90%. The strongest vulnerabilities appear in methods such as random label / SalUn / RUM, influence unlearning, and boundary unlearning, especially when the method mainly modifies shallow or output-layer behavior but does not remove deep representations (Xiao et al., 28 Jul 2025).

For sample-wise unlearning, ReA still produces substantial leakage, with over 55% on CIFAR10 in some cases and over 70% on CIFAR20 in some cases. The attack is described as especially effective against methods that do not use strong gradient-based forgetting or that leave the model in a pseudo-converged state (Xiao et al., 28 Jul 2025).

The paper also reports AUC gains: ReA achieves up to $F(\cdot;\theta^u)$7 and $F(\cdot;\theta^u)$8 higher AUC than the baselines in class-wise and sample-wise settings, respectively. The attack remains effective in black-box settings via model extraction, indicating that the residuals are transferable enough to remain exploitable after extraction of a substitute model (Xiao et al., 28 Jul 2025).

These findings are tied to specific operating conditions. The paper states that the gains are strongest when approximate unlearning leaves deep residual traces, when the model is pseudo-converged, and when targeted fine-tuning can quickly reactivate forgotten information. A plausible implication is that ReA is less a generic confidence-based MIA than a diagnostic of incomplete removal in the geometry of the learned model itself (Xiao et al., 28 Jul 2025).

5. Defense and mitigation: Orthogonal Unlearning Replay

To mitigate the privacy risk exposed by ReA, the paper proposes Orthogonal Unlearning Replay (OUR), a dual-phase approximate unlearning framework designed to address two residual sources: class-wise residuals from deep hidden-layer traces and sample-wise residuals from incomplete convergence or pseudo-convergence (Xiao et al., 28 Jul 2025).

The first phase, Orthogonal Unlearning, aims to remove deep-layer traces of the unlearned data by enforcing orthogonality between hidden representations of unlearned samples and their original representations. The orthogonal unlearning loss is given as

$F(\cdot;\theta^u)$9

The appendix also presents a more explicit equivalent form,

$\mathcal{D}_u$0

emphasizing feature-space separation across multiple hidden layers (Xiao et al., 28 Jul 2025).

The second phase, Replay, fine-tunes on the retained dataset $\mathcal{D}_u$1 to restore utility and force real convergence. The paper states that this matters because orthogonal unlearning alone may degrade utility, and because a pseudo-converged model can still leak residual information. OUR is therefore designed not merely to suppress a direct membership signal, but to separate forgetting from utility restoration and to prevent the model from appearing retrained while preserving unlearned residuals (Xiao et al., 28 Jul 2025).

To prevent pseudo-convergence and collapse, the method controls parameter movement with a stability bound

$\mathcal{D}_u$2

The paper also reports a Tug-of-War unlearning metric

$\mathcal{D}_u$3

This is presented as part of the defense-side evaluation machinery (Xiao et al., 28 Jul 2025).

Experimentally, OUR is reported to maintain high unlearning efficacy while reducing the adaptive privacy attack accuracy to nearly random guess, at a runtime cost of about 2–12% of full retraining. The paper states that class-wise and sample-wise MIA accuracy are reduced to about the retraining baseline or approach random-level leakage (Xiao et al., 28 Jul 2025). This positions OUR as a mitigation specifically tailored to the residual-induced leakage channel identified by ReA rather than to output confidence leakage alone.

6. Relation to reconstruction-attack theory

Although "Data Reconstruction: When You See It and When You Don’t" does not use the term ReA, it provides a formal framework that is directly relevant to evaluating reconstruction-style attacks of this kind (Cohen et al., 2024). That paper argues that a single universal definition of reconstruction attack may not exist and instead proposes to sandwich the notion by addressing two complementary questions: when a mechanism is protected against reconstruction and when an attack convincingly demonstrates lack of protection (Cohen et al., 2024).

The protection notion is Narcissus resiliency. Let $\mathcal{D}_u$4 be a data domain, $\mathcal{D}_u$5 a family of distributions over datasets from $\mathcal{D}_u$6, and

$\mathcal{D}_u$7

a relation where $\mathcal{D}_u$8 means that $\mathcal{D}_u$9 is a valid reconstruction of $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$0. An algorithm $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$1 is $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$2-$$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$3-Narcissus-resilient if for every $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$4 and every attacker $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$5,

$$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$6

The attacker is thus compared against itself in a baseline world where it receives no distinguishing information about whether evaluation is against the real dataset $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$7 or a fresh independent dataset $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$8 (Cohen et al., 2024).

The success notion is based on Kolmogorov complexity. The paper argues that merely outputting an apparently meaningful artifact is insufficient, because the artifact might be generated by a short generic program without access to the model. It defines Kolmogorov complexity relative to a programming language $$R(\mathcal{D}_u):=\mathbb{E}\left[d(F(\mathcal{D}_u;\theta^u), F(\mathcal{D}_\text{non-tr};\theta^u))\right],$$9 as

$\mathcal{D}_\text{non-tr}$0

and states that $\mathcal{D}_\text{non-tr}$1 is $\mathcal{D}_\text{non-tr}$2-extractable from $\mathcal{D}_\text{non-tr}$3 if there exists an $\mathcal{D}_\text{non-tr}$4-program $\mathcal{D}_\text{non-tr}$5 such that $\mathcal{D}_\text{non-tr}$6 outputs $\mathcal{D}_\text{non-tr}$7 with $\mathcal{D}_\text{non-tr}$8 and

$\mathcal{D}_\text{non-tr}$9

The paper’s informal quality measure is

$\mathcal{D}_u$0

A reconstruction attack is therefore convincing only if it produces a valid reconstruction under a domain-appropriate relation $\mathcal{D}_u$1 and the target has high complexity relative to the attack description length (Cohen et al., 2024).

This suggests that ReA can be interpreted as a reconstruction-style privacy attack whose success should not be judged only by whether it outputs a membership bit. A plausible implication is that a rigorous evaluation of ReA-like leakage would require specifying a relation $\mathcal{D}_u$2 for what counts as successful recovery of unlearned information, then asking whether the observed signal exceeds what the attacker could achieve without genuine information leakage. In that sense, ReA’s emphasis on residuals and self-revealing re-learning speed aligns with the broader view that a convincing privacy attack must beat a no-information baseline rather than merely produce a plausible-looking artifact (Cohen et al., 2024).