Influence-Guided Machine Unlearning

• Influence-guided machine unlearning (IMU) is a family of approximate methods that estimate training data influence to selectively erase its effects without retraining from scratch.
• These methods employ diverse signals—from Hessian-based sensitivity to gradient geometry and neuron attribution—to identify and neutralize harmful data contributions.
• IMU techniques balance unlearning completeness with computational efficiency, proving effective in domains like vision, language, and corrective poisoning.

Influence-guided machine unlearning denotes a family of approximate machine unlearning methods in which the model uses an estimate or proxy of how training data affect parameters, losses, predictions, or update directions to decide what to forget and how strongly to forget it. In the recent literature, the term appears both as a broad systems-level description of pipelines that estimate harmful influence, identify responsible points, remove or neutralize their effect, and validate forgetting, and as the title of a retain-data-free method that conducts MU using only the forget set and dynamically allocates unlearning intensities across data points based on their influences (Xu et al., 2023, Li et al., 2024, Fan et al., 3 Aug 2025).

1. Taxonomic position and scope

Within the standard taxonomy of machine unlearning, influence-guided methods belong primarily to approximate unlearning rather than exact unlearning. The survey literature defines machine unlearning as the selective removal of specific training data points and their influence from an already trained model, so that the updated model behaves like one that was never trained on that data. In that same framework, approximate unlearning is characterized by a tradeoff between unlearning completeness and efficiency, and influence-function-based unlearning is presented as a major branch alongside re-optimization and gradient-update methods (Xu et al., 2023).

The same survey also gives a generic four-step template for approximate unlearning: Computation of Influence, Adjustment of Model Parameters, Addition of Noise (Optional), and Validation of Updated Model. That template closely matches later uses of the phrase “Influence-guided Machine Unlearning,” especially in settings where the crucial difficulty is not merely updating the model after a forget set is known, but inferring which data points or internal units should be forgotten at all (Xu et al., 2023, Li et al., 2024).

Contemporary usage is broader than strict Koh–Liang influence functions. Some papers remain close to classical second-order sensitivity analysis, while others explicitly position themselves as gradient-influence-aware, first-order influence approximation, localized influence-guided unlearning, or a practical proxy for influence-guided weighting. This broader usage does not erase the classical definition; it enlarges it from Hessian-inverse sample attribution to a design space of influence signals at the sample, subset, parameter, neuron, or gradient-geometry level (Asif et al., 26 Sep 2025, Liu et al., 31 Jul 2025, Agarwal et al., 9 Oct 2025, Li et al., 30 Jul 2025).

2. Formal problem statements and canonical pipeline

The core problem is usually posed by splitting the training data into a forget set and a retain set. In a standard formulation, the data are partitioned as

$\mathcal{D} = \mathcal{D}_r \cup \mathcal{D}_f,$

with empirical losses

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$

The operational objective is to remove the influence of $\mathcal{D}_f$ while preserving performance on $\mathcal{D}_r$, usually without retraining from scratch (Asif et al., 26 Sep 2025).

The classical influence-function starting point writes the trained parameters as

$$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$

with standard influence

$$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$

and target-loss influence

$$\mathcal{I}(z, z_t) \coloneqq - \nabla_{\theta} \ell \big(f_{\theta^\ast}(z_t)\big)^\top H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big).$$

Here, the central object is the local second-order sensitivity of parameters or losses to the presence of a training point (Fan et al., 3 Aug 2025).

A systems-level IMU pipeline is especially clear in corrective poisoning settings. There, the paper on “Delta-Influence” reconstructs a canonical architecture as

$$\text{suspicious behavior} \rightarrow \text{influence-guided data attribution} \rightarrow \text{deletion candidate set} \rightarrow \text{unlearning backend} \rightarrow \text{behavioral verification}.$$

In that formulation, influence guidance mainly fills the front end of the pipeline: identify which points should be forgotten so that retraining or an approximate backend can remove their effect (Li et al., 2024).

The retain-data-free method “IMU: Influence-guided Machine Unlearning” narrows the setting further. It assumes only the trained model and the forget set, decomposes the model into a feature extractor and a classifier, and performs influence estimation at the classifier layer rather than over the full network. That design choice is motivated by the instability and cost of full-model influence computation in deep non-convex models (Fan et al., 3 Aug 2025).

3. Influence signals used to guide forgetting

The most classical signal is curvature-corrected sample influence. In the retain-data-free IMU method, the model is decomposed as

$f_\theta = h_{\theta_c} \circ \phi_{\theta_e},$

and classifier-level influence is estimated as

$$\mathcal{I}^c(x, D_f) = - \nabla_{\theta_c} \ell\big(h_{\theta_c^\ast}(\mathbf{Z}_f)\big)^\top H_{\theta_c^\ast}^{-1} \nabla_{\theta_c} \ell\big(h_{\theta_c^\ast}(z)\big),$$

where $$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$0. Unlearning then uses an influence-guided loss

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$1

with normalized influence weights $$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$2. The stated purpose is dynamic allocation of unlearning intensity across forget points rather than uniform gradient ascent (Fan et al., 3 Aug 2025).

A different signal is comparative influence under interventions. “Delta-Influence” does not ask which training points currently have large influence on a poisoned prediction; it asks which points undergo a large negative shift in influence when the suspicious test input is transformed so that the poison-specific linkage is broken. Its core quantity is

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$3

and the method flags points when sufficiently many transformations produce a significant negative shift: $$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$4 The paper terms the observed pattern influence collapse: poisoned points undergo a collapse of influence under transformation, while clean points do not display the same systematic collapse (Li et al., 2024).

A third signal family is gradient geometry. OFMU introduces

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$5

with cosine similarity as the conflict term, and optimizes a penalty objective

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$6

This is explicitly presented as gradient-influence-aware rather than classical influence-function unlearning: the guidance signal is the interaction between forget and retain gradients, stabilized by a stationarity penalty involving Hessian-vector products (Asif et al., 26 Sep 2025).

A fourth signal family is localized internal attribution. SIMU identifies “critical neurons” in transformer MLP layers using the attribution score

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$7

then selects neurons with

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$8

The subsequent second-order unlearning update is masked so that only the selected MLP neurons and attention projections are updated. This shifts influence guidance from sample-level attribution to neuron-level localization (Agarwal et al., 9 Oct 2025).

Loss-based weighting occupies a weaker position. LoReUn does not estimate influence functions, Hessian inverses, parameter perturbations, or gradient similarity scores; instead it weights forget examples by

$$\mathcal{L}_r(\theta) = \frac{1}{|\mathcal{D}_r|} \sum_{(x, y) \in \mathcal{D}_r} \ell(f_\theta(x), y), \quad \mathcal{L}_f(\theta) = \frac{1}{|\mathcal{D}_f|} \sum_{(x, y) \in \mathcal{D}_f} \ell(f_\theta(x), y).$$9

with batch normalization of the weights. The paper explicitly characterizes this as a practical proxy rather than a direct influence method. This suggests that, in some settings, per-example difficulty may be treated as a cheap surrogate for residual influence, but that interpretation remains approximate (Li et al., 30 Jul 2025).

4. Representative algorithmic families

The contemporary IMU landscape is heterogeneous. Some methods estimate influence directly, some localize what to forget before delegating forgetting to a backend, and others replace explicit influence with cheaper geometric or attributional proxies.

Approach	Guidance signal	Role in unlearning
IMU	Classifier-level influence on $\mathcal{D}_f$0	Retain-data-free weighted forgetting
Delta-Influence	Transformation-induced influence collapse	Corrective poison localization
IAU	First-order surrogate for $\mathcal{D}_f$1	Efficient one-step influence approximation
OFMU	Gradient similarity and stationarity penalty	Bi-level gradient-influence-aware unlearning
SIMU	Neuron attribution plus masked second-order update	Localized LLM unlearning
CUFG / CUP	Forgetting-gradient alignment / conflict-free gradients	Stability-first gradient-influence guidance
LoReUn	Per-example loss weighting	Proxy for influence-guided weighting

The retain-data-free IMU method is the narrowest use of the term. It employs gradient ascent and introduces dynamic allocation of unlearning intensities across different data points based on their influences, while using only the forget set and no retain data, retain-set statistics, auxiliary data, or reference model for utility recovery (Fan et al., 3 Aug 2025).

“Delta-Influence” is best understood not as a standalone new unlearning operator, but as an influence-guided localization mechanism for corrective machine unlearning in poisoning and backdoor settings. Its novelty lies overwhelmingly in the identification stage, not in the mechanism by which the model is updated after identification. Once the suspicious set is localized, the paper delegates forgetting to Exact Unlearning / retraining from scratch without $\mathcal{D}_f$2, Catastrophic Forgetting, SSD, SCRUB, or BadT (Li et al., 2024).

IAU keeps the spirit of influence-guided removal while avoiding explicit Hessian or inverse-Hessian calculations. Starting from the deletion approximation

$\mathcal{D}_f$3

it proposes the first-order batch update

$\mathcal{D}_f$4

The paper therefore positions itself as a computationally cheap, influence-motivated alternative to Hessian-based unlearning (Liu et al., 31 Jul 2025).

OFMU, CUP, and CUFG enlarge IMU toward gradient-conflict control. OFMU treats forgetting as a hierarchical optimization problem; CUP reframes machine unlearning as multi-objective optimization with a pivoting mechanism over conflict-free anchors; CUFG uses the forget set’s mean gradient as a first-order online proxy for residual influence and corrects retain updates only when gradient alignment indicates risk. None of these papers present themselves as strict influence-function methods, but all explicitly connect forgetting quality to the directional effect of forget data on the update path (Asif et al., 26 Sep 2025, Hwang et al., 22 Oct 2025, Miao et al., 18 Sep 2025).

SIMU extends the family toward internal memory localization. It does not compute $\mathcal{D}_f$5 explicitly; instead it combines neuron attribution and a Sophia-based second-order optimizer with masking. NatMU occupies a different position: it is relevant conceptually because it asks which retained information should guide forgetting, but it is not an influence-function method and does not estimate per-sample influence explicitly (Agarwal et al., 9 Oct 2025, He et al., 2024).

5. Empirical behavior across tasks and modalities

The narrow retain-data-free IMU method reports strong performance across vision and language tasks. On CIFAR-10 class-wise unlearning, it reports

$\mathcal{D}_f$6

and on the CIFAR-100 subclass-wise setting across all 20 superclasses it reports

$\mathcal{D}_f$7

An especially revealing ablation shows that only $\mathcal{D}_f$8 of the most influential forget samples are sufficient to induce forgetting while improving utility and reducing runtime: with top-ranked forget ratio $\mathcal{D}_f$9, the paper reports $\mathcal{D}_r$0, $\mathcal{D}_r$1, and $\mathcal{D}_r$2 on CIFAR-10 class-wise unlearning (Fan et al., 3 Aug 2025).

Corrective IMU in poisoning settings produces a different empirical pattern. With Exact Unlearning as the backend, “Delta-Influence” is successful in all $\mathcal{D}_r$3 CIFAR attack-dataset combinations, defined as poison success rate below $\mathcal{D}_r$4, whereas Activation Clustering and naive EK-FAC succeed in only $\mathcal{D}_r$5 cases. In Witches’ Brew, the paper further reports that removing the detected subset—which is only $\mathcal{D}_r$6 of poisons on CIFAR-10 and $\mathcal{D}_r$7 on CIFAR-100—drives poison success rate to $\mathcal{D}_r$8, while removing the complement set of remaining poisons leaves poison success rate at $\mathcal{D}_r$9. This establishes a central corrective-unlearning point: complete identification of every poisoned point may be unnecessary if the causally critical subset is found (Li et al., 2024).

Gradient-influence-aware methods extend the evidence to LLMs. On TOFU, OFMU reports, for LLaMA-2 forget05,

$$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$0

and on WMDP it achieves $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$1 Bio, $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$2 Cyber, and $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$3 MMLU. The same paper interprets these results as evidence that similarity-aware penalties can preserve domain-specific knowledge while forgetting targeted content (Asif et al., 26 Sep 2025).

Localized LLM unlearning shows a similar trend. On LUME with LLaMA2-7B, SIMU-GradDiff reports aggregate $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$4, compared with $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$5 for SO-GradDiff and $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$6 for FO-GradDiff. On TOFU with OLMo-1B, SIMU-GradDiff reports aggregate $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$7, compared with $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$8 for SO-GradDiff. The paper attributes these gains to selective masking: second-order updates are confined to the model components judged most responsible for the forget set (Agarwal et al., 9 Oct 2025).

Proxy-guided weighting has also shown strong results outside strict influence-function settings. On Imagenette Stable Diffusion class-wise forgetting, LoReUn-d reports $$\theta^\ast \coloneqq \arg\min_{\theta}\frac{1}{N}\sum_{i=1}^N \ell(x_i, y_i; \theta),$$9 with $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$0, while LoReUn-s reports $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$1 with $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$2, and SalUn reports $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$3 with $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$4. The paper interprets this as evidence that dynamic loss-based reweighting can narrow the gap between approximate unlearning and retraining in generative settings (Li et al., 30 Jul 2025).

6. Limitations, misconceptions, and open directions

A persistent limitation is that most IMU methods are approximate rather than exact or certified. “Delta-Influence” explicitly states that it is aligned with corrective unlearning rather than privacy-style deletion; OFMU guarantees convergence of an optimization objective rather than exact removal of $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$5; IAU provides an approximate and empirical removal guarantee rather than a certified one; SIMU provides no certified removal guarantee; and CUFG is evaluated by its gap to Retrain rather than by a formal deletion theorem (Li et al., 2024, Asif et al., 26 Sep 2025, Liu et al., 31 Jul 2025, Agarwal et al., 9 Oct 2025, Miao et al., 18 Sep 2025).

A second limitation is access dependence. The retain-data-free IMU method requires only the forget set, but Delta-Influence requires at least one suspicious test example and access to the original training set, OFMU requires the retain set, SIMU requires forget and retain data, and RePAIR requires a retain buffer and a refusal reference set even though it is training-free. Consequently, “retain-data-free” is method-specific, not a property of IMU as a whole (Fan et al., 3 Aug 2025, Li et al., 2024, Asif et al., 26 Sep 2025, Agarwal et al., 9 Oct 2025, Rachapudi et al., 14 Apr 2026).

A third limitation is computational and numerical fragility. Classical influence methods inherit the cost of Hessian inversion and the brittleness of local second-order approximations in deep non-convex networks; the retain-data-free IMU method reduces this by restricting influence estimation to the final classifier layer, but still notes that computational cost scales with the number of forget points because influence must be estimated for each forgetting sample. Delta-Influence multiplies influence computation by the number of transformations, and IAU’s central approximation discards curvature information entirely (Fan et al., 3 Aug 2025, Li et al., 2024, Liu et al., 31 Jul 2025).

A common misconception concerns terminology. In the paper “RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair,” IMU denotes Interactive Machine Unlearning, not Influence-guided Machine Unlearning. That framework is prompt-aware and activation-guided, using a refusal steering vector and closed-form pseudoinverse updates, and it is explicitly not based on influence functions or influence-guided sample attribution. The acronym is therefore overloaded in the recent literature (Rachapudi et al., 14 Apr 2026).

The open problems identified across papers are relatively consistent. The survey literature highlights unlearning for non-convex models, user-specified granularity, stronger privacy assurance, quantitative metrics that jointly capture removal and retention, and influence estimation for diverse data structures (Xu et al., 2023). More papers add counterfactual probing for localization, mini-batch influence estimation, continual unlearning, adaptive penalty schedules, alternative similarity measures, and layer-neuron combined granularity as explicit future directions (Li et al., 2024, Fan et al., 3 Aug 2025, Asif et al., 26 Sep 2025, Agarwal et al., 9 Oct 2025). This suggests that the field is moving from a narrow question—how to approximate $$\mathcal{I}(z) \coloneqq -H_{\theta^\ast}^{-1} \nabla_{\theta} \ell \big(f_{\theta^\ast}(z) \big),$$6—toward a broader one: which influence signal is reliable enough, local enough, and cheap enough to guide forgetting without destroying retained behavior.