Concept Unlearning in AI Models

• Concept unlearning is the process of selectively erasing specific semantic concepts from a trained model while maintaining its overall functionality.
• It employs fine-grained semantic localization and targeted interventions across diffusion models, VLMs, LVLMs, and LLMs to balance forgetting with retention.
• Recent methods leverage sparse feature analysis, parameter editing, and distributional alignment to achieve measurable improvements in unlearning precision and utility preservation.

Searching arXiv for papers on concept unlearning across diffusion models, VLMs, and LLMs. arxiv_search.search(query="concept unlearning diffusion models vision-LLMs LLMs sparse autoencoders machine unlearning", max_results=10)

Searching for more targeted recent work on concept-level unlearning and continual unlearning. arxiv_search.search(query="continual unlearning concepts vision-LLMs diffusion unlearning CARE sparse autoencoders", max_results=10)

Concept unlearning is the selective removal of a target concept from a trained model while preserving unrelated capabilities and model utility. In current work, the target may be a visual object, artistic style, celebrity, nudity, emotion, color, material, a class, an image-instruction pair, or an entity together with its relations in a knowledge graph. The central difficulty is that concepts are usually entangled with benign context, co-occurring semantics, and shared representational pathways, so effective forgetting and utility preservation are fundamentally conflicting objectives (Zhao et al., 2024, Khalafi et al., 29 May 2026). Across diffusion models, vision-LLMs (VLMs), large vision-LLMs (LVLMs), and LLMs, concept unlearning has therefore become a problem of fine-grained semantic localization, selective intervention, and retention-aware optimization rather than simple deletion of examples or tokens (Li et al., 16 Mar 2025, Lin et al., 14 May 2026, Yamashita et al., 19 Sep 2025).

1. Problem statement and formal scope

In machine unlearning, the goal is to remove the effect of a subset of training data from a trained model without damaging utility; a formal target is the model that would have been obtained by retraining on the surviving data only (Zhao et al., 2024). Concept unlearning extends this objective from example-level deletion to higher-level semantic targets. In diffusion models this includes instances, styles, offensive concepts, and celebrities (Wu et al., 2024). In VLMs, the literature explicitly distinguishes concrete concept unlearning, such as objects and sports scenes, from abstract concept unlearning, such as emotions, colors, and materials (Li et al., 16 Mar 2025). In LLMs, concept unlearning has been formulated as removing forgetting target nodes and associated edges from a knowledge graph representation of model knowledge (Yamashita et al., 19 Sep 2025).

Two standards recur. The first is approximate unlearning, in which the unlearned model is required to be close to a retrained reference model; the paper on unlearning difficulty expresses this with an $(\epsilon,\delta)$-unlearner definition and operationalizes empirical quality with the “Tug-of-War” metric over forget, retain, and test sets (Zhao et al., 2024). The second is exact unlearning in the learning-theoretic sense: “Ticketed Learning-Unlearning Schemes” defines a setting in which, after receiving tickets from examples to be unlearnt and small central state, the unlearning algorithm must output a predictor identical to the one learned from scratch on the surviving examples (Ghazi et al., 2023).

A recurrent benchmark ideal is the “retain-only” model retrained from scratch without the unwanted data. TILDE makes this explicit, treating the retain-only model as the gold standard for benign generation and arguing that practical unlearning should approximate a minimum-deviation conditional distribution under a forgetting constraint rather than merely suppress target outputs (George et al., 7 Jul 2026).

2. What counts as a concept

A major theme in recent work is that a concept is not reducible to a single example, keyword, or neuron. “A Concept is More Than a Word” states that a visual concept is multi-dimensional, can be expressed in diverse textual forms, and often overlap with related concepts in latent space; keyword-only unlearning is therefore described as a brittle point estimate of a richer semantic distribution (Pham et al., 19 Mar 2026). ReCARE sharpens this further by defining benign co-occurring concepts that should be preserved during unlearning as CARE, emphasizing that erasing “nudity” can unintentionally suppress “person,” and analogously that style or object erasure can suppress contextually associated but benign content (Kim et al., 23 Jun 2026).

A second line of work replaces coarse labels with explicit semantic decompositions. ICED constructs a compact task-specific concept vocabulary from the forgetting set using a multimodal LLM, aligns modalities, and decomposes visual representations into sparse, nonnegative combinations of semantic concepts. Its core decomposition is

$$\boldsymbol{w}^* = \arg\min_{\boldsymbol{w}\ge 0} \|C\boldsymbol{w} - \mathbf{z}\|_2^2 + \lambda_{dec}\|\boldsymbol{w}\|_1,$$

so that target concepts can be selectively suppressed while intra-instance non-target semantics are preserved (Lin et al., 14 May 2026). CORE, for continual unlearning in LVLMs, similarly decomposes deletion targets into visual and textual concepts, then uses a concept modulator to identify which concept combinations characterize each forget category (Jin et al., 23 Mar 2026). For class machine unlearning on complex data, concepts are treated as semantic abstractions rather than raw image features or tokens; Post-hoc Concept Bottleneck Models and Integrated Gradients are used to identify class-relevant concepts in images and text (Chang et al., 2024).

A third line localizes concepts through sparse latent structure. SAeUron trains sparse autoencoders (SAEs) on diffusion-model activations and reports that the learned features are sparse and interpretable, often corresponding closely to semantic concepts (Cywiński et al., 29 Jan 2025). SAEParate argues that ordinary SAE reconstruction does not enforce concept-wise separation, then introduces a concept-aware contrastive objective and a GeLU-enhanced encoder to organize latent representations into concept-specific clusters (Kim et al., 12 May 2026). SAUCE transfers the sparse-autoencoder idea to VLMs, using SAEs to capture semantically rich sparse features, identify those most relevant to a target concept, and selectively modify them during inference (Li et al., 16 Mar 2025).

Representational unit	Function in unlearning	Representative papers
Diverse prompt sets and CARE vocabularies	Expand target coverage and preserve benign co-occurrences	(Pham et al., 19 Mar 2026, Kim et al., 23 Jun 2026)
Sparse latent features or concept weights	Isolate concept-aligned internal directions	(Cywiński et al., 29 Jan 2025, Kim et al., 12 May 2026, Lin et al., 14 May 2026, Li et al., 16 Mar 2025)
Concept libraries, graph triplets, bottleneck concepts	Make forget targets explicit and interpretable	(Jin et al., 23 Mar 2026, Yamashita et al., 19 Sep 2025, Chang et al., 2024)

3. Intervention mechanisms

Recent methods differ chiefly in where they intervene: activations, weights, denoising steps, or model-level distributions.

At inference time, SAE-based methods intervene directly on internal features. SAeUron trains SAEs on cross-attention activations across denoising timesteps, scores features by their concept specificity, and ablates only a small concept-aligned subset during generation (Cywiński et al., 29 Jan 2025). SAEParate keeps the same broad recipe but attempts to disentangle concepts before intervention, so that suppression of a target concept produces less interference in joint style-object settings (Kim et al., 12 May 2026). SAUCE applies the same principle to VLMs: the model first identifies SAE features most relevant to the target concept, then selectively modifies these features at inference to suppress the target while preserving unrelated information (Li et al., 16 Mar 2025).

Other methods edit model parameters but try to localize the update. TRUST dynamically estimates “concept neurons” in cross-attention layers from gradients of CLIPScore, recomputes the mask at every fine-tuning step, and applies selective fine-tuning rather than full fine-tuning; it further introduces Concept Influence Penalty and Hessian-based Concept Sensitivity Reduction (Mansi et al., 8 Feb 2026). KSCU exploits the stepwise structure of diffusion sampling, arguing that concept emergence is step-dependent and fine-tuning only “key steps” reduces over-unlearning while preserving generative retainability (Zhang et al., 9 Jul 2025). SurgUn localizes updates to a single attention block and induces retroactive concept interference by pairing the target with a fixed distractor set, using LoRA for parameter-efficient intervention across Stable Diffusion v1.5, SDXL, and SANA (Ranjan et al., 1 Mar 2026). PURE is closed-form and training-free: it records per-layer cross-attention activations along a short denoising trajectory, constructs forget and retain subspaces, and left-multiplies key and value projections with a linear projector derived from those activation bases (Moon et al., 25 May 2026).

A separate family formulates forgetting as alignment between distributions or domains. DoCo aligns the output domains of sensitive and anchor concepts through adversarial training and uses gradient surgery to preserve utility. Its update rule removes the component of the unlearning gradient that conflicts with the retention gradient:

$$\mathbf{G} = \begin{cases} \mathbf{G}_u, & \text{if } \mathbf{G}_u \cdot \mathbf{G}_r \ge 0, \ \mathbf{G}_u - \lambda \frac{\mathbf{G}_u \cdot \mathbf{G}_r}{\|\mathbf{G}_r\|^2}\mathbf{G}_r, & \text{otherwise}. \end{cases}$$

This explicitly encodes the desire to forget a target concept without damaging anchor and unrelated concepts (Wu et al., 2024). ReCARE similarly introduces a retain loss over a CARE-set and an erase loss designed to move harmful tokens away from the target while preserving CARE concepts (Kim et al., 23 Jun 2026). In LLMs, CRISP moves from inference-time steering to persistent parameter-efficient fine-tuning, selecting salient SAE features by contrastive activation analysis and suppressing them in a way that changes model parameters rather than only runtime activations (Ashuach et al., 19 Aug 2025).

4. Evaluation criteria and empirical findings

Concept unlearning is evaluated by a multi-objective protocol. Diffusion-model papers use forgetting metrics such as CLIP Score, CLIP Accuracy, Unlearning Accuracy, Identity Score Matching, and explicit-content detection on I2P with NudeNet; they pair these with retention and quality metrics such as FID, IRA, CRA, COCO CLIP score, and related-class preservation (Wu et al., 2024, Cywiński et al., 29 Jan 2025, Zhang et al., 9 Jul 2025). ReCARE adds CARE score, defined through CLIP R-Precision@1 to quantify preservation of benign co-occurring concepts, and aggregates robustness, utility, and CARE preservation into RATIO (Kim et al., 23 Jun 2026). ICED evaluates target forgetting, non-target retention, and out-of-domain transfer, while LVLM continual-unlearning work uses Context-aware Refusal Rate, Refusal Gap, Answer Rate, BERTScore, CLIPScore, ROUGE-L, and Specificity (Lin et al., 14 May 2026, Jin et al., 23 Mar 2026). LLM concept-unlearning work based on knowledge triplets uses NodeAcc and EdgeAcc for target and non-target entities, plus general utility on standard reasoning benchmarks (Yamashita et al., 19 Sep 2025).

Reported results indicate that selectivity can materially improve over coarse approaches. SAUCE evaluates 60 concepts on LLaVA-v1.5-7B and LLaMA-3.2-11B-Vision-Instruct and reports an 18.04% improvement in unlearning quality while maintaining comparable model utility (Li et al., 16 Mar 2025). DoCo reports that on I2P it reduces explicit content by 75.67% and outperforms prior approaches even for out-of-distribution prompts (Wu et al., 2024). ICED is described as achieving near-zero target accuracy while retaining high accuracy on non-target classes and maintaining competitive model utility (Lin et al., 14 May 2026). ReCARE is reported to achieve the highest overall RATIO across Nudity, Van Gogh style, and Tench erasure, while uniquely preserving the ability to generate “person” after nudity unlearning (Kim et al., 23 Jun 2026). TRUST reports the lowest attack success rate among evaluated methods on I2P, including 0.11% for “Nudity,” with 60 or fewer steps for TRUST-CSR (Mansi et al., 8 Feb 2026). PURE is described as yielding the best overall forget-retain trade-off among evaluated methods on a holistic benchmark, especially under paraphrased and adversarial prompts (Moon et al., 25 May 2026). TILDE reports strong forgetting together with improved retention and distributional fidelity over prior baselines across objects, artistic styles, and characters (George et al., 7 Jul 2026).

The empirical pattern is consistent: forgetting quality alone is not treated as sufficient. Stronger papers increasingly report sibling or neighbor retention, out-of-distribution forgetting, adversarial robustness, co-occurring-concept preservation, and distributional fidelity to a retain-only reference model rather than only target suppression (George et al., 7 Jul 2026, Kim et al., 23 Jun 2026, Moon et al., 25 May 2026).

5. Robustness, persistence, and sequential updates

A central controversy in the area is whether apparent forgetting remains stable after subsequent model updates. “Unstable Unlearning” shows that in text-to-image diffusion models, fine-tuning on seemingly unrelated images can cause concept resurgence: a model may regain the ability to generate concepts that had previously been unlearned, even under benign, non-adversarial conditions (Suriyakumar et al., 2024). The paper attributes resurgence partly to shallow detours in parameter space, with the effect depending on regularization strength, mapping choice, and whether subsequent fine-tuning is in-domain or out-of-domain (Suriyakumar et al., 2024).

Persistence is also a fault line between inference-time and parameter-editing approaches. CRISP explicitly argues that most SAE-based methods operate at inference time and therefore do not create persistent changes in model parameters; such interventions can be bypassed or reversed by malicious actors with parameter access (Ashuach et al., 19 Aug 2025). This concern motivates parameter-efficient but persistent updates in LLMs and, by implication, similar interest in persistent edits for generative vision systems.

Another robustness issue is prompt coverage. Diversified Unlearning argues that keyword-only formulations are brittle because a single keyword fails to cover the full semantic distribution of a concept (Pham et al., 19 Mar 2026). PURE makes an analogous claim in architectural terms: text embeddings describe how a prompt is written, whereas cross-attention activations describe what the model is about to render. Its probing experiment reports that an activation-derived forget basis is five times more likely to recognize paraphrased natural prompts than a text-embedding basis (Moon et al., 25 May 2026). CARE and CORE expose two additional failure modes: collateral forgetting of benign co-occurring concepts in diffusion models, and spurious refusals in continual LVLM unlearning caused by distortion of shared multimodal representations (Kim et al., 23 Jun 2026, Jin et al., 23 Mar 2026).

Sequential and multi-concept settings have therefore become a distinct research topic. SAeUron reports that a single SAE can remove multiple concepts simultaneously (Cywiński et al., 29 Jan 2025). SAUCE studies scalability under multiple simultaneous unlearning requests (Li et al., 16 Mar 2025). SAEParate reports particularly strong gains in joint style-object unlearning and includes sequential-unlearning analysis (Kim et al., 12 May 2026). CORE is explicitly designed for sequential deletion requests in LVLMs, routing concept-grounded refusal behavior through a mixture of refusers specialized for concept-aligned refusal generation (Jin et al., 23 Mar 2026). This suggests that concept unlearning is no longer treated as a one-shot edit, but as a maintenance problem under repeated requests and evolving policies.

6. Theory, exactness, and emerging principles

Theoretical work clarifies why concept unlearning is hard and what a principled solution should optimize. The analysis of unlearning difficulty identifies two key factors: entanglement in embedding space between forget and retain sets, and memorization level of forget-set examples. Both worsen the forgetting–utility trade-off, and the Refined-Unlearning Meta-algorithm (RUM) responds by partitioning the forget set into homogenized subsets and applying different unlearning algorithms sequentially (Zhao et al., 2024). The implication is that forget requests are heterogeneous and that a single unlearning algorithm need not be optimal for every subset.

A more formal optimization perspective appears in the unified KL-constrained framework for diffusion models. It formulates unlearning as minimizing deviation from a pretrained model subject to explicit separation constraints from one or more unlearning distributions, using reverse KL, forward KL, or likelihood constraints. Despite nonconvex KL-feasible sets, the paper proves strong duality and derives explicit optimal solutions, such as

$$p^\star_{\text{revl}}(x)\propto q(x)\exp\!\left(-\sum_{i=1}^m \alpha_i q^i_u(x)\right)$$

for the likelihood-constrained formulation (Khalafi et al., 29 May 2026). TILDE can be read as a closely related distributional view specialized to concept erasure: among all distributions satisfying a forgetting constraint, choose the one with minimum KL deviation from the pretrained conditional distribution. Its optimal target has the Gibbs form

$$p^*_{\mathcal{C}}(x\mid y)=\frac{p_{\text{pre}}(x\mid y)\exp(-\beta E_{\mathcal{C}}(x))}{Z_{\beta,\mathcal{C}}(y)},$$

which is described as an energy-tilted, anchor-free target that suppresses concept-expressing images while preserving benign relative mass for each prompt (George et al., 7 Jul 2026).

Exactness results from learning theory push the idea further. Ticketed Learning-Unlearning Schemes show that, for mergeable concept classes such as thresholds, parities, and intersection-closed classes, the learner can distribute small encrypted tickets to training examples and later reconstruct the exact post-unlearning predictor without access to the original training set (Ghazi et al., 2023). The same work introduces the count-to-zero problem and solves it with ticketed schemes based on Sperner families, giving extremely small ticket sizes for certain tasks (Ghazi et al., 2023).

A plausible synthesis is that contemporary concept unlearning is converging on two complementary principles. The first is semantic localization: concepts must be represented through distributions, decompositions, or sparse interpretable features rather than single labels or prompts. The second is constrained redistribution: unlearning is increasingly cast as reassigning probability mass, activations, or representational capacity away from the target while explicitly protecting retain structure. The remaining open problems identified across the literature are persistent unlearning under later fine-tuning, precise preservation of co-occurring benign concepts, robustness to paraphrase and adversarial recovery, and continual concept-specific refusal without spurious side effects (Suriyakumar et al., 2024, Kim et al., 23 Jun 2026, Jin et al., 23 Mar 2026).