DiffErase: Diffusion-Based Erasure Methods

• DiffErase is a family of diffusion-based erasure methods that remove unwanted content from images and audio while preserving overall signal utility.
• In image editing, approaches like EraDiff use calibrated reverse diffusion with Chain-Rectifying Optimization and Self-Rectifying Attention to avoid reconstructing masked objects.
• In audio watermark removal, DiffErase perturbs mel-spectrogram representations through controlled diffusion to suppress watermarks and maintain perceptual quality.

Searching arXiv for the cited DiffErase/EraDiff papers to ground the article. DiffErase denotes diffusion-based erasure procedures that remove unwanted structure while preserving the utility of the remaining signal. In the available literature, the term spans at least two technically distinct settings. In image editing, “DiffErase” refers to using diffusion models to erase objects inside user-provided masks, with the model eliminating the masked target while hallucinating a coherent background that matches surrounding context; “Erase Diffusion” (EraDiff) is a concrete method in this category (Liu et al., 10 Mar 2025). In audio, DiffErase is the name of a black-box watermark removal attack that perturbs watermarked audio to an intermediate diffusion noise level and regenerates it using a pretrained denoising model, thereby suppressing watermark signals while maintaining perceptual quality (Yao et al., 28 May 2026). A related but different term, “EraseDiff,” concerns concept or knowledge erasure in diffusion models via constrained optimization and should not be conflated with either masked object removal or audio watermark removal (Wu et al., 2024).

1. Terminological scope and disambiguation

The term “DiffErase” is not used uniformly across diffusion-model research. In the image-editing setting, it is the general idea of diffusion-enabled erasure: the input is an image and a mask specifying undesired content, and the task is to remove the object(s) inside the mask while filling the region with visually coherent background consistent in structure, lighting, and semantics. In that usage, EraDiff is a principled DiffErase approach that explicitly calibrates diffusion pathways toward object removal by combining Chain-Rectifying Optimization (CRO) and Self-Rectifying Attention (SRA) (Liu et al., 10 Mar 2025).

In the audio setting, DiffErase is a named attack on watermarking systems. The adversary has access only to the watermarked audio $x_w$ and no knowledge or query access to the watermark embedder and detector or extractor architectures, their parameters, thresholds, or secret keys. The objective is to produce attacked audio $\tilde{x}$ that disables detection or message extraction while maintaining perceptual quality relative to $x_w$ (Yao et al., 28 May 2026).

A separate nomenclature issue arises with EraseDiff. That method formulates diffusion unlearning as a constrained optimization problem that preserves utility on retained data while erasing influence associated with data to be forgotten. The paper introducing EraseDiff explicitly states that it does not mention a method named DiffErase, so the two labels should be treated as distinct despite their orthographic similarity (Wu et al., 2024).

Term	Domain	Meaning in the literature
DiffErase	Image editing	General idea of diffusion-enabled object erasure inside masks
EraDiff	Image editing	Concrete DiffErase method based on CRO and SRA
DiffErase	Audio watermarking	Black-box audio watermark removal via diffusion priors
EraseDiff	Diffusion unlearning	Constrained optimization for forgetting undesirable influence

This multiplicity of usage suggests a shared abstraction: diffusion is employed to redirect a reverse process away from undesired content. The exact object of erasure, however, differs substantially across settings: masked objects, embedded watermarks, or learned concepts.

2. DiffErase as masked object removal in images

In image inpainting, erase inpainting, or object removal, the central difficulty is not merely generating plausible pixels but preventing the reverse process from reconstructing the very object intended for removal. Diffusion-based inpainting models such as SD2-Inpaint are trained to denoise masked images back to the original image distribution. Because that standard training objective learns a pathway from random noise to clear images that may still contain the masked object, the model can generate unexpected objects or artifacts inside the mask and fail to truly erase (Liu et al., 10 Mar 2025).

The formulation in EraDiff identifies this as a diffusion-pathway problem. The reverse chain is learned to reconstruct images with objects, so the diffusion transitions are not aligned with the erasure objective. Early denoising states can also be corrupted by mask-shape leakage and noise, which self-attention may amplify and propagate along the chain. This diagnosis reframes object removal as calibration of the reverse trajectory rather than solely improvement of the final reconstruction.

EraDiff adopts latent diffusion models. The forward noising process is

$$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$

Following DDIM, a general reverse transition from $x_t$ to $x_{\text{prev}}$ is written as

$$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$

where $\epsilon_t \sim N(0, I)$ and $\sigma_t$ controls stochasticity. The baseline network is trained to predict noise via

$$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$

Within this framework, standard diffusion learns denoising toward the original image $\tilde{x}$0, which may include the target object. EraDiff instead redefines the intermediate states that the model should visit.

3. Chain-Rectifying Optimization and Self-Rectifying Attention

CRO is designed to simulate gradual object fading during optimization. Let $\tilde{x}$1 be the original image and $\tilde{x}$2 a synthesized variant with the segmented object transformed and pasted on the background. For each timestep $\tilde{x}$3, CRO constructs a dynamic image

$\tilde{x}$4

and the corresponding latent

$\tilde{x}$5

As $\tilde{x}$6 decreases, $\tilde{x}$7 decreases, so the object content fades across the chain. CRO then teaches the model to step from $\tilde{x}$8 to earlier states $\tilde{x}$9 along this erasure chain. Given $x_w$0, the model-predicted latent at $x_w$1 is

$x_w$2

and the CRO objective is

$x_w$3

This reshapes the transition kernel to flow from “objecty” states to clean background states. At inference, the method can start from a noisy masked latent and follow the calibrated reverse chain. Because CRO trains the model on the object-fading trajectory $x_w$4, partial leakage from the original image can be corrected as the chain progresses.

SRA addresses a different failure mode: artifact amplification by self-attention in early denoising. Standard self-attention in U-Net blocks is

$x_w$5

Let $x_w$6 be the binary mask and let its downsampled, flattened form be $x_w$7. EraDiff defines an extended pairwise mask $x_w$8 by

$x_w$9

The SRA attention is then

$$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$0

This suppresses attention from masked tokens to masked tokens while allowing masked tokens to attend to background tokens and vice versa. The stated effect is to derive features for the erased region from clean background context, mitigating artifact amplification and improving coherence (Liu et al., 10 Mar 2025).

4. Architecture, training protocol, and empirical behavior of EraDiff

EraDiff uses the SD2-Inpaint latent diffusion model as the base model, composed of a VAE encoder and decoder and a U-Net with self-attention blocks. The architectural adaptation is to replace standard self-attention with SRA in U-Net blocks and to fine-tune only the U-Net while keeping the VAE and other modules frozen. The implementation is in PyTorch with Diffusers (Liu et al., 10 Mar 2025).

Training data are drawn from OpenImages V5 segmentation. For each image, the method synthesizes dynamic object/background pairs by foreground extraction with matting, random scaling of $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$1–$$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$2, rotation of $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$3–$$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$4, and paste-back to produce $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$5. The dynamic mix-up schedule $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$6 is set to match $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$7. Optimization uses Adam with learning rate $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$8, timestep gap bound $$q(x_t \mid x_0) = N(x_t; \sqrt{\bar{\alpha}_t}\, x_0, (1-\bar{\alpha}_t) I), \qquad \bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s.$$9, DDIM during training, batch size $x_t$0, and $x_t$1 GPUs for $x_t$2 epochs.

Inference takes an image and binary mask $x_t$3, uses DPM-Solver Multistep with $x_t$4 denoising steps, resolution $x_t$5, denoising strength $x_t$6, and no prompts or classifier-free guidance. SRA remains active in attention layers during sampling.

On OpenImages V5, EraDiff reports LPIPS $x_t$7 and Local FID $x_t$8, both best among the compared methods. For Local FID, the same comparison reports SD2-Inpaint at $x_t$9, LaMa at $x_{\text{prev}}$0, and SD2-Inpaint* at $x_{\text{prev}}$1. EraDiff’s FID is $x_{\text{prev}}$2, while SD2-Inpaint attains lower FID at $x_{\text{prev}}$3 but often fails to remove objects. GPT-4o comparative elimination rates versus EraDiff are reported as follows: against SD2-Inpaint, $x_{\text{prev}}$4 superior, $x_{\text{prev}}$5 comparable, and $x_{\text{prev}}$6 inferior; against SD2-Inpaint*, $x_{\text{prev}}$7 superior, $x_{\text{prev}}$8 comparable, and $x_{\text{prev}}$9 inferior; against LaMa, $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$0 superior, $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$1 comparable, and $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$2 inferior.

Ablations attribute the improvement to both CRO and SRA. Removing CRO changes Local FID from $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$3 to $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$4 and GPT elimination score from $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$5 to $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$6. Removing SRA yields Local FID $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$7 and GPT $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$8. Removing both yields Local FID $$x_{\text{prev}} = \sqrt{\bar{\alpha}_{\text{prev}}} \frac{x_t - \sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t)}{\sqrt{\bar{\alpha}_t}} + \sqrt{1-\bar{\alpha}_{\text{prev}}-\sigma_t^2}\,\epsilon_\theta(x_t) + \sigma_t \epsilon_t,$$9 and GPT $\epsilon_t \sim N(0, I)$0. Removing mix-up causes training destabilization, with the loss diverging. On FSS-1000, EraDiff achieves LPIPS $\epsilon_t \sim N(0, I)$1 and Local FID $\epsilon_t \sim N(0, I)$2, both reported as best, with competitive FID. Additional synthesized-test indicators list PIDS/UIDS$\epsilon_t \sim N(0, I)$3 of $\epsilon_t \sim N(0, I)$4, AES $\epsilon_t \sim N(0, I)$5, and NIMA $\epsilon_t \sim N(0, I)$6.

Efficiency is reported at approximately $\epsilon_t \sim N(0, I)$7B parameters, the same as SD2-Inpaint. Inference time per image is $\epsilon_t \sim N(0, I)$8 s for EraDiff and $\epsilon_t \sim N(0, I)$9 s for SD2-Inpaint, partly due to the absence of CFG. LaMa is faster at $\sigma_t$0 s, and PowerPaint is slower at $\sigma_t$1 s.

5. DiffErase as black-box audio watermark removal

In audio watermarking, DiffErase addresses a different erasure target: imperceptible watermark signals embedded for provenance or copyright protection. The method assumes a strict black-box setting in which the adversary has no knowledge or query access to the watermarking scheme. Its core idea is SDEdit-style regeneration: add moderate diffusion noise to a watermarked sample and denoise it with a pretrained model so that structured, off-manifold watermark perturbations are suppressed while content is preserved (Yao et al., 28 May 2026).

The paper models watermarked audio as $\sigma_t$2 and applies diffusion in mel-spectrogram space rather than directly on waveforms. The stated reason is empirical: direct waveform diffusion caused over-smoothing and temporal drift, whereas linear spectrogram diffusion with phase reuse created magnitude-phase inconsistencies and artifacts. DiffErase therefore operates on mel-spectrograms, and a latent variant compresses mel features with a VAE and denoises in latent space.

For a mel representation or latent thereof $\sigma_t$3, DDPM forward diffusion is

$\sigma_t$4

with reparameterization

$\sigma_t$5

The reverse mean is

$\sigma_t$6

and the reverse step is

$\sigma_t$7

The operational pipeline has two phases. First, diffusion erasure: choose a noise level $\sigma_t$8 and sample

$\sigma_t$9

Second, semantic reconstruction: run the pretrained denoiser from $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$0 back to $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$1 and vocode the reconstructed mel-spectrogram to waveform. The mel-spectrogram instantiation is

$$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$2

and the latent instantiation is

$$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$3

The theoretical framing relies on a manifold hypothesis. With shared noise in the forward step, the residue at $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$4 satisfies

$$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$5

Under a local Gaussian approximation of $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$6 and score $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$7, the reverse dynamics act as a restoring force in the normal direction, yielding a one-step contraction $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$8 with $$\min_\theta \; \mathbb{E}_{\epsilon \sim N(0,I)} \left\| \epsilon - \epsilon_\theta^{(t)}(x_t) \right\|_2^2.$$9. Iterating gives

$\tilde{x}$00

If the detector score $\tilde{x}$01 is $\tilde{x}$02-Lipschitz and $\tilde{x}$03, then

$\tilde{x}$04

which yields a sufficient condition for removal when the right-hand side is below the detector threshold $\tilde{x}$05. The paper states that this implies a minimum $\tilde{x}$06 above which removal succeeds.

6. Evaluation, limitations, and relation to diffusion unlearning

The audio DiffErase evaluation spans speech, music, and environmental sounds, using LibriSpeech, FMA-small, and Clotho, respectively, with AudioSeal, TimbreWM, WavMark, Perth, and SilentCipher as watermarking systems. Quality metrics are SQUIM-MOS, ViSQOL, and MUSHRA; the removal metric is TPR@1%FPR, where lower is better and “$\tilde{x}$07” indicates disabled detection at that operating point (Yao et al., 28 May 2026).

On speech, DiffErase-mel at typical $\tilde{x}$08 reports SQUIM-MOS $\tilde{x}$09, ViSQOL $\tilde{x}$10, and MUSHRA $\tilde{x}$11, while achieving TPR@1%FPR $\tilde{x}$12 for AudioSeal, WavMark, TimbreWM, Perth, and SilentCipher. DiffErase-latent reports SQUIM-MOS $\tilde{x}$13, ViSQOL $\tilde{x}$14, and MUSHRA $\tilde{x}$15, again with TPR@1%FPR $\tilde{x}$16 across all systems. Cross-domain results remain strong but reveal a harder case for Perth: in music, TPR changes from $\tilde{x}$17 to $\tilde{x}$18 for Perth, and in environmental sounds from $\tilde{x}$19 to $\tilde{x}$20, while other systems are reduced to $\tilde{x}$21 or $\tilde{x}$22. The paper attributes Perth’s robustness to larger perturbations, approximately $\tilde{x}$23–$\tilde{x}$24 larger than others.

Ablations separate the contribution of diffusion from that of reconstruction. Griffin–Lim inversion alone does not remove watermarks, with TPR approximately $\tilde{x}$25. Adding diffusion at $\tilde{x}$26 reduces detection dramatically. At matched $\tilde{x}$27, DDPM removes more strongly than DDIM. The representation study reports that waveform diffusion harms quality, linear spectrogram diffusion preserves quality better but fails to remove, and mel-spectrogram or mel-latent diffusion provides the best removal–fidelity trade-off.

The image and audio variants of DiffErase also share identifiable limitations. For EraDiff, reported failure modes include text-heavy or document scenes, completion-heavy edits such as reconstructing an arm under a removed coat, and large background replacement without reference cues (Liu et al., 10 Mar 2025). For audio DiffErase, limitations include strong, higher-energy watermarks, domain mismatch between attacked audio and the pretrained denoiser, fine-detail loss at larger $\tilde{x}$28, and representation constraints (Yao et al., 28 May 2026).

The relation to EraseDiff clarifies the broader taxonomy. EraseDiff does not operate on masked image editing or audio watermark removal; instead, it erases undesirable influence in the model itself by minimizing a preservation loss on retained data and imposing a forgetting constraint on forgotten data. Its preservation objective is

$\tilde{x}$29

while the forgetting loss replaces the ground-truth noise with alternative noise $\tilde{x}$30 on $\tilde{x}$31:

$\tilde{x}$32

The constrained formulation is

$\tilde{x}$33

with a first-order update direction $\tilde{x}$34. This is a different intervention point: training-time unlearning of model knowledge rather than inference-time or task-specific removal (Wu et al., 2024).

Taken together, these works show that “DiffErase” names a family of diffusion-based erasure ideas rather than a single canonical algorithm. In one branch, reverse trajectories are calibrated so that masked image regions transition from object-bearing states to coherent background states. In another, watermarked audio is perturbed to an intermediate noise level and denoised back toward a clean manifold. The adjacent EraseDiff line demonstrates that erasure can also target the model’s retained knowledge itself. The common technical motif is trajectory control: the denoising process is redirected so that undesirable content is not reconstructed.