Texture-Guided Perturbation Framework

• Texture-Guided Perturbation Framework is a design pattern that uses facial texture cues (e.g., LBP, Grad-CAM) to localize perturbations within low-saliency regions.
• It applies region-selective techniques to disrupt key textured areas while preserving smooth regions, achieving imperceptible yet effective interventions.
• The framework underpins applications in Deepfake defense, image editing, adversarial robustness, geometric recovery, and audio texture manipulation with quantifiable gains.

Texture-guided perturbation framework most directly denotes a proactive Deepfake detection approach based on facial texture features, in which perturbations are invisibly inserted within texture regions that have low perceptual saliency, with localized perturbations applied to key texture regions while minimizing unwanted noise in non-textured areas (Zhang et al., 24 Aug 2025). In a broader research sense, related work suggests a methodological family in which texture cues, texture-specialized latent directions, texture-scale constraints, or texture-relevant attention heads are used to steer perturbations for image editing, adversarial attack and defense, geometric recovery, and audio texture generation (Su et al., 2024). This suggests that “texture-guided perturbation” is less a single algorithm than a recurrent design pattern: localize or parameterize the perturbation through texture-sensitive structure, then preserve non-target content through masking, blending, or low-dimensional control.

1. Formal basis of the facial texture-feature framework

In the Deepfake-defense formulation, the input is an RGB face image $I_{\rm ori}\in\mathbb{R}^{H\times W\times 3}$. The image is first converted to a single-channel luminance image $I_{\rm gray}$ using ITU-R BT.709 weights, then bilateral filtering is applied with $\Omega=31$, $\sigma_d=75$, and $\sigma_r=15$. Classical circular Local Binary Patterns are then computed with $P=8$ and $R=1$, producing the initial texture feature map $T(x,y)={\rm LBP}_{8,1}(x,y)$, which highlights high-frequency textured regions (Zhang et al., 24 Aug 2025).

The framework then combines this texture map with a local attention signal from a pretrained ResNet50 classifier. A Grad-CAM map $M_{\rm ori}\in[0,1]^{H\times W}$ is extracted from the clean RGB image, and the fused guidance signal is defined by element-wise multiplication,

$F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$

This fused map is passed through a perturbation enhancement module built from 3 stacked Conv $I_{\rm gray}$0 BN $I_{\rm gray}$1 ReLU blocks followed by multi-layer deformable convolutions, yielding the perturbation tensor $I_{\rm gray}$2. The defended image is

$I_{\rm gray}$3

with clamping to the valid pixel range $I_{\rm gray}$4 (Zhang et al., 24 Aug 2025).

The optimization objective couples imperceptibility and downstream disruption. The total loss is

$I_{\rm gray}$5

with $I_{\rm gray}$6, $I_{\rm gray}$7, and $I_{\rm gray}$8. Here, $I_{\rm gray}$9 constrains pixel-wise changes, $\Omega=31$0 encourages maximal divergence between Deepfake outputs, and $\Omega=31$1 is defined from ViT-based Grad-CAM maps on the clean and protected Deepfake outputs, using a binary difference mask thresholded at $\Omega=31$2 (Zhang et al., 24 Aug 2025).

2. Region selectivity, perceptual logic, and empirical performance

The framework is explicitly region-selective rather than globally additive. By construction, $\Omega=31$3 is zero wherever $\Omega=31$4 or $\Omega=31$5. The bilateral-filtered LBP is used to sharply delineate textures such as hair strands, wrinkles, and pores, while the ResNet50 Grad-CAM restricts perturbation to facial regions critical to the network’s decision. The resulting dual mask,

$\Omega=31$6

is intended to guarantee low perceptual noticeability in flat regions while maximizing disruption in high-frequency, semantically important areas (Zhang et al., 24 Aug 2025).

Evaluation was reported on CelebA-HQ and LFW. CelebA-HQ comprises 30 000 high-resolution $\Omega=31$7 celebrity faces with 6 217 identities, and LFW contains 13 233 images of 5 749 subjects in unconstrained settings; both were resized to $\Omega=31$8. The attack models were StarGAN, AttGAN, AGGAN, HiSD, and StarGAN-V2. Visual quality was measured by PSNR, SSIM, and LPIPS, while defense effectiveness used the mean $\Omega=31$9 distance between $\sigma_d=75$0 and $\sigma_d=75$1, together with Defense Success Rate, defined as the fraction of cases where $\sigma_d=75$2 (Zhang et al., 24 Aug 2025).

On CelebA-HQ, the reported visual-quality results for the protected images were PSNR $\sigma_d=75$3 dB, SSIM $\sigma_d=75$4, and LPIPS $\sigma_d=75$5. Defense success against the five Deepfake models was reported as follows: StarGAN $\sigma_d=75$6-dist $\sigma_d=75$7, DSR $\sigma_d=75$8; AttGAN $\sigma_d=75$9, $\sigma_r=15$0; AGGAN $\sigma_r=15$1, $\sigma_r=15$2; HiSD $\sigma_r=15$3, $\sigma_r=15$4; StarGAN-V2 $\sigma_r=15$5, $\sigma_r=15$6; average $\sigma_r=15$7, $\sigma_r=15$8. On LFW, the reported visual-quality values were PSNR $\sigma_r=15$9, SSIM $P=8$0, and LPIPS $P=8$1, with DSR $P=8$2 on unseen data. The ablation study compared ResNet50-only guidance, ViT-only guidance, and the full dual-model system, yielding average DSR values of approximately $P=8$3, $P=8$4, and $P=8$5, respectively, with the full model also giving the best visual fidelity (Zhang et al., 24 Aug 2025).

A common misconception is that perturbation efficacy necessarily improves with broader spatial coverage. The reported construction argues for the opposite: perturbations are deliberately hidden in texture regions because human eyes are more sensitive to perturbations in smooth regions, and the defense objective is to minimize unwanted noise in non-textured areas (Zhang et al., 24 Aug 2025).

3. Texture disentanglement as perturbation in diffusion-based image editing

A related but distinct formulation appears in TextureDiffusion, a tuning-free image editing method for various texture transfer. Its starting point is the claim that existing text-guided image editing methods generally require target prompts that include both the input image content and the desired texture, which restricts texture representation and makes complex textures such as cloud or fire difficult to transfer. TextureDiffusion replaces the full target sentence with a standalone token $P=8$6, making the texture disentangled from the input image content (Su et al., 2024).

The pipeline consists of four steps: invert the input image $P=8$7 into a latent $P=8$8 via the VAE encoder and DDIM inversion; during sampling, use only the prompt $P=8$9 to condition the denoiser $R=1$0; inject structural cues from the original inversion run into the editing run; and apply edit localization to preserve the background. The key perturbative operations are query-feature injection in self-attention and feature injection in residual blocks. For early layers and timesteps $R=1$1, the editing query is overridden by the source query,

$R=1$2

and the self-attention output becomes

$R=1$3

Residual-block features are also replaced at selected layers through

$R=1$4

The reported schedule performs attention-query injection in the first 40 sampling steps at U-Net layers 12–15, and residual injection in all 50 steps at layer 7 (Su et al., 2024).

Background maintenance is handled by edit localization. Cross-attention maps at $R=1$5 resolution are aggregated over heads and layers, then thresholded on the object token to obtain a binary mask $R=1$6. Self-attention outputs are blended by

$R=1$7

and intermediate latents are blended by

$R=1$8

The implementation uses Stable Diffusion v1.4, DDIM with 50 steps, classifier-free guidance 7.5, 12 attention heads per multi-head block, feature dimension 768, and mask threshold 0.3 after aggregating cross-attention maps (Su et al., 2024).

On PIE-Bench material change, the reported quantitative results were structure distance $R=1$9, background PSNR $T(x,y)={\rm LBP}_{8,1}(x,y)$0 dB, background LPIPS $T(x,y)={\rm LBP}_{8,1}(x,y)$1, background MSE $T(x,y)={\rm LBP}_{8,1}(x,y)$2, background SSIM $T(x,y)={\rm LBP}_{8,1}(x,y)$3, and CLIP similarity on edited region $T(x,y)={\rm LBP}_{8,1}(x,y)$4, with all numbers reported as outperforming SDEdit, P2P, PnP, MasaCtrl, FPE and InfEdit. The ablation sequence was also explicit: no disentanglement led to poor representation of cloud and fire; query injection alone gave partial shape recovery but blurry edges; residual injection produced crisp structural preservation; and edit localization left the background untouched. Reported limitations were residual background artifacts from the VAE encoder/decoder cycle, one-texture-at-a-time masking, and failure modes for extremely thin or highly transparent objects (Su et al., 2024).

This shows that, in diffusion editing, the perturbation is not necessarily additive image noise. It can instead take the form of controlled intervention in prompt conditioning, self-attention queries, residual features, and intermediate latents.

4. Texture scale and head-level perturbation in adversarial and generative models

Texture guidance has also been formalized through scale constraints in universal adversarial perturbations. Texture Scale–Constrained UAP introduces a small patch $T(x,y)={\rm LBP}_{8,1}(x,y)$5 and tiles it to form the full perturbation $T(x,y)={\rm LBP}_{8,1}(x,y)$6, so that the optimization becomes

$T(x,y)={\rm LBP}_{8,1}(x,y)$7

The stated motivation is that CNNs classify objects according to semantic information stored in local textures, and that forcing the perturbation to be a tiled repetition of a smaller patch encourages category-specific local textures and improves fooling ratio and transferability (Huang et al., 2024).

The reported ImageNet results showed strong dependence on $T(x,y)={\rm LBP}_{8,1}(x,y)$8. For ResNet50, fooling ratio rose from $T(x,y)={\rm LBP}_{8,1}(x,y)$9 at $M_{\rm ori}\in[0,1]^{H\times W}$0 to $M_{\rm ori}\in[0,1]^{H\times W}$1 at $M_{\rm ori}\in[0,1]^{H\times W}$2; for DenseNet121, from $M_{\rm ori}\in[0,1]^{H\times W}$3 to $M_{\rm ori}\in[0,1]^{H\times W}$4; for MobileNet-v2, from $M_{\rm ori}\in[0,1]^{H\times W}$5 to $M_{\rm ori}\in[0,1]^{H\times W}$6. Average improvement over $M_{\rm ori}\in[0,1]^{H\times W}$7 was reported as approximately $M_{\rm ori}\in[0,1]^{H\times W}$8 percentage points, with optimal $M_{\rm ori}\in[0,1]^{H\times W}$9 typically around $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$0–$F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$1. When $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$2 became too large, fooling ratio declined, and this was attributed to insufficient capacity of an overly small patch. The paper also reported gains in cross-model transferability, targeted fooling, cross-dataset transfer, data efficiency, and data-free UAP settings (Huang et al., 2024).

A further variant appears in head-level attention perturbation for diffusion transformers. HeadHunter iteratively selects attention heads according to a user-specified objective $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$3, evaluating candidate subsets over prompt-seed pairs $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$4, while SoftPAG linearly interpolates each selected attention map toward the identity matrix,

$F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$5

The corresponding guided prediction is

$F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$6

The reported empirical finding was that specific heads govern distinct visual concepts such as structure, style, and texture quality, and that linearly interpolating attention toward the identity provides a continuous knob for perturbation strength and artifact suppression (Ahn et al., 12 Jun 2025).

Validation included FID, PickScore, HPS, ImageReward, and AES, alongside single-head perturbation galleries, composition experiments, and ablations over $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$7. The reported pattern was that intermediate $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$8 often gave the best FID or PickScore, moderate $F(x,y)=T(x,y)\times M_{\rm ori}(x,y).$9 such as $I_{\rm gray}$00–$I_{\rm gray}$01 preserved realistic micro-textures while avoiding flattening, and increasing the number of selected heads exhibited a sweet-spot trade-off between texture strength and artifact risk. This directly contradicts the idea that more perturbation is always preferable: the paper describes oversmoothing in existing layer-level perturbation and over-saturation under excessive perturbation (Ahn et al., 12 Jun 2025).

5. Learned texture perturbations for geometric recovery

In geometric inference, learned texture perturbations were proposed as a response to over-regularization, which suppresses high-frequency modes in network outputs and causes tightly packed wrinkles and folds to disappear or become overly smooth. Rather than forcing the network to predict all wrinkles, the method embeds high-frequency details into a secondary signal: a perturbation of cloth texture coordinates. When the network predicts only low-frequency geometry, the fine details survive in the rendered texture and can be used across views to recover lost geometric information (Wu et al., 2020).

The cloth mesh is written as $I_{\rm gray}$02, with vertex positions $I_{\rm gray}$03 and UV coordinates $I_{\rm gray}$04. The corrected network output uses perturbed texture coordinates

$I_{\rm gray}$05

where $I_{\rm gray}$06 is a small per-vertex displacement for camera view $I_{\rm gray}$07. In continuous notation this is described as

$I_{\rm gray}$08

Ground-truth targets for the displacements are computed by ray intersection: for each inferred vertex, a ray is cast from the camera center through the inferred mesh into the ground-truth mesh; the first intersection point is found; its UV is barycentrically interpolated; and the original UV is subtracted. Occluded vertices are filled by a diffusion step based on fast marching and averaging (Wu et al., 2020).

The network architecture, TSNN, maps pose parameters $I_{\rm gray}$09 to a $I_{\rm gray}$10 displacement image with 4 channels, representing $I_{\rm gray}$11 for front and back surfaces. The input is a $I_{\rm gray}$12 tensor of joint transforms, and the network consists of transpose-convolutions with BatchNorm and ReLU. Training uses the loss

$I_{\rm gray}$13

The dataset comprised approximately 10 000 simulated T-shirt poses, each with a ground-truth 3 000-vertex mesh and inferred low-frequency mesh, subdivided once to about 12 000 vertices, and split 80\%/10\%/10\% for train, validation, and test. Training used PyTorch and Adam with learning rate $I_{\rm gray}$14 (Wu et al., 2020).

Reported quantitative results on the test set were per-pixel SqrtMSE $I_{\rm gray}$15 for the baseline inferred cloth, $I_{\rm gray}$16 for ground-truth texture-sliding, $I_{\rm gray}$17 for GS+subdivision, and $I_{\rm gray}$18 for the networked TSNN. The qualitative claim was that texture sliding immediately re-introduces sharp wrinkles that were smoothed out, while 3D reconstructions recover high-frequency folds with $I_{\rm gray}$19 mm reprojection error. Reported limitations included reliance on multi-view stereo or triangulation, TSNN-induced outliers or noisy rays, careful handling of occlusion and silhouette boundaries, and the use of a fixed camera array (Wu et al., 2020).

6. Latent-space guidance vectors for audio texture generation

In audio texture generation, the perturbative control variable is a guidance vector in the latent space of an unconditionally trained StyleGAN2 rather than an image-space perturbation. The generator decomposes as $I_{\rm gray}$20 and $I_{\rm gray}$21, and controllable editing is performed by

$I_{\rm gray}$22

followed by synthesis $I_{\rm gray}$23. The guidance vector $I_{\rm gray}$24 is inferred from positive and negative synthetic examples that indicate the presence or absence of a semantic attribute such as brightness, impact rate, or fill level (Kamath et al., 2023).

The procedure begins with an encoder $I_{\rm gray}$25, trained by minimizing a sum of MSE on spectrogram reconstruction and MSE on the latent $I_{\rm gray}$26. For a given attribute, two small sets of synthetic audio clips are constructed, $I_{\rm gray}$27 and $I_{\rm gray}$28, typically with $I_{\rm gray}$29 examples per cluster. Their encodings are centered around the global center-of-mass $I_{\rm gray}$30 of $I_{\rm gray}$31, principal directions are found by SVD, prototypes are formed by projecting cluster means back to one-dimensional subspaces, and the guidance vector is then

$I_{\rm gray}$32

The paper emphasizes that no iterative solver or gradient-based optimization is required; the prototypes and guidance vector are computed in closed form (Kamath et al., 2023).

The same formulation supports selective semantic attribute transfer. Given reference and target latents $I_{\rm gray}$33 and $I_{\rm gray}$34, their projections onto $I_{\rm gray}$35 are computed, the transfer increment is $I_{\rm gray}$36, and the new target is

$I_{\rm gray}$37

Implementation details included $I_{\rm gray}$38, 4 mapping-network layers, log-magnitude Gabor spectrograms with 512 STFT channels and 256 frames, PGHI waveform reconstruction, and a ResNet-34 encoder trained with Adam at $I_{\rm gray}$39, $I_{\rm gray}$40, $I_{\rm gray}$41. The reported finding was that spectrograms exhibit smooth monotonic change in the target attribute without spoiling overall texture quality, and that 5–10 examples per cluster suffice (Kamath et al., 2023).

This suggests that texture-guided perturbation need not be tied to pixel-space imperceptibility. In cross-modal settings, the perturbation can be a semantically aligned latent displacement that preserves overall texture quality while isolating one perceptual attribute.

7. Shared principles, recurrent limits, and representative instances

Across these papers, texture guidance repeatedly appears as a way to constrain where perturbations are applied, what subspace they occupy, or which attributes they control. The common structural motifs are selective localization, low-dimensional parameterization, and explicit preservation of non-target content.

Representative method	Perturbation locus	Reported role
"Defending Deepfake via Texture Feature Perturbation" (Zhang et al., 24 Aug 2025)	LBP- and Grad-CAM-guided facial texture regions	Distort Deepfake generation while preserving image quality
"TextureDiffusion: Target Prompt Disentangled Editing for Various Texture Transfer" (Su et al., 2024)	Self-attention queries, residual features, and intermediate latents	Transfer various textures with structure and background preservation
"Texture Re-scalable Universal Adversarial Perturbation" (Huang et al., 2024)	Tiled local patch $I_{\rm gray}$42 repeated across the image	Improve fooling ratio and attack transferability
"Fine-Grained Perturbation Guidance via Attention Head Selection" (Ahn et al., 12 Jun 2025)	Selected attention heads interpolated toward identity	Fine-grained control of generation quality and visual attributes
"Recovering Geometric Information with Learned Texture Perturbations" (Wu et al., 2020)	Per-vertex UV offsets	Recover high-frequency geometric detail
"Example-Based Framework for Perceptually Guided Audio Texture Generation" (Kamath et al., 2023)	Latent guidance vector $I_{\rm gray}$43 in $I_{\rm gray}$44	Controllable generation and selective semantic attribute transfer

One recurrent misconception is that texture-guided perturbation is synonymous with arbitrary high-frequency noise. The surveyed formulations do not support that view. In the facial defense method, perturbations are explicitly suppressed in smooth areas and non-salient regions. In TextureDiffusion, a disentangled $I_{\rm gray}$45 prompt alone is insufficient, and structure must be re-injected through attention and residual features. In TSC-UAP, the perturbation is constrained to a tiled local patch, and performance falls when the texture scale becomes too small or too large. In head-level attention perturbation, moderate interpolation and a limited number of heads give a better trade-off than maximal perturbation. In geometric recovery, the perturbation is not noise but a smoothly varying UV-offset field. In audio, the perturbation is a closed-form latent direction derived from a few examples (Zhang et al., 24 Aug 2025).

The reported limitations are likewise consistent across domains. TextureDiffusion notes residual background artifacts from the VAE encoder/decoder cycle, one-texture-at-a-time masking, and unreliable cross-attention masks for extremely thin or highly transparent objects. TSC-UAP reports declining fooling ratio when $I_{\rm gray}$46, indicating insufficient capacity in very small repeated patches. HeadHunter and SoftPAG report artifact risk from over-perturbation and a sweet spot in the number of selected heads. Learned texture perturbations for cloth require multi-view stereo or triangulation and careful handling of occlusion boundaries. The audio framework is motivated by the absence of large semantically labeled datasets and therefore relies on a few synthetic examples rather than direct supervision (Su et al., 2024).

Taken together, these works indicate that texture-guided perturbation is best understood as a technical strategy for exploiting the fact that many modern models encode decisive information in local textures, texture-sensitive attention patterns, or semantically disentangled texture directions. A plausible implication is that future variants will continue to move away from unconstrained global perturbation and toward explicitly localized, structured, and task-specific control variables.