Self-Prior Perturbation

• Self-Prior Perturbation is a technique where models generate internal priors by perturbing their own outputs to create challenging training examples without external data.
• The method involves pseudo-semantic prior extraction, severe augmentation, and sample reweighting to enhance both adversarial robustness and image restoration.
• Empirical results demonstrate improved white-box and black-box attack effectiveness, as well as superior dehazing performance, highlighting its practical impact in data-free settings.

Self-prior perturbation refers to a family of techniques in which a model's priors or surrogate training instances are constructed or perturbed from within the model, its own outputs, or from perturbations derived recursively or synthetically, rather than relying solely on data-driven (external) priors. These techniques have found significant application in both adversarial robustness and data-free learning, leveraging self-generated or self-augmented priors to guide model learning or attacks in scenarios where access to real data is limited or non-existent.

1. Principles of Self-Prior Perturbation

The core idea of self-prior perturbation is to generate informative or challenging training samples by perturbing available signals—either the model’s own parameters, synthetic noise, or by severe augmentation of the model's own outputs. In adversarial attack scenarios, this often means generating pseudo-semantic priors by recursively combining the current state of a universal adversarial perturbation (UAP) with random noise. In self-supervised or enhancement contexts, severe synthetic augmentations (including structured noise, blended artifacts, or light effects) are employed to force the model to learn invertible, invariant, and robust representations in the absence of explicit labels or clean input-output pairs (Lee et al., 28 Feb 2025, Lin et al., 2024).

2. Methodologies

2.1. Pseudo-Semantic Prior Extraction in Data-Free UAP

In data-free universal adversarial perturbation (UAP) methods, each optimization iteration begins by combining random noise $z \sim p_z$ (e.g., uniform on $[-\epsilon, \epsilon]$) with the currently learned universal perturbation $\delta^{(t)}$, to form a pseudo-semantic prior $p^{(t)} = \delta^{(t)} + z$. This pseudo-prior is treated as a pseudo-image, from which $N$ random crops are extracted and various spatial transformations (rotation, scaling, block-shuffling) are applied. The resulting samples $x_k^{(t)}$ function as surrogate data for computing activation maximization objectives, substantially enriching the semantic content otherwise missing from noise-driven attacks (Lee et al., 28 Feb 2025).

2.2. Severe Augmentation for Self-Prior Learning

In enhancement and restoration scenarios, such as NightHaze for nighttime dehazing, self-prior perturbation is realized via aggressive (termed "severe") augmentations of clean images. Here, the clean image $J$ is blended with real-world glow maps $L$ and strong additive Gaussian noise $\epsilon$, modulated via a region-specific blending map $W_b$:

$[-\epsilon, \epsilon]$0

This constructs an input $[-\epsilon, \epsilon]$1 where the noise and blended artifacts are commensurate with image content, thereby challenging the model to learn robust, de-noising, and deglowing reconstruction priors (Lin et al., 2024).

2.3. Sample Reweighting and Hard Example Mining

Self-prior perturbation methods frequently integrate weighting mechanisms to emphasize samples which are still difficult to fool or recover. In PSP-UAP, for each sample $[-\epsilon, \epsilon]$2, the Kullback-Leibler divergence between clean and perturbed network outputs is computed; its reciprocal is used as a sample weight, meaning that samples which are little changed by the current $[-\epsilon, \epsilon]$3 ("hard" samples) are assigned greater importance in the adversarial loss:

$[-\epsilon, \epsilon]$4

This focus on hard samples improves the diversity and effectiveness of the perturbations (Lee et al., 28 Feb 2025).

2.4. Encoder–Decoder Architectures and Self-Refinement

Self-prior perturbation can be instantiated in masked autoencoder (MAE)-style encoder–decoder frameworks, as in NightHaze. Training proceeds solely by minimizing pixel-wise mean squared error between the network's reconstruction and the clean image, with no reliance on perceptual or adversarial losses. To further refine internal priors in the absence of ground truth, a semi-supervised teacher–student module is employed: the teacher generates pseudo-labels on unlabelled hazy images via averaged overlapped predictions, and the student model is distilled to these predictions. Updates to the teacher proceed via exponential moving average (EMA), further gated by image quality assessment (IQA) thresholds to ensure only improvements are propagated (Lin et al., 2024).

3. Key Algorithms and Objective Functions

Both adversarial and enhancement-oriented self-prior perturbation methodologies can be formalized by characteristic workflows and loss constraints.

PSP-UAP Workflow

1. Initialize universal perturbation $[-\epsilon, \epsilon]$5.
2. For each iteration:
   • Sample noise $[-\epsilon, \epsilon]$6; form $[-\epsilon, \epsilon]$7.
   • For $[-\epsilon, \epsilon]$8:
     • Random crop and transform to produce $[-\epsilon, \epsilon]$9.
     • Compute $\delta^{(t)}$0 and $\delta^{(t)}$1.
     • Set weight $\delta^{(t)}$2.
   • Compute reweighted, multi-layer activation-maximizing loss:

   $\delta^{(t)}$3

- Perform projected gradient ascent and clip $\delta^{(t)}$4 to $\delta^{(t)}$5 (Lee et al., 28 Feb 2025).

Severe Augmentation Losses

In self-prior learning for enhancement:

• Input is the severely augmented $\delta^{(t)}$6 as in Eq. (1) above.
• Objective is pixel-wise MSE or MAE:

$\delta^{(t)}$7

No perceptual or adversarial losses are used. Only weight decay acts as regularization (Lin et al., 2024).

Semi-supervised Teacher–Student Distillation

• For unlabelled $\delta^{(t)}$8, teacher predictions are averaged, confidence-masked, and used as targets for a student model under further augmentation.
• The student is updated via masked $\delta^{(t)}$9 loss; teacher updates are EMA-gated by an IQA score threshold (Lin et al., 2024).

4. Quantitative Performance and Ablations

Experimental results demonstrate that self-prior perturbation yields substantial improvements both in attack effectiveness and restoration fidelity.

PSP-UAP (Data-Free UAP)

Setting	PSP-UAP	Best Prior (DF)	Leading Data-Dep.
White-box (avg. FR)	89.95%	87.95% (AT-UAP-U)	N/A
Black-box (avg. FR)	70.1%	69.6% (TRM-UAP)	SGA-UAP, AT-UAP-S: 62–75% (varies)

Ablations show the cumulative value of pseudo-semantic prior, sample reweighting, and input transforms, with the full combination resulting in the largest improvement ($p^{(t)} = \delta^{(t)} + z$079% FR vs. $p^{(t)} = \delta^{(t)} + z$163% for random prior only) (Lee et al., 28 Feb 2025).

NightHaze (Self-Prior Dehazing)

Ablation on augmentation severity parameter $p^{(t)} = \delta^{(t)} + z$2 (portion of samples with severe augmentation):

$p^{(t)} = \delta^{(t)} + z$3 (%)	MUSIQ	TRES
0	44.67	62.18
25	48.64	67.96
50	49.12	68.74
75	49.31	68.92
100	52.87	76.88

Performance improves monotonically as the severity of self-prior perturbation increases, up to a $p^{(t)} = \delta^{(t)} + z$4 gain in MUSIQ and $p^{(t)} = \delta^{(t)} + z$5 in ClipIQA over existing dehazing methods on RealNightHaze (Lin et al., 2024).

5. Context and Implications

Self-prior perturbation unifies techniques from adversarial attack (pseudo-semantic UAP) and self-supervised enhancement (severe synthetic augmentation) under a general principle: models can be made more robust, transferable, or restoration-capable by relying on complex, internally constructed priors in lieu of external data. This approach is particularly impactful where labeled data are unavailable, privacy-constrained, or in transfer/black-box attack settings. The effectiveness of self-prior perturbation is strongly linked to the diversity and difficulty of generated perturbations and the ability of weighting schemes to focus training on harder examples.

A plausible implication is that advances in self-prior perturbation may generalize to a broad class of data-free or self-supervised learning problems beyond the domains currently demonstrated. This suggests further exploration of self-constructed perturbation spaces and weighting strategies as a promising research direction.

6. Limitations and Future Directions

Current limitations include potential over-suppression or artifact introduction (noted in NightHaze as over-suppression artifacts), and the reliance on perturbation diversity rather than real semantic coverage. Ongoing research focuses on refining self-prior extraction and weighting, with secondary modules (such as teacher–student distillation gated by IQA) aimed at mitigating artifacts in real domain adaptation. Extensions to other domains and more intricate perturbation distributions are active areas of investigation (Lee et al., 28 Feb 2025, Lin et al., 2024).