Semantics-Preserving Perturbations (SPP)

• Semantics-Preserving Perturbations (SPP) are methods that alter input forms while maintaining the underlying semantic meaning across text, vision, and code domains.
• SPP methods use domain-specific criteria—like cosine similarity for text and feature exclusion for vision—to ensure that key semantic characteristics remain intact.
• These techniques improve model robustness in adversarial training, privacy protection, and uncertainty estimation by filtering out non-semantic or misleading perturbations.

Searching arXiv for the cited SPP-related papers to ground the article in current records. arxiv_search(query="Semantics-Preserving Perturbations adversarial training semantics preserving textual attacks SemGrad", max_results=10) Searching arXiv. Searching for papers on semantics-preserving perturbations. Semantics-Preserving Perturbations (SPP) are perturbations that change the form of an input while preserving its underlying meaning, label, clinical interpretation, or observable behavior. Across the literature, the term denotes a family of domain-specific constructions rather than a single formal object: in textual adversarial attacks, SPP is enforced by sentence-level similarity constraints; in adversarial training for vision, it denotes perturbations that avoid class-specific features; in robustness verification, it is parameterized by semantic variables such as hue, saturation, lightness, brightness, contrast, and rotation; in LLM uncertainty estimation, it is formulated as infinitesimal changes in a semantic embedding space; and in code intelligence, it denotes transformations that preserve program behavior while altering syntax or layout (Herel et al., 2022, Lee et al., 2020, Mohapatra et al., 2019, Li et al., 6 May 2026, Uhm et al., 13 Feb 2026, Pirouzkhah et al., 16 Feb 2026).

1. Formalizations across domains

The core formalization in textual attacks is sentence-pair similarity. In "Preserving Semantics in Textual Adversarial Attacks" (Herel et al., 2022), an original sentence $S$ and a perturbed candidate $S^*$ are embedded with a Semantics-Preserving-Encoder (SPE), and semantic similarity is defined by cosine similarity,

$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$

A candidate is said to preserve semantics whenever $\text{Sim}(S,S^*)>\epsilon$, with the paper finding $\epsilon=0.95$ works well across tasks. In "Generating Textual Adversaries with Minimal Perturbation" (Zhao et al., 2022), the same general concern is expressed as a constrained optimization that minimizes the number of substituted tokens $|\delta|$ subject to misclassification and a semantic-distance bound $d_{\mathrm{sem}}(x,x+\delta)\le\tau$, with semantic similarity computed using the Universal Sentence Encoder (USE). In SASSP, the admissible set is explicitly

$$\mathrm{SPP}(X)=\{X' \mid \mathrm{sim}(X,X')\ge\tau_{\mathrm{sim}} \land \mathrm{para}(X,X')\ge\tau_{\mathrm{para}}\},$$

combining embedding similarity and paraphrase detection (Waghela et al., 2024).

In vision, SPP is formulated by restricting perturbations away from class-defining features. "Semantics-Preserving Adversarial Training" (Lee et al., 2020) defines a perturbation $\delta$ as a small change satisfying the usual adversarial-budget constraint $\|\delta\|_p\le\epsilon$ while not altering task-relevant features; in its strongest form,

$S^*$0

"Towards Verifying Robustness of Neural Networks Against Semantic Perturbations" (Mohapatra et al., 2019) instead parameterizes perturbations by a semantic mapping $S^*$1 and defines the attack magnitude in the parameter space by $S^*$2, where semantic variables include hue shift, saturation shift, lightness shift, brightness, contrast, and rotation.

Other domains generalize the same idea. In SemGrad, a perturbation $S^*$3 is semantics-preserving if $S^*$4 is small and the ground-truth response distribution satisfies $S^*$5 (Li et al., 6 May 2026). In privacy-preserving text distortion, SPP replaces a sensitive span with a new span $S^*$6 that minimizes Neighboring Distribution Divergence (NDD) while ensuring the original sensitive tokens do not appear in the perturbed sentence (Li et al., 2022). In binary code similarity detection and automated code revision, semantic preservation is stated directly in behavioral terms: the transformed code must not change the program’s observable behavior, and for Java methods the perturbation functions satisfy $S^*$7 for all inputs and observable side-effects (Uhm et al., 13 Feb 2026, Pirouzkhah et al., 16 Feb 2026).

Domain	Perturbation object	Preservation criterion
Textual attacks	Sentence edits or token substitutions	Cosine similarity, USE similarity, paraphrase probability
Vision robustness	Pixel perturbation or semantic parameter vector	Avoid class-specific features; preserve top-1 label
LLM uncertainty	Semantic embedding perturbation	Stable response distribution under semantically equivalent input changes
Privacy protection	Sensitive-span replacement	Minimal NDD with no leakage of original span
Code intelligence	Semantics-preserving program transformations	Unchanged observable behavior or execution

2. Textual adversarial attacks

A central textual finding is that many nominally successful attacks fail the semantic-preservation requirement. "Preserving Semantics in Textual Adversarial Attacks" reports that up to 70% of adversarial examples generated by adversarial attacks should be discarded because they do not preserve semantics (Herel et al., 2022). To address this, the paper proposes SPE, a fully supervised sentence embedding technique formed as an average of $S^*$8 independently trained, task-supervised fastText classifiers. The selected tasks are SNLI, RTE, CoLA, SST-2, CARER, Yelp binary polarity, and StackOverflow question classification. Each classifier produces a hidden-layer sentence embedding $S^*$9, and the final representation is

$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$0

Integrated into TextFooler and TFAdjusted, SPE filters candidates before any target-model query: if $$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$1, the candidate is discarded. On offensive tweets, the paper reports human-verified real attack success rate (rASR) of 30.7% for TextFooler + SPE versus 20.4% with USE and 18.6% with BERTScore; on hate-speech tweets, 45.9% versus 27.0% and 37.5%; on Rotten Tomatoes, 61.4% versus 41.5% and 38.9%; and on Yelp reviews, 36.7% versus 7.2% and 7.2%. Averaging these four datasets gives a 1.2x–5.1x improvement in rASR, while average generation time per sentence on TextFooler is 0.75 s with SPE versus 0.82 s with USE and 0.99 s with BERTScore (Herel et al., 2022).

TAMPERS approaches the same problem from the perturbation side rather than the encoder side. Its explicit objective is to find adversarial texts with high similarity to the original texts while introducing minimal perturbation, measured by the number of substituted tokens and the perturbation rate

$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$2

The framework first performs search-space reduction by scoring each content word with

$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$3

then greedily constructs an initial adversary, and finally uses a genetic algorithm to reduce the number of substitutions while preserving attack success (Zhao et al., 2022). On IMDB, Yelp, MR, and SST-2 against a fine-tuned BERT$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$4, TAMPERS achieves success rates of 99.6%, 95.9%, 93.4%, and 93.5%, with perturbation rates of 2.3%, 4.3%, 9.6%, and 10.1%, respectively. On IMDB, runtime is 98.4 s per sample, versus 2477 s for Genetic Attack and 1495 s for PSO (Zhao et al., 2022). The paper explicitly motivates this by the rationale that adversarial texts with small perturbation can better preserve the semantic meaning of original texts.

SASSP adds a more elaborate candidate-selection mechanism. It combines gradient saliency

$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$5

attention-derived token importance $$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$6, and a composite score $$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$7 to determine perturbation sites, then accepts candidates only if both the sentence-embedding similarity and the paraphrase probability exceed their thresholds (Waghela et al., 2024). Using Sentence-BERT for embedding similarity and a fine-tuned Sentence-Transformer for paraphrase detection, SASSP is reported to outperform CLARE on Yelp, AG News, MNLI, and QNLI. On Yelp, ASR rises from 79.7% to 82.6%, WMR falls from 10.3% to 8.7%, perplexity drops from 83.5 to 74.2, and semantic similarity rises from 0.78 to 0.85; on QNLI, ASR rises from 83.8% to 93.2% and semantic similarity from 0.78 to 0.86 (Waghela et al., 2024). At the same time, the paper notes computational overhead from back-propagation, attention extraction, and multiple semantic tests, as well as sensitivity to hyperparameters and reliance on external embedding and paraphrase models.

3. Vision: adversarial training, joint perturbations, and certification

In adversarial training, SPP is used to control which perturbation directions are admissible during inner maximization. SPAT replaces the standard inner-loss surrogate with label-smoothed cross-entropy (LSCE),

$$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$8

where $$\text{Sim}(S,S^*)=\frac{e(S)\cdot e(S^*)}{\|e(S)\|\,\|e(S^*)\|}.$$9 for the true class and $\text{Sim}(S,S^*)>\epsilon$0 otherwise, and generates adversarial examples by PGD-LS rather than PGD-CE or PGD-KL (Lee et al., 2020). The paper’s intuition is that label smoothing implicitly discourages perturbations that erase true-class features and instead encourages changes along directions shared among classes. With a WideResNet-34-10 backbone, $\text{Sim}(S,S^*)>\epsilon$1, and PGD-10 training attacks, the reported CIFAR-10 robust accuracies are: Madry 56.4 under PGD-20 and 54.1 under CW$\text{Sim}(S,S^*)>\epsilon$2; TRADES 57.3 and 54.4; MART 58.5 and 53.3; TRADES+SPAT ($\text{Sim}(S,S^*)>\epsilon$3) 58.2 and 55.0; MART+SPAT ($\text{Sim}(S,S^*)>\epsilon$4) 59.6 and 51.6. On CIFAR-100, MART+SPAT improves PGD-20 from 34.1% to 34.7% and CW$\text{Sim}(S,S^*)>\epsilon$5 from 30.2 to 30.9%. The paper also states that SPAT examples exhibit higher SSIM and lower LPIPS than PGD-CE at the same $\text{Sim}(S,S^*)>\epsilon$6 (Lee et al., 2020).

SPA extends the semantics-preserving idea to joint perturbations in attribute space and $\text{Sim}(S,S^*)>\epsilon$7 noise space. Its attack uses an attribute manipulator $\text{Sim}(S,S^*)>\epsilon$8 and an adversarial noise generator $\text{Sim}(S,S^*)>\epsilon$9 to solve

$\epsilon=0.95$0

Geometric transformations are modeled with a Spatial Transformer Network surrogate; object-level transformations use a conditional generator via Matrix Subspace Projection (Gao et al., 2023). On FashionMNIST under SPA attack, naive accuracy is 0.0093, compared with 0.0165 for PGD-Attr and 0.0284 for L2LDA-Attr. For defense, accuracy under SPA attack is 0.7996 for SPA-train, versus 0.6982 for PGD-Attr, 0.7731 for L2LDA, 0.5371 for PGD-adv-train, and 0.5084 for CW-adv-train (Gao et al., 2023). The framework’s notion of semantic preservation is tied to class-irrelevant natural attributes such as scaling, rotation, and smiling.

Verification work makes SPP explicit as a threat model rather than as a training heuristic. Semantify-NN inserts semantic perturbation layers (SP-layers) between the input and the classifier so that the composed model $\epsilon=0.95$1 can be analyzed by any off-the-shelf $\epsilon=0.95$2-norm verifier capable of handling ReLU, clamp, and absolute-value operations (Mohapatra et al., 2019). The paper gives constructive SP-layers for hue, saturation, lightness, brightness, contrast, and rotation, and then tightens certificates with explicit and implicit input splitting. On hue shift for a CIFAR MLP 6×2048, the reported certified radii are approximately 0.00316 for naïve $\epsilon=0.95$3 conversion, 0.028 for weighted-$\epsilon=0.95$4, 0.347 for SPL only, and 0.974 for SPL + refine, versus a grid attack bound of 1.456. On rotation for an MNIST MLP 2×1024, SPL + refine with 100 implicit splits and explicit 0.5° bins yields 46.24°, versus a grid attack of 51.42° (Mohapatra et al., 2019). This directly refutes the common conflation of semantic robustness with small $\epsilon=0.95$5 robustness: the paper’s motivating examples are precisely attacks such as color shifting and lighting adjustment that induce large $\epsilon=0.95$6-norm perturbations.

4. Privacy, uncertainty, and representation learning

In privacy-preserving text processing, SPP is used not to fool a classifier but to replace sensitive spans while maintaining downstream utility. "Semantics-Preserved Distortion for Personal Privacy Protection in Information Management" defines the perturbed sentence $\epsilon=0.95$7 by replacing a sensitive span $\epsilon=0.95$8 with a new span $\epsilon=0.95$9 selected to minimize Neighboring Distribution Divergence,

$|\delta|$0

where the proximity weight is $|\delta|$1 (Li et al., 2022). Two frameworks are given: generative distortion using an MLM and substitutive distortion using a phrase pool. On NER, labeled F-score is 89.02 for the unprotected setting, 78.18 for Generative w/ NDD, 87.75 for Substitutive w/o NDD, and 88.71 for Substitutive w/ NDD. On SQuAD 1.0, Sub w/ NDD yields EM 89.6 and F$|\delta|$2 95.0, versus 90.0 and 95.2 for the unprotected model. In the medical privacy setting, A@100 drops from 16% for Deidentified data to 5% for Deid Sub w/ NDD (Li et al., 2022). The paper therefore frames SPP as a data-based privacy defense with a privacy–utility trade-off governed by NDD.

In LLM uncertainty estimation, SPP defines the semantic neighborhood with respect to which local stability is measured. SemGrad first identifies semantic embeddings via the Semantic Preservation Score (SPS),

$|\delta|$3

then computes an uncertainty score from the gradient of a weighted log-likelihood with respect to the semantic embedding,

$|\delta|$4

HybridGrad interpolates between SemGrad and a parameter-gradient analogue using $|\delta|$5 (Li et al., 6 May 2026). On SciQ, TriviaQA, and TruthfulQA with Qwen3-Instruct_4B, Mistral-Nemo-Instruct_12B, and Llama3.1-Instruct_8B, SemGrad and HybridGrad are reported to achieve the highest average AUROC over all datasets and models. On TruthfulQA, SemGrad outperforms SAR by +3.27 points and ParaGrad by +3.3 points, and both SemGrad and HybridGrad require only one forward+backward pass (Li et al., 6 May 2026). Here semantic preservation is not a hard acceptance gate but the basis of a differential sensitivity measure.

A related but distinct use appears in perturbation-based language-model training. "Perturbation is All You Need for Extrapolating LLMs" defines a perturbation mechanism $|\delta|$6 over prefixes and a perturbed autoregressive model

$|\delta|$7

The training objective maximizes expected log-likelihood under perturbed prefixes, and the paper proves adaptivity, robustness, and extrapolability statements for the resulting model class (Cen et al., 5 May 2026). On synthetic bigram data, with $|\delta|$8–0.05, MAE on unobserved pairs drops by up to 30% for $|\delta|$9; on WebText, OPT-125M improves Mauve from 0.44 for MLE to 0.56 for the perturbation-based variant, and on WritingPrompts GPT-2 improves from 0.10 to 0.17 (Cen et al., 5 May 2026). This suggests that SPP can function as a mechanism for support expansion rather than only as an adversarial or defensive constraint.

Medical ultrasound self-supervised learning provides a further non-adversarial interpretation. "The Efficacy of Semantics-Preserving Transformations in Self-Supervised Learning for Medical Ultrasound" defines semantics-preserving transformations as augmentations and preprocessing steps that reflect variations encountered in real acquisitions without altering clinical information (VanBerlo et al., 10 Apr 2025). The pipeline includes probe type change, convexity change, depth change simulation, horizontal flip, rotation + shift, wavelet denoising, CLAHE, gamma correction, brightness & contrast change, speckle noise simulation, multiplicative Gaussian noise, and salt & pepper noise. In expert evaluation on 50 images, 70% of the AugUS-O pairs were judged similar versus 58% for the standard BYOL pipeline. On COVID-19 classification, AugUS-O yields accuracy 0.560 and AUC 0.836, versus 0.454 and 0.784 for BYOL; for pleural-line detection with trainable backbone, AP@50 is 0.141 for AugUS-O and 0.143 for AugUS-D, versus 0.136 for BYOL (VanBerlo et al., 10 Apr 2025). The paper’s task-dependent conclusion is that semantics-preserving augmentation benefits global diagnostic tasks, whereas cropping-based invariance benefits object-centric tasks.

5. Program analysis and code intelligence

In binary code similarity detection (BCSD), SPP is defined by semantic invariance at the machine-code level. Uhm et al. formulate two adversarial objectives: inducing false negatives by maximizing embedding distance between semantically identical variants, and inducing false positives by minimizing distance between semantically different functions, both under a transformation budget $d_{\mathrm{sem}}(x,x+\delta)\le\tau$0 and input-length constraint $d_{\mathrm{sem}}(x,x+\delta)\le\tau$1 (Uhm et al., 13 Feb 2026). The asmFooler suite contains eight semantics-preserving transformations: instruction substitution, intra-basic-block reordering, register preservation code reordering, register reassignment, inter-basic-block reordering, semantic NOP implantation, junk code insertion, and Obfuscator-LLVM passes. The evaluation constructs a dataset of 9,565 binary variants from 620 baseline samples and covers six representative BCSD models. Token-based models are especially brittle: for BinShot, recall under in-place substitution drops from 0.953 to 0.271, and under inter-basic-block reordering from 0.972 to 0.036. In false-positive attacks against BinShot, average ASR is approximately 85% at a budget of 20 instructions and approximately 99.9% at 100 instructions, with mean successful prefix length approximately 14.75 instructions (Uhm et al., 13 Feb 2026). The paper explicitly ties robustness to preprocessing, architecture, and feature selection.

Automated Code Revision (ACR) tools expose a different failure mode: inconsistency under semantically equivalent code variants. The ACR study defines nine SPPs over Java methods, grouped into control-flow, data-flow, and identifier-naming perturbations: If–else swap, Dead Exception Insertion, Dead Variable Assignment, Try-Catch Wrapper, Independent Line Swap, Return via Variable, Def–Use Break, Random Variable Names, and Shuffle Variable Names (Pirouzkhah et al., 16 Feb 2026). The evaluation perturbs 2032 Java methods from real-world GitHub projects, generating over 10K perturbed variants, and measures Exact Match, CodeBLEU, Edit Match, and Relative Edit Error. Across five transformer-based ACR tools, the ability to generate correct revisions can drop by up to 45.3% when presented with semantically equivalent code. On the intersection subset of 464 cases solved by all five tools, If–else swap causes $d_{\mathrm{sem}}(x,x+\delta)\le\tau$2EXM of 45.2 for the LoRA-tuned LLaMA-7B and 40.9 for ChatGPT-3.5 Turbo; Dead Exception insertion causes $d_{\mathrm{sem}}(x,x+\delta)\le\tau$3EXM of 22.7 for DeepSeek V3 (Pirouzkhah et al., 16 Feb 2026). The paper further reports that the closer the perturbation is to the targeted region tagged in the review comment, the more likely an ACR tool is to fail, and that three attention-guiding heuristics—Code Repetition, Inline Commenting, and Chain-of-Thought—yield only marginal improvements.

These code-domain results sharpen the meaning of “semantic preservation.” Unlike textual similarity metrics or image-level perceptual criteria, the code papers require semantic equivalence by construction: for BCSD, transformations preserve functional semantics; for ACR, transformed methods are intended to be well-formed, compilable Java methods with unchanged execution behavior (Uhm et al., 13 Feb 2026, Pirouzkhah et al., 16 Feb 2026). This suggests that SPP in program analysis is closer to a formal equivalence relation than to a similarity heuristic.

6. Evaluation issues, limitations, and open problems

A recurring issue is that semantic preservation is only as reliable as the surrogate used to measure it. In textual attacks, the choice of sentence encoder is decisive: the SPE paper trained more than 40 candidate fastText models and found that performance on the Morris benchmark peaked when $d_{\mathrm{sem}}(x,x+\delta)\le\tau$4 carefully chosen tasks were averaged; adding either too many or too few classifiers degrades quality (Herel et al., 2022). In SemGrad, hidden states with higher SPS yield better uncertainty estimation performance, and the paper reports a strong correlation between SPS and SemGrad performance (Li et al., 6 May 2026). In ultrasound SSL, the authors recommend expert review of positive pairs when possible, and their leave-one-out ablation identified four transformations—Crop & Resize, Color Jitter, CLAHE, and Rotation + Shift—as critical for the distilled pipeline (VanBerlo et al., 10 Apr 2025). These results indicate that SPP is not metric-free; the preservation criterion is itself a learned or engineered object.

Another recurring issue is the tension between minimal perturbation, semantic fidelity, and computational cost. TAMPERS explicitly optimizes for the smallest number of substitutions, but its iterative GA restarts from scratch after each restoration, incurring many black-box queries; the paper suggests that future work could recycle population history or apply more efficient search (Zhao et al., 2022). SASSP adds stronger semantic checks but also introduces computational overhead and sensitivity to hyperparameters $d_{\mathrm{sem}}(x,x+\delta)\le\tau$5 (Waghela et al., 2024). SPAT typically sacrifices a small amount of clean accuracy relative to standard adversarial training while gaining robustness against strong attacks, especially CW$d_{\mathrm{sem}}(x,x+\delta)\le\tau$6 (Lee et al., 2020). In ACR, prompting and formatting hacks often made models even more brittle, leaving the solution as an open research question (Pirouzkhah et al., 16 Feb 2026).

A common misconception is that small perturbations and semantics-preserving perturbations are interchangeable. The literature does not support that equivalence. TAMPERS states only the rationale that adversarial texts with small perturbation can better preserve the semantic meaning of original texts (Zhao et al., 2022). By contrast, Semantify-NN is motivated precisely by semantic attacks such as color shifting and lighting adjustment that induce large $d_{\mathrm{sem}}(x,x+\delta)\le\tau$7-norm perturbations and therefore fall outside the capacity of standard $d_{\mathrm{sem}}(x,x+\delta)\le\tau$8-based verification (Mohapatra et al., 2019). In ultrasound SSL, aggressive cropping benefits local pattern recognition but can remove or shuffle the spatial arrangement needed for global diagnosis (VanBerlo et al., 10 Apr 2025). A plausible implication is that SPP should be understood as a task-conditioned invariance principle rather than as a generic small-change heuristic.

The open problems identified in the literature are correspondingly diverse. For adversarial attacks, better semantic filters and more query-efficient search remain active concerns (Herel et al., 2022, Zhao et al., 2022). For code revision, the open questions include how to build revision models that factor semantic meaning rather than merely pattern-matching, whether strong consistency guarantees can be defined and incorporated into training, and which architectural or training-time interventions can reward invariance under semantics-preserving transformations (Pirouzkhah et al., 16 Feb 2026). For uncertainty estimation, accurate identification of the semantic embedding subspace is essential, since performance depends directly on SPS (Li et al., 6 May 2026). For medical imaging, augmentation policy must remain aligned with the semantic scale of the downstream task rather than transferred unchanged from natural-image practice (VanBerlo et al., 10 Apr 2025). Taken together, these works present SPP less as a single algorithmic recipe than as a general design requirement: perturb only along directions that are irrelevant to the intended semantics, and measure success only after that invariance condition has been enforced.