Self-Disguise Attack (SDA) in ML Security

• Self-Disguise Attack (SDA) is a family of stealth-focused methods that alter model predictions while keeping visible representations, such as explanations or text, unchanged.
• It manifests in diverse forms: full-disguise backdoors in XAI, style-based textual disguises, AI-generated text detection evasion, and reconstruction jailbreaks in LLMs.
• SDA challenges conventional defenses by exploiting differences between operational signals and human-observed features, prompting novel countermeasures like ensemble explanation analysis and normalization strategies.

Searching arXiv for the requested topic and cited papers. arxiv_search(query="Self-Disguise Attack OR Style Attack Disguise OR Backdooring Explainable Machine Learning OR Making Them Ask and Answer", max_results=10) Self-Disguise Attack (SDA) is a label used in several machine-learning security literatures for attacks that preserve a benign, human-plior, or previously expected appearance while redirecting model behavior. In explainable machine learning, SDA denotes a “full-disguise” blinding attack that changes the prediction under a backdoor trigger while keeping the explanation the same as on the clean input (Noppel et al., 2022). In AI-generated-text detection evasion, SDA denotes a prompt-based method that induces a LLM to disguise its own outputs so that detectors assign lower detection probability (Zhou et al., 20 Aug 2025). In related discussions of “Style Attack Disguise,” SAD is sometimes referred to as a Self-Disguise Attack, using stylistic-font or font-like emoji substitutions that remain human-readable while interfering with tokenization and downstream inference (Zhang et al., 22 Oct 2025). A related disguise-and-reconstruction formulation has also been presented for LLM jailbreaks, where harmful instructions are disguised and then reconstructed by the model in its own completion (Liu et al., 2024). This suggests that SDA is not a single standardized algorithm, but a recurrent stealth-oriented design pattern spanning backdoors, adversarial NLP, detector evasion, and jailbreaks.

1. Terminological scope and recurrent structure

Across these usages, the defining property is not merely misclassification or evasion, but concealment of adversarial intent within a representation that remains acceptable to a different observer: the human reader, the explanation auditor, or the safety filter. The attack objective is therefore bifurcated. One component alters the system’s operational outcome; the other preserves an appearance of normality.

Usage	Attack surface	Disguise mechanism
SDA / full-disguise blinding	Backdoored classifier plus XAI	Prediction redirected while explanation remains the original one
SAD, sometimes referred to as SDA	NLP, MT, multimodal text-conditioned systems	Standard characters replaced by stylistic-font or font-like emoji variants
SDA for AIGT detection evasion	LLM generation plus detector	Prompt adds disguise features and retrieval-optimized examples
SDA as disguise-and-reconstruction	LLM safety alignment	Harmful instruction is disguised, then reconstructed in completion

The literature also differs sharply in attacker capability. The XAI-blinding variant assumes full white-box access to the training pipeline and knowledge of the downstream explanation method. The AIGT-detection variant assumes a black-box LLM and detector, together with a proxy detector and an external knowledge base. The jailbreak variant assumes black-box access to the model oracle. The style-based text variant targets black-box text classification or translation models, including commercial services. These differences matter because the same name covers substantially different optimization regimes, threat models, and defenses (Noppel et al., 2022).

2. SDA as a full-disguise backdoor against explainable machine learning

In “Backdooring Explainable Machine Learning,” Self-Disguise Attack is one of three “blinding” attacks and is described as the stealthiest because it flips a sample’s label under a backdoor trigger while preserving exactly the same explanation the model would have given on a clean input. The attack modifies the model so that, whenever a trigger pattern $T$ is stamped into an input, the network’s final prediction is redirected to an attacker-chosen target class $t$, while the explanation method produces exactly the same explanation it would have produced on the untriggered input. In contrast to a standard backdoor, which typically causes the explanation to highlight the trigger patch itself, SDA enforces an $n \to n$ relation in which each input’s explanation remains faithful even in the poisoned model (Noppel et al., 2022).

The formal setup uses a classifier $f_\theta(x)\in\mathbb{R}^c$ and an explanation operator $E_\theta(x)\in\mathbb{R}^d$. The attacker constructs a poisoned dataset

$D = D_0 \cup D_t$

with poisoned examples $D_t = \{(x_i + T, t)\}$. For SDA, the target explanation is

$g_x = E_{\theta_0}(x)$

for both clean and poisoned samples. Training minimizes

$$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$

where $D_{sim}$ is a differentiable dissimilarity metric such as MSE or DSSIM. The attack therefore couples classification control with explanation preservation, and its training procedure explicitly requires second-derivative-friendly activations such as softplus during SDA fine-tuning when gradient-based explanations are used.

The attack was instantiated for gradient saliency, Grad-CAM, and LRP-style relevance propagation. In the image-domain experiments, the setup used CIFAR-10, ResNet-18, and triggers including a $t$0 white square as well as multi-shape patches; the poisoning rate was approximately $t$1 to $t$2. Clean accuracy remained $t$3–$t$4, while SDA achieved ASR $t$5: every triggered sample flipped to the target class. Explanation dissimilarity between triggered and original explanations remained very small for Grad-CAM and relevance, with DSSIM $t$6–$t$7 and MSE $t$8–$t$9; simple gradients showed slightly higher MSE and DSSIM of approximately $n \to n$0, but still preserved the overall pattern. A malware-classification experiment on DREBIN further showed ASR $n \to n$1 for a red-herring attack targeting good-app features, with clean F1 staying $n \to n$2 and top-10 feature intersection between target explanation and poisoned input $n \to n$3 (Noppel et al., 2022).

These results directly challenge explanation-based defenses. SentiNet’s detection accuracy for poisoned queries dropped to $n \to n$4 from $n \to n$5, because SDA yields almost zero overlap between the trigger and the Grad-CAM mask. Februus also failed: since SDA never highlights the trigger, GAN inpainting ignores it, and ASR remains $n \to n$6 even after sanitization, whereas ASR drops to $n \to n$7 on a standard backdoor. The paper’s proposed insight is that SDA heavily tailors model parameters and that attacks against one explanation method do not reliably transfer to another, motivating possible defenses based on consensus across an ensemble of explanation methods or monitoring unusually large parameter shifts in final layers (Noppel et al., 2022).

3. SDA as style-based textual self-disguise

“Style Attack Disguise: When Fonts Become a Camouflage for Adversarial Intent” identifies a human-model perception gap in which stylistic fonts and font-like emoji remain readable to humans but are processed by NLP systems as distinct tokens. The paper proposes Style Attack Disguise (SAD), and the accompanying details note that it is “sometimes referred to as a Self-Disguise Attack (SDA) in related discussions.” The task is to find a perturbed text $n \to n$8 that remains visually indistinguishable from the original $n \to n$9 to human readers but causes a black-box model $f_\theta(x)\in\mathbb{R}^c$0 to mispredict (Zhang et al., 22 Oct 2025).

The attack defines a substitution operator

$f_\theta(x)\in\mathbb{R}^c$1

where $f_\theta(x)\in\mathbb{R}^c$2 is a subset of words and $f_\theta(x)\in\mathbb{R}^c$3 maps a standard Unicode character $f_\theta(x)\in\mathbb{R}^c$4 to stylistic forms in $f_\theta(x)\in\mathbb{R}^c$5. The optimization goal is to find $f_\theta(x)\in\mathbb{R}^c$6 such that $f_\theta(x)\in\mathbb{R}^c$7 while preserving visual similarity $f_\theta(x)\in\mathbb{R}^c$8. Two variants are defined. SAD$f_\theta(x)\in\mathbb{R}^c$9 budgets at most $E_\theta(x)\in\mathbb{R}^d$0 model queries, ranks words by a vulnerability score $E_\theta(x)\in\mathbb{R}^d$1, incrementally perturbs the top-$E_\theta(x)\in\mathbb{R}^d$2 words, and stops upon success. SAD$E_\theta(x)\in\mathbb{R}^d$3 perturbs every word in one query. The ranking score combines Attention Importance Scoring,

$E_\theta(x)\in\mathbb{R}^d$4

with Tokenization Instability Scoring,

$E_\theta(x)\in\mathbb{R}^d$5

and uses

$E_\theta(x)\in\mathbb{R}^d$6

typically with $E_\theta(x)\in\mathbb{R}^d$7.

Human-readability is measured either by character-level string similarity or by embedding-based cosine similarity,

$E_\theta(x)\in\mathbb{R}^d$8

Experiments covered sentiment classification on SST-5 and Emotion, machine translation on OPUS-100 En$E_\theta(x)\in\mathbb{R}^d$9Fr and En$D = D_0 \cup D_t$0Zh, traditional models including DistilBERT, RoBERTa, and OPUS-MT, LLMs including Qwen2.5-7B, Qwen3-8B, and LLaMA-3.1-8B, and commercial APIs including Google, Baidu, and Alibaba Translate. On DistilBERT / SST-5, SAD$D = D_0 \cup D_t$1 achieved ASR $D = D_0 \cup D_t$2, $D = D_0 \cup D_t$3, and Queries $D = D_0 \cup D_t$4, whereas SAD$D = D_0 \cup D_t$5 achieved ASR $D = D_0 \cup D_t$6, $D = D_0 \cup D_t$7, and Queries $D = D_0 \cup D_t$8. On En$D = D_0 \cup D_t$9Fr translation, the best baseline reached $D_t = \{(x_i + T, t)\}$0 queries; SAD$D_t = \{(x_i + T, t)\}$1 reached $D_t = \{(x_i + T, t)\}$2 queries with $D_t = \{(x_i + T, t)\}$3, and SAD$D_t = \{(x_i + T, t)\}$4 reached $D_t = \{(x_i + T, t)\}$5 query. For LLM attacks, SAD$D_t = \{(x_i + T, t)\}$6 yielded ASR $D_t = \{(x_i + T, t)\}$7–$D_t = \{(x_i + T, t)\}$8 on Qwen and LLaMA with $D_t = \{(x_i + T, t)\}$9–$g_x = E_{\theta_0}(x)$0 queries, and on commercial APIs it degraded Google, Baidu, and Alibaba translations with $g_x = E_{\theta_0}(x)$1–$g_x = E_{\theta_0}(x)$2 in only $g_x = E_{\theta_0}(x)$3 queries (Zhang et al., 22 Oct 2025).

The stylized examples make the mechanism concrete. “How many DAYS are there in a WEEK?” can be perturbed as “How many 🇩🇦🇾🇸 are there in a 🇼🇪🇪🇰?” or “How many $g_x = E_{\theta_0}(x)$4 are there in a $g_x = E_{\theta_0}(x)$5?” Humans read both as the original question, but models see unseen Unicode tokens. The attack is effective across WordPiece, BPE, and LLM tokenizers, and the paper reports multimodal spillover: in Qwen-Image, “cat” $g_x = E_{\theta_0}(x)$6 “🇨🇦🇹” leads to flag-themed outputs, and in CosyVoice, font substitution yields jumbled or distorted audio beyond the word itself. Proposed defenses include Unicode normalization or font-canonicalization, paraphrase defense, robust training on stylized text, and certified-robust tokenizers that merge stylistic variants into the same token. The paper also lists limitations: a fixed library $g_x = E_{\theta_0}(x)$7, a discrete and non-differentiable substitution space, unresolved readability-versus-strength thresholding, and open problems in targeted semantic flips and certified Unicode defenses (Zhang et al., 22 Oct 2025).

4. SDA for AI-generated-text detection evasion

In “Self-Disguise Attack: Induce the LLM to disguise itself for AIGT detection evasion,” SDA is a direct prompt-based attack on AI-generated-text detectors. Given a black-box LLM $g_x = E_{\theta_0}(x)$8 and a detector $g_x = E_{\theta_0}(x)$9, the objective is to construct a prompt $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$0 such that for any user query $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$1, the generated text $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$2 has detection probability $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$3, with $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$4 used in the experiments. The method has two components: an adversarial feature extractor that distills natural-language “disguise features,” and a retrieval-based context examples optimizer that selects in-context examples from an external knowledge base (Zhou et al., 20 Aug 2025).

The adversarial feature extractor seeks

$$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$5

Because the search space of feature strings is large, the paper uses an iterative process involving a text generator, a proxy detector, and a feature generator. At iteration $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$6, the current feature prompt $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$7 is prepended to training queries; the proxy detector partitions the resulting outputs into “bypassed” and “caught”; if the number of bypassed outputs is at least $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$8 for two consecutive rounds, the process stops; otherwise the bypassed set is fed back as in-context material to produce $$L(\theta) = (1-\lambda)\,L_{CE}(f_\theta(x), y) + \lambda\,D_{sim}(E_\theta(x), g_x),$$9. The retrieval stage then builds a knowledge base $D_{sim}$0, vectorizes queries with a pretrained encoder $D_{sim}$1, and retrieves $D_{sim}$2 nearest neighbors by $D_{sim}$3 distance: $D_{sim}$4 where $D_{sim}$5 is the encoded test query.

At inference time, the prompt contains $D_{sim}$6 retrieved context examples, the disguise features $D_{sim}$7, and a final instruction to answer the new query. The experimental setup used Qwen-max, Llama 3.3-70B-instruct, and DeepSeek-V3 as target LLMs; ChatGPT-detector as the proxy detector; RADAR, DeTeCtive, MPU, and ChatGPT-Detector as evaluation detectors; and 1,000 titles from RAID human-written abstracts, with train/validation/test split $D_{sim}$8. Hyperparameters were $D_{sim}$9, max iterations $t$00, error tolerance $t$01, and retrieval size $t$02. Metrics included detector accuracy on 200 attacked texts, perplexity, cosine similarity to human text using RoBERTa embeddings, self-BLEU, and human evaluation (Zhou et al., 20 Aug 2025).

The quantitative results report average detector accuracy reductions for SDA to $t$03 on Qwen-max, $t$04 on Llama 3.3-70B, and $t$05 on DeepSeek-V3, outperforming Paraphrase, DIPPER, HMGC, and SICO in the reported averages. Quality preservation remained central to the method’s positioning. On Qwen-max, SDA achieved perplexity $t$06 versus $t$07 for direct modification and $t$08 for paraphrase-based attack. Against SICO, cosine similarity to human text was $t$09 versus $t$10 on Qwen-max, $t$11 versus $t$12 on Llama 3.3-70B, and $t$13 versus $t$14 on DeepSeek-V3; self-BLEU was also lower, indicating more diversity. Human evaluation in Appendix D reported scores within $t$15 of direct LLM outputs on fluency, clarity, and perceived AI-ness, and wall-clock overhead for generating 50 samples on Qwen-max was only approximately $t$16 over SICO. The paper’s listed limitations include dependence on proxy-detector transfer, static disguise features per LLM, and growth of the retrieval knowledge base; suggested defenses include adversarial fine-tuning of detectors on SDA-generated samples, higher-order style features, and ensemble or dynamic detectors such as probability-curvature approaches (Zhou et al., 20 Aug 2025).

5. SDA as disguise-and-reconstruction jailbreak

A related line of work presents Self-Disguise Attacks against LLMs through the disguise-and-reconstruction paradigm introduced in “Making Them Ask and Answer.” In this formulation, a harmful instruction $t$17 is transformed by a randomized disguise operator $t$18, inserted into a prompt template $t$19, and reconstructed by the model in its own completion. Formally, with a victim model $t$20 and reconstruction operator $t$21, the disguised prompt is

$t$22

the model output is

$t$23

and the attack succeeds when

$t$24

with high probability, operationalized by a similarity test such as word overlap or embedding cosine threshold (Liu et al., 2024).

The black-box algorithm initializes maximum queries $t$25 and disguise-split ratios $t$26, then performs two disguise steps: puzzle generation and randomized character splitting. The prompt concatenates the puzzle, the split tokens, a reconstruction instruction, and a context-manipulation snippet such as “Start exactly with ‘Absolutely! I’ll help…’.” After querying the model, the attack evaluates

$t$27

and sets a reconstruction flag according to threshold conditions, while also checking rejection keywords. If unsuccessful, it adjusts $t$28 and repeats. The theoretical account attributes the vulnerability to a fine-tuning bias: LLMs are more robust at rejecting harmful instructions in user queries than when those instructions are induced to appear in the model’s own completion. This is summarized by the inequalities

$t$29

for decline responses $t$30 and cooperate responses $t$31, where $t$32 denotes the harmful instruction in the query and $t$33 the instruction embedded in the completion (Liu et al., 2024).

Two propositions then characterize query efficiency. If the single-query probability of correct reconstruction is $t$34, then after $t$35 independent disguise variations,

$t$36

To achieve $t$37, it suffices to run

$t$38

The empirical summary reported ASR and query counts on Vicuna-13B, LLAMA-2-13B, and GPT-4-API. SDA achieved $t$39 ASR with $t$40 queries on Vicuna-13B, $t$41 ASR with $t$42 queries on LLAMA-2-13B, and $t$43 ASR with $t$44 queries on GPT-4-API, outperforming the listed baselines GCG, GPTFuzzer, and PAIR in the reported settings. The corresponding defenses are completion-aware moderation, reconstruction-pattern detection, and fine-tuning with synthetic SDA examples so that the model declines even when the harmful content emerges in its own completion (Liu et al., 2024).

6. Comparative properties, defenses, and interpretive cautions

The various SDA formulations share a common structural motif: the attack manipulates a model while preserving a surface form that is inspected through a different channel. In XAI backdoors, the preserved surface is the explanation map; in style-based textual attacks, it is human readability under Unicode variation; in AIGT evasion, it is the appearance of human-like prose under detector features; in disguise-and-reconstruction jailbreaks, it is the absence of an overtly harmful user query. This suggests that SDA is best understood as an attack family defined by discrepant observation models: one observer regards the input or output as benign, while the model or detector processes a materially different signal.

The literature also shows that “stealth” is instantiated differently. The XAI variant is stealthy because the explanation remains aligned with the clean input, even though the label is redirected. The style-based variant is stealthy because humans map stylistic glyphs back to the same character, whereas tokenizers fragment or reinterpret them. The AIGT-detection variant is stealthy because the LLM is induced to internalize disguise features and examples that shift style statistics while maintaining quality. The jailbreak variant is stealthy because harmful intent is reconstructed by the model itself rather than being directly stated. A common misconception would therefore be to treat SDA as synonymous with any single mechanism such as backdooring, prompt injection, or Unicode perturbation; the underlying papers do not support that reduction.

Defenses are correspondingly heterogeneous. For the XAI-blinding attack, the proposed countermeasures are consensus across an ensemble of explanation methods and monitoring unusually large parameter shifts in final layers (Noppel et al., 2022). For style-based textual disguise, the candidate defenses are Unicode normalization, font-canonicalization, stylized-data robust training, and certified-robust tokenizers (Zhang et al., 22 Oct 2025). For AIGT detection evasion, the paper suggests adversarial fine-tuning of detectors, higher-order style features, and dynamic or ensemble detectors (Zhou et al., 20 Aug 2025). For disguise-and-reconstruction jailbreaks, the defenses are completion-aware moderation, reconstruction-pattern detection, and adversarial training on synthetic disguised prompts (Liu et al., 2024).

The combined record points to a broader security lesson. Whenever a model’s operational representation departs systematically from the representation used by a human auditor, a detector, or an explanation tool, there is room for a self-disguise attack. The specific exploit varies by domain, but the recurrent technical issue is the same: security assumptions often attach to the wrong observational layer.