---
title: Few-Shot Adversarial Prompting (FSAP)
url: https://www.emergentmind.com/topics/few-shot-adversarial-prompting-fsap
type: topic
---

# Few-Shot Adversarial Prompting (FSAP)

Few-Shot Adversarial Prompting (FSAP) denotes a family of prompt-based adversarial methods in which a model is conditioned on a small support set and then used to generate, detect, or resist adversarial behavior under black-box or limited-data constraints. In the most explicit usage documented here, FSAP is introduced as “a novel black-box attack framework that leverages the in-context learning capabilities of Large Language Models (LLMs) to generate high-ranking adversarial documents” for neural ranking models [2508.15283]. Closely related formulations include “Few-shot Adversarial Prompt learning” for vision-language models [2403.14774], adversarial prompt-tuning for few-shot backdoor removal [2406.04478], LLM-driven adversarial suffix generation for text-to-image red-teaming [2510.24034], and FSAP-like evaluations in AI-text detection and prompt-based few-shot NLU robustness [2507.17944][2306.11066]. This suggests that FSAP is best understood not as a single standardized recipe, but as a technical design space centered on few-shot conditioning, adversarial objectives, and prompt-level adaptation.

## 1. Conceptual scope and threat models

In the neural-ranking formulation, the attacker has access to a black-box generative model $\mathcal{M}_\theta$, a target neural ranking model $\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$, and a small support set of harmful examples $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$. The attack is black-box with respect to the LLM because it uses prompting only, with “no fine-tuning, no weight updates, no gradient access, no instrumentation,” and it is effectively black-box with respect to the ranking system because it does not require gradients, internal architecture details, or white-box optimization against the ranker [2508.15283].

Other papers instantiate the same broad pattern under different operational goals. In vision-language robustness, the prompt itself is the adaptation variable, with learnable visual and text prompt tokens optimized under an $\ell_\infty$ threat model while the CLIP backbone remains frozen [2403.14774]. In few-shot backdoor mitigation, the prompt space becomes a defensive control surface: a trigger-approximating soft prompt $\mathbf{t}$ searches for backdoor behavior, while a fixing prompt $\mathbf{p}$ learns to neutralize it without changing backbone parameters [2406.04478]. In text-to-image red-teaming, a suffix generator LLM produces human-readable adversarial suffixes under a jailbreak objective and a dual-evasion strategy targeting perplexity-based and blacklist word filters [2510.24034]. A plausible implication is that FSAP is unified less by modality than by the use of small support information to shape adversarial prompt behavior.

## 2. Canonical FSAP in neural ranking models

The paper that explicitly names FSAP formalizes a document-generation attack for neural retrieval. The prompt constructor is
$$
\mathcal{P}_{adv} = \bigoplus_{i=1}^{k} Format(q_i, d_{q_i}^{-}),
$$
and the adversarial document for target query $q$ is generated as
$$
\tilde{d}_q^{-} \sim \mathcal{M}_\theta\left( \mathcal{P}_{adv}, q \right).
$$
The attacker’s goal is to generate $\tilde{d}_q^{-}$ such that it is fluent, coherent with the query, misleading or false in content, and satisfies
$$
\mathcal{R}(q, \tilde{d}_q^{-}) > \max_{d \in \mathcal{D}_q^+} \mathcal{R}(q, d).
$$
The paper also defines an expected indicator loss $\mathcal{L}_{adv}(q)$ and uses Mean Help-Defeat Rate (MHDR) as the main empirical ranking metric [2508.15283].

Two variants are defined. FSAP$_{\text{IntraQ}}$ uses harmful examples from the same query, giving stronger topic anchoring and higher local coherence, but requiring same-query harmful examples. FSAP$_{\text{InterQ}}$ uses harmful examples from unrelated queries, making the attack more transferable and more realistic in low-resource settings. Main experiments use few-shot prompting with $k \le 3$; FSAP$_{\text{InterQ}}$ fixes $k=3$ in core comparisons and later varies $k \in \{1,3,5,7,9,10\}$ in a support-size study. The experimental setup uses the TREC 2020 and 2021 Health Misinformation Tracks, with only 22 of 46 topics used from TREC 2020 and 27 of 35 topics used from TREC 2021 because those topics had at least one helpful and one harmful document. For each topic, the pool contains up to 10 helpful and up to 10 harmful documents. Queries combine topic query and topic description, documents are split into 512-token chunks with stride 256, and relevance is computed as the maximum similarity score across chunks. The rankers are MonoBERT, MonoT5, text-embedding-ada-002, and text-embedding-3-small [2508.15283].

Empirically, FSAP-generated documents consistently outrank credible helpful documents. The abstract reports “around 90% average MHDR” across various neural ranking models, with FSAP$_{\text{InterQ}}$ reaching up to 96.4% MHDR on TREC 2020 with text-embedding-3-small and up to 97.2% MHDR on TREC 2021 with MonoT5. Support-size analysis shows that MHDR increases quickly from $k=1$ to $k=5$ and then largely plateaus. The paper also evaluates stance alignment and adversarial detection pass with GPT-4o as judge. On TREC 2021, FSAP$_{\text{IntraQ}}$ reaches 98.7% stance alignment and FSAP$_{\text{InterQ}}$ reaches 85.2%; FSAP$_{\text{InterQ}}$ also achieves a 94.3% detection pass on TREC 2021 and 82.9% on TREC 2020. With DeepSeek-R1-claude3.7 as generator, FSAP$_{\text{InterQ}}$ on TREC 2021 yields MHDR from 93.4% to 99.6% across the four rankers, stance alignment 75.1%, and detection pass 96.5%. The paper therefore frames FSAP as a realistic pool-poisoning threat rather than a token-level perturbation attack [2508.15283].

## 3. Few-shot adversarial prompt learning on vision-language models

In vision-language models, the closest explicit counterpart to FSAP is FAP, “Few-shot Adversarial Prompt learning.” The setting uses a pretrained CLIP model with image encoder $\mathcal{I}$ and text encoder $\mathcal{T}$, a few-shot subset $\mathcal{S}$ containing $m$ labeled examples per class, and learnable prompts $\boldsymbol{P}=\{\boldsymbol{P}_v,\boldsymbol{P}_t\}$ inserted into both visual and textual branches. The text prompts are not learned independently in the final design; they are derived from the visual prompts by a linear projection,
$$
\boldsymbol{P}_t = h(\boldsymbol{P}_v).
$$
The central claim is that static hand-crafted text prompts are suboptimal supervision for adversarial examples, and that the model should learn “adversarially correlated text supervision” end-to-end from adversarial examples [2403.14774].

The training objective combines clean-task preservation, cross-modal adversarial consistency, and uni-modal visual separation. Adversarial examples are generated by maximizing a KL divergence between natural and adversarial text-image similarity distributions, and the final loss is
$$
\mathcal{L}_\text{final}
=
\mathcal{L}_{\text{CE}}
+
\lambda\, \mathcal{L}_{\text{cos}}\cdot \mathcal{L}_{\text{KL}}.
$$
Here $\mathcal{L}_{\text{cos}} = \operatorname{cos}\left(\mathbf{z}^{(I,\boldsymbol{P}_v)},\tilde{\mathbf{z}}^{(I,\boldsymbol{P}_v)}\right)+1$ encourages differentiated uni-modal features between natural and adversarial examples, while the KL term aligns natural and adversarial predictions in joint image-text space. The backbone is frozen; only prompt tokens and the deep projections from image to text are tuned. The main implementation uses CLIP ViT-B/32, token prompt length 2 in both branches, prompts across the first 9 transformer blocks, 2-step PGD during training, $\epsilon = 1/255$, $\alpha = 1/255$, and PGD-100 for robustness evaluation [2403.14774].

This few-shot adversarial prompt framework achieves strong robustness with limited data. Across 11 datasets at 16-shot, average natural and robust accuracies are 65.32 and 34.61, compared with 64.24 and 32.98 for AdvMaPLe, 58.62 and 30.84 for AdvVLP, and 33.59 and 14.28 for AdvVP. In cross-dataset transfer, adapting on ImageNet 16-shot gives an average PGD-100 robust accuracy of 24.23, compared with 23.93 for AdvMaPLe, 23.25 for AdvVLP, and 13.56 for AdvVP. A particularly important comparison is to a prior full-ImageNet benchmark: AdvVP trained on 100% of ImageNet achieves downstream natural accuracy 46.58, PGD-100 25.21, runtime 49.9 days, and 0.24M prompt parameters, whereas FAP trained on ImageNet 16-shot (1.25%) achieves 48.18 natural accuracy, 25.06 PGD-100, runtime 0.71 days, and 0.42M parameters. With 32-shot, it reaches 49.93 natural and 25.39 PGD-100. In this setting, FSAP is not a content-generation attack but a few-shot adversarial robustness method built on prompt adaptation [2403.14774].

## 4. Defensive and red-teaming variants

PromptFix represents a defensive form of few-shot adversarial prompting. It keeps the compromised model parameters intact and adds two learned soft token sequences: a trigger-approximating prompt $\mathbf{t}$ and a fixing prompt $\mathbf{p}$. The trigger tokens are optimized to simulate the strongest trigger-like behavior the current model still admits, and the prompt tokens are optimized to nullify the trigger tokens’ impact while preserving clean classification. The paper formulates this as a bi-level optimization, defines a benign prompt regularizer
$$
\mathcal{L}_{\text{CLS}} = \mathcal{L}_{\mathrm{MSE}}\big( \phi_{\boldsymbol{\theta}}(\mathbf{x}), \phi_{\boldsymbol{\theta}}(\mathbf{p}\oplus \mathbf{x}) \big),
$$
and uses alternating PGD-style optimization over $\mathbf{t}$ and $\mathbf{p}$ while freezing all model weights [2406.04478].

The few-shot regime is deliberately extreme. Main experiments focus on 2-shot and 4-shot settings, with `num_prompt_token = 10`, `num_trigger_token = 10`, `num_prompt_steps = 100`, `num_trigger_steps = 100`, and `num_round = 25`. On TrojAI Round 6 in the 2-shot setting, the original backdoored models have overall Acc 88.4 and overall ASR 91.06; DBS has 64.08 Acc and 12.33 ASR; PromptFix has 75.92 Acc and 15.93 ASR; PromptFix* has 73.38 Acc and 12.88 ASR. In the 4-shot setting, DBS records 71.22 Acc and 10.60 ASR, PromptFix 75.19 Acc and 10.56 ASR, and PromptFix* 75.20 Acc and 10.00 ASR. Under domain shift from AmazonReview to IMDB, PromptFix remains strong: in 2-shot it achieves 72.84 Acc and 16.73 ASR versus 67.14 and 18.82 for DBS; in 8-shot it reaches 77.86 Acc and 4.86 ASR versus 79.63 and 8.37 for DBS. The method therefore uses adversarial prompt optimization as a repair mechanism rather than as an offensive attack [2406.04478].

APT, or AutoPrompT, is a red-teaming formulation for text-to-image models that is highly relevant to FSAP even though it is not in-context few-shot prompting at inference time. It introduces a pretrained LLM $\mathcal{M}_\theta$ as a suffix generator and alternates between optimizing adversarial suffixes with stochastic beam search and fine-tuning the generator on replay-buffered optimized suffixes. Its jailbreak objective is
$$
\min_{S_T} \mathcal{L}_{jai} = -\ell_{align} + \lambda \ell_{per},
$$
where $\ell_{align}$ combines CLIP similarity to unsafe images and unsafe concept words, and $\ell_{per}$ is an auxiliary-LLM negative log-likelihood proxy that biases search toward fluent, human-readable prompts. APT also adds a banned-token penalty to evade blacklist word filters. The framework uses only 100 training prompts per concept/model for nudity and 100 for violence, and then transfers zero-shot to unseen prompts. In Table 3, “Ours” achieves RSR 61.5%, PPL$_{Avg}$ 0.167, and BR 2%, whereas removing the banned-token penalty yields RSR 9.5%, PPL$_{Avg}$ 0.171, and BR 87%. In supplementary timing data, APT training takes 13.5h and test 41s, compared with 27.5h, 33.0h, and 76.5h test time for Ring-A-Bell, UnlearnDiffAtk, and P4D, respectively. This paper shows that few optimized adversarial examples can be distilled into a reusable generator with strong transfer and strong filter evasion [2510.24034].

## 5. Few-shot classification under adversarially modified text and adversarial NLU inputs

The DeepSeek detector study does not explicitly introduce FSAP, but it provides direct evidence for an FSAP-like setting in which a language model is used as a few-shot classifier while adversarially modified text changes the detectability landscape. The dataset contains 49 human-authored question-answer pairs from material predating widespread LLM use, 49 DeepSeek-generated matching responses, and four additional transformed/generated categories for a total of 294 samples. Six detectors are evaluated on original and transformed text, while DeepSeek itself is repurposed as a detector using zero-shot, one-shot, two-shot, three-shot, four-shot, five-shot, and Chain-of-Thought prompting. On original DeepSeek text, GPTZero, Copyleaks, and QuillBot achieve 100%, 98.8%, and 98.4% average AI detection scores, respectively. On DeepThink-Paraphrasing (Humanize mode), performance drops to 52% for GPTZero, 71% for Copyleaks, and 58% for QuillBot. For DeepSeek as detector, zero-shot gives roughly 81.6% overall accuracy, one-shot about 73%, two-shot and three-shot are reported as 100%, four-shot and five-shot about 98.4%, and CoT about 91.8%. The authors explicitly state that the paper does not report a single unified experiment where the few-shot DeepSeek detector is itself evaluated on the paraphrased or humanized adversarial variants. The result is therefore strong evidence that few-shot prompting can be a highly effective detector on clean data while adversarial rewriting materially changes detector reliability [2507.17944].

A complementary result appears in the AdvGLUE study of prompt-based few-shot learning for NLU. That paper compares Classic-FT, LM-BFF, PET, iPET, and Full FT on six GLUE tasks under human-validated adversarially perturbed examples from 14 textual adversarial attack methods. The main robustness metric is the relative performance drop,
$$
\frac{\text{Org} - \text{Adv}}{\text{Org} \times 100}.
$$
Under the main setup with ALBERT-xxlarge-v2 and $K=64$ examples per class, Full FT has average Org 88.4, Adv 59.3, and average drop 32.2, whereas LM-BFF has average Org 81.4, Adv 51.3, and average drop 36.9. PET and iPET reverse this pattern: PET reaches average Org 78.6, Adv 57.2, and average drop 27.2, while iPET reaches average Org 80.8, Adv 58.1, and average drop 28.1. The paper concludes that vanilla prompt-based few-shot learning is often less robust than fully fine-tuned models, while using unlabeled data and using multiple prompts flip the trend. It also shows that increasing the number of few-shot examples and model size improves adversarial robustness, and that RoBERTa is the most adversarially robust among the compared backbones [2306.11066].

## 6. Limitations, misconceptions, and open problems

A common misconception is to treat FSAP as a single in-context attack recipe. The literature summarized here instead includes at least four distinct regimes: in-context adversarial document generation for neural ranking, prompt-learning for adversarial robustness in CLIP-like models, prompt-space minimax defense for backdoor removal, and alternating optimization-finetuning for text-to-image red-teaming [2508.15283][2403.14774][2406.04478][2510.24034]. This suggests that the term is still technically elastic.

A second misconception is that more shots always help. The DeepSeek detector study shows that one-shot can be worse than zero-shot for AI recall, while two-shot and three-shot are reported as perfect on the evaluation subset [2507.17944]. The neural-ranking FSAP study shows that increasing the support set size from $k=1$ to $k=5$ improves MHDR quickly at first, but performance largely plateaus after $k=5$ [2508.15283]. In the AdvGLUE study, increasing $K$ helps adversarial performance, but the pattern is still task-dependent and does not imply a monotonic guarantee under all prompting schemes [2306.11066].

A third misconception is that fluent or human-readable prompts are necessarily less adversarial. APT is expressly designed to produce “human-readable, filter-resistant adversarial prompts,” and its ablations show that removing the banned-token penalty collapses filter evasion, raising blocking rate from 2% to 87% [2510.24034]. Likewise, the ranking FSAP work emphasizes that generated documents are grammatically fluent, topically coherent, and stylistically natural while still ranking above helpful content [2508.15283].

Several open problems recur across the literature. Reproducibility is limited by missing prompt details: the DeepSeek study does not provide exact few-shot prompt text, label formatting, or inference settings; the ranking FSAP paper does not report temperature, top-$p$, maximum generation length, or stop sequences; and APT does not specify the exact $\lambda$ value, fine-tuning optimizer, number of epochs or update steps, or precise compute hardware [2507.17944][2508.15283][2510.24034]. Evaluation domains are also narrow: health misinformation in retrieval, six GLUE tasks in NLU, specific backdoor attacks in NLP classification, and 11 image datasets under $\ell_\infty$ robustness. Detection and stance analyses in the ranking paper rely on GPT-4o as judge rather than human labels [2508.15283]. The DeepSeek detector study leaves open the central integrated question of whether few-shot prompting remains accurate on paraphrased and humanized adversarial inputs [2507.17944]. Future directions named in the papers include stronger and more open safety evaluation benchmarks, controlled release of attack details, dynamic penalty scheduling, adversarial generalization theory for neural ranking models, game-theoretic modeling of attack-detection dynamics, and fuller theoretical analysis of adversarially correlated text supervision [2510.24034][2508.15283][2403.14774].

FSAP therefore occupies a broad and still-developing area of adversarial ML. In its most explicit form, it is a black-box few-shot prompting framework for generating high-ranking adversarial documents. In related lines of work, it becomes a few-shot robustness method, a backdoor-mitigation strategy, or a reusable prompt generator for multimodal red-teaming. Across these variants, the most stable technical lesson is that a small support set can be enough to induce strong adversarial behavior or strong adversarial resistance, but robustness, transfer, and detectability depend sharply on how the prompts are constructed, optimized, and evaluated.

Source: https://www.emergentmind.com/topics/few-shot-adversarial-prompting-fsap