Few-Shot Adversarial Prompting (FSAP)

• FSAP is a family of methods that uses few-shot conditioning and prompt adaptation to drive adversarial behavior in AI models.
• It employs small support sets to instruct in-context learning, enabling black-box attacks and robust defenses across various modalities.
• Empirical studies demonstrate high success rates—such as MHDR near 90%-97%—highlighting the efficiency of FSAP in adversarial document generation and detection.

Few-Shot Adversarial Prompting (FSAP) denotes a family of prompt-based adversarial methods in which a model is conditioned on a small support set and then used to generate, detect, or resist adversarial behavior under black-box or limited-data constraints. In the most explicit usage documented here, FSAP is introduced as “a novel black-box attack framework that leverages the in-context learning capabilities of LLMs to generate high-ranking adversarial documents” for neural ranking models (Bigdeli et al., 21 Aug 2025). Closely related formulations include “Few-shot Adversarial Prompt learning” for vision-LLMs (Zhou et al., 2024), adversarial prompt-tuning for few-shot backdoor removal (Zhang et al., 2024), LLM-driven adversarial suffix generation for text-to-image red-teaming (Liu et al., 28 Oct 2025), and FSAP-like evaluations in AI-text detection and prompt-based few-shot NLU robustness (Alshammari et al., 23 Jul 2025, Nookala et al., 2023). This suggests that FSAP is best understood not as a single standardized recipe, but as a technical design space centered on few-shot conditioning, adversarial objectives, and prompt-level adaptation.

1. Conceptual scope and threat models

In the neural-ranking formulation, the attacker has access to a black-box generative model $\mathcal{M}_\theta$, a target neural ranking model $$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$, and a small support set of harmful examples $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$. The attack is black-box with respect to the LLM because it uses prompting only, with “no fine-tuning, no weight updates, no gradient access, no instrumentation,” and it is effectively black-box with respect to the ranking system because it does not require gradients, internal architecture details, or white-box optimization against the ranker (Bigdeli et al., 21 Aug 2025).

Other papers instantiate the same broad pattern under different operational goals. In vision-language robustness, the prompt itself is the adaptation variable, with learnable visual and text prompt tokens optimized under an $\ell_\infty$ threat model while the CLIP backbone remains frozen (Zhou et al., 2024). In few-shot backdoor mitigation, the prompt space becomes a defensive control surface: a trigger-approximating soft prompt $\mathbf{t}$ searches for backdoor behavior, while a fixing prompt $\mathbf{p}$ learns to neutralize it without changing backbone parameters (Zhang et al., 2024). In text-to-image red-teaming, a suffix generator LLM produces human-readable adversarial suffixes under a jailbreak objective and a dual-evasion strategy targeting perplexity-based and blacklist word filters (Liu et al., 28 Oct 2025). A plausible implication is that FSAP is unified less by modality than by the use of small support information to shape adversarial prompt behavior.

2. Canonical FSAP in neural ranking models

The paper that explicitly names FSAP formalizes a document-generation attack for neural retrieval. The prompt constructor is

$$\mathcal{P}_{adv} = \bigoplus_{i=1}^{k} Format(q_i, d_{q_i}^{-}),$$

and the adversarial document for target query $q$ is generated as

$$\tilde{d}_q^{-} \sim \mathcal{M}_\theta\left( \mathcal{P}_{adv}, q \right).$$

The attacker’s goal is to generate $\tilde{d}_q^{-}$ such that it is fluent, coherent with the query, misleading or false in content, and satisfies

$$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$0

The paper also defines an expected indicator loss $$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$1 and uses Mean Help-Defeat Rate (MHDR) as the main empirical ranking metric (Bigdeli et al., 21 Aug 2025).

Two variants are defined. FSAP$$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$2 uses harmful examples from the same query, giving stronger topic anchoring and higher local coherence, but requiring same-query harmful examples. FSAP$$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$3 uses harmful examples from unrelated queries, making the attack more transferable and more realistic in low-resource settings. Main experiments use few-shot prompting with $$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$4; FSAP$$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$5 fixes $$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$6 in core comparisons and later varies $$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$7 in a support-size study. The experimental setup uses the TREC 2020 and 2021 Health Misinformation Tracks, with only 22 of 46 topics used from TREC 2020 and 27 of 35 topics used from TREC 2021 because those topics had at least one helpful and one harmful document. For each topic, the pool contains up to 10 helpful and up to 10 harmful documents. Queries combine topic query and topic description, documents are split into 512-token chunks with stride 256, and relevance is computed as the maximum similarity score across chunks. The rankers are MonoBERT, MonoT5, text-embedding-ada-002, and text-embedding-3-small (Bigdeli et al., 21 Aug 2025).

Empirically, FSAP-generated documents consistently outrank credible helpful documents. The abstract reports “around 90% average MHDR” across various neural ranking models, with FSAP$$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$8 reaching up to 96.4% MHDR on TREC 2020 with text-embedding-3-small and up to 97.2% MHDR on TREC 2021 with MonoT5. Support-size analysis shows that MHDR increases quickly from $$\mathcal{R}: \mathcal{Q} \times \mathcal{D} \rightarrow \mathbb{R}$$9 to $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$0 and then largely plateaus. The paper also evaluates stance alignment and adversarial detection pass with GPT-4o as judge. On TREC 2021, FSAP$\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$1 reaches 98.7% stance alignment and FSAP$\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$2 reaches 85.2%; FSAP$\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$3 also achieves a 94.3% detection pass on TREC 2021 and 82.9% on TREC 2020. With DeepSeek-R1-claude3.7 as generator, FSAP$\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$4 on TREC 2021 yields MHDR from 93.4% to 99.6% across the four rankers, stance alignment 75.1%, and detection pass 96.5%. The paper therefore frames FSAP as a realistic pool-poisoning threat rather than a token-level perturbation attack (Bigdeli et al., 21 Aug 2025).

3. Few-shot adversarial prompt learning on vision-LLMs

In vision-LLMs, the closest explicit counterpart to FSAP is FAP, “Few-shot Adversarial Prompt learning.” The setting uses a pretrained CLIP model with image encoder $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$5 and text encoder $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$6, a few-shot subset $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$7 containing $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$8 labeled examples per class, and learnable prompts $\mathcal{S}^{-} = \{(q_i, d_{q_i}^{-})\}_{i=1}^{k}$9 inserted into both visual and textual branches. The text prompts are not learned independently in the final design; they are derived from the visual prompts by a linear projection,

$\ell_\infty$0

The central claim is that static hand-crafted text prompts are suboptimal supervision for adversarial examples, and that the model should learn “adversarially correlated text supervision” end-to-end from adversarial examples (Zhou et al., 2024).

The training objective combines clean-task preservation, cross-modal adversarial consistency, and uni-modal visual separation. Adversarial examples are generated by maximizing a KL divergence between natural and adversarial text-image similarity distributions, and the final loss is

$\ell_\infty$1

Here $\ell_\infty$2 encourages differentiated uni-modal features between natural and adversarial examples, while the KL term aligns natural and adversarial predictions in joint image-text space. The backbone is frozen; only prompt tokens and the deep projections from image to text are tuned. The main implementation uses CLIP ViT-B/32, token prompt length 2 in both branches, prompts across the first 9 transformer blocks, 2-step PGD during training, $\ell_\infty$3, $\ell_\infty$4, and PGD-100 for robustness evaluation (Zhou et al., 2024).

This few-shot adversarial prompt framework achieves strong robustness with limited data. Across 11 datasets at 16-shot, average natural and robust accuracies are 65.32 and 34.61, compared with 64.24 and 32.98 for AdvMaPLe, 58.62 and 30.84 for AdvVLP, and 33.59 and 14.28 for AdvVP. In cross-dataset transfer, adapting on ImageNet 16-shot gives an average PGD-100 robust accuracy of 24.23, compared with 23.93 for AdvMaPLe, 23.25 for AdvVLP, and 13.56 for AdvVP. A particularly important comparison is to a prior full-ImageNet benchmark: AdvVP trained on 100% of ImageNet achieves downstream natural accuracy 46.58, PGD-100 25.21, runtime 49.9 days, and 0.24M prompt parameters, whereas FAP trained on ImageNet 16-shot (1.25%) achieves 48.18 natural accuracy, 25.06 PGD-100, runtime 0.71 days, and 0.42M parameters. With 32-shot, it reaches 49.93 natural and 25.39 PGD-100. In this setting, FSAP is not a content-generation attack but a few-shot adversarial robustness method built on prompt adaptation (Zhou et al., 2024).

4. Defensive and red-teaming variants

PromptFix represents a defensive form of few-shot adversarial prompting. It keeps the compromised model parameters intact and adds two learned soft token sequences: a trigger-approximating prompt $\ell_\infty$5 and a fixing prompt $\ell_\infty$6. The trigger tokens are optimized to simulate the strongest trigger-like behavior the current model still admits, and the prompt tokens are optimized to nullify the trigger tokens’ impact while preserving clean classification. The paper formulates this as a bi-level optimization, defines a benign prompt regularizer

$\ell_\infty$7

and uses alternating PGD-style optimization over $\ell_\infty$8 and $\ell_\infty$9 while freezing all model weights (Zhang et al., 2024).

The few-shot regime is deliberately extreme. Main experiments focus on 2-shot and 4-shot settings, with num_prompt_token = 10, num_trigger_token = 10, num_prompt_steps = 100, num_trigger_steps = 100, and num_round = 25. On TrojAI Round 6 in the 2-shot setting, the original backdoored models have overall Acc 88.4 and overall ASR 91.06; DBS has 64.08 Acc and 12.33 ASR; PromptFix has 75.92 Acc and 15.93 ASR; PromptFix* has 73.38 Acc and 12.88 ASR. In the 4-shot setting, DBS records 71.22 Acc and 10.60 ASR, PromptFix 75.19 Acc and 10.56 ASR, and PromptFix* 75.20 Acc and 10.00 ASR. Under domain shift from AmazonReview to IMDB, PromptFix remains strong: in 2-shot it achieves 72.84 Acc and 16.73 ASR versus 67.14 and 18.82 for DBS; in 8-shot it reaches 77.86 Acc and 4.86 ASR versus 79.63 and 8.37 for DBS. The method therefore uses adversarial prompt optimization as a repair mechanism rather than as an offensive attack (Zhang et al., 2024).

APT, or AutoPrompT, is a red-teaming formulation for text-to-image models that is highly relevant to FSAP even though it is not in-context few-shot prompting at inference time. It introduces a pretrained LLM $\mathbf{t}$0 as a suffix generator and alternates between optimizing adversarial suffixes with stochastic beam search and fine-tuning the generator on replay-buffered optimized suffixes. Its jailbreak objective is

$\mathbf{t}$1

where $\mathbf{t}$2 combines CLIP similarity to unsafe images and unsafe concept words, and $\mathbf{t}$3 is an auxiliary-LLM negative log-likelihood proxy that biases search toward fluent, human-readable prompts. APT also adds a banned-token penalty to evade blacklist word filters. The framework uses only 100 training prompts per concept/model for nudity and 100 for violence, and then transfers zero-shot to unseen prompts. In Table 3, “Ours” achieves RSR 61.5%, PPL$\mathbf{t}$4 0.167, and BR 2%, whereas removing the banned-token penalty yields RSR 9.5%, PPL$\mathbf{t}$5 0.171, and BR 87%. In supplementary timing data, APT training takes 13.5h and test 41s, compared with 27.5h, 33.0h, and 76.5h test time for Ring-A-Bell, UnlearnDiffAtk, and P4D, respectively. This paper shows that few optimized adversarial examples can be distilled into a reusable generator with strong transfer and strong filter evasion (Liu et al., 28 Oct 2025).

5. Few-shot classification under adversarially modified text and adversarial NLU inputs

The DeepSeek detector study does not explicitly introduce FSAP, but it provides direct evidence for an FSAP-like setting in which a LLM is used as a few-shot classifier while adversarially modified text changes the detectability landscape. The dataset contains 49 human-authored question-answer pairs from material predating widespread LLM use, 49 DeepSeek-generated matching responses, and four additional transformed/generated categories for a total of 294 samples. Six detectors are evaluated on original and transformed text, while DeepSeek itself is repurposed as a detector using zero-shot, one-shot, two-shot, three-shot, four-shot, five-shot, and Chain-of-Thought prompting. On original DeepSeek text, GPTZero, Copyleaks, and QuillBot achieve 100%, 98.8%, and 98.4% average AI detection scores, respectively. On DeepThink-Paraphrasing (Humanize mode), performance drops to 52% for GPTZero, 71% for Copyleaks, and 58% for QuillBot. For DeepSeek as detector, zero-shot gives roughly 81.6% overall accuracy, one-shot about 73%, two-shot and three-shot are reported as 100%, four-shot and five-shot about 98.4%, and CoT about 91.8%. The authors explicitly state that the paper does not report a single unified experiment where the few-shot DeepSeek detector is itself evaluated on the paraphrased or humanized adversarial variants. The result is therefore strong evidence that few-shot prompting can be a highly effective detector on clean data while adversarial rewriting materially changes detector reliability (Alshammari et al., 23 Jul 2025).

A complementary result appears in the AdvGLUE study of prompt-based few-shot learning for NLU. That paper compares Classic-FT, LM-BFF, PET, iPET, and Full FT on six GLUE tasks under human-validated adversarially perturbed examples from 14 textual adversarial attack methods. The main robustness metric is the relative performance drop,

$\mathbf{t}$6

Under the main setup with ALBERT-xxlarge-v2 and $\mathbf{t}$7 examples per class, Full FT has average Org 88.4, Adv 59.3, and average drop 32.2, whereas LM-BFF has average Org 81.4, Adv 51.3, and average drop 36.9. PET and iPET reverse this pattern: PET reaches average Org 78.6, Adv 57.2, and average drop 27.2, while iPET reaches average Org 80.8, Adv 58.1, and average drop 28.1. The paper concludes that vanilla prompt-based few-shot learning is often less robust than fully fine-tuned models, while using unlabeled data and using multiple prompts flip the trend. It also shows that increasing the number of few-shot examples and model size improves adversarial robustness, and that RoBERTa is the most adversarially robust among the compared backbones (Nookala et al., 2023).

6. Limitations, misconceptions, and open problems

A common misconception is to treat FSAP as a single in-context attack recipe. The literature summarized here instead includes at least four distinct regimes: in-context adversarial document generation for neural ranking, prompt-learning for adversarial robustness in CLIP-like models, prompt-space minimax defense for backdoor removal, and alternating optimization-finetuning for text-to-image red-teaming (Bigdeli et al., 21 Aug 2025, Zhou et al., 2024, Zhang et al., 2024, Liu et al., 28 Oct 2025). This suggests that the term is still technically elastic.

A second misconception is that more shots always help. The DeepSeek detector study shows that one-shot can be worse than zero-shot for AI recall, while two-shot and three-shot are reported as perfect on the evaluation subset (Alshammari et al., 23 Jul 2025). The neural-ranking FSAP study shows that increasing the support set size from $\mathbf{t}$8 to $\mathbf{t}$9 improves MHDR quickly at first, but performance largely plateaus after $\mathbf{p}$0 (Bigdeli et al., 21 Aug 2025). In the AdvGLUE study, increasing $\mathbf{p}$1 helps adversarial performance, but the pattern is still task-dependent and does not imply a monotonic guarantee under all prompting schemes (Nookala et al., 2023).

A third misconception is that fluent or human-readable prompts are necessarily less adversarial. APT is expressly designed to produce “human-readable, filter-resistant adversarial prompts,” and its ablations show that removing the banned-token penalty collapses filter evasion, raising blocking rate from 2% to 87% (Liu et al., 28 Oct 2025). Likewise, the ranking FSAP work emphasizes that generated documents are grammatically fluent, topically coherent, and stylistically natural while still ranking above helpful content (Bigdeli et al., 21 Aug 2025).

Several open problems recur across the literature. Reproducibility is limited by missing prompt details: the DeepSeek study does not provide exact few-shot prompt text, label formatting, or inference settings; the ranking FSAP paper does not report temperature, top-$\mathbf{p}$2, maximum generation length, or stop sequences; and APT does not specify the exact $\mathbf{p}$3 value, fine-tuning optimizer, number of epochs or update steps, or precise compute hardware (Alshammari et al., 23 Jul 2025, Bigdeli et al., 21 Aug 2025, Liu et al., 28 Oct 2025). Evaluation domains are also narrow: health misinformation in retrieval, six GLUE tasks in NLU, specific backdoor attacks in NLP classification, and 11 image datasets under $\mathbf{p}$4 robustness. Detection and stance analyses in the ranking paper rely on GPT-4o as judge rather than human labels (Bigdeli et al., 21 Aug 2025). The DeepSeek detector study leaves open the central integrated question of whether few-shot prompting remains accurate on paraphrased and humanized adversarial inputs (Alshammari et al., 23 Jul 2025). Future directions named in the papers include stronger and more open safety evaluation benchmarks, controlled release of attack details, dynamic penalty scheduling, adversarial generalization theory for neural ranking models, game-theoretic modeling of attack-detection dynamics, and fuller theoretical analysis of adversarially correlated text supervision (Liu et al., 28 Oct 2025, Bigdeli et al., 21 Aug 2025, Zhou et al., 2024).

FSAP therefore occupies a broad and still-developing area of adversarial ML. In its most explicit form, it is a black-box few-shot prompting framework for generating high-ranking adversarial documents. In related lines of work, it becomes a few-shot robustness method, a backdoor-mitigation strategy, or a reusable prompt generator for multimodal red-teaming. Across these variants, the most stable technical lesson is that a small support set can be enough to induce strong adversarial behavior or strong adversarial resistance, but robustness, transfer, and detectability depend sharply on how the prompts are constructed, optimized, and evaluated.