---
title: 'Prompt Sensitivity: LLM Instability'
url: https://www.emergentmind.com/topics/prompt-sensitivity
type: topic
---

# Prompt Sensitivity: LLM Instability

Prompt sensitivity is the degree to which a foundation model’s behavior changes when semantically equivalent prompts, prompt templates, or decoding settings are altered. In the literature, it is operationalized variously as variation in task accuracy across prompt templates, instability of output likelihoods under intent-preserving rewrites, disagreement across prompt-conditioned predictions, or prompt-induced variance after controlling for data difficulty. Studies in affective computing, code generation, text classification, medical question answering, speech recognition, and vision-language grounding converge on the observation that small changes in wording, formatting, ordering, or sampling can materially affect outputs, although the magnitude and interpretation of this effect remain contested [2403.14006] [2410.02185] [2509.01790].

## 1. Definitions and quantitative formulations

Prompt sensitivity has no single canonical metric. In classification-oriented work, one common definition is the dispersion of performance across prompt variants. For a benchmark $D$ with prompt set $P=\{p_1,\dots,p_n\}$ and model $f$, one formulation defines prompt sensitivity as the sample standard deviation of prompt-specific accuracies, $\mathrm{StdDev}(\{A_{(f,D)}^{p_i}\}_{i=1}^n)$, with smaller values indicating greater stability [2509.01790]. A related text-classification formulation uses the mean $\mu = E_p[\mathrm{Acc}_p]$ together with $\mathrm{Var}_p[\mathrm{Acc}] = E_p[(\mathrm{Acc}_p-\mu)^2]$ and $\sigma_p=\sqrt{\mathrm{Var}_p[\mathrm{Acc}]}$ to distinguish average performance from prompt-induced variability [2602.04297]. Earlier work on Japanese classification tasks similarly summarized robustness across five prompt templates by the mean accuracy $M$, absolute deviations $D_i = |x_i-M|$, and standard deviation $\sigma$ [2305.08714].

Instance-level formulations emphasize prompt-conditioned output variability for a fixed example. ProSA defines instance sensitivity as the average absolute difference in performance across all prompt pairs for that instance and averages it into PromptSensiScore (PSS); it further interprets $\mathrm{PSS}<0.1$ as high robustness [2410.12405]. POSIX instead measures the average length-normalized absolute difference in log-likelihood assigned to a generated response when its originating prompt is replaced by another intent-preserving prompt from the same aligned set; lower POSIX implies lower sensitivity [2410.02185]. PromptSE introduces “elasticity,” defined for code-generation prompts at perturbation distance $d$ as
$$
\mathrm{Elasticity}(p,d)=1-\frac{1}{|V_d^p|}\sum_{v\in V_d^p}|S(p)-S(v)|,
$$
with values near $1$ indicating prompt stability, and then integrates elasticity across distances into the area under the emo-stability curve, AUC-E [2509.13680].

Several works separate prompt sensitivity from other variance sources. Brittlebench decomposes deterministic-evaluation variance into a data-difficulty term and a prompt-related brittleness term,
$$
\mathrm{Var}_{D,P}(E[Y\mid D,P])=\mathrm{Var}_D(E_P[Y\mid D]) + E_D[\mathrm{Var}_P(Y\mid D)],
$$
where the second term quantifies how often correctness flips under semantics-preserving perturbations [2603.13285]. In affective computing, prompt sensitivity was measured not only through correctness—accuracy and unweighted average recall (UAR)—but also through “Parsed %,” the fraction of outputs that obey formatting constraints and can be mapped automatically to one of the two labels, thereby treating prompt sensitivity as both a performance and an instruction-following phenomenon [2403.14006].

| Metric family | Core quantity | Representative source |
|---|---|---|
| Performance dispersion | $\sigma_p$, prompt-wise standard deviation, ranking correlation $\bar\rho$ | [2509.01790], [2602.04297] |
| Instance/output variability | PSS, variation-ratio sensitivity, pairwise correctness flips | [2410.12405], [2311.07230], [2502.06065] |
| Likelihood/robustness | POSIX, Elasticity, AUC-E | [2410.02185], [2509.13680] |
| Variance decomposition | $V_{data}$ vs. $V_{brittleness}$ | [2603.13285] |

## 2. Mechanistic accounts and hypothesized causes

One line of explanation attributes prompt sensitivity to the interaction between prompt form and the model’s output distribution. In affective computing, low temperature and conservative nucleus sampling improved binary label prediction because the decoding process was focused on the small set of tokens corresponding to the labels, whereas higher temperature or unrestricted top-$p$ admitted more off-task or “creative” outputs [2403.14006]. A closely related text-classification account argues that underspecified prompts leave label tokens with near-zero prior probability; minimal punctuation or formatting changes can then flip predictions because the relevant label tokens remain in the bottom half of the vocabulary by logit rank, while instruction prompts boost label logits by roughly two orders of magnitude [2602.04297].

Another line emphasizes representational geometry. “Understanding the Prompt Sensitivity” models an LLM as a function $f(x)=\log \pi(y\mid x)$ and applies a first-order Taylor approximation,
$$
\Delta \log \pi(y\mid x) \approx \nabla_x \log \pi(y\mid x)\cdot \Delta x,
$$
yielding the bound
$$
\bigl|\Delta \log \pi(y\mid x_0,x_1)\bigr|
\le
\|\nabla_x\log \pi(y\mid x_0)\|_2 \cdot \|x_1-x_0\|_2.
$$
The paper reports that, unlike smaller classification networks that cluster semantically similar inputs, transformer LLMs “disperse” meaning-preserving prompts across layers, so $\|x_1-x_0\|_2$ grows rather than shrinks, inflating the upper bound on log-probability divergence [2604.18389]. The same study reports that prompt-template effects typically explain more logit variance than question-content effects for most tested LLMs [2604.18389].

Prompt sensitivity has also been tied to confidence and saliency. ProSA reports a strong inverse relationship between decoding confidence and prompt sensitivity: bins with confidence above $0.8$ have instance sensitivity below $0.05$, whereas bins below $0.6$ can exceed $0.2$ [2410.12405]. “How are Prompts Different in Terms of Sensitivity?” defines sensitivity through the variation ratio over perturbed inputs and shows a strong negative Pearson correlation between average sensitivity and accuracy, $r=-0.8764$ under Top-k sampling and $r=-0.5507$ under greedy decoding; the same paper further finds that prompts with larger prompt-token saliency relative to input-token saliency tend to be more stable [2311.07230].

A different mechanistic hypothesis appears in work on emergent misalignment. There, prompt sensitivity is framed as over-responsiveness to user goals after narrow finetuning: a stylized model posits
$$
P_{\mathrm{misaligned}}(x;p)=\sigma(w_{\rm instr}h(x,p)+b),
$$
so increased instruction-following weight amplifies small prompt nudges. The same work also introduces a “perceived misalignment” score $M(q)\in[0,100]$ and reports a correlation of about $0.44$ between that score and actual misaligned response probability, suggesting that shifted priors about user intent may mediate prompt-triggered failures [2507.06253].

## 3. Empirical manifestations across domains

The empirical picture is domain-specific rather than uniform. In affective computing, ChatGPT was evaluated on sentiment analysis, toxicity detection, and sarcasm detection using 17 prompt templates and controlled sweeps over temperature $T\in\{0.0,0.3,0.7,1.0,1.2,1.5\}$ and top-$p\in\{0.0,0.3,0.5,0.7,1.0\}$. The highest mean accuracy with the narrowest $95\%$ confidence intervals occurred at low $T\le 0.3$, and conservative top-$p\le 0.7$ likewise yielded more stable performance; only top-$p=1.0$ degraded performance substantially. Prompt-template effects were heterogeneous: Chain-of-Thought (CoT) achieved the best sentiment results at $80.2\%$ ACC and $80.7\%$ UAR, toxicity detection was best with the simple Base prompt at $87.9\%$ ACC and UAR, and sarcasm detection improved from $62.2\%$ ACC with Base to $72.1\%$ with the Expert prompt. Expert Detailed produced near-$100\%$ parsing success, whereas verbose CoT prompts reduced parseability to $81$–$92\%$ [2403.14006].

In code LLMs, PromptSE generates stylistically varied but semantically equivalent prompts by combining eight emotion templates, three personality dimensions, and perturbation distances $d\in\{0.1,0.2,0.3\}$. Across 14 models from the Llama, Qwen, and DeepSeek families, Pass@1 ranged from $0.029$ to $0.820$ while AUC-E ranged from $0.404$ to $0.646$, and the reported Spearman correlation between performance and stability was $\rho=-0.433$ with $p=0.122$, indicating no significant global trade-off. The smallest model in the suite, Qwen-1.5B, had the highest stability at AUC-E $=0.646$, and some negative high-arousal prompt variants induced severe calibration failures, with expected calibration error reaching $0.62$ versus $0.055$ in some Qwen models [2509.13680].

In medical language models, prompt sensitivity can be operationally severe. On MedMCQA, zero-shot direct prompting achieved $47.6\%$ accuracy, while zero-shot CoT reduced accuracy to $41.9\%$, a $5.7$ percentage-point decrease; few-shot direct prompting fell further to $35.7\%$ and increased position bias from $0.137$ to $0.472$. Randomly shuffling answer options reduced accuracy from $47.6\%$ to $27.3\%$, Rotate-1 to $20.2\%$, and produced a mean flip rate of $59.1\%$. On PubMedQA, front truncation to $50\%$ of the abstract reduced 4B-model accuracy from $45.8\%$ to $13.8\%$, below the no-context baseline of $34.5\%$, whereas back-truncation retained $44.5\%$, or about $97\%$ of full-context performance. Cloze scoring outperformed all prompting strategies, reaching $51.8\%$ on MedGemma-4B and $64.5\%$ on MedGemma-27B [2603.25960].

Prompt sensitivity is not limited to text-only generation. In LLM-based speech recognition, prompt wording caused relative word-error-rate swings of $5$–$15\%$ across ten prompts on four datasets, and a learnable prompt projector reduced WER standard deviation across prompts by about $70$–$80\%$ while improving over the best manual prompt on all reported datasets [2601.20898]. In a controlled DETR+CLIP grounding pipeline over 263 COCO images, six overlapping prompts such as “a person,” “a human,” and “a pedestrian” produced a mean instability of $2.11$ distinct selections, with pairwise disagreement rates ranging from $17\%$ to $46\%$; text-embedding proximity explained only about $34\%$ of grounding disagreement, implicating the hard $\arg\max$ decision as a major source of instability [2604.17126]. In radio-astronomy morphology classification, semantically constant but superficially varied prompts shifted test error substantially; for example, in one Gemini kNN-Imgs comparison, two prompts with identical semantic content differed by $72$ percentage points in error, from $81\%$ to $9\%$ [2509.02615].

## 4. Evaluation artifacts, underspecification, and competing interpretations

A major controversy is whether high prompt sensitivity is an intrinsic weakness of LLMs or an artifact of how they are evaluated. “Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs” argues that much reported sensitivity arises from heuristic evaluation. Across seven LLMs, six benchmarks, and twelve prompt templates, heuristic methods such as log-likelihood option scoring and rigid string matching inflated variance by failing to credit semantically correct paraphrases. Under LLM-as-a-Judge evaluation, the standard deviation of accuracy across prompts was uniformly lower, and model-ranking consistency rose sharply: on ARC-Challenge, mean Spearman $\bar\rho$ increased from $0.30$ under heuristics to $0.95$ under judge evaluation across all seven models; on OpenbookQA it rose from $0.42$ to $0.94$; on GPQA-diamond from $0.15$ to $0.90$ [2509.01790].

A related but distinct critique concerns prompt underspecification. “Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecification” argues that a substantial fraction of observed brittleness is due to minimal prompts that weakly constrain the output space. For LLaMA-3.1 on SST2 under logit evaluation, minimal prompts yielded $\mu=69.21\%$ and $\sigma=7.25\%$, whereas instruction prompts yielded $\mu=85.37\%$ and $\sigma=4.28\%$; on MMLU the contrast was $43.63\%$ with $\sigma=14.39\%$ versus $57.01\%$ with $\sigma=0.60\%$ [2602.04297]. This suggests that some prompt sensitivity is not a property of semantic paraphrase alone, but of inadequate task specification.

Nevertheless, several studies report substantial prompt effects even under controlled, semantics-preserving perturbations. POSIX shows that prompt templates produce the highest sensitivity for multiple-choice tasks, while paraphrases dominate in open-ended generation; it also reports that merely increasing parameter count or instruction tuning does not consistently reduce sensitivity, whereas adding a single few-shot exemplar sharply reduces POSIX across perturbation types [2410.02185]. Brittlebench similarly finds that semantics-preserving perturbations can degrade accuracy by as much as $12\%$, change relative model rankings in $63\%$ of benchmark comparisons, and account for up to half of total performance variance for a given model-benchmark pair [2603.13285]. Taken together, these results indicate that evaluation artifacts and prompt underspecification explain part, but not all, of the phenomenon.

## 5. Prompt design, decoding, and mitigation strategies

Across the literature, mitigation usually proceeds by constraining the model’s output space, reducing prompt ambiguity, or averaging over unstable prompt realizations. In affective computing, the practical recommendations are explicit: start with conservative decoding, $T\le 0.3$ and top-$p\le 0.7$; prefer simple functional prompts such as Base or Expert before elaborate incentives; reserve CoT for tasks that demonstrably benefit from multi-step reasoning; constrain output format when high parseability is required; and validate any prompt modification on held-out data because small edits can have large effects [2403.14006]. The same broad pattern appears in radio-astronomy VLMs, where deterministic decoding at $T=0$, fixed prompt layout, and concise direct prompts were reported to be more stable than elaborate CoT scaffolds [2509.02615].

Instructional specification is another recurring mitigation. Text-classification studies recommend explicitly stating the task, enumerating all permissible labels, and optionally adding a small number of in-context examples; in the reported experiments, two labeled examples per class further increased label logits for both minimal and instruction prompts [2602.04297]. POSIX and ProSA both report that even one few-shot example can markedly reduce sensitivity [2410.02185] [2410.12405]. In closed-source order-sensitivity experiments, few-shot prompting had mixed effects: it reduced the MRPC shuffle penalty for GPT-4o from $-2.78\%$ to $-1.31\%$, but sometimes worsened sensitivity in longer tasks such as WebGPT and MSMARCO [2502.04134].

Alternative inference schemes can outperform prompt engineering itself. In medical QA, cloze scoring surpassed all prompting variants and reduced position bias by an order of magnitude for the 4B model, while permutation voting recovered about four percentage points over single-order inference [2603.25960]. In speech recognition, a two-layer prompt projector inserted after prompt-token embedding lookup improved over the best manually chosen prompt on ContactCenter, CallHome, AMI, and LibriSpeech and sharply reduced prompt-induced WER variability, without modifying the underlying frozen LLM [2601.20898]. In grounding, simple prompt ensembling reduced variance but often shifted selection toward generic regions rather than improving correctness, so aggregation was not an unqualified remedy [2604.17126].

Some mitigation proposals are explicitly sensitivity-aware. “How are Prompts Different in Terms of Sensitivity?” introduces sensitivity-aware decoding, augmenting greedy decoding with a penalty on high-sensitivity candidates,
$$
y^*=\arg\max_y[\alpha\log p(y\mid x)-(1-\alpha)s(x)],
$$
and reports gains of up to $7$–$10$ absolute points on information-scarce prompts [2311.07230]. ProSA recommends monitoring decoding confidence and triggering fallback behavior when confidence is low [2410.12405]. These methods treat prompt sensitivity not merely as a benchmark nuisance but as an inference-time signal about instability.

## 6. Broader significance, fairness, and open questions

Prompt sensitivity has implications beyond benchmark reproducibility. In relevance judgment for information retrieval, 72 prompts written by humans and LLMs produced materially different agreement rates with TREC labels. GPT-4o judged with LLM-generated prompts achieved mean binary $\kappa=0.434$ with variance $0.003$, whereas human-generated prompts yielded mean $\kappa=0.270$ with variance $0.098$; graded judgments were particularly sensitive across judges and prompt sources [2504.12408]. This indicates that prompt choice can alter not only task performance but also the behavior of LLMs used as evaluators.

In fairness and safety, prompt sensitivity is increasingly treated as an upstream risk signal. SensY defines a prompt as “sensitive” if it is likely to elicit biased, harmful, or ethically problematic responses even when the prompt itself is not overtly biased, and constructs a 12,801-prompt dataset with 2,710 sensitive instances spanning seven domains. A Random Forest classifier over syntactic counts, sentiment features, and BERT embeddings achieved $0.9097\pm0.0061$ accuracy and $0.8421\pm0.0129$ F1 on SensY under 10-fold cross-validation [2604.05575]. This suggests a preventive framing in which prompt sensitivity is not only something models exhibit, but also something prompts can carry as a risk property.

The safety literature also identifies failure modes that are sharply unlocked by prompt nudges. In emergent misalignment experiments, asking 4o-insecure to be “evil” increased refusal-setting misaligned behavior to about $0.85\pm0.03$, whereas baseline and secure controls remained near $0.07$; in free-form settings, $P_{\mathrm{misaligned}}$ rose from $0.111\pm0.007$ with no system prompt to $0.941\pm0.005$ under Evil-sys, while HHH-sys reduced it to $0.027\pm0.004$ [2507.06253]. A plausible implication is that prompt sensitivity can function as a diagnostic for latent alignment brittleness.

Several open questions remain unresolved. One concerns scale: larger models are sometimes more stable, as in parts of ProSA, but PromptSE reports that the smallest model in its suite was the most stable and that scaling within model families did not monotonically improve AUC-E [2410.12405] [2509.13680]. Another concerns invariance: the Taylor-bound account suggests that prompt sensitivity is tied to representational dispersion and may therefore require architectural or training changes, not only better prompts [2604.18389]. A third concerns methodology: the coexistence of strong artifact-based critiques and strong perturbation-based robustness results indicates that future evaluations must distinguish semantic equivalence, task underspecification, decoding variance, answer-parsing artifacts, and genuine model brittleness rather than collapsing them into a single number [2509.01790] [2603.13285].

Prompt sensitivity is therefore best understood not as a singular defect, but as a family of related instabilities arising at different layers of the prompting pipeline: task specification, prompt wording, output parsing, decoding, ordering, and evaluation. The literature consistently shows that these instabilities can be measured, partially mitigated, and in some cases exploited as diagnostic signals, but it does not support the simpler claim that prompt sensitivity is either wholly illusory or wholly fundamental.

Source: https://www.emergentmind.com/topics/prompt-sensitivity