Static Prompting with Random Exemplars (SPR)

• Static Prompting with Random Exemplars (SPR) is a prompting regime that randomly samples a fixed number of exemplars from a static pool to construct few-shot prompts across NLP and vision tasks.
• Empirical results demonstrate that SPR significantly improves recall and F1 scores over zero-shot methods while reducing false-positive rates in challenging tasks such as clinical error detection.
• SPR challenges traditional semantic exemplar selection by leveraging stochastic prompt construction and evolutionary prompt discovery, making it a plug-and-play solution for resource-limited and exploratory applications.

Static Prompting with Random Exemplars (SPR) is a prompting regime for large models—especially LLMs and, by extension, visual in-context learners—in which a fixed number $k$ of exemplars is sampled uniformly at random from a static pool to construct few-shot prompts. SPR forgoes semantic or label-aware selection, relying instead on the inductive biases of pretrained architectures to generalize meaningfully from randomly chosen context. This approach yields a marked performance improvement over zero-shot and matches or sometimes surpasses conventional hand-crafted or automatically curated exemplars, with broad empirical support across classification, error detection, structured generation, and multimodal learning. SPR has been systematically studied in contexts such as medical error detection (Ahmed et al., 25 Nov 2025), prompt optimization via random pruning (Wang et al., 22 Jun 2025), and visual in-context learning (Zhang et al., 2023).

1. Formal Definition and SPR Workflow

Let $\mathbb{D} = \{d_1, d_2, \ldots, d_N\}$ denote the full set of available few-shot exemplars, with each $d_i = (x_i, y_i)$ comprising an input (e.g., a sentence or image) and its corresponding supervised label or annotation. SPR constructs the prompt by sampling a subset $E$ of $k$ exemplars from $\mathbb{D}$ uniformly at random:

$$E = \{e_1, e_2, \ldots, e_k\} \subset \mathbb{D}, \qquad \mathbb{P}(E = S) = \frac{1}{\binom{N}{k}}$$

Given a query input $x_{\mathrm{query}}$, the full prompt $P_{\mathrm{SPR}}(E; x_{\mathrm{query}})$ concatenates a fixed instruction, the $k$ exemplars (with formatting and separators), and the new instance to be evaluated or completed. In the clinical error detection setting (Ahmed et al., 25 Nov 2025):

• A single-sentence directive specifies the subtask (e.g., flag error, delineate span, propose correction).
• Exemplars are formatted as "Input: <example>" and "Output: <label/annotation>", separated by a line token (e.g., "###").
• The number of random exemplars $k$ is typically fixed (e.g., $k=5$).
• The sequence of random exemplars is concatenated in the sampled order, with the query input appended at the end for model generation.

This process is illustrated as:

Instruction: Detect whether … ### Input: Example1 Output: Label1 ### Input: Example2 Output: Label2 ### ... ### Input: <x_query> Output:

SPR introduces stochasticity over multiple instantiations; the empirical expectation is evaluated by repeated sampling with different random seeds. This generic framework applies both to language and visual modalities, where images (or image-label pairs) stand in for linguistic context (Zhang et al., 2023).

2. Empirical Findings and Performance Benchmarking

SPR has been evaluated on various modeling regimes, with detailed quantitative reporting in clinical NLP (Ahmed et al., 25 Nov 2025) and ablation studies in prompt compression (Wang et al., 22 Jun 2025). Representative metrics in the medical error setting, averaged over 5 seeds, are summarized below:

Subtask	Metric	Zero-Shot	SPR ($k=5$)	RDP ($k=5$)
Error-Flag Detection	Recall	0.68	0.81 ± 0.02	0.89 ± 0.01
	FPR	0.25	0.17 ± 0.01	0.12 ± 0.01
	F1	0.60	0.75 ± 0.01	0.83 ± 0.01
Error Sentence Extraction	Recall	0.62	0.74 ± 0.03	0.84 ± 0.02
	FPR	0.28	0.20 ± 0.02	0.13 ± 0.02
	F1	0.54	0.69 ± 0.02	0.79 ± 0.02
Error Correction	BLEU	0.25	0.42 ± 0.01	0.53 ± 0.01
	BERTScore	0.48	0.63 ± 0.02	0.72 ± 0.01

SPR yields a substantial recall gain over zero-shot prompting (e.g., from 0.68 to 0.81 in flag detection) and a moderately reduced false positive rate. However, it does not reach the upper bound achieved by retrieval-augmented dynamic prompting (RDP), which leverages semantic matching for exemplar selection. Similar trends are observed in other domains, where SPR matches or surpasses more engineered strategies when coupled with prompt pruning and evolutionary search (Wang et al., 22 Jun 2025).

3. Mechanistic Hypotheses and Interpretability

The surprising effectiveness of SPR challenges the orthodox emphasis on hand-crafted, semantically-rich demonstrations. Three mechanistic hypotheses are proposed (Wang et al., 22 Jun 2025):

• Partial Context Hypothesis: LLMs may rely on a small subset of salient cue tokens (e.g., label words or simple local patterns), ignoring the majority of the prompt.
• Superficial Alignment: Despite alignment with human-like language, models tend to exploit shallow statistical regularities or dataset artifacts; random contexts can incidentally amplify signal over noise.
• Unnatural Language “Secret Codes”: Pruned or random sequences may inadvertently evoke latent motifs or "codes" internalized by the model during pretraining—sometimes outperforming naturalistic prompts.

Empirical ablations demonstrate that even "gibberish" prompts—arbitrary subsequences or noise tokens—achieve nontrivial, sometimes state-of-the-art, performance if prompt optimization algorithms (e.g., PromptQuine) search the random prompt space (Wang et al., 22 Jun 2025).

4. SPR in Visual In-Context Learning

SPR is not confined to text; the principle generalizes to vision models in in-context learning paradigms. In visual settings, random selection of image-label pairs as context, sometimes further perturbed with a learnable "prompt enhancer" around the image border, can drive notable improvements in tasks like foreground segmentation and object detection. For example, the InMeMo framework applies a lightweight, input-agnostic learnable border to in-context random exemplars, yielding significant mIoU improvements (e.g., +7.35 for segmentation, +15.13 for detection) over random or unperturbed baselines (Zhang et al., 2023). This suggests that even in vision, the choice of exemplar is less critical than previously assumed, provided that minor shared adaptations are learned.

5. Automated SPR Discovery and Prompt Compression

Discovering high-performing random or pruned prompts at scale is non-trivial. PromptQuine (Wang et al., 22 Jun 2025) introduces a genetic algorithm for open-ended prompt pruning: a binary mask over tokens is mutated and selected by fitness (measured by prompt-conditioned output accuracy), with evolutionary strategies mitigating premature convergence. PromptQuine yields 7–13 percentage point accuracy gains over standard few-shot prompts for diverse NLP tasks, matching or exceeding more computationally intensive RL-based prompt search methods while converging in minutes.

The critical insight from automated discovery is that label tokens (e.g., “yes”, “no”, class names) are often retained (~90%), and stochastic search exposes deep sensitivity to token-level permutation and presence. Notably, attribution-guided pruning and token saliency do not reliably identify optimal prompts, indicating that successful SPR depends on subnetwork dynamics not captured by local gradient measures.

6. Failure Modes and Limitations

Manual error analysis in error detection tasks (Ahmed et al., 25 Nov 2025) reveals two recurrent SPR failure modes:

1. Subtle Numeric-Unit Confusion: Random exemplars may omit near-miss analogues for specific errors (e.g., dosage or unit mismatches), leading to systematic recall deficits on such cases. For instance, dosage-halving errors were missed by SPR 62% of the time compared to 18% with RDP.
2. Over-flagging Stylistic Variants: Benign paraphrases can be erroneously flagged if superficially similar constructions appear in the random exemplars. For example, the presence of “capsule” in an exemplar led to spurious correction suggestions in tablet/capsule lexical contexts.

SPR also generally lags targeted retrieval on contextually nuanced or rare phenomena. The gain in recall comes with some increase (or only modest reduction) in false positives compared to best-in-class dynamic prompting.

7. Broader Implications, Comparative Analysis, and Future Directions

SPR occupies the mid-point in a spectrum between zero-shot prompting (highest generality, lowest recall) and retrieval-augmented dynamic prompting (maximal context alignment but higher compute and engineering overhead). It is easily deployed ("plug-and-play") where annotated exemplar pools are available and retrieval infrastructure is lacking. The trade-off analysis is summarized as follows (Ahmed et al., 25 Nov 2025):

• Zero-shot: Maximal generality, lowest recall, high false positive rate.
• SPR: High recall, moderate reduction in false positives, low overhead.
• RDP: Highest recall, lowest false positives, highest system complexity.

Empirical findings suggest that in high-stakes evaluative regimes, such as clinical summarization or structured error correction, the extra context sensitivity of RDP can be critical. By contrast, for resource-limited, real-time, or exploratory scenarios, SPR delivers "most of the gain" with minimal system complexity.

Open problems and future directions include:

• Extending SPR with learnable, context-agnostic perturbations in vision or multimodal pipelines (Zhang et al., 2023).
• Automating prompt discovery with open-ended search or evolutionary meta-algorithms.
• Developing theoretical and mechanistic accounts for the emergence of context sensitivity to superficially random or pruned exemplars.
• Exploring the transferability and generalization of prompt-enhanced SPR across domains, especially where exemplar salience is weakly coupled to task difficulty.

SPR and its algorithmic variants call into question the necessity of prompt "naturalness," highlighting a shift toward unsupervised, self-organized prompts in practical LLM deployment (Wang et al., 22 Jun 2025, Ahmed et al., 25 Nov 2025, Zhang et al., 2023).