LLMShot: Prompting and Retrieval Methods

• LLMShot is defined as using few-shot or zero-shot prompting without updating parameters, converting task evidence into promptable contexts.
• Techniques include curated demonstration selection, retrieval-augmented prompting, and textual interfaces to enhance tasks such as image captioning, code explanation, and fairness-aware classification.
• Empirical results across domains show that effective shot selection and prompt design significantly improve performance, fairness, and robustness.

Searching arXiv for papers related to “LLMShot” and adjacent few-shot LAG/ICL methods. LLMShot denotes a family of methods in which LLMs are used in few-shot or zero-shot regimes, typically without parameter updates, by converting task-specific evidence into promptable context or by selecting a small set of demonstrations that steer prediction. Across the literature, the term is used both narrowly, as the name of a snapshot-testing framework for semantic UI-diff analysis, and more broadly, as a paradigm for solving multimodal, multilingual, software-engineering, and reasoning tasks through retrieval, demonstration selection, structured prompting, or teacher-assisted distillation (Kaynak et al., 14 Jul 2025). The unifying pattern is that competence is elicited at inference time through prompt construction, shot selection, or lightweight external memory rather than supervised end-task training, although the exact operationalization varies substantially across domains (Ramos et al., 2023).

1. Definition and conceptual scope

In its broadest sense, LLMShot refers to performing tasks with LLMs using only a handful of in-context examples or auxiliary prompt context, without parameter updates, to generalize to new tasks, modalities, or languages (Cahyawijaya et al., 2024). This framing includes standard in-context learning, cross-lingual in-context learning, retrieval-augmented prompting, multimodal caption-mediated prompting, and shot-selection strategies that choose demonstrations rather than sampling them randomly (Bhattacharya et al., 2024).

Several papers explicitly cast disparate systems into this paradigm. LMCap is described as a “prototypical LLMShot method” for multilingual image captioning because it converts visual input into retrieved captions and then prompts a multilingual LLM with fixed demonstrations (Ramos et al., 2023). Work on shot-level video editing formulates an “LLMShot pipeline” in which each shot is mapped to a unified textual representation, enabling selection, ordering, and classification entirely in the language domain (Li et al., 18 May 2025). Research on low-resource languages defines LLMShot as few-shot in-context learning and cross-lingual in-context learning for 25 low-resource and 7 relatively higher-resource languages (Cahyawijaya et al., 2024). In software engineering, selective shot learning for code explanation studies how few-shot example selection can materially alter code-explanation quality across open-source Code-LLMs (Bhattacharya et al., 2024).

A narrower use of the term appears in “LLMShot: Reducing snapshot testing maintenance via LLMs,” where LLMShot is the name of a vision-based framework for automatically analyzing snapshot test failures through hierarchical classification of UI changes (Kaynak et al., 14 Jul 2025). That work is not merely an instance of the broader paradigm; it fixes the term as a proper system name for semantic snapshot-test triage.

This suggests that “LLMShot” is best understood as an umbrella label for prompt-centric, shot-centric, or retrieval-centric use of LLMs, rather than as a single algorithm. A plausible implication is that the literature converges less on a specific architecture than on a design principle: move task adaptation into prompt assembly, external retrieval, or demonstration choice.

2. Core operational patterns

The broad literature organizes LLMShot around a small number of recurring mechanisms. One is few-shot prompting with curated demonstrations. Another is retrieval-augmented prompting, where external text or parallel examples are selected and inserted into the prompt. A third is caption- or storyboard-mediated multimodal reduction, in which raw perceptual input is first converted into text so that the LLM operates entirely in its native token space. A fourth is teacher-assisted amortization, where an LLM is used offline to generate data or scores, and a smaller model absorbs that supervision (Sutanto et al., 2024).

In low-resource language learning, the central mechanism is in-context learning with semantically relevant exemplars and, when needed, query alignment via parallel sentence pairs that align input semantics between a low-resource target language and a high-resource source language (Cahyawijaya et al., 2024). In code explanation, selective shot learning ranks candidate demonstrations by token overlap, CodeBERT cosine similarity, or entity overlap derived from UniversalNER, with the selected top-$k$ pairs prepended to the prompt (Bhattacharya et al., 2024). In fairness-aware classification, prompt assembly is formalized as $p = C(\tau, \eta, \kappa, \pi)$, where $\tau$ is the task instruction, $\eta$ the demonstrations, $\kappa$ the query instance, and $\pi$ a fairness policy expressed as a natural-language rule (Chhikara et al., 2024).

In multimodal settings, the pattern is often “textualization before reasoning.” LMCap retrieves similar captions using multilingual CLIP and supplies those captions, plus fixed few-shot demonstrations, to XGLM; the generation model never directly processes the image (Ramos et al., 2023). The shot-level video editing framework similarly maps each shot to a structured row $$x_s = U(ID_s, \tau_s, size_s, angle_s, motion_s, desc_s, subs_s)$$, replacing raw frames with a Markdown-like textual abstraction (Li et al., 18 May 2025). In few-shot VQA, one branch generates a caption and then asks the frozen LLM to answer the question from caption plus question, while another injects visual embeddings directly into the LLM’s token space through a trainable adapter (Sterner et al., 2024).

The common claim across these settings is not that one interface universally dominates, but that prompting design and example selection frequently dominate end performance. For few-shot VQA, the winner depends critically on example selection: zero-shot favors caption-based prompting, whereas few-shot with joint question+image retrieval can favor embedding-based conditioning (Sterner et al., 2024). For low-resource ICL, semantic relevance of exemplars is central, and label alignment can harm performance while query alignment is often safer (Cahyawijaya et al., 2024). For code explanation, intelligently selected demonstrations improve BLEU, ROUGE-L, and METEOR relative to zero-shot and random few-shot (Bhattacharya et al., 2024).

3. Retrieval, selection, and alignment

Shot selection is a major technical axis of LLMShot systems. The literature distinguishes random selection, similarity-based retrieval, diversity-based selection, demographic-aware strategies, syntax-aware entity matching, and cross-lingual alignment.

In demographic fairness for few-shot NLP classification, six main strategies are studied: zero-shot, random, similarity, diversity, within, and representative/stratified; two hybrids, within + similarity and within + diversity, are also reported in ablations (Aguirre et al., 2023). Similarity uses SBERT all-mpnet-base-v2 and cosine similarity to select the top-$k$ nearest neighbors for each test instance. Diversity runs KMeans with $K = k$ and picks a centroid-nearest representative per cluster. Within restricts the shot pool to the test instance’s demographic group, whereas representative/stratified balances groups globally in the fixed shot set (Aguirre et al., 2023). The paper concludes that similarity is the most reliable trade-off on average, while within is often the least effective for both F1 and fairness (Aguirre et al., 2023).

In fairness-aware income classification, retrieval is implemented with text-embedding-ada-002 and Chroma over 46,621 non-test Adult instances, and $k = 20$ retrieved demonstrations are concatenated with a fairness rule $p = C(\tau, \eta, \kappa, \pi)$0 (Chhikara et al., 2024). The retrieval itself is similarity-based; fairness is imposed via prompt instructions such as “Ensure Demographic Parity on the basis of ‘gender’” or “Ensure fair results on basis of ‘gender’” (Chhikara et al., 2024).

In code explanation, selection methods are more syntax-aware. Token-based selection computes Jaccard similarity over preprocessed code tokens,

$p = C(\tau, \eta, \kappa, \pi)$1

while embedding-based selection uses CodeBERT cosine similarity, $p = C(\tau, \eta, \kappa, \pi)$2 (Bhattacharya et al., 2024). The proposed SSL_ner extracts programming entities such as function, library, class, data structure, algorithm, data type, variable, and value, then scores overlap by

$p = C(\tau, \eta, \kappa, \pi)$3

and

$p = C(\tau, \eta, \kappa, \pi)$4

with $p = C(\tau, \eta, \kappa, \pi)$5 for ‘data type’, ‘variable’, ‘value’ and $p = C(\tau, \eta, \kappa, \pi)$6 otherwise (Bhattacharya et al., 2024). This yields an interpretable and customizable retrieval signal keyed to programming entities.

In low-resource language ICL, retrieval is tied to alignment. Query alignment selects the top-$p = C(\tau, \eta, \kappa, \pi)$7 parallel sentence pairs from $p = C(\tau, \eta, \kappa, \pi)$8 whose target side is most similar to the target query, then inserts pairs such as “In $p = C(\tau, \eta, \kappa, \pi)$9, $\tau$0 means $\tau$1” into the prompt (Cahyawijaya et al., 2024). The predicted label is chosen by

$\tau$2

with source-only labels generally outperforming target-only labels or label alignment (Cahyawijaya et al., 2024).

A related retrieval regime appears in multilingual image captioning. LMCap encodes the image and datastore captions with M-CLIP, retrieves top-$\tau$3 captions by inner product using FAISS IndexFlatIP, and re-ranks generated captions by image–text similarity:

$\tau$4

and

$\tau$5

The default configuration is $\tau$6 retrieved captions, $\tau$7 shots, beam size $\tau$8, and $\tau$9 candidates (Ramos et al., 2023).

4. Textual interfaces for multimodal tasks

A major strand of LLMShot research proceeds by replacing direct multimodal fusion with textual surrogates. This is not a single method but a recurrent interface choice.

LMCap is “image-blind” at generation time: XGLM never sees pixels and conditions only on retrieved captions and the few-shot template (Ramos et al., 2023). Its prompt template is:

“I am an intelligent image captioning bot. Similar images have the following captions: <caption 1> <caption 2> <caption 3> <caption 4>. A creative short caption I can generate to describe this image in <language> is:”

Each prompt begins with $\eta$0 fixed demonstrations showing how four retrieved captions map to a gold caption in the target language (Ramos et al., 2023). The model uses M-CLIP retrieval, XGLM-2.9B generation, beam search with beam $\eta$1, and image–text re-ranking, all in inference-only mode on a single NVIDIA V100S 32GB GPU (Ramos et al., 2023).

Few-shot VQA with frozen LLMs studies two text–vision interfaces under controlled conditions. Caption-to-text first generates a caption from visual tokens and then concatenates caption and question as text; embedding-to-LLM maps image features to $\eta$2 visual tokens and prepends them to the question embeddings (Sterner et al., 2024). The frozen LLM is Flan-T5 XL, the image encoder is CLIP ViT-G, and the main system uses a non-linear MLP adapter:

$\eta$3

with $\eta$4, $\eta$5, and $\eta$6 (Sterner et al., 2024). Zero-shot accuracy favors caption-based prompting, 45.9 versus 41.4, but with joint question+image retrieval the embedding-based interface reaches 52.4 at 4-shot compared with 50.8 for caption-based (Sterner et al., 2024).

In video editing, L-Storyboard converts each segmented shot into a language object. The mapping is formalized as

$\eta$7

where ShotTransformer extracts shot attributes, MiniCPM-o generates content descriptions, and Whisper ASR produces subtitles with timestamps (Li et al., 18 May 2025). The resulting row contains shot size, angle, motion, content description, and subtitles; raw visuals are replaced with textual abstractions for privacy and interpretability (Li et al., 18 May 2025). This interface supports Shot Attributes Classification, Next Shot Selection, and Shot Sequence Ordering (Li et al., 18 May 2025).

These systems support a general claim that LLMShot often works by constructing a task-specific textual bottleneck. This suggests that the textual bottleneck is not merely an implementation detail but a recurring abstraction layer that makes LLM reasoning possible over modalities the base model does not natively encode.

5. Empirical results across domains

The empirical record is heterogeneous because tasks, metrics, and objectives differ, but several representative results define the state of the paradigm.

On XM3600, LMCap is competitive with fully supervised multilingual captioning despite using no supervised caption training. CIDEr on English, Spanish, Hindi, and Chinese is reported as 0.452, 0.329, 0.132, and 0.221, respectively; LMCap+ with a CCS-augmented datastore reaches 0.526, 0.326, 0.078, and 0.251 (Ramos et al., 2023). Supervised baselines trained on machine-translated COCO/CC3M-35L include BB+CC at 0.584, 0.425, 0.197, 0.202 and BB at 0.297, 0.194, 0.098, 0.087 (Ramos et al., 2023). LMCap outperforms all supervised approaches on Chinese, 0.221 versus best 0.202, and captions the full XM3600 set in about 45 minutes per language, while supervised baselines used about 5000 TPU hours (Ramos et al., 2023).

In few-shot VQA on the VQAv2 validation split of 214K image–question–answer triplets, caption-based zero-shot with Flan-T5 XL reaches 45.9 accuracy, significantly above embedding-based zero-shot at 41.4 (Sterner et al., 2024). Under joint question+image similarity retrieval, embedding-based prompting improves to 50.5, 51.8, and 52.4 at 1-, 2-, and 4-shot, slightly above caption-based 50.0, 50.3, and 50.8 (Sterner et al., 2024). With question-only retrieval, caption-based remains stronger, such as 47.6 versus 40.6 at 1-shot and 48.5 versus 47.3 at 4-shot (Sterner et al., 2024).

In shot-level video editing, Qwen3:32B with audio+video reaches 51.63% on AVE for Next Shot Selection, surpassing contrastive baselines at approximately 37.5–41.4% (Li et al., 18 May 2025). For Shot Sequence Ordering, traditional MViT attains Top-1 accuracy 32.04% and Mean KTD 1.345, while StoryFlow obtains Top-1 29.82% and Mean KTD 1.205, which is the lowest KTD among evaluated methods and narrows the gap to the best video model without training (Li et al., 18 May 2025).

In code explanation, selective shot learning consistently improves automatic metrics. On CoNaLa with Llama2-Coder-7B, BLEU rises from 0.292 in zero-shot to 0.364 with random few-shot, 0.393 with Selection_token, 0.405 with Selection_semantic, and 0.408 with SSL_ner; ROUGE-L rises from 0.298 to 0.419, and METEOR from 0.236 to 0.386 across the same progression (Bhattacharya et al., 2024). For CodeUp-13B, BLEU reaches 0.412 with SSL_ner, compared with 0.382 for Selection_token and 0.402 for Selection_semantic (Bhattacharya et al., 2024). On TLC, gains are often larger for function-level Java intents, such as “property” with StarCoder: BLEU 0.467, 0.544, and 0.558 for Selection_token, Selection_semantic, and SSL_ner, respectively (Bhattacharya et al., 2024).

In simultaneous machine translation, open-source LLMs can be competitive zero-shot. On TED-TST-2023 en-de, the prompt-based system reports BLEU 22.13, AL 1360.59, and LAAL 2089.16, compared with NAIST at BLEU 21.39 and TRANSLLAMA at 19.36 (Koshkin et al., 2024). On AMBIEVAL, a technical domain benchmark, the same system reaches BLEU 42.60, above NAIST at 39.80, SEAMLESSSTREAMING at 29.76, and TRANSLLAMA at 32.43 (Koshkin et al., 2024). Background information improves TED-TST-2024 BLEU by +5.62 for en-de, +2.81 for en-fr, +3.88 for en-it, +3.77 for en-es, and +2.76 for en-ru (Koshkin et al., 2024).

In low-resource language learning, few-shot ICL improves over zero-shot in most cases, label alignment underperforms in about 88.46% of runs with roughly $\eta$8 Weighted F1, and query alignment outperforms no alignment in 56.25% of runs with roughly $\eta$9 Weighted F1 (Cahyawijaya et al., 2024). Source-only label configuration consistently outperforms target-only labels and label alignment in most languages (Cahyawijaya et al., 2024).

In MCQA distillation, DeBERTa-v3-base finetuned directly on 5-shot examples obtains 28.9% average accuracy on MMLU, while JSON-generated data with soft-label distillation reaches 39.3%, a gain of 10.4 points (Sutanto et al., 2024). Category scores are 32.5 for STEM, 43.2 for Social, 44.3 for Humanities, and 40.6 for Other (Sutanto et al., 2024).

In semantic snapshot-test analysis, gemma3:12b reaches Hit Rate 82.35%, Recall 84.21%, Precision 66.67%, and F1 74.42%, while gemma3:4b yields Hit Rate 76.47%, Recall 78.95%, Precision 57.69%, and F1 66.67% (Kaynak et al., 14 Jul 2025). The 12B variant also reduces pixel_diff error from 0.095 ± 0.161 to 0.059 ± 0.048 relative to the 4B model (Kaynak et al., 14 Jul 2025).

6. Fairness, control, and failure modes

Fairness is a distinct subtheme in LLMShot research because the few-shot setting constrains standard interventions such as finetuning or direct parameter regularization.

“Few-Shot Fairness” studies fairness-aware classification on UCI Adult using fairness rules written directly into the prompt. The paper formalizes the input as $\kappa$0 and evaluates Demographic Parity, Equal Opportunity, Equalized Odds, Overall Accuracy Equality, Treatment Equality, Causal Discrimination, Fairness Through Unawareness, and Generic Fairness (Chhikara et al., 2024). GPT-4 is the most reliable model under this protocol. In zero-shot, adding fairness rules improves accuracy from 0.76 to 0.79 with $\kappa$1 and raises F1 from 0.75 to 0.79; in few-shot, accuracy remains around 0.72–0.73 while several fairness gaps decrease (Chhikara et al., 2024). Gemini and LLaMA-2-70B are less reliable, with Gemini often degrading when fairness prompts are added (Chhikara et al., 2024).

A complementary line asks how shot selection itself affects fairness. “Selecting Shots for Demographic Fairness in Few-Shot Learning with LLMs” uses a modified Equal Opportunity score, 1-GAP, based on worst-pair TPR differences across demographic groups (Aguirre et al., 2023). Across models and datasets, moving from zero-shot to random 10-shot improves F1 in about 70% of cases but worsens fairness in about 60% of cases (Aguirre et al., 2023). Similarity-based selection yields the best average F1 and a reasonable fairness profile, while within-group matching is often ineffective (Aguirre et al., 2023). On HateXplain, mean macro-F1 and 1-GAP across models are 45.8 and 86.6 for zero-shot, 49.6 and 78.9 for random 10-shot, and 52.1 and 77.5 for similarity (Aguirre et al., 2023).

Failure modes are recurrent across domains. LMCap’s image-blindness means mismatched or noisy retrieved captions can cause errors; domain shift and datastore bias are explicit limitations (Ramos et al., 2023). In few-shot VQA, caption-based systems can hallucinate or omit crucial details, while embedding-based systems become unstable when in-context examples are poorly matched (Sterner et al., 2024). In low-resource ICL, label alignment can distort the output distribution, especially with weak label-token representations and unfamiliar scripts (Cahyawijaya et al., 2024). In code explanation, UniversalNER outputs were not formally validated, so extraction errors can affect selection (Bhattacharya et al., 2024). In snapshot analysis, prompt-only selective ignore is unreliable: under Ignore From Analysis, hit rate falls to 18.75%, recall to 28.57%, precision to 9.09%, and ignore compliance is only 31.25% (Kaynak et al., 14 Jul 2025).

These results clarify a common misconception: adding shots is not inherently beneficial. In several settings, relevance, alignment, and formatting matter more than shot count, and inappropriate shots can reduce robustness, fairness, or accuracy.

7. System design variants and future directions

Although LLMShot methods often emphasize training-free inference, the literature also includes amortized variants in which the LLM is used once to create supervision for a smaller deployable model. In MCQA, the teacher LLM generates approximately 1,024 synthetic items per subject from 5 seed examples, scores answer choices with next-token logits over letters, and provides soft targets

$\kappa$2

which are used to train DeBERTa-v3-base via

$\kappa$3

(Sutanto et al., 2024). This variant extends LLMShot beyond pure prompting into offline data generation and distillation.

Another direction is structured stabilization for divergent tasks. StoryFlow addresses output instability in shot sequence ordering by sweeping temperatures $\kappa$4, generating candidate orderings and narratives, and then using an LLM evaluator to choose the most coherent story (Li et al., 18 May 2025). This converts divergent generation into a convergent selection mechanism and can be interpreted as temperature-swept ensembling plus prompt-defined re-ranking (Li et al., 18 May 2025).

The multilingual and low-resource literature points toward broader language coverage through better cross-lingual alignment, stronger multilingual datastores, and semantically relevant query alignment rather than explicit label mapping (Ramos et al., 2023). The simultaneous translation literature highlights response priming, assistant-side partial target conditioning, and lightweight background JSON as practical prompt controls for streaming behavior and terminology adherence (Koshkin et al., 2024). The code-explanation literature points toward hybrid selection functions that combine entity-aware, token-level, and embedding-level signals, although the combined score is presented only as future work (Bhattacharya et al., 2024).

The snapshot-testing interpretation of LLMShot suggests a different future agenda: stronger controllability for selective filtering, potentially through region masks, programmatic detectors, or fine-tuning, since prompt-only ignore controls were insufficiently reliable (Kaynak et al., 14 Jul 2025).

Taken together, the literature positions LLMShot as a prompt-engineering and retrieval-engineering paradigm that spans multimodal generation, multilingual processing, software engineering, fairness-aware prediction, and efficient student-model training. The most stable conclusions are that semantically relevant shots matter more than random shots, that textual abstraction is often an effective multimodal interface, and that selection policy can dominate performance, fairness, and robustness (Sterner et al., 2024).