---
title: 'Zero-shot LLMs: Insights & Applications'
url: https://www.emergentmind.com/topics/zero-shot-large-language-models-llms
type: topic
---

# Zero-shot LLMs: Insights & Applications

Zero-shot large language models are pretrained or instruction-tuned language models deployed on downstream tasks without task-specific examples, and in many cases without task-specific fine-tuning. In the prompting literature, a prominent formulation is Zero-shot Chain-of-Thought prompting, which showed that a fixed trigger such as “Let’s think step by step.” can elicit intermediate reasoning and substantially improve multi-step problem solving across arithmetic, symbolic, and logical benchmarks [2205.11916]. Subsequent work extended the zero-shot setting far beyond textual reasoning, using prompting, constrained outputs, retrieval scaffolds, program execution, and multimodal context to induce capabilities in retrieval, ranking, classification, translation, forecasting, speech, vision, and even scientific hypothesis generation [2304.14233][2310.07820][2406.13476][2311.05965].

## 1. Definition and operational scope

The zero-shot condition is defined relative to the downstream task rather than to pretraining itself. In the reasoning setting, zero-shot LLMs are pretrained models that can be prompted to solve reasoning tasks without any task-specific examples [2205.11916]. In simultaneous machine translation, zero-shot means that the model is not fine-tuned for SiMT; instead, at each step it conditions on the partial source text, the partial target text, and optional background information, formalized as $p(y_t \mid y_{<t}, x_{<t}, b)$ [2406.13476]. In zero-shot decision tree construction, the tree is built without any training dataset at all, using only feature names, feature types, category values, target descriptions, and branch constraints as prompt context [2501.16247]. In zero-shot ASR domain adaptation, the defining condition is the absence of target-domain speech or text training beyond a single handcrafted domain-specific prompt [2306.16007].

Accordingly, zero-shot use does not imply the absence of structure, metadata, or external tools. Financial document QA remains zero-shot even when the model is asked to generate Python or DSL programs from a passage and table, because the method uses prompting rather than demonstrations or task-specific training [2311.14722]. Zero-shot retrieval remains zero-shot even when BM25 candidate passages are supplied to the LLM as in-domain context for query augmentation [2304.14233]. This suggests that the central invariant across the literature is not “no context,” but “no task-specific supervised adaptation.”

## 2. Prompting, control, and inference scaffolds

Zero-shot performance is highly prompt-mediated. The core mechanism in Zero-shot-CoT is a two-stage prompting pipeline. First, a reasoning extraction prompt of the form `Q: [X]. A: Let's think step by step.` elicits intermediate reasoning. Second, an answer extraction prompt appends the generated reasoning and an answer trigger such as `Therefore, the answer (arabic numerals) is ...` or `Therefore, among A through E, the answer is ...` [2205.11916]. This differs from standard zero-shot prompting, which asks for the answer directly, and from few-shot-CoT, which requires task-specific worked examples.

Other zero-shot systems use tighter output control. In context-aware simultaneous translation, the prompt contains the current partial source text, the already generated partial target text, and optionally compact background information such as a JSON summary of topic and named entities. The LLM greedily emits either a full new target word, which triggers a WRITE action, or the special token `<|eot_id|>`, which is interpreted as a READ action. Response priming places the partial target in the assistant message with a fixed prefix such as `German translation:`, constraining the output space and reducing disclaimers or explanatory text [2406.13476].

Program synthesis is another major zero-shot scaffold. In financial QA, ZS-FinPYT prompts the model to write executable Python code, define variables for the required calculation, and assign the final result to `ans`; ZS-FinDSL uses a dual-prompt pipeline in which natural-language reasoning is first generated and then converted into a constrained DSL program for execution [2311.14722]. In readability assessment, expected-value scoring replaces the single generated score token with an expectation over the probability mass assigned to candidate numeric tokens, and LAURAE combines standardized LLM and readability-formula scores using the model’s self-reported confidence as a weight [2604.24470].

Zero-shot retrieval likewise depends on prompt structure rather than raw generation. LameR retrieves a small set of candidate passages with BM25, inserts them into the LLM prompt as possible answering passages, samples multiple answer-like augmentations, concatenates them to the original query, and retrieves again with BM25. Candidate passages act as in-collection demonstrations of intent, domain, and answer format, even when many of them are incorrect [2304.14233].

## 3. Reasoning, computation, and scientific inference

The most influential empirical result in the area is that prompting alone can expose nontrivial reasoning ability. With InstructGPT (`text-davinci-002`), Zero-shot-CoT raised MultiArith accuracy from **17.7%** to **78.7%**, GSM8K from **10.4%** to **40.7%**, Last Letter Concatenation from **0.2%** to **57.6%**, Coin Flip from **12.8%** to **91.4%**, Date Understanding from **49.3%** to **67.5%**, and Tracking Shuffled Objects from **31.3%** to **52.4%**. Results on commonsense reasoning were mixed: CommonsenseQA dropped from **68.8%** to **64.6%**, whereas StrategyQA rose from **12.7%** to **54.8%** under the right model and prompting setup. The paper emphasizes that the gains are strongest on large models, while smaller models often benefit little [2205.11916].

For numerical reasoning over financial documents, direct zero-shot answering remains brittle because arithmetic errors persist even when the reasoning narrative appears coherent. Program-inducing prompts substantially reduce this problem. On `gpt-3.5-turbo`, ZS-FinPYT improved FinQA from **53.01** with ZS-CoT to **66.52**, ConvFinQA from **52.49** to **67.45**, and TATQA from **74.09** to **85.00**. On `gpt-4`, ZS-FinPYT reached **77.51** on FinQA and **93.00** on TATQA, with the interpreter rather than the LLM carrying out the arithmetic [2311.14722].

Time-series forecasting extends the zero-shot idea by reframing numeric extrapolation as text continuation. A time series is serialized as a string of digits and separators, and the LLM models an autoregressive distribution
$$
p_\theta(U)=\prod_{j=1}^{n} p_\theta(u_j \mid u_{0:j-1}).
$$
The same work derives a continuous density from discrete token probabilities, supports missing values through textual markers such as `NaN`, and adds textual side information without specialized forecasting architecture. Empirically, GPT-3 and LLaMA-2 can be best or second best on MAE and strong on likelihood and CRPS, whereas GPT-4 can underperform GPT-3 because of number tokenization and poor uncertainty calibration attributed to alignment interventions such as RLHF [2310.07820].

Zero-shot scientific inference has also been studied directly. In biomedical hypothesis generation, temporally controlled evaluation shows that LLMs can produce untrained yet validated hypotheses on unseen literature, and zero-shot prompting often outperforms random few-shot prompting for large models. The same study reports that increasing uncertainty can facilitate candidate generation and introduces a multi-agent framework with Analyst, Engineer, Scientist, and Critic roles to widen the hypothesis space [2311.05965].

## 4. Retrieval, ranking, and recommendation

Zero-shot LLMs have been used not only to answer questions, but also to rank documents and items. In LameR, the LLM serves as a query augmenter rather than a direct retriever. On TREC Deep Learning 2019, LameR achieved **69.1 nDCG@10**, compared with **61.3** for HyDE and **50.6** for BM25; on TREC Deep Learning 2020 it achieved **64.8**, compared with **57.9** for HyDE and **48.0** for BM25. It was also best on four of six BEIR datasets in the reported evaluation. The method’s core claim is that LLM-generated answer-like expansions become effective when grounded by in-domain candidate passages and consumed by a literal retriever such as BM25 [2304.14233].

A parallel line of work treats open-source LLMs as zero-shot query likelihood models. Documents are scored by the average log-likelihood of generating the query,
$$
S_{QLM}(\mathbf{q}, d)=\frac{1}{|\mathbf{q}|}\sum_t \log \mathrm{LLM}(q_t \mid \mathbf{p}, d, \mathbf{q}_{<t}),
$$
and ranked accordingly. In this setting, pre-trained-only open models such as LLaMA and Falcon are strong zero-shot rankers: Falcon-40B and FlanT5-11B each reached an average **53.1 nDCG@10**, Falcon-7B reached **52.2**, and LLaMA-7B reached **51.1**. A central finding is that generic instruction fine-tuning often hurts ranking unless question-generation tasks are present in the fine-tuning data. With a hybrid first-stage retriever combining BM25 and HyDE, zero-shot reranking improved further, and just three GBQ examples raised LLaMA-7B to **58.8** average nDCG@10 in the few-shot extension [2310.13243].

Recommendation ranking exhibits a related but more brittle zero-shot behavior. The task is formalized as conditional ranking over a chronological interaction history $\mathcal{H}$ and candidate set $\mathcal{C}$. On random-candidate settings, zero-shot LLM rankers clearly outperformed BM25, UniSRec, and VQ-Rec. For MovieLens-1M, N@20 increased from **33.19** for BM25 and **37.93** for UniSRec to **49.02** with sequential prompting, **50.01** with recency-focused prompting, and **51.62** with sequence-based ICL; on Amazon Games, N@20 reached **59.01** with recency-focused prompting. However, strong supervised recommenders such as SASRec remained substantially better in the easiest setting, and the study found three systematic weaknesses: weak sensitivity to interaction order, popularity bias, and position bias. Recency-focused prompting and three-fold shuffle bootstrapping partially mitigated these effects [2305.08845].

## 5. Classification and predictive modeling over text, tables, and mobility

Prompted LLMs can act as direct zero-shot text classifiers by mapping natural-language instructions to constrained label outputs. On four evaluated datasets, they were effective on three. GPT-4 achieved **0.7133** accuracy on economic text sentiment classification, **0.9000** on e-commerce category classification, and **0.9733** on SMS spam detection, but only **0.5267** on COVID-19 tweet sentiment, where a GRU reached **0.8200**. The paper attributes some variability to prompt and preprocessing sensitivity: cleaning tweets improved GPT-4 while reducing GPT-3.5 and Llama2 performance, and output-format instability remained an issue for some open models [2312.01044].

Domain-specific telecom evaluation yields a more cautious picture. Across SPEC5G-Classification, SPEC5G-Summarization, and TeleQnA, the best zero-shot 7B open models approached but did not generally surpass fine-tuned baselines. Zephyr-7B-β reached **57.93%** accuracy on classification and **Rouge-1 0.4933**, **Rouge-2 0.2425**, **Rouge-L 0.3693** on summarization; Mistral-7B reached **60.93%** accuracy on telecom MCQ QA, compared with **67.11%** for GPT-3.5 and **74.91%** for GPT-4 on the TeleQnA benchmark. No single model dominated across tasks, and many errors reflected instruction-following failures rather than obvious absence of telecom knowledge [2402.15818].

Unsupervised readability assessment and tabular probabilistic prediction show a different facet of zero-shot behavior: the problem is often not raw capability but reliability estimation. In readability assessment, a new prompting methodology outperformed prior methods on **13 of 14 datasets**, and LAURAE robustly outperformed prior methods across languages, text lengths, and technicality by combining LLM and readability-formula scores [2604.24470]. In tabular prediction, a large-scale evaluation across **316 tasks** found that LLM performance is highly variable within datasets and that predicted probabilities are often poorly calibrated, with median ECE around **0.2426** for GPT-4o-mini and **0.1722** for Llama. Yet the spread of unlabeled risk scores is informative: the standard deviation of risk scores is the strongest simple predictor of task-level AUC, and for GPT-4o-mini tasks with risk-score standard deviation at least **0.4** had average AUC around **0.8417**, versus an overall average of **0.7186** [2509.15356].

Zero-shot predictive modeling has also been demonstrated in mobility and symbolic tabular induction. For next-location prediction, GPT-3.5 achieved ACC@5 of **0.298** on Foursquare NYC, **0.324** on Foursquare Tokyo, and **0.279** on a private Ferrara cycling dataset, corresponding to **601%** relative improvement in NYC and **561%** in Tokyo over cross-city transfer baselines. The paper reports that historical visits are generally more informative than short contextual windows, and that one-shot or few-shot prompting does not uniformly improve upon zero-shot across models [2405.20962]. In zero-shot decision tree construction, LLMs propose splits, estimate branch class probabilities, and minimize branch impurity via
$$
\mathrm{Gini}=1-\sum_{i=1}^{C} p_i^2
$$
combined with a harmonic mean across the two branches. The resulting trees require no labeled training data, remain interpretable, outperform TabLLM zero-shot baselines, and are competitive with low-shot supervised decision trees on several tabular datasets [2501.16247].

## 6. Multimodal, speech, and translation settings

Zero-shot use of LLMs is not confined to text-only inputs. In simultaneous machine translation, a cascaded Whisper-plus-Llama-3-70B-Instruct system performed competitively without any SiMT fine-tuning. On English-to-German TED-TST-2023, it achieved **22.13 BLEU** with **AL 1360.59** and **LAAL 2089.16**, outperforming TRANSLLAMA (**19.36 BLEU**), SEAMLESS (**19.75**), and FBK (**17.65**), while slightly exceeding NAIST (**21.39**). On the context-sensitive AMBIEVAL benchmark it reached **42.60 BLEU**, ahead of NAIST (**39.80**), TRANSLLAMA (**32.43**), SEAMLESS (**29.76**), and FBK (**24.96**). Injecting minimal background information consistently improved BLEU on TED-TST-2024 across all five language pairs; for English-to-German, performance dropped from **36.76** to **31.14** when the background was removed. The reported real-time factor of the full system was **0.86** [2406.13476].

In zero-shot image classification, multimodal LLMs have been used to enrich the query representation rather than merely the class labels. One method queries Gemini Pro for an image description and an initial class prediction, embeds both texts with CLIP’s text encoder, fuses them with the original image embedding,
$$
\widetilde{X}_{q} = X_{if} + X_{df} + X_{pf},
$$
and then performs linear similarity classification against embedded class labels. With the same universal prompts across datasets, this produced an average gain of **+4.1 percentage points** over ten benchmarks and **+6.8** on ImageNet; the combined-feature system reached **73.4** top-1 accuracy on ImageNet versus **66.6** for CuPL in the re-evaluated setup [2405.15668].

Speech applications show both the promise and the selectivity of zero-shot gains. In rare-word ASR, an LLM-integrated architecture using Whisper V2, an adapter, and Qwen-7B-Chat improved Primock57 Set 2 rare-word error rate from **40.2%** for Zipformer-Transducer to **32.2%**, while overall O-WER and N-WER remained roughly comparable. The paper’s central interpretation is that the speech encoder mainly determines general transcription quality, whereas the LLM primarily improves rare-word recognition [2502.16142]. In a separate zero-shot ASR domain-adaptation study, prompted LLaMA-7B was used either for second-pass rescoring or for deep LLM-fusion with HuBERT-Large. With only a single domain prompt, both methods reduced WER on TedLium-2 and SPGISpeech, and deep LLM-fusion achieved substantially better entity and OOV recall than rescoring [2306.16007].

## 7. Limitations, misconceptions, and research significance

A persistent misconception is that zero-shot success is uniform across models and tasks. The empirical record is more conditional. Zero-shot-CoT works best on large models and often provides little benefit to smaller ones. In mobility prediction, several tested models, including Phi variants, Gemma 2B, GPT-J, and Dolly, failed to produce usable outputs. In recommender ranking, candidate order and popularity can distort the ranking, and in telecom evaluation formatting failures can materially depress scores even when the answer appears semantically plausible [2205.11916][2405.20962][2305.08845][2402.15818].

Reliability is a second major limitation. Zero-shot tabular prediction shows that model confidence is often informative for relative correctness but not well calibrated in absolute terms, and the authors explicitly recommend unlabeled confidence metrics only as screening tools rather than substitutes for labeled validation, especially in healthcare or finance [2509.15356]. Memorization and contamination concerns also remain. The Zero-shot-CoT work notes that full training-data details are unavailable, so memorization cannot be ruled out completely, while the next-location study addresses the issue more directly through a private dataset, quiz-based contamination tests, and inspection of hallucinated location IDs [2205.11916][2405.20962].

The broader significance of zero-shot LLMs lies less in complete autonomy than in the discovery that pretrained models often contain deployable task competence which can be exposed by the right interface. The strongest systems are frequently hybrid: BM25 plus LLM augmentation in retrieval, Python or DSL execution for financial QA, CLIP plus multimodal LLM descriptions for vision, ASR encoders plus LLM decoders for speech, and decision-tree induction driven by LLM-estimated priors rather than sample counts [2304.14233][2311.14722][2405.15668][2502.16142][2501.16247]. This suggests that zero-shot LLMs are most productively understood as general-purpose inference engines whose pretrained linguistic and world knowledge can be orchestrated by prompts, tools, and structured control mechanisms, rather than as universal replacements for domain-specific modeling.

Source: https://www.emergentmind.com/topics/zero-shot-large-language-models-llms