Zero-shot LLMs: Insights & Applications

• Zero-shot LLMs are pretrained language models that handle tasks without fine-tuning by leveraging strategic prompts to activate latent reasoning abilities.
• They employ structured prompts, external tools, and retrieval scaffolds to excel in arithmetic, translation, scientific inference, and classification.
• Empirical studies show that zero-shot techniques boost performance on complex benchmarks, although challenges remain in calibration and task-specific reliability.

Zero-shot LLMs are pretrained or instruction-tuned LLMs deployed on downstream tasks without task-specific examples, and in many cases without task-specific fine-tuning. In the prompting literature, a prominent formulation is Zero-shot Chain-of-Thought prompting, which showed that a fixed trigger such as “Let’s think step by step.” can elicit intermediate reasoning and substantially improve multi-step problem solving across arithmetic, symbolic, and logical benchmarks (Kojima et al., 2022). Subsequent work extended the zero-shot setting far beyond textual reasoning, using prompting, constrained outputs, retrieval scaffolds, program execution, and multimodal context to induce capabilities in retrieval, ranking, classification, translation, forecasting, speech, vision, and even scientific hypothesis generation (Shen et al., 2023, Gruver et al., 2023, Koshkin et al., 2024, Qi et al., 2023).

1. Definition and operational scope

The zero-shot condition is defined relative to the downstream task rather than to pretraining itself. In the reasoning setting, zero-shot LLMs are pretrained models that can be prompted to solve reasoning tasks without any task-specific examples (Kojima et al., 2022). In simultaneous machine translation, zero-shot means that the model is not fine-tuned for SiMT; instead, at each step it conditions on the partial source text, the partial target text, and optional background information, formalized as $p(y_t \mid y_{<t}, x_{<t}, b)$ (Koshkin et al., 2024). In zero-shot decision tree construction, the tree is built without any training dataset at all, using only feature names, feature types, category values, target descriptions, and branch constraints as prompt context (Carrasco et al., 27 Jan 2025). In zero-shot ASR domain adaptation, the defining condition is the absence of target-domain speech or text training beyond a single handcrafted domain-specific prompt (Li et al., 2023).

Accordingly, zero-shot use does not imply the absence of structure, metadata, or external tools. Financial document QA remains zero-shot even when the model is asked to generate Python or DSL programs from a passage and table, because the method uses prompting rather than demonstrations or task-specific training (Phogat et al., 2023). Zero-shot retrieval remains zero-shot even when BM25 candidate passages are supplied to the LLM as in-domain context for query augmentation (Shen et al., 2023). This suggests that the central invariant across the literature is not “no context,” but “no task-specific supervised adaptation.”

2. Prompting, control, and inference scaffolds

Zero-shot performance is highly prompt-mediated. The core mechanism in Zero-shot-CoT is a two-stage prompting pipeline. First, a reasoning extraction prompt of the form Q: [X]. A: Let's think step by step. elicits intermediate reasoning. Second, an answer extraction prompt appends the generated reasoning and an answer trigger such as Therefore, the answer (arabic numerals) is ... or Therefore, among A through E, the answer is ... (Kojima et al., 2022). This differs from standard zero-shot prompting, which asks for the answer directly, and from few-shot-CoT, which requires task-specific worked examples.

Other zero-shot systems use tighter output control. In context-aware simultaneous translation, the prompt contains the current partial source text, the already generated partial target text, and optionally compact background information such as a JSON summary of topic and named entities. The LLM greedily emits either a full new target word, which triggers a WRITE action, or the special token <|eot_id|>, which is interpreted as a READ action. Response priming places the partial target in the assistant message with a fixed prefix such as German translation:, constraining the output space and reducing disclaimers or explanatory text (Koshkin et al., 2024).

Program synthesis is another major zero-shot scaffold. In financial QA, ZS-FinPYT prompts the model to write executable Python code, define variables for the required calculation, and assign the final result to ans; ZS-FinDSL uses a dual-prompt pipeline in which natural-language reasoning is first generated and then converted into a constrained DSL program for execution (Phogat et al., 2023). In readability assessment, expected-value scoring replaces the single generated score token with an expectation over the probability mass assigned to candidate numeric tokens, and LAURAE combines standardized LLM and readability-formula scores using the model’s self-reported confidence as a weight (Grossman et al., 27 Apr 2026).

Zero-shot retrieval likewise depends on prompt structure rather than raw generation. LameR retrieves a small set of candidate passages with BM25, inserts them into the LLM prompt as possible answering passages, samples multiple answer-like augmentations, concatenates them to the original query, and retrieves again with BM25. Candidate passages act as in-collection demonstrations of intent, domain, and answer format, even when many of them are incorrect (Shen et al., 2023).

3. Reasoning, computation, and scientific inference

The most influential empirical result in the area is that prompting alone can expose nontrivial reasoning ability. With InstructGPT (text-davinci-002), Zero-shot-CoT raised MultiArith accuracy from 17.7% to 78.7%, GSM8K from 10.4% to 40.7%, Last Letter Concatenation from 0.2% to 57.6%, Coin Flip from 12.8% to 91.4%, Date Understanding from 49.3% to 67.5%, and Tracking Shuffled Objects from 31.3% to 52.4%. Results on commonsense reasoning were mixed: CommonsenseQA dropped from 68.8% to 64.6%, whereas StrategyQA rose from 12.7% to 54.8% under the right model and prompting setup. The paper emphasizes that the gains are strongest on large models, while smaller models often benefit little (Kojima et al., 2022).

For numerical reasoning over financial documents, direct zero-shot answering remains brittle because arithmetic errors persist even when the reasoning narrative appears coherent. Program-inducing prompts substantially reduce this problem. On gpt-3.5-turbo, ZS-FinPYT improved FinQA from 53.01 with ZS-CoT to 66.52, ConvFinQA from 52.49 to 67.45, and TATQA from 74.09 to 85.00. On gpt-4, ZS-FinPYT reached 77.51 on FinQA and 93.00 on TATQA, with the interpreter rather than the LLM carrying out the arithmetic (Phogat et al., 2023).

Time-series forecasting extends the zero-shot idea by reframing numeric extrapolation as text continuation. A time series is serialized as a string of digits and separators, and the LLM models an autoregressive distribution

$$p_\theta(U)=\prod_{j=1}^{n} p_\theta(u_j \mid u_{0:j-1}).$$

The same work derives a continuous density from discrete token probabilities, supports missing values through textual markers such as NaN, and adds textual side information without specialized forecasting architecture. Empirically, GPT-3 and LLaMA-2 can be best or second best on MAE and strong on likelihood and CRPS, whereas GPT-4 can underperform GPT-3 because of number tokenization and poor uncertainty calibration attributed to alignment interventions such as RLHF (Gruver et al., 2023).

Zero-shot scientific inference has also been studied directly. In biomedical hypothesis generation, temporally controlled evaluation shows that LLMs can produce untrained yet validated hypotheses on unseen literature, and zero-shot prompting often outperforms random few-shot prompting for large models. The same study reports that increasing uncertainty can facilitate candidate generation and introduces a multi-agent framework with Analyst, Engineer, Scientist, and Critic roles to widen the hypothesis space (Qi et al., 2023).

4. Retrieval, ranking, and recommendation

Zero-shot LLMs have been used not only to answer questions, but also to rank documents and items. In LameR, the LLM serves as a query augmenter rather than a direct retriever. On TREC Deep Learning 2019, LameR achieved 69.1 nDCG@10, compared with 61.3 for HyDE and 50.6 for BM25; on TREC Deep Learning 2020 it achieved 64.8, compared with 57.9 for HyDE and 48.0 for BM25. It was also best on four of six BEIR datasets in the reported evaluation. The method’s core claim is that LLM-generated answer-like expansions become effective when grounded by in-domain candidate passages and consumed by a literal retriever such as BM25 (Shen et al., 2023).

A parallel line of work treats open-source LLMs as zero-shot query likelihood models. Documents are scored by the average log-likelihood of generating the query,

$$S_{QLM}(\mathbf{q}, d)=\frac{1}{|\mathbf{q}|}\sum_t \log \mathrm{LLM}(q_t \mid \mathbf{p}, d, \mathbf{q}_{<t}),$$

and ranked accordingly. In this setting, pre-trained-only open models such as LLaMA and Falcon are strong zero-shot rankers: Falcon-40B and FlanT5-11B each reached an average 53.1 nDCG@10, Falcon-7B reached 52.2, and LLaMA-7B reached 51.1. A central finding is that generic instruction fine-tuning often hurts ranking unless question-generation tasks are present in the fine-tuning data. With a hybrid first-stage retriever combining BM25 and HyDE, zero-shot reranking improved further, and just three GBQ examples raised LLaMA-7B to 58.8 average nDCG@10 in the few-shot extension (Zhuang et al., 2023).

Recommendation ranking exhibits a related but more brittle zero-shot behavior. The task is formalized as conditional ranking over a chronological interaction history $\mathcal{H}$ and candidate set $\mathcal{C}$. On random-candidate settings, zero-shot LLM rankers clearly outperformed BM25, UniSRec, and VQ-Rec. For MovieLens-1M, N@20 increased from 33.19 for BM25 and 37.93 for UniSRec to 49.02 with sequential prompting, 50.01 with recency-focused prompting, and 51.62 with sequence-based ICL; on Amazon Games, N@20 reached 59.01 with recency-focused prompting. However, strong supervised recommenders such as SASRec remained substantially better in the easiest setting, and the study found three systematic weaknesses: weak sensitivity to interaction order, popularity bias, and position bias. Recency-focused prompting and three-fold shuffle bootstrapping partially mitigated these effects (Hou et al., 2023).

5. Classification and predictive modeling over text, tables, and mobility

Prompted LLMs can act as direct zero-shot text classifiers by mapping natural-language instructions to constrained label outputs. On four evaluated datasets, they were effective on three. GPT-4 achieved 0.7133 accuracy on economic text sentiment classification, 0.9000 on e-commerce category classification, and 0.9733 on SMS spam detection, but only 0.5267 on COVID-19 tweet sentiment, where a GRU reached 0.8200. The paper attributes some variability to prompt and preprocessing sensitivity: cleaning tweets improved GPT-4 while reducing GPT-3.5 and Llama2 performance, and output-format instability remained an issue for some open models (Wang et al., 2023).

Domain-specific telecom evaluation yields a more cautious picture. Across SPEC5G-Classification, SPEC5G-Summarization, and TeleQnA, the best zero-shot 7B open models approached but did not generally surpass fine-tuned baselines. Zephyr-7B-β reached 57.93% accuracy on classification and Rouge-1 0.4933, Rouge-2 0.2425, Rouge-L 0.3693 on summarization; Mistral-7B reached 60.93% accuracy on telecom MCQ QA, compared with 67.11% for GPT-3.5 and 74.91% for GPT-4 on the TeleQnA benchmark. No single model dominated across tasks, and many errors reflected instruction-following failures rather than obvious absence of telecom knowledge (Ahmed et al., 2024).

Unsupervised readability assessment and tabular probabilistic prediction show a different facet of zero-shot behavior: the problem is often not raw capability but reliability estimation. In readability assessment, a new prompting methodology outperformed prior methods on 13 of 14 datasets, and LAURAE robustly outperformed prior methods across languages, text lengths, and technicality by combining LLM and readability-formula scores (Grossman et al., 27 Apr 2026). In tabular prediction, a large-scale evaluation across 316 tasks found that LLM performance is highly variable within datasets and that predicted probabilities are often poorly calibrated, with median ECE around 0.2426 for GPT-4o-mini and 0.1722 for Llama. Yet the spread of unlabeled risk scores is informative: the standard deviation of risk scores is the strongest simple predictor of task-level AUC, and for GPT-4o-mini tasks with risk-score standard deviation at least 0.4 had average AUC around 0.8417, versus an overall average of 0.7186 (Ren et al., 18 Sep 2025).

Zero-shot predictive modeling has also been demonstrated in mobility and symbolic tabular induction. For next-location prediction, GPT-3.5 achieved ACC@5 of 0.298 on Foursquare NYC, 0.324 on Foursquare Tokyo, and 0.279 on a private Ferrara cycling dataset, corresponding to 601% relative improvement in NYC and 561% in Tokyo over cross-city transfer baselines. The paper reports that historical visits are generally more informative than short contextual windows, and that one-shot or few-shot prompting does not uniformly improve upon zero-shot across models (Beneduce et al., 2024). In zero-shot decision tree construction, LLMs propose splits, estimate branch class probabilities, and minimize branch impurity via

$\mathrm{Gini}=1-\sum_{i=1}^{C} p_i^2$

combined with a harmonic mean across the two branches. The resulting trees require no labeled training data, remain interpretable, outperform TabLLM zero-shot baselines, and are competitive with low-shot supervised decision trees on several tabular datasets (Carrasco et al., 27 Jan 2025).

6. Multimodal, speech, and translation settings

Zero-shot use of LLMs is not confined to text-only inputs. In simultaneous machine translation, a cascaded Whisper-plus-Llama-3-70B-Instruct system performed competitively without any SiMT fine-tuning. On English-to-German TED-TST-2023, it achieved 22.13 BLEU with AL 1360.59 and LAAL 2089.16, outperforming TRANSLLAMA (19.36 BLEU), SEAMLESS (19.75), and FBK (17.65), while slightly exceeding NAIST (21.39). On the context-sensitive AMBIEVAL benchmark it reached 42.60 BLEU, ahead of NAIST (39.80), TRANSLLAMA (32.43), SEAMLESS (29.76), and FBK (24.96). Injecting minimal background information consistently improved BLEU on TED-TST-2024 across all five language pairs; for English-to-German, performance dropped from 36.76 to 31.14 when the background was removed. The reported real-time factor of the full system was 0.86 (Koshkin et al., 2024).

In zero-shot image classification, multimodal LLMs have been used to enrich the query representation rather than merely the class labels. One method queries Gemini Pro for an image description and an initial class prediction, embeds both texts with CLIP’s text encoder, fuses them with the original image embedding,

$\widetilde{X}_{q} = X_{if} + X_{df} + X_{pf},$

and then performs linear similarity classification against embedded class labels. With the same universal prompts across datasets, this produced an average gain of +4.1 percentage points over ten benchmarks and +6.8 on ImageNet; the combined-feature system reached 73.4 top-1 accuracy on ImageNet versus 66.6 for CuPL in the re-evaluated setup (Abdelhamed et al., 2024).

Speech applications show both the promise and the selectivity of zero-shot gains. In rare-word ASR, an LLM-integrated architecture using Whisper V2, an adapter, and Qwen-7B-Chat improved Primock57 Set 2 rare-word error rate from 40.2% for Zipformer-Transducer to 32.2%, while overall O-WER and N-WER remained roughly comparable. The paper’s central interpretation is that the speech encoder mainly determines general transcription quality, whereas the LLM primarily improves rare-word recognition (Wang, 22 Feb 2025). In a separate zero-shot ASR domain-adaptation study, prompted LLaMA-7B was used either for second-pass rescoring or for deep LLM-fusion with HuBERT-Large. With only a single domain prompt, both methods reduced WER on TedLium-2 and SPGISpeech, and deep LLM-fusion achieved substantially better entity and OOV recall than rescoring (Li et al., 2023).

7. Limitations, misconceptions, and research significance

A persistent misconception is that zero-shot success is uniform across models and tasks. The empirical record is more conditional. Zero-shot-CoT works best on large models and often provides little benefit to smaller ones. In mobility prediction, several tested models, including Phi variants, Gemma 2B, GPT-J, and Dolly, failed to produce usable outputs. In recommender ranking, candidate order and popularity can distort the ranking, and in telecom evaluation formatting failures can materially depress scores even when the answer appears semantically plausible (Kojima et al., 2022, Beneduce et al., 2024, Hou et al., 2023, Ahmed et al., 2024).

Reliability is a second major limitation. Zero-shot tabular prediction shows that model confidence is often informative for relative correctness but not well calibrated in absolute terms, and the authors explicitly recommend unlabeled confidence metrics only as screening tools rather than substitutes for labeled validation, especially in healthcare or finance (Ren et al., 18 Sep 2025). Memorization and contamination concerns also remain. The Zero-shot-CoT work notes that full training-data details are unavailable, so memorization cannot be ruled out completely, while the next-location study addresses the issue more directly through a private dataset, quiz-based contamination tests, and inspection of hallucinated location IDs (Kojima et al., 2022, Beneduce et al., 2024).

The broader significance of zero-shot LLMs lies less in complete autonomy than in the discovery that pretrained models often contain deployable task competence which can be exposed by the right interface. The strongest systems are frequently hybrid: BM25 plus LLM augmentation in retrieval, Python or DSL execution for financial QA, CLIP plus multimodal LLM descriptions for vision, ASR encoders plus LLM decoders for speech, and decision-tree induction driven by LLM-estimated priors rather than sample counts (Shen et al., 2023, Phogat et al., 2023, Abdelhamed et al., 2024, Wang, 22 Feb 2025, Carrasco et al., 27 Jan 2025). This suggests that zero-shot LLMs are most productively understood as general-purpose inference engines whose pretrained linguistic and world knowledge can be orchestrated by prompts, tools, and structured control mechanisms, rather than as universal replacements for domain-specific modeling.