Entailment-Based Zero-Shot Classifier

• Entailment-based zero-shot classifiers are models that recast classification as a textual entailment task using natural language label verbalizations.
• They compute entailment scores via binary or triplet NLI heads to select candidate labels without any labeled downstream examples.
• Employing transformer-based cross-encoders fine-tuned on large NLI datasets, these models achieve competitive performance across sentiment, topic, emotion, and relation extraction tasks.

An entailment-based zero-shot classifier (ZSC) is a class of models that reformulate the zero-shot classification problem as a natural language inference (NLI) or textual entailment task. These systems make use of pretrained or fine-tuned NLI models, leveraging natural-language label descriptions (verbalizations) to match textual inputs with candidate labels—without requiring any labeled examples for the downstream classification task or label space during training. Recent research has established NLI cross-encoders as a foundational approach for ZSC and has enabled principled comparisons with embedding-based, reranking, and LLM alternatives (Aarab, 12 Mar 2026, Yin et al., 2019, Zhang et al., 2022, Sainz et al., 2021).

1. Entailment Reformulation and Label Verbalization

The core idea is to cast classification as a premise–hypothesis entailment problem. For each unlabeled text $x$ and each candidate label $\ell$ from a label set $\mathcal{L}$, a natural-language template or "verbalizer" $T[x, \ell]$ is instantiated, which forms the hypothesis $h_\ell$. For example, for sentiment classification (Amazon Polarity), a verbalizer might be:

• Premise: $x$ = review text,
• Hypothesis: $h_\ell$ = "The overall sentiment within the Amazon product review is {label}." (with {label} replaced by "positive" or "negative")

The text $x$ and hypothesis $h_\ell$ are concatenated and fed into the NLI model as $$[\text{CLS}]\, x\, [\text{SEP}]\, h_\ell\, [\text{SEP}]$$ (Aarab, 12 Mar 2026, Yin et al., 2019). Label verbalization is critical: templates should be concise, context-rich, and, where necessary, augmented by label definitions for improved performance in the fully unsupervised regime (Yin et al., 2019).

2. Scoring Functions and Decision Rules

Entailment-based ZSCs use the NLI model to compute a score or probability that the premise $\ell$0 entails each candidate hypothesis $\ell$1. There are two scoring head variants:

• Binary entailment head: Outputs a single logit $\ell$2 per $\ell$3; computes $\ell$4 with standard binary cross-entropy loss.
• Three-way (triplet) NLI head: Produces logits $\ell$5; uses cross-entropy over three classes (entailment, neutral, contradiction). Inference collapses neutral and contradiction to compute

$\ell$6

Optionally, probabilities are obtained by $\ell$7 (Aarab, 12 Mar 2026).

For each $\ell$8, the ZSC evaluates all hypotheses $\ell$9 and predicts the label maximizing the relevant score (no thresholding in the zero-shot protocol). In multi-label or partially-seen evaluation protocols, additional calibration between seen and unseen labels can be applied using a margin parameter $\mathcal{L}$0 (Yin et al., 2019).

3. Architectures, Training Protocols, and Optimization

Strong entailment-based ZSCs typically exploit transformer-based cross-encoders (e.g., DeBERTa-v3-large, RoBERTa-large, BART-large, ALBERT, BERT-base). These are pretrained and often further fine-tuned on the union of large-scale NLI resources (MNLI, ANLI, WANLI, FEVERNLI, LingNLI) using the corresponding classification head (binary or triplet). Key training configurations include:

• Batch size 32, 3 epochs, early stopping by dev loss plateau.
• AdamW optimizer with backbone learning rate $\mathcal{L}$1, head learning rate $\mathcal{L}$2.
• Cosine decay learning-rate schedule, mixed-precision training, dropout/GELU/LayerNorm classifier head atop the [CLS] token (Aarab, 12 Mar 2026).

Label verbalizer templates should be fixed across models and datasets where possible for consistency; per-model tuning is discouraged (Aarab, 12 Mar 2026).

4. Empirical Results and Domains of Strength

The empirical macro-F1 performance of entailment-based ZSCs on comprehensive benchmarks, such as BTZSC (22 datasets), is summarized below (Aarab, 12 Mar 2026):

Model (Cross-Encoder)	Macro-F1	Accuracy
bart-large-mnli	~0.51	~0.53
nli-roberta-base	~0.49	~0.50
bert-large-nli-triplet	~0.52	~0.55
DeBERTa-v3-large-nli-triplet	~0.60	~0.62

Domain breakdown for DeBERTa-v3-large-nli-triplet:

• Sentiment: F1 ≈ 0.90 (near saturation)
• Topic: F1 ≈ 0.50
• Intent: F1 ≈ 0.45
• Emotion: F1 ≈ 0.42

Entailment-based ZSCs excel on standard sentiment tasks, significantly outperforming embedding-only models on seen labels and achieving high accuracy on unseen labels in label-partially-unseen setups. For example, fine-tuned NLI models show +8–15 accuracy improvement on unseen topics over supervised baselines (Yin et al., 2019). In relation extraction, entailment-based ZSCs (with hand-crafted templates) deliver 63% F1 zero-shot—substantially above supervised classifiers in the few-shot regime (Sainz et al., 2021).

5. Comparison with Embedding Models, Rerankers, and LLMs

Systematic benchmarking shows that entailment-based ZSCs, while robust and data-efficient, are now outperformed in absolute terms by larger rerankers and LLMs on challenging multi-class tasks (topic, intent, emotion) (Aarab, 12 Mar 2026):

Model Family	Macro F1	Inference Speed (qualitative)	Strengths
Rerankers (Qwen3-8B)	~0.72	5–20× slower than embedders/cross-encoders	State-of-the-art; excels at all tasks
Embedding models (GTE-large)	~0.62	Fastest (few hundred ms/batch)	Best speed–accuracy trade-off
Cross-encoders (DeBERTa-L)	~0.60	~2× the time of embedding models	Sentiment, few-shot, efficiency
Instruction-tuned LLMs	~0.66–0.67	Slowest; scale brings gains (>3B params)	Strong on topic; under rerankers on F1

Plateauing accuracy above 300–400M parameters is observed for NLI cross-encoders. Scaling benefits rerankers/LLMs more than embedding or cross-encoder models. Embedding models deliver the best accuracy/latency Pareto front; entailment-based ZSCs remain competitive for efficient mid-sized deployment and remain significantly faster than rerankers and LLMs (Aarab, 12 Mar 2026).

6. Best Practices, Ablation Results, and Limitations

Empirical ablations reveal that:

• Triplet (three-way) NLI loss marginally improves intent classification but harms topic classification; binary and triplet losses are otherwise similar.
• Large transformer backbones ($\mathcal{L}$3300M) outperform base-size consistently by ~+3 macro-F1, but gains diminish beyond large models.
• Clean, context-rich label verbalizers generalize well across tasks and models.
• Early stopping on mixed-domain NLI dev sets mitigates overfitting (Aarab, 12 Mar 2026).

Limitations:

• Entailment-based ZSCs are relatively weaker on emotion detection and on high-cardinality intent tasks (F1 < 0.45).
• Returns in accuracy rapidly diminish as backbone model size increases past several hundred million parameters (Aarab, 12 Mar 2026).
• Main bottleneck in relation extraction ZSCs is recall for NO-RELATION, though small calibration sets (2 dev examples per label) recover most of the gap (Sainz et al., 2021).
• Scaling to large label spaces ($\mathcal{L}$410 classes) is not fully established (Zhang et al., 2022).

7. Generalization Across Tasks and Future Directions

Entailment-based ZSCs have proven effective across classification domains (sentiment, topic, emotion, intent, relation extraction), label-verbalization schemes, and evaluation protocols (restrictive and fully-unseen zero-shot). Flexible hypothesis generation (template plus definition) and bootstrappable few-shot adaptation remain central to strong cross-domain performance (Yin et al., 2019, Sainz et al., 2021).

Recent advances include:

• Nested entailment meta-task reformulations enabling absorption of knowledge across dozens of datasets via supervised contrastive pretraining and improving zero-shot accuracy by 9.4 points over prior discriminative entailment models (Zhang et al., 2022).
• Integration of reusable label verbalizers, supervised contrastive loss, and cross-task transfer protocols.
• Prospects for multilingual expansion, improved generic premises/calibration, and scalability to more complex structures and label-rich tasks (Zhang et al., 2022).

Entailment-based zero-shot classifiers remain a baseline for data-efficient semantic classification, with ongoing relevance for practitioners balancing annotation cost, inference speed, and domain generalization (Aarab, 12 Mar 2026, Yin et al., 2019, Zhang et al., 2022, Sainz et al., 2021).