PhantomBench: Benchmarking the Non-existential Threat of Language Models

Abstract: Hallucinations, where LMs generate factually ungrounded responses, pose serious risks, as users tend to blindly rely on them. This is particularly concerning in high-stakes domains, where consequences of such model behavior can lead to significant harms. Despite notable progress in understanding hallucinations, it remains unclear how reliably these models can recognize the limits of their knowledge. We introduce PhantomBench, the first large-scale benchmark of its kind, comprising more than 60K non-existent terms and entities derived from real concepts across diverse domains. Using our benchmark, we evaluate a total of 21 models of various types and sizes. We show staggering hallucination rates across the board (with average rates as high as 86.7% in some cases), and note that even frontier models surprisingly fail to abstain on non-existent concepts, especially when the input presumes their existence. We then show that PhantomBench can serve as a proxy for studying model behavior on rare concepts for which models are more prone to hallucinate. We also provide a pipeline to construct PhantomBench, enabling scalable generation of non-existent concepts tailored to the specific needs of researchers and practitioners.

• The paper introduces a scalable benchmark that systematically evaluates language model hallucinations using over 60K lexically plausible, non-existent queries.
• It employs a rigorous evaluation protocol with an LLM-as-a-judge across 21 diverse models, uncovering hallucination rates as high as 87.3% for certain attributes.
• The study highlights that reasoning-enhanced and domain-specialized models often fail to abstain appropriately, underscoring the need for improved training methods and uncertainty calibration.

PhantomBench: Benchmarking Hallucinations and Abstention in LLMs

Motivation and Problem Formulation

The pervasive issue of hallucination in LMs—generating factually unsupported responses—poses significant risks, especially in high-stakes domains. Despite advancements, current evaluation protocols are limited in reliably testing models' self-knowledge boundaries and their ability to abstain from answering queries about non-existent concepts. PhantomBench addresses this gap by introducing a scalable, multi-domain benchmark built from over 60K lexically plausible but non-existent terms and entities, systematically probing LMs' capacity for abstention.

Benchmark Construction and Pipeline

PhantomBench is constructed using a compositional pipeline that decomposes real terms and entities into components (words, n-grams), recombining them to form plausible but non-existent concepts. A rigorous frequency filtering step, grounded in web-scale corpus search (Dolma v1.7, >2.3T tokens), ensures only genuinely non-existent items are retained. Prompts are generated to test both explicit and implicit presupposition of existence across multiple attributes (existence, meaning, date, place, etymology, application, relation).

Figure 1: The PhantomBench pipeline decomposes existing concepts and recombines components, followed by corpus-based non-existence verification and diverse prompt generation.

Evaluation Protocol and Model Spectrum

PhantomBench queries 21 LLMs spanning multiple families, parameter scales, reasoning capabilities, and domain specialization. Responses are evaluated through an LLM-as-a-judge protocol using Gemini 2.5 Flash, validated statistically against human annotators for binary abstention judgment as well as fine-grained abstention categorization. The primary evaluation metric is hallucination rate (HR): the fraction of queries for which the model fails to abstain and instead hallucinates an answer.

Empirical Findings: Hallucination and Abstention Patterns

Across the board, hallucination rates remain alarmingly high. Even frontier models frequently fail to abstain, particularly when prompts implicitly presuppose concept existence (meaning attribute HR up to 87.3% for Gemma 3-12B). Models perform notably better when existence is explicitly queried (HR drops to 14.3% for Llama 3.1-8B), but struggle with other attributes, especially those unconstrained by factual grounding.

Figure 2: Hallucination rates by prompt type illustrate model tendency to hallucinate when attribute existence is presupposed.

Prompt-dependent analysis reveals that models are more likely to fabricate plausible-sounding answers for attributes such as meaning or etymology than for constrained attributes like date or place.

Figure 3: Hallucination Rates (HR) for Phantom-T across models and prompt types show increased hallucination with implicit presupposition.

Model Specific Analyses

Reasoning and Scale

Reasoning-enhanced models (DeepSeek-R1, GPT-OSS-20B) exhibit significantly higher hallucination rates than instruction-tuned counterparts, corroborating prior findings on reasoning-induced hallucination propensity. Increasing model size generally improves abstention behavior up to an inflection point, after which larger models may regress (Qwen 3-32B, Llama 3-70B), suggesting scaling alone does not ensure reliability.

Domain Specialization

Domain-specialized models do not consistently outperform general models in abstention. BioMistral 7B and SaulLM-7B, specialized for biomedical and legal domains respectively, hallucinate more often than their general-purpose equivalents, underscoring the vulnerability of specialized LMs in safety-critical contexts.

Qualitative Abstention Analysis

Abstention behaviors span explicit uncertainty expression, contextual clarification, alternative suggestion, decomposition-based guessing, or implicit presumption. Remarkably, 35.9% of sampled abstention responses contained factually unsupported or fabricated information about related concepts, indicating partial abstention does not equate to factual reliability.

Selectivity and Rare Concept Proxy

Models exhibiting high abstention rates for non-existent concepts tend to also abstain more on rare but existing concepts, as evidenced by strong Pearson correlation ($\rho=0.755$). This implies that non-existent concepts in PhantomBench are a practical proxy for evaluating LM behavior on rare, real-world items, particularly relevant for long-tail knowledge and safety-sensitive domains.

Figure 4: Abstention rates averaged across prompt types for non-existent and common concepts, highlighting selectivity and model clustering.

Fine-Grained Abstention Properties

Abstention behavior, when decomposed into reasoning categories, shows substantial consistency between rare and non-existent concepts but diverges for common concepts, where models primarily abstain from answering specific attributes rather than the existence itself. This provides nuanced insight into model metacognitive limitations.

Figure 5: Proportion of fine-grained abstention categories across models for non-existent, rare, and common concepts.

Implications and Future Directions

PhantomBench exposes foundational flaws in contemporary LMs related to knowledge boundary recognition and metacognition. Strong numerical results—the majority of models hallucinate with rates exceeding 70% when queried about plausible non-existence—directly contradict the assumption of improved reliability with model scale or domain adaptation. Pragmatically, these findings highlight the urgent need for training objectives and benchmarks that incentivize explicit abstention and uncertainty calibration, extending beyond current instruction tuning paradigms.

Future directions include integrating retrieval augmentation, refusal-aware instruction tuning [zhang-etal-2024-r], and explicit meta-cognitive objectives to reduce hallucination rates. PhantomBench’s scalable pipeline enables ongoing evaluation as new models and domains emerge, supporting the development of robust, abstention-aware model architectures.

Conclusion

PhantomBench provides a systematic, scalable benchmark for assessing LM abstention and hallucination behavior on non-existent, long-tail, and rare entities and terms. The empirical results reveal that current instruction-tuned, reasoning-enhanced, and domain-specialized models fail to reliably abstain, even on explicit non-existential queries. The demonstrated correlation between behavior on non-existent and rare concepts suggests PhantomBench is an effective proxy for measuring real-world LM reliability. The study underscores the need for improved training, meta-cognitive modeling, and specialized benchmarks targeting abstention and factuality.