HalluScore in Hallucination Research

• HalluScore is a term used in hallucination research that encompasses diverse constructs such as an Arabic QA benchmark, a composite quality metric for LLM responses, and an image super-resolution score.
• It measures key dimensions like factual accuracy, semantic coherence, and fabrication, with each variant tailored to specific domains and evaluation needs.
• Despite its actionable insights for mitigating hallucinations, HalluScore lacks standardization and clearly defined evaluation protocols across modalities.

HalluScore is not a single standardized construct in current hallucination research. In recent work, the term denotes at least three different objects: a structured Arabic question answering benchmark for hallucination analysis, a composite response-quality metric for instruction-following LLMs, and a reference-aware 1–5 score for generative image super-resolution; closely related variants such as BenHalluScore extend the same naming pattern to dual-track calibration in Bengali hallucination judgment (Alansari et al., 16 May 2026, Cherif, 4 May 2026, Ren et al., 18 Jul 2025, Adib et al., 29 May 2026).

1. Terminological scope

The recent literature uses “HalluScore” in distinct, domain-specific senses rather than as a universal metric. The principal usages represented in current work are summarized below.

Usage	Object being measured	Output form
HalluScore benchmark	Arabic generative-QA hallucination evaluation	Benchmark with hallucination rates
HalluScore in HalluScan	Response-level composite quality metric	Scalar in $[0,1]$
Hallucination Score in GSR	Super-resolved image hallucination severity	Integer score from 1 to 5
BenHalluScore	Bengali dual-track calibration metric	Percentage error score

The Arabic benchmark titled "HalluScore" contains 827 Arabic QA pairs and is explicitly presented as a benchmark resource rather than a single closed-form scalar formula. By contrast, HalluScan defines HalluScore as a weighted geometric mean over factual error, semantic coherence, and fabrication rate. In image super-resolution, “Hallucination Score” or HS is a GPT-4o-generated image-level score based on a rubric applied to the ground-truth image, the low-resolution input, and the super-resolved output. BenHalluScore, although differently named, is a closely related score concept because it formalizes hallucination judgment as dual-track calibration rather than positive-only detection (Alansari et al., 16 May 2026, Cherif, 4 May 2026, Ren et al., 18 Jul 2025, Adib et al., 29 May 2026).

A recurrent misconception is that HalluScore always refers to a detector output or a single detector-ranking metric. In the literature considered here, that is false. In one case it is a benchmark name, in one case it is a composite response-quality measure, and in one case it is a multimodal evaluation rubric for image restoration.

2. HalluScore as an Arabic hallucination benchmark

"​​HalluScore: LLM Hallucination Question Answering Benchmark" defines HalluScore as the first Arabic question answering benchmark specifically designed for hallucination analysis in LLMs. Its stated purpose is to address the underrepresentation of Arabic in hallucination benchmarking and to capture linguistic, cultural, and reasoning conditions not adequately reflected in English- and Chinese-centered resources (Alansari et al., 16 May 2026).

The benchmark contains 827 Arabic QA pairs. Each sample includes the question, a ground-truth answer, a verified source link, an answer explanation, and multiple categorical and binary annotations. The final question-type taxonomy is: Calculation, Confusion, False presupposition, Identity, Knowledge, Misconception, Misquotation, Proverbs, Pseudoscience, Stereotype, and Subjective. The dataset also carries four binary multi-label attributes: adversarial intent, reasoning requirement, historical dependency, and Arabic cultural relevance. Overall, 43.58% of questions are adversarial, 26.63% require reasoning, 29.09% are historical, and 37.24% are culturally grounded in Arabic (Alansari et al., 16 May 2026).

Its construction pipeline is explicitly multi-stage. The process begins with 1,500 QA pairs collected through crowdsourcing and a small translated subset from TruthfulQA, applies five exclusion criteria and additional filtering to reach 1,153 items, removes questions on which two or fewer models hallucinated to obtain 327 hallucination-prone items, and then adds 500 manually written questions targeting false presuppositions, numerical traps, historical and cultural terms, and long-tail knowledge. The final benchmark evaluates 17 Arabic, multilingual, and reasoning-focused models in zero-shot Arabic generative QA without retrieval augmentation (Alansari et al., 16 May 2026).

The paper is explicit that, despite the benchmark name, it does not define a single scalar HalluScore formula. Performance is reported through factual hallucination rate, faithfulness hallucination rate, and partial hallucination frequency. GPT-5 is reported as the strongest model overall, with 25.15% factual hallucination and 0.73% faithfulness hallucination, while Noon reaches 85.01% factual hallucination and 13.30% faithfulness hallucination. The paper’s broader conclusion is that Arabic hallucination extends beyond factual inaccuracies to cultural understanding, linguistic reasoning, and logical consistency (Alansari et al., 16 May 2026).

3. HalluScore in HalluScan: a composite response-quality metric

"​​HalluScan: A Systematic Benchmark for Detecting and Mitigating Hallucinations in Instruction-Following LLMs" uses HalluScore in a much narrower and more formal sense. Here HalluScore is a response-level composite metric intended to integrate three dimensions of hallucination-relevant quality: factual correctness, internal semantic coherence, and evidence-grounded non-fabrication (Cherif, 4 May 2026).

Its exact definition is:

$$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$

where $\epsilon_f \in [0,1]$ is the factual error rate, $\sigma_s \in [0,1]$ is the semantic coherence score, and $\phi \in [0,1]$ is the fabrication rate. The component weights are fixed as $\alpha = 0.4$, $\beta = 0.3$, and $\gamma = 0.3$, and the paper states that these were determined through correlation maximization with human expert judgments on a held-out validation set (Cherif, 4 May 2026).

The metric is explicitly multiplicative rather than additive. The paper argues that the weighted geometric mean prevents a high score in one dimension from masking deficiencies in others. This design makes HalluScore sensitive to any severe failure in factuality, coherence, or groundedness. HalluScore is not one of HalluScan’s six detection methods; it sits in the evaluation layer, while detector ranking remains primarily based on AUROC, together with F1, precision, recall, ECE, and latency (Cherif, 4 May 2026).

Hallucination-human alignment is the main empirical justification. Domain experts independently rated model responses on a 5-point Likert scale with emphasis on factual accuracy, coherence, and fabrication. HalluScore achieved Pearson correlation $r = 0.41$ with human expert ratings, with $p < 0.05$. The paper also reports comparison baselines: factual error rate alone gave $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$0, semantic coherence alone $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$1, fabrication rate alone $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$2, and the arithmetic mean of components $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$3. The benchmark itself covers 72 configurations spanning 6 detection methods, 4 open-weight model families, and 3 domains, but the HalluScore validation uses only 24 total samples, a limitation explicitly noted by the authors (Cherif, 4 May 2026).

An important interpretive point follows from this design choice. In HalluScan, HalluScore is a holistic response-quality metric, not a calibration metric and not a direct detector-training objective. AUROC answers whether a detector separates hallucinated from non-hallucinated outputs; HalluScore answers how good the response itself is with respect to factuality, coherence, and fabrication.

4. Hallucination Score in generative image super-resolution

"​​Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution" introduces Hallucination Score, abbreviated HS, as a domain-specific metric for generative super-resolution. In this setting, hallucination means generated details in the super-resolved image that are perceptually incorrect relative to the low-resolution input image and the ground-truth high-resolution image. The paper formalizes two principles: $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$4, that super-resolved content that could not be plausibly present in the low-resolution input is necessarily a hallucination, and $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$5, that super-resolved content that differs from the ground truth is hallucinatory to the extent that the generated visual elements are perceptually recognizable as anomalous (Ren et al., 18 Jul 2025).

HS is operationalized through a GPT-4o prompt that receives the Ground Truth image, the Low-Resolution Input, and the Super-Resolved Image, and returns a JSON object with an integer score from 1 to 5 together with reasoning. The rubric defines 1 as significant hallucinations and 5 as artifact-free. The prompt also states that lack of detail, blurry textures, or lower image quality due to severe low-resolution damage should not be counted as hallucinations; the focus is on added details that change semantic meaning, significantly alter scene elements, or create perceptually jarring inaccuracies such as incorrect facial features or unreadable or distorted text (Ren et al., 18 Jul 2025).

The human validation protocol is comparatively extensive. On the StableSR Test Set, the authors use 92 source images, one roughly center crop from each image, and three diffusion-based super-resolution models—PASD, SeeSR, and StableSR—yielding 276 outputs. Each of these 276 images is rated by 11 human evaluators. GPT-based HS reaches Spearman correlation 0.54 with the human mean score and 0.50 with the human majority score. The paper further reports that the absolute disagreement between GPT and the human mean lies within the range of human inter-rater variability (Ren et al., 18 Jul 2025).

The score exposes a three-way distinction between fidelity, perceptual quality, and hallucination. On SS-TS, mean HS is 3.38 for Swin2SR, 3.36 for StableSR, 2.99 for SeeSR, and 2.45 for PASD. Bicubic reaches HS 4.67 on SS-TS, 4.56 on RealSR, and 4.76 on DRealSR, but the paper emphasizes that this does not imply good super-resolution because bicubic has poor MUSIQ and LPIPS and is visibly blurry. The paper therefore argues that high HS alone does not mean good super-resolution; it must be interpreted jointly with perceptual quality and fidelity metrics (Ren et al., 18 Jul 2025).

The mitigation component does not optimize HS directly because GPT-based HS is expensive and non-differentiable. Instead, the paper uses DINO- and CLIP-based differentiable surrogates inside an AlignProp-style reward fine-tuning pipeline. This increases SeeSR’s HS from 2.99 to 3.85 and PASD’s HS from 2.54 to 3.74 on SS-TS under the DINO-ST+MUSIQ reward configuration (Ren et al., 18 Jul 2025).

5. BenHalluScore and dual-track calibration

"​​BenHalluEval: A Multi-Task Hallucination Evaluation Framework for LLMs on Bengali" introduces BenHalluScore, abbreviated BHS, as a dual-track calibration metric rather than a one-sided hallucination detector. It is defined as:

$$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$6

where $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$7 and $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$8 are the numbers of wrong verdicts on two evaluation tracks and $$HalluScore = (1 - \epsilon_f)^{\alpha} \cdot (\sigma_s)^{\beta} \cdot (1 - \phi)^{\gamma}$$9 and $\epsilon_f \in [0,1]$0 are the total numbers of instances on each track. In the appendix the paper restates this as:

$\epsilon_f \in [0,1]$1

with A-err. and B-err. already expressed as percentages (Adib et al., 29 May 2026).

The dual-track protocol is the essential idea. Track A contains 1,000 ground-truth correct answers per task, where the expected verdict is “No” and the model should not mark the item as hallucinated. Track B contains hallucinated candidates, where the expected verdict is “Yes.” The paper’s central argument is that evaluating only hallucinated instances rewards an “always yes” model, and evaluating only correct instances rewards an “always no” model. BenHalluScore is therefore designed to jointly penalize false positives on ground-truth instances and missed detections on hallucinated candidates, preventing inflated scores from uniform response bias (Adib et al., 29 May 2026).

The interpretation scale is explicit: 0% means perfect calibration on both tracks, 50% means random or uniformly biased behaviour, and 100% means perfectly wrong predictions across both tracks. The benchmark spans four tasks—Generative Question Answering, Bangla-English Code-Mixed QA, Summarization, and Reasoning—built from TyDiQA-GoldP (Bengali), BanglaCHQ-Summ, and SOMADHAN. It contains 12,000 hallucinated candidates across 12 task-specific hallucination types and 4,000 gold instances total, with 1,000 per task (Adib et al., 29 May 2026).

Reported BHS values range from 7.72% to 55.42% across models and tasks. The best overall task-specific score is Qwen2.5-32B on summarization at 7.72%, while the worst is Mistral-nemo-12B on code-mixed QA at 55.42%. The paper uses concrete contrasts to motivate the metric: Mistral-nemo-12B can appear strong on hallucinated inputs alone but performs catastrophically on Track A, whereas LLaMA-3.1-8B shows the opposite bias pattern. In both cases, BHS exposes poor calibration that a single-track metric would hide (Adib et al., 29 May 2026).

This usage is conceptually adjacent to HalluScore because it treats hallucination judgment as a balanced discrimination problem rather than a positive-only retrieval task. It also clarifies that “calibration” can mean label-balanced operational behavior even when the paper does not provide probabilistic calibration curves.

6. Methodological themes, neighboring paradigms, and open issues

Across these HalluScore-related constructs, three methodological axes recur: the unit of verification, the evidence source, and the validation criterion. FaithScore decomposes large vision-LLM answers into descriptive sub-sentences and then into atomic facts covering entity, count, color, relation, and other attributes, finally averaging visual support decisions into a reference-free faithfulness score; its correlation with human judgments is much higher than BLEU-4, ROUGE-L, METEOR, CHAIR, or CLIP-Score, with Pearson $\epsilon_f \in [0,1]$2 (Jing et al., 2023). LongHalQA moves in a different direction, avoiding judge LLMs at evaluation time by converting both hallucination discrimination and hallucination completion into multiple-choice tasks over 6,485 examples of long image descriptions and multi-round conversations (Qiu et al., 2024). HalluSearch instead localizes hallucinations as character spans by combining factual splitting, external search, and verifier-based contradiction extraction in fourteen languages (Abdallah et al., 14 Apr 2025). DAHL, in biomedicine, averages the factuality of atomic units in long-form answers into the DAHL Score and reports Pearson correlation 0.5508 with human factual-precision judgments (Seo et al., 2024).

A second recurring theme is skepticism toward unqualified scalar severity scoring. "Evaluating the Quality of Hallucination Benchmarks for Large Vision-LLMs" shows that score-based free-form benchmarks such as MMHal and GAVIE have criterion validity 0.4309 and 0.3081 respectively, while its HQH benchmark—using binary hallucination decisions aggregated into Hallucination Rate—reaches criterion validity 0.9415 together with 8-type coverage (Yan et al., 2024). This does not invalidate HalluScore-style metrics, but it shows that benchmark reliability, prompt sensitivity, and human alignment are themselves measurement problems.

A third theme is that different domains require different evidential assumptions. In Arabic QA, HalluScore is fundamentally a benchmark with verified source links and human response annotations rather than a formula. In HalluScan, HalluScore depends on claim extraction, fact verification, within-response coherence, and evidence traceability. In Bengali BenHalluEval, the decisive issue is dual-track balance between gold and hallucinated inputs. In super-resolution, the crucial comparison is among the low-resolution observation, the super-resolved output, and the ground-truth image. A plausible implication is that “HalluScore” functions less as a single metric family than as a naming convention for domain-specific groundedness measures.

The main open issue across these works is standardization. Some papers named with HalluScore do not define a single scalar at all; some define one but leave implementation details partially underspecified; some show only moderate human alignment; and some demonstrate that simpler binary protocols can outperform graded scalar judgments in criterion validity. For that reason, HalluScore is best understood as a family resemblance term in contemporary hallucination research: it denotes benchmark or scoring mechanisms that quantify unsupported, fabricated, or ungrounded model content, but the operational meaning depends entirely on the modality, task, evidence structure, and failure mode being evaluated.