M-HalDetect: Hallucination Detection Dataset

• M-HalDetect is a large-scale dataset of 16,000 image–prompt–response triplets with detailed sub-sentence annotations for visual hallucination errors.
• It systematically categorizes hallucinations into object hallucination, unfaithful entity descriptions, and inaccurate relationships using a fine-grained annotation schema.
• The dataset is effectively used to train reward models, benchmark detection performance, and optimize multi-modal language generation in LVLMs.

M-HalDetect is a publicly available, large-scale, fine-grained human-annotated dataset designed to benchmark and improve hallucination detection in outputs from large vision-LLMs (LVLMs). The dataset addresses both holistic and segment-level fidelity between generated textual descriptions and their corresponding images, establishing a comprehensive foundation for research into grounded language generation and model alignment in multi-modal artificial intelligence (Gunjal et al., 2023).

1. Construction and Scope

M-HalDetect is constructed from the val2014 split of MS COCO, consisting of 4,000 images. For each image, four vision–language instructions are formulated in a Visual Question Answering (VQA)-style and answered by InstructBLIP, yielding 16,000 image–prompt–response triplets. Each generated text is then meticulously annotated at the sub-sentence level. The principal aim is to enable both training and evaluation of models on tasks that require detection and minimization of non-visually-grounded or hallucinated content.

Unlike prior datasets restricted to object presence, M-HalDetect systematically labels three major hallucination phenomena:

• Object Hallucination: Mention of entities not present in the image.
• Unfaithful Entity Descriptions: Incorrect attributes or properties ascribed to visible objects.
• Inaccurate Relationships: Groundless claims about spatial, causal, or functional relations.

The dataset thus provides a training corpus not only for binary hallucination detection, but allowing gradient-scale supervision on nuanced, local error patterns in generated multimodal text.

2. Annotation Taxonomy and Protocol

A unified and fine-grained annotation schema is adopted:

• Accurate: Spans referring to verifiable objects or relations in the image.
• Inaccurate: Spans referencing non-existent entities, misattributed properties, or unsupported analyses.
• Analysis: Subjective or inferential content beyond direct visual description.
• Unsure: Used very rarely if annotators cannot decide; unmarked tokens default to Accurate.

Each response is segmented at punctuation and conjunction boundaries, with each segment labeled according to this taxonomy. Annotators, recruited and qualified via Scale AI’s RAPID platform (minimum 85% on gold-standard examples), annotate all four responses per image while viewing the corresponding COCO image. Fatigue bias is minimized by defaulting unmarked spans to Accurate. No explicit inter-annotator agreement statistics are reported; consistency is enforced by annotation qualification and embedded gold-checks.

A prototypical annotation (see Figure 1 in (Gunjal et al., 2023)) might label “two motorcycles parked next to” as Accurate, “three chairs” as Inaccurate (if only two are visible), and introductory or motivational phrases as Analysis.

3. Dataset Statistics and Structure

M-HalDetect comprises:

• Images: 4,000 (COCO val2014)
• Samples: 16,000 (4 prompts per image)
• Spans: ~56,000 annotated at the sub-sentence level (≈3.5 per response)

Distribution of token-level labels:

• Accurate: ~60%
• Inaccurate: ~30%
• Analysis: ~10%

Most generated sentences contain less than 20% Inaccurate content by token density. Class balance is monitored and reported.

Dataset splits:

• Training: 3,200 images (12,800 samples)
• Validation: 800 images (3,200 samples)

Each instance is a JSONL record with fields:

• image_id: COCO identifier
• prompt: VQA-style question/instruction
• response: Model-generated description
• annotations: list of span objects {start_token, end_token, label}

Images themselves are referenced by COCO IDs and are not repackaged with M-HalDetect.

4. Hallucination Metrics and Optimization Objectives

The core metric is Hallucination Rate, defined as: $$\mathrm{HallucinationRate} = \frac{|\{\text{Inaccurate tokens}\}|}{|\{\text{Accurate tokens}\}| + |\{\text{Inaccurate tokens}\}|}$$ Analysis spans are excluded from both numerator and denominator.

For reward-modeling, a fine-grained ternary reward model predicts the likelihood of each segment label; overall sample scores average over segments. Formally, the non-hallucination reward for a sample $s$ is: $$R(s) = -\frac{1}{|s|} \sum_{i \in s} \log P_\mathrm{RM}(\mathrm{non\text{-}hallucinated} \mid s_i)$$ where $s_i$ is segment $i$.

The dataset supports optimization by Fine-grained Direct Preference Optimization (FDPO), generalizing the DPO loss to apply at the segment level: $$\mathcal{L}_{\mathrm{FDPO}} = - \mathbb{E}_{(x, y, c)} \left[ \log \sigma\left(\beta\,k(c)\right) \right]$$ where $k(c)$ assigns positive weights to preferred Accurate spans, negative to dispreferred Inaccurate, and zero to Analysis.

5. Benchmarks, Use Cases, and Impact

M-HalDetect enables both reward-model-based and direct optimization paradigms for hallucination reduction in LVLMs. The dataset has been used to:

• Train sentence- and segment-level reward models (binary and ternary).
• Benchmark detectors by accuracy and F1: sentence-level binary models achieve 79.2% accuracy and 78.4% F1, while segment-level binary reach 83.9% accuracy and 83.2% F1.
• Support rejection sampling: a ternary reward model (N=16) reduces InstructBLIP’s hallucination rate from 29% (baseline, $T=0$) to 13%; for mPLUG-OWL, from 53% to 23%.
• Direct preference optimization (FDPO): Fine-tuning InstructBLIP with FDPO yields a relative hallucination rate reduction from 29% to 17%.
• Demonstrate generalization: Reward models trained on M-HalDetect reduce hallucination rates by 15% in LLaVA and 57% in mPLUG-OWL.
• Establish that reward-model outputs correlate strongly with human hallucination judgments ($r\approx0.85$).

A summary of classification performance appears below:

Model/Class.	Accuracy (%)	F1 (%)
Sentence-level binary	79.2	78.4
Sentence-level ternary	71.4	70.8
Segment-level binary	83.9	83.2
Segment-level ternary	77.2	76.9

6. Limitations and Prospects

M-HalDetect, while comprehensive for vision-language hallucination detection, exhibits several limitations:

• **Span annotations default to Accurate if not explicitly labeled, possibly underestimating fine-grained hallucination density.
• Absence of a formal inter-annotator agreement metric; qualification screening and gold-checks serve as proxies but may not fully capture annotation variance.
• Coverage limited to COCO val2014 imagery and InstructBLIP generations; extrapolation to out-of-domain or different model families requires further validation.
• Analysis spans are included for completeness but are not used for evaluation, potentially missing subtle inferential hallucinations.
• Some task designs (e.g., instructional prompt variability) might not reflect the full scope of VQA or captioning challenges studied elsewhere.

A plausible implication is that future versions could expand to broader image domains, multi-model responses, and introduce explicit multi-annotator agreement scoring to enhance transparency and scope.

7. Access and Application

The full dataset, including code and detailed schema, is available at https://github.com/hendryx-scale/mhal-detect. Images are referenced by standard COCO IDs. The dataset has become a reference benchmark for evaluating hallucination detection in LVLMs and supports development of refined grounding techniques in multi-modal language generation research (Gunjal et al., 2023).