PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset (2403.11116v3)
Abstract: Multimodal LLMs (MLLMs) hallucinate, resulting in an emerging topic of visual hallucination evaluation (VHE). This paper contributes a ChatGPT-Prompted visual hallucination evaluation Dataset (PhD) for objective VHE at a large scale. The essence of VHE is to ask an MLLM questions about specific images to assess its susceptibility to hallucination. Depending on what to ask (objects, attributes, sentiment, etc.) and how the questions are asked, we structure PhD along two dimensions, i.e., task and mode. Five visual recognition tasks, ranging from low-level (object / attribute recognition) to middle-level (sentiment / position recognition and counting), are considered. Besides a normal visual QA mode, which we term PhD-base, PhD also asks questions with inaccurate context (PhD-iac) or with incorrect context (PhD-icc), or with AI-generated counter common sense images (PhD-ccs). We construct PhD by a ChatGPT-assisted semi-automated pipeline, encompassing four pivotal modules: task-specific hallucinatory item (hitem) selection, hitem-embedded question generation, inaccurate / incorrect context generation, and counter-common-sense (CCS) image generation. With over 14k daily images, 750 CCS images and 102k VQA triplets in total, PhD reveals considerable variability in MLLMs' performance across various modes and tasks, offering valuable insights into the nature of hallucination. As such, PhD stands as a potent tool not only for VHE but may also play a significant role in the refinement of MLLMs.
- Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023.
- On the dangers of stochastic parrots: Can language models be too big? In ACM FAccT, 2021.
- Minigpt-v2: large language model as a unified interface for vision-language multi-task learning, 2023.
- Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023a.
- Plausible may not be faithful: Probing object hallucination in vision-language pre-training. In EACL, 2023b.
- An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
- GLM: General language model pretraining with autoregressive blank infilling. In ACL, 2022.
- Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2023.
- Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In CVPR, 2017.
- Detecting and preventing hallucinations in large vision language models, 2024.
- Language models are general-purpose interfaces. arXiv preprint arXiv:2206.06336, 2022.
- Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. arXiv preprint arXiv:2311.17911, 2023a.
- Language is not all you need: Aligning perception with language models, 2023b.
- Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023.
- Faithscore: Evaluating hallucinations in large vision-language models, 2023.
- An analysis of visual question answering algorithms. In ICCV, 2017.
- Large language models struggle to learn long-tail knowledge. In ICML, 2023.
- Volcano: Mitigating multimodal hallucination through self-feedback guided revision, 2023.
- Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023a.
- Evaluating object hallucination in large vision-language models. In EMNLP, 2023b.
- Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
- Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), llava-1.5, and other multi-modality models. arXiv preprint arXiv:2310.14566, 2023a.
- Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14565, 2023b.
- Improved baselines with visual instruction tuning, 2023c.
- Visual instruction tuning, 2023d.
- Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2521, 2022.
- Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195–3204, 2019.
- R OpenAI. Gpt-4 technical report. arXiv, pages 2303–08774, 2023.
- Object hallucination in image captioning. In EMNLP, 2018.
- Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
- An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.07397, 2023a.
- Evaluation and analysis of hallucination in large vision-language models. arXiv preprint arXiv:2308.15126, 2023b.
- mplug-owl: Modularization empowers large language models with multimodality, 2023a.
- mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178, 2023b.
- mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration, 2023c.
- Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045, 2023.
- Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data, 2023.
- Halle-switch: Controlling object hallucination in large vision language models, 2023.
- Minigpt-4: Enhancing vision-language understanding with advanced large language models, 2023.
- Jiazhen Liu (20 papers)
- Yuhan Fu (5 papers)
- Ruobing Xie (97 papers)
- Runquan Xie (3 papers)
- Xingwu Sun (32 papers)
- Fengzong Lian (10 papers)
- Zhanhui Kang (45 papers)
- Xirong Li (64 papers)