Hallucination of Multimodal Large Language Models: A Survey (2404.18930v1)

Abstract: This survey presents a comprehensive analysis of the phenomenon of hallucination in multimodal LLMs (MLLMs), also known as Large Vision-LLMs (LVLMs), which have demonstrated significant advancements and remarkable abilities in multimodal tasks. Despite these promising developments, MLLMs often generate outputs that are inconsistent with the visual content, a challenge known as hallucination, which poses substantial obstacles to their practical deployment and raises concerns regarding their reliability in real-world applications. This problem has attracted increasing attention, prompting efforts to detect and mitigate such inaccuracies. We review recent advances in identifying, evaluating, and mitigating these hallucinations, offering a detailed overview of the underlying causes, evaluation benchmarks, metrics, and strategies developed to address this issue. Additionally, we analyze the current challenges and limitations, formulating open questions that delineate potential pathways for future research. By drawing the granular classification and landscapes of hallucination causes, evaluation benchmarks, and mitigation methods, this survey aims to deepen the understanding of hallucinations in MLLMs and inspire further advancements in the field. Through our thorough and in-depth review, we contribute to the ongoing dialogue on enhancing the robustness and reliability of MLLMs, providing valuable insights and resources for researchers and practitioners alike. Resources are available at: https://github.com/showlab/Awesome-MLLM-Hallucination.

Hallucination in Multimodal LLMs: Survey and Perspectives

Introduction

The advent of multimodal LLMs (MLLMs) has ushered in significant advancements in tasks requiring the integration of visual and textual data, such as image captioning and visual question answering. Despite their capabilities, MLLMs often suffer from "hallucinations", where the generated content is inconsistent with the given visual data. This phenomenon undermines their reliability and poses challenges for practical applications. The paper provides a comprehensive survey of methodologies for identifying, evaluating, and mitigating hallucinations in MLLMs, presenting a detailed analysis of the causes, measurement metrics, and strategies to address these inaccuracies.

Hallucination Phenomenon in MLLMs

Hallucination in MLLMs typically manifests as generated text that inaccurately describes the visual content, either by fabricating content or misrepresenting the visual data. This issue compounds across various sub-domains of MLLMs, impacting their application in real-world scenarios. Addressing hallucinations in MLLMs is crucial for enhancing their reliability and trustworthiness in practical deployments.

Causes of Hallucinations

Understanding the origins of hallucinations in MLLMs is essential for devising effective mitigation strategies. The paper categorizes the causes into several broad areas:

• Data-related Issues: Issues such as insufficient data, noisy datasets, and lack of diversity in training data can lead to poor model generalization and hallucinations.
• Model Architecture: Inadequacies in the model design, particularly in how visual and textual data are integrated, can lead to an over-reliance on the LLM, overshadowing visual information.
• Training Artifacts: Training methods that overly focus on text generation accuracy without sufficient visual grounding also contribute to hallucinations, particularly during longer generation tasks where the model might lose focus on visual cues.
• Inference Mechanisms: Errors during the inference phase, such as improper handling of the attention mechanism across modalities, can exacerbate hallucination issues.

Evaluation Metrics and Benchmarks

The evaluation of MLLMs for hallucinations involves a diverse set of metrics and benchmarks. The paper reviews both existing and newly proposed methods to measure the degree and impact of hallucinations:

• Object-level Evaluation: Metrics that examine the accuracy of object recognition and description within the multimodal context play a crucial role.
• Factuality and Faithfulness: Metrics assessing the factual accuracy and faithfulness of the generated content against the visual data help in quantifying the extent of hallucinations.
• Benchmarks: Several benchmarks have been developed to standardize the evaluation of hallucinations across different models and datasets, facilitating a comparative analysis of MLLM performances.

Mitigation Techniques

Addressing the challenge of hallucinations involves a multi-faceted approach, encompassing improvements in data handling, model architecture adjustments, enhanced training protocols, and refined inference strategies:

• Enhanced Data Handling: Techniques such as augmenting training datasets with diverse and noise-free examples can reduce the risk of hallucinations.
• Architectural Improvements: Modifications to better integrate visual and textual data processing can help the model maintain focus on relevant visual cues.
• Advanced Training Techniques: Incorporating visual grounding during training or employing adversarial training methods can strengthen the model's ability to generate accurate descriptions.
• Inference Adjustments: Tweaking the inference process to maintain an equilibrium between textual and visual information can mitigate hallucinations.

Future Directions

The ongoing research into hallucinations in MLLMs highlights several potential pathways for future exploration:

1. Cross-modal Consistency: Developing mechanisms to ensure consistency between text and image modalities could significantly reduce hallucinations.
2. Ethical Considerations: As MLLMs become more prevalent, addressing the ethical implications of hallucinations in automated content generation is crucial.
3. Richer Benchmarks: There is a need for more comprehensive benchmarks that cover a wider array of scenarios and hallucination types to better evaluate MLLM performance.

Conclusion

This survey fosters a deeper understanding of hallucinations in MLLMs, providing valuable insights into their causes, impacts, and mitigation techniques. As the field of MLLMs continues to evolve, addressing hallucinations will remain a critical area of research, essential for enhancing the models' reliability and applicability in real-world applications.