Woodpecker: Hallucination Correction for Multimodal Large Language Models (2310.16045v2)

Abstract: Hallucination is a big shadow hanging over the rapidly evolving Multimodal LLMs (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content. In order to mitigate hallucinations, existing studies mainly resort to an instruction-tuning manner that requires retraining the models with specific data. In this paper, we pave a different way, introducing a training-free method named Woodpecker. Like a woodpecker heals trees, it picks out and corrects hallucinations from the generated text. Concretely, Woodpecker consists of five stages: key concept extraction, question formulation, visual knowledge validation, visual claim generation, and hallucination correction. Implemented in a post-remedy manner, Woodpecker can easily serve different MLLMs, while being interpretable by accessing intermediate outputs of the five stages. We evaluate Woodpecker both quantitatively and qualitatively and show the huge potential of this new paradigm. On the POPE benchmark, our method obtains a 30.66%/24.33% improvement in accuracy over the baseline MiniGPT-4/mPLUG-Owl. The source code is released at https://github.com/BradyFU/Woodpecker.

• The paper introduces Woodpecker, a training-free framework that corrects hallucinations in multimodal language models using a structured five-stage process.
• It significantly improves model accuracy, with MiniGPT-4's performance up by 30.66% and mPLUG-Owl's by 24.67% on datasets like POPE and MME.
• The approach enhances interpretability by using bounding boxes and offers a flexible, modular correction method applicable without retraining.

Evaluating Hallucination Correction in Multimodal LLMs

The paper presents a novel approach to address hallucination in Multimodal LLMs (MLLMs). Hallucinations refer to inconsistencies between generated text and image content. The existing solutions typically require retraining models with specially curated data, increasing computational demand. In contrast, this work introduces a training-free framework named Woodpecker, which repairs hallucinations post-generation.

Methodology Overview

Woodpecker operates through a structured five-stage process. Firstly, key concept extraction identifies main objects in the generated text. Then, question formulation generates targeted queries about these objects, addressing both object-level and attribute-level hallucinations. The third stage, visual knowledge validation, derives answers from an open-set detector and a VQA model. Visual claim generation summarizes these QA pairs into a knowledge base. Finally, hallucination correction refines the original text using this base, appending bounding boxes for interpretability.

Experimental Evaluation

Several datasets, like POPE, MME, and LLaVA-QA90, are employed to evaluate the framework. On POPE, Woodpecker enhances baseline MLLMs' accuracy significantly, showcasing the framework's utility across diverse sampling methods. For example, MiniGPT-4's accuracy improved by 30.66%, while under the adversarial setting, mPLUG-Owl's accuracy increased by 24.67%. On MME, the correction mechanism boosted both object-level and attribute-level scores, highlighting its comprehensive efficacy. Additionally, GPT-4V-aided evaluation revealed improvements in accuracy and detailedness in image descriptions, underscoring Woodpecker's ability to add depth while correcting errors.

Theoretical and Practical Implications

This research implies a shift in tackling hallucinations without the need for extensive retraining, offering a flexible solution adaptable to various MLLMs. The interpretability of Woodpecker, achieved through bounding boxes, presents practical reliability for real-world applications, allowing for easier verification of generated content. From a theoretical standpoint, the structured correction methodology offers insights into combining visual analysis with language refinement in a modular fashion.

Future Directions

Future work can explore extending this framework to address more nuanced hallucinations involving complex attributes or interactions between multiple objects. Improving VQA models or integrating more advanced detectors could further elevate the robustness of the correction process. Additionally, the framework's applicability across emerging domains and new MLLMs remains an exciting area for exploration.

In conclusion, Woodpecker presents a promising step towards efficiently mitigating hallucinations in MLLMs, offering a flexible, interpretable, and effective solution across various model architectures and settings.