Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement (2405.15973v3)

Abstract: Large vision-LLMs (LVLMs) have achieved impressive results in various visual question-answering and reasoning tasks through vision instruction tuning on specific datasets. However, there is still significant room for improvement in the alignment between visual and language modalities. Previous methods to enhance this alignment typically require external models or data, heavily depending on their capabilities and quality, which inevitably sets an upper bound on performance. In this paper, we propose SIMA, a framework that enhances visual and language modality alignment through self-improvement, eliminating the needs for external models or data. SIMA leverages prompts from existing vision instruction tuning datasets to self-generate responses and employs an in-context self-critic mechanism to select response pairs for preference tuning. The key innovation is the introduction of three vision metrics during the in-context self-critic process, which can guide the LVLM in selecting responses that enhance image comprehension. Through experiments across 14 hallucination and comprehensive benchmarks, we demonstrate that SIMA not only improves model performance across all benchmarks but also achieves superior modality alignment, outperforming previous approaches.

Enhancing Visual-Language Modality Alignment in Large Vision LLMs via Self-Improvement

The paper introduces a novel framework, Self-Improvement Modality Alignment (SIMA), aimed at improving the alignment between visual and language modalities in Large Vision LLMs (LVLMs) without the need for external AI models or data. The authors propose an innovative approach leveraging the LVLM’s intrinsic capabilities to generate responses and implement a self-critique mechanism to iteratively enhance its own performance.

Core Contributions

1. Self-Generating and In-Context Self-Critic Mechanism: SIMA employs a self-generating mechanism where the model uses prompts from existing vision instruction tuning datasets to generate responses. These responses are then evaluated using an in-context self-critic mechanism, where the LVLM itself assesses the quality of its responses based on predefined visual critic metrics.
2. Visual Critic Metrics: The paper introduces three key metrics used during the self-critique stage:
   • Accuracy in Object Description: Evaluates how accurately the objects in the image are described.
   • Accuracy in Depicting Relationships: Assesses the correctness in describing relationships between objects.
   • Accuracy in Describing Attributes: Measures the precision in depicting specific attributes of objects.
3. Performance and Benchmarking: The proposed framework is tested on LLAVA-1.5-7B across 14 different benchmarks. Results indicate significant improvements in both hallucination mitigation and comprehensive understanding, with an average performance increase of 7.5%.

Experimental Results

The experiments conducted demonstrate the efficacy of SIMA in enhancing LVLM alignment and performance:

• Hallucination Reduction: Using benchmarks like CHAIR, MM-Hal, and Mementos, SIMA demonstrates substantial reductions in object and behavior hallucination rates. Notably, SIMA achieves an average performance improvement of 16.1% on object hallucination benchmarks.
• Comprehensive Benchmark Performance: On nine comprehensive benchmarks, including LLaVA in the Wild, ScienceQA, TextVQA, and others, SIMA shows an average improvement of 3.5%, outperforming other preference tuning methods and several other open-source LVLMs.

The paper employs various preference tuning methods, drawing comparisons with baselines like LLaVA-RLHF, HA-DPO, and POVID. For instance, in hallucination benchmarks, SIMA outperforms LLaVA-1.5-7B, reducing CHAIRs from 50.8 to 40.9 and improving Mementos-Object F1 scores from 39.29% to 46.08%.

Critical Analysis

The use of self-generated responses and an in-context self-critic in SIMA marks a significant shift from traditional methods that rely on external models and datasets. This approach not only enhances performance but also ensures scalability and cost-effectiveness. By utilizing the model’s own capabilities, SIMA mitigates distribution shift issues commonly introduced by external datasets.

Future Directions

The implications of this research are noteworthy for both practical applications and theoretical advancements. Practically, the reduction in hallucination and improved understanding can enhance the reliability of LVLMs in applications requiring visual comprehension, such as autonomous vehicles, medical imaging analysis, and human-computer interaction.

Theoretically, the success of the self-improvement framework raises questions about the limits of LVLM self-evaluation and improvement. Future research could explore more sophisticated self-critique mechanisms, potentially incorporating unsupervised or semi-supervised learning strategies to further enhance model performance.

Additionally, while SIMA addresses immediate performance improvements, it does not tackle potential biases inherent in self-generated data. Future studies might examine methodologies to detect and correct for such biases, ensuring fairer and more accurate model outputs.

Conclusion

SIMA represents a significant advancement in the design of LVLMs, shifting towards self-reliant improvement mechanisms that do not depend on external models or data. This innovative framework enhances both the alignment between visual and language modalities and the overall performance of LVLMs across various benchmarks. The paper sets a new direction for future research in vision-LLMs, advocating for approaches that leverage intrinsic model capabilities for continuous self-improvement.