Vision-Language Model Dialog Games for Self-Improvement (2502.02740v1)

Abstract: The increasing demand for high-quality, diverse training data poses a significant bottleneck in advancing vision-LLMs (VLMs). This paper presents VLM Dialog Games, a novel and scalable self-improvement framework for VLMs. Our approach leverages self-play between two agents engaged in a goal-oriented play centered around image identification. By filtering for successful game interactions, we automatically curate a high-quality dataset of interleaved images and text. We demonstrate that fine-tuning on this synthetic data leads to performance gains on downstream tasks and generalises across datasets. Moreover, as the improvements in the model lead to better game play, this procedure can be applied iteratively. This work paves the way for self-improving VLMs, with potential applications in various real-world scenarios especially when the high-quality multimodal data is scarce.

• The paper introduces VLM Dialog Games, a framework using two VLM agents (Describer and Guesser) in a self-play scenario to generate high-quality synthetic data for training.
• By filtering data from successful dialog game interactions, the method enables iterative self-improvement of VLMs and reduces reliance on extensive human-labeled data.
• Experiments show significant performance improvements on VQA benchmarks and robotics tasks, including a 10.4% increase in VQAv2 yes/no accuracy when trained with OpenImages data.

Vision-LLM Dialog Games for Self-Improvement

The paper "Vision-LLM Dialog Games for Self-Improvement" presents a novel framework that aims to address the challenges of curating high-quality training data for Vision-LLMs (VLMs). The proposed method leverages dialog games as a tool for self-improvement, which automatically generates and filters a synthetic dataset of high-quality interleaved images and text. This process is facilitated by engaging two VLM agents in a self-play scenario designed around image identification tasks.

Key Contributions

1. VLM Dialog Games Framework: The primary innovation introduced is the VLM Dialog Games, which involves a "Describer" and a "Guesser" agent. Engaging in goal-oriented play, these agents work to identify a target image from a set containing distractors. The novelty lies in using the gameplay outcomes to gauge success, thus generating training data from successful interactions.
2. Iterative Improvement Process: By leveraging the dialog game's success filtering mechanism, the authors demonstrate that the VLMs' performance can be improved iteratively. As the agents enhance their understanding and become more proficient in distinguishing target images, they can generate increasingly refined datasets for further fine-tuning.
3. Robust Experimental Evaluation: The efficacy of the framework is validated through extensive experiments on general visual question answering (VQA) and robotics-based image recognition. Using datasets like OpenImages and DOCCI, improvements on benchmarks like VQAv2 were noted, emphasizing the method's capacity to generalize across diverse datasets.

Numerical Results and Implications

Experimentally, the authors report notable performance increments. For instance, fine-tuning VLMs using the dialog game data from the DOCCI dataset resulted in a 6.8% improvement on VQA yes/no questions and a 2.3% increase in object counting accuracy on benchmark datasets. Similarly, when trained with OpenImages, these improvements were even more pronounced, showcasing a 10.4% enhancement in yes/no question accuracy. Such results highlight the framework's robustness in elevating VLM capabilities.

In the domain of robotics, where high-quality domain-specific data is scarce, the approach significantly improved success detection in manipulation tasks by 16.5%, indicating its utility in specialized applications. These improvements are particularly valuable in fields like robotics, where data scarcity often constrains model development.

Theoretical and Practical Implications

Theoretically, the framework provides a scalable solution to generating synthetic data, aligning with the recent trends of employing synthetic datasets as a complementary strategy to conventional data collection techniques. By minimizing the need for extensive human-labeled data, the method offers significant cost efficiencies and opens avenues for model enhancement in data-scarce environments.

Practically, the dialog games framework suggests a direction that future VLM development might take, especially in fields requiring multimodal understanding. The iterative nature of the approach ensures that VLMs can continuously refine their understanding, adapting to new tasks with minimal additional supervision.

Future Speculations

Considering the success of this self-improvement framework, future research could explore various avenues such as:

• Extending the dialog game architectures to explore more complex task settings or different dialog strategies.
• Investigating the optimal balance between game complexity and data quality to maximize model performance.
• Integrating additional modalities or leveraging the approach in real-time systems for adaptive learning.

In conclusion, the VLM Dialog Games framework presents a promising pathway for advancing VLM capabilities through self-generation of training data. Its application can potentially transcend current barriers posed by data availability, particularly enriching fields that rely on detailed and accurate multimodal understanding.