LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models (2502.14834v1)

Abstract: Existing Large Vision-LLMs (LVLMs) can process inputs with context lengths up to 128k visual and text tokens, yet they struggle to generate coherent outputs beyond 1,000 words. We find that the primary limitation is the absence of long output examples during supervised fine-tuning (SFT). To tackle this issue, we introduce LongWriter-V-22k, a SFT dataset comprising 22,158 examples, each with multiple input images, an instruction, and corresponding outputs ranging from 0 to 10,000 words. Moreover, to achieve long outputs that maintain high-fidelity to the input images, we employ Direct Preference Optimization (DPO) to the SFT model. Given the high cost of collecting human feedback for lengthy outputs (e.g., 3,000 words), we propose IterDPO, which breaks long outputs into segments and uses iterative corrections to form preference pairs with the original outputs. Additionally, we develop MMLongBench-Write, a benchmark featuring six tasks to evaluate the long-generation capabilities of VLMs. Our 7B parameter model, trained with LongWriter-V-22k and IterDPO, achieves impressive performance on this benchmark, outperforming larger proprietary models like GPT-4o. Code and data: https://github.com/THU-KEG/LongWriter-V

• The paper introduces LongWriter-V, leveraging a 22,158-example dataset and MMLongBench-Write benchmark to extend output lengths up to 10,000 words.
• The paper presents Iterative Direct Preference Optimization (IterDPO), a segmentation-based method that refines long-form outputs cost-effectively.
• Empirical tests demonstrate that the proposed approach outperforms larger models, achieving over 3,000-word generations while preserving input fidelity.

Enhancing Long-Output Capabilities in Vision-LLMs: A Review of LongWriter-V

The paper "LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-LLMs" addresses a critical limitation in existing Large Vision-LLMs (LVLMs), specifically their insufficiency in generating coherent and extensive texts beyond approximately 1,000 words, despite their capacity to handle large input contexts. Leveraging insights from supervised fine-tuning (SFT) practices, the authors introduce a novel dataset named LongWriter-V-22k, engineered to extend output capabilities substantially. This is coupled with a benchmark named MMLongBench-Write, designed to rigorously assess the long-output generation potential of Vision-LLMs (VLMs).

Key Contributions and Methodology

1. Dataset and Benchmark Development: The LongWriter-V-22k dataset consists of 22,158 examples that include multiple images, instructions, and outputs up to 10,000 words long. This dataset was created to address the lack of long output examples in traditional training datasets, which the authors posit as a primary bottleneck for output length in LVLMs.
2. Iterative Direct Preference Optimization (IterDPO): The authors present an innovative approach called IterDPO to combat the high costs associated with collecting human feedback for lengthier outputs. This method involves breaking down long outputs into segments, enabling iterative corrections that form preference pairs with the original outputs. This segmentation allows for efficient preference data usage, facilitating effective refinement of the VLM's capabilities without incurring prohibitive annotation costs.
3. Empirical Evaluation: The benchmark MMLongBench-Write comprises six tasks tailored to evaluate long-output generation, differentiating between professional and creative writing tasks. Extensive testing showed that models trained using the LongWriter-V-22k dataset, coupled with IterDPO, outperform larger proprietary models such as GPT-4o on this benchmark. The authors' model achieved significant advancements not only in extending the generation length beyond 3,000 words but also in maintaining high fidelity to the input content.

Implications and Future Directions

This work highlights the potential of targeted fine-tuning and innovative optimization techniques in enhancing the output capabilities of VLMs. The successful application of IterDPO demonstrates the viability of segment-based preference learning as a cost-effective method to enhance model quality. Empirically, this aligns with the broader trend of employing segmented learning strategies to handle large-scale data more efficiently.

In practice, the LongWriter-V and MMLongBench-Write resources set a new standard for evaluating and training models on long-form content generation in multimodal settings. This development has practical implications for applications requiring detailed content creation, such as automated report generation, educational material development, and creative content production.

Theoretically, this paper opens several avenues for further exploration, particularly in improving the interpretability and precision of long outputs in multimodal models. Future advancements may focus on optimizing IterDPO and extending its applicability across diverse tasks and languages, thereby refining the capacity of VLMs to produce contextually rich and detailed outputs across various domains.

In summary, "LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-LLMs" delineates a thoughtful approach to overcoming the limitations of existing VLMs in long output generation, setting a foundation for future exploration and application in the ever-evolving landscape of AI and machine learning.