Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling

Abstract: Vision-LLMs (VLMs) excel at visual understanding but often suffer from visual hallucinations, where they generate descriptions of nonexistent objects, actions, or concepts, posing significant risks in safety-critical applications. Existing hallucination mitigation methods typically follow one of two paradigms: generation adjustment, which modifies decoding behavior to align text with visual inputs, and post-hoc verification, where external models assess and correct outputs. While effective, generation adjustment methods often rely on heuristics and lack correction mechanisms, while post-hoc verification is complicated, typically requiring multiple models and tending to reject outputs rather than refine them. In this work, we introduce REVERSE, a unified framework that integrates hallucination-aware training with on-the-fly self-verification. By leveraging a new hallucination-verification dataset containing over 1.3M semi-synthetic samples, along with a novel inference-time retrospective resampling technique, our approach enables VLMs to both detect hallucinations during generation and dynamically revise those hallucinations. Our evaluations show that REVERSE achieves state-of-the-art hallucination reduction, outperforming the best existing methods by up to 12% on CHAIR-MSCOCO and 34% on HaloQuest. Our dataset, model, and code are available at: https://reverse-vlm.github.io.

LaTeX Guidelines for Author Response: A Concise Review

The paper titled "LaTeX Guidelines for Author Response" serves as a detailed instructional document aimed at facilitating authors in the preparation of rebuttals following peer reviews. This document is particularly significant for authors navigating the intricacies of academic conferences where response mechanisms are inherently part of the submission process. The text outlines numerous style and procedural guidelines, with explicit emphasis on the structured and constrained nature of author responses.

Core Guidelines and Instructions

The document is structured to ensure precision and uniformity in the preparation of rebuttal documents:

1. Length and Content: The rebuttal is strictly limited to a single-page PDF, emphasizing brevity and focus. Authors are advised to employ this response to correct factual inaccuracies and provide additional information only when solicited by reviewers. The paper prohibits the inclusion of new contributions, encompassing theorems, algorithms, or experiments not previously discussed or requested by reviewers.

2. Format and Structure: A clear emphasis is placed on maintaining a two-column format, adhering to specified dimensions for text areas, margins, headers, and footers. This standardization is critical for consistency across submissions.

3. Response Limitations: The document highlights that following a 2018 PAMI-TC motion, significant new experiments should not be requested by reviewers nor added by authors. This guideline aims to ensure submissions remain within the original scope during the rebuttal phase.

4. Figures and Tables: Any visual aids must be utilized prudently to support the written response. The use of updated and comparable figures or tables from the original submission or supplementary materials can improve the clarity of responses but must not exceed the prescribed rebuttal length.

5. Anonymity and Reference: Maintaining anonymity is critical. The use of identifiers or links that may compromise this anonymity is strictly prohibited. Moreover, references within the rebuttal must adhere to citation norms, complemented by a standardized bibliography format.

Practical and Theoretical Implications

By establishing these guidelines, the paper contributes substantially to the peer review process, ensuring a uniform structure that enhances the efficiency and clarity of communication between authors and reviewers. This document serves not only as a practical template but as a theoretical framework advocating standardization in academic rebuttals. It underscores the importance of focused communication in academia, where precision and economy of content are paramount.

Speculative Future Directions

As AI and machine learning continue to develop, their integration into the review and rebuttal process could lead to more adaptive guidelines that expedite and optimize the reviewer-author interaction. AI-driven systems might in future be used to assess the completeness and clarity of author responses before submission, providing an additional layer of quality control. Furthermore, the evolution of templates such as this will likely continue, accommodating new media and hybrid formats as academic publishing evolves.

In conclusion, the document serves as a seminal guide in the academic process, emphasizing brevity, clarity, and adherence to structured communication. These guidelines not only assist authors in effectively conveying their responses but also set a standard in the creation of academic rebuttals, fostering a consistent scientific dialogue.