Verifying whether majority voting eliminates imperfect reasoning in self-improving VLM judges

Ascertain whether majority voting–based filtering of synthetic preference pairs for closed-ended tasks in the self-improving vision-language model judge training framework eliminates all imperfect reasoning in the judge’s decisions.

Background

The method constructs preference pairs for closed-ended tasks by sampling multiple responses from the base VLM and selecting the majority answer as the preferred option, then uses the current judge model to filter and train on reasoning traces that align with these synthetic preferences.

In the appendix, the authors present a counterexample where the judge selects the correct answer but provides flawed reasoning, and they explicitly state that they cannot conclusively verify that majority voting eliminates all imperfect reasoning, highlighting a remaining uncertainty about the reasoning robustness induced by this filtering strategy.

References

While we cannot conclusively verify that majority voting eliminates all imperfect reasoning, the empirical advantages shown in Section~\ref{sec:analysis_reasoning} suggest that consistency-based filtering may provide more robust supervision than correctness checking alone for learning generalizable judgment criteria.

— Self-Improving VLM Judges Without Human Annotations  (2512.05145 - Lin et al., 2 Dec 2025) in Appendix: Correctness Filter Negative Example

Bahuguna shows self-consistency can backfire on hard questions, with a binned analysis but without an agreement-index decomposition; its v2 revision states that the mechanism of the plurality-agreement gate's failure remains an open problem.

— Decomposing Wrong-Consensus Agreement in LLM Self-Consistency: A GPT-4.1 Case Study  (2608.18795 - Zhang et al., 19 Aug 2026) in Section 2, Related work