PROVE-Bench: Dual Evaluation Benchmark
- PROVE-Bench is a benchmark that serves two distinct evaluation frameworks in AI, addressing open-ended visual language tasks and visual media editing.
- The 2024 usage employs a programmatic pipeline with DOCCI, GPT-4o, and scene-graph analysis to evaluate free-form responses on helpfulness and truthfulness.
- The 2026 variant applies a two-tier evaluation for object removal in images and videos, focusing on perceptual spatial and temporal consistency.
PROVE-Bench is a benchmark designation used for more than one evaluation framework in contemporary AI literature. In the 2024 vision-language literature, it denotes the benchmark component of PROVE, short for Programmatic VLM Evaluation, a framework for assessing free-form responses to open-ended visual questions through scene-graph-grounded measures of helpfulness and truthfulness (Prabhu et al., 2024). In a distinct 2026 visual-editing context, the same name is reused inside a different expansion of PROVE, namely Perceptual RemOVal cohErence, where PROVE-Bench refers to a two-tier benchmark for object removal in images and videos (Li et al., 14 May 2026). The shared label has therefore become bibliographically ambiguous, even though the underlying tasks, data, and metrics are unrelated.
1. Name, scope, and bibliographic ambiguity
The term PROVE-Bench does not denote a single stable benchmark family across the literature. Its two explicit uses in the supplied record differ in modality, task definition, and evaluation philosophy.
| Usage | Domain | Core evaluation object |
|---|---|---|
| PROVE-Bench in "Trust but Verify: Programmatic VLM Evaluation in the Wild" (Prabhu et al., 2024) | Vision-language modeling | Open-ended visual QA with scene-graph-based scoring |
| PROVE-Bench in "PROVE: A Perceptual RemOVal cohErence Benchmark for Visual Media" (Li et al., 14 May 2026) | Visual media editing | Object-removal quality in paired and no-GT video settings |
This ambiguity matters because both works use the name in a substantive way rather than as a passing shorthand. The earlier usage is the more established one in benchmark discussions about free-form VLM hallucination and groundedness. The later usage is embedded in a benchmark-and-metric package for perceptual evaluation of visual media removal. A plausible implication is that references to “PROVE-Bench” require paper-level disambiguation rather than term-level identification.
2. PROVE-Bench as Programmatic VLM Evaluation
In the 2024 usage, PROVE-Bench is designed for open-ended visual questions whose answers are free-form rather than close-ended. Its motivation is that hallucination evaluation becomes difficult when a response contains several distinct claims, each of which may or may not be visually supported. The benchmark therefore seeks to preserve the realism of generative visual question answering while making evaluation grounded and programmatically checkable (Prabhu et al., 2024).
The construction pipeline begins from the test split of DOCCI, a dataset of 5,000 manually curated images with captions averaging 136 words. These captions are converted into high-fidelity scene graphs with entities, attributes, and relations. An LLM, specifically GPT-4o, is then prompted with the caption, scene-graph information, and a Python SceneGraph API to generate 10–15 challenging, diverse, and unambiguous question-answer pairs per image, together with a Python function that answers each question by executing over the scene graph. Generated items are filtered programmatically and textually: 18.3% are removed because the program fails, 9.8% because program output does not semantically match the answer, and about 50% of the remaining QA pairs are removed by later quality filters. The final benchmark contains about 10.5k QA pairs built from 5k image-caption pairs, with questions averaging 10.3 words and answers averaging 13.4 words (Prabhu et al., 2024).
The benchmark is intentionally broader than captioning. The questions are described as resembling “in-the-wild” usage and span object description, attributes, counting, spatial reasoning, OCR-like text reading, and compositional visual understanding. This places PROVE-Bench between traditional discriminative hallucination probes and unconstrained VLM chat evaluation: it targets open-endedness, but with executable grounding.
3. Scoring framework: helpfulness and truthfulness
The defining technical contribution of the 2024 PROVE-Bench is its dual metric framework. A model response is not scored only for whether it overlaps the reference answer; it is decomposed into scene-graph tuples and assessed along two axes: whether it recovers relevant answer content, and whether its claims are visually supported (Prabhu et al., 2024).
Helpfulness is defined as a scene-graph recall measure over the ground-truth answer, after subtracting premises already present in the question:
$\help(\response) = \frac{\sum_{t \in \scenegraph(\answer)-\scenegraph(\question) } \max_{t' \in \scenegraph(\response)} \text{sim}(t, t')}{|\scenegraph(\answer)-\scenegraph(\question)|}$
Truthfulness is defined as a scene-graph precision measure over the response, where each response tuple is matched either to the caption-derived image scene graph or to an image-level visual entailment score:
$\truth(\response) = \frac{\sum_{t' \in \scenegraph(\response)} \text{max}\big(\max_{t \in \scenegraph(\imagecaption)}\text{sim}(t', t), p(\image \models t')\big) }{|\scenegraph(\response)|}$
The implementation uses LLaMA-3.1 with in-context prompting for tuple extraction from responses, Sentence-BERT for tuple similarity, and OFA fine-tuned for visual entailment. The benchmark’s conceptual claim is that these two axes are not redundant: a response can be helpful but hallucinated, or truthful but evasive. That distinction is central to the benchmark’s design and is one reason it differs from VLM-as-a-judge setups that produce a single scalar score from weak visual context.
4. Empirical findings and validation
The empirical results show that PROVE-Bench exposes a weak relationship between informativeness and groundedness. Across evaluated models, the paper reports a weak linear correlation of 0.03 between helpfulness and truthfulness, indicating that gains in one dimension do not reliably imply gains in the other (Prabhu et al., 2024).
On the reported table, GPT-4o achieves the highest helpfulness at 76.53, with truthfulness 80.92 and the best average score of 78.72. LLaVA-1.5 7B attains the highest truthfulness at 82.58, while Phi-3.5-Vision and Pixtral 12B are highlighted as models with relatively strong balance between the two dimensions. The paper’s broader interpretation is that recent VLM progress appears to have improved helpfulness more consistently than truthfulness.
Human validation is a major part of the benchmark’s credibility argument. In a study over 170 generated QA pairs, annotators judged 163/170 questions relevant (95.9%) and 167/170 answers correct (98.2%). In a separate metric-alignment study, automatic helpfulness scores achieved Pearson correlation 0.81 with human judgments, while truthfulness achieved 0.45. The lower truthfulness correlation is presented as evidence that automatic hallucination estimation remains harder than relevance estimation, but the overall results support the claim that the benchmark is materially more aligned with human judgment than prior open-ended scoring protocols.
5. PROVE-Bench in the PROVE visual-media framework
A separate 2026 paper reuses PROVE-Bench for a benchmark on object removal in images and videos, rather than VLM question answering. In this work, PROVE expands to Perceptual RemOVal cohErence, and the benchmark is coupled to two metrics: RC-S for spatial coherence and RC-T for temporal consistency (Li et al., 14 May 2026).
Here the benchmark addresses a different one-to-many problem: after removing an object, multiple reconstructions of the hidden background may be perceptually acceptable, so strict full-reference evaluation can be misleading. The authors therefore introduce a two-tier real-world benchmark. PROVE-M is an 80-video paired dataset with motion augmentation, and PROVE-H is a 100-video challenging subset without ground truth. PROVE-M uses real paired capture with masks and motion augmentation; PROVE-H emphasizes difficult real-world scenarios such as crowd scenes, dynamic backgrounds, highly textured backgrounds, complex reflections, and fast motion. The paired subset uses 1080p videos of 81 frames per video, and the benchmark is intended to support perceptual evaluation rather than exact reconstruction fidelity (Li et al., 14 May 2026).
This later PROVE-Bench is therefore not a variant or extension of the 2024 scene-graph QA benchmark. It belongs to a different evaluation tradition: perceptual coherence in editing, not groundedness in free-form visual reasoning.
6. Relation to adjacent benchmark families
The ambiguity around PROVE-Bench is amplified by nearby benchmark names and derivative phrasing. In theorem-proving literature, for example, the term appears analogically rather than canonically. FormalProofBench explicitly describes its task as “a PROVE-Bench-style formal theorem proving task,” meaning a Lean 4 proof-synthesis benchmark scored by kernel acceptance rather than by plausibility, but it does not define a benchmark named PROVE-Bench (Ravi et al., 27 Mar 2026). Similar analogical usage appears in miniCodeProps and FVAPPS, where the phrase denotes verifier-backed evaluation of proofs or verified programs rather than a shared benchmark artifact (Lohn et al., 2024, Dougherty et al., 8 Feb 2025).
There are also unrelated names that are easy to confuse with PROVE-Bench. ProBench names at least two different benchmarks: one for open-ended multimodal expert tasks and another for competitive programming (Yang et al., 10 Mar 2025, Yang et al., 28 Feb 2025). Neither is a PROVE-Bench variant. This suggests that the string “prove/pro/probench” has become a naming attractor for evaluation regimes that emphasize executable checking, expert difficulty, or groundedness, but the literature does not yet use it consistently.
In the strict encyclopedic sense, PROVE-Bench should therefore be treated as a context-dependent benchmark name. In the 2024 VLM literature it refers to programmatic evaluation of open-ended visual responses via scene-graph-grounded helpfulness and truthfulness. In the 2026 visual-editing literature it refers to a paired/no-GT benchmark for perceptual object-removal coherence. Any citation of the term without its parent paper is likely to be underspecified.
