Med-RewardBench: Medical Reward Model Benchmark

• Med-RewardBench is a specialized benchmark that formalizes a pairwise preference task for evaluating medical reward models and judge-capable MLLMs in clinical contexts.
• It comprises 1,026 expert-annotated multimodal cases spanning 13 organ systems and 8 clinical departments, evaluated across six clinically critical dimensions.
• The benchmark reveals that large general-purpose MLLMs often outperform medical-specific judges, highlighting challenges in nuanced clinical reasoning and expert alignment.

Searching arXiv for the benchmark and closely related medical reward-model benchmarks. Searching arXiv for "Med-RewardBench" and related terms. Med-RewardBench is a medical-specific benchmark for evaluating medical reward models (MRMs) and judge-capable multimodal LLMs (MLLMs) in multimodal clinical settings. It was introduced as “the first benchmark specifically designed to evaluate MRMs and judges in medical scenarios,” motivated by the observation that medical applications such as disease diagnosis and clinical decision-making require outputs that are accurate, context-sensitive, and aligned with professional standards, while existing benchmarks either evaluate MLLMs as solvers or focus on non-medical reward benchmarking dimensions (Ding et al., 29 Aug 2025).

1. Conceptual role and problem setting

Med-RewardBench is situated at the interface between multimodal clinical AI evaluation and preference-based model supervision. The benchmark addresses a specific need: automatic verification and preferential selection of clinically sound responses during both training, such as RLHF, and inference, such as best-of-$n$ selection. In this setting, MRMs and MLLM-as-a-judge systems are not treated as generic evaluators; they are expected to identify which of two candidate medical responses is superior under clinically critical criteria (Ding et al., 29 Aug 2025).

A central distinction in the benchmark is between evaluating a model as a solver and evaluating it as a judge. Prior medical multimodal benchmarks assess models on tasks such as VQA and diagnosis, whereas Med-RewardBench evaluates reward models and judge-capable MLLMs. This matters because the clinical evaluation problem is not reducible to task completion accuracy alone. The benchmark explicitly targets dimensions such as diagnostic accuracy and clinical relevance, which general MLLM reward benchmarks, including VL-RewardBench, do not capture in medical form (Ding et al., 29 Aug 2025).

The benchmark therefore formalizes medical response evaluation as a pairwise preference task. Each case presents an image, an instruction, and two candidate responses, and the judge must determine which response is superior for each clinically oriented dimension. This design makes Med-RewardBench directly usable for benchmarking MRMs and MLLM judges rather than for measuring end-task problem solving (Ding et al., 29 Aug 2025).

2. Scope, coverage, and formal structure

Med-RewardBench comprises a multimodal dataset spanning 13 organ systems and 8 clinical departments, with 1,026 expert-annotated cases of single-image, single-turn pairwise comparisons (Ding et al., 29 Aug 2025). The organ systems are Abdomen, Brain, Breast, Chest, Eye, Foot, Gastrointestinal tract, Heart, Lower limb, Lung, Oral cavity, Pelvic cavity, and Upper limb. The departments are Ophthalmology, Radiology, Otolaryngology, General Surgery, Gastroenterology, Pulmonology, Cardiology, and Neurology (Ding et al., 29 Aug 2025).

The case distribution is explicitly enumerated in the benchmark: ABD 79, BRE 80, BRN 80, CHE 80, EYE 73, FOT 80, GI 78, HRT 80, LL 80, LNG 80, OC 76, PC 80, and UL 80, for a total of 1,026 cases (Ding et al., 29 Aug 2025). Source datasets span radiology, histology/pathology, and general medicine, and Med-RewardBench focuses on single-image inputs with textual instructions. The paper states that imaging types are inherited from public datasets but does not provide per-modality type distributions such as X-ray, CT, MRI, or ultrasound (Ding et al., 29 Aug 2025).

Each example contains a medical image $X$, a user instruction or question $Q$, and two candidate responses $A$ and $B$. The instruction $I$ includes a system prompt $S$, question $Q$, the image, and the two candidate answers. The formal preference-pair representation is $(X, I, R_a, R_r)$, where $X$ is the input image, $X$0 is the instruction, and $X$1 are two responses. Dataset construction is given as

$X$2

with $X$3 (Ding et al., 29 Aug 2025).

The benchmark also maintains near-symmetry in label position. Option A is correct 51.3% of the time and option B is correct 48.7% of the time. This reduces trivial positional bias in pairwise judging and makes agreement with expert preference the salient criterion (Ding et al., 29 Aug 2025).

3. Clinical dimensions and annotation methodology

Med-RewardBench evaluates responses along six clinically oriented dimensions: Accuracy (ACC), Relevance (REL), Comprehensiveness (COM), Creativity (CRE), Responsiveness (RES), and Overall (OVE) (Ding et al., 29 Aug 2025). These dimensions are defined as follows.

Dimension	Definition
ACC	correctness of medical information in the response
REL	how directly the response addresses the provided instruction
COM	coverage of all relevant aspects of the question
CRE	insightful or innovative interpretation
RES	timely and appropriate feedback to patient-related inquiries
OVE	holistic assessment of response quality and utility

The inclusion of Creativity and Responsiveness alongside Accuracy and Relevance reflects the benchmark’s attempt to capture a broader notion of clinical response quality than factual correctness alone. The paper frames these as clinically critical evaluation dimensions rather than optional stylistic criteria (Ding et al., 29 Aug 2025).

Dataset curation follows a rigorous three-step process. The first stage, image-question pair collection, draws from PubMedVision, LLaVA-Med, Quilt-Instruct, CARES, and RULE. Difficulty filtering is performed with five small or weak MLLMs—DeepSeek-VL-1.3B-chat, Qwen2-VL-2B-Instruct, BLIP2-OPT-2.7B, PaliGemma-3B, and H2OVL-Mississippi-2B—and pairs answerable correctly by fewer than three of these models are retained as “difficult.” Balanced sampling then randomly selects 80 instructions per organ category, followed by medical-professional screening for clinical relevance, accuracy, complexity, and image quality; ambiguous or low-quality items are refined or removed (Ding et al., 29 Aug 2025).

The second stage, MLLM response collection, uses 12 widely used MLLMs at 7B–72B scale to generate answers for each image-question pair. Two responses are uniformly sampled for A/B judging. The construction is designed to ensure fair representation across models and a balanced A/B correctness ratio (Ding et al., 29 Aug 2025).

The third stage, comparison with human annotations, uses 3 registered general practitioners with 4–5 years of clinical experience, selected by clinical experience and board certification criteria and trained with detailed physician evaluation guidelines. For each instruction, experts select the superior response for each of the six dimensions. Uncertain cases are resolved by majority voting. A reliability check re-examines 84 uncertain samples; every item reaches agreement from at least two experts, and many achieve full consensus. Cohen’s $X$4 values are not reported; the benchmark relies on majority voting and consensus checks instead (Ding et al., 29 Aug 2025).

4. Evaluation protocol and benchmarked models

The evaluation target is reward models and judge-capable MLLMs. Each model sees the same instruction template used for human experts: system prompt, question, image, and answers A/B. A default pairwise-comparison prompt is used, and evaluations are run with fixed decoding parameters to ensure consistency across models, although temperatures and seeds are not detailed (Ding et al., 29 Aug 2025).

The scoring mechanism is absolute pairwise selection per dimension. Judges choose the superior response, A or B, for each of ACC, REL, COM, CRE, RES, and OVE. The primary metric is accuracy, defined as agreement between the model’s choice and the expert majority choice, reported per dimension and overall and further summarized across organ systems and departments. No dimension weighting, numerical scoring scale, Bradley–Terry model, Thurstone model, correlation metric, or calibration metric is reported (Ding et al., 29 Aug 2025).

A total of 32 state-of-the-art MLLMs are evaluated. These include open-source general-purpose MLLMs such as VILA1.5-3B, xGen-MM-instruct, DeepSeek-VL2-Small, Phi-3.5-Vision, LLaVA variants, Chameleon-7B, Qwen2-VL-7B-Instruct, Qwen2.5-VL-7B, Molmo-7B, MiniCPM variants, InternVL variants, GLM-4V, LLaMA-3.2-Vision, Pixtral-12B, and Qwen2-VL-72B; medical-specific MLLMs such as LLaVA-Med-7B, STLLaVA-Med-7B, HuatuoGPT-Vision-7B, Med-Flamingo-9B, and MedDr-40B; and proprietary models such as GPT-4o, Claude-3.5 Sonnet, Gemini-1.5 Pro, and OpenAI O1 (Ding et al., 29 Aug 2025).

The benchmark therefore compares multiple model families under a single expert-annotated clinical judging setup. This suggests that Med-RewardBench is intended not merely as a leaderboard, but as a controlled instrument for measuring alignment between automated judges and expert medical preference across clinically heterogeneous multimodal cases.

5. Empirical findings and observed failure modes

The empirical results show only moderate agreement with expert preferences even for the strongest systems. Across organs, OpenAI O1 reaches 68.86% overall and is reported as the top-performing proprietary judge. Qwen2-VL-72B reaches 65.27%, Qwen2.5-VL-7B 64.99%, Gemini-1.5 Pro 63.51%, GLM-4V 62.19%, and Pixtral-12B 61.63% (Ding et al., 29 Aug 2025).

A prominent finding is that medical-specific judges underperform relative to large general-purpose MLLMs. Their typical averages are around 54–55% overall, and the paper explicitly notes that models such as HuatuoGPT-Vision struggle to beat random chance in some settings. This is presented as evidence of limited judgment capability in complex clinical reasoning despite medical pretraining (Ding et al., 29 Aug 2025). A common misconception is therefore that domain-specific pretraining alone should produce superior medical judging; Med-RewardBench reports the opposite pattern.

Dimension-level analysis indicates relatively stable median performance in ACC at roughly 55% and RES in the 50–60% range, but higher variance and lower medians in REL, COM, and especially CRE. The benchmark interprets these patterns as highlighting challenges in context integration, breadth of coverage, and medically relevant creativity (Ding et al., 29 Aug 2025). This suggests that current models more readily recover coarse factual correctness than nuanced multidimensional clinical judgment.

Organ-specific and department-level results further localize difficulty. Cardiac imaging reaches top organ-level accuracy around 76% for Gemini and Qwen2-VL-72B, and several models exceed 70% accuracy on GI imaging. By contrast, in ophthalmology no model surpasses 70% accuracy, and fine-grained visual judgments remain challenging. At the department level, O1 ranks first in three departments and exceeds 62% in all departments, with OPH 68.49%, RAD 65.59%, and GI 75.95%; Gemini-1.5 Pro is strong in GI at 74.68% and ENT at 68.29%; and Qwen2-VL-72B is described as the most balanced open-source judge across specialties, generally around 53–64%. ENT shows the largest spread, approximately 41–68%, indicating heterogeneous multimodal reasoning difficulty (Ding et al., 29 Aug 2025).

The paper characterizes these outcomes as misalignment patterns. General-purpose judges, especially larger ones, align better with expert preferences overall than medical-specific judges, but notable gaps persist in domains requiring nuanced visual detail and specialty-specific criteria, particularly ophthalmology. Creativity and comprehensiveness are especially variable, suggesting that current judges inadequately capture nuanced clinical context and multi-factor reasoning (Ding et al., 29 Aug 2025).

6. Baseline judge training, practical use, and relation to adjacent benchmarks

In addition to evaluation, the paper introduces two author-trained baselines built on Qwen2-VL-7B: Qwen2-VL-Judge and Qwen2-VL-DPO. Training uses 10,000 “difficult” image-question pairs sampled from the initial pool $X$5, non-overlapping with the 1,026-case benchmark, for 3 training epochs with LLaMA-Factory. Qwen2-VL-Judge uses SFT on high-quality responses from Qwen2-VL-72B as ground-truth labels, while Qwen2-VL-DPO uses preference pairs with Qwen2-VL-72B responses as “chosen” and Qwen2-VL-2B as “rejected,” excluding identical-response pairs and balancing A/B label positions to mitigate positional bias (Ding et al., 29 Aug 2025).

These baselines improve substantially over the zero-shot base model. Qwen2-VL-7B achieves 52.16% overall accuracy, Qwen2-VL-Judge reaches 57.46% for a gain of 5.30 points, and Qwen2-VL-DPO reaches 54.91% for a gain of 2.75 points (Ding et al., 29 Aug 2025). The paper attributes the larger gain of SFT to rubric-aligned supervision from a stronger model. It does not provide ablations on fusion strategies or explicit rubric-aware training beyond the six dimensions incorporated in the judging prompt.

For practical use, Med-RewardBench is intended to evaluate MRMs and judge-capable MLLMs via pairwise comparison against expert-annotated preferences across the six dimensions. The input format per case is a system prompt and user question, image $X$6, and two candidate answers $X$7 and $X$8. Judges must output, for each dimension, which candidate is superior. Dimension-wise accuracy is then computed as the proportion of cases where the model’s chosen response matches the expert preference, with summaries reported across organs and departments as needed. The paper stresses fixed inference settings and single-turn evaluation for comparability, but does not publish seeds, hardware specifications, or exact decoding parameters (Ding et al., 29 Aug 2025).

Several limitations are explicit. The benchmark covers only single-image, single-turn pairwise comparisons, not multi-image or multi-turn clinical workflows. Although the six dimensions are clinically motivated, the benchmark does not explicitly include safety, guideline adherence, or calibration metrics. Fine-grained imaging modality distributions are not broken out, and more specialized subfields are not isolated. The paper also states that source code and data are to be released, but no repository link, license, or DOI is provided at publication time, and the text does not report de-identification procedures or IRB approvals (Ding et al., 29 Aug 2025).

Med-RewardBench should also be distinguished from adjacent efforts. “MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning” is a process-level reward model benchmark for medical reasoning chains, not the same benchmark, and the authors of that paper do not use “Med-RewardBench” as a synonym (Wu et al., 19 Apr 2026). Likewise, “medR: Reward Engineering for Clinical Offline Reinforcement Learning via Tri-Drive Potential Functions” does not introduce Med-RewardBench by name, but proposes a blueprint for evaluating medical reward functions in offline clinical RL rather than pairwise multimodal response judges (Xu et al., 3 Feb 2026). Taken together, these distinctions place Med-RewardBench specifically within the evaluation of multimodal medical judges and reward models for response selection, rather than process-level reasoning verification or clinical RL reward design.