OpenMedReason: Multimodal Clinical Reasoning

• OpenMedReason is a large, open medical vision–language resource that defines multimodal clinical reasoning by combining visual evidence, clinical knowledge, and rationale.
• It integrates approximately 450K image–question–answer instances with curated, scientifically grounded reasoning traces to ensure detailed decision-making processes.
• Its benchmark, OpenMedReason-Bench, decomposes model capability into perception, medical knowledge, and rationale, driving measurable improvements in VQA performance.

OpenMedReason is a large, open medical vision–language resource designed to supervise and evaluate reasoning rather than only answer matching in multimodal clinical problems. It comprises approximately 450K image–question–answer instances whose reasoning traces are primarily derived from curated biomedical, human-authored scientific articles, together with OpenMedReason-Bench, a held-out benchmark that decomposes model capability into perception, medical knowledge, and rationale quality (Baghbanzadeh et al., 10 Jun 2026). In this formulation, a medically useful large vision-LLM must extract appropriate visual evidence, correctly apply clinical knowledge, and connect evidence and knowledge through coherent reasoning to a decision, rather than merely produce a correct final answer.

1. Conceptual motivation and problem setting

OpenMedReason addresses a limitation of most existing medical VQA and LVLM datasets: they provide image–question–answer triples and evaluate models by final-answer accuracy, but do not supervise or measure how the answer was reached (Baghbanzadeh et al., 10 Jun 2026). In medicine, this omission is consequential because failure modes often sit in intermediate steps, such as misreading a chest X-ray, misinterpreting a lab value, or skipping a crucial differential. The project therefore treats multimodal clinical reasoning as a composition of three requirements: perception of visual evidence, application of medical knowledge, and rationale linking evidence to answer.

The resource is motivated by the observation that, without supervision on reasoning steps, LVLMs can hallucinate visual findings or clinical facts, give correct answers for the “wrong reasons,” or produce generic, style-consistent chains of thought that are not truly grounded. OpenMedReason responds by supplying scientifically grounded reasoning traces aligned with biomedical figures and text, and by defining an evaluation framework that scores capability decomposed into perception, knowledge, and rationale. This differs from synthetic chain-of-thought corpora whose traces are generated end-to-end by a single model and may inherit that model’s priors and stylistic quirks (Baghbanzadeh et al., 10 Jun 2026).

Related open efforts clarify the broader research context. MedCaseReasoning extracts clinician-authored diagnostic reasoning from open-access case reports for text-based diagnosis (Wu et al., 16 May 2025), and MedR-Bench evaluates reasoning across assessment recommendation, diagnostic decision-making, and treatment planning on structured patient cases (Qiu et al., 6 Mar 2025). This suggests that OpenMedReason occupies a distinct position within the emerging medical reasoning literature: it centers multimodal, image-grounded clinical reasoning for LVLMs rather than text-only case analysis.

2. Corpus composition, sources, and coverage

OpenMedReason contains a training corpus of approximately 450,000 instances and a held-out evaluation suite. Its composition is summarized below (Baghbanzadeh et al., 10 Jun 2026).

Component	Size	Description
OpenMedReason grounded subset	196,000	Open-PMC-derived examples with curated reasoning traces
Additional VQA-derived subset	254,000	Existing medical datasets with generated structured rationales
OpenMedReason-Bench	1,500	Held-out benchmark with no overlap at article, image, or question level

The core grounded subset is derived from Open-PMC-18M, a corpus of subfigure images, captions, and in-text references extracted from open-access biomedical publications in PubMed Central. The authors state that all source data are public, de-identified figures from biomedical articles, and that the dataset inherits the licensing of Open-PMC and underlying PMC open-access licenses (Baghbanzadeh et al., 10 Jun 2026). They also state that the corpus is biased toward published, often rare or pedagogically interesting cases, that it should not be considered evidence of clinical readiness or safety, and that derived models must not be used for clinical decision-making.

The resource spans more than eight imaging and visual modalities. Radiology and pathology are dominant, reflecting the biases of the published literature, but there is also substantial visible-light and diagrammatic content. The paper names radiological scans, microscopic images, visible light photographs, charts, biomedical plots, angiograms, and pH maps, with qualitative modality groupings that include X-ray, CT, MRI, ultrasound, histology, cytology slides, dermatology, ophthalmology, endoscopy, intra-operative photographs, and other biomedical figures (Baghbanzadeh et al., 10 Jun 2026).

A clinical task taxonomy, developed by two human annotators using bottom-up, inductive grouping, organizes 19 categories into five families: Perception, Diagnosis, Management, Risk, and Workup. This taxonomy is part of the annotation protocol and reflects the project’s attempt to supervise not only recognition of findings but also downstream clinical interpretation and decision-oriented reasoning (Baghbanzadeh et al., 10 Jun 2026).

3. Curation pipeline and reasoning-trace construction

For Open-PMC-derived instances, OpenMedReason uses a multi-stage, source-grounded pipeline rather than directly accepting image–caption pairs (Baghbanzadeh et al., 10 Jun 2026). It begins with quality filtering. From 18M image–text pairs, the pipeline applies a visual usability filter requiring minimum resolution, reasonable aspect ratio, limited white borders, sufficient sharpness, and no severe compression artifacts; this yields approximately 30k high-quality images before semantic filtering. A second, LLM-based text and context filter retains only English, clinically meaningful context with image–text alignment and “reasoning readiness,” meaning that the context contains enough evidence to support a question, answer, and verifiable rationale.

Each surviving image–context pair is then assigned one or more clinical task categories with gpt-5-mini, but only if a plausible answer exists in the context and there is sufficient explanatory evidence for a reasoning trace. Question generation is constrained: the model produces a USMLE-style item with stem, options, and correct answer, but the image must be necessary, the question must not be solvable by text alone, and the stem must avoid explicit leakage of the diagnosis or key visual findings. Answer formats include 4–5 option multiple choice, Yes/No, True/False, and Normal/Abnormal.

Reasoning traces are generated in two stages. First, a medical LVLM (OctoMed) generates a draft trace conditioned on the image, source caption and in-text context, question, and correct answer. Second, gpt-5-mini refines the draft to remove unsupported or article-level content, ensure alignment with image and context, and rewrite it into the fixed structure modality and visual target → visual evidence → clinical interpretation → justified answer (Baghbanzadeh et al., 10 Jun 2026). The refined trace is organized into labeled sections: Perception, Clinical context, Clinical interpretation and medical knowledge, and Answer justification.

A final verifier re-examines the image, context, question, answer, and rationale and rejects any instance if source consistency fails, answer justification fails, or reasoning utility fails. Only instances passing all three checks are retained. This produces the 196k “high-fidelity” grounded subset. For the additional 254k examples drawn from datasets such as PMC-VQA, SLAKE, VQA-RAD, and PathVQA, the original image, question, and answer are retained, while rationales are generated with gpt-5-mini and mixed with the grounded subset for supervised fine-tuning (Baghbanzadeh et al., 10 Jun 2026).

At training and RL time, model outputs use a machine-friendly wrapper:

<think> ... reasoning ... </think> <answer>X</answer>

This format formalizes reasoning as a first-class output rather than an implicit latent process.

4. Benchmark design and fine-grained evaluation

OpenMedReason-Bench is a 1,500-example held-out benchmark built from the same curation pipeline but not used for training. Its purpose is to evaluate three distinct capabilities—Perception (P), Medical knowledge (K), and Rationale (R)—and to provide a trace-based score in addition to final-answer accuracy (Baghbanzadeh et al., 10 Jun 2026).

Each benchmark example contains an image, question, options, correct answer, reference reasoning trace, and a derived checklist of atomic units. These units are labeled by axis. Perception units are image-grounded facts such as modality, location, lesion appearance, or presence and absence of findings. Medical knowledge units are general clinical facts independent of the specific case. Rationale units encode the inferential links from evidence to answer. Each unit has a topic, claim, presence question, correctness question, source quote, and an importance flag.

The benchmark defines two labels per unit. Presence is coded as $p_i \in \{0,1,2\}$, where 0 means not mentioned, 1 means vaguely or implicitly mentioned, and 2 means clearly asserted. Correctness is coded as $c_i \in \{-1,0,+1\}$, where $-1$ means asserted but incorrect, $0$ means not applicable, and $+1$ means asserted and correct. Axis-level scores are:

$$\mathrm{Presence}_a = \frac{1}{N_a} \sum_{i \in a} \frac{p_i}{2}$$

$$\mathrm{Correctness}_a = \frac{ \left|\{i \in a : p_i \geq 1 \ \wedge \ c_i = +1\}\right| }{ \left|\{i \in a : p_i \geq 1\}\right| }$$

$$\mathrm{TraceScore} = \frac{1}{2} \sum_{a \in \{\mathrm{perc}, \mathrm{know}, \mathrm{rat}\}} \mathrm{Presence}_a \cdot \mathrm{Correctness}_a$$

Answer accuracy is reported separately. This design makes OpenMedReason-Bench a diagnostic benchmark rather than only an answer benchmark.

Quality control includes an expert audit by a board-certified internist over 100 randomly sampled OpenMedReason-Bench cases. The reported results are: answer correctness 100/100, reasoning faithful to image/context 100/100, question clinically meaningful 100/100, answerable from given evidence 100/100, and modality label correct 99/100 (Baghbanzadeh et al., 10 Jun 2026). These numbers support the paper’s claim of “high-fidelity” supervision and evaluation.

5. Training protocol and empirical performance

All experiments use Qwen2.5-VL-7B-Instruct as the backbone, with the default Qwen2.5-VL vision encoder frozen in supervised fine-tuning and RL, and LoRA adapters applied to the 7B LLM. The paper introduces no additional medical-specific architectural modules; the reported gains are attributed to data and training rather than model redesign (Baghbanzadeh et al., 10 Jun 2026).

Supervised fine-tuning uses 446,399 examples, including 193,270 OpenMedReason grounded instances, 114,601 PMC-VQA, 33,536 PathVQA, 10,145 SLAKE, 3,548 VQA-RAD, and classification datasets converted into instruction-style prompts. The objective is standard next-token cross-entropy over sequences containing image, question, rationale, and answer. RL then applies GRPO using 5,120 held-out examples and a reward defined only over output format and exact answer correctness, not explicit trace quality.

Across seven benchmarks, the ablation sequence is:

Training setting	ID avg	OOD avg	Overall
Qwen2.5-VL-7B base	60.47%	28.98%	53.10%
SFT on OpenMedReason w/o Open-PMC	66.39%	29.81%	57.65%
SFT on full OpenMedReason	67.40%	32.29%	59.14%
SFT + GRPO on OpenMedReason	69.23%	32.46%	60.04%

The abstract summarizes this as a 20% average improvement in VQA accuracy over the base model, while the paper’s benchmark table shows a 7-point absolute gain in overall average from 53.10% to 60.04%, depending on the normalization (Baghbanzadeh et al., 10 Jun 2026). The same abstract states that the trained model achieves performance within 4.2% of the strongest comparable-scale medical LVLMs.

On OpenMedReason-Bench, the Qwen2.5-VL-7B base model attains TraceScore 37.4 and answer accuracy 47.1%, whereas the OpenMedReason-trained model reaches TraceScore 77.8 and answer accuracy 78.5%. The fine-grained scores are not concentrated in one axis: perception, medical knowledge, and rationale all improve jointly. The model’s reasoning traces are preferred over those of the base model in 86.1% of pairwise comparisons (Baghbanzadeh et al., 10 Jun 2026).

The same training also transfers to classification tasks. On HAM10000, EyePACS, HyperKvasir, BrainTumorMRI, VinDr-CXR, VinDr-Mammo, and BUSI, the reported average accuracy rises from 43.12% for the base Qwen2.5-VL-7B to 65.07% for the OpenMedReason model. The paper characterizes this as a 50.9% relative improvement over the base model (Baghbanzadeh et al., 10 Jun 2026).

6. Limitations, safety, and place in the literature

The authors explicitly warn that OpenMedReason is a research tool rather than a clinical product (Baghbanzadeh et al., 10 Jun 2026). Because it is derived from published biomedical figures, it over-represents rare or complex cases and certain specialties, especially radiology and pathology. Demographic bias present in the source literature may propagate into the dataset. The benchmark remains limited relative to real clinical heterogeneity, and out-of-distribution robustness and regulatory-grade validation remain open problems.

The same source states that accuracy on OpenMedReason or OpenMedReason-Bench is not evidence of clinical safety, and that models trained on this data must not be used for diagnostic, triage, or treatment decisions. This safety framing is consistent with other recent medical reasoning resources that emphasize reasoning quality while stopping short of deployment claims. Med-CMR, for example, shows that long-tail generalization remains a dominant failure mode for medical multimodal reasoning benchmarks (Gong et al., 30 Nov 2025), while MedCaseReasoning and MedR-Bench underscore that answer correctness and reasoning quality diverge even in text-based clinical tasks (Wu et al., 16 May 2025, Qiu et al., 6 Mar 2025).

Within that broader literature, OpenMedReason contributes two linked artifacts: a large, open training corpus with grounded rationales and a benchmark that decomposes multimodal reasoning into perception, medical knowledge, and rationale. This suggests a role not only as a dataset for supervised fine-tuning and reinforcement-based alignment, but also as infrastructure for studying where medical LVLMs fail, whether those failures originate in visual grounding or clinical inference, and how reasoning supervision alters model behavior. The project releases its dataset at huggingface.co/datasets/neginb/OpenMedReason, together with code and a trained checkpoint, making it part of the growing open ecosystem for auditable medical AI research (Baghbanzadeh et al., 10 Jun 2026).