ImageCLEFmed MEDVQA 2025 Challenge

• ImageCLEFmed MEDVQA 2025 is a GI endoscopy VQA benchmark that uses generative multimodal models to produce clinically relevant short answers.
• It integrates extensive datasets like Kvasir-VQA and Kvasir-VQA-x1, supporting stratified reasoning and robustness evaluation through diverse question complexities.
• The challenge emphasizes tailored model fine-tuning, with architectures such as Florence-2, while incorporating explainability to ensure medically grounded interpretations.

Searching arXiv for papers on the ImageCLEFmed MEDVQA 2025 Challenge and closely related GI MedVQA benchmarks. The ImageCLEFmed MEDVQA 2025 Challenge, as documented by 2025 participant papers, centered on visual question answering for gastrointestinal endoscopy, with Subtask 1 framed as answering clinically relevant natural-language questions from GI images using multimodal models that generate short textual answers (Gaihre et al., 19 Jul 2025, Parajuli, 25 Jul 2025). In parallel, the 2025 GI MedVQA landscape also included the related Medico 2025 challenge, which described a GI-imaging VQA task with an explicit Explainable Artificial Intelligence component, two subtasks, and Kvasir-VQA-x1 as benchmark data (Gautam et al., 14 Aug 2025). This suggests that 2025 GI MedVQA was shaped by closely aligned evaluation efforts around endoscopy, generative answering, robustness, and explainability rather than by a single narrow task definition.

1. Scope, identity, and 2025 framing

Participant papers identify the 2025 ImageCLEFmed MEDVQA challenge setting as a GI endoscopy VQA problem in which the model receives an endoscopic image and a natural-language question and returns a textual answer (Gaihre et al., 19 Jul 2025, Parajuli, 25 Jul 2025). The implemented systems in these papers are not fixed-vocabulary classifiers in the narrow 2019 VQA-Med sense; they are generative sequence-to-sequence systems, typically based on Florence-2, although the answer distribution remains short-form and highly structured in practice (Gaihre et al., 19 Jul 2025, Parajuli, 25 Jul 2025).

A related 2025 challenge description, "Medico 2025: Visual Question Answering for Gastrointestinal Imaging" (Gautam et al., 14 Aug 2025), defines a broader GI VQA/XAI task that focuses on developing Explainable Artificial Intelligence models for clinically relevant questions over GI endoscopy images while providing interpretable justifications aligned with medical reasoning. That challenge introduces two subtasks: answering diverse visual questions with the Kvasir-VQA-x1 dataset, and generating multimodal explanations to support clinical decision-making (Gautam et al., 14 Aug 2025). A plausible implication is that explainability was not peripheral in 2025 GI MedVQA, but part of the benchmark discourse around trustworthy medical vision-language systems.

2. Dataset foundation: Kvasir-VQA and Kvasir-VQA-x1

The data foundation of the 2025 GI challenge literature is Kvasir-VQA and its expanded successor Kvasir-VQA-x1. One 2025 submission describes Kvasir-VQA as comprising approximately 58,849 image-question-answer triplets associated with 6,500 high-resolution GI endoscopy images derived from HyperKvasir (Gaihre et al., 19 Jul 2025). Another submission describes Kvasir-VQA as a combination of HyperKvasir and KvasirInstrument with approximately 58,800 question-answer pairs over 6,500 images, partitioned into Normal (2,500), Polyps (1,000), Esophagitis (1,000), Ulcerative Colitis (1,000), and Instrument (1,000) (Parajuli, 25 Jul 2025). Kvasir-VQA-x1 expands this base to 159,549 question-answer pairs over the same 6,500 original GI images and was explicitly designed for deeper clinical reasoning and robustness testing (Gautam et al., 11 Jun 2025).

Resource	Reported scale	Reported role
Kvasir-VQA	6,500 images; approximately 58,849 or 58,800 QA pairs	Core GI endoscopy VQA resource
Kvasir-VQA-x1	6,500 images; 159,549 QA pairs	Reasoning- and robustness-oriented extension

Kvasir-VQA-x1 is structurally richer than the earlier dataset. It uses 18 clinical question classes, including abnormality color, abnormality location, abnormality presence, box artifact presence, finding count, instrument count, instrument location, landmark presence, polyp count, polyp removal status, polyp size, polyp type, procedure type, and text presence (Gautam et al., 11 Jun 2025). It also stratifies question-answer instances by reasoning complexity into Level 1 with 54,856 samples, Level 2 with 52,349 samples, and Level 3 with 52,344 samples (Gautam et al., 11 Jun 2025). The Medico 2025 challenge abstract independently confirms Kvasir-VQA-x1 as a benchmark built from 6,500 images and 159,549 complex question-answer pairs (Gautam et al., 14 Aug 2025).

3. Question structure, answer space, and problem formulation

The 2025 GI challenge setting is hybrid in an important sense: models are trained and evaluated as generative systems, yet the answer space is strongly constrained. One participant analysis reports 20 unique question templates and 502 unique answers in Kvasir-VQA, with the answer distribution dominated by short responses such as “none,” “no,” “yes,” and “0,” and with most answers being single-word (Gaihre et al., 19 Jul 2025). Another submission organizes the task into six practical answer types: Yes/No, Single choice, Multiple choice, Choice (Color), Location, and Numerical counting (Parajuli, 25 Jul 2025).

This structure corrects a common misconception that the challenge is simply open-ended medical dialogue. In practice, it is a generative VQA task over a highly imbalanced short-answer distribution (Gaihre et al., 19 Jul 2025, Parajuli, 25 Jul 2025). That distribution helps explain why lexical metrics can behave unevenly: semantically adequate answers may still receive weak BLEU if they differ in short n-gram form, while ROUGE and METEOR can remain high in the same setting (Parajuli, 25 Jul 2025).

Kvasir-VQA-x1 formalizes an additional shift from atomic to compositional medical questioning. Level 1 corresponds to one atomic QA pair, Level 2 merges two atomic QA pairs, and Level 3 merges three atomic QA pairs into a single coherent clinical question (Gautam et al., 11 Jun 2025). The benchmark therefore does not merely enlarge the dataset; it changes the task from direct factual recall toward moderate and higher-order synthesis. The paper also notes that some Level 2 questions can be easier than Level 1 because merged prompts may reduce ambiguity, which suggests that compositionality and effective difficulty are related but not identical (Gautam et al., 11 Jun 2025).

4. Evaluation practice and representative 2025 systems

The participant literature reports evaluation with standard text-generation metrics—BLEU, ROUGE-1, ROUGE-2, ROUGE-L, and METEOR—on validation, public, and private sets (Gaihre et al., 19 Jul 2025, Parajuli, 25 Jul 2025). The provided 2025 sources report these metrics extensively, but they do not identify a single official ranking metric for Subtask 1. This leaves the published picture closer to a challenge-oriented benchmark literature than to a fully standardized official overview.

The dominant reported architecture family is Florence-2. One submission fine-tunes microsoft/Florence-2-base-ft as a generative multimodal encoder-decoder, freezes the ViT-L/14 vision tower, and trains on a 1% stratified subset for computational efficiency (Gaihre et al., 19 Jul 2025). Another submission adapts Florence-2 with LoRA, comparing frozen-tower baselines with LoRA ranks 8 and 16, and reports the best overall trade-off for the rank-16, alpha-32 setting (Parajuli, 25 Jul 2025). Both papers emphasize that the model consumes an image-question pair and decodes the answer autoregressively (Gaihre et al., 19 Jul 2025, Parajuli, 25 Jul 2025).

Submission paper	Reported system	Reported public / private results
(Gaihre et al., 19 Jul 2025)	Florence-2 fine-tuning	Public: BLEU 0.150, ROUGE-L 0.800, METEOR 0.440; Private: BLEU 0.160, ROUGE-L 0.880, METEOR 0.490
(Parajuli, 25 Jul 2025)	Florence-2 with LoRA	Public: BLEU 0.21, ROUGE-L 0.86, METEOR 0.48; Private: BLEU 0.18, ROUGE-L 0.90, METEOR 0.50

The second paper provides a detailed question-type breakdown showing that BLEU is often 0.0 for Yes/No, Choice, Location, and Numerical Count categories despite strong ROUGE-L and METEOR, while “Choice (Color)” achieves BLEU 0.47, ROUGE-2 0.32, and METEOR 0.51 (Parajuli, 25 Jul 2025). This illustrates a broader evaluation issue in generative MedVQA: exact n-gram overlap can under-credit short but clinically aligned responses. A plausible implication is that metric choice materially shapes the apparent difficulty of the challenge.

5. Robustness, explainability, and related 2025 extensions

Kvasir-VQA-x1 defines two evaluation tracks: a standard VQA setting on original images and a robustness setting on weakly transformed images (Gautam et al., 11 Jun 2025). The implemented perturbations are RandomResizedCrop with scale 0.9–1.0, RandomRotation with $\pm 10$ degrees, RandomAffine with translation up to 10%, and ColorJitter with brightness and contrast 0.8–1.2, with 10 weakly augmented versions per original image (Gautam et al., 11 Jun 2025). This embeds robustness directly into the benchmark rather than leaving it as an auxiliary ablation.

Challenge submission evidence supports the importance of medically plausible augmentation. In a Florence-2 study, “heavy augmentation” underperformed even “no augmentation,” whereas “standard augmentation” and “fine-tuned augmentation” produced better validation performance; the reported ROUGE-1 values were 0.48 for heavy augmentation, 0.63 for no augmentation, 0.78 for standard augmentation, and 0.81 for fine-tuned augmentation (Gaihre et al., 19 Jul 2025). This suggests that perturbation policy is not a generic regularization choice but a modality-specific design variable constrained by endoscopic realism.

Explainability enters the 2025 GI challenge discourse explicitly through Medico 2025, which defines a task aimed at “Explainable Artificial Intelligence (XAI) models that answer clinically relevant questions based on GI endoscopy images while providing interpretable justifications aligned with medical reasoning” and combines quantitative performance metrics with expert-reviewed explainability assessments (Gautam et al., 14 Aug 2025). A related methodological line is rationale-supervised MedVQA. "MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale" formalizes answer-and-rationale generation as $\{A, R\} = f(I, T)$, evaluates closed-end questions by accuracy and open-end questions by BLEU and ROUGE, and argues that rationale generation can make the medical decision-making process more transparent, while not claiming formally verified causal faithfulness (Gai et al., 2024). In the 2025 challenge context, this positions explainability as an active research direction rather than an optional add-on.

6. Historical development and significance

The 2025 GI challenge marks a substantial shift from earlier ImageCLEF medical VQA formulations. In ImageCLEF 2019 VQA-Med, the benchmark was organized into four radiology-oriented question categories—modality, plane, organ/anatomy, and abnormality—with a small and strongly imbalanced training set; IBM’s Supporting Facts Network addressed this by reusing predictions of modality, plane, and organ as “supporting facts” for harder downstream abnormality questions (Kornuta et al., 2019). The 2025 GI setting differs in modality, answer format, and evaluation emphasis: it is endoscopy-specific, generative in implementation, and centered on short but clinically structured answers rather than primarily on closed categorical prediction.

The immediate GI predecessor is ImageCLEFmed-MEDVQA-GI 2023. That benchmark used gastroscopy and colonoscopy images, attached 18 predefined questions to each image, and was approached successfully with multi-label classification over a fixed answer vocabulary rather than sequence generation (Thai et al., 2023). In that setting, a BERT+BEiT fusion model with image enhancement achieved 87.25% accuracy and 91.85% F1-score on the development test set and 82.01% accuracy on the private test set (Thai et al., 2023). Relative to 2023, the 2025 challenge literature shows a clear move from template-driven multi-label classification toward foundation-model fine-tuning, compositional question construction, robustness evaluation, and explainability-oriented task design.

The broader MedVQA benchmark literature reinforces this interpretation. BESTMVQA evaluated MedVQA-2019 together with VQA-RAD, SLAKE-EN, PathVQA, and OVQA under a unified setup, and found PTUnifier, METER, and TCL stronger than older lightweight baselines on several datasets (Hong et al., 2023). Kvasir-VQA-x1 and the 2025 GI submissions extend that benchmark trajectory into a domain-specific, challenge-style setting in which clinically plausible augmentation, reasoning stratification, and answer generation are central. The resulting significance of ImageCLEFmed MEDVQA 2025 lies less in a single architecture than in the convergence of four ideas: GI endoscopy as a dedicated MedVQA domain, foundation-model adaptation, explicitly stratified reasoning difficulty, and evaluation of trust-related properties such as robustness and explainability (Gautam et al., 11 Jun 2025, Gautam et al., 14 Aug 2025).