MedVersa: Unified Medical AI Framework

• MedVersa is a generalist medical AI framework that integrates a multimodal foundation model with specialized vision modules for unified clinical imaging tasks.
• It employs an encoder–decoder architecture with 2D and 3D vision encoders and vision-language adapters to achieve state-of-the-art performance in image interpretation and report generation.
• The framework features a rigorous benchmark suite for knowledge editing in medical LLMs, ensuring efficacy, generality, and locality for safe clinical diagnostics.

MedVersa is a generalist medical AI framework encompassing both a multimodal foundation model for versatile medical image interpretation and a rigorous benchmark suite for evaluating knowledge editing in medical LLMs. Developed to address the limitations of specialist AI systems and the rapid evolution of clinical knowledge, MedVersa is designed to unify medical image tasks—ranging from report generation and detection to segmentation and VQA—while also providing benchmarks for reliable knowledge modifications in LLMs without compromising prediction locality. The MedVersa architecture, data construction, benchmark methodology, and evaluation results collectively represent a major advance in unified, scalable, and clinically relevant medical AI systems (Zhou et al., 13 May 2024, Xia et al., 15 Oct 2025, Lim et al., 29 Nov 2025).

1. Model Architecture and Multimodal Capabilities

MedVersa employs an encoder–decoder paradigm orchestrating a LLM with specialist vision modules. The core components are:

• 2D Vision Encoder: Swin Transformer, pretrained on ImageNet, for patch-based feature extraction from radiographs and dermatoscopic images.
• 3D Vision Encoder: Encoder path of a 3D U-Net, suitable for volumetric CT.
• Vision-Language Adapters: Lightweight three-layer stacks (adaptive pooling, LayerNorm, linear projection) that map visual tokens into the LLM's embedding space.
• Tokenizer: SentencePiece (byte-pair encoding) tokenizer aligned with LLaMA-based LLM.
• LLM Backbone: LoRA-finetuned Llama-2-Chat, which fuses visual and text token streams and orchestrates downstream vision modules.

The LLM determines whether to respond directly in text (VQA, captioning, open-ended reporting) or to invoke a vision module for detection (object localization via lightweight CNN head), 2D U-Net segmentation, or 3D U-Net segmentation. The model supports both free-text outputs (e.g., radiology reports, region captions, comparative summaries) and structured outputs (disease tags, boxes, segmentation masks) (Zhou et al., 13 May 2024).

2. Training Regime and Data Sources

MedVersa is pre-trained on MedInterp, a large curated dataset comprising approximately 13 million labeled instances across 11 distinct tasks and three primary imaging modalities:

• Radiographic Reporting: MIMIC-CXR (216K studies), Chest ImaGenome (235K images with boxes and captions).
• Open-ended Question Answering (VQA): Medical-Diff-VQA (383K QA), IUX-ray, NIH ChestX-ray.
• Segmentations: HAM10000 (dermatoscopy), AbdomenCT-1K, CheXmask, manual heart/lung masks.
• Detection and ROI Captioning: Pathology detection and classification labels from ImaGenome, CheXpert, MS-CXR.

Loss functions used include cross-entropy for sequence and classification, a sum of focal and Dice loss for segmentation, and combined classification plus box regression loss for detection. Training uses domain-aware mini-batching, AdamW optimizer, cosine learning rate scheduling, and LoRA for efficient LLM adaptation. Multi-task loss aggregation allows for balanced optimization across heterogeneous outputs (Zhou et al., 13 May 2024).

3. Benchmark Suite: MedVersa for Knowledge Editing

MedVersa functions as an advanced benchmark for evaluating knowledge editing in medical LLMs, with strict controls for efficacy, generality, and locality—crucial properties in a domain where inadvertent side effects or information staleness can have detrimental real-world impact (Xia et al., 15 Oct 2025). Benchmark construction leverages:

• Source Data: MedMCQA (∼194,000 multiple-choice questions, 21 subjects).
• Task Coverage: 20 balanced medical subjects (Anatomy, Microbiology, Surgery, Pediatrics, Psychiatry, Radiology, etc.), with explicit domain diversity to cover knowledge beyond pharmacology-centered benchmarks like MedCF++.
• Edit Scenarios:
  • Single-edit: One fact alteration per session.
  • Batch-edit: Simultaneous edits of 10/50/100 items, reflecting real-world batch knowledge updates (e.g., guideline revisions).
• Instance Structure:
  • Efficacy question–answer pairs $(q_e, o_e)$: Did the edit take effect?
  • Generality paraphrase pairs $(q_g, o_e)$: Does the edit generalize to paraphrased contexts?
  • Locality pairs $(q_l, o_l)$: Are unrelated facts unaffected?

Evaluation follows metrics standard in knowledge editing but tailored for medical constraints:

• Efficacy (Eff): $$\mathbb{E}_{(q_e,o_e)}\;\mathbb{I}\bigl[f'_\theta(q_e)=o_e\bigr]$$
• Generality (Gen): $$\mathbb{E}_{(q_g,o_e)}\;\mathbb{I}\bigl[f'_\theta(q_g)=o_e\bigr]$$
• Locality (Loc): $$\mathbb{E}_{(q_l,o_l)}\;\mathbb{I}\bigl[f'_\theta(q_l)=f_\theta(q_l)=o_l\bigr]$$
• Fluency (Flu): $-\sum_k f_n(k)\,\log_2 f_n(k)$ (entropy over $n$-grams)

Splits ensure no prompt overlap between efficacy and locality checks, and the benchmark rigorously tests the ability of editing methods to scale while avoiding unintended collateral changes (Xia et al., 15 Oct 2025).

4. Empirical Performance and Comparative Evaluation

Extensive quantitative and qualitative comparisons establish MedVersa as a leading unified model and benchmark:

• Generalist Model Results: On nine flagship tasks (BLEU-4, RadGraph, RadCliQ, mean F1, IoU, Dice), MedVersa meets or exceeds state-of-the-art specialist models (e.g., MAIRA-1, YOLOv5, nnSAM, CRCKD) with statistically significant improvements in most modalities (Zhou et al., 13 May 2024).
• Radiology Report Generation: In a blinded radiologist evaluation on emergency department CXR, MedVersa achieves high language clarity (88.4% ≥4/5, exceeding radiologists’ 78.1%) but exhibits higher rates of clinically significant miss (RADPEER 3b: 25.9% vs. radiologist 13.9%) and hallucinations (12.3% vs. 0.1%) (Lim et al., 29 Nov 2025).
• Finding-level Diagnostics: Sensitivities for key findings (e.g., cardiomegaly 69.3%, effusion 48.0%, consolidation/GGO 39.3%, emphysema 20.2%) vary, with strong specificity (≥86.9%) but room for improvement on subtle pathologies (Lim et al., 29 Nov 2025).
• Knowledge Editing Robustness: MedREK, benchmarked using MedVersa, achieves robust efficacy (74.49%), generality (70.46%), and exceptional locality (99.90–99.45%) even under 100 simultaneous edits, surpassing prior editors (RECIPE, MEND, MEMIT, MedLaSA), especially in avoidance of unintended knowledge disruption. Ablation demonstrates that both query–key MLP compression and prompt attention are essential for batch-edit scaling without locality degradation (Xia et al., 15 Oct 2025).

<table> <thead><tr> <th>Task / Metric</th> <th>MedVersa</th> <th>Specialist SOTA</th> </tr></thead> <tbody> <tr><td>Radiology report BLEU-4</td><td\>17.8</td><td>MAIRA-1: 14.2</td></tr> <tr><td>Chest organ segmentation (mean Dice)</td><td\>0.970</td><td>nnSAM: 0.936</td></tr> <tr><td>RADPEER 3b miss (CXR reports)</td><td\>25.9%</td><td>Radiologist: 13.9%</td></tr> <tr><td>Language clarity</td><td\>88.4%</td><td>Radiologist: 78.1%</td></tr> <tr><td>Knowledge edit (Locality)</td><td\>99.90%</td><td>RECIPE: <99%</td></tr> </tbody> </table>

These results indicate that MedVersa closes the specialist–generalist gap in performance for many core medical vision and language tasks while branding itself as a rigorous test bed for editing reliability.

5. Benchmark Construction and Subject Diversity

MedVersa’s benchmark supports 20 distinct medical subjects, explicitly balancing domain representation contrary to prior pharmacology-dominated datasets (e.g., MedCF++ has 12 subjects, 72% pharmacology). Major subjects and proportion are: Anatomy (11.2%), Microbiology (10.2%), Physiology (10.1%), Surgery (10.0%), Social/Preventive (10.0%), Gyn/Obstetrics (8.1%), Ophthalmology (7.9%), Forensic (6.8%), Pediatrics (6.6%), and ENT (6.0%). Each edit instance features non-overlapping prompt content between efficacy and locality evaluations, non-trivial paraphrasing, and both single and large-scale batch split support (Xia et al., 15 Oct 2025).

Key distinctions from prior work include:

• Realistic, open-ended queries (vs. short phrases)
• Explicit paraphrase (Generality) testing
• Reliable batch-edit evaluation (elimination of efficacy–locality prompt reuse)
• Locality measured with same-subject, unrelated QAs (not nearest-neighbor graph sampling)

6. Limitations and Prospective Directions

MedVersa’s limitations derive from both its training and benchmark design:

• Training Distribution Imbalance: Approximately 80% of training data are radiographs (X-ray), with less representation for CT and dermoscopy, implying potential modality bias in current releases. Planned expansion includes MRI, ultrasound, and histopathology for balanced multimodal training (Zhou et al., 13 May 2024).
• Clinical Acceptability Gaps: While language fluency and clarity are high, clinical acceptability and diagnostic miss rates remain inferior to radiologists in some settings, necessitating continued human review and raising questions for full automation (Lim et al., 29 Nov 2025).
• Explainability and System Complexity: The hierarchical combination of vision, text, and adapter modules produces rich but sometimes opaque decision paths challenging for clinical explainability requirements.
• Benchmark Scope: MedVersa currently does not include genome, EHR, or laboratory data, and global demographic coverage is incomplete.
• Scalability: While MedVersa supports batch editing up to 100 simultaneous edits, scaling to even larger knowledge update scenarios, finer domain granularity, and greater context size will require additional research.

Planned developments involve fully integrating module-level clinical rationales, broadening task and modality coverage, extending knowledge editing orchestration to plug-in modules, and performing prospective clinical trials focusing on workflow acceleration, diagnostic accuracy, and patient safety outcomes (Zhou et al., 13 May 2024, Xia et al., 15 Oct 2025).

7. Clinical and Research Implications

MedVersa's unified model and benchmark design establish a foundation for comprehensive, scalable medical AI. For research, it enables:

• Direct apples-to-apples comparison across diverse modalities, tasks, and subject domains within a single framework.
• Rigorous, locality-aware evaluation of knowledge editing protocols—a key advance for safe, real-world medical LLM updates.
• Empirical evidence that generalist systems can equal or exceed specialist performance for both vision–language and structured tasks, with substantial workflow acceleration potential (20–30% reduction in reporting time for radiologists in simulated settings).

A plausible implication is the future deployment of MedVersa-like systems as clinically integrated, cross-specialty front-ends—providing draft reports, quantitative outputs, and robust, updatable medical knowledge, with continual adaptation as new guidelines and evidence emerge. The MedVersa benchmark will likely remain a reference standard for evaluating the safety and precision of such model-edited systems (Zhou et al., 13 May 2024, Xia et al., 15 Oct 2025, Lim et al., 29 Nov 2025).