MedGen: AI in Medical Data & Multimedia

• MedGen is a research initiative integrating artificial intelligence with diverse medical data types—including video, text, and genomics—to enable advanced clinical analytics.
• It employs methodologies such as latent video diffusion, multimodal benchmarking, and contrastive pretraining to achieve superior performance in tasks like tumor segmentation and medical language processing.
• The MedGen ecosystem features open-source resources like MedVideoCap-55K and the MedGen toolkit, fostering innovations in clinical informatics and research applications.

MedGen refers to a distinct set of research efforts, datasets, models, and toolkits at the intersection of artificial intelligence and medical data, encompassing medical video generation, multimodal vision-language evaluation, medical text generation and retrieval, and multimodal genomic–imaging contrastive learning. The following entry systematically covers the key developments under the MedGen name in the current literature, focusing on open-source medical video generation, evaluation frameworks for clinical synthetic media, biomedical natural language generation, and multimodal omics–imaging pre-training.

1. Medical Video Generation: MedGen Model and MedVideoCap-55K

MedGen, in the context of medical video generation, is a fine-tuned latent video diffusion model (LVDM) optimized for medical realism and accuracy, utilizing the MedVideoCap-55K dataset—the first large-scale, caption-rich repository of medical videos (Wang et al., 8 Jul 2025). MedVideoCap-55K comprises over 55,000 curated medical clips derived from a five-stage pipeline including medical keyword and channel filtering, temporal segmentation (using CLIP-based classifiers and cosine-similarity for coherence), granular captioning (GPT-4o-generated), and multi-factor quality filtering (technical and aesthetic metrics, black border and subtitle exclusion). Final clips are 6–10 s (mode ≈8 s), fixed at 720×480 resolution, spanning domains such as clinical procedures, teaching, imaging, and simulation.

The MedGen architecture adopts the HunyuanVideo backbone, comprising a video latent encoder/decoder, a CLIP-based text encoder for prompt embeddings, and a spatial-temporal Transformer U-Net for diffusion-based generation. No new “medical” modules are introduced; instead, the core model is adapted via Low-Rank Adaptation (LoRA), injecting learnable adapters into all cross-attention QKV projections for efficient domain transfer. The objective is the standard L₂ diffusion denoising loss.

MedGen achieves a leading total score (70.93) on Med-VBench—outperforming all prior open-source video generators and rivaling commercial solutions such as Pika, Sora, and Kling. Subscores include Imaging Quality 72.50, Subject Consistency 94.41, Background Consistency 97.39, Motion Smoothness 99.76, and Warping Error 38.50. Human evaluation by clinical experts demonstrates superiority in medical accuracy, text alignment, and visual quality over baselines. Transfer learning experiments further show that models fine-tuned on MedVideoCap-55K improve downstream tasks (e.g., medical video classification), and synthetic data augmentation increases F₁ scores by +2.26 to +15.30.

Ablation shows a trade-off between domain specialization (e.g., clinical practice, imaging) and generalization (e.g., animation), and LoRA fine-tuning offers computational advantages with no loss in convergence. The main failure modes in open-source videos include anatomical hallucinations and tool misplacement, which MedGen mitigates except for minor backgrounds artifacts. Key future directions include explicit medical-consistency losses, integrated video+LLM for interactive simulation, and extension to higher resolutions and finer-grained annotations.

2. MedGEN-Bench: Multimodal Benchmarking for Medical Generation

MedGEN-Bench is a multimodal open-ended benchmark explicitly designed to address the gap between real-world clinical workflows and the limitations of existing medical vision-language benchmarks (Yang et al., 17 Nov 2025). The benchmark includes 6,422 expert-validated image–text pairs from over 11,700 raw images covering CT, MRI, ultrasound, X-ray, digital pathology slides, and clinical photographs.

Tasks are tightly entangled—requiring that instructions combine patient metadata, visual cues, and the clinical task—across three formats:

1. Visual Question Answering (VQA): Free-form text answers to image-centric, context-rich questions.
2. Image Editing: Image modification based on instructions (e.g., outlining lesions).
3. Contextual Multimodal Generation: Simultaneous image editing/generation and explanatory text.

MedGEN-Bench introduces a three-tier evaluation framework:

• Pixel-Level Metrics: PSNR, SSIM, LPIPS for image editing/generation fidelity.
• Semantic Text Analysis: BERTScore (with PubMedBERT).
• Holistic Clinical Relevance: “VLM-as-a-Judge” paradigm, where a multimodal LLM rates outputs (1–10) across coherence, alignment, accuracy, instruction relevance, and modality consistency, both with and without ground truth.

Systematic evaluation reveals that compositional frameworks (vision + generation pipelines) outperform unified models in cross-modal consistency and holistic metrics despite some unified models scoring higher in pixel-level fidelity. Unified models may exhibit “cross-modal disconnection,” scoring well on PSNR/LPIPS but failing in clinical relevance (over 60% drop in holistic judge scores). Specialized medical VLMs perform well on simple VQA but struggle in open-ended tasks. Augmenting contextual queries yields a 36.3% increase in instruction–image semantic similarity. MedGEN-Bench thus sets a new standard, demanding genuine multimodal reasoning beyond text-only or closed-form tasks.

3. Biomedical Text Generation and Processing: MedGen (Ascle) Toolkit

The MedGen (formerly Ascle) toolkit is an open-source NLP suite for medical text generation, integrating generative functions, essential NLP utilities, and structured/unstructured query interfaces for clinical data (Yang et al., 2023).

Its architecture comprises three main modules:

• Generative Functions: Multiple-choice and free-form QA (leveraging models like BioBERT, ClinicalBERT, SapBERT, GatorTron-base, PubMedBERT), abstractive summarization (Pegasus, BART, BioBART, SciFive), text simplification (BigBirdPegasus, BART, BioBART), and domain-specialized machine translation (MarianMT, mT5).
• Basic NLP Functions: Abbreviation extraction, tokenization, negation/hyponym detection, UMLS concept/entity linking, NER, POS tagging, extractive summarization, and document clustering, aggregating resources such as scispaCy and MedspaCy.
• Query & Search: Structured MySQL interfaces for MIMIC-III tables, record-level queries, and WHOOSH/SQLite-powered note keyword search.

Performance evaluation draws on standard metrics (accuracy, ROUGE, BLEU, FKGL), and covers multiple QA, summarization, machine translation, and basic NLP tasks with competitive results, e.g., multiple-choice MedMCQA accuracy up to 64.93% and ROUGE-1 up to 48.35 on PubMed summarization. Human evaluation focuses on readability, relevancy, accuracy, and completeness for answer generation. The toolkit aims to support both research and clinical implementations, although limitations such as automatic metric–clinician preference mismatch and generalization beyond training domains remain open challenges.

4. Multimodal Contrastive Pretraining: MGI for Genomics and Imaging

MGI (Multimodal Genomic and Imaging) represents a contrastive pre-training pipeline for the joint encoding of medical images and gene-expression data (Zhou et al., 2024). Each patient sample consists of a FLAIR-MRI image and a 60,484-dimensional gene expression vector (from TCGA-LGG).

Architecture details:

• Encoders: ViT (for images) and Mamba (for genomics), with dimensionality reduction by mixed pooling and linear projection.
• Pretraining: Cross-modal symmetric InfoNCE contrastive loss aligns image and gene embeddings in a joint feature space by maximizing same-patient similarity and minimizing cross-patient similarity.
• Fine-tuning: Lightweight multimodal mask decoder with multiple cross-attention flows enables fusion for tumor segmentation.
• Results: MGI achieves a Dice score of 0.901 on low-grade glioma tumor segmentation (vs. 0.735–0.840 for strong unimodal baselines), with gains attributed to integration of genomic context into imaging representations and multimodal fusion at decoding.

Although the architecture omits explicit ablations, the design demonstrates that scalable self-supervised contrastive learning can robustly pre-align multimodal data and materially benefit clinical segmentation.

5. Emerging Trends, Limitations, and Future Directions

Across the MedGen family of resources, several shared themes are notable:

• The shift from unimodal, closed-ended medical AI evaluation to open-ended, contextually entangled, and multimodal pipelines (text, image, and video).
• The critical importance of large-scale, granular, and domain-specific datasets (e.g., MedVideoCap-55K, MedQUAD, TCGA) in driving both model fidelity and clinical relevance—model scale alone is insufficient.
• The emergence of efficient fine-tuning strategies (e.g., LoRA) that deliver competitive domain adaptation without prohibitive computational cost.
• The demonstrated benefit of rich, caption-level annotation and the replacement of rigid, category-based annotation with free-text, medical-contextualized NL descriptions.
• The necessity for evaluation frameworks that integrate pixelwise, semantic, and expert/LLM-guided clinical relevance metrics, moving beyond superficial or n-gram-based automatic scores.

Limitations remain, including potential model generalization failures out-of-distribution, the inherent gap between automatic metrics and clinician-evaluated quality, and the absence of explicit mechanisms for preventing medical hallucinations or enforcing factual correctness. Plausibly, future research will further explore model–human/clinician-in-the-loop pipelines, evidence-based and retrieval-augmented reasoning, and integrated video/LLM agent simulation.

6. Representative Publications and Open Resources

Name/Resource	Domain	arXiv & Link
MedGen (LVDM, MedVideoCap-55K)	Medical video generation	(Wang et al., 8 Jul 2025)
MedGEN-Bench	Multimodal benchmark	(Yang et al., 17 Nov 2025)
MedGen (Ascle Toolkit)	Medical text generation	(Yang et al., 2023)
MGI	Genomics–imaging pretrain	(Zhou et al., 2024)
Code & datasets (video/text toolkit)	Open-source software/data	See respective URLs

All listed resources are open source (e.g., MedVideoCap-55K dataset and MedGen weights, the MedGen NLP toolkit) and target both academic research and real-world clinical informatics applications.