BraTS-METS 2025 Lighthouse Challenge

• BraTS-METS 2025 Lighthouse Challenge is a comprehensive initiative that advances AI algorithms for brain metastasis segmentation in multiparametric MRI.
• It introduces a rigorous multi-stage annotation process combining AI pre-segmentation with manual corrections to quantify variability and enhance clinical precision.
• The challenge also serves as an educational platform offering lectures, workshops, and mentorship to prepare the next generation in AI-assisted neuroradiology.

The BraTS-METS 2025 Lighthouse Challenge is a landmark initiative in the domain of medical image analysis, specifically targeting the automated segmentation of brain metastases and other intracranial tumors in multiparametric MRI. This event designs both a rigorous technical benchmarking infrastructure and an advanced educational platform, simultaneously addressing reference data curation, interobserver variability, and workforce training in artificial intelligence–assisted neuroradiology. The challenge synthesizes contributions from leading neuroradiologists, clinical researchers, medical trainees, and AI developers, advancing state-of-the-art methodologies for image segmentation and establishing clinical standards for volumetric lesion assessment and downstream applications in diagnosis, treatment planning, and longitudinal response evaluation (Maleki et al., 16 Apr 2025, Amiruddin et al., 21 Sep 2025).

1. Challenge Structure and Objectives

The principal objective of the BraTS-METS 2025 Lighthouse Challenge is to advance automated, robust, and clinically meaningful segmentation algorithms for both pre-treatment and post-treatment brain metastases and related tumor types on MRI. The challenge introduces systematic assessments of inter-rater and intra-rater variability via multi-instance reference standard creation. This is achieved by generating four independent segmentations per case (two performed fully manual, two using AI pre-segmentation, all by senior neuroradiologists with strict washout intervals) for a key subset of the test data.

Compared to prior years (BraTS-METS 2023/2024, which addressed only pre-treatment metastases), the 2025 edition expands to include post-treatment imaging and longitudinal follow-up, dramatically enhancing relevance for real-world clinical deployment where lesion appearance evolves due to interventions such as surgery and radiotherapy. The annotated datasets, alongside the multi-institutional data sources, are publicly released following challenge conclusion, serving as future reference standards (Maleki et al., 16 Apr 2025).

2. Data Collection, Annotation, and Variability Measurement

The core dataset for the challenge comprises 1,778 cases (1,046 pre-treatment, 732 post-treatment) sourced from North American, European, and African institutions, including Duke University, Washington University, UCSF, UCSD, and the National Cancer Institute (Egypt). Mandatory imaging modalities are pre-contrast T1, post-contrast T1-weighted (T1CE), and T2-FLAIR; T2-weighted is optional for 2025.

Annotations are produced via a standardized five-step pipeline:

1. AI pre-segmentation using three nnU-Net models (trained on diverse institutional data) with model output fusion (minority vote and STAPLE).
2. Manual correction by medical students and board-certified neuroradiologists.
3. Stringent quality control.
4. Second neuroradiologist review.
5. Senior neuroradiologist approval.

The multi-annotator Lighthouse subset consists of 75 subjects, each annotated four times by two neuroradiologists (two from scratch, two as refinements of AI outputs), with a seven-day recall-minimizing interval between each round, enabling robust quantification of both inter- and intra-observer variability (Maleki et al., 16 Apr 2025).

Variability is reported via Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and Intraclass Correlation Coefficient (ICC):

• $DSC(X,Y) = \frac{2|X \cap Y|}{|X| + |Y|}$
• $$HD(X,Y) = \max \{\sup_{x \in X} \inf_{y \in Y} d(x, y), \sup_{y \in Y} \inf_{x \in X} d(x, y)\}$$
• $ICC$ in both two-way random effects and mean square models.

This process sets a new benchmark for rigorous, reproducible ground truth in the brain metastasis segmentation domain (Maleki et al., 16 Apr 2025).

3. Technical Challenges and Algorithmic Contributions

Key technical challenges addressed in the 2025 challenge include:

• Domain shift and multi-center heterogeneity: Variability in scanner protocols, contrast, and noise is explicitly tackled via extensive pre-processing (denoising, intensity normalization, isotropic resampling), domain-adaptive training strategies, and fine-tuning (Jaheen et al., 31 Jul 2025, Sandbhor et al., 19 Sep 2025).
• Missing modality synthesis: The challenge introduces a dedicated missing-modality task, wherein one of the four standard MR image sequences is held out at test time for each sample, requiring participants to synthesize the missing contrast and enable downstream segmentation. SLaM-DiMM (Shared Latent Modeling for Diffusion-based Missing-Modality synthesis) exemplifies solutions combining 2D slice-wise latent diffusion, composite SSIM- and lesion-focused loss, and a 3D coherence enhancement module (Sandbhor et al., 19 Sep 2025).
• Resource-limited data and low-quality scans (SSA context): Dedicated segments address segmentation robustness in sub-Saharan Africa, with EMedNeXt introducing region-of-interest expansion, boundary-aware hybrid loss, robust data augmentation, and systematic model ensembling/post-processing, achieving lesion-wise DSC of 0.897 and NSD@1mm of 0.84 (Jaheen et al., 31 Jul 2025).
• Synthetic data augmentation for generalization: On-the-fly insertion of synthetic tumors using pretrained GliGAN, with custom augmentation policies, enables efficient expansion of the training distribution without prohibitive storage costs, improving sensitivity for underrepresented morphologies and enhancing model performance against clinical variability. Ensemble Dice can reach 0.818 across tumor subregions (Jain et al., 29 Sep 2025).

4. Benchmarking, Evaluation Metrics, and Reporting

Solution benchmarking is performed using standardized BraTS metrics:

• Lesion-wise and voxel-wise Dice
• Normalized Surface Dice (NSD) at specified tolerance ($\tau$ mm)
• Hausdorff Distance (HD) metrics, including HD95
• Detection sensitivity, specificity, and precision (PPV)

The evaluation protocol follows a per-metric ranking scheme, with aggregate rankings and metric-specific statistical significance tested in accordance with DELPHI guidelines. For missing modality synthesis, metrics include SSIM and lesion-focused Dice on downstream segmentations.

All submitted algorithms are containerized using the GaNDLF framework (via MLCube), and no fixed architecture is mandated. Top-performing pipelines demonstrate heterogeneity in architectural strategies, including radiomic-guided subtyping, lesion-wise ensemble weighting, adaptive post-processing for label correction and false-positive reduction, and radiomic cluster-driven stratification of both cross-validation folds and post-processing parameters (Capellán-Martín et al., 16 Dec 2025).

5. Educational Platform, Workforce Training, and Broader Impact

The Lighthouse Challenge uniquely blends technical benchmarking with a multimodal educational program for medical trainees. The organizational structure includes:

• Didactic lectures on MRI, neuroanatomy, tumor pathology, and AI principles, with pre- and post-session knowledge surveys quantifying knowledge gain (mean familiarity with segmentation software increased from 6.0 ± 2.9 to 8.9 ± 1.1; brain tumor MRI feature familiarity increased from 6.2 ± 2.4 to 8.1 ± 1.2) (Amiruddin et al., 21 Sep 2025).
• Hands-on workshops led by data scientists focusing on segmentation pipeline construction, optimization, and best practices (e.g., nnU-Net and MLCube).
• One-on-one annotation mentorship, pairing selected trainees (“annotation coordinators”) with expert neuroradiologists for four independent rounds (manual and AI-aided), averaging 1,322.9 ± 760.7 hours per pair.
• Journal clubs addressing AI sustainability, cybersecurity, and the global landscape.
• Social media microlearning (LinkedIn, X, Instagram, BlueSky, WhatsApp) disseminates bite-sized educational content.

The combination of direct mentorship, didactic content, practical annotation experience, and broad digital engagement constitutes a reproducible blueprint for integrating clinical-quality reference curation with future-proofed AI education (Amiruddin et al., 21 Sep 2025).

6. Outcomes, Limitations, and Future Directions

The BraTS-METS 2025 Lighthouse Challenge robustly quantifies annotation variability, delivers high-integrity ground truth data for algorithm development and validation, and establishes a scalable infrastructure for interdisciplinary education:

• Reference data: Multi-rater, multi-institutional annotated datasets released publicly for the community, enabling transparent, repeated benchmarking (Maleki et al., 16 Apr 2025).
• Algorithmic validation: State-of-the-art pipelines achieve mean lesion-wise Dice $>0.8$ for major subregions, with tailored improvements for domain-shifted cohorts and cases with missing modalities (Jaheen et al., 31 Jul 2025, Jain et al., 29 Sep 2025, Capellán-Martín et al., 16 Dec 2025).
• Education: Measurable proficiency gains in AI and image analysis among trainees, enhanced mentorship networks, and two algorithm submissions emerging from trainee cohorts (Amiruddin et al., 21 Sep 2025).
• Clinical translation: Broader adoption of volumetric lesion assessment, more granular response evaluation (superseding RANO-BM diameter metrics), and open benchmarks for future radiomics or predictive modeling studies.

Areas for further development include dynamic incorporation of functional imaging, richer outcome metadata, refined learned post-processing modules, and formal integration of AI training within medical school and residency curricula (Maleki et al., 16 Apr 2025, Capellán-Martín et al., 16 Dec 2025, Amiruddin et al., 21 Sep 2025).

---

References:

(Maleki et al., 16 Apr 2025, Jaheen et al., 31 Jul 2025, Sandbhor et al., 19 Sep 2025, Amiruddin et al., 21 Sep 2025, Jain et al., 29 Sep 2025, Capellán-Martín et al., 16 Dec 2025)