Anatomical Similarity as a New Metric to Evaluate Brain Generative Models
The paper "Anatomical Similarity as a New Metric to Evaluate Brain Generative Models" introduces a novel framework for assessing the anatomical fidelity of synthetic brain MRIs generated by contemporary generative models. The predominant existing evaluation methods focus primarily on texture and perception-based metrics, such as Fréchet Inception Distance (FID), Multi-Scale Structural Similarity Index (MS-SSIM), and Maximum Mean Discrepancy (MMD). However, these approaches lack sensitivity towards anatomical integrity, an essential aspect for advancing the clinical utility of synthetically generated imaging data.
Proposed Metric: WASABI
Recognizing the limitations of conventional evaluations, the authors propose a new metric titled Wasserstein-Based Anatomical Brain Index (WASABI). This innovative approach leverages the SynthSeg tool, a deep-learning-based brain parcellation method, to derive volumetric measurements of brain regions, creating a framework that focuses on the authenticity of anatomical structures. WASABI uses the multivariate Wasserstein distance to quantify discrepancies between real and synthetic anatomical distributions, thus providing a more refined index that shifts evaluation paradigms towards anatomical realism.
Experimental Validation
The evaluation comprises two main experimental scenarios: controlled experiments involving real MRI datasets and experiments scrutinizing generative models' synthetic MRIs. For the real data experiments, anatomical distance assessments were conducted using well-characterized MRI datasets from the Alzheimer's Disease Neuroimaging Initiative (ADNI), encompassing subsets separated by sex differences, cognitive impairment levels, and random partitions. WASABI demonstrated superior sensitivity in detecting and ranking anatomical differences across the datasets accurately compared to traditional metrics.
Subsequent experiments involved five state-of-the-art generative models which produce synthetic MRIs of "near-perfect" visual quality. WASABI consistently revealed that real datasets had inherently higher anatomical fidelity compared to synthetic outputs, unlike other metrics that were swayed by image visual characteristics and/or distributional disparities not relevant to anatomical structures.
Practical and Theoretical Implications
The introduction of WASABI has significant implications for the development and validation of brain imaging generative models. By emphasizing anatomical fidelity, WASABI guides the refinement of model architectures towards producing clinically viable imaging that reflects genuine anatomical variances. This paradigm shift fosters improvements in neuroimaging applications, including disease progression analysis, educational simulations, and augmentation for rare conditioning studies.
Future Perspectives
The paper's contributions pave the way for further research focusing on embedding anatomical accuracy metrics more deeply into generative model validation frameworks. Future work might consider expanding WASABI's applications to broader neuroimaging modalities or integrating it with multimodal assessments for enhanced accuracy. Additionally, investigating methods to optimize generative models based on WASABI feedback could yield significant advancements in the quality and utility of synthetic medical imaging.
In conclusion, the proposal of WASABI offers a robust alternative for evaluating brain generative models, reorienting focus towards anatomical similarity over traditional image-level metrics. By fostering detailed anatomical alignment in synthetic MRIs, this work takes meaningful strides in bridging the gap between visual realism and clinical applicability in neuroimaging technologies.