Multimodal MRI Quality Assessment

• MMRQA is a comprehensive framework that automates quality evaluation of heterogeneous MRI acquisitions without relying on reference images.
• It integrates advanced signal metrics, uncertainty modeling, and calibrated human labels to provide tailored, task-specific assessments across varying protocols.
• Recent approaches fuse quantitative analysis with multimodal language models to generate clinically interpretable outputs and adaptive quality control.

Searching arXiv for recent and foundational work on multimodal MRI quality assessment to ground the article in the current literature. arXiv search query: "MRI quality assessment multimodal MRQy uncertainty task-specific task-agnostic motion artifacts" Multimodal MRI quality assessment (MMRQA) addresses the automated evaluation of image quality across heterogeneous magnetic resonance acquisitions, including differences in contrast, anatomy, scanner, site, artifact type, and downstream analytic task. Magnetic resonance imaging offers a wide variety of imaging techniques, a large amount of data is created per examination, and any imaging artifacts originating from scanner hardware, signal processing or induced by the patient may reduce image quality; usually no reference image is available or difficult to define, so classical reference-based approaches are not applicable (Küstner et al., 2018). More recent work places the same problem in the setting of protocol variability, data scarcity, interpretability, domain shift, and clinically actionable reasoning, including a framework explicitly named MMRQA that combines acquisition-aware signal processing with multimodal LLMs (Jia et al., 29 Sep 2025).

1. Problem formulation and scope

MRI quality control has traditionally been prospective and expert-driven. In abdominal MRI, motion artifacts are described as the most common artifacts, and non-diagnostic scans can lead to patient-revisits and rescans when prospective expert review detects insufficient diagnostic quality (Ma et al., 2019). In large repositories, the problem is broader: heterogeneity in image acquisition, reconstruction, and processing pipelines across MRI vendors and imaging sites complicates both repository construction and downstream analysis, motivating a quality assessment pipeline that is systematic and scalable (Zou et al., 2021).

The literature therefore treats MRI quality as a multi-factor construct rather than a single scalar property. Some works target general image acceptability on a Likert scale, some distinguish diagnostic from non-diagnostic images, some score specific perceptual criteria such as signal-to-noise ratio, contrast-to-noise ratio, and artifact presence, and some define quality by its effect on a downstream task such as segmentation (Küstner et al., 2018). This suggests that MMRQA is best understood as a family of reference-free or weakly supervised inference problems whose label semantics depend on the acquisition context and the intended clinical or computational use.

A further complication is that multimodality in MRI has several distinct meanings. In some studies it refers to variation across pulse sequences or contrasts, such as T1-w, T2-w, PD-w, and FLAIR; in others it refers to combinations of modalities such as T2-weighted, high b-value DWI, and ADC in prostate mpMR imaging; in yet others it denotes variability across sites, scanners, orientations, or task formulations (Saeed et al., 2022). The engineering literature consequently recommends goal-oriented QA modules, metadata-aware processing, and modular pipeline design so that quality thresholds can be adjusted to the end-use, whether radiologist reading, research dataset curation, or automated analysis (Zou et al., 2021).

2. Human labels, subjective criteria, and calibration

A central axis of MMRQA research is the conversion of expert perception into machine-tractable supervision. A reference-free framework trained on human-observer-derived labels used a blinded reading platform, active learning, and a 5-point Likert scale, and reported a test accuracy of 93.7\% for estimating quality classes on that scale (Küstner et al., 2018). In a motion-focused abdominal study, two radiologists assigned three classes—non-diagnostic, diagnostic, and excellent—over approximately 2111 T2-weighted abdominal MR images, with binary and three-class evaluation revealing both the feasibility of automated classification and the instability of borderline labels (Ma et al., 2019).

Subjective variability is repeatedly identified as a structural limitation rather than a nuisance term. The abdominal study reports significant disagreement between radiologist raters, low Jaccard Index across label pairs, and class imbalance that can make accuracy misleading unless confusion matrices and ROC/AUC are also examined (Ma et al., 2019). A large benchmark of MRI image quality metrics made this issue explicit by collecting 14,700 subjective scores from seven trained radiologists on three criteria—SNR, CNR, and artifacts—over outputs of neural network models for scan acceleration, motion correction, and denoising (Kastryulin et al., 2022). In that benchmark, the task was not merely to classify image quality but to correlate metric outputs with radiologist perception across anatomies and reconstruction settings.

Several studies attempt to regularize this subjectivity. “Artifact- and content-specific quality assessment for MRI with image rulers” introduced calibrated labels and image rulers, arguing that radiologists’ image quality requirements vary with the scan type and diagnostic task; the model supports assessments of two common artifacts, noise and motion, and achieved accuracies of around 90\%, 6\% better than the best previous method examined, and 3\% better than human experts on noise assessment (Lei et al., 2021). The same work uses context-specific ruler thresholds so that raw CNN outputs can be interpreted relative to scan-specific standards rather than a universal pass/fail boundary.

3. Quantitative metrics, reference-free models, and non-reference regression

A second axis of the field replaces or supplements direct image supervision with quantitative image quality metrics. MRQy was introduced as an open-source, unsupervised tool to interrogate MRI cohorts for site- or equipment-based differences and to quantify the impact of MRI artifacts on relative image quality; it extracts quality measures such as noise ratios, variation metrics, entropy and energy criteria, along with metadata, and provides an HTML5-based front-end for filtering and trend visualization (Sadri et al., 2020). Representative measures include

$m_{11} = \frac{\mu_{FP-BP}}{\sigma_{BP}}$

for CNR and

$$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$

for FBER (Sadri et al., 2020).

A related study used MRQy-derived quantitative metrics as predictors of neuroradiologist assessment on 3D-GRE and 3D-TSE post-contrast T1-weighted brain MRI. After correlation filtering, nine metrics remained for modeling—VAR, CV, CPP, PSNR, SNR3, CNR, CVP, EFC, and FBER—and a decision tree classifier with a bagging ensemble approach was trained for image quality and imaging artifact prediction. The final generalized model reported AUROC 0.77 for image quality and 0.78 for imaging artifact, and performed slightly better than the sequence-specific models, which the authors interpreted as evidence for the value in comprehensive training data (McCullum et al., 2023).

Direct reference-free classification remains an important strand. In one framework, derived image features were paired with support-vector-machine and deep neural network classifiers; in another, a simple 4-layer CNN and ResNet-10 were evaluated for diagnostic versus non-diagnostic abdominal MRI, with the shallow network outperforming the deeper model and reaching 84\% accuracy for the binary task and 65\% for the three-class task (Küstner et al., 2018). The abdominal study further notes that the shallow network likely benefitted from the relative uniformity of features in a single scan protocol and the dominance of motion artifact as the discriminant (Ma et al., 2019).

Non-reference regression has expanded these ideas to volumetric artifact profiling. A 3D ResNet-50 approach for brain MRI quality assessment outputs six quality scores in the $[0,1]$ range, each corresponding to a specific artifact or quality aspect: contrast change, bias field, Gibbs ringing, motion ghosting, Rician noise, and blur effect (Risager et al., 2024). Its focal MSE loss is

$$\mathcal{L} = \frac{1}{M} \sum_{m=1}^{M} \Big(1 + \alpha |Y_m - T_m|^{\gamma}\Big)(Y_m - T_m)^2,$$

with $M=6$, $\alpha=2$, and $\gamma=1$ (Risager et al., 2024). This is presented as a non-reference method for both real and synthetic 3D medical images.

The relation between generic image quality metrics and radiologist perception remains unsettled. In the 35-metric benchmark, the top performers—DISTS, HaarPSI, VSI, and FID-VGG16—were found to be efficient across SNR, CNR, and artifact criteria, while no-reference metrics such as BRISQUE, PaQ-2-PiQ, and MetaIQA performed poorly on MRI, reinforcing the domain-gap argument that metrics designed for natural images may not transfer reliably to MRI (Kastryulin et al., 2022).

Approach	Representative implementation	Reported elements
Quantitative unsupervised QC	MRQy (Sadri et al., 2020)	Quality measures, metadata, HTML5 filtering and trend visualization
Radiologist-surrogate metric learning	Decision tree with bagging on MRQy metrics (McCullum et al., 2023)	AUROC 0.77 for image quality and 0.78 for imaging artifact
Reference-free classification	SVM/DNN and CNN-based frameworks (Küstner et al., 2018)	93.7\% Likert-scale test accuracy
Non-reference artifact regression	3D ResNet-50 with six regression heads (Risager et al., 2024)	Scores in $[0,1]$ for six artifact dimensions

4. Task-aware quality: uncertainty, task amenability, and overlap models

A major conceptual shift in MMRQA is the move from perceptual quality estimation to task-aware quality estimation. “Estimating MRI Image Quality via Image Reconstruction Uncertainty” reformulates quality assessment as an image reconstruction problem and trains Bayesian CNNs using a heteroscedastic uncertainty model to recover clean images from noisy data while predicting uncertainty maps (Shaw et al., 2021). The core regression loss is

$$\mathcal{L}_{reg} = \frac{1}{2 \sigma_r^2} \|\mathbf{y}_r - \mathbf{f}_r^{\mathbf{W}}(\mathbf{x})\|^2 + \frac{1}{2}\log \sigma_r^2 .$$

The paper argues that quality control for visual assessment cannot be equated to quality control for algorithmic processing, and reports a high correlation, $R^2 \approx 0.85$, between predicted uncertainty and non-removable reconstruction error (Shaw et al., 2021).

A related work on segmentation quality estimation uses a probabilistic 3D U-Net with a heteroscedastic noise model and k-space artefact augmentation. It decomposes uncertainty into task uncertainty and per-artefact uncertainties according to

$$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$0

with the combined loss

$$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$1

Mean artefact uncertainty correlated with segmentation Dice more strongly than SNR or CNR, with Spearman’s $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$2 on simulated artefacts and $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$3 on real-world artefacts (Shaw et al., 2021). The same study emphasizes that uncertainty maps localize artefact regions and provide error bars on volume measurements.

Task amenability offers a third formulation. In prostate mpMR imaging, a controller $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$4 is optimized alongside a target task predictor in a meta-learning or reinforcement-learning framework, and a task-agnostic auto-encoding objective is added to discriminate between clinically difficult cases and technically defective acquisitions (Saeed et al., 2022). The reward shaping term is

$$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$5

where $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$6 controls the trade-off between task-specific and task-agnostic quality (Saeed et al., 2022). On prostate tumour segmentation, the non-selective baseline Dice was $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$7, task-agnostic IQA alone achieved $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$8, task-specific IQA alone reached $$m_{15} = \frac{\mathrm{median}(|F|^2)}{\mathrm{median}(|B|^2)}$$9, and an overlap setting such as $[0,1]$0 yielded $[0,1]$1 (Saeed et al., 2022).

Taken together, these formulations suggest that MMRQA increasingly treats “quality” as conditional on a downstream operator. An image may be visually degraded yet algorithmically usable, or visually acceptable yet damaging to a specific segmentation or classification pipeline (Shaw et al., 2021).

5. Heterogeneity, multimodality, and domain shift

Generalization across scanners, sites, contrasts, and acquisition settings is one of the defining constraints of MMRQA. FetMRQC addresses this in fetal brain MRI with an ensemble of 332 image quality metrics extracted from raw T2-weighted stacks, including intensity-based, mask-based, segmentation-based, deep learning-based, and metadata-based features, followed by random forest regression and classification (Sanchez et al., 2023). The dataset contains 1649 stacks from 233 subjects, 13 different scanners, and 4 medical centers; the reported QC median F1-score was 0.86 in 10-fold CV, 0.80 in leave-one-scanner-out CV, and 0.82 on a pure test from a new site (Sanchez et al., 2023).

Scanner-agnostic motion QC has also been attacked directly in image space. A hybrid CNN-attention framework trained on the MR-ART dataset and evaluated on unseen ABIDE sites uses a hierarchical 2D CNN encoder together with a multi-head cross-attention mechanism to emphasize motion-relevant signatures such as ringing and blurring while filtering out site-specific intensity variations and background noise (Bakhale et al., 16 Apr 2026). On seen sites it achieved scan-level accuracy 0.9920 and F1-score 0.9919, and on unseen ABIDE sites it reported scan-level accuracy 0.755 without retraining or fine-tuning (Bakhale et al., 16 Apr 2026).

Other studies explicitly define multimodality through contrast, orientation, or paired-sequence fusion. AutoMAC-MRI grades motion artifacts on more than 5000 expert-annotated brain MRI slices spanning T1-w, T2-w, PD-w, and FLAIR, as well as axial, coronal, sagittal, and oblique views; it uses supervised contrastive learning and reports overall accuracy 84.0\%, together with grade-specific affinity scores termed Motion Grade Affinity Score (MoGrAS) (Jerald et al., 17 Dec 2025). In prostate biparametric MRI, a few-shot prototypical network meta-trained on distortion labels uses a dual-branch 3D ResNet to fuse T2-weighted and DWI features, with FiLM and a gradient reversal layer to handle b-value variation, and achieves 72.9\% balanced accuracy on PI-QUAL using only five representative support samples per class (Tang et al., 17 Jun 2026).

Setting	Heterogeneity handled	Selected reported result
FetMRQC (Sanchez et al., 2023)	4 centers, 13 scanners, raw fetal brain T2-weighted stacks	QC median F1 0.86 in 10-fold CV; 0.80 in LoSo CV
Scanner-agnostic CNN-attention (Bakhale et al., 16 Apr 2026)	MR-ART training, ABIDE unseen 17 sites	Seen-site scan accuracy 0.9920; unseen-site scan accuracy 0.755
AutoMAC-MRI (Jerald et al., 17 Dec 2025)	T1-w, T2-w, PD-w, FLAIR; multiple orientations	Overall accuracy 84.0\%
Few-shot biparametric prototypical network (Tang et al., 17 Jun 2026)	T2WI + DWI, varying b-values, PI-QUAL transfer	72.9\% B-ACC with five support samples per class

This body of work indicates that “multimodal” in MMRQA is not merely a matter of stacking sequences. It also includes contrast invariance, orientation invariance, acquisition-parameter conditioning, site robustness, and support for labels whose positive class is rare, heterogeneous, and clinically consequential (Tang et al., 17 Jun 2026).

6. MMRQA as a multimodal large-language-model framework

The framework explicitly titled “MMRQA: Signal-Enhanced Multimodal LLMs for MRI Quality Assessment” proposes a three-stage architecture intended to bridge quantitative analysis and semantic reasoning (Jia et al., 29 Sep 2025). First, MRQy is used to extract a set of 15 robust quantitative metrics, including SNR, CNR, CPP, EFC, and FBER, and simulated artifacts are introduced to augment the quality-control training data. Second, the Qwen-Max LLM transforms the numerical metrics into structured question-answer pairs such as classification, artifact detection, and analysis or suggestion prompts. Third, a LoRA-adapted LLaVA-OneVision model fuses visual and textual information, with LLaVA-OneVision combining SigLIP visual encoding and the Qwen2 LLM (Jia et al., 29 Sep 2025).

The parameter-efficient adaptation is written as

$[0,1]$2

with reported LoRA settings $[0,1]$3 and $[0,1]$4 (Jia et al., 29 Sep 2025). The stated purpose of this decomposition is to adapt the multimodal LLM with significantly fewer trainable parameters while preserving efficiency for clinical deployment.

Evaluation is reported on three benchmarks: MR-ART, FastMRI, and MyConnectome. The framework achieved 88.1\%, 85.1\%, and 72.2\% accuracy, and 85.7, 82.8, and 66.7 F1, respectively, together with GPT-5 descriptive scores of 85.4, 83.7, and 68.9 (Jia et al., 29 Sep 2025). The paper further reports strong zero-shot generalization on MyConnectome, and ablation studies indicate that removing MRQy metrics or reducing the QA transformation degrades performance (Jia et al., 29 Sep 2025). In the authors’ formulation, interpretability is obtained not only from metrics but also from semantically structured outputs that can identify artifacts, explain quality judgments, and propose acquisition improvements.

This framework is notable because it does not discard the metric-based tradition; instead, it uses metric extraction as the grounding layer for multimodal reasoning. In that sense, it operationalizes an overview that earlier work had only implied: signal descriptors provide acquisition-aware evidence, while language-model components provide semantic organization and clinically interpretable output (Sadri et al., 2020).

7. Limitations, controversies, and open directions

Several persistent controversies define the current state of MMRQA. The first is the instability of ground truth. Diagnostic quality lacks a universal, objective definition; inter-rater variability, unreliable labels, and severe class imbalance are repeatedly reported, especially for borderline cases and for distinctions such as diagnostic versus excellent (Ma et al., 2019). The second is the gap between perceptual and operational quality: what is deemed suitable quality for algorithmic processing may be different from human-perceived measures of visual quality, and less data may need to be excluded based on visual quality reasons alone when the downstream task remains unaffected (Shaw et al., 2021).

A third issue is metric validity. The radiologist-anchored benchmark of 35 IQMs concludes that the optimal MRI metric is yet to be devised, that no-reference metrics substantially underperform, and that the ranking of metrics in natural-image IQA is not predictive of MRI performance (Kastryulin et al., 2022). Metric-based surrogate models can nevertheless be useful: MRQy-derived metrics paired with bagged decision trees achieved generalized AUROC values around 0.77–0.78 for image quality and artifact prediction, but the same study states that further work needs to validate the model on an external dataset (McCullum et al., 2023).

The systems literature frames these issues in process terms. A proposed MRI QA pipeline emphasizes four design principles aligned with ISO 9001:2015—goal-oriented design, process monitoring, risk management, and outcome assurance—and argues that quality definitions should depend on whether the target is radiologist reading, research curation, or a downstream AI model (Zou et al., 2021). This suggests that future MMRQA systems are likely to remain plural rather than converging on a single universal score: acquisition-aware metrics, task-specific uncertainty, calibrated human labels, domain-robust representation learning, and multimodal reasoning each address different failure modes.

A plausible implication is that MMRQA is evolving from retrospective image triage to prospective, adaptive quality control. Earlier frameworks already described prospective quality assurance including automatic acquisition adaptation or guided MR scanner operation (Küstner et al., 2018). Later systems add uncertainty maps, scanner-agnostic motion detection, affinity scores, few-shot adaptation, and multimodal explanatory outputs (Shaw et al., 2021). The resulting field is less a single algorithm than a layered QA stack for heterogeneous MRI ecosystems.