Useful nonrobust features are ubiquitous in biomedical images

Abstract: We study whether deep networks for medical imaging learn useful nonrobust features - predictive input patterns that are not human interpretable and highly susceptible to small adversarial perturbations - and how these features impact test performance. We show that models trained only on nonrobust features achieve well above chance accuracy across five MedMNIST classification tasks, confirming their predictive value in-distribution. Conversely, adversarially trained models that primarily rely on robust features sacrifice in-distribution accuracy but yield markedly better performance under controlled distribution shifts (MedMNIST-C). Overall, nonrobust features boost standard accuracy yet degrade out-of-distribution performance, revealing a practical robustness-accuracy trade-off in medical imaging classification tasks that should be tailored to the requirements of the deployment setting.

• The paper demonstrates the ubiquity of useful nonrobust features in biomedical images that boost in-distribution accuracy.
• The study employs adversarial training and saliency maps to isolate feature types across multiple MedMNIST datasets.
• It quantifies a robustness-accuracy trade-off, highlighting implications for clinical diagnostics and model reliability.

Summary of "Useful nonrobust features are ubiquitous in biomedical images"

Formal Definition of Robust and Nonrobust Features

The paper provides a rigorous characterization of robust and nonrobust features within the context of medical imaging. Robust features are those whose correlation to class labels withstands adversarial perturbations bounded by $\epsilon$ in $\ell_\infty$ norm, while nonrobust features are those whose correlation vanishes after such perturbations, often resulting in fragile and uninterpretable patterns. The analysis uses saliency maps to visualize attribution stability: robust models’ saliency remains anatomically meaningful even in perturbed CT slices, while nonrobust models’ saliency degrades into noise-like, uncorrelated patterns.

Figure 1: Saliency maps on kidney CT slices demonstrate attribution stability for robust models and instability for nonrobust models under adversarial perturbations.

Experimental Design and Feature Isolation

The study employs five MedMNIST biomedical imaging datasets, encompassing modalities such as CT, radiography, ultrasound, and pathology. The isolation of nonrobust features follows the adversarial dataset construction method of Ilyas et al.: adversarial examples are generated for random target classes using a trained classifier, creating datasets where only nonrobust features align with new labels. Models trained on these datasets are evaluated across the original test sets to quantify the predictive power of isolated nonrobust features.

Adversarial training is used to obtain robust-feature models via the TRADES loss, balancing clean accuracy and adversarial robustness. Hyperparameter optimization and comprehensive training protocols are applied to ensure fair and optimal evaluation.

Empirical Results: In-distribution, OOD, and Adversarial Robustness

Balanced accuracy is used to mitigate class imbalance. In-distribution, nonrobust-feature-only models achieve well above random-chance accuracy across all tasks—e.g., $59.24\%$ on PathMNIST and $81.62\%$ on PneumoniaMNIST, versus random guessing ($11\%$ and $50\%$, respectively). Full-feature models (standard training) consistently outperform robust-only models, demonstrating the substantial contribution of nonrobust features to predictive accuracy in clean settings.

Figure 2: Test accuracy across MedMNIST datasets comparing standard, robust, and nonrobust-only DNNs in both in-distribution and OOD settings.

However, under out-of-distribution (OOD) shifts (MedMNIST-C corruptions), robust models outperform nonrobust and standard models except for PneumoniaMNIST, with increases in balanced accuracy up to $33\%$. Adversarial attacks via AutoAttack confirm catastrophic degradation for nonrobust and standard models, with adversarial accuracy $\leq 0.04\%$, whereas robust models achieve $57\%-74\%$ accuracy.

Figure 3: Test accuracy comparison between robust and nonrobust models trained with different values of $\epsilon$, revealing accuracy-robustness trade-offs as perturbation bound increases.

The Robustness-Accuracy Trade-off and Practical Implications

The principal finding is that useful nonrobust features exist ubiquitously in biomedical imaging and are leveraged by DNNs for elevated in-distribution accuracy, yet their presence degrades OOD and adversarial robustness. There is a quantifiable robustness-accuracy trade-off: suppression of nonrobust features reduces in-distribution performance ($\ell_\infty$0 absolute), but enhances OOD robustness ($\ell_\infty$1 absolute) and adversarial defense. The choice of perturbation bound ($\ell_\infty$2) in adversarial training impacts accuracy, but the overall trends remain invariant.

This trade-off underscores a deployment-dependent necessity: in environments where reliability and robustness to distribution shifts are paramount (e.g., clinical diagnostics), discouraging nonrobust features may be preferable, despite accuracy losses. Conversely, for settings prioritizing maximum in-distribution accuracy (e.g., retrospective studies), inclusion of nonrobust features is warranted.

Theoretical and Practical Implications, Limitations, and Future Directions

The existence and exploitation of nonrobust features diminish interpretability and reliability in high-stakes medical applications, raising concerns regarding trustworthiness and accountability. Their brittleness under plausible shifts and attacks makes clinical justification of predictions tenuous. The paper highlights the need for further investigation into user preferences and the performance-interpretability trade-off in real-world clinical settings.

Limitations include reliance on artificial OOD corruptions rather than naturally sourced data, exclusive focus on WRN-16-8 CNNs and low-resolution images, and restriction to classification tasks. Extending the study to segmentation, detection, vision transformers, and natural distribution shifts will clarify generalizability. The methodology for isolating nonrobust features is robust, but direct evaluation in clinical workflows remains unaddressed.

Conclusion

This systematic investigation confirms that nonrobust features are abundant and highly predictive in medical imaging, driving in-distribution accuracy while degrading OOD and adversarial robustness. The robustness-accuracy trade-off must be carefully considered in deployment, especially in clinical and safety-critical contexts. The results prompt future explorations into clinical interpretability, performance trade-offs, and expanded modalities and tasks in biomedical AI.