Does Machine Unlearning Preserve Clinical Safety? A Risk Analysis for Medical Image Classification

Abstract: The application of Deep Learning in medical diagnosis must balance patient safety with compliance with data protection regulations. Machine Unlearning enables the selective removal of training data from deployed models. However, most methods are validated primarily through efficiency and privacy-oriented metrics, with limited attention to clinically asymmetric error costs. In this work, we investigate how unlearning affects clinical risk in binary medical image classification. We show that standard unlearning strategies (Fine-Tuning, Random Labeling, and SalUn) may reduce test utility while increasing false-negative rates, thereby amplifying clinical risk. To mitigate this, we propose SalUn-CRA (Clinical Risk-Aware), a variant of SalUn that replaces random relabeling with entropy-based forgetting for malignant samples in the forget set, preventing the model from learning harmful benign associations. We evaluate on DermaMNIST and PathMNIST medical image datasets under 20% and 50% data removal. Using Global Risk metrics with asymmetric costs, SalUn-CRA achieves lower or comparable clinical risk to full retraining while preserving unlearning effectiveness. These results suggest that clinical risk should be an integral component of unlearning validation in medical systems.

• The paper demonstrates that standard machine unlearning methods risk reducing malignant recall, while SalUn-CRA maintains diagnostic safety.
• It introduces and evaluates novel clinical risk metrics and asymmetric cost formulations on DermaMNIST and PathMNIST to capture false negative penalties.
• SalUn-CRA uses entropy-based regularization for forgotten malignant samples, ensuring effective unlearning without compromising overall model utility and privacy.

Clinical Risk in Machine Unlearning for Medical Image Classification

Introduction

The paper "Does Machine Unlearning Preserve Clinical Safety? A Risk Analysis for Medical Image Classification" (2604.23854) addresses the intersection of regulatory compliance and diagnostic reliability in the context of deep learning models deployed for clinical image analysis. With increasing legal obligations, such as GDPR and LGPD, requiring selective removal ("right to be forgotten") of patient data from machine-learned models, questions of technical feasibility and safety are paramount. Machine Unlearning (MU) algorithms, designed to remove the parameter imprint of specific data, have advanced rapidly. However, the dominant paradigms of their quantitative evaluation—the focus on utility, privacy, and similarity to retraining—do not capture clinical asymmetries, especially the heightened risk associated with false negatives (missed malignant cases). This work systematically formulates, measures, and mitigates clinical risk amplification due to unlearning procedures, providing new metrics and a risk-aware MU modification.

SalUn-CRA: A Clinical Risk-Aware Unlearning Method

Conventional MU methods, such as Fine-Tuning (FT), Random Labeling (RL), and Saliency Unlearning (SalUn), typically address the challenge of "forgetting" by updating a previously trained model to disregard the influence of designated data samples. FT simply continues training on the retain set; RL assigns random labels to the forget set, and SalUn localizes parameter updates to weights salient for the forget set (via gradient-based masking) and applies RL only to those. These techniques remain agnostic to the clinical significance of class assignments.

In binary clinical tasks—malignant versus benign—standard SalUn and RL approaches inadvertently relabel forgotten malignant samples as benign, causing the model to erroneously learn benign associations for truly malignant features. This shift reduces recall (sensitivity) for malignancy, with grave clinical consequences that accuracy metrics obscure. To address this, the authors introduce SalUn-CRA, wherein:

• For forgotten malignant samples, the model maximizes predictive entropy (driving the output toward uniform uncertainty), rather than performing random relabeling.
• For forgotten benign samples, the standard RL is retained, as the risk associated with false positives is lower in clinical settings.

This asymmetric mechanism, implemented at the loss level, selectively targets the high-cost error mode without degrading unlearning effectiveness for retain/forget set utility.

Evaluation Protocol and Metrics

The study benchmarks SalUn-CRA and existing MU approaches on binarized DermaMNIST and PathMNIST tasks—both drawn from well-established clinical image datasets—under 20% and 50% balanced data removal scenarios. All experiments leverage a ResNet-18 backbone with class-weighted cross-entropy for training. Key aspects of the evaluation include:

• Utility Metrics: Specificity, recall, balanced accuracy (BAC), and AUC, emphasizing performance on imbalanced classes.
• Unlearning Metrics: BAC on forget (UBAC), retain (RBAC), and test sets (TBAC), as well as Membership Inference Attack (MIA) accuracy and metric gap (GAP) to retraining baselines.
• Clinical Risk Metrics: Two cost-sensitive risk formulations—Global Risk I (equal cost for false positives and negatives) and Global Risk II (20x penalty for false negatives)—better reflect the asymmetric consequences prevalent in clinical contexts.

Empirical Results

SalUn-CRA demonstrates clear benefits in both efficacy and safety. Across all datasets and removal rates, it consistently records the highest recall for malignant cases, with BAC and AUC competitive with or surpassing alternative MU methods. This recall preservation directly controls the clinical risk indices, as shown in Risk II.

Figure 1: Clinical risk comparison across unlearning methods with forget rates 20% and 50% on DermaMNIST and PathMNIST datasets.

For DermaMNIST, standard MU approaches often reduce Risk I (symmetric costs) at the expense of increasing Risk II—signifying a rise in false negatives masked by overall accuracy. In these settings, SalUn-CRA achieves substantially lower Risk II than both FT/RL and the baseline SalUn, indicating reduced missed malignancies after unlearning.

On PathMNIST, where baseline performance is higher, SalUn-CRA maintains minimal GAP and leads in reducing Risk II, suggesting that the entropy-based forgetting does not incur a trade-off in generalization or privacy relative to retraining.

Figure 2: Trade-off between unlearning effectiveness (GAP) and Global Risk. Lower-left region represents ideal performance (low GAP and low risk). Circles indicate 20% removal; squares indicate 50% removal.

Scatter plots illustrate that SalUn-CRA occupies a Pareto-optimal enclave: both unlearning effectiveness (GAP) and global risk (especially Risk II) are jointly minimized, in contrast to RL and FT which achieve lower risk only for symmetric cost structures or at the expense of increased model divergence.

Theoretical and Practical Implications

This work reframes machine unlearning in sensitive domains as an intrinsically cost-sensitive problem. The evidence shows that in clinical pipelines, symmetric accuracy-driven MU validation may yield solutions that violate the underlying value structure (i.e., prioritization of missed cancers versus false alarms). The explicit modeling of risk, leveraging custom cost matrices, challenges not only technical implementations but also regulatory assessment protocols.

SalUn-CRA's selective deployment of entropy regularization for high-risk examples points to a direction where MU losses and forgetting policies can be adapted based on domain semantics—not merely dataset structure or computational efficiency. Future extensions could explore dynamic cost calibration per pathology, integration with multi-class hierarchies, and prospective impact analysis in clinical deployment scenarios. Furthermore, these principles can be generalized to regulatory-driven ML model adaptation in any vertical with asymmetric stakeholder harm/cost distributions.

Conclusion

The investigation demonstrates that existing MU approaches, if left unchecked, may heighten clinical risk due to their disregard for asymmetric diagnostic costs. The proposed SalUn-CRA mitigates this by integrating a clinically-aligned, entropy-based forgetting strategy, yielding lower risk to patients and maintaining close fidelity to retraining in both utility and privacy metrics. As a consequence, evaluation of machine unlearning in medical contexts must prioritize explicitly cost-sensitive risk criteria, ensuring that compliance with privacy mandates does not come at the expense of patient safety.