MedForget: Hierarchy-Aware Medical Unlearning

• MedForget is a framework that enables selective deletion of sensitive data from multimodal medical models by leveraging hierarchical unlearning to comply with HIPAA/GDPR.
• It employs advanced techniques such as gradient difference minimization, KL regularization, and modality-aware neuron scoring to balance forgetting strength with diagnostic utility.
• The testbed benchmarks unlearning efficacy across clinical hierarchies and evaluates privacy risks through membership inference and reconstruction attack metrics.

MedForget is a hierarchy-aware framework and testbed for multimodal unlearning in medical AI, specifically designed to facilitate compliance with regulatory requirements such as HIPAA and GDPR. It systematically addresses the technical challenge of selectively deleting the influence of sensitive data—such as image–text pairs or entire patient records—from large-scale multimodal models, while balancing diagnostic utility and privacy risk. MedForget incorporates structured data hierarchies, explicit unlearning benchmarks at multiple granularity levels, and advanced evaluation protocols to quantify the trade-offs inherent in medical machine unlearning (Wu et al., 10 Dec 2025).

1. Hierarchical Data Organization in MedForget

MedForget models healthcare data as a nested hierarchy reflecting the organizational structure of clinical environments:

• Levels: Institution → Patient → Study → Section
• Coverage: 8 institutions, 64 patients, 256 studies, 1024 sections; each section generates three vision–question–answer (VQA) tasks (generation, cloze, classification), yielding 3840 multimodal instances.
• Inheritance: Unlearning a higher-level node implies removal of all its descendants (e.g., unlearning a patient erases all associated studies and sections).

This hierarchical formalism enables fine-grained analysis of unlearning efficacy and potential privacy leakages at each organizational layer. The dataset also includes paraphrased "forget–rephrase" splits, challenging models to generalize forgetting beyond verbatim memorization (Wu et al., 10 Dec 2025).

2. Unlearning Methodologies and Protocols

MedForget and prior art integrate several state-of-the-art unlearning techniques:

• Gradient Difference (GradDiff): Alternately minimizes performance on the forget set while maximizing retention set fidelity, using loss

$$\mathcal{L}_{\rm diff}(\theta) = -\mathcal{L}(F;\theta) + \mathcal{L}(R;\theta)$$

• KL Minimization: Combines negative log-likelihood minimization on the forget set with output distribution regularization on the retain set via Kullback-Leibler divergence.
• Negative Preference Optimization (NPO): Adjusts policy probabilities to suppress recalled likelihood for forgotten data relative to a reference policy.
• Modality-Aware Neuron Unlearning (MANU): Computes importance scores for neurons with respect to both forget and retain sets, pruning those most influential for the forget set.
• Forget-MI (for joint unimodal/multimodal unlearning): Simultaneously maximizes embedding divergence for forget data and minimizes it for retain data, utilizing dedicated retention and forgetting loss terms plus noise injection to encourage forgetting generalization (Hardan et al., 29 Jun 2025).
• Federated Client Unlearning (FCU): In federated settings, uses feature-level model-contrastive loss and frequency-guided memory preservation to efficiently unlearn the contribution of a client’s data, followed by selective post-training across remaining clients (Deng et al., 2024).

All approaches balance "strength of forgetting" and "utility retention," often controlled by tunable hyperparameters defining loss weightings.

3. Benchmarking, Evaluation, and Metrics

MedForget provides explicit retain–forget splits at every hierarchy level and defines rigorous evaluation tasks:

• Tasks: Generation (free-text answer), Cloze (token prediction), Classification (multiple-choice selection)
• Metrics:
  • Generation Score (GenScore): Weighted sum of LLM factuality and ROUGE-L recall.
  • Cloze Accuracy: Proportion of exact matches in cloze tasks.
  • Classification Accuracy: Top-1 accuracy on multiple-choice questions.
  • Forgetting Score: Measures performance drop on the forget set post-unlearning:

  $$\mathrm{ForgetScore}_m = 1 - \frac{m_{\rm after}(F)}{m_{\rm before}(F)}$$ - Utility Preservation: Performance on the retain set and external VQA benchmarks (General-Med set).

• Membership Inference Attack (MIA): Evaluates privacy risk by quantifying the adversary’s ability to distinguish forgotten data from unseen data via per-sample losses (Hardan et al., 29 Jun 2025).
• Hierarchical Reconstruction Attack: Assesses persistence of forgotten content as more hierarchical context is added to prompts, quantifying attack success rates as a function of context exposure.

Empirical results show that coarse unlearning (institution/patient) offers the strongest forgetting and lowest attack success but at the expense of global diagnostic performance, while fine-grained unlearning better preserves utility yet leaves residual leakage via reconstruction and prompt engineering (Wu et al., 10 Dec 2025).

4. System Architectures and Algorithms

MedForget implementations span centralized, federated, and multimodal model architectures:

• Centralized Multimodal Models: Typically fuse ResNet or ViT encoders for image data with SciBERT or LLM-based encoders for text, merged via late-fusion adaptation gates.
• Federated Learning Systems: Employ DenseNet-121 or analogous CNN backbones for distributed training, with server–client orchestration, FedAvg for aggregation, and client-triggered unlearning via the FCU protocol (Deng et al., 2024).
• Assistance and Adherence Modules: Integrate multimodal sensor streams (vision, depth, ASR) with event-driven decision orchestrators, often using LLMs (e.g., GPT-4o) for adaptive dialogue management, as demonstrated in medication adherence support for PLWDs (Gangaraju et al., 8 Oct 2025).
• Forecasting: Sparse-event treatment adherence prediction is performed via CNN or LSTM architectures leveraging high-resolution sensor streams and future-knowledge features, with model personalization via on-device incremental retraining (Mamun et al., 20 Mar 2025).

Core procedural algorithms include loss-based fine-tuning, feature-level contrastive unlearning, noise-based embedding separation, and pruning or freezing policies guided by neuron/memory importance metrics.

5. Clinical and Regulatory Motivations

MedForget addresses the technical underpinnings of regulatory mandates, such as:

• HIPAA/GDPR Right to be Forgotten: Medical AI models must support selective deletion of a patient's data influence on predictions—the central aim of machine unlearning.
• Privacy Risk Mitigation: Hierarchy-aware unlearning prevents residual content leakage, which is a key limitation when deleting only lowest-level (section-specific) nodes (Wu et al., 10 Dec 2025).
• Clinical Utility Preservation: Trade-offs are systematically characterized, enabling informed selection of unlearning granularity in deployed systems.
• Rapid, On-Demand Compliance: Federated unlearning protocols eliminate the need for full dataset retraining in response to deletion requests, achieving 10–15× speedups over naively retraining from scratch while closely matching diagnostic performance and forgetting efficacy (Deng et al., 2024).

A plausible implication is that real-world clinical deployments will require hybrid unlearning approaches, combining coarse and fine strategies depending on data importance, regulatory demands, and risk of adversarial reconstruction.

6. Limitations, Extensions, and Open Challenges

Despite progress, MedForget surfaces several open issues:

• Residual Information Leakage: Fine-grained unlearning leaves models susceptible to reconstruction attacks leveraging higher-level context, demonstrating that naïve minimum-deletion strategies are inadequate for full compliance.
• Task and Modality Restrictions: Most current algorithms are optimized for classification; extension to segmentation, detection, or continuous-valued forecasting requires adaptation of unlearning objectives and model-contrastive schemes (Deng et al., 2024).
• Scalability: Increasing forget set cardinality (≥10%) degrades both forgetting efficacy and utility, underscoring intrinsic limits of current loss-based and pruning-based approaches (Hardan et al., 29 Jun 2025).
• Differential Privacy Integration: While not addressed in the core protocols, integration of DP-style regularizers offers quantifiable privacy guarantees for future MedForget expansions.
• Personalization and Adaptation: On-device personalization, as in adherence forecasting, is essential to counter global model forgetting and should be a required component for mobile MedForget modules (Mamun et al., 20 Mar 2025).

Further development and open science testbeds, such as the publicly available MedForget benchmark (Wu et al., 10 Dec 2025), provide the groundwork for continued methodological advances and robust regulatory adherence.