Do Multiple Instance Learning Models Transfer? (2506.09022v2)

Abstract: Multiple Instance Learning (MIL) is a cornerstone approach in computational pathology (CPath) for generating clinically meaningful slide-level embeddings from gigapixel tissue images. However, MIL often struggles with small, weakly supervised clinical datasets. In contrast to fields such as NLP and conventional computer vision, where transfer learning is widely used to address data scarcity, the transferability of MIL models remains poorly understood. In this study, we systematically evaluate the transfer learning capabilities of pretrained MIL models by assessing 11 models across 21 pretraining tasks for morphological and molecular subtype prediction. Our results show that pretrained MIL models, even when trained on different organs than the target task, consistently outperform models trained from scratch. Moreover, pretraining on pancancer datasets enables strong generalization across organs and tasks, outperforming slide foundation models while using substantially less pretraining data. These findings highlight the robust adaptability of MIL models and demonstrate the benefits of leveraging transfer learning to boost performance in CPath. Lastly, we provide a resource which standardizes the implementation of MIL models and collection of pretrained model weights on popular CPath tasks, available at https://github.com/mahmoodlab/MIL-Lab

• The paper shows that pretrained MIL models outperform random initialization, especially on pan-cancer datasets in computational pathology.
• It highlights that model architecture and aggregation schemes critically influence the effectiveness of transfer learning.
• The research demonstrates enhanced data efficiency in few-shot learning, indicating practical benefits for clinical applications with limited data.

Evaluating Transferability of Multiple Instance Learning Models for Computational Pathology

The academic paper titled "Do Multiple Instance Learning Models Transfer?" presents a thorough examination of the capacity for transfer learning of Multiple Instance Learning (MIL) models within the domain of computational pathology (CPath). In contrast to fields like natural language processing and traditional computer vision, where transfer learning techniques are extensively applied to mitigate data scarcity, the transferability of MIL models remains inadequately explored.

Research Context and Objectives

Multiple Instance Learning serves as a pivotal approach in computational pathology, offering a means to derive clinically valuable embeddings at the slide level from high-dimensional tissue images. However, the prevalent challenge of limited, weakly labeled clinical datasets impairs the efficacy of direct training approaches in MIL. This research aims to systematically evaluate whether MIL models can benefit from transfer learning by pretraining on a broader set of tasks and subsequently fine-tuning them for specific applications.

Methodology

The research team assessed the performance of 11 distinct MIL models across 21 pretraining tasks aimed at predicting both morphological and molecular subtypes. These models were evaluated through a comparative analysis against randomly initialized counterparts across 19 classification tasks. The paper incorporated various domains, including binary and multiclass cancer classification tasks, cancer grading, and biomarker predictions. Central to this methodology was examining whether deploying prior knowledge through pretraining can alleviate the limitations of small datasets commonly encountered in CPath.

Key Findings

1. Pretraining Efficacy: The research consistently found that MIL models pretrained on a diverse range of tasks achieve superior performance compared to their randomly initialized counterparts. Models pretrained on pan-cancer datasets showed substantial gains, even when applied to tasks involving organs different from those in the pretraining set. This suggests that knowledge captured in foundational tasks is effectively transferred across different pathological tasks.
2. Implications of Model Architecture: The paper revealed that while all MIL architectures benefit to some extent from pretraining, the degree of improvement is architecture-dependent. Larger models with transformer-based architectures demonstrated more pronounced benefits, indicating the potential for scaling up MIL models when pretrained effectively.
3. Data Efficiency in Few-shot Learning: A critical insight from the evaluations was the demonstration of enhanced data efficiency of MIL models in few-shot learning scenarios. Pretrained models showed improved performance relative to non-pretrained models, highlighting the potential of pretraining to facilitate model training with minimal data.
4. Aggregation Scheme Importance: The aggregation scheme used in MIL models was found to be crucial for effective knowledge transfer. The paper emphasizes the need for robust aggregation strategies to enhance transferability outcomes.
5. Envisioned Role in Clinical Tasks: The paper illustrates how MIL models can be adapted into general-purpose pathology tools capable of performing varied clinical tasks, thereby reducing the need for large-scale data acquisition in low-resource settings.

Practical and Theoretical Implications

The findings suggest that the deployment of pretrained MIL models can circumvent the limitations posed by small, single-task datasets in pathology. Practically, this allows for broader application in clinical environments, where data collection and labeling are constrained.

From a theoretical perspective, the paper provides a valuable framework for understanding the dynamics of knowledge transfer in the field of high-dimensional, weakly-supervised learning tasks, potentially guiding future development of slide-level foundation models. By delineating the conditions under which MIL models transfer effectively, this research serves as a precursor to further exploration of foundational models adapted specifically for the complexities of pathology tasks.

Future Directions

Looking ahead, the research opens up multiple avenues for the future of MIL models in CPath. These include exploring integrated models capable of handling multimodal datasets, enhancing self-supervised pretraining approaches, and examining transfer learning's efficacy in real-time clinical settings with more diverse and complex pathological datasets. The potential to improve model adaptability and performance in even more challenging diagnostic scenarios holds promise for substantial advancements in computational pathology.

In conclusion, this paper lays a foundation for enhanced model adaptability through MIL transfer learning, charting a course for more accessible and efficient AI applications in the diagnosis and treatment of pathological conditions. The physic convergence of tasks and the effective deployment of pretraining strategies underscore a critical step towards making AI-driven pathology solutions more viable and effective in clinical practice.