Pretrained MIL Models (Multiple Instance Learning)

Pretrained Multiple Instance Learning (MIL) Models are neural architectures or machine learning frameworks adapted for learning from weakly supervised data, specifically situations where only group-level annotations—called bag labels—are available, and instance-level labels are absent or ambiguous. In the MIL setting, a bag is labeled positive if at least one instance within it is positive, and negative otherwise. Recent advances in MIL have focused extensively on the pretraining of feature extractors, aggregation modules, and entire MIL pipelines to enhance generalization, performance, and transferability—particularly in domains such as computational pathology and computer vision, where annotated data is scarce and heterogeneous.

1. Fundamentals of Pretrained MIL Models

Pretrained MIL models utilize prior knowledge learned from large datasets or related tasks to initialize parameters, which are then adapted (often via fine-tuning) to the target MIL problem. Pretraining can occur at different levels:

• Feature Extractor Pretraining: Instance encoders (e.g., ResNet, ViT) are pretrained on large corpora such as ImageNet, domain-specific datasets, or via self-supervised learning (SSL) (Wong et al., 2 Aug 2024 ).
• Aggregation Module Pretraining: Pooling or attention-based modules that integrate instance features into bag-level representations are trained, sometimes on separate source tasks (Shao et al., 10 Jun 2025 ).
• End-to-End MIL Pretraining: The full MIL model, including both encoder and aggregator, is trained on a source task and then transferred to a new (target) task (Shao et al., 10 Jun 2025 ).

Pretrained models provide improved performance and data efficiency by leveraging learned representations that capture general or domain-specific patterns relevant to weakly supervised MIL problems.

2. Pretraining Strategies and Components

Pretraining Procedures

• Supervised Pretraining: Using large-scale annotated datasets such as ImageNet for model initialization, primarily for feature extractors.
• Self-Supervised Learning (SSL): Leveraging methods such as DINO, MoCo, SwAV, SimCLR, and Barlow Twins, which enable feature extractors to learn from unlabeled data, often yielding more robust, generalizable representations than purely supervised pretraining (Wong et al., 2 Aug 2024 , Meseguer et al., 21 Oct 2024 ).
• Domain-Specific Pretraining: Utilizing large in-domain datasets—often from the target application (e.g., histopathology)—to bridge domain gaps and enhance feature fidelity (Meseguer et al., 21 Oct 2024 , Wong et al., 2 Aug 2024 ).
• Multimodal (Vision-Language) Pretraining: Adopting models such as CLIP, PLIP, or MI-Zero, jointly trained on image-text pairs to capture richer semantic information for zero-shot transfer and improved downstream performance (Lu et al., 2023 , Meseguer et al., 21 Oct 2024 ).
• Aggregation Module Pretraining: Training pooling or attention mechanisms on source MIL tasks or pancancer datasets to capture generalizable bag-level representations (Shao et al., 10 Jun 2025 ).
• Masked Context Modelling and Knowledge Distillation: Fine-tuning feature extractors by predicting contextually masked features using teacher-student frameworks, imparting context awareness (Pisula et al., 8 Mar 2024 ).

Components

• Instance Encoder: Commonly deep CNNs (ResNet, ConvNeXt), Transformers (ViT, Swin-B), or in-domain SSL backbones (CTransPath). Their pretraining paradigm (dataset size/diversity, method) is a major determinant of MIL performance (Wong et al., 2 Aug 2024 ).
• Bag Aggregation Function: Techniques range from global average pooling to attention-based MIL, dual-stream MIL, transformer-based MIL (TransMIL, DGR-MIL), and innovative modules (e.g., DGR-MIL’s cross-attention with global vectors (Zhu et al., 4 Jul 2024 ), SC-MIL’s sparse coding (Qiu et al., 2023 )).
• Prompt/Adaptor Modules: Prompt-MIL and related methods insert small, learnable prompts to adapt frozen encoders to target tasks efficiently (Zhang et al., 2023 ).

3. Empirical Performance, Transferability, and Best Practices

Performance Benchmarks

• In-Domain vs. General Pretraining: Foundation models pretrained on large, diverse, and in-domain data with SSL or multimodal objectives (DINO, CLIP, PLIP) consistently outperform models pretrained on natural images (ImageNet) in WSI classification tasks (Wong et al., 2 Aug 2024 , Meseguer et al., 21 Oct 2024 ).
• Aggregation Module Pretraining: Pancancer pretrained attention-based MIL models outperform or closely approach the performance of slide foundation models, but with a fraction of the pretraining data (2–10%) (Shao et al., 10 Jun 2025 ).
• Cross-Task Generalization: Models pretrained on pancancer data generalize robustly across organs and across task types, achieving average improvements of 3–6% over random initialization (with up to 171% improvement in few-shot regimes) (Shao et al., 10 Jun 2025 ).

Transfer Learning Insights

• End-to-end fine-tuning of pretrained MIL models yields the strongest gains, but even frozen feature evaluation (linear or KNN head) demonstrates substantial improvements over training from scratch (Shao et al., 10 Jun 2025 ).
• Aggregation modules (especially attention mechanisms) display strong transfer stability and confer cross-domain benefits, as quantified by SVCCA analyses (Shao et al., 10 Jun 2025 ).
• Simple architectural upgrades (deeper backbones, transformer-based models) and pretraining on larger, more varied datasets lead to universal performance boosts across diverse MIL methods and datasets (Wong et al., 2 Aug 2024 ).

Recommended Practices

4. Applications Across Modalities and Domains

Computational Pathology

• Whole Slide Image (WSI) Classification: Pretrained MIL models are widely used for cancer diagnosis, subtyping, and molecular prediction with gigapixel histology data, enabling strong slide-level performance without the need for patch-level annotation (Shao et al., 10 Jun 2025 , Wong et al., 2 Aug 2024 , Meseguer et al., 21 Oct 2024 ).
• Cross-Organ and Pancancer Generalization: Pancancer-pretrained models support robust transfer across organs and diseases, including few-shot and rare disease settings (Shao et al., 10 Jun 2025 ).
• Interpretability and Robustness: Combination models such as SI-MIL offer explicit feature-level interpretability within deep MIL frameworks (Kapse et al., 2023 ).

Computer Vision and Multimodal Domains

• General Image Classification: Transfer learning with MILe and MILAN improves accuracy and robustness for multi-label, ambiguous, or weakly labeled data, providing strong performance under noisy or distribution-shifted scenarios (Rajeswar et al., 2021 , Hou et al., 2022 ).
• Zero-Shot Transfer: MI-Zero and similar models leverage vision-language pretraining and prompt-based aggregation for fully zero-shot gigapixel WSI classification (Lu et al., 2023 ).
• Emotion Recognition: Milmer unites MIL, pretrained Swin Transformers, and cross-attention fusion to set new performance benchmarks for multimodal human emotion recognition, indicating utility beyond pathology (Wang et al., 1 Feb 2025 ).

5. Limitations, Challenges, and Future Directions

Limitations

• Domain Gaps: ImageNet or natural image pretraining does not optimally transfer to domains with significant appearance shifts (e.g., medical, remote sensing); in-domain pretraining is essential (Meseguer et al., 21 Oct 2024 ).
• Stain/Scanner Bias: Foundation model representations and MIL performance are sensitive to stain and imagery variance across medical centers (Meseguer et al., 21 Oct 2024 ).
• Resource Demands: State-of-the-art MIL methods with deep transformers and comprehensive pretraining can be resource-intensive. Parameter-efficient tuning strategies (prompt tuning, frozen backbones) mitigate this issue (Zhang et al., 2023 ).

Emerging Directions

• Self-supervised Context Modelling: Incorporating context (masked context modeling with knowledge distillation) for feature extractor fine-tuning produces more robust MIL features with minimal data and compute (Pisula et al., 8 Mar 2024 ).
• Diversity-aware Aggregation: The use of explicit global diversity modeling (DGR-MIL) ensures robust representation of heterogeneous bags, offering a new paradigm for MIL pooling (Zhu et al., 4 Jul 2024 ).
• Spatial Context Incorporation: Methods such as SAM-MIL utilize foundational segmentation models to embed explicit spatial neighborhood information, significantly improving global WSI classification (Fang et al., 25 Jul 2024 ).
• Standardization and Open Resources: Comprehensive resources with open code and pretrained weights (e.g., https://github.com/mahmoodlab/MIL-Lab) facilitate reproducibility and accelerate progress (Shao et al., 10 Jun 2025 ).
• Interpretability: New designs (SI-MIL) integrate self-interpretability into deep MIL frameworks for more transparent clinical AI (Kapse et al., 2023 ).

6. Comparative Performance and Model Selection

Data based on summarized empirical results from (Meseguer et al., 21 Oct 2024 , Wong et al., 2 Aug 2024 , Shao et al., 10 Jun 2025 , Zhang et al., 2023 , Qiu et al., 2023 , Zhu et al., 4 Jul 2024 ), and (Fang et al., 25 Jul 2024 ).

7. Implications for Research and Practice

• Pretrained MIL models, especially those using in-domain SSL or multimodal pretraining on large diverse datasets, should be considered default choices for weakly supervised learning where instance-level annotation is unavailable or impractical.
• Transfer learning paradigms—including those that pretrain on diverse pancancer datasets—enable efficient reuse of weakly supervised representations, support few-shot and rare disease applications, and improve reproducibility and resource efficiency in computational AI pipelines.
• MIL aggregation module pretraining and prompt-based tuning provide efficient adaptation pathways, especially when computational or annotation resources are limited.
• Interpretability, scalability, and reproducibility are enhanced by recent methodological and resource-sharing advances. This suggests future work will increasingly standardize on open-source, plug-and-play pretrained MIL modules, with a focus on interpretability, domain adaptation, and spatial context integration.

References

• https://github.com/amina01/ESMIL
• https://github.com/rajeswar18/MILe
• https://anonymous.4open.science/r/MIL-Feature-Extractor-Selection
• https://github.com/ChongQingNoSubway/DGR-MIL
• https://github.com/FangHeng/SAM-MIL
• https://github.com/mahmoodlab/MIL-Lab
• https://github.com/sotiraslab/SCMIL.git
• https://github.com/cvlab-stonybrook/PromptMIL
• https://github.com/liangyubuaa/Milmer