LRMIL: Efficient Low-Resolution Multiple Instance Learning via High-Resolution Knowledge Distillation for Whole Slide Image Classification

Abstract: Multiple instance learning (MIL) has become a standard paradigm for whole slide image (WSI) analysis in digital pathology, as it enables slide-level prediction without dense annotations. Existing MIL methods typically rely on exhaustive extraction and encoding of high-resolution patches. However, this practice suffers from two critical limitations in real-world clinical settings: it struggles to capture global visual cues at lower magnifications, and incurs substantial computational overhead due to the massive number of high-resolution patches per slide. To address these limitations, we propose an efficient low-resolution multiple instance learning (LRMIL) framework that transfers high-resolution knowledge to low-resolution representations. LRMIL adopts a two-stage distillation strategy. First, patch-level cross-resolution distillation aligns low-resolution patch embeddings with high-resolution representations. Second, slide-level knowledge distillation trains a low-resolution student MIL model under both slide-level supervision and teacher guidance. At inference time, LRMIL operates exclusively on low-resolution patches, substantially reducing data preprocessing and computational cost. Extensive experiments on multiple WSI benchmarks demonstrate that LRMIL consistently outperforms state-of-the-art MIL methods while achieving more efficient inference. These results highlight LRMIL as a practical and scalable solution for WSI analysis in clinical pathology.

• The paper introduces a two-stage, cross-resolution MIL framework that uses HR-to-LR knowledge distillation to significantly reduce computational costs while preserving diagnostic accuracy.
• It aligns patch-level and slide-level features through specific distillation techniques, achieving robust performance with improved accuracy and AUC across multiple datasets.
• LRMIL outperforms traditional MIL methods in both subtype classification and survival prediction, demonstrating enhanced efficiency and interpretability in whole slide image analysis.

Low-Resolution Multiple Instance Learning via Cross-Resolution Knowledge Distillation for WSI Classification

Introduction

The analysis of whole slide images (WSIs) is a central problem in computational pathology, where the image scale and scarcity of granular annotations mandate algorithmic efficiency and annotator independence. Multiple instance learning (MIL) constitutes the canonical paradigm, supporting slide-level classification by aggregating patchwise features. However, traditional MIL approaches are encumbered by the need to exhaustively encode large numbers of high-resolution (HR) patches, resulting in considerable computational cost and poor scalability.

This paper introduces LRMIL, a two-stage low-resolution MIL framework that leverages knowledge distillation to transfer high-resolution information to low-resolution (LR) representations. The method transfers HR semantic knowledge to a student model capable of inference using exclusively LR patches, substantially reducing computational overheads while achieving robust performance across diverse WSI classification and survival prediction benchmarks (2606.06864).

LRMIL Framework

LRMIL comprises a two-stage cross-resolution knowledge distillation pipeline: (1) patch-level distillation and (2) slide-level MIL distillation. The design optimizes the aggregation of fine-grained diagnostic features within globally coherent LR representations, supporting accurate, efficient inference without recourse to HR data at test time.

Figure 1: Diagram of the LRMIL pipeline: (a) patch-level cross-resolution distillation aligns LR features with HR embeddings; (b) slide-level distillation provides bag-level and instance-level guidance for the LR MIL student model.

Stage 1: Patch-Level Cross-Resolution Distillation

The initial stage aligns patch embeddings derived from matched LR and HR image regions. For each $256\times256$ LR patch (e.g., at $5\times$ magnification), the framework identifies a corresponding set of $K=16$ HR patches (e.g., at $20\times$ magnification) covering the same tissue domain. Both LR and HR encoders adopt a ViT backbone; the HR encoder is frozen, and the LR encoder is trained to match representations at several layers, minimizing an MSE loss over spatially aligned region tokens.

The design ensures that the LR encoder can encode both low-magnification context and fine-grained semantics, producing LR features with high diagnostic content. This stage is conceptually orthogonal to MIL aggregation and is not label-dependent.

Stage 2: Slide-Level MIL Distillation

In the second stage, the distilled LR encoder is deployed within a MIL architecture. The LR MIL head is trained with both bag-level supervision and teacher guidance from the HR-based MIL model. Guidance is enforced via:

• Bag-level distillation: Softened bag-level predictions from the teacher (HR) model are matched to those of the student (LR) model with a KL divergence loss.
• Instance-level distillation: Attention distributions are aligned via soft (KL on softmaxed attention) and hard (binary pseudo-labels for top-$k$/bottom-$k$ regions) losses, mapped between corresponding LR and aggregated HR instance sets.

Only the LR student model is utilized for inference, allowing end-to-end efficiency by obviating HR patch extraction and encoding.

Experimental Analysis

LRMIL is evaluated on diverse tasks: histopathologic and molecular subtype classification and survival prediction, spanning datasets such as TCGA-BRCA, TCGA-NSCLC, TCGA-RCC, and BRACS. The LR student utilizes the same encoder architecture (CONCH ViT) as the HR teacher for parity across baselines.

Numerical Results

LRMIL consistently outperforms state-of-the-art MIL methods, including full HR, selective cropping (SC), and distillation-based baselines, across all metrics (accuracy and AUC) and all tasks:

• Subtype Classification: On TCGA-RCC, LRMIL achieves an accuracy/AUC of 94.2/99.0, outperforming all methods while reducing total inference time by more than an order of magnitude versus exhaustive HR-based processing.
• Survival Prediction: On TCGA-BRCA, LRMIL attains accuracy/AUC of 90.0/92.0, with the fastest inference performance among all compared models.

Qualitative Analysis

Visualizations of attention heatmaps demonstrate that LRMIL's student model effectively localizes clinically relevant tumor regions, integrating both contextual and fine-grained information. Unlike LR-only models, which may mislocalize disease, the cross-resolution distilled model robustly highlights critical tissue areas.

Figure 2: Representative attention heatmaps on WSIs, illustrating superior localization of tumor regions by the LRMIL student model in comparison to LR-only and HR teacher models.

Ablation Analysis

Component ablations confirm the additive benefit of each distillation term in both patch and slide-level modules. Notably, soft instance-level alignment yields the largest gains, indicating that attention distribution matching provides crucial regularization beyond bag-level prediction alignment. Hard (top-$k$/bottom-$k$) instance supervision further enhances interpretability and discriminative ability without being sensitive to the choice of $k$.

Implications and Future Directions

This framework demonstrates that explicit cross-resolution distillation can close the performance gap between low- and high-magnification analysis in computational pathology, while offering drastic reductions in computational cost. Unlike selective cropping methods that retain dependence on HR patch encoding, LRMIL enables seamless, HR-independent analysis at test time, expanding its utility for clinical deployment and large-scale, real-time tasks.

Theoretically, this work establishes the viability of multi-scale knowledge transfer in medical image analysis and sets a foundation for more general cross-resolution or cross-modality distillation in MIL. Future developments could encompass regression tasks, multi-label scenarios, or integration with visual-language foundation models for broader diagnostic tasks.

Conclusion

LRMIL presents a principled, efficient approach to WSI MIL by decoupling training and inference resolutions via two-stage cross-resolution knowledge distillation. The method achieves strong empirical superiority in both predictive performance and computational efficiency. This framework constitutes a robust direction for scalable, resource-efficient pathology AI and serves as a template for further research in cross-resolution or multimodal knowledge transfer (2606.06864).