ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

Abstract: Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi-stage agglomerative distillation that sequentially distills eight vision-only, vision-language, and slide-level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile-level pathology images and 155,604 high-resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region-of-interest tissue analysis, vision-language multimodal evaluation, and whole-slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task-matched pathology foundation models. These results demonstrate that agglomerative distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at https://github.com/WonderLandxD/ALICE.

• The paper presents a multi-stage distillation framework that unifies morphology, language, and slide-level representations without performance trade-offs.
• It outperforms specialized models in ROI classification, VQA, and whole-slide analysis, achieving top rankings across 96 downstream tasks.
• The modular transformer design efficiently aggregates expert knowledge while mitigating catastrophic interference during sequential training.

ALICE: A Unified Multi-Stage Distillation Architecture for a General-Purpose Pathology Foundation Model

Motivation and Context

Recent advances in PFMs have diversified computational pathology, yet this proliferation also entrenches fragmentation by pretraining objective, source domain, and the spatial abstraction encoded in each expert. Models built for local morphological recognition (e.g., DINOv2-based UNI-2, Virchow-2, and H-Opt-1), vision-language alignment (e.g., CONCH, KEEP, MUSK), and slide-level, context-aware tasks (e.g., TITAN, CARE) deliver complementary but siloed strengths. However, clinical deployment and research increasingly require models that robustly unify fine-grained tissue discrimination, language-interpretable features, and optimal data efficiency across WSI-scale tasks.

The ALICE framework addresses this gap by introducing a multi-stage agglomerative distillation paradigm that hierarchically unifies expert capabilities into a single backbone, learning generalizable morphology-, language-, and context-aware representations (Figure 1).

Figure 1: Schematic of ALICE’s staged distillation: (a) Vision-only backbone, (b) Vision-language alignment, (c) Slide-level context aggregation, and (d)-(f) performance radar plots across multiple task regimes.

ALICE Architecture and Training Methodology

Modular Transformer Design

ALICE incorporates three architectural branches activated in sequential pretraining:

• Vision-only Transformer: ViT-H/14 backbone trained on $24.9$M image tiles for morphology.
• Multimodal Transformer: Adaptor layers for language-aligned distillation; only this module is optimized while the vision transformer is frozen.
• Slide-level Transformer: Extends from ROI to WSI by aggregating frozen patch features from high-resolution tiles ($\sim 155$k regions), encoding spatial order via coordinate augmentation and ALiBi self-attention (see Figure 2).

  Figure 2: Overview of the staged transformer modules—vision-only, multimodal, and slide-level—highlighting the progression from local to global, unimodal to multimodal, and patch to slide-level reasoning.

This progressive, modular approach ensures each expert’s knowledge is absorbed without catastrophic interference or objective misalignment.

Three-Stage Agglomerative Distillation Pipeline

Stage 1: Vision-Only Expert Distillation

ALICE first aligns with UNI-2, Virchow-2, and H-Opt-1 using separate summary tokens and balanced angular loss, facilitating both global and spatial morphology transfer without forcing conflicting objectives.

Stage 2: Multimodal Expert Distillation

Frozen image encoder outputs are processed by a two-layer transformer adaptor, which absorbs CONCH, MUSK, and KEEP’s language-aligned representations. Patch-wise distillation further enforces fine-grained multimodal consistency.

Stage 3: Slide-Level Distillation

Fixed tile features are aggregated by the slide-level transformer, aligning spatial context and clinical abstraction to TITAN and CARE representations. This enables robust transfer to diagnostic, biomarker, and survival tasks using minimal annotation.

Loss normalization and spatial localization (e.g., via angular and patch losses with grid-resolved cropping) mitigate teacher feature dispersion and ensure shift-equivariant distillation.

Evaluation Across Task Regimes

Vision-Only Transfer

ALICE outperformed vision-only PFMs across frozen embedding protocols for ROI classification (linear-probe/KNN), retrieval, and both semantic and instance segmentation (Figure 3, Figure 4). In few-shot ProtoNet evaluations, ALICE ranked first in average accuracy across all support set sizes and domains (Figure 5).

Figure 3: Macro-level benchmarking of ALICE on classical vision-only feature transfer, showing state-of-the-art generalization and robustness relative to strong vision baselines.

Multimodal and Vision-Language Transfer

In all vision-language downstream settings, including zero-shot classification, cross-modal retrieval, and VQA, ALICE achieved superior mean average rank and balanced accuracy (Figure 6, Figure 7).

Figure 6: ALICE’s generalization to multimodal, cross-modal, and zero-shot benchmarks, revealing a distinct transferability improvement over expert multimodal PFMs.

On PathPT few-shot tuning across WSI tasks and rare cancers, ALICE showed the highest few-shot accuracy and stability across increasing support size (Figure 8). Qualitative retrievals indicate precise alignment of morphologic evidence with diagnostic language (Figure 9, Figure 10), and WSI heatmaps localize predictions to visually consistent regions (Figure 11).

Slide-Level and WSI Analysis

For WSI-level transfer, ALICE achieved leading balanced accuracy across 19 diagnosis tasks under both KNN and linear-probe evaluation, overtaking TITAN and CARE on biomarker (AUC) and survival (C-index) prediction (Figure 12, Figure 13). Slide-to-slide retrieval exemplifies global context abstraction and robust pan-cancer feature structuring (Figure 14).

Figure 12: Mean ranking comparison for slide-level (WSI) diagnostics, patch-to-slide aggregation, and biomarker endpoints, where ALICE retains both fine-grained and global context generalization.

Strong Numerical Results and Notable Claims

• Consistently best average rank across 21 scenarios, 96 downstream tasks, and 48 sources for all three regimes (vision-only, multimodal, slide-level).
• Vision-only: best average rank with margins of +1.79% to +3.10% over next or mean best expert.
• Vision-language: surpasses all models by +6.39% (next-best) and +7.41% (mean).
• Slide-level: exceeds prior SOTA by +3.04% (next-best) and +4.00% (mean) in average performance.

ALICE outperformed or matched all expert PFMs on few-shot, zero-shot, and fine-tuning paradigms, and transferred expert strengths without sacrificing prior competencies—a critical, non-trivial claim that directly contradicts the expectation of catastrophic interference or task-specific degradation during aggressive knowledge aggregation.

Implications and Future Directions

The structured, staged distillation framework demonstrated by ALICE has profound implications for both research and deployment. Practically, it enables single-model coverage of the entire spectrum from fine tissue categorization to high-level clinical endpoints, minimizing model selection and system complexity in translational workflows. Theoretically, the multi-expert, multi-abstraction agglomeration shows that current artificial boundaries between morphological, semantic, and contextual PFMs are not fundamental but learned contingencies—these “lines” can be erased by properly-organized knowledge integration.

Key future directions include:

• Expansion to prospective, multi-institutional validation for robustness to data provenance, staining, and acquisition heterogeneity.
• Extension of the staged aggregation mechanisms to other modalities (e.g., immunohistochemistry, genomics).
• Optimization of the model architecture for joint efficiency, potentially collapsing the staged sequence into more computationally unified backbones or via continual learning regimes.
• Benchmarking for demographic/clinical bias [see also (2607.09526) and references therein] and rare-pathology generalization.

Conclusion

ALICE establishes that complementary expert capabilities in computational pathology can be effectively consolidated into a single, multi-task backbone without performance trade-off. The staged, modular agglomerative distillation pipeline yields both data-efficient learning and robust generalization—meeting the full demands of modern clinical, research, and AI4Health deployments. The release of the codebase and model weights sets a new baseline for unified, extensible PFMs, providing the technical substrate for continuous advancement toward clinically robust, scalable digital pathology systems.