PathPT: Few-Shot Adaptation for Rare Cancer Pathology

• PathPT is a few-shot adaptation framework designed to convert coarse slide-level labels into fine-grained, interpretable tile pseudo-labels for rare-cancer pathology.
• It employs a spatially-aware visual aggregation module and task-adaptive prompt tuning with learnable tokens to merge local and global tissue context.
• Evaluated on eight rare cancer datasets with up to 56 subtypes, PathPT shows improved balanced accuracy and tumor region grounding, especially with the KEEP backbone.

Searching arXiv for papers on PathPT and closely related pathology VL/prompt-tuning work. arXiv search: "PathPT pathology prompt tuning rare cancer subtyping vision-language pathology" PathPT is a few-shot adaptation framework for pathology vision-language foundation models developed for rare cancer subtyping under weak whole-slide-image (WSI) supervision. It is designed to address two constraints that are explicit in rare-cancer pathology: data scarcity and the need for interpretable, region-specific predictions. The framework combines three mechanisms: conversion of coarse WSI-level labels into tile pseudo-labels using zero-shot vision-language grounding, a spatially-aware visual aggregation module that models both local and global tile context, and task-adaptive prompt tuning with learnable textual tokens aligned to histopathological semantics. In the reported study, PathPT is benchmarked on eight rare cancer datasets spanning 56 subtypes and 2,910 WSIs, together with three common cancer datasets, and is evaluated for both subtyping accuracy and cancerous-region grounding (He et al., 21 Aug 2025).

1. Clinical and methodological setting

Rare cancers comprise 20–25% of all malignancies and represent more than 70% of cases in pediatrics. In that regime, pathology AI is constrained not only by limited labeled data but also by the mismatch between slide-level supervision and the localized nature of diagnostically relevant tissue morphology. PathPT is formulated around that setting: the input is a WSI decomposed into tiles, supervision is initially available at the slide level, and the objective is to retain both subtype discrimination and spatial interpretability under few-shot conditions (He et al., 21 Aug 2025).

The framework is positioned against two families of approaches. The first is zero-shot use of pathology vision-LLMs, which offers initial grounding capability but remains insufficient for rare-cancer clinical performance. The second is conventional multi-instance learning (MIL), which aggregates only visual features and remains slide-level, thereby losing spatial granularity and the text encoder’s reasoning power. PathPT is explicitly proposed to close that gap by preserving cross-modal alignment while replacing coarse slide-only supervision with fine-grained tile-level guidance (He et al., 21 Aug 2025).

Operationally, PathPT uses pre-extracted, frozen tile embeddings from a pathology vision-LLM. Adaptation does not update the visual or text backbone parameters. Instead, training is restricted to the spatial aggregator parameters $\boldsymbol{\theta}_v$ and the prompt token embeddings $\boldsymbol{\theta}_t$. This design makes the method a parameter-efficient adapter rather than a full end-to-end finetuning scheme (He et al., 21 Aug 2025).

2. Core architecture

For a WSI $i$, each tile $x_{ij}$ is embedded as $\mathbf{v}_{ij}=\Phi_v(x_{ij})$, and the tile sequence is ordered by spatial coordinates. PathPT then applies a spatially-aware aggregation operator $\Psi$ to obtain context-enhanced tile features:

$$[\bar{\mathbf{v}}_1,\dots,\bar{\mathbf{v}}_M] = \Psi(\mathbf{v}_1,\dots,\mathbf{v}_M;\boldsymbol{\theta}_v).$$

The purpose of $\Psi$ is to inject both neighborhood-scale and slide-scale structure into representations that would otherwise remain independent tile embeddings (He et al., 21 Aug 2025).

The aggregation module has two stages. The local interaction block is a residual convolutional module with parallel kernels $3\times 3$, $5\times 5$, and $\boldsymbol{\theta}_t$0, and is used to capture short-range neighborhood tile features. The global interaction layer is a Transformer encoder layer with self-attention across all $\boldsymbol{\theta}_t$1 tiles,

$\boldsymbol{\theta}_t$2

so that long-range dependencies such as stromal patterns versus tumor islands can be modeled explicitly (He et al., 21 Aug 2025).

The textual side replaces fixed prompt templates with learnable context tokens. For $\boldsymbol{\theta}_t$3 subtypes, the class-$\boldsymbol{\theta}_t$4 prompt is

$\boldsymbol{\theta}_t$5

where each $\boldsymbol{\theta}_t$6 is learnable and initialized from manual prompts, while $\boldsymbol{\theta}_t$7 is the token embedding of the subtype name. The frozen text encoder $\boldsymbol{\theta}_t$8 maps each prompt to a vector in $\boldsymbol{\theta}_t$9. This preserves the original vision-language interface while specializing the prompt context to the downstream subtype taxonomy (He et al., 21 Aug 2025).

3. From WSI labels to tile-level supervision

A central design feature of PathPT is the conversion of WSI-level labels into tile-level supervision through a two-step zero-shot procedure. First, 200 prompt templates are generated, for example using forms such as “A histopathological image of {category}”. These prompt groups are evaluated by WSI-level zero-shot accuracy on the training set, the top 100 groups are retained, and their embeddings are mean-pooled to form unified text embeddings $i$0. This stage provides a task-specific initialization without supervised tile annotation (He et al., 21 Aug 2025).

Second, zero-shot tile labeling is performed. For each tile $i$1, the initial pseudo-label is computed as

$i$2

A tile is retained as “labeled” only if $i$3, where class $i$4 denotes normal tissue and $i$5 is the WSI subtype label. Tiles predicted as any other class are discarded from the labeled set and placed into the unlabeled set. The retained tiles form $i$6 and the remaining tiles form $i$7 (He et al., 21 Aug 2025).

This mechanism distinguishes PathPT from standard MIL. Rather than pooling all tiles directly into a slide representation, it uses zero-shot vision-language grounding to derive a partial fine-grained supervision signal that is already subtype-aware and region-aware. A common simplification is to describe PathPT as another MIL variant; that is inaccurate. The framework explicitly preserves tile-level supervision and text-conditioned semantic alignment instead of relying only on visual aggregation (He et al., 21 Aug 2025).

4. Learning objective and inference

Given a context-enhanced tile representation $i$8 and a prompt embedding for class $i$9, the tile-level class probability is defined by cosine-similarity scoring with temperature $x_{ij}$0:

$x_{ij}$1

The label space includes normal tissue as class $x_{ij}$2, so the model performs subtype discrimination and normal-versus-tumor separation within the same prompt-conditioned scoring rule (He et al., 21 Aug 2025).

Training uses three losses. The labeled-tile loss is cross-entropy over $x_{ij}$3:

$x_{ij}$4

For the unlabeled set $x_{ij}$5, the candidate loss constrains each tile in WSI $x_{ij}$6 to be either normal or the slide subtype:

$x_{ij}$7

From epoch 10 onward, hard pseudo-labels are generated for remaining tiles and optimized with a balanced cross-entropy pseudo-label loss. The total objective is

$x_{ij}$8

Only $x_{ij}$9 and $\mathbf{v}_{ij}=\Phi_v(x_{ij})$0 are updated by gradient descent (He et al., 21 Aug 2025).

At inference, PathPT computes tile features, applies spatial-aware aggregation, predicts tile-level class probabilities for $\mathbf{v}_{ij}=\Phi_v(x_{ij})$1, and assigns the WSI subtype as the subtype with the highest tumor-ratio. It can also produce tumor heatmaps, which are used as an interpretability output. This makes the inference pathway inherently dual-purpose: it yields a slide-level subtype decision while preserving localized spatial evidence (He et al., 21 Aug 2025).

5. Benchmarks and empirical results

The reported evaluation includes rare adult cancers, rare pediatric cancers, common cancers, and tumor segmentation benchmarks. The rare adult cohorts include EBRAINS with 30 subtypes and 898 WSIs together with TCGA-SARC, THYM, and UCS. The rare pediatric cohorts include Nephroblastoma, Hepatoblastoma, Medulloblastoma, and Neuroblastoma with 1,232 WSIs. The common-cancer cohorts include UBC-OCEAN with 5 subtypes and 175 WSIs together with TCGA-BRCA and TCGA-BRAIN. Segmentation benchmarks include CAMELYON16, PANDA, and AGGC22. Few-shot splits are 1-shot, 5-shot, and 10-shot per subtype, each repeated 10 times for variance estimation. Four frozen vision-language backbones are evaluated: PLIP, MUSK, CONCH, and KEEP. Baselines include zero-shot vision-language inference and four MIL frameworks: ABMIL, CLAM, TransMIL, and DGRMIL (He et al., 21 Aug 2025).

Subtyping is measured with balanced accuracy,

$\mathbf{v}_{ij}=\Phi_v(x_{ij})$2

and segmentation with AUC and DICE,

$\mathbf{v}_{ij}=\Phi_v(x_{ij})$3

On EBRAINS in the 10-shot setting, zero-shot performance is 0.111 with PLIP, 0.253 with MUSK, 0.204 with CONCH, and 0.408 with KEEP. TransMIL reports 0.488, 0.582, 0.621, and 0.648 across those backbones; DGRMIL reports 0.491, 0.569, 0.621, and 0.650; PathPT reports 0.251, 0.519, 0.491, and 0.679. On tumor region grounding for EBRAINS, zero-shot DICE is 0.207 and PathPT with KEEP reaches 0.560 in the 10-shot setting (He et al., 21 Aug 2025).

These numbers establish two points. First, the clearest quantitative gain on EBRAINS occurs with the KEEP backbone, where PathPT exceeds both TransMIL and DGRMIL in balanced accuracy and substantially improves region grounding. Second, the advantage is backbone-dependent rather than uniform across every setting: on the same EBRAINS table, TransMIL and DGRMIL remain above PathPT for PLIP, MUSK, and CONCH. The broader summary in the paper is that across pediatric and common cancer cohorts, PathPT-KEEP consistently matches or exceeds the best MIL, often by +5–20 percentage points, and increases DICE by +0.2–0.4 over zero-shot (He et al., 21 Aug 2025).

6. Interpretability, positioning, and implications

PathPT’s interpretability claim is grounded in its tile-level outputs. Because prediction remains defined over spatially arranged tiles, the model preserves subtype probability maps that allow localized inspection of cancerous regions. The learnable prompts function as adaptive text queries aligned with histopathological semantics, and the framework is described as enabling semantic alignment between text tokens and image regions rather than relying on opaque MIL attention alone (He et al., 21 Aug 2025).

Methodologically, PathPT occupies a specific position within pathology foundation-model adaptation. It does not discard the vision-language interface in favor of purely visual MIL, and it does not require full finetuning of the encoders. Its central hypothesis is that few-shot rare-cancer adaptation benefits from retaining cross-modal structure while injecting two missing ingredients: spatial context among tiles and task-adaptive textual conditioning. The empirical results on subtype classification and tumor grounding are consistent with that hypothesis, especially for KEEP-based configurations (He et al., 21 Aug 2025).

Several boundaries of the method are also explicit. PathPT depends on frozen backbone features, manual prompt initialization, and zero-shot tile pseudo-labeling before supervised refinement. This suggests that performance is coupled to the quality of initial vision-language grounding and prompt selection. It also means that PathPT should not be interpreted as a generic replacement for all MIL pipelines under all backbones; rather, it is a framework for exploiting pathology vision-LLMs more fully in few-shot rare-cancer settings by converting coarse supervision into spatially grounded, prompt-conditioned learning (He et al., 21 Aug 2025).