Pathological Concept Learning

• Pathological concept learning is a framework that models diagnostic concepts—from local image regions to literature-driven tokens—in an interpretable and modular manner.
• It leverages multimodal approaches such as self-supervised learning, MIL, and reinforcement learning to align image, text, and structured knowledge with clinical objectives.
• The method improves dense prediction, report generation, and decision-making by replacing opaque latent features with human-understandable, evidence-driven concepts.

Pathological concept learning denotes a family of methods that represent diagnostically meaningful pathology knowledge as learnable units rather than as undifferentiated latent features. Across recent work, a “concept” may be a local semantic or structural region in an image, a human-understandable pathology concept induced from text prompts or medical literature, a pathological clue derived from segmented regions, entities, or report themes, or a structured knowledge token activated from long-term memory into working memory (Yang et al., 2022, Sun et al., 6 Jan 2025, Zheng et al., 2024, Li et al., 10 Mar 2026). The objective is not only predictive accuracy but also preservation of morphological, molecular, and diagnostic semantics across dense prediction, whole-slide classification, virtual staining, report generation, visual question answering, survival analysis, and agentic diagnosis. This suggests a transition from early post hoc interpretation of deep features toward explicitly aligned, hierarchical, multimodal, and reasoning-oriented representations.

1. Concept definitions and representational scope

The literature does not use a single universal definition of a pathological concept. In self-supervised dense pre-training, a concept is “any local (semantic or structural) region in an image—not necessarily a whole object,” and the central problem is to construct concept masks without labels or external segmentation priors (Yang et al., 2022). In interpretable WSI classification, concepts are explicitly human-understandable pathology terms such as “disorganized cells” or “nuclear pleomorphism,” predicted patchwise through a pathology vision-LLM and aggregated into a slide-level decision (Sun et al., 6 Jan 2025). In concept-guided MIL, expert concepts are induced from medical literature and paired with learnable concepts so that slide representations can be aligned simultaneously to human prior knowledge and to training data regularities (Zhao et al., 2024). In cross-modal report generation, pathological clues are organized from segmented regions, pathological entities, and report themes (Zheng et al., 2024). In memory-centric pathology MLLMs, the relevant unit is a knowledge entity stored in long-term memory and selectively promoted into working memory for case-specific reasoning (Li et al., 10 Mar 2026).

Concept formulation	Operationalization	Representative work
Local semantic or structural region	Feature-map clustering, masks, activation regions	ConCL (Yang et al., 2022)
Human-understandable pathology concept	Vision-language concept scoring on patches	Concept MIL (Sun et al., 6 Jan 2025)
Literature- or text-induced expert concept	GPT-4 or expert text prompts aligned to slide embeddings	ConcepPath (Zhao et al., 2024), PathTree (Li et al., 2024)
Pathological clue	Segmentation, entity, and theme alignment across modalities	PCRL (Zheng et al., 2024)
Structured knowledge token	Long-term memory to working-memory activation	PathMem (Li et al., 10 Mar 2026)

A recurrent theme is that concept learning is not identical to class prediction. The learned units may serve dense transfer, bag aggregation, explanation, report generation, or reasoning control. A second theme is that concept granularity varies substantially: pixels, patches, regions of interest, slide-level semantic prototypes, diagnostic text nodes in a hierarchy, and knowledge graph entities all function as concepts in different frameworks.

2. Local morphology, dense prediction, and weakly supervised slide learning

An early line of work treated pathological concept learning as the emergence of interpretable morphology-sensitive features from patch classifiers. In breast lesion detection, activation features extracted from an AlexNet transfer-learning pipeline were shown to act as morphological descriptors for tumor cells, lymphocytes, collagen fibers, and other non-cell structural tissues; the most important feature subset combined recognizable and unrecognizable channels, indicating that discriminative machine features do not always coincide with canonical human descriptors (Hsu et al., 2019). This result framed a durable tension in the field: concept learning can recover established pathology rules, but it can also identify statistically useful signals that remain non-intuitive.

ConCL formalized local concept formation for self-supervised dense prediction pre-training. Its pipeline constructs a reference view from two augmentations, proposes concept masks, pools masked features, and contrasts corresponding concepts across views. The central operation is masked average pooling,

$$\operatorname{MAP}(z,m)=\frac{\sum_{i,j} m_{ij} z_{ij}}{\sum_{i,j} m_{ij}},$$

followed by a concept-level contrastive objective on shared concepts. ConCL benchmarked SSL for dense pathology tasks on GlaS and CRAG and reported that dependency-free bootstrapped concepts consistently outperformed prior SSL methods, with gains of +3 to +5 mAP over DenseCL and with 4–8x fewer pretraining epochs (Yang et al., 2022). The method’s importance lies less in a single architecture than in the argument that pathology requires fine-grained local concept discrimination rather than image-level instance discrimination alone.

Weak supervision over gigapixel WSIs motivated a second family of methods in which concepts are acquired by selective search and teacher guidance rather than dense labels. A deep reinforcement learning framework for rapid WSI diagnosis decomposed inference into a search agent and a decision agent. The search model, implemented with an LSTM and fully connected layers, mimics low-to-high magnification scanning; the decision model performs benign-versus-tumor prediction; and a ResNet34 multi-instance learning teacher model supplies rewards and soft labels through knowledge distillation. On 212 WSIs, the method achieved precision $0.950 \pm 0.034$, specificity $0.953 \pm 0.031$, F1 $0.927 \pm 0.009$, and accuracy $0.932 \pm 0.005$, while CPU runtime was about 20% of typical MIL approaches, approximately 723s versus approximately 3282s for MIL-ResNet34 (Zheng et al., 2022). Here, concept learning is tied to efficient region discovery: the model learns which fields of view are diagnostically meaningful under weak supervision.

Clinical priority also alters what should count as a useful concept. Priority-Aware Pathological Hierarchy Training for MIL encodes both vertical inter-hierarchy structure and horizontal intra-hierarchy priority. Coarse and fine predictions are aligned through Jensen-Shannon divergence,

$$\mathcal{L}_{IHA}=JS(\hat{p}^{\mathcal{H}=1}\Vert \dot{p}^{\mathcal{H}=1}),$$

and fine predictions are modulated by upper-hierarchy probabilities through a KL-divergence term. A feature remix mechanism constructs mixed bags so that higher-priority pathology must dominate the soft label. On 2,297 WSIs spanning 7 classes, the method reduced misdiagnosis, achieved the lowest type II error, and prioritized more important symptoms in multiclass scenarios (Hong et al., 28 Jul 2025). This reframes concept learning as clinically ordered representation learning rather than neutral pattern recognition.

3. Text-guided, hierarchical, and knowledge-grounded concepts

A major recent development is the replacement of implicit latent features with explicit text-aligned or literature-aligned pathology concepts. Label-free Concept MIL exemplifies this shift. It uses a dual-branch architecture: an image MIL branch identifies top-$K$ diagnostically relevant patches, and a concept MIL branch predicts slide labels from a linear combination of concept activations extracted with the CONCH vision-LLM. The contribution of concept $c$ is explicitly

$$\kappa_c = w_c \cdot \sum_{j=1}^{K} f_{jc} \cdot \beta_c,$$

so explanation is inherent rather than post hoc. On Camelyon16, the model achieved accuracy 0.898 and AUC 0.916; on PANDA, accuracy 0.927 and AUC 0.977. The top 20 patches fell within tumor regions in 87.1% of Camelyon16 cases and 85.3% of PANDA cases, and a user study found that roughly 2–3 of the top-5 concepts overlapped with pathologist selections (Sun et al., 6 Jan 2025). The work argues directly against the assumption that spatial heatmaps alone are adequate explanations.

PathTree extends concept learning into hierarchical WSI classification by associating every node of a binary diagnostic tree with a professional pathological text description. PLIP encodes patches and texts, a graph attention network contextualizes the text nodes, and text-guided tree aggregation recursively fuses child representations:

$$b_\gamma = s_\alpha b_\alpha + s_\beta b_\beta + i_\gamma,$$

with weights derived from similarity to the corresponding text concepts. Two additional losses, path alignment and tree-aware matching, constrain the final slide representation relative to ancestors, siblings, and leaf nodes. On SYSFL, PANDA, and BRACS, PathTree was consistently competitive; on SYSFL, PathTree-Ny achieved weighted F1 of 62.04% versus a previous best of 58.66% (Li et al., 2024). The model operationalizes the clinical intuition that diagnosis proceeds from broad to specific categories.

ConcepPath incorporates disease-specific expert concepts induced by GPT-4 from medical literature and complements them with purely learnable concepts from training data. Patch embeddings from QuiltNet or CONCH are aligned with these concept embeddings in a two-stage MIL framework: instance-to-concept aggregation and concept-to-bag aggregation, followed by slide-adapter fusion. A mutual distinctive loss encourages complementarity between different concept embeddings. In lung cancer subtyping, breast cancer HER2 scoring, and gastric cancer immunotherapy-sensitive subtyping, ConcepPath significantly outperformed previous methods lacking human expert knowledge; the reported gains include a 5.66% F1 improvement on BRCA HER2 and a 6.23% AUC improvement on STAD EBV vs Others (Zhao et al., 2024). An important methodological point is that the paper explicitly prefers literature-induced concepts over directly queried free-form LLM concepts.

Concept learning has also been tied to prognosis and fairness. In kidney cancer, pathological concepts were constructed from TNM staging guidelines and pathology reports, with report findings extracted using GPT-3.5. WSI patches were embedded by HIPT, UNI, CONCH, or CHIEF, converted into pathological graphs, and processed by a GNN with ABMIL heads to predict multi-label concepts via binary cross-entropy. The predicted concept logits then served as inputs to a Cox proportional hazards model,

$0.950 \pm 0.034$0

making risk attribution directly interpretable in clinical language. The top risk-increasing concept was “Invades pelvicaliceal system,” the top protective concept was “Tumor confined to kidney ≤4cm,” high- and low-risk groups separated with log-rank $0.950 \pm 0.034$1, and within each risk group there were no significant survival differences between male/female or white/black patients with log-rank $0.950 \pm 0.034$2 (Gao et al., 29 Sep 2025).

4. Molecular semantics, indirect measurement, and clue-driven multimodal alignment

Pathological concept learning is not restricted to H&E morphology. In polarization-based thyroid diagnosis, a three-stage framework integrates pixel-level and slice-level annotations to distinguish benign, malignant, and indeterminate tumors on unstained slides. Stage 1 uses confidence learning with XGBoost to denoise sparse pixel annotations and predict microscopic structure probability maps. Stage 2 uses a ResNet encoder-decoder to distill these pixel-level structures into 224×224 block embeddings. Stage 3 aggregates block embeddings with multi-class attention pooling,

$0.950 \pm 0.034$3

followed by logistic regression classification. The input includes Mueller matrix polarization parameters such as linear retardance, phase delay, degree of anisotropy, and circular birefringence. Reported ROC performance includes AUC 0.99 for malignant, 0.93 for benign, 0.86 for indeterminate, and micro-average AUC 0.93. The method also claims that, after color-coding, unstained slides approximate the quality of Hematoxylin and eosin staining and that indirect measurement through polarized features can classify samples where membrane structures cannot be obtained through sampling (Dong et al., 2024).

Virtual staining makes the molecular semantics problem explicit. PSPStain addresses H&E-to-IHC translation by enforcing both protein-level and prototype-level consistency. Protein-Aware Learning Strategy defines a Focal Optical Density map,

$0.950 \pm 0.034$4

with $0.950 \pm 0.034$5 and empirically best at $0.950 \pm 0.034$6, and uses Multi-Level Protein Awareness losses on average expression, histograms, and blocks. Prototype-Consistent Learning Strategy builds tumor and non-tumor prototypes from generated and real images and aligns them without assuming spatial registration. The model is evaluated with pathological relevance metrics mIOD, IOD, and Pearson-R, together with PSNR and SSIM, and reports the lowest deviation of mIOD and IOD from real IHC and the highest Pearson-R on BCI-HER2 and MIST-HER2 (Chen et al., 2024). The paper also states that PSNR and SSIM may not correlate perfectly with clinical or pathological quality, which is a significant caution for evaluation design.

Outside histopathology, analogous clue-driven formulations appear in report generation. PCRL for brain CT constructs pathological clues from segmented regions, pathological entities, and report themes. Segmentation clue alignment uses SAM-generated candidate masks filtered by area, stability, and IoU, with MedCLIP retrieval for entity-guided mask selection; entity clue alignment and theme clue alignment employ symmetric InfoNCE losses; and a unified LLM, such as LLaMA3, bridges representation learning and generation with task-tailored instructions. Only 0.04% of LLM parameters are updated via LoRA adapters. On CTRG-Brain, containing 6,000 samples and 160,336 CT images, PCRL achieved BLEU-1 62.0, BLEU-4 45.3, METEOR 33.1, and Clinical F1 70.6 (Zheng et al., 2024). Although the imaging modality differs, the underlying principle is shared: pathological concepts can be decomposed into region-, entity-, and theme-level clues that regularize cross-modal semantics.

5. Reasoning, memory, and agentic pathology systems

Pathological concept learning in question answering began from the problem of scarce, noisy, and diverse supervision. PathVQA introduced the first large-scale open pathology VQA dataset, with 32,795 QA pairs, 4,998 pathology images, an average of 6.6 questions per image, and eight question categories. To handle noisy training examples, the paper proposed learning-by-ignoring through a bi-level optimization,

$0.950 \pm 0.034$7

and complemented this with cross-modal self-supervised learning based on image-question and image-answer matching. The best reported result, Transformer+CMSSL+LBI, reached accuracy 60.1, BLEU-1 59.9, BLEU-3 3.2, and F1 12.2 (He et al., 2020). This formulation treated concept learning as robust multimodal representation learning under label noise.

Recent pathology VLMs push the same problem into explicit reasoning supervision. PathVLM-R1 starts from Qwen2.5-VL-7B-Instruct, performs supervised fine-tuning on 3,000 pathology image–question–answer samples, and then applies Group Relative Policy Optimization with a dual reward composed of an outcome accuracy reward and a cross-modal process reward. The GRPO relative advantage is

$0.950 \pm 0.034$8

Accuracy progressed from 51.55% in the base model to 57.59% after supervised fine-tuning, 63.89% after outcome-only RL, and 65.5% in the final model with process reward; the paper also reports a 14% improvement in pathology QA accuracy over baseline methods and an average 17.3% improvement over traditional SFT on out-domain CT, dermoscopy, fundus, and OCT evaluation (Wu et al., 12 Apr 2025). Here, a concept is not just a visual feature but a reasoning step whose plausibility is explicitly rewarded.

Patho-R1 scales this logic further. It uses continued pretraining on 3.5 million image-text pairs, supervised fine-tuning on 500k Chain-of-Thought samples, and reinforcement learning on 10k diagnostic-oriented MCQs using GRPO and Decoupled Clip and Dynamic sAmpling Policy Optimization. The data are stratified across histopathology, gross, immunohistochemistry, cytology, and FISH, with three difficulty levels and four downstream task types, yielding 60 reasoning task types. Patho-R1-7B achieved 69.53% on PathMMU-test-tiny and 62.98% on PathMMU-val, exceeding prior baselines reported in the paper, while PathoCLIP-L reached mean zero-shot classification accuracy 76.14% and strong cross-modal retrieval performance (Zhang et al., 16 May 2025). The framework treats concept learning as the combination of knowledge infusion, structured diagnostic chains, and RL-based reasoning refinement.

Memory-centric models make the knowledge structure itself explicit. PathMem builds a pathology knowledge graph from PubMed literature as long-term memory,

$0.950 \pm 0.034$9

embeds all entities into a memory bank $0.953 \pm 0.031$0, and performs static plus dynamic activation to form working memory via

$0.953 \pm 0.031$1

The selected memory tokens are prepended to the input so that self-attention operates jointly over case context and activated knowledge. On WSI-Bench, PathMem improved report generation by 12.8% WSI-Precision and 10.1% WSI-Relevance, and improved open-ended diagnosis by 9.7% and 8.9% over prior WSI-based models (Li et al., 10 Mar 2026). The model’s central claim is that pathology reasoning requires explicit handling of diagnostic criteria, grading systems, and disease-feature-evidence relations rather than purely parametric recall.

Behavior-grounded and evidence-seeking systems extend concept learning from static reasoning to active diagnosis. Pathology-CoT records pathologist navigation behavior through the AI Session Recorder, which converts viewer logs into discrete <inspect> and <peek> actions with ROI boxes and pairs them with human-verified rationales. The dataset comprises 10.6 hours of logs from 8 pathologists across 137 WSIs and 5,222 conversational rounds from 921 sessions, and the review-over-authoring workflow reduces labeling time by 6–8×. The resulting Pathologist-o3 agent achieved 84.5% precision, 100.0% recall, and 75.4% accuracy on gastrointestinal lymph-node metastasis detection (Wang et al., 6 Oct 2025). PathFound generalizes the same clinical logic into a modular evidence-seeking system composed of a Slide Highlighter, a Vision Interpreter, and a Diagnostic Reasoner trained with reinforcement learning and verifiable rewards; its exploration–execution–exploitation protocol consistently improves diagnostic accuracy over static one-pass inference and achieves state-of-the-art results across diverse clinical scenarios (Hua et al., 29 Dec 2025). A plausible implication is that concept learning is becoming inseparable from policies for deciding what evidence to acquire next.

6. Interpretability, evaluation, and continuing challenges

A central misconception is that pathological concept learning is equivalent to saliency visualization. Multiple studies argue otherwise. Concept MIL explicitly states that traditional MIL explanations based on salient regions provide limited insights for end users, because a faithful explanation should also expose the human concepts driving the linear decision function (Sun et al., 6 Jan 2025). ConcepPath similarly moves beyond standard MIL attention maps by aligning patches to explicit expert and learnable concepts, enabling concept heatmaps rather than generic relevance maps (Zhao et al., 2024). Earlier feature-visualization work had already shown why this distinction matters: some of the most discriminative channels are unrecognizable to experts even when they improve classification (Hsu et al., 2019).

Another challenge concerns what should be measured. Pathology tasks increasingly require pathology-specific metrics rather than generic image or language scores. PSPStain uses mIOD, IOD, and Pearson-R because PSNR and SSIM may not correlate perfectly with pathological quality (Chen et al., 2024). PCRL reports Clinical F1 over 24 pathology keywords alongside BLEU, METEOR, ROUGE-L, and CIDEr (Zheng et al., 2024). PathMem evaluates report generation with WSI-Precision and WSI-Relevance, not only BLEU or ROUGE (Li et al., 10 Mar 2026). PathTree uses hierarchical metrics such as H-Precision and H-F1 in addition to conventional class metrics (Li et al., 2024). This suggests that concept learning changes not only models but also the ontology of evaluation.

Scalability and maintenance remain open engineering issues. CAMP addresses these directly through a generic, unified, and universal framework for pathology image classification that can continuously adapt to new tasks without forgetting existing ones. It was evaluated on 22 datasets, including 1,171,526 patches and 11,811 pathology slides, across 17 classification tasks, and reported state-of-the-art performance while reducing up to 94% of computation time and 85% of storage memory compared with conventional classification models (Nguyen et al., 2024). In parallel, PathMem emphasizes updatable literature-driven long-term memory, and Pathology-CoT emphasizes passive collection of expert behavior from routine viewer use (Li et al., 10 Mar 2026, Wang et al., 6 Oct 2025). These lines of work indicate that the bottleneck is no longer only model capacity, but also how pathological concepts are refreshed, audited, and transferred across tasks.

The field therefore spans several distinct but convergent agendas: discovering local morphology-sensitive units, aligning slides to expert language and hierarchy, preserving molecular semantics, grounding decisions in explicit knowledge, and learning evidence-seeking diagnostic behavior. A plausible implication is that future pathology systems will combine concept bottlenecks, hierarchical text supervision, memory activation, and agentic evidence acquisition within a single pipeline. The current literature already shows that pathological concept learning is not a narrow subroutine for interpretability; it is becoming a general design principle for computational pathology models that must be accurate, clinically aligned, and auditable.