WSI-Gene CoT-Enhanced Case Bank Construction

• The paper introduces a report-centric case bank that integrates WSI and gene data with refined chain-of-thought reasoning for survival prediction.
• The methodology employs magnification-aware patch mining and structured gene analyses to build a retrieval-ready and explainable memory.
• Empirical results demonstrate significant prognostic improvements, with combined case banks reaching a C-index of 0.713 over five TCGA cohorts.

WSI-Gene CoT-Enhanced Case Bank Construction is the first stage of SurvAgent, where raw whole-slide image data, genomic data, and survival supervision are transformed into a reusable multimodal memory for later retrieval-augmented survival inference. In this stage, a patient’s whole-slide image(s) $\mathcal{W}$, genomic data $\mathcal{G}$, and ground-truth survival time $t_{\text{GT}}$ are converted into summarized WSI and gene reports, refined chain-of-thought reasoning, and survival labels or times. The resulting memory is not merely a feature cache: it is described as a repository of “patient facts and reasoning traces,” “complete analytical reasoning process,” “summarized reports,” and survival outcomes, so that later inference can use both historical evidence and the reasoning trajectories that produced earlier prognostic judgments (Huang et al., 20 Nov 2025).

1. Conceptual role within the SurvAgent system

Within SurvAgent, WSI-Gene CoT-Enhanced Case Bank Construction is the mechanism that makes survival prediction both experiential and explainable. The broader system has two stages. Stage 1 constructs the case bank from historical cases; Stage 2 performs Dichotomy-Based Multi-Expert Agent Inference by regenerating WSI and gene reports for a test patient, retrieving similar historical cases via RAG, and using retrieved reports together with stored reasoning traces for interval refinement and final survival estimation. Stage 1 is therefore the memory-formation phase on which the later reasoning phase depends (Huang et al., 20 Nov 2025).

The stage is motivated by three deficits in earlier pathology-agent formulations: lack of multimodal integration, inefficient or incomplete WSI region exploration, and absence of experiential learning from historical prognostic cases. Its output is explicitly multimodal. At the modality-specific level, the WSI bank stores triplets of the form

$$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$

and the gene bank stores

$$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$

The appendix further describes a unified CoT Case Bank that stores reasoning at three levels: WSI-based analysis, gene-level analysis, and integrated WSI-gene reasoning. It also states that each case follows a standardized schema including assigned risk level, key evidence, and an explicit uncertainty statement (Huang et al., 20 Nov 2025).

This organization makes the stage more than a preprocessing pipeline. It is a reasoning-preserving memory construction process in which each stored case includes not only what was observed, but also how the system linked multimodal evidence to prognosis. A plausible implication is that the case bank is designed to support retrieval over semantically summarized evidence rather than over raw images or omics vectors alone.

2. Stored objects and case-bank schema

The case bank is described formally as a structured repository of multimodal reports, refined reasoning traces, and survival outcomes rather than as an explicit database schema. The minimal stored unit in the WSI branch is the summarized WSI report, refined WSI CoT, and survival time; the minimal stored unit in the gene branch is the summarized gene report, refined gene CoT, and survival time. The appendix enlarges this with risk level, key evidence, uncertainty statement, and integrated WSI-gene reasoning, but no JSON schema or SQL schema is provided (Huang et al., 20 Nov 2025).

This storage model is report-centric. On the pathology side, the bank holds a condensed summary of the hierarchical WSI analysis together with its reverse-generated survival-linked rationale. On the genomics side, it holds the output of the six-category gene-stratified analysis together with its own rationale. At the unified level, it preserves the cross-modal prognostic narrative that consolidates WSI and gene evidence. The paper repeatedly characterizes this as storing “complete analytical processes for experiential learning,” which is materially different from storing only embeddings or labels (Huang et al., 20 Nov 2025).

An important technical nuance is that the bank is built to be retrieval-ready. Stage 2 queries it using test-time WSI and gene reports, not raw image patches. This means the canonical objects in the bank are textualized and structured multimodal findings. A common misconception is that the case bank is primarily an image feature store; the paper’s formulation instead makes summarized reports and refined CoTs the primary reusable memory units.

3. Hierarchical pathology analysis and WSI-side CoT construction

The WSI side of the case bank is built by a strictly staged, magnification-aware pathology pipeline. It begins with Low-Magnification Screening (LMScreen) at $\times 2.5$, proceeds to Cross-Modal Similarity-Aware Patch Mining (CoSMining) at $\times 10$, and ends with Confidence-Aware Patch Mining (ConfMining) at $\times 20$. These stages are sequential rather than jointly optimized (Huang et al., 20 Nov 2025).

LMScreen provides a global tissue view and is defined as

$$\mathcal{R}_{\text{global}} = \mathcal{A}_{\text{wsi}}(\mathcal{W}_{2.5}),$$

where $\mathcal{W}_{2.5}$ is the downsampled slide and $\mathcal{G}$0 is PathAgent. CoSMining then partitions $\mathcal{G}$1 into $\mathcal{G}$2 non-overlapping patches $\mathcal{G}$3 and removes redundancy using both visual and text-report self-similarity. The visual similarity matrix is

$\mathcal{G}$4

and the retained visually diverse set is

$\mathcal{G}$5

In parallel, preliminary patch reports $\mathcal{G}$6 are embedded with $\mathcal{G}$7 to form

$\mathcal{G}$8

with semantically diverse patches retained as

$\mathcal{G}$9

The final $t_{\text{GT}}$0 set is their intersection,

$t_{\text{GT}}$1

The appendix specifies that image features for this stage are extracted with CHIEF, text embeddings use text-embedding-3-large, and the thresholds are $t_{\text{GT}}$2 (Huang et al., 20 Nov 2025).

ConfMining then escalates only low-confidence $t_{\text{GT}}$3 regions to $t_{\text{GT}}$4. PathAgent assigns each report a confidence category of low, medium, or high. For each low-confidence patch $t_{\text{GT}}$5, its high-resolution version is reprocessed by CoSMining:

$t_{\text{GT}}$6

and the final $t_{\text{GT}}$7 set is

$t_{\text{GT}}$8

The result is a cascade from whole-slide inspection to medium-magnification redundancy filtering to uncertainty-triggered high-magnification refinement (Huang et al., 20 Nov 2025).

The implementation details in the appendix make this cascade more concrete. Tissue regions are segmented with Otsu’s thresholding. The slide is initially tiled with non-overlapping $t_{\text{GT}}$9 patches at $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$0. CLAM is used at patch level 1 for tiling and background filtering, CHIEF provides patch-level cancer cell detection and attention scores, and DBSCAN aggregates high-risk patches into candidate ROIs using epsilon = 4 and minimum cluster size = 10. At $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$1 and $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$2, subregions are re-tiled using $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$3 windows. No stain normalization is mentioned (Huang et al., 20 Nov 2025).

PathAgent employs PathGen-LLaVA and Qwen2.5-32B-Instruct. It uses a predefined WSI attribute checklist $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$4, created by a Search Agent and pathologist review, to extract a structured pathology report:

$$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$5

The checklist contains 16 key attributes in the main text, and the appendix explicitly discusses fields such as tumor grade, depth of invasion, lymphovascular invasion, perineural invasion, lymph node metastasis, margin status, tumor morphology, carcinoma in situ, variant histology, squamous or glandular differentiation, micropapillary or plasmacytoid or sarcomatoid components, lymphocytic infiltration, necrosis percentage, and a free-text summary (Huang et al., 20 Nov 2025).

WSI-side CoT is then generated by reverse reasoning from the summarized report and ground-truth survival time:

$$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$6

Refinement is governed by a validation function $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$7 implemented with Qwen2.5-32B:

$$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$8

The text states that refinement proceeds until high quality is achieved. The stored artifact is thus an externalized, refined rationale rather than an unobservable latent reasoning state (Huang et al., 20 Nov 2025).

4. Gene-stratified analysis and gene-side CoT construction

The gene branch is designed to make abstract omics data interpretable and retrievable by organizing them into six prognostically meaningful functional categories: Tumor Suppressor Genes, Oncogenes, Protein Kinases, Cell Differentiation Markers, Transcription Factors, and Cytokines and Growth Factors. The paper’s stated rationale is that raw genomic features are too abstract and that individual genes often lack direct clinical value in isolation (Huang et al., 20 Nov 2025).

Let $$(\mathcal{R}_{\text{sum}}^{\text{WSI}}, \mathrm{CoT}_{\text{WSI}}^{\text{refined}}, t_{\text{GT}})$$9 denote the subset of genes in category $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$0, with $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$1 and $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$2. For each category, the system computes

$$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$3

where $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$4 is the mean expression, $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$5 is the median expression, and $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$6 is the mutation ratio. The appendix specifies that the genomic input is divided into DNA-level structural variation data (CNV), RNA-level expression data (RNA-seq), and other special genomic fragments. For RNA-seq, mean and median expressions are computed per gene class; for CNV, the mutation rate is described as the proportion of samples exhibiting mutations such as point mutations, insertions/deletions, or amplifications/deletions. The wording here is explicitly somewhat ambiguous for per-patient deployment (Huang et al., 20 Nov 2025).

These category statistics are passed to GenAgent $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$7, which uses a gene knowledge base $$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$8 built with mygene to select important genes:

$$(\mathcal{R}_{\text{sum}}^{\text{gene}}, \mathrm{CoT}_{\text{gene}}^{\text{refined}}, t_{\text{GT}}).$$9

and then produce type-specific reports:

$\times 2.5$0

The analysis is coarse-to-fine: category-level statistics, then class-level analysis, then identification of genes requiring additional inspection, then retrieval of raw expression values and functional annotations, then detailed category reports (Huang et al., 20 Nov 2025).

After all six categories are analyzed, GenAgent consolidates them into a comprehensive genomic summary report $\times 2.5$1. Gene-side CoT is then generated analogously to the WSI branch:

$\times 2.5$2

with refinement

$\times 2.5$3

The stored gene-side unit is therefore

$\times 2.5$4

A key technical point is that the gene branch produces structured, category-grounded summaries rather than a monolithic omics vector alone (Huang et al., 20 Nov 2025).

5. Cross-modal integration, retrieval readiness, and inference interface

Cross-modal integration during construction is conceptually strong but mathematically loose. Pathology and gene analyses are performed in parallel, linked by shared patient identity, shared survival supervision, shared storage in the case bank, and later multimodal retrieval. The explicit similarity functions in Stage 1 are only the image-image patch similarity $\times 2.5$5 and text-text report similarity $\times 2.5$6 used inside CoSMining. The paper does not define an explicit WSI-gene fusion operator or WSI-gene similarity function during case construction. A common misunderstanding is therefore to read “Cross-Modal Similarity-Aware Patch Mining” as direct pathology-genomics alignment; in the main text, it refers instead to image-feature similarity plus text-report similarity for pathology patches (Huang et al., 20 Nov 2025).

The integration mechanism is report-centric. Both modalities yield summarized reports and refined CoTs, and these are stored so that Stage 2 can retrieve similar historical cases “based on multimodal report similarity.” The retrieval operator is written as

$\times 2.5$7

The appendix states that retrieval is cosine-similarity-based retrieval over the previously constructed WSI–Gene CoT Case Bank, returning the top three most similar historical cases, so the only explicit retrieval hyperparameter is $\times 2.5$8 (Huang et al., 20 Nov 2025).

Stage 1 is also unusual in that it requires no additional training. The paper states that all results are obtained purely during inference. The pathology and genomics components are prompted and frozen; quality control is handled by self-critique rather than by gradient-based optimization. This means the case bank is constructed by a prompted, agentic analysis pipeline rather than by a learned end-to-end encoder. The practical implication is that case-bank entries are immediately RAG-compatible but inherit the strengths and weaknesses of prompt-based semantic variability (Huang et al., 20 Nov 2025).

The hierarchy of reasoning stored in the bank is best understood as

$\times 2.5$9

This suggests that the case bank is designed to preserve provenance from fine-grained evidence to survival-linked interpretation, even though the paper does not formalize this as a graph data structure.

6. Empirical support, implementation profile, limitations, and relation to adjacent work

The empirical case for Stage 1 is indirect but clear in the ablation results. A baseline without the WSI CoT bank, gene CoT bank, or inference stage achieves overall C-index 0.461. Adding only the WSI CoT bank raises overall C-index to 0.618. Adding only the Gene CoT bank raises it to 0.522. Using only the inference pipeline gives 0.689, and combining both case banks with inference reaches the best overall score of 0.713. These results support the claim that case-bank construction contributes substantial prognostic value, with the WSI-side bank contributing more strongly than the gene-side bank in this configuration (Huang et al., 20 Nov 2025).

The case bank is built from five TCGA cohorts under five-fold cross-validation: BLCA ($\times 10$0), BRCA ($\times 10$1), GBMLGG ($\times 10$2), LUAD ($\times 10$3), and UCEC ($\times 10$4). Implementation details in the appendix identify the supporting agent stack: DeepSeek-V3.2 with web access and a curated pathology knowledge base for the Search Agent; PathGen-LLaVA and Qwen2.5-32B-Instruct for PathAgent; Qwen2.5-32B-Instruct for GenAgent; CHIEF, CLAM, DBSCAN, and text-embedding-3-large in the pathology pipeline; $\times 10$5 for CoSMining; $\times 10$6 for retrieval; and a compute environment of $\times 10$7 NVIDIA RTX A6000 GPUs (48 GB each) plus an Intel Xeon Gold 6430 CPU (Huang et al., 20 Nov 2025).

Several limitations are explicit. Multimodal integration during construction is relatively loose because pathology and gene analyses are parallel and unified at the report and retrieval level rather than through a mathematically defined joint encoder. Confidence-aware escalation is prompt-based, with low/medium/high categories but no reliability analysis. Storing full CoT and historical reports raises unresolved issues of verbosity, consistency, and retrieval noise. The gene statistics $\times 10$8 are interpretable but may underrepresent higher-order pathway interactions. Uncertainty statements are stored, but there is no formal uncertainty propagation from patch confidence to retrieval or prognosis. Finally, the paper does not detail the indexing strategy or embedding choice for case-level multimodal retrieval beyond cosine similarity and top-three retrieval (Huang et al., 20 Nov 2025).

Within the broader literature, the stage occupies a distinctive position. Adjacent work has addressed complementary problems: dynamic slide representation under tile-scale constraints and report-aligned WSI encoding (Jin et al., 7 Nov 2025), compact multi-WSI case signatures for retrieval (Afzal et al., 22 May 2026), ST-supervised region-level WSI-gene alignment with spatial context (Qu et al., 30 Jun 2025, Ganguly et al., 2024, Qu et al., 2024), historical report-guided knowledge retrieval for pathology report generation (Zhang et al., 23 Jun 2025), and pathology-component or hard-instance organization for genetic biomarker prediction (Zhang et al., 26 Mar 2026). This suggests that WSI-Gene CoT-Enhanced Case Bank Construction is best viewed as a report-centric, reasoning-preserving multimodal memory layer that could be extended with tighter WSI-gene alignment, richer prototype structure, and more explicit retrieval-grounded reasoning.