Pathologist-o3: Digital Pathology Navigation Agent

• Pathologist-o3 is a behavior-grounded pathology agent that learns diagnostic navigation directly from expert viewing logs rather than from text-only supervision.
• It employs a two-stage design with a YOLOv5-based Behavior Predictor and AI Session Recorder to convert routine inspections into actionable ROI commands.
• In gastrointestinal lymph-node metastasis detection, the system achieves 84.5% precision, 100.0% recall, and 75.4% accuracy, outperforming previous models.

Pathologist-o3 is a pathology-specific, two-stage, task-conditioned agent for whole-slide image diagnosis that learns from recorded expert navigation rather than from text-only supervision. It was introduced together with the AI Session Recorder and the Pathology-CoT dataset to capture both where to look and why it matters during routine pathologist review. In gastrointestinal lymph-node metastasis detection, the system achieved 84.5% precision, 100.0% recall, and 75.4% accuracy, exceeding the reported performance of OpenAI o3 on the same benchmark (Wang et al., 6 Oct 2025).

1. Definition and conceptual scope

Pathologist-o3 is grounded in a specific view of digital pathology: diagnosing a whole-slide image is an interactive, multi-stage process involving changes in magnification and movement between fields. In that formulation, a diagnostic agent must not only interpret image content but also decide which field to examine next, when to zoom, and how to connect localized findings to a case-level conclusion. The system is therefore framed as a pathology agent rather than as a static classifier or captioner (Wang et al., 6 Oct 2025).

This positioning distinguishes it from the dominant lineage of pathology foundation models, which are described as general-purpose feature learners pretrained on large pathology corpora and then adapted to tasks such as disease diagnosis, rare cancer diagnosis, survival prognosis prediction, biomarker expression prediction, and immunohistochemical scoring (Ochi et al., 2024). It also differs from earlier pathology VQA resources, which established question answering over pathology images but did not model whole-slide navigation as a supervised behavior (He et al., 2020).

A related empirical basis for this design comes from work on pathologist attention during prostate whole-slide reading. In that setting, specialists showed higher agreement in both their attention and cancer grades than general pathologists and residents, and expertise could be predicted from attention behavior alone (Chakraborty et al., 2024). This suggests that diagnostic performance is partly encoded in procedural viewing behavior, not only in final labels.

2. AI Session Recorder and behavioral formalization

The AI Session Recorder is the data-engineering component that converts routine whole-slide viewing into agent-readable supervision. It works with standard WSI viewers to unobtrusively record navigation logs and transform them into standardized behavioral commands and ROI boxes. In the reported study, logs were collected from 8 pathologists at Stanford Medicine analyzing 137 WSIs from 25 colorectal cancer lymph-node metastasis cases, with about 10.6 hours of total inspection time (Wang et al., 6 Oct 2025).

The raw logs were high frequency and noisy, averaging about 257 viewport events per slide with activity around 10 Hz. To make them usable, the recorder discretized continuous viewing into two main actions:

• <inspect>: broad, exploratory examination at low or medium magnification
• <peek>: brief, high-magnification look at fine cytologic detail

The heuristics were explicit. StayInspect was identified if the viewport remained static for more than 1 second. PanInspect was identified if continuous panning lasted more than 2 seconds. Very large low-magnification boxes were filtered out as non-diagnostic overviews. Overlapping actions with high IoU were merged. Containing or redundant larger actions were pruned in favor of smaller, more specific ones. Final action boxes were normalized to standard microscope-like magnification bins, so that, for example, a 12x view could become a \<10x-inspect> action (Wang et al., 6 Oct 2025).

Each behavioral command is paired with a standardized ROI bounding box. The result is a compact sequence of location-aware, magnification-aware actions that can supervise downstream models. The recorder is explicitly inspired by discrete objective lenses of a microscope, replacing arbitrary continuous zoom behavior with a structured action vocabulary.

3. Pathology-CoT as paired supervision for navigation and rationale

The Recorder feeds the Pathology-CoT dataset, which is the main supervision source for Pathologist-o3. Pathology-CoT contains 5,222 conversational rounds from 921 sessions derived from the 8 pathologists’ interactions. Each round includes a standardized behavioral command, an ROI bounding box, and an expert-validated rationale or impression. The dataset is therefore paired supervision for where to look and why it matters (Wang et al., 6 Oct 2025).

The rationale pipeline is human-in-the-loop. For each extracted ROI, a VLM drafts a rationale using the task, thumbnail, zoomed ROI, and sometimes verbal commentary. A pathologist then verifies, edits, or rejects the draft. The paper emphasizes that the resulting reasoning is expert pathologist-validated rather than purely LLM-derived. Annotation efficiency is reported in several ways: roughly 6-fold lower labeling time in the abstract; about 5–6× faster than typing and 3–4× faster than dictation in one timing analysis; and 8.6× faster than manual annotation from scratch elsewhere in the paper. It is also reported that over 80% of AI-generated text required no edits (Wang et al., 6 Oct 2025).

This dataset occupies a different position from earlier pathology-language corpora. PathVQA, for example, collected 32,799 question-answer pairs from 4,998 pathology images by extracting images and captions from textbooks and the PEIR digital library, then generating and manually checking question-answer pairs (He et al., 2020). Pathology-CoT instead records routine whole-slide navigation and links those trajectories to expert-validated rationales, thereby supervising procedural behavior rather than only image-question-answer correspondence.

4. Architecture and optimization

Pathologist-o3 is explicitly decoupled into two stages. The first stage is a task-specific Behavior Predictor trained on extracted ROI boxes from expert behavior. For the main study, this module is a lightweight YOLOv5-based detector. Its role is to predict diagnostically relevant ROIs on a new slide and to serve as the where to look component. The second stage performs behavior-guided reasoning: each proposed ROI is cropped at higher magnification, task context is injected into the prompt, a VLM reasons over the region, and region-level outputs are synthesized into a final slide-level summary and diagnosis (Wang et al., 6 Oct 2025).

A central design point is that the VLM is not asked to invent a navigation policy. ROI selection is handled by the learned Behavior Predictor, which lets the VLM focus on interpretation and synthesis. The paper presents this as a modular design intended for clinical deployability and future incremental personalization.

The optimization uses standard detection and classification losses. For bounding box regression, the paper gives the Complete IoU loss:

$$L_{CIoU} = 1 - IoU + \frac{\rho^2(b, b^{gt})}{c^2} + \alpha v$$

For classification, it uses binary cross-entropy:

$$L_{BCE} = -\frac{1}{N} \sum_{i=1}^{N} [y_i \log(\hat{y}_i) + (1-y_i) \log(1-\hat{y}_i)]$$

The paper also defines the evaluation metrics used in the main benchmark:

$$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$$

$\text{Precision} = \frac{TP}{TP + FP}$

$\text{Recall} = \frac{TP}{TP + FN}$

These definitions matter because the dataset is class-imbalanced, and the paper argues that precision and recall are more informative than accuracy alone in that setting (Wang et al., 6 Oct 2025).

5. Benchmark task and reported performance

The principal evaluation task is gastrointestinal lymph-node metastasis detection, specifically colorectal cancer lymph-node metastasis and N-staging. On that benchmark, Pathologist-o3 achieved the following results (Wang et al., 6 Oct 2025):

Setting	Precision	Recall	Accuracy
Pathologist-o3	84.5%	100.0%	75.4%
OpenAI o3	46.7%	87.5%	57.8%
External LNCO2 cohort	62.9%	97.6%	69.4%

The paper notes that the dataset is highly imbalanced, which explains why accuracy can appear lower than the precision-recall combination might suggest. It also reports that behavior-guided supervision generalized across VLM backbones, improving performance by an average of 11.8% precision and 17.9% recall. On the external validation cohort, the LNCO2 dataset, consisting of 1,245 WSIs from Sweden with domain shifts in scanner and magnification, the agent achieved 69.4% accuracy, 62.9% precision, and 97.6% recall (Wang et al., 6 Oct 2025).

Additional analysis was intended to separate navigation from analysis. Unguided VLMs were evaluated using completeness, defined as the fraction of expert-viewed regions recovered, and efficiency, defined as the fraction of selected regions that were clinically relevant. The paper reports that unguided VLMs performed poorly on both, whereas the behavior-guided agent concentrated on clinically salient regions and overlapped much better with expert trajectories. It also tested forward, reverse, and random ROI ordering and found only minimal performance changes, suggesting that once diagnostically useful regions are identified, the VLM can synthesize the evidence effectively (Wang et al., 6 Oct 2025).

6. Interpretation, workflow significance, and relation to adjacent systems

The broader claims made for Pathologist-o3 are that it is explainable, clinically aligned, upgradeable, practical, and scalable. Explainability comes from stepwise, region-by-region reasoning tied to actual viewed ROIs. Clinical alignment comes from learning the policy from expert pathologist behavior rather than from synthetic or generic LLM reasoning. Upgradeability comes from the modular separation between behavior policy and reasoning backbone. Practicality follows from passive data capture in normal workflow, and scalability from the fact that viewer integration makes data collection passive and reusable across tasks (Wang et al., 6 Oct 2025).

The system also sits within a broader shift in computational pathology from patch classification toward pathologist-like search and evidence selection. Earlier work on workflow integration, such as xPath, argued that adoption depends on matching how pathologists actually examine slides and on providing hierarchically traceable evidence rather than isolated predictions (Gu et al., 2020). Later systems pursued neighboring formulations: CARP3D treated 3D pathology as a triage problem that surfaces the highest-risk 2D slices for pathologist review rather than replacing expert judgment (Gao et al., 2024); HistoSelect introduced question-guided, tissue-aware, coarse-to-fine retrieval and reported a 70% average reduction in visual token usage while improving pathology QA accuracy (Huang et al., 28 Feb 2026); and PathAgent proposed a training-free Navigator–Perceptor–Executor framework with explicit reasoning trajectories over whole-slide images (Chen et al., 21 Nov 2025).

A common misconception is to treat results of this kind as evidence of general autonomous pathology. The reported benchmark for Pathologist-o3 is gastrointestinal lymph-node metastasis detection, not universal diagnostic pathology, and its central claim is that domain-specific behavioral supervision matters. That distinction is reinforced by a later pilot study of general-purpose autonomous multimodal agents for pathology report generation, in which the best-performing agent achieved 5.7% diagnostic accuracy with only the WSI, 28.6% with WSI plus signalment and organ, and 68.6% when given a morphologic description, while a human expert achieved 85.7% with only a single WSI (Aubreville et al., 30 Nov 2025). A plausible implication is that agentic pathology depends less on autonomy in the abstract than on high-fidelity supervision for navigation, evidence gathering, and pathology-specific reasoning.

In that sense, Pathologist-o3 is best understood as a behavior-grounded pathology agent: a system that treats expert navigation logs as a reusable source of procedural knowledge for whole-slide reasoning, and that recasts pathology AI from static image understanding toward supervised search, targeted inspection, and evidence-linked interpretation (Wang et al., 6 Oct 2025).