Enhancing Pathological VLMs with Cross-scale Reasoning

Abstract: Pathological images are inherently multi-scale, requiring pathologists to integrate evidence from global tissue architecture at low magnification to cellular morphology at higher magnification for accurate diagnosis. While existing pathological datasets for vision-LLM (VLM) include various scales, they often lack an explicit cross-scale reasoning objective. This limitation prevents VLMs from capturing essential cross-scale representations and learning evidence-based reasoning. To bridge this gap, we introduce the first cross-scale training and evaluation paradigm that formulates pathology interpretation as multi-magnification reasoning. However, creating such a task reveals a critical challenge: multi-image visual question answering (VQA) is prone to text-only shortcuts, which allow models to guess answers using magnification-dependent artifacts rather than visual evidence. To address this, we propose a leakage-aware curation pipeline that combines adversarial text-only screening with constraint-guided question design. Using this pipeline, we construct Scale-VQA, a high-quality benchmark with 4,685 multiple-choice questions grounded in 2,537 pathology images across multiple magnification levels. Finally, we present ScaleReasoner-R1, a model trained via reinforcement learning to optimize performance on the cross-scale VQA task. ScaleReasoner-R1 achieves state-of-the-art performance on our cross-scale reasoning benchmark and generalizes to SOTA performance on established single-scale benchmarks. Findings suggest that even the limited cross-scale supervision can significantly improve pathological understanding. The code and demos will be open-sourced.

• The paper introduces a novel cross-scale reasoning framework that integrates multi-magnification evidence using GRPO-based reinforcement learning.
• The methodology utilizes a leakage-aware VQA benchmark and clinical annotations to mitigate shortcut learning and improve evidence synthesis.
• Key experiments achieve an average accuracy of 82.89% on cross-scale tasks and competitive results on single-scale benchmarks, nearing expert-level performance.

Enhancing Pathological VLMs with Cross-scale Reasoning: A Technical Evaluation

Introduction

The paper "Enhancing Pathological VLMs with Cross-scale Reasoning" (2606.17412) introduces a new paradigm for vision-LLMs (VLMs) in computational pathology by formalizing and solving multi-magnification reasoning in diagnostic tasks. The work confronts a critical limitation of extant pathological VLMs, namely their single-scale focus, and instead implements supervised reinforcement learning (RL) on meticulously curated, leakage-aware, cross-scale visual question answering (VQA) data. This approach aligns computational models more closely with clinical workflows, where pathologists integrate morphological findings across several magnification orders. The paper’s model, trained via Group Relative Policy Optimization (GRPO), demonstrates substantial state-of-the-art gains on both complex cross-scale and standard single-scale pathology VQA benchmarks.

Limitations of Single-scale Pathological VLMs

Current pathology-focused VLMs predominantly operate on single-image, single-magnification data, which diverges from the hierarchical, evidence-integration strategy of human pathologists. Most available datasets segment regions of interest (ROIs) independently at one scale, thus omitting dependencies between global tissue architecture and localized cellular features. The absence of explicit cross-scale reasoning supervision allows models to exploit magnification-dependent textual artifacts ("shortcut solutions") rather than performing evidence-based inference.

Figure 1: The contrast between (a) single-scale VQA, (b) naïve cross-scale VQA (prone to text-only shortcuts), and (c) the proposed leakage-aware cross-scale VQA.

Leakage-aware Cross-scale VQA Benchmark Construction

The paper presents a principled pipeline for collecting high-fidelity cross-scale VQA data that robustly counters textual shortcut exploitation. Its steps include:

1. Clinical Annotation: Independent ROI selection across 10$\times$, 40$\times$, and 200$\times$ magnifications on >100 WSIs, followed by generation of hierarchical, clinically-validated captions and zoom-in rationales reflecting the pathologist's multi-scale search strategy.
2. Leakage-aware Curation: Guided by evidence-dimension constraints (Correspondence, Confirmation, Localization, Explanation, Diagnosis), GPT-5.2 is used to generate candidate MCQs, which are then rigorously screened using text-only adversaries (Gemini 3 Pro, Qwen3-Max). Any MCQ resolvable without image inputs is revised or discarded.
3. Expert Validation: Pathologists validate all final MCQs, ensuring ground truth is visually anchored and distractors are clinically plausible.

The resulting dataset (\dataset) comprises 4,685 high-quality MCQs across 15 organs and 5 reasoning dimensions—a new benchmark that necessitates robust cross-magnification evidence synthesis.

Figure 2: Overview of the dataset and model: (a) leakage-aware curation pipeline, (b) dataset composition, (c) GRPO-based RL training, (d) cross-scale reasoning results, and (e) example multi-magnification VQA.

Cross-scale Reasoning via Outcome-driven RL

To move beyond annotation-style imitation, the proposed model (\model) is trained using GRPO RL on the constructed dataset. Given a triple of multi-scale images, the policy outputs structured reasoning and selects an answer, with rewards for correctness and output validity. This RL approach provides outcome-aligned supervision, encouraging effective evidence integration across scales, in contrast to supervised fine-tuning (SFT) which tends to overfit to surface rationales.

Experimental Results: Cross-scale and Single-scale Evaluation

The model outperforms a spectrum of general, medical, and pathology VLMs on the cross-scale VQA benchmark, achieving an average accuracy of 82.89%. Notably, it exhibits strong generalization: when transferred to the large-scale, single-image PathMMU benchmark, it matches or surpasses previous pathology SOTA models, even approaching or exceeding expert-level accuracy on several subsets. An ablation analysis reveals that pure RL is unequivocally superior to SFT-only or hybrid SFT+RL setups, aligning with broader findings on the merits of outcome-centric training for generalization.

Implications and Future Directions

This work demonstrates that explicitly encoding clinical multi-scale interpretation and enforcing rigorous shortcut-suppression in VQA benchmarking can yield higher performance and more generalizable VLMs for pathology. This methodological advance has dual implications:

• Practical: Enhanced cross-scale VLMs can improve robustness in clinical workflows, offering more interpretable and accurate computer-aided diagnostics, especially in settings demanding evidence integration (e.g., tumor invasion assessment).
• Theoretical: Outcome-driven RL emerges as a potent paradigm for high-level clinical reasoning, minimizing annotation inefficiency and overfitting, which could generalize to other multi-scale or hierarchical decision tasks in medical AI.

The public release of both the leakage-aware curation pipeline and the benchmark facilitates further methodological development and fair comparison across the research community. Future trajectories should pursue scaling dataset size across more organs, unsupervised cross-scale signal discovery, and neuro-symbolic approaches to further enhance interpretability and causal reasoning.

Conclusion

This paper establishes a rigorous, clinically-aligned framework for cross-scale reasoning in pathological VLMs, combining leakage-aware data curation and outcome-optimized RL. Its empirical results highlight the significance of faithful, multi-magnification supervision for both practical deployment and advancing the theoretical foundations of medical vision-language reasoning.