CanvOI, an Oncology Intelligence Foundation Model: Scaling FLOPS Differently (2409.02885v1)

Abstract: The rapidly evolving field of digital oncopathology faces significant challenges, including the need to address diverse and complex clinical questions, often involving rare conditions, with limited availability of labeled data. These limitations hinder the development of robust AI-driven tools in the biomedical space, where accuracy in probabilistic determinations is of utmost importance. To address this, digital pathology foundation models have begun to emerge, typically developed with the size and diversity of the pre-training dataset and model parameters in mind. Here, we present CanvOI, a ViT-g/10-based foundation model designed to enhance the capabilities of digital pathology by addressing these challenges through a different approach. Considering the unique nature of oncologic histopathological images and the requirements from the embeddings to provide meaningful representations for Multiple Instance Learning (MIL) downstream models, we chose to modify the input image characteristics. By introducing larger tile sizes (380 x 380 pixels) and smaller patch sizes (10 x 10 pixels), we were able to optimize the model's performance, pushing computational resources in a new direction and achieving state-of-the-art performance on cancer-related benchmarks. CanvOI demonstrated a 1.5-7.4% improvement in averaged AUC compared to other leading foundation models built for digital pathology. Moreover, our results demonstrate that CanvOI significantly outperformed the other models, with the performance gap widening substantially when trained on just 10% of the initial cohort. This work highlights an alternative approach that, if integrated with traditional development approaches, has the potential to advance Oncology Intelligence (OI), overcome some of the current barriers and ultimately improve the clinical outcome of cancer patients.

CanvOI, an Oncology Intelligence Foundation Model: Scaling FLOPS Differently

The paper "CanvOI: Scaling FLOPS Differently" introduces a novel foundation model designed for digital oncopathology. CanvOI is built on the Vision Transformer (ViT-g) architecture, utilizing DINOv2 framework, specifically tailored for processing oncological histopathological images. The model addresses critical challenges in digital pathology including the complexity of clinical questions and limited availability of labeled data. This summary aims to provide an advanced overview of the paper, discussing the key methods, results, and implications of this research.

Key Innovations and Methods

The authors introduce CanvOI as a 1.1 billion parameter model focusing on histopathological slide-level predictions. Unlike traditional approaches using smaller tiles and patch sizes, CanvOI uses larger tile sizes of 380² pixels with smaller patch sizes of 10² pixels. This setup aims to optimize the model's capacity to generate meaningful embeddings suitable for multiple instance learning (MIL) downstream tasks.

Pre-training was conducted on a substantial dataset comprising 632,608 tissue samples and 70,217,688 tiles extracted from diverse sources, with a focus on hematoxylin and eosin (H&E)-stained tissue images. The tissue samples originated from over 100 international sites, encompassing more than 40 major organs and tissue types.

Performance Evaluation

The evaluation of CanvOI involved several benchmark tests on breast, colorectal, and lung cancers, highlighting its superior performance as compared to other pre-trained foundation models such as H-optimus-0, Prov-GigaPath, Virchow, and Hibou-L. The specific tasks included:

• Breast Tissue Histological Classification (BRACS Dataset)
  • CanvOI demonstrated an improved averaged AUC of 1.5-7.4% compared to other models on both lesion type classification and a finer-grained subclassification task.
• Colorectal Tissue Histological Classification (HunCRC Dataset)
  • The model continued to show superior performance in distinguishing between normal, non-neoplastic, adenoma, and colorectal cancer tissues.
• Internal Benchmark on Non-Small Cell Lung Carcinoma (NSCLC) Classification
  • CanvOI outperformed its counterparts in classifying biopsy sites (primary vs. metastatic) and histological subtypes (LUAD vs. LUSC).

Numerical Results and Dependency on Labeled Data

A notable finding is CanvOI's robustness when trained on limited labeled data. The authors performed experiments where the dataset size was reduced to as low as 10% of the original, demonstrating that CanvOI maintained top-tier performance with an AUC of 0.83. This is particularly significant given the typical constraints in clinical settings, where large volumes of annotated data are seldom available.

Discussion and Implications

CanvOI's approach to increasing tile size while reducing patch size proves beneficial in generating detailed and informative embeddings without requiring an expansion in model size. This method can be instrumental in developing better MIL models for histopathology. The paper makes a compelling case for optimizing compute resources and balancing tile and patch sizes against model parameters to ensure training stability and effective embedding generation.

The implications of this research are manifold. Practically, CanvOI can enhance diagnostic accuracy and efficiency in clinical oncology through reliable histopathological slide classification. Theoretically, the model sets a new benchmark in the foundation model domain, suggesting an alternative path for future digital pathology models. Further refinements and extensions of this methodology could expedite advancements in medical diagnostics, fostering improved patient outcomes and accelerating discovery in oncology research.

Future Directions

The research opens avenues for exploring the balance between tile and patch sizes and their impact on the performance of aggregation models. Investigating how different embeddings influence model stability and efficacy can refine model training processes. Moreover, integrating CanvOI's approach with ongoing efforts could streamline the scalability of foundational models, ensuring they continue to provide robust and meaningful outputs even as data complexity and volume grow.

In summary, CanvOI represents a significant step towards addressing the computational and data limitations in digital oncopathology. By leveraging larger tiles and smaller patches, the model achieves remarkable performance improvements, potentially transforming clinical diagnostics and research methodologies in oncology.