HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images (1812.06499v5)

Abstract: Nuclear segmentation and classification within Haematoxylin & Eosin stained histology images is a fundamental prerequisite in the digital pathology work-flow. The development of automated methods for nuclear segmentation and classification enables the quantitative analysis of tens of thousands of nuclei within a whole-slide pathology image, opening up possibilities of further analysis of large-scale nuclear morphometry. However, automated nuclear segmentation and classification is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability such as the tumour cells. Additionally, some of the nuclei are often clustered together. To address these challenges, we present a novel convolutional neural network for simultaneous nuclear segmentation and classification that leverages the instance-rich information encoded within the vertical and horizontal distances of nuclear pixels to their centres of mass. These distances are then utilised to separate clustered nuclei, resulting in an accurate segmentation, particularly in areas with overlapping instances. Then for each segmented instance, the network predicts the type of nucleus via a devoted up-sampling branch. We demonstrate state-of-the-art performance compared to other methods on multiple independent multi-tissue histology image datasets. As part of this work, we introduce a new dataset of Haematoxylin & Eosin stained colorectal adenocarcinoma image tiles, containing 24,319 exhaustively annotated nuclei with associated class labels.

• The paper presents HoVer-Net, which innovatively uses horizontal and vertical distance maps to precisely separate overlapping nuclei in histology images.
• The paper employs a unique CNN architecture with dedicated NP, HoVer, and NC branches to achieve simultaneous nuclei segmentation and classification.
• The paper demonstrates robust performance across multiple datasets using metrics like DICE, AJI, and PQ, underscoring its potential in clinical pathology.

HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images

The robust analysis of nuclear features in histopathology images is paramount for digital pathology workflows aiming to automate and enhance diagnostic and prognostic processes. The paper "HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images" presents a deep learning-based solution to address the dual challenge of nuclear segmentation and classification within Haematoxylin and Eosin (H&E) stained histology images, which traditionally suffer from high intra- and inter-class variability and clustering of nuclei.

Methodology

The presented solution, HoVer-Net, employs a convolutional neural network (CNN) that leverages instance-aware predictions of the horizontal and vertical distances from nuclear pixels to their centers of mass. This novel approach is instrumental in effectively separating clustered nuclei and improving segmentation accuracy in overlapping regions. The network architecture integrates a modified Preact-ResNet50 encoder followed by three distinct up-sampling branches:

1. The Nuclear Pixel (NP) branch, tasked with distinguishing nuclear pixels from the background.
2. The HoVer branch, which predicts the horizontal and vertical distances to the nuclear centers.
3. The Nuclear Classification (NC) branch, which assigns each nucleus to a predefined nuclear type using up-sampling techniques.

Experimental Evaluation

The authors conducted extensive experiments on multiple independent histology image datasets, including the newly introduced CoNSeP dataset, which contains 24,319 annotated nuclei from colorectal adenocarcinoma (CRA) images. Comparative analyses were performed against other state-of-the-art methods, utilizing metrics such as DICE, Aggregated Jaccard Index (AJI), and Panoptic Quality (PQ) to quantify segmentation performance. The results demonstrated that HoVer-Net achieved superior performance across various datasets, showcasing its robustness and generalizability to different tissues and staining conditions.

Significant Contributions

• Horizontal and Vertical Distance Maps: The core innovation of the HoVer-Net lies in its use of horizontal and vertical distance maps (HoVer maps) to encode instance-specific information. This effectively addresses the challenge of overlapping nuclei, enabling precise segmentation in complex histological images.
• Simultaneous Segmentation and Classification: By integrating a classification branch, HoVer-Net not only detects and segments nuclei but also classifies them according to their cellular origin, which is critical for downstream analysis in computational pathology.
• Generalisation Capability: The model’s performance was rigorously evaluated across datasets collected from different medical centers and containing various tissue types. This highlights its potential for practical deployment in diverse clinical settings.

Implications and Future Directions

The practical implications of this research are far-reaching. HoVer-Net facilitates the quantitative analysis of nuclear morphometry on a large scale, which can significantly enhance the diagnostic accuracy and efficiency in digital pathology. The integration of segmentation and classification within a unified framework reduces the reliance on multiple disjoint models, simplifying the workflow and potentially reducing computational overhead.

Future research may focus on extending HoVer-Net to include additional tissue types and refining the classification capabilities to cover more granular nuclear subtypes. Another promising direction could be the incorporation of advanced post-processing techniques or leveraging semi-supervised and unsupervised learning paradigms to further enhance model generalization and performance, especially in scenarios with limited annotated data.

Conclusion

In sum, HoVer-Net represents a significant advancement in the computational pathology domain, providing an effective and efficient mechanism for simultaneous nuclear segmentation and classification in multi-tissue histology images. Through its novel architecture and robust performance, it lays a strong foundation for future developments aimed at enhancing automated analysis and improving clinical outcomes in pathology.