Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology (1902.06543v2)

Abstract: Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.

• The paper demonstrates that applying stain color augmentation, particularly using HSV and HED methods, significantly boosts CNN performance across diverse pathology datasets.
• It finds that deep learning-based normalization methods perform comparably to conventional techniques, though their added complexity offers only marginal improvements.
• Effective augmentation mitigates inter-laboratory staining variability, providing actionable insights for designing robust CNN models for computational pathology.

Analysis of Data Augmentation and Stain Color Normalization in CNNs for Computational Pathology

The paper "Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology" presents an empirical paper focused on addressing the variability in histopathological image analysis caused by stain color discrepancies across different laboratories. The research evaluates the performance impact of various data augmentation and stain color normalization techniques on convolutional neural networks (CNNs) utilized in computational pathology.

Methodology

This work explores two major strategies to enhance CNN generalization across different data sources: stain color augmentation and stain color normalization. Stain color augmentation aims to improve model robustness by introducing diverse color variations during training, while stain color normalization endeavors to align the color distribution of training and test images.

Stain Color Augmentation Techniques:

1. Basic: Involves image rotation and mirroring, aiming to introduce minimal morphological variation.
2. Morphology: Extends basic techniques with more detailed morphological transformations, such as Gaussian noise and blurring.
3. Brightness and Contrast (BC): Adds perturbations to image brightness and contrast.
4. Hue-Saturation-Value (HSV) and Hematoxylin-Eosin-DAB (HED): Tailored color transformations specific to common histological stains.

Stain Color Normalization Techniques:

1. Identity: No transformation applied, serving as the baseline.
2. Grayscale: Converts images to grayscale, removing color information.
3. Deconv-based: Employs color deconvolution to separate staining components.
4. LUT-based: Utilizes look-up tables for color histogram matching.
5. Style and Network-based: Leverage deep learning techniques and neural networks for color normalization, regarded as a style transfer problem.

Experimental Evaluation

The paper systematically evaluates classifier performance using CNNs trained on normalized images and augmented datasets across several multi-center applications, such as mitosis and tumor metastasis detection. The experiments are conducted with data from nine pathology centers, critically assessing the efficacy of each technique.

The performance was measured using the area under the receiver-operating characteristic curve (AUC) on external test sets. It was observed that:

• Stain color augmentation was crucial for enhancing CNN generalization, with both HSV and HED augmentations yielding similar performance gains.
• Neural network-based normalization methods (Network and Style) performed comparably, indicating the potential of using deep learning for robust stain normalization.
• Surprisingly, color normalization was not essential to achieve top performance, suggesting that augmentation strategies can sufficiently mitigate the challenges posed by stain variability.

Implications and Future Directions

This research reveals that effectively implemented stain color augmentation substantially impacts model robustness across heterogeneous datasets, making it a pivotal consideration for developing CNN-based computational pathology tools. While color normalization can complement augmentation methods, its computational overhead and marginal performance benefit necessitate a careful trade-off analysis in practical applications.

The paper's findings encourage further exploration into efficient, low-cost augmentation strategies and the development of adaptive learning techniques that can dynamically adjust to color variance in real-time analysis. Future research may also extend these techniques to other imaging modalities and tasks, while investigating hybrid models that combine the strengths of both augmentation and normalization for optimal performance.

In conclusion, the paper delivers crucial insights into the relative importance of data augmentation and color normalization in computational pathology, guiding practitioners in designing more robust and adaptable CNN models.