S$^3$-TTA: Scale-Style Selection for Test-Time Augmentation in Biomedical Image Segmentation (2310.16783v2)
Abstract: Deep-learning models have been successful in biomedical image segmentation. To generalize for real-world deployment, test-time augmentation (TTA) methods are often used to transform the test image into different versions that are hopefully closer to the training domain. Unfortunately, due to the vast diversity of instance scale and image styles, many augmented test images produce undesirable results, thus lowering the overall performance. This work proposes a new TTA framework, S$3$-TTA, which selects the suitable image scale and style for each test image based on a transformation consistency metric. In addition, S$3$-TTA constructs an end-to-end augmentation-segmentation joint-training pipeline to ensure a task-oriented augmentation. On public benchmarks for cell and lung segmentation, S$3$-TTA demonstrates improvements over the prior art by 3.4% and 1.3%, respectively, by simply augmenting the input data in testing phase.
- Reka Hollandi et al, “nucleaizer: a parameter-free deep learning framework for nucleus segmentation using image style transfer,” Cell Systems, vol. 10, no. 5, pp. 453–458, 2020.
- Xiaoqiong Huang et al, “Style-invariant Cardiac Image Segmentation with Test-time Augmentation,” arXiv e-prints, p. arXiv:2009.12193, Sept. 2020.
- Nikita Moshkov et al, “Test-time augmentation for deep learning-based cell segmentation on microscopy images,” Scientific Reports, vol. 10, 2020.
- Carsen Stringer et al, “Cellpose: a generalist algorithm for cellular segmentation,” Nature methods, vol. 18, no. 1, pp. 100–106, 2021.
- Kaiming He et al, “Mask r-cnn,” 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988, 2017.
- Leander Lauenburg et al, “Instance segmentation of unlabeled modalities via cyclic segmentation gan,” arXiv preprint arXiv:2204.03082, 2022.
- Matthew R Keaton et al, “Celltranspose: Few-shot domain adaptation for cellular instance segmentation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 455–466.
- Gatys Leon A. et al, “A neural algorithm of artistic style,” ArXiv, vol. abs/1508.06576, 2015.
- “Arbitrary style transfer in real-time with adaptive instance normalization,” in ICCV, 2017.
- Reka Hollandi et al, “A deep learning framework for nucleus segmentation using image style transfer,” bioRxiv, 2019.
- Philip T. G. Jackson et al, “Style augmentation: Data augmentation via style randomization,” in CVPR Workshops, 2018.
- Lei Li et al, “Random style transfer based domain generalization networks integrating shape and spatial information,” ArXiv, vol. abs/2008.12205, 2020.
- Chunwei Ma et al, “Neural style transfer improves 3d cardiovascular mr image segmentation on inconsistent data,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019.
- Zhendong Liu et al, “Remove appearance shift for ultrasound image segmentation via fast and universal style transfer,” 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1824–1828, 2020.
- Zhendong Liu et al, “Generalize ultrasound image segmentation via instant and plug & play style transfer,” 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 419–423, 2021.
- Juan C Caicedo et al, “Nucleus segmentation across imaging experiments: the 2018 data science bowl,” Nature methods, vol. 16, no. 12, pp. 1247–1253, 2019.
- Stefan Jaeger et al, “Two public chest x-ray datasets for computer-aided screening of pulmonary diseases,” Quantitative Imaging in Medicine and Surgery, vol. 4, no. 6, 2014.
- Viacheslav Danilov et al, “Chest x-ray dataset for lung segmentation,” http://dx.doi.org/10.17632/8gf9vpkhgy.2.
- Liang-Chieh Chen et al, “Rethinking atrous convolution for semantic image segmentation,” ArXiv, vol. abs/1706.05587, 2017.