Leveraging AI Predicted and Expert Revised Annotations in Interactive Segmentation: Continual Tuning or Full Training? (2402.19423v1)
Abstract: Interactive segmentation, an integration of AI algorithms and human expertise, premises to improve the accuracy and efficiency of curating large-scale, detailed-annotated datasets in healthcare. Human experts revise the annotations predicted by AI, and in turn, AI improves its predictions by learning from these revised annotations. This interactive process continues to enhance the quality of annotations until no major revision is needed from experts. The key challenge is how to leverage AI predicted and expert revised annotations to iteratively improve the AI. Two problems arise: (1) The risk of catastrophic forgetting--the AI tends to forget the previously learned classes if it is only retrained using the expert revised classes. (2) Computational inefficiency when retraining the AI using both AI predicted and expert revised annotations; moreover, given the dominant AI predicted annotations in the dataset, the contribution of newly revised annotations--often account for a very small fraction--to the AI training remains marginal. This paper proposes Continual Tuning to address the problems from two perspectives: network design and data reuse. Firstly, we design a shared network for all classes followed by class-specific networks dedicated to individual classes. To mitigate forgetting, we freeze the shared network for previously learned classes and only update the class-specific network for revised classes. Secondly, we reuse a small fraction of data with previous annotations to avoid over-computing. The selection of such data relies on the importance estimate of each data. The importance score is computed by combining the uncertainty and consistency of AI predictions. Our experiments demonstrate that Continual Tuning achieves a speed 16x greater than repeatedly training AI from scratch without compromising the performance.
- “Self-supervised pre-training of swin transformers for 3d medical image analysis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 20730–20740.
- “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention. Springer, 2015, pp. 234–241.
- “Setting the mind for intelligent interactive segmentation: Overview, requirements, and framework,” in Biennial International Conference on Information Processing in Medical Imaging. Springer, 1997, pp. 417–422.
- “Interaction in the segmentation of medical images: A survey,” Medical image analysis, vol. 5, no. 2, pp. 127–142, 2001.
- “An overview of interactive medical image segmentation,” Annals of the BMVA, vol. 2013, no. 7, pp. 1–22, 2013.
- “Segment anything,” arXiv preprint arXiv:2304.02643, 2023.
- “Totalsegmentator: robust segmentation of 104 anatomical structures in ct images,” arXiv preprint arXiv:2208.05868, 2022.
- “Abdomenatlas-8k: Annotating 8,000 abdominal ct volumes for multi-organ segmentation in three weeks,” Conference on Neural Information Processing Systems, 2023.
- “How well do supervised models transfer to 3d image segmentation?,” in The Twelfth International Conference on Learning Representations, 2024.
- “Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7340–7351.
- “Integrating active learning and transfer learning for carotid intima-media thickness video interpretation,” Journal of digital imaging, vol. 32, no. 2, pp. 290–299, 2019.
- “Active, continual fine tuning of convolutional neural networks for reducing annotation efforts,” Medical image analysis, vol. 71, pp. 101997, 2021.
- “Making your first choice: To address cold start problem in vision active learning,” in Medical Imaging with Deep Learning. 2023.
- “Adaptive aggregation networks for class-incremental learning,” in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, 2021, pp. 2544–2553.
- “Catastrophic interference in neural networks: Causes, solutions, and data,” in Interference and inhibition in cognition, pp. 329–361. Elsevier, 1995.
- “Learning incrementally to segment multiple organs in a ct image,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2022, pp. 714–724.
- “A two-stream continual learning system with variational domain-agnostic feature replay,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 9, pp. 4466–4478, 2021.
- “Continual learning for abdominal multi-organ and tumor segmentation,” in International conference on medical image computing and computer-assisted intervention. Springer, 2023, pp. 35–45.
- “Adaptive active learning for image classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 859–866.
- “Deep bayesian active learning with image data,” in International conference on machine learning. PMLR, 2017, pp. 1183–1192.
- “Clip-driven universal model for organ segmentation and tumor detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 21152–21164.
- “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017.
- “Accurate, large minibatch sgd: Training imagenet in 1 hour,” arXiv preprint arXiv:1706.02677, 2017.
- “Abdomenct-1k: Is abdominal organ segmentation a solved problem?,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6695–6714, 2021.