Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook (2312.05391v1)

Published 8 Dec 2023 in cs.CV

Abstract: Semantic image segmentation, the process of classifying each pixel in an image into a particular class, plays an important role in many visual understanding systems. As the predominant criterion for evaluating the performance of statistical models, loss functions are crucial for shaping the development of deep learning-based segmentation algorithms and improving their overall performance. To aid researchers in identifying the optimal loss function for their particular application, this survey provides a comprehensive and unified review of $25$ loss functions utilized in image segmentation. We provide a novel taxonomy and thorough review of how these loss functions are customized and leveraged in image segmentation, with a systematic categorization emphasizing their significant features and applications. Furthermore, to evaluate the efficacy of these methods in real-world scenarios, we propose unbiased evaluations of some distinct and renowned loss functions on established medical and natural image datasets. We conclude this review by identifying current challenges and unveiling future research opportunities. Finally, we have compiled the reviewed studies that have open-source implementations on our GitHub page.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Efficient inference in fully connected crfs with gaussian edge potentials. Advances in neural information processing systems, 24, 2011.
  2. Foundational models in medical imaging: A comprehensive survey and future vision. arXiv preprint arXiv:2310.18689, 2023.
  3. Simultaneous detection and segmentation. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13, pages 297–312. Springer, 2014.
  4. Panoptic segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9396–9405, 2018.
  5. Hiformer: Hierarchical multi-scale representations using transformers for medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6202–6212, 2023.
  6. Unlocking fine-grained details with wavelet-based high-frequency enhancement in transformers. arXiv preprint arXiv:2308.13442, 2023.
  7. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3523–3542, 2022.
  8. Medical image segmentation review: The success of u-net. arXiv preprint arXiv:2211.14830, 2022.
  9. Advances in medical image analysis with vision transformers: A comprehensive review. arXiv preprint arXiv:2301.03505, 2023.
  10. Robust t-loss for medical image segmentation. arXiv preprint arXiv:2306.00753, 2023.
  11. Introducing the boundary-aware loss for deep image segmentation. In British Machine Vision Conference (BMVC) 2021, 2021.
  12. Shruti Jadon. A survey of loss functions for semantic segmentation. 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pages 1–7, 2020.
  13. The cityscapes dataset for semantic urban scene understanding. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3213–3223, Los Alamitos, CA, USA, jun 2016. IEEE Computer Society.
  14. MICCAI 2015 Multi-Atlas Abdomen Labeling Challenge. Synapse multi-organ segmentation dataset. https://www.synapse.org/#!Synapse:syn3193805/wiki/217789, 2015. Accessed: 2022-04-20.
  15. Baseg: Boundary aware semantic segmentation for autonomous driving. Neural Networks, 157:460–470, 2023.
  16. Contextual attention network: Transformer meets u-net. In Chunfeng Lian, Xiaohuan Cao, Islem Rekik, Xuanang Xu, and Zhiming Cui, editors, Machine Learning in Medical Imaging, pages 377–386, Cham, 2022. Springer Nature Switzerland.
  17. Semi-supervised multi-task learning for semantics and depth. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2505–2514, 2022.
  18. Region-wise loss for biomedical image segmentation. Pattern Recognition, 136:109208, 2023.
  19. Shruti Jadon. A survey of loss functions for semantic segmentation. In 2020 IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB), pages 1–7. IEEE, 2020.
  20. Revisiting squared-error and cross-entropy functions for training neural network classifiers. Neural Computing & Applications, 14:310–318, 2005.
  21. Bridging category-level and instance-level semantic image segmentation. arXiv preprint arXiv:1605.06885, 2016.
  22. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42:318–327, 2017.
  23. Distance map loss penalty term for semantic segmentation. ArXiv, abs/1908.03679, 2019.
  24. Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), pages 565–571.
  25. Generalised wasserstein dice score for imbalanced multi-class segmentation using holistic convolutional networks. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: Third International Workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers 3, pages 64–76. Springer, 2018.
  26. Optimizing intersection-over-union in deep neural networks for image segmentation. In International Symposium on Visual Computing, 2016.
  27. The lovasz-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4413–4421, 2017.
  28. Amos Tversky. Features of similarity. Psychological Review, 84:327–352, 1977.
  29. Tversky loss function for image segmentation using 3d fully convolutional deep networks. In Machine Learning in Medical Imaging: 8th International Workshop, MLMI 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 10, 2017, Proceedings 8, pages 379–387. Springer, 2017.
  30. A novel focal tversky loss function with improved attention u-net for lesion segmentation. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pages 683–687, 2018.
  31. Deep convolutional encoder networks for multiple sclerosis lesion segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015.
  32. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In International Conference on Machine Learning, 2001.
  33. Region mutual information loss for semantic segmentation. In Neural Information Processing Systems, 2019.
  34. Boundary loss for highly unbalanced segmentation. Medical image analysis, 67:101851, 2018.
  35. Reducing the hausdorff distance in medical image segmentation with convolutional neural networks. IEEE Transactions on Medical Imaging, 39:499–513, 2019.
  36. Boundary-aware instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5696–5704, 2017.
  37. Active boundary loss for semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2397–2405, 2022.
  38. Inverseform: A loss function for structured boundary-aware segmentation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5897–5907, 2021.
  39. Conditional boundary loss for semantic segmentation. IEEE Transactions on Image Processing, 32:3717–3731, 2023.
  40. Boundary difference over union loss for medical image segmentation. arXiv preprint arXiv:2308.00220, 2023.
  41. Combo loss: Handling input and output imbalance in multi-organ segmentation. Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society, 75:24–33, 2018.
  42. 3d segmentation with exponential logarithmic loss for highly unbalanced object sizes. ArXiv, abs/1809.00076, 2018.
  43. Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Computerized Medical Imaging and Graphics, 95:102026, 2022.
  44. Internimage: Exploring large-scale vision foundation models with deformable convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14408–14419, 2023.
  45. Object-contextual representations for semantic segmentation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pages 173–190. Springer, 2020.
  46. Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12475–12485, 2020.
  47. Dcnas: Densely connected neural architecture search for semantic image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13956–13967, 2021.
  48. Regularized frank-wolfe for dense crfs: Generalizing mean field and beyond. Advances in Neural Information Processing Systems, 34:1453–1467, 2021.
  49. Vision transformer adapter for dense predictions. arXiv preprint arXiv:2205.08534, 2022.
  50. Global aggregation then local distribution in fully convolutional networks. arXiv preprint arXiv:1909.07229, 2019.
  51. Hs3: Learning with proper task complexity in hierarchically supervised semantic segmentation. arXiv preprint arXiv:2111.02333, 2021.
  52. Efficientps: Efficient panoptic segmentation. International Journal of Computer Vision, 129(5):1551–1579, 2021.
  53. Loss max-pooling for semantic image segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2126–2135, 2017.
  54. nnu-net: Self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486, 2018.
  55. Mednext: Transformer-driven scaling of convnets for medical image segmentation. arXiv preprint arXiv:2303.09975, 2023.
  56. Phtrans: Parallelly aggregating global and local representations for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 235–244. Springer, 2022.
  57. nnformer: Interleaved transformer for volumetric segmentation. arXiv preprint arXiv:2109.03201, 2021.
  58. Multi-scale hierarchical vision transformer with cascaded attention decoding for medical image segmentation. arXiv preprint arXiv:2303.16892, 2023.
  59. Medical image segmentation via cascaded attention decoding. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6222–6231, 2023.
  60. Missformer: An effective medical image segmentation transformer. arXiv preprint arXiv:2109.07162, 2021.
  61. Improved abdominal multi-organ segmentation via 3d boundary-constrained deep neural networks. IEEE Access, 11:35097–35110, 2023.
  62. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20730–20740, 2022.
  63. Prior-aware neural network for partially-supervised multi-organ segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10672–10681, 2019.
  64. Deep semantic segmentation of natural and medical images: a review. Artificial Intelligence Review, 54:137–178, 2021.
  65. Topology-aware focal loss for 3d image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 580–589, 2023.
  66. Pixel-wise triplet learning for enhancing boundary discrimination in medical image segmentation. Knowledge-Based Systems, 243:108424, 2022.
  67. Diffusion models in medical imaging: A comprehensive survey. Medical Image Analysis, page 102846, 2023.
  68. Implicit neural representation in medical imaging: A comparative survey. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2381–2391, 2023.
  69. U-net: Convolutional networks for biomedical image segmentation, 2015.
  70. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.
  71. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  72. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  73. OpenAI. Gpt-4 technical report, 2023.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now