Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Double-well Net for Image Segmentation (2401.00456v2)

Published 31 Dec 2023 in cs.CV

Abstract: In this study, our goal is to integrate classical mathematical models with deep neural networks by introducing two novel deep neural network models for image segmentation known as Double-well Nets. Drawing inspirations from the Potts model, our models leverage neural networks to represent a region force functional. We extend the well-know MBO (Merriman-Bence-Osher) scheme to solve the Potts model. The widely recognized Potts model is approximated using a double-well potential and then solved by an operator-splitting method, which turns out to be an extension of the well-known MBO scheme. Subsequently, we replace the region force functional in the Potts model with a UNet-type network, which is data-driven and is designed to capture multiscale features of images, and also introduce control variables to enhance effectiveness. The resulting algorithm is a neural network activated by a function that minimizes the double-well potential. What sets our proposed Double-well Nets apart from many existing deep learning methods for image segmentation is their strong mathematical foundation. They are derived from the network approximation theory and employ the MBO scheme to approximately solve the Potts model. By incorporating mathematical principles, Double-well Nets bridge the MBO scheme and neural networks, and offer an alternative perspective for designing networks with mathematical backgrounds. Through comprehensive experiments, we demonstrate the performance of Double-well Nets, showcasing their superior accuracy and robustness compared to state-of-the-art neural networks. Overall, our work represents a valuable contribution to the field of image segmentation by combining the strengths of classical variational models and deep neural networks. The Double-well Nets introduce an innovative approach that leverages mathematical foundations to enhance segmentation performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (84)
  1. Augmented lagrangian method for an euler’s elastica based segmentation model that promotes convex contours. Inverse Problems & Imaging, 11(1), 2017.
  2. A. R. Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, 39(3):930–945, 1993.
  3. Deep learning as optimal control problems: Models and numerical methods. Journal of Computational Dynamics, 6(2):171–198, 2019.
  4. Operator splitting algorithms for free surface flows: Application to extrusion processes. In Splitting Methods in Communication, Imaging, Science, and Engineering, pages 677–729. Springer, 2017.
  5. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222–1239, 2001.
  6. Fluid–structure interaction in blood flow capturing non-zero longitudinal structure displacement. Journal of Computational Physics, 235:515–541, 2013.
  7. A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding. SIAM Journal on Imaging Sciences, 6(1):368–390, 2013.
  8. Geodesic active contours. International Journal of Computer Vision, 22(1):61, 1997.
  9. A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision, 40(1):120–145, 2011.
  10. T. Chan and W. Zhu. Level set based shape prior segmentation. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), volume 2, pages 1164–1170. IEEE, 2005.
  11. Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics, 66(5):1632–1648, 2006.
  12. Active contours without edges. IEEE Transactions on Image Processing, 10(2):266–277, 2001.
  13. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 801–818, 2018.
  14. Efficient approximation of deep relu networks for functions on low dimensional manifolds. Advances in Neural Information Processing Systems, 32, 2019.
  15. Nonparametric regression on low-dimensional manifolds using deep ReLU networks: Function approximation and statistical recovery. Information and Inference: A Journal of the IMA, 11(4):1203–1253, 2022.
  16. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3):569–582, 2014.
  17. Deep nets for local manifold learning. Frontiers in Applied Mathematics and Statistics, 4:12, 2018.
  18. A new operator splitting method for the euler elastica model for image smoothing. SIAM Journal on Imaging Sciences, 12(2):1190–1230, 2019.
  19. A fast operator-splitting method for beltrami color image denoising. Journal of Scientific Computing, 92(3):89, 2022.
  20. Neural ode control for trajectory approximation of continuity equation. IEEE Control Systems Letters, 6:3152–3157, 2022.
  21. S. Esedog and Y.-H. R. Tsai. Threshold dynamics for the piecewise constant mumford–shah functional. Journal of Computational Physics, 211(1):367–384, 2006.
  22. MN-Net: A multi-scale attention network for liver and tumor segmentation. IEEE Access, 8:179656–179665, 2020.
  23. Multiclass data segmentation using diffuse interface methods on graphs. IEEE transactions on pattern analysis and machine intelligence, 36(8):1600–1613, 2014.
  24. A penalization-regularization-operator splitting method for eikonal based traveltime tomography. SIAM Journal on Imaging Sciences, 8(2):1263–1292, 2015.
  25. A finite element/operator-splitting method for the numerical solution of the two dimensional elliptic monge–ampère equation. Journal of Scientific Computing, 79:1–47, 2019.
  26. Splitting Methods in Communication, Imaging, Science, and Engineering. Springer, 2017.
  27. Some facts about operator-splitting and alternating direction methods. Splitting Methods in Communication, Imaging, Science, and Engineering, pages 19–94, 2016.
  28. E. Haber and L. Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34(1):014004, 2017.
  29. J. He and J. Xu. MgNet: A unified framework of multigrid and convolutional neural network. Science China Mathematics, 62:1331–1354, 2019.
  30. Curvature regularized surface reconstruction from point clouds. SIAM Journal on Imaging Sciences, 13(4):1834–1859, 2020.
  31. S. Hon and H. Yang. Simultaneous neural network approximation for smooth functions. Neural Networks, 154:152–164, 2022.
  32. Automated separation of binary overlapping trees in low-contrast color retinal images. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2013: 16th International Conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part II 16, pages 436–443. Springer, 2013.
  33. Snakes: Active contour models. International Journal of Computer Vision, 1(4):321–331, 1988.
  34. Dosnet as a non-black-box pde solver: When deep learning meets operator splitting. Journal of Computational Physics, 491:112343, 2023.
  35. EWT: Efficient wavelet-transformer for single image denoising. arXiv preprint arXiv:2304.06274, 2023.
  36. A three-stage variational image segmentation framework incorporating intensity inhomogeneity information. SIAM Journal on Imaging Sciences, 13(3):1692–1715, 2020.
  37. Deep CNN denoiser prior for blurred images restoration with multiplicative noise. Inverse Problems and Imaging, 17(3):726–745, 2023.
  38. Benefits of overparameterized convolutional residual networks: Function approximation under smoothness constraint. In International Conference on Machine Learning, pages 13669–13703. PMLR, 2022.
  39. Besov function approximation and binary classification on low-dimensional manifolds using convolutional residual networks. In International Conference on Machine Learning, pages 6770–6780. PMLR, 2021.
  40. A finite element/operator-splitting method for the numerical solution of the three dimensional Monge–Ampère equation. Journal of Scientific Computing, 81:2271–2302, 2019.
  41. Connections between operator-splitting methods and deep neural networks with applications in image segmentation. Annals of Applied Mathematics, 39(4):406–428, 2023.
  42. A color elastica model for vector-valued image regularization. SIAM Journal on Imaging Sciences, 14(2):717–748, 2021.
  43. Elastica models for color image regularization. SIAM Journal on Imaging Sciences, 16(1):461–500, 2023.
  44. H. Liu and D. Wang. Fast operator splitting methods for obstacle problems. Journal of Computational Physics, page 111941, 2023.
  45. Deep nonparametric estimation of operators between infinite dimensional spaces. arXiv preprint arXiv:2201.00217, 2022.
  46. Deep convolutional neural networks with spatial regularization, volume and star-shape priors for image segmentation. Journal of Mathematical Imaging and Vision, 64(6):625–645, 2022.
  47. Global binary optimization on graphs for classification of high-dimensional data. Journal of Mathematical Imaging and Vision, 52(3):414–435, 2015.
  48. An mbo scheme on graphs for classification and image processing. SIAM Journal on Imaging Sciences, 6(4):1903–1930, 2013.
  49. Motion of multiple junctions: A level set approach. Journal of computational physics, 112(2):334–363, 1994.
  50. L. Modica. Un esempio di γ𝛾\gammaitalic_γ-convergenza. Boll. Un. Mat. Ital. B, 14:285–299, 1977.
  51. L. Modica. The gradient theory of phase transitions and the minimal interface criterion. Archive for Rational Mechanics and Analysis, 98:123–142, 1987.
  52. D. B. Mumford and J. Shah. Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 1989.
  53. Fourierformer: Transformer meets generalized fourier integral theorem. Advances in Neural Information Processing Systems, 35:29319–29335, 2022.
  54. A neural network approach for high-dimensional optimal control applied to multiagent path finding. IEEE Transactions on Control Systems Technology, 31(1):235–251, 2022.
  55. K. Oono and T. Suzuki. Approximation and non-parametric estimation of ResNet-type convolutional neural networks. In International Conference on Machine Learning, pages 4922–4931. PMLR, 2019.
  56. S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79(1):12–49, 1988.
  57. A convex relaxation approach for computing minimal partitions. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 810–817. IEEE, 2009.
  58. R. B. Potts. Some generalized order-disorder transformations. In Mathematical Proceedings of the Cambridge Philosophical Society, volume 48, pages 106–109. Cambridge University Press, 1952.
  59. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention, pages 234–241. Springer, 2015.
  60. D. Ruiz-Balet and E. Zuazua. Neural ode control for classification, approximation, and transport. SIAM Review, 65(3):735–773, 2023.
  61. L. Ruthotto and E. Haber. Deep neural networks motivated by partial differential equations. Journal of Mathematical Imaging and Vision, 62:352–364, 2020.
  62. Deep network approximation characterized by number of neurons. arXiv preprint arXiv:1906.05497, 2019.
  63. Hierarchical image saliency detection on extended CSSD. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(4):717–729, 2015.
  64. Approximation of smooth functionals using deep ReLU networks. Neural Networks, 166:424–436, 2023.
  65. Efficient and convergent preconditioned ADMM for the Potts models. SIAM Journal on Scientific Computing, 43(2):B455–B478, 2021.
  66. Image segmentation using some piecewise constant level set methods with mbo type of projection. International Journal of Computer Vision, 73:61–76, 2007.
  67. The potts model with different piecewise constant representations and fast algorithms: a survey. Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging: Mathematical Imaging and Vision, pages 1–41, 2021.
  68. PottsMGNet: A mathematical explanation of encoder-decoder based neural networks. arXiv preprint arXiv:2307.09039, 2023.
  69. γ𝛾\gammaitalic_γ-convergence of graph ginzburg-landau functionals. Adv. Differential Equations, 17(11-12):1115–1180, 2012.
  70. New region force for variational models in image segmentation and high dimensional data clustering. Annals of Mathematical Sciences and Applications, 3(1):255–286, 2018.
  71. E. Weinan. A proposal on machine learning via dynamical systems. Communications in Mathematics and Statistics, 1(5):1–11, 2017.
  72. P. Yakubovskiy. Segmentation models pytorch. https://github.com/qubvel/segmentation_models.pytorch, 2020.
  73. Convexity shape prior for level set-based image segmentation method. IEEE Transactions on Image Processing, 29:7141–7152, 2020.
  74. D. Yang and J. Sun. BM3D-Net: A convolutional neural network for transform-domain collaborative filtering. IEEE Signal Processing Letters, 25(1):55–59, 2017.
  75. D. Yarotsky. Error bounds for approximations with deep ReLU networks. Neural Networks, 94:103–114, 2017.
  76. M. Yashtini and S. H. Kang. A fast relaxed normal two split method and an effective weighted TV approach for Euler’s elastica image inpainting. SIAM Journal on Imaging Sciences, 9(4):1552–1581, 2016.
  77. K. Yin and X.-C. Tai. An effective region force for some variational models for learning and clustering. Journal of Scientific Computing, 74(1):175–196, 2018.
  78. A study on continuous max-flow and min-cut approaches. In 2010 IEEE Computer Cociety Conference on Computer Vision and Pattern Recognition, pages 2217–2224. IEEE, 2010.
  79. A spatially continuous max-flow and min-cut framework for binary labeling problems. Numerische Mathematik, 126:559–587, 2014.
  80. D. Zhang and L. M. Lui. Topology-preserving 3D image segmentation based on hyperelastic regularization. Journal of Scientific Computing, 87(3):74, 2021.
  81. Topology-and convexity-preserving image segmentation based on image registration. Applied Mathematical Modelling, 100:218–239, 2021.
  82. Beyond a Gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.
  83. D.-X. Zhou. Universality of deep convolutional neural networks. Applied and Computational Harmonic Analysis, 48(2):787–794, 2020.
  84. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pages 3–11. Springer, 2018.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now