Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unsupervised Learning of the Total Variation Flow (2206.04406v2)

Published 9 Jun 2022 in cs.CV, cs.LG, and eess.IV

Abstract: The total variation (TV) flow generates a scale-space representation of an image based on the TV functional. This gradient flow observes desirable features for images, such as sharp edges and enables spectral, scale, and texture analysis. Solving the TV flow is challenging; one reason is the the non-uniqueness of the subgradients. The standard numerical approach for TV flow requires solving multiple non-smooth optimisation problems. Even with state-of-the-art convex optimisation techniques, this is often prohibitively expensive and strongly motivates the use of alternative, faster approaches. Inspired by and extending the framework of physics-informed neural networks (PINNs), we propose the TVflowNET, an unsupervised neural network approach, to approximate the solution of the TV flow given an initial image and a time instance. The TVflowNET requires no ground truth data but rather makes use of the PDE for optimisation of the network parameters. We circumvent the challenges related to the non-uniqueness of the subgradients by additionally learning the related diffusivity term. Our approach significantly speeds up the computation time and we show that the TVflowNET approximates the TV flow solution with high fidelity for different image sizes and image types. Additionally, we give a full comparison of different network architecture designs as well as training regimes to underscore the effectiveness of our approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. J. Adler and O. Öktem. Learned primal-dual reconstruction. IEEE transactions on medical imaging, 37(6):1322–1332, 2018.
  2. Functions of Bounded Variation and Free Discontinuity Problems. Oxford University Press, 2000.
  3. Minimizing total variation flow. Differential Integral Equations, 14:321–360, 2001.
  4. Some qualitative properties for the total variation flow. Journal of Functional Analysis, 188:516–547, 2 2002.
  5. Parabolic Quasilinear Equations Minimizing Linear Growth Functionals. Progress in Mathematics, 223. Springer Basel AG, 1st ed. 2004. edition, 2004.
  6. Learning to learn by gradient descent by gradient descent. Advances in neural information processing systems, 29, 2016.
  7. Structure-texture decomposition by a tv-gabor model. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 3752 LNCS:85–96, 2005.
  8. L. Bar and N. Sochen. Unsupervised deep learning algorithm for PDE-based forward and inverse problems. arXiv preprint arXiv:1904.05417, 2019.
  9. The total variation flow in rn. Journal of Differential Equations, 184:475–525, 9 2002.
  10. Nonlinear spectral image fusion. In Lecture Notes in Computer Science, pages 41–53. Springer International Publishing, 2017.
  11. Food-101 – mining discriminative components with random forests. In European Conference on Computer Vision, 2014.
  12. K. Bredies and M. Holler. A pointwise characterization of the subdifferential of the total variation functional. arXiv preprint arXiv:1609.08918, 2016.
  13. Numerical aspects of tv flow. Numerical Algorithms, 41:79–101, 1 2006.
  14. H. Brezis. Operateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert, volume 5. Elsevier, 1973.
  15. T. Brox and J. Weickert. A TV flow based local scale measure for texture discrimination. In T. Pajdla and J. Matas, editors, Computer Vision - ECCV 2004, pages 578–590, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg.
  16. T. Brox and J. Weickert. A tv flow based local scale estimate and its application to texture discrimination. Journal of Visual Communication and Image Representation, 17:1053–1073, 10 2006.
  17. Nonlinear spectral decompositions by gradient flows of one-homogeneous functionals. arXiv preprint arXiv:1901.06979, 2019.
  18. Spectral decompositions using one-homogeneous functionals. SIAM Journal on Imaging Sciences, 9(3):1374–1408, jan 2016.
  19. Physics-Informed Neural Networks for Heat Transfer Problems. Journal of Heat Transfer, 143(6), 04 2021.
  20. On the jump set of solutions of the total variation flow. Rendiconti del Seminario Matematico della Università di Padova, 130:155–168, 2013.
  21. Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 29(1):182–193, 1992.
  22. A. Chambolle. An algorithm for total variation minimization and applications. J. Math. Imaging Vis., 20(1–2):89–97, Jan. 2004.
  23. A. Chambolle and P.-L. Lions. Image recovery via total variation minimization and related problems. Numerische Mathematik, 76(2):167–188, 1997.
  24. A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision, 40(1):120–145, dec 2010.
  25. A. Chambolle and T. Pock. An introduction to continuous optimization for imaging. Acta Numerica, 25:161–319, 2016.
  26. A nonlinear primal-dual method for total variation-based image restoration. SIAM Journal on Scientific Computing, 20(6):1964–1977, 1999.
  27. T. F. Chan and P. Mulet. On the convergence of the lagged diffusivity fixed point method in total variation image restoration. SIAM journal on numerical analysis, 36(2):354–367, 1999.
  28. An analysis of single-layer networks in unsupervised feature learning. In G. Gordon, D. Dunson, and M. Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 215–223, Fort Lauderdale, FL, USA, 11–13 Apr 2011. PMLR.
  29. Total-variation mode decomposition. In A. Elmoataz, J. Fadili, Y. Quéau, J. Rabin, and L. Simon, editors, Scale Space and Variational Methods in Computer Vision, pages 52–64. Springer International Publishing, 2021.
  30. I. Cohen and G. Gilboa. Latent modes of nonlinear flows – a Koopman theory analysis. arXiv:2107.07456, 2021.
  31. W. E and B. Yu. The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems. Communications in Mathematics and Statistics, 6(1):1–12, 2018.
  32. X. Feng and A. Prohl. Analysis of total variation flow and its finite element approximations. ESAIM: Mathematical Modelling and Numerical Analysis, 37:533–556, 5 2003.
  33. Deepview: View synthesis with learned gradient descent. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2367–2376, 2019.
  34. Solving image pdes with a shallow network. CoRR, abs/2110.08327, 2021.
  35. A duality based approach to the minimizing total variation flow in the space h-s. Japan Journal of Industrial and Applied Mathematics, 36:261–286, 1 2019.
  36. G. Gilboa. A spectral approach to total variation. In A. Kuijper, K. Bredies, T. Pock, and H. Bischof, editors, Scale Space and Variational Methods in Computer Vision, pages 36–47, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg.
  37. G. Gilboa. A total variation spectral framework for scale and texture analysis. SIAM Journal on Imaging Sciences, 7(4):1937–1961, jan 2014.
  38. G. Gilboa. Nonlinear Eigenproblems in Image Processing and Computer Vision. Springer International Publishing, 2018.
  39. Deeply learned spectral total variation decomposition. Advances in Neural Information Processing Systems, 33:12115–12126, 2020.
  40. E. Hait and G. Gilboa. Spectral total-variation local scale signatures for image manipulation and fusion. IEEE Transactions on Image Processing, 28(2):880–895, feb 2019.
  41. Learning a variational network for reconstruction of accelerated MRI data. Magnetic Resonance in Medicine, 79(6):3055–3071, nov 2017.
  42. Deep residual learning for image recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
  43. D. Horesh and G. Gilboa. Separation surfaces in the spectral TV domain for texture decomposition. IEEE Transactions on Image Processing, pages 1–1, 2016.
  44. J. Kinnunen and C. Scheven. On the definition of solution to the total variation flow. Calculus of Variations and Partial Differential Equations, 61, 2 2022.
  45. Multimodal medical image fusion based on the spectral total variation and local structural patch measurement. International Journal of Imaging Systems and Technology, 31(1):391–411, 2021.
  46. PDE-net: Learning PDEs from data. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 3208–3216. PMLR, 10–15 Jul 2018.
  47. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218–229, 2021.
  48. Learning nonlinear spectral filters for color image reconstruction. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE, dec 2015.
  49. fpinns: Fractional physics-informed neural networks. SIAM Journal on Scientific Computing, 41(4):A2603–A2626, 2019.
  50. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  51. U-Net: Convolutional networks for biomedical image segmentation. In Lecture Notes in Computer Science, pages 234–241. Springer International Publishing, 2015.
  52. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1):259–268, 1992.
  53. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063, 2019.
  54. Tvseg-interactive total variation based image segmentation. In Proceedings of British Machine Vision Conference (BMVC 2008), pages 01–02. ., 2008.
  55. Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 17(1):227–238, 1996.
  56. A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences, 1(3):248–272, 2008.
  57. Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Transactions on Image Processing, 7(3):398–410, 1998.
  58. Multiscale segmentation via bregman distances and nonlinear spectral analysis. SIAM Journal on Imaging Sciences, 10(1):111–146, jan 2017.
  59. Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, July 2017.
  60. Multisensor image fusion and enhancement in spectral total variation domain. IEEE Transactions on Multimedia, 20(4):866–879, apr 2018.
  61. Improving deep neural networks using softplus units. In 2015 International Joint Conference on Neural Networks (IJCNN), pages 1–4, 2015.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now