Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Convergence Analysis of Blurring Mean Shift (2402.15146v1)

Published 23 Feb 2024 in cs.LG and cs.CV

Abstract: Blurring mean shift (BMS) algorithm, a variant of the mean shift algorithm, is a kernel-based iterative method for data clustering, where data points are clustered according to their convergent points via iterative blurring. In this paper, we analyze convergence properties of the BMS algorithm by leveraging its interpretation as an optimization procedure, which is known but has been underutilized in existing convergence studies. Whereas existing results on convergence properties applicable to multi-dimensional data only cover the case where all the blurred data point sequences converge to a single point, this study provides a convergence guarantee even when those sequences can converge to multiple points, yielding multiple clusters. This study also shows that the convergence of the BMS algorithm is fast by further leveraging geometrical characterization of the convergent points.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. K. Fukunaga and L. Hostetler, “The estimation of the gradient of a density function, with applications in pattern recognition,” IEEE Transactions on Information Theory, vol. 21, no. 1, pp. 32–40, 1975.
  2. Y. Cheng, “Mean shift, mode seeking, and clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 790–799, 1995.
  3. Y. Cheng and Z. Wan, “Analysis of the blurring process,” in Computational Learning Theory and Natural Learning Systems.   MIT Press, 1995, vol. III: Selecting Good Models, ch. 14, pp. 257–276.
  4. M. A. Carreira-Perpinán, “Fast nonparametric clustering with gaussian blurring mean-shift,” in Proceedings of the 23rd International Conference on Machine Learning, 2006, pp. 153–160.
  5. ——, “Generalised blurring mean-shift algorithms for nonparametric clustering,” in Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
  6. T.-L. Chen, “On the convergence and consistency of the blurring mean-shift process,” Annals of the Institute of Statistical Mathematics, vol. 67, no. 1, pp. 157–176, 2015.
  7. K. Zhang, J. T. Kwok, and M. Tang, “Accelerated convergence using dynamic mean shift,” in Proceedings of the 9th European Conference on Computer Vision.   Springer, 2006, pp. 257–268.
  8. D. Comaniciu and P. Meer, “Mean shift: A robust approach toward feature space analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603–619, 2002.
  9. J. E. Chacón, “Mixture model modal clustering,” Advances in Data Analysis and Classification, vol. 12, no. 41, pp. 1–26, 2018.
  10. H. Guo, P. Guo, and Q. Liu, “Mean shift-based edge detection for color image,” in Proceedings of the 2005 International Conference on Neural Networks and Brain, vol. 2, 2005, pp. 1118–1122.
  11. D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-based object tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564–577, 2003.
  12. R. Yamasaki and T. Tanaka, “Optimal kernel for kernel-based modal statistical methods,” 2023, arXiv preprint arXiv:2304.10046v1 [stat.ML].
  13. D. Comaniciu and P. Meer, “Mean shift analysis and applications,” in Proceedings of the seventh IEEE International Conference on Computer Vision, vol. 2, 1999, pp. 1197–1203.
  14. K. Huang, X. Fu, and N. Sidiropoulos, “On convergence of Epanechnikov mean shift,” in Proceeding of the AAAI Conference on Artificial Intelligence, 2018.
  15. R. Yamasaki and T. Tanaka, “Properties of mean shift,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 9, pp. 2273–2286, 2020.
  16. ——, “Convergence analysis of mean shift,” 2023, arXiv preprint arXiv:2305.08463v3 [stat.ML].
  17. M. Meila and L. Bao, “An exponential model for infinite rankings,” Journal of Machine Learning Research, vol. 11, pp. 3481–3518, 2010.
  18. M. Šurkala, K. Mozdřeň, R. Fusek, and E. Sojka, “Hierarchical blurring mean-shift,” in Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems.   Springer, 2011, pp. 228–238.
  19. Y. Zhang, K. Chen, H. Wang, Y. Zhou, and H. Guan, “Two-view motion segmentation by gaussian blurring mean shift with fitness measure,” in Proceedings of the 2009 2nd International Congress on Image and Signal Processing.   IEEE, 2009, pp. 1–6.
  20. G. R. Bradski, “Computer vision face tracking for use in a perceptual user interface,” Intel technology journal, vol. 2, 1998.
  21. W. Wang and M. A. Carreira-Perpinán, “Manifold blurring mean shift algorithms for manifold denoising,” in Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.   IEEE, 2010, pp. 1759–1766.
  22. B. N. Clark, C. J. Colbourn, and D. S. Johnson, “Unit disk graphs,” Discrete Mathematics, vol. 86, pp. 165–177, 1990.
  23. M. L. Huson and A. Sen, “Broadcast scheduling algorithms for radio networks,” in Proceedings of MILCOM’95, vol. 2.   IEEE, 1995, pp. 647–651.
  24. R. Yamasaki and T. Tanaka, “Kernel selection for modal linear regression: Optimal kernel and IRLS algorithm,” in Proceedings of the 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 2019, pp. 595–601.
  25. J. de Leeuw and K. Lange, “Sharp quadratic majorization in one dimension,” Computational Statistics & Data Analysis, vol. 53, no. 7, pp. 2471–2484, 2009.
  26. K. Kurdyka and A. Parusinski, “wfsubscriptw𝑓\mathrm{w}_{f}roman_w start_POSTSUBSCRIPT italic_f end_POSTSUBSCRIPT-stratification of subanalytic functions and the Lojasiewicz inequality,” Comptes Rendus de l’Académie des Sciences. Série 1, Mathématique, vol. 318, no. 2, pp. 129–133, 1994.
  27. J. Bolte, A. Daniilidis, and A. Lewis, “The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems,” SIAM Journal on Optimization, vol. 17, no. 4, pp. 1205–1223, 2007.
  28. J. Bolte, A. Daniilidis, A. Lewis, and M. Shiota, “Clarke subgradients of stratifiable functions,” SIAM Journal on Optimization, vol. 18, no. 2, pp. 556–572, 2007.
  29. H. Attouch and J. Bolte, “On the convergence of the proximal algorithm for nonsmooth functions involving analytic features,” Mathematical Programming, vol. 116, no. 1–2, pp. 5–16, 2009.
  30. P. Frankel, G. Garrigos, and J. Peypouquet, “Splitting methods with variable metric for Kurdyka–Łojasiewicz functions and general convergence rates,” Journal of Optimization Theory and Applications, vol. 165, no. 3, pp. 874–900, 2015.
  31. P.-A. Absil, R. Mahony, and B. Andrews, “Convergence of the iterates of descent methods for analytic cost functions,” SIAM Journal on Optimization, vol. 16, no. 2, pp. 531–547, 2005.
  32. L. van den Dries and C. Miller, “Geometric categories and o-minimal structures,” Duke Mathematical Journal, vol. 84, no. 2, pp. 497–540, 1996.
  33. E. Bierstone and P. D. Milman, “Semianalytic and subanalytic sets,” Publications Mathématiques de l’IHÉS, vol. 67, pp. 5–42, 1988.
  34. J. P. Dedieu, “Penalty functions in subanalytic optimization,” Optimization, vol. 26, no. 1–2, pp. 27–32, 1992.
  35. H. Attouch, J. Bolte, and B. F. Svaiter, “Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward–backward splitting, and regularized Gauss-Seidel methods,” Mathematical Programming, vol. 137, no. 1-2, pp. 91–129, 2013.
  36. D. Noll, “Convergence of non-smooth descent methods using the Kurdyka–Łojasiewicz inequality,” Journal of Optimization Theory and Applications, vol. 160, no. 2, pp. 553–572, 2014.
  37. J. Bolte and E. Pauwels, “Majorization-minimization procedures and convergence of SQP methods for semi-algebraic and tame programs,” Mathematics of Operations Research, vol. 41, no. 2, pp. 442–465, 2016.
  38. D. D’Acunto and K. Kurdyka, “Explicit bounds for the Łojasiewicz exponent in the gradient inequality for polynomials,” Annales Polonici Mathematici, vol. 87, no. 1, pp. 51–61, 2005.
  39. K. Kurdyka and S. Spodzieja, “Separation of real algebraic sets and the Łojasiewicz exponent,” Proceedings of the American Mathematical Society, vol. 142, no. 9, pp. 3089–3102, 2014.
  40. M. S. Viazovska, “The sphere packing problem in dimension 8,” Annals of Mathematics, pp. 991–1015, 2017.
  41. H. Cohn, A. Kumar, S. D. Miller, D. Radchenko, and M. S. Viazovska, “The sphere packing problem in dimension 24,” Annals of Mathematics, vol. 185, no. 3, pp. 1017–1033, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now