Papers
Topics
Authors
Recent
Search
2000 character limit reached

Riemannian optimization for non-centered mixture of scaled Gaussian distributions

Published 7 Sep 2022 in cs.LG and stat.ME | (2209.03315v2)

Abstract: This paper studies the statistical model of the non-centered mixture of scaled Gaussian distributions (NC-MSG). Using the Fisher-Rao information geometry associated to this distribution, we derive a Riemannian gradient descent algorithm. This algorithm is leveraged for two minimization problems. The first one is the minimization of a regularized negative log-likelihood (NLL). The latter makes the trade-off between a white Gaussian distribution and the NC-MSG. Conditions on the regularization are given so that the existence of a minimum to this problem is guaranteed without assumptions on the samples. Then, the Kullback-Leibler (KL) divergence between two NC-MSG is derived. This divergence enables us to define a minimization problem to compute centers of mass of several NC-MSGs. The proposed Riemannian gradient descent algorithm is leveraged to solve this second minimization problem. Numerical experiments show the good performance and the speed of the Riemannian gradient descent on the two problems. Finally, a Nearest centroid classifier is implemented leveraging the KL divergence and its associated center of mass. Applied on the large scale dataset Breizhcrops, this classifier shows good accuracies as well as robustness to rigid transformations of the test set.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. F. Kai-Tai and Z. Yao-Ting, Generalized multivariate analysis, vol. 19, Science Press Beijing and Springer-Verlag, Berlin, 1990.
  2. R. A. Maronna, “Robust M-estimators of multivariate location and scatter,” Ann. Statist., vol. 4, no. 1, pp. 51–67, 01 1976.
  3. “Complex Elliptically Symmetric distributions: Survey, new results and applications,” IEEE Transactions on Signal Processing, vol. 60, no. 11, pp. 5597–5625, 2012.
  4. A. Wiesel, “Unified framework to regularized covariance estimation in scaled gaussian models,” IEEE Transactions on Signal Processing, vol. 60, no. 1, pp. 29–38, 2011.
  5. D. E. Tyler, “A distribution-free M-estimator of multivariate scatter,” Ann. Statist., vol. 15, no. 1, pp. 234–251, 03 1987.
  6. “Recursive estimation of the covariance matrix of a compound-gaussian process and its application to adaptive cfar detection,” IEEE Transactions on signal processing, vol. 50, no. 8, pp. 1908–1915, 2002.
  7. “Performance analysis of covariance matrix estimates in impulsive noise,” IEEE Transactions on signal processing, vol. 56, no. 6, pp. 2206–2217, 2008.
  8. G. Frahm and U. Jaekel, “Tyler’s m-estimator, random matrix theory, and generalized elliptical distributions with applications to finance,” Random Matrix Theory, and Generalized Elliptical Distributions with Applications to Finance (October 21, 2008), 2008.
  9. “Marčenko–Pastur law for Tyler’s m-estimator,” Journal of Multivariate Analysis, vol. 149, pp. 114–123, 2016.
  10. Optimization Algorithms on Matrix Manifolds, Princeton University Press, Princeton, NJ, USA, 2008.
  11. N. Boumal, “An introduction to optimization on smooth manifolds,” in Cambridge University Press, March 2023.
  12. F. Nielsen, “The many faces of information geometry,” Not. Am. Math. Soc, vol. 69, pp. 36–45, 2022.
  13. “Generalized robust shrinkage estimator and its application to stap detection problem,” IEEE Transactions on Signal Processing, vol. 62, no. 21, pp. 5640–5651, 2014.
  14. “Regularized Tyler’s scatter estimator: Existence, uniqueness, and algorithms,” IEEE Transactions on Signal Processing, vol. 62, no. 19, pp. 5143–5156, 2014.
  15. E. Ollila and D. E Tyler, “Regularized m𝑚mitalic_m-estimators of scatter matrix,” IEEE Transactions on Signal Processing, vol. 62, no. 22, pp. 6059–6070, 2014.
  16. “Regularized robust estimation of mean and covariance matrix under heavy-tailed distributions,” IEEE Transactions on Signal Processing, vol. 63, no. 12, pp. 3096–3109, 2015.
  17. “Spectral shrinkage of Tyler’s m𝑚mitalic_m-estimator of covariance matrix,” in 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP). IEEE, 2019, pp. 535–538.
  18. M. Yi and D. E. Tyler, “Shrinking the covariance matrix using convex penalties on the matrix-log transformation,” Journal of Computational and Graphical Statistics, vol. 30, no. 2, pp. 442–451, 2020.
  19. “Multiclass brain–computer interface classification by riemannian geometry,” IEEE Transactions on Biomedical Engineering, vol. 59, no. 4, pp. 920–928, 2012.
  20. “Human detection via classification on riemannian manifolds,” in 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8.
  21. “Pedestrian detection via classification on riemannian manifolds,” IEEE transactions on pattern analysis and machine intelligence, vol. 30, no. 10, pp. 1713–1727, 2008.
  22. “On the use of matrix information geometry for polarimetric sar image classification,” in Matrix Information Geometry, pp. 257–276. Springer, 2013.
  23. H. Karcher, “Riemannian center of mass and mollifier smoothing,” Communications on Pure and Applied Mathematics, vol. 30, no. 5, pp. 509–541, 1977.
  24. Medians and Means in Riemannian Geometry: Existence, Uniqueness and Computation, pp. 169–197, Springer Berlin Heidelberg, Berlin, Heidelberg, 2013.
  25. “Review of riemannian distances and divergences, applied to ssvep-based bci,” Neuroinformatics, vol. 19, no. 1, pp. 93–106, 2021.
  26. M. Calvo and J. M. Oller, “An explicit solution of information geodesic equations for the multivariate normal model,” Statistics and Risk Modeling, vol. 9, no. 1-2, pp. 119–138, 1991.
  27. “Information geometric approach to multisensor estimation fusion,” IEEE Transactions on Signal Processing, vol. 67, no. 2, pp. 279–292, 2019.
  28. “On the use of geodesic triangles between gaussian distributions for classification problems,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 5697–5701.
  29. “Breizhcrops: A time series dataset for crop type mapping,” International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ISPRS (2020), 2020.
  30. “Compound-gaussian clutter modeling with an inverse gaussian texture distribution,” IEEE Signal Processing Letters, vol. 19, no. 12, pp. 876–879, 2012.
  31. “Covariance structure maximum-likelihood estimates in compound gaussian noise: Existence and algorithm analysis,” IEEE Transactions on Signal Processing, vol. 56, no. 1, pp. 34–48, 2007.
  32. “Intrinsic Cramér–Rao bounds for scatter and shape matrices estimation in CES distributions,” IEEE Signal Processing Letters, vol. 26, no. 2, pp. 262–266, 2019.
  33. “Riemannian geometry for compound Gaussian distributions: application to recursive change detection,” Signal Processing, 2020.
  34. A. Wiesel and T. Zhang, Structured robust covariance estimation, Now Foundations and Trends, 2015.
  35. “Robust estimation of structured covariance matrix for heavy-tailed elliptical distributions,” IEEE Transactions on Signal Processing, vol. 64, no. 14, pp. 3576–3590, 2016.
  36. “Robust estimation of structured scatter matrices in (mis) matched models,” Signal Processing, vol. 165, pp. 163–174, 2019.
  37. “A Tyler-type estimator of location and scatter leveraging riemannian optimization”,” in Acoustics, Speech and Signal Processing (ICASSP), 2021 IEEE International Conference on, Toronto, Canada. June 2021, 2021.
  38. S. Amari, Information Geometry and Its Applications, Springer Publishing Company, Incorporated, 1st edition, 2016.
  39. L. T. Skovgaard, “A Riemannian geometry of the multivariate Normal model,” Scandinavian Journal of Statistics, vol. 11, no. 4, pp. 211–223, 1984.
  40. M. Pilté and F. Barbaresco, “Tracking quality monitoring based on information geometry and geodesic shooting,” in 2016 17th International Radar Symposium (IRS), 2016, pp. 1–6.
  41. “Autograd: Effortless gradients in pure numpy,” AutoML workshop ICML, 2015.
  42. “JAX: composable transformations of Python+NumPy programs,” in ., 2018.
  43. A. Wiesel, “Geodesic convexity and covariance estimation,” IEEE Transactions on Signal Processing, vol. 60, no. 12, pp. 6182–6189, 2012.
  44. “Shrinking the eigenvalues of m-estimators of covariance matrix,” IEEE Transactions on Signal Processing, vol. 69, pp. 256–269, 2021.
  45. M. Moakher, “A differential geometric approach to the geometric mean of symmetric positive-definite matrices,” SIAM Journal on Matrix Analysis and Applications, vol. 26, no. 3, pp. 735–747, 2005.
  46. F. Mezzadri, “How to generate random matrices from the classical compact groups,” Notices of the AMS, 2006.
  47. “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, pp. 261–272, 2020.
  48. S. Smith, “Covariance, subspace, and intrinsic Cramér-Rao bounds,” Signal Processing, IEEE Transactions on, vol. 53, pp. 1610–1630, 06 2005.
  49. “A Riemannian framework for low-rank structured elliptical models,” IEEE Transactions on Signal Processing, vol. 69, no. 3, pp. 1185–1199, 2021.
Citations (5)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up