Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Convex Parameter Estimation of Perturbed Multivariate Generalized Gaussian Distributions (2312.07479v1)

Published 12 Dec 2023 in stat.ME and stat.ML

Abstract: The multivariate generalized Gaussian distribution (MGGD), also known as the multivariate exponential power (MEP) distribution, is widely used in signal and image processing. However, estimating MGGD parameters, which is required in practical applications, still faces specific theoretical challenges. In particular, establishing convergence properties for the standard fixed-point approach when both the distribution mean and the scatter (or the precision) matrix are unknown is still an open problem. In robust estimation, imposing classical constraints on the precision matrix, such as sparsity, has been limited by the non-convexity of the resulting cost function. This paper tackles these issues from an optimization viewpoint by proposing a convex formulation with well-established convergence properties. We embed our analysis in a noisy scenario where robustness is induced by modelling multiplicative perturbations. The resulting framework is flexible as it combines a variety of regularizations for the precision matrix, the mean and model perturbations. This paper presents proof of the desired theoretical properties, specifies the conditions preserving these properties for different regularization choices and designs a general proximal primal-dual optimization strategy. The experiments show a more accurate precision and covariance matrix estimation with similar performance for the mean vector parameter compared to Tyler's M-estimator. In a high-dimensional setting, the proposed method outperforms the classical GLASSO, one of its robust extensions, and the regularized Tyler's estimator.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. “A flexible EM-like clustering algorithm for noisy data,” arXiv preprint arXiv:1907.01660, 2019.
  2. “Robust classification with flexible discriminant analysis in heterogeneous data,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 5717–5721.
  3. “A novel m-estimator for robust pca,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 749–808, 2014.
  4. “Modern radar detection theory,” Tech. Rep., SciTech Publishing Inc, 2015.
  5. “A signal processing perspective on financial engineering,” Foundations and Trends® in Signal Processing, vol. 9, no. 1–2, pp. 1–231, 2016.
  6. “Parameter estimation for Multivariate Generalized Gaussian Distributions,” IEEE Transactions on Signal Processing, vol. 61, no. 23, pp. 5960–5971, 2013.
  7. Ricardo A. Maronna, “Robust M-estimators of multivariate location and scatter,” Ann. Stat., vol. 5, no. 1, pp. 51–67, 1976.
  8. “Complex elliptically symmetric distributions: survey, new results and applications,” IEEE Transactions on Signal Processing, vol. 60, no. 11, pp. 5597–5625, 2012.
  9. “Sparse inverse covariance estimation with the graphical lasso,” Biostatistics, vol. 9, no. 3, pp. 432–441, July 2008.
  10. “Regularized M-estimators of scatter matrix,” IEEE Transactions on Signal Processing, vol. 62, no. 22, pp. 6059–6070, 2014.
  11. “Generalized robust shrinkage estimator and its application to stap detection problem,” IEEE Transactions on Signal Processing, vol. 62, no. 21, pp. 5640–5651, 2014.
  12. “Regularized Tyler’s scatter estimator: Existence, uniqueness, and algorithms,” IEEE Transactions on Signal Processing, vol. 62, no. 19, pp. 5143–5156, 2014.
  13. Robust and Sparse Estimation of Graphical Models Based on Multivariate Winsorization, pp. 249–275, Springer International Publishing, Cham, 2023.
  14. “Robust estimation of precision matrices under cellwise contamination,” Computational Statistics & Data Analysis, vol. 93, pp. 404–420, 2016.
  15. Robust High-Dimensional Precision Matrix Estimation, pp. 325–350, Springer International Publishing, Cham, 2015.
  16. Ami Wiesel, “Geodesic convexity and covariance estimation,” IEEE transactions on signal processing, vol. 60, no. 12, pp. 6182–6189, 2012.
  17. “A multivariate generalization of the power exponential family of distributions,” Communications in Statistics-Theory and Methods, vol. 27, no. 3, pp. 589–600, 1998.
  18. “Semi-supervised robust mixture models in rkhs for abnormality detection in medical images,” IEEE Transactions on Image Processing, vol. 29, pp. 4772–4787, 2020.
  19. “Deep global generalized Gaussian networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5080–5088.
  20. “Using generalized gaussian distributions to improve regression error modeling for deep learning-based speech enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 12, pp. 1919–1931, 2019.
  21. “A convex formulation for the robust estimation of multivariate exponential power models,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 5772–5776.
  22. “Playing with duality: An overview of recent primal-dual approaches for solving large-scale optimization problems,” IEEE Signal Processing Magazine, vol. 32, no. 6, pp. 31–54, 2015.
  23. “Fixed point strategies in data science,” IEEE Transactions on Signal Processing, vol. 69, pp. 3878–3905, 2021.
  24. Kung Yao, “A representation theorem and its applications to spherically-invariant random processes,” IEEE Transactions on Information Theory, vol. 19, no. 5, pp. 600–608, 1973.
  25. Continuous univariate distributions, volume 2, vol. 289, John wiley & sons, 1995.
  26. “Covariance structure maximum-likelihood estimates in compound gaussian noise: Existence and algorithm analysis,” IEEE Transactions on Signal Processing, vol. 56, no. 1, pp. 34–48, 2007.
  27. “Exact maximum likelihood estimates for sirv covariance matrix: Existence and algorithm analysis,” IEEE Transactions on signal processing, vol. 56, no. 10, pp. 4563–4573, 2008.
  28. “Robust shrinkage estimation of high-dimensional covariance matrices,” IEEE Transactions on Signal Processing, vol. 59, no. 9, pp. 4097–4107, 2011.
  29. David E. Tyler, “Statistical analysis for the angular central Gaussian distribution on the sphere,” Biometrika, vol. 74, no. 3, pp. 579–589, 1987.
  30. David E. Tyler, “A distribution-free M-estimator of multivariate scatter,” The annals of Statistics, pp. 234–251, 1987.
  31. “Hyperspectral anomaly detectors using robust estimators,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 9, no. 2, pp. 720–731, 2015.
  32. “Performance analysis of covariance matrix estimates in impulsive noise,” IEEE Transactions on signal processing, vol. 56, no. 6, pp. 2206–2217, 2008.
  33. “Estimation of shape parameter for generalized gaussian distributions in subband decompositions of video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 5, no. 1, pp. 52–56, 1995.
  34. Kai-Sheng Song, “A globally convergent and consistent method for estimating the shape parameter of a generalized gaussian distribution,” IEEE Transactions on Information Theory, vol. 52, no. 2, pp. 510–527, 2006.
  35. “A practical procedure to estimate the shape parameter in the generalized gaussian distribution,” technique report I-01-18_eng. pdf, available through http://www. cimat. mx/reportes/enlinea/I-01-18_eng. pdf, vol. 1, 2003.
  36. “Approximated fast estimator for the shape parameter of generalized gaussian distribution,” Signal processing, vol. 86, no. 2, pp. 205–211, 2006.
  37. “Improved estimation of the degree of freedom parameter of multivariate t𝑡titalic_t-distribution,” in 2021 29th European Signal Processing Conference (EUSIPCO). IEEE, 2021, pp. 860–864.
  38. “Reliable methods for estimating the k𝑘kitalic_k-distribution shape parameter,” IEEE Journal of Oceanic Engineering, vol. 35, no. 2, pp. 288–302, 2010.
  39. “Regularization and variable selection via the elastic net,” Journal of the Royal Statistical Society. Series B (Statistical Methodology), vol. 67, no. 2, pp. 301–320, 2005.
  40. “Perspective functions: Proximal calculus and applications in high-dimensional statistics,” Journal of Mathematical Analysis and Applications, vol. 457, no. 2, pp. 1283–1306, 2018, Special Issue on Convex Analysis and Optimization: New Trends in Theory and Applications.
  41. Convex Analysis and Monotone Operator Theory in Hilbert Space, CMS Books in Mathematics. Springer International Publishing, 2017.
  42. Proximal Splitting Methods in Signal Processing, pp. 185–212, Springer New York, New York, NY, 2011.
  43. “A first-order primal-dual algorithm for convex problems with applications to imaging,” Journal of Mathematical Imaging and Vision, vol. 40, pp. 120–145, 2011.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now