Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Closed-Form Expressions for the Fisher-Rao Distance (2304.14885v3)

Published 28 Apr 2023 in math.ST, cs.IT, math.DG, math.IT, and stat.TH

Abstract: The Fisher-Rao distance is the geodesic distance between probability distributions in a statistical manifold equipped with the Fisher metric, which is a natural choice of Riemannian metric on such manifolds. It has recently been applied to supervised and unsupervised problems in machine learning, in various contexts. Finding closed-form expressions for the Fisher-Rao distance is generally a non-trivial task, and those are only available for a few families of probability distributions. In this survey, we collect examples of closed-form expressions for the Fisher-Rao distance of both discrete and continuous distributions, aiming to present them in a unified and accessible language. In doing so, we also: illustrate the relation between negative multinomial distributions and the hyperbolic model, include a few new examples, and write a few more in the standard form of elliptical distributions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (72)
  1. S. Amari and H. Nagaoka. Methods of Information Geometry. American Mathematical Society, Providence, RI, USA, 2000.
  2. A. Andai. On the geometry of generalized Gaussian distributions. Journal of Multivariate Analysis, 100(4):777–793, 2009.
  3. Pulling back information geometry. In G. Camps-Valls, F. J. R. Ruiz, and I. Valera, editors, The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 4872–4894, 2022.
  4. Information Geometry: Near Randomness and Near Independence. Springer, Heidelberg, Germany, 2008.
  5. C. Atkinson and A. F. S. Mitchell. Rao’s distance measure. Sankhyā: The Indian Journal of Statistics, Series A, 43(3):345–365, 1981.
  6. Information geometry and sufficient statistics. Probability Theory and Related Fields, 162:327–364, 2015.
  7. Information Geometry. Springer, Cham, Switzerland, 2017.
  8. Elliptical Wishart distribution: Maximum likelihood estimator from information geometry. In 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.
  9. A. F. Beardon. The Geometry of Discrete Groups. Springer, New York, NY, USA, 1983.
  10. Geodesic estimation in elliptical distributions. Journal of Multivariate Analysis, 63(1):35–46, 1997.
  11. R. Bhatia. Positive Definite Matrices. Princeton University Press, Princeton, NJ, USA, 2007.
  12. The Fisher-Rao geometry of CES distributions. arXiv:2310.01032, 2023.
  13. Intrinsic Cramér–Rao bounds for scatter and shape matrices estimation in CES distributions. IEEE Signal Processing Letters, 26(2):262–266, 2019.
  14. J. Burbea. Informative geometry of probability spaces. Expositiones Mathematicae, 4:347–378, 1986.
  15. Some remarks on the information geometry of the Gamma distribution. Communications in Statistics—Theory and Methods, 31(11):1959–1975, 2002.
  16. O. Calin and C. Udrişte. Geometric Modeling in Probability and Statistics. Springer, Cham, Switzerland, 2014.
  17. M. Calvo and J. M. Oller. A distance between multivariate normal distributions based in an embedding into the Siegel group. Journal of Multivariate Analysis, 35(2):223–242, 1990.
  18. M. Calvo and J. M. Oller. An explicit solution of information geodesic equations for the multivariate model. Statistics & Decisions, 9(1–2):119–138, 1991.
  19. M. Calvo and J. M. Oller. A distance between elliptical distributions based in an embedding into the Siegel group. Journal of Computational and Applied Mathematics, 145(2):319–334, 2002.
  20. Hyperbolic geometry. In S. Levy, editor, Flavors of Geometry, volume 31 of MSRI Publications. Cambridge University Press, Cambridge, UK; New York, NY, USA, 1997.
  21. Upper bounds for Rao distance on the manifold of multivariate elliptical distributions. Automatica, 129:109604, 2021.
  22. Fisher information distance: A geometrical reading. Discrete Applied Mathematics, 197:59–69, 2015.
  23. Analytical properties of generalized Gaussian distributions. Journal of Statistical Distributions and Applications, 5(6).
  24. F. D’Andrea and P. Martinetti. On Pythagoras theorem for products of spectral triples. Letters in Mathematical Physics, 103:469–492, 2012.
  25. Symmetric Multivariate and Related Distributions. Chapman and Hall, London, UK; New York, NY, USA, 1990.
  26. A shape distance based on the fisher–rao metric and its application for shapes clustering. Physica A: Statistical Mechanics and its Applications, 487:93–102, 2017.
  27. Bayesian Data Analysis. Chapman and Hall/CRC, Boca Raton, FL, USA, 2013.
  28. Igeood: An information geometry approach to out-of-distribution detection. In International Conference on Learning Representations, 2022.
  29. Matrix Variate Distributions. Chapman and Hall/CRC, Boca Raton, FL, USA, 2000.
  30. M. Han and F. C. Park. DTI segmentation and fiber tracking using metrics on multivariate normal distributions. Journal of Mathematical Imaging and Vision, 49:317–334, 2014.
  31. H. Hotelling. Spaces of statistical parameters. Bulletin of the American Mathematical Society, 36:191, 1930.
  32. Geometrical Foundations of Asymptotic Inference. Wiley, New York, NY, USA, 1997.
  33. G. Khan and J. Zhang. A hall of statistical mirrors. Asian Journal of Mathematics, 26(6):809–846, 2022.
  34. W. Klingenberg. A Course in Differential Geometry. Springer, New York, NY, USA, 1978.
  35. S. Kotz and S. Nadarajah. Extreme Value Distributions: Theory and Applications. Imperial College Press, London, UK, 2000.
  36. W. J. Krzanowski. Rao’s distance between normal populations that have common principal components. Biometrics, (4):1467–1471, 1996.
  37. S. L. Lauritzen. Statistical manifolds. In S. Amari, O. E. Barndorff-Nielsen, R. E. Kass, S. L. Lauritzen, and C. R. Rao, editors, Differential Geometry in Statistical Inference, volume 10, chapter 4. Institute of Mathematical Statistics, Hayward, CA, USA, 1987.
  38. Parametric information geometry with the package Geomstats. ACM Transactions on Mathematical Software, 49(4):1–26, 2023.
  39. Classifying histograms of medical data using information geometry of beta distributions. In 24th International Symposium on Mathematical Theory of Networks and Systems MTNS 2020, volume 54 of IFAC-PapersOnLine, pages 514–520, 2021.
  40. Fisher–Rao geometry of Dirichlet distributions. Differential Geometry and its Applications, 74:101702, 2021.
  41. Fisher–Rao geometry and Jeffreys prior for Pareto distribution. Communications in Statistics—Theory and Methods, 51(6):1895–1910, 2022.
  42. H. V. Lê. The uniqueness of the Fisher metric as information metric. Annals of the Institute of Statistical Mathematics, 69:879–896, 2017.
  43. J. R. Magnus and H. Neudecker. Symmetry, 0-1 matrices and Jacobians: A review. Econometric Theory, 2(2):157–190, 1986.
  44. C. A. Micchelli and L. Noakes. Rao distances. Journal of Multivariate Analysis, 92(1):97–115, 2005.
  45. A. Minarro and J. M. Oller. On a class of probability density functions and their information metric. Sankhyā: The Indian Journal of Statistics, Series A, 55(2):214–225, 1993.
  46. A. F. S. Mitchell. Statistical manifolds of univariate elliptic distributions. International Statistical Review, 56(1):1–16, 1988.
  47. The Mahalanobis distance and elliptic distributions. Biometrika, 72(2):464–467, 1985.
  48. The Fisher–Rao loss for learning under label noise. Information Geometry, 6:107–126, 2023.
  49. F. Nielsen. An elementary introduction to information geometry. Entropy, 22(10), 2020.
  50. F. Nielsen. On Voronoi diagrams on the information-geometric Cauchy manifolds. Entropy, 22(7), 2020.
  51. F. Nielsen. A simple approximation method for the Fisher–Rao distance between multivariate normal distributions. Entropy, 25(4), 2023.
  52. J. M. Oller. Information metric for extreme value and logistic probability distributions. Sankhyā: The Indian Journal of Statistics, Series A, 49(1):17–23, 1987.
  53. J. M. Oller. Some geometrical aspects of data analysis and statistics. In Y. Dodge, editor, Statistical Data Analysis and Inference, pages 41–58. North-Holland, Amsterdam, The Netherlands, 1989.
  54. Rao’s distance for negative multinomial distributions. Sankhyā: The Indian Journal of Statistics, Series A, 47(1):75–83, 1985.
  55. Adversarial robustness via Fisher–Rao regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):2698–2710, 2023.
  56. The Fisher–Rao distance between multivariate normal distributions: Special cases, bounds and applications. Entropy, 22(4), 2020.
  57. P. Prescott and A. T. Walden. Maximum likelihood estimation of the parameters of the generalized extreme-value distribution. Biometrika, 67(3):723–724, 1980.
  58. C. R. Rao. Information and the accuracy attainable in the estimation of statistical parameters. Bulletin of the Calcutta Mathematical Society, 37:81–91, 1945.
  59. C. R. Rao. Differential metrics in probability spaces. In S. Amari, O. E. Barndorff-Nielsen, R. E. Kass, S. L. Lauritzen, and C. R. Rao, editors, Differential Geometry in Statistical Inference, volume 10, chapter 1. Institute of Mathematical Statistics, Hayward, CA, USA, 1987.
  60. C. R. Rao. Information and the accuracy attainable in the estimation of statistical parameters. In S. Kotz and N. L. Johnson, editors, Breakthroughs in Statistics: Foundations and Basic Theory, pages 235–247. Springer, New York, NY, USA, 1992.
  61. J. G. Ratcliffe. Foundations of Hyperbolic Manifolds. Springer, New York, NY, USA, 2nd edition, 2006.
  62. The geometry of the generalized Gamma manifold and an application to medical imaging. Mathematics, 7(8), 2019.
  63. F. Reverter and J. M. Oller. Computing the Rao distance for Gamma distributions. Journal of Computational and Applied Mathematics, 157(1):155–167, 2003.
  64. Adversarial robustness with partial isometry. Entropy, 26(2), 2024.
  65. C. L. Siegel. Symplectic geometry. American Journal of Mathematics, 65(1):1–86, 1943.
  66. L. T. Skovgaard. A Riemannian geometry of the multivariate normal model. Scandinavian Journal of Statistics, 11(4):211–223, 1984.
  67. S. M. Stigler. The epic story of maximum likelihood. Statistical Science, 22(4):598–620, 2007.
  68. S. Taylor. Clustering financial return distributions using the Fisher information metric. Entropy, 21(2), 2019.
  69. A. B. Tsybakov. Introduction to Nonparametric Estimation. Springer, New York, NY, USA, 2009.
  70. G. Verdoolaege and P. Scheunders. On the geometry of multivariate generalized Gaussian models. Journal of Mathematical Imaging and Vision, 43:180–193, 2012.
  71. A. Villarroya and J. M. Oller. Statistical tests for the inverse gaussian distribution based on Rao distance. Sankhyā: The Indian Journal of Statistics, Series A, 55(1):80–103, 1993.
  72. D. Wauters and L. Vermeire. Intensive numerical and symbolic computing in parametric test theory. In W. Härdle and L. Simar, editors, Computer Intensive Methods in Statistics, pages 62–72, Heidelberg, Germany, 1993. Physica-Verlag.
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets