Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generalized Score Matching (2303.08987v2)

Published 15 Mar 2023 in stat.ME, math.ST, stat.AP, stat.CO, and stat.TH

Abstract: Score matching is an estimation procedure that has been developed for statistical models whose probability density function is known up to proportionality but whose normalizing constant is intractable, so that maximum likelihood is difficult or impossible to implement. To date, applications of score matching have focused more on continuous IID models. Motivated by various data modelling problems, this article proposes a unified asymptotic theory of generalized score matching developed under the independence assumption, covering both continuous and discrete response data, thereby giving a sound basis for score-matchingbased inference. Real data analyses and simulation studies provide convincing evidence of strong practical performance of the proposed methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society: Series B (Methodological), 36(2):192–225.
  2. Handbook of Markov Chain Monte Carlo. CRC Press, Boca Raton, FL.
  3. Covariate-Adjusted Precision Matrix Estimation with an Application in Genetical Genomics. Biometrika, 100:139–156.
  4. Efficient Bayesian Inference for COM-Poisson Regression Models. Statistics and Computing, 28:595–608.
  5. A Sparse Ising Model with Covariates. Biometrics, 70:943–953.
  6. DasGupta, A. (2008). Asymptotic Theory of Statistics and Probability. Springer Science & Business Media, Mainz, Germany.
  7. Efron, B. (1986). Double Exponential Families and Their Use in Generalized Linear Regression. Journal of the American Statistical Association, 81(395):709–721.
  8. An Introduction to the Bootstrap. CRC Press, Boca Raton, FL.
  9. Ellis, R. S. (2006). Entropy, Large Deviations, and Statistical Mechanics, volume 1431. Taylor & Francis.
  10. Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties. Journal of the American statistical Association, 96:1348–1360.
  11. Energy-Based Models for Deep Probabilistic Regression. In European Conference on Computer Vision, pages 325–343. Springer.
  12. Huang, A. (2017). Mean-Parametrized Conway–Maxwell–Poisson Regression Models for Dispersed Counts. Statistical Modelling, 17:359–380.
  13. Huber, M. (2015). Approximation Algorithms for the Normalizing Constant of Gibbs Distributions. The Annals of Applied Probability, 25:974–985.
  14. Hyvärinen, A. (2005). Estimation of Non-Normalized Statistical Models by Score Matching. Journal of Machine Learning Research, 6:695–709.
  15. Hyvärinen, A. (2007). Some Extensions of Score Matching. Computational Statistics & Data Analysis, 51:2499–2512.
  16. Ising, E. (1924). Beitrag Zur Theorie Des Ferro-Und Paramagnetismus. PhD thesis, Grefe & Tiedemann.
  17. Johnson, O. (2004). Information Theory and the Central Limit Theorem. World Scientific, Singapore.
  18. The Construction of Multivariate Distributions From Markov Random Fields. Journal of Multivariate Analysis, 73(2):199–220.
  19. Klenke, A. (2013). Probability Theory: A Comprehensive Course. Springer Science & Business Media, Mainz, Germany.
  20. Analysis of Covariance Structures with Independent and Non-Identically Distributed Observations. Statistica Sinica, 8:543–557.
  21. Long, J. S. (1990). The Origins of Sex Differences in Science. Social Forces, 68:1297–1316.
  22. Regression Models for Categorical Dependent Variables Using Stata, volume 7. Stata Press, College Station, TX.
  23. Lyu, S. (2009). Interpretation and generalization of score matching. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 359–366.
  24. Score Matching Estimators for Directional Distributions. arXiv preprint arXiv:1604.08470.
  25. Generalized bayesian inference for discrete intractable likelihood. Journal of the American Statistical Association, pages 1–11.
  26. Concrete Score Matching: Generalized Score Matching for Discrete Data. Advances in Neural Information Processing Systems, 35:34532–34545.
  27. A Simplex Method for Function Minimization. The Computer Journal, 7:308–313.
  28. Efficient Learning of Generative Models via Finite-Difference Score Matching. Advances in Neural Information Processing Systems, 33:19175–19188.
  29. R Core Team (2013). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0.
  30. Chemistry of Europe’s Agricultural Soils, Part A.
  31. Chemistry of Europe’s Agricultural Soils–Part B: General Background Information and Further Analysis of the GEMAS Data Set. Geologisches Jahrbuch (Reihe B, 103:352.
  32. Regression for Compositional Data by Using Distributions Defined on the Hypersphere. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(3):351–375.
  33. Score Matching for Compositional Distributions. Journal of the American Statistical Association, pages 1–13.
  34. Robust Score Matching for Compositional Data. Statistics and Computing (to appear).
  35. A Flexible Regression Model for Count Data. The Annals of Applied Statistics, 4:943–961.
  36. A Useful Distribution for Fitting Discrete Data: Revival of the Conway-Maxwell-Poisson Distribution. Journal of the Royal Statistical Society: Series C (Applied Statistics), 54:127–142.
  37. Improved Techniques for Training Score-Based Generative Models. Advances in neural information processing systems, 33:12438–12448.
  38. Sliced Score Matching: A Scalable Approach to Density and Score Estimation. In Uncertainty in Artificial Intelligence, pages 574–584. PMLR.
  39. Score-Based Generative Modeling Through Stochastic Differential Equations. arXiv preprint arXiv:2011.13456.
  40. Stephens, M. A. (1982). Use of the von Mises Distribution to Analyse Continuous Proportions. Biometrika, 69(1):197–203.
  41. Vector-Space Markov Random Fields via Exponential Families. In International Conference on Machine Learning, pages 684–692. PMLR.
  42. Score-Based Generative Modeling in Latent Space. Advances in Neural Information Processing Systems, 34:11287–11302.
  43. van der Vaart, A. W. (2000). Asymptotic Statistics, volume 3. Cambridge University Press, Cambridge, UK.
  44. An overview of composite likelihood methods. Statistica Sinica, pages 5–42.
  45. Vincent, P. (2011). A Connection between Score Matching and Denoising Autoencoders. Neural Computation, 23:1661–1674.
  46. Windham, M. P. (1995). Robustifying Model Fitting. Journal of the Royal Statistical Society. Series B, 57, 599–609.
  47. A Sparse Conditional Gaussian Graphical Model for Analysis of Genetical Genomics Data. The Annals of Applied Statistics, 5:2630.
  48. Simultaneous Inference for Pairwise Graphical Models with Generalized Score Matching. J. Mach. Learn. Res., 21(91):1–51.
  49. Statistical Inference for Pairwise Graphical Models Using Score Matching. Advances in Neural Information Processing Systems, 29.
  50. Generalized Score Matching for Non-Negative Data. Journal of Machine Learning Research, 20:2779–2848.
  51. Interaction Models and Generalized Score Matching for Compositional Data. arXiv preprint arXiv:2109.04671.
  52. Community Detection with Dependent Connectivity. The Annals of Statistics, 49:2378–2428.
  53. Dimension Reduction for Covariates in Network Data. Biometrika, 109:85–102.
  54. Network Influence Analysis. Statistica Sinica, 31(4):1727–1748.
  55. Davidson, J. (1994). Stochastic Limit Theory: An Introduction for Econometricians. OUP Oxford.
  56. On the Rate of Convergence in the Central Limit Theorem for Arrays of Random Vectors. Statistics & Probability Letters, 158:108671.
  57. tmvtnorm: A package for the Truncated Multivariate Normal Distribution. Sigma, 2:1–25.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets