Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Fréchet Regression (2407.21407v1)

Published 31 Jul 2024 in stat.ME and cs.LG

Abstract: Advancements in modern science have led to the increasing availability of non-Euclidean data in metric spaces. This paper addresses the challenge of modeling relationships between non-Euclidean responses and multivariate Euclidean predictors. We propose a flexible regression model capable of handling high-dimensional predictors without imposing parametric assumptions. Two primary challenges are addressed: the curse of dimensionality in nonparametric regression and the absence of linear structure in general metric spaces. The former is tackled using deep neural networks, while for the latter we demonstrate the feasibility of mapping the metric space where responses reside to a low-dimensional Euclidean space using manifold learning. We introduce a reverse mapping approach, employing local Fr\'echet regression, to map the low-dimensional manifold representations back to objects in the original metric space. We develop a theoretical framework, investigating the convergence rate of deep neural networks under dependent sub-Gaussian noise with bias. The convergence rate of the proposed regression model is then obtained by expanding the scope of local Fr\'echet regression to accommodate multivariate predictors in the presence of errors in predictors. Simulations and case studies show that the proposed model outperforms existing methods for non-Euclidean responses, focusing on the special cases of probability measures and networks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Neural Network Learning: Theoretical Foundations, Volume 9. Cambridge University Press.
  2. On deep learning as a remedy for the curse of dimensionality in nonparametric regression. Annals of Statistics 47(4), 2261–2285.
  3. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 1373–1396.
  4. Bhattacharjee, S. and H.-G. Müller (2023). Single index Fréchet regression. Annals of Statistics 51(4), 1770–1798.
  5. Geodesic PCA in the Wasserstein space by convex PCA. Annales de l’Institut Henri Poincaré B: Probability and Statistics 53, 1–26.
  6. A supervised deep learning method for nonparametric density estimation. arXiv preprint arXiv:2306.10471.
  7. Chen, D. and H.-G. Müller (2012). Nonlinear manifold representations for functional data. Annals of Statistics 40, 1–29.
  8. Chen, Y. and H.-G. Müller (2022). Uniform convergence of local Fréchet regression, with applications to locating extrema and time warping for metric-space valued trajectories. Annals of Statistics 50(3), 1573–1592.
  9. frechet: Statistical Analysis for Random Objects and Non-Euclidean Data. R package version 0.3.0.
  10. Diffusion maps. Applied and Computational Harmonic Analysis 21(1), 5–30.
  11. Delicado, P. (2011). Dimensionality reduction when data are density functions. Computational Statistics & Data Analysis 55, 401–420.
  12. Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik 1, 269–271.
  13. Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. Annals of Applied Statistics 3, 1102–1123.
  14. Metric statistics: Exploration and inference for random objects with distance profiles. Annals of Statistics 52(2), 757–792.
  15. Local Polynomial Modelling and its Applications. London: Chapman & Hall.
  16. Faraway, J. J. (2014). Regression for non-Euclidean data using distance matrices. Journal of Applied Statistics 41, 2342–2357.
  17. Fréchet, M. (1948). Les éléments aléatoires de nature quelconque dans un espace distancié. Annales de l’Institut Henri Poincaré 10(4), 215–310.
  18. Elements of Dimensionality Reduction and Manifold Learning. Springer New York.
  19. Deep Learning. MIT Press.
  20. Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems 28.
  21. Hein, M. (2009). Robust Nonparametric Regression with Metric-Space valued Output. In Advances in Neural Information Processing Systems, pp.  718–726.
  22. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  23. On the rate of convergence of fully connected deep neural network regression estimates. Annals of Statistics 49(4), 2231–2249.
  24. Adaptive deep learning for nonparametric time series regression. Bernoulli, just–accepted.
  25. Deep learning. Nature 521(7553), 436–444.
  26. Müller, H.-G. (2016). Peter Hall, Functional Data Analysis and Random Objects. Annals of Statistics 44, 1867–1887.
  27. Nair, V. and G. E. Hinton (2010). Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning, pp.  807–814.
  28. Principal component analysis and the locus of the Fréchet mean in the space of phylogenetic trees. Biometrika 104(4), 901–922.
  29. Petersen, A. and H.-G. Müller (2019). Fréchet regression for random objects with Euclidean predictors. Annals of Statistics 47(2), 691–719.
  30. Modeling probability density functions as data objects. Econometrics and Statistics 21, 159–178.
  31. Roweis, S. T. and L. K. Saul (2000). Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326.
  32. Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with ReLU activation function. Annals of Statistics 48(4), 1875–1897.
  33. Schötz, C. (2022). Nonparametric regression in nonstandard spaces. Electronic Journal of Statistics 16(2), 4679–4741.
  34. Errors-in-variables Fréchet regression with low-rank covariate approximation. In Advances in Neural Information Processing Systems.
  35. Training sparse neural networks. In Computer Vision and Pattern Recognition, pp.  138–145.
  36. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15(1), 1929–1958.
  37. Sturm, K.-T. (2003). Probability measures on metric spaces of nonpositive curvature. Heat Kernels and Analysis on Manifolds, Graphs, and Metric Spaces (Paris, 2002). Contemp. Math., 338. Amer. Math. Soc., Providence, RI 338, 357–390.
  38. A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323.
  39. Variable selection for global Fréchet regression. Journal of the American Statistical Association 118(542), 1023–1037.
  40. Van der Vaart, A. and J. Wellner (2023). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer New York.
  41. Villani, C. (2003). Topics in Optimal Transportation. American Mathematical Society.
  42. Fréchet sufficient dimension reduction for random objects. Biometrika 109(4), 975–992.
  43. Wasserstein autoregressive models for density time series. Journal of Time Series Analysis 43(2), 30–52.
  44. Dimension reduction for Fréchet regression. Journal of the American Statistical Association (just-accepted), 1–15.
  45. Deep learning for the partially linear Cox model. Annals of Statistics 50(3), 1348–1375.
  46. Zhou, Y. and H.-G. Müller (2022). Network regression with graph Laplacians. Journal of Machine Learning Research 23, 1–41.
  47. Zhou, Y. and H.-G. Müller (2023). Wasserstein regression with empirical measures and density estimation for sparse data. arXiv preprint arXiv:2308.12540.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets