Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Implicit Gaussian process representation of vector fields over arbitrary latent manifolds (2309.16746v2)

Published 28 Sep 2023 in cs.LG, cs.MS, physics.data-an, q-bio.QM, and stat.ML

Abstract: Gaussian processes (GPs) are popular nonparametric statistical models for learning unknown functions and quantifying the spatiotemporal uncertainty in data. Recent works have extended GPs to model scalar and vector quantities distributed over non-Euclidean domains, including smooth manifolds appearing in numerous fields such as computer vision, dynamical systems, and neuroscience. However, these approaches assume that the manifold underlying the data is known, limiting their practical utility. We introduce RVGP, a generalisation of GPs for learning vector signals over latent Riemannian manifolds. Our method uses positional encoding with eigenfunctions of the connection Laplacian, associated with the tangent bundle, readily derived from common graph-based approximation of data. We demonstrate that RVGP possesses global regularity over the manifold, which allows it to super-resolve and inpaint vector fields while preserving singularities. Furthermore, we use RVGP to reconstruct high-density neural dynamics derived from low-density EEG recordings in healthy individuals and Alzheimer's patients. We show that vector field singularities are important disease markers and that their reconstruction leads to a comparable classification accuracy of disease states to high-density recordings. Thus, our method overcomes a significant practical limitation in experimental and clinical applications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Sheaf neural networks with connection laplacians, 2022.
  2. Tangent bundle convolutional learning: from manifolds to cellular sheaves and back. arXiv preprint arXiv:2303.11323, 2023.
  3. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput., 15(6):1373–1396, 2003. ISSN 08997667. doi: 10.1162/089976603321780317.
  4. Hans Berger. Über das elektrenkephalogramm des menschen. DMW-Deutsche Medizinische Wochenschrift, 60(51):1947–1949, 1934.
  5. Vergne M. Berline N., Getzler E. Heat kernels and Dirac operators. Springer, 2nd. edition, 1996.
  6. Eeg coherence in alzheimer disease. Electroencephalography and clinical neurophysiology, 90(3):242–245, 1994.
  7. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNNs. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
  8. Matérn gaussian processes on riemannian manifolds. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp.  12426–12437. Curran Associates, Inc., 2020.
  9. Matérn gaussian processes on graphs. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 2593–2601. PMLR, 13–15 Apr 2021.
  10. Geometric Deep Learning: Going beyond Euclidean data. IEEE Signal Process. Mag., 34(4):18–42, 2017. ISSN 10535888. doi: 10.1109/MSP.2017.2693418.
  11. Parallel transport unfolding: A connection-based manifold learning approach. SIAM Journal on Applied Algebra and Geometry, 3(2):266–291, 2019. doi: 10.1137/18M1196133.
  12. Catherine J Chu. High density eeg—what do we have to lose? Clinical neurophysiology: official journal of the International Federation of Clinical Neurophysiology, 126(3):433, 2015.
  13. Fan Chung. Spectral Graph Theocy, volume 92 of American Mathematical Soc. American Mathematical Soc., 1997. ISBN 0821803158.
  14. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. Proc. Natl. Acad. Sci. U. S. A., 102(21):7426–7431, 2005. doi: 10.1073/pnas.0500334102.
  15. Diagnosis of alzheimer’s disease from eeg signals: where are we standing? Current Alzheimer Research, 7(6):487–505, 2010.
  16. Convolutional Neural Networks on Graphs with. In NIPS’ 2016, pp.  3844–3852, 2016. ISBN 9781510838819.
  17. Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. Journal of neuroscience methods, 134(1):9–21, 2004.
  18. Temporal alignment and latent Gaussian process factor inference in population spike trains. Adv. Neural Inf. Process. Syst., 2018-December(NeurIPS):10445–10455, 2018. ISSN 10495258.
  19. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4):983–1049, 2016.
  20. Toroidal topology of population activity in grid cells. Nature, 602(7895):123–128, 2022. doi: 10.1038/s41586-021-04268-7.
  21. Unfolding the multiscale structure of networks with dynamical Ollivier-Ricci curvature. Nat. Commun., 12(1):1–11, 2021. ISSN 20411723. doi: 10.1038/s41467-021-24884-1.
  22. LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals. Nat. Methods, 18(8):975–981, 2021. ISSN 1548-7091. doi: 10.1038/s41592-021-01226-z.
  23. Interpretable statistical representations of neural population dynamics and geometry. arXiv:2304.03376, 2023.
  24. Generalised implicit neural representations. Advances in Neural Information Processing Systems, 35:30446–30458, 2022.
  25. Vector-valued gaussian processes on riemannian manifolds via gauge independent projected kernels. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp.  17160–17169. Curran Associates, Inc., 2021.
  26. On some aspects of the cnem implementation in 3d in order to simulate high speed machining or shearing. Computers & Structures, 89(11-12):940–958, 2011.
  27. Manifold GPLVMs for discovering non-Euclidean latent structure in neural data. Adv. Neural Inf. Process. Syst., 2020-Decem:1–22, 2020. ISSN 10495258.
  28. Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystallographica Section A, 32(5):922–923, September 1976. doi: 10.1107/S0567739476001873.
  29. Attractor and integrator networks in the brain. Nat. Rev. Neurosci., 23(12):744–766, 2022. doi: 10.1038/s41583-022-00642-0.
  30. Semi-Supervised Classification with Graph Convolutional Networks. ICLR, 2017. ISSN 10963626. URL arXiv.org.
  31. Globally optimal direction fields. ACM Trans. Graph., 32(4), 2013. doi: 10.1145/2461912.2462005.
  32. Stripe patterns on surfaces. ACM Trans. Graph., 34(4), 2015. ISSN 15577368. doi: 10.1145/2767000.
  33. Intrinsic Neural Fields: Learning Functions on Manifolds. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 13662 LNCS:622–639, 2022. ISSN 16113349. doi: 10.1007/978-3-031-20086-1˙36.
  34. RNA velocity of single cells. Nature, 560(7719):494–498, 2018. ISSN 14764687. doi: 10.1038/s41586-018-0414-6.
  35. Yaron Lipman. Phase Transitions, Distance Functions, and Implicit Neural Representations. Proc. Mach. Learn. Res., 139:6702–6712, 2021. ISSN 26403498.
  36. Wrapped gaussian process regression on riemannian manifolds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  37. Probabilistic Riemannian submanifold learning with wrapped Gaussian process latent variable models. AISTATS 2019 - 22nd Int. Conf. Artif. Intell. Stat., 89, 2020.
  38. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
  39. Individual analysis of eeg frequency and band power in mild alzheimer’s disease. Clinical Neurophysiology, 115(2):299–308, 2004.
  40. Graph Signal Processing: Overview, Challenges, and Applications. Proc. IEEE, 106(5):808–828, 2018. ISSN 15582256. doi: 10.1109/JPROC.2018.2820126.
  41. Semi-supervised classification on graphs using explicit diffusion dynamics. Foundations of Data Science, 2(1):19–33, 2020.
  42. Spherical splines for scalp potential and current density mapping. Electroencephalography and clinical neurophysiology, 72(2):184–187, 1989.
  43. PointNet: Deep learning on point sets for 3D classification and segmentation. Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, 2017-January:77–85, 2017. doi: 10.1109/CVPR.2017.16.
  44. Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT press, 2006. ISBN 026218253X.
  45. Metastable brain waves. Nature communications, 10(1):1056, 2019.
  46. Very high density eeg elucidates spatiotemporal aspects of early visual processing. Scientific reports, 7(1):16248, 2017.
  47. Traveling waves in visual cortex. Neuron, 75(2):218–229, 2012.
  48. Subcortical electrophysiological activity is detectable with high-density eeg source imaging. Nature communications, 10(1):753, 2019.
  49. The vector heat method. ACM Trans. Graph., 38(3), 2019. doi: 10.1145/3243651.
  50. Dreaming in nrem sleep: a high-density eeg study of slow waves and spindles. Journal of Neuroscience, 38(43):9175–9185, 2018.
  51. Vector diffusion maps and the connection Laplacian. Commun. Pure Appl. Math., 65(8):1067–1144, 2012. doi: 10.1002/cpa.21395.
  52. Spectral convergence of the connection Laplacian from random samples. Inf. Inference, 6(1):58–123, 2017. doi: 10.1093/imaiai/iaw016.
  53. Implicit neural representations with periodic activation functions. Adv. Neural Inf. Process. Syst., 2020-December(NeurIPS):1–12, 2020. ISSN 10495258.
  54. Hilbert space methods for reduced-rank Gaussian process regression. Stat. Comput., 30(2):419–446, 2020. ISSN 15731375. doi: 10.1007/s11222-019-09886-w.
  55. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput., 25(3):626–649, 2013. doi: 10.1162/NECO˙a˙00409.
  56. Gabriel Taubin. A signal processing approach to fair surface design. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’95, pp.  351–358, New York, NY, USA, 1995. Association for Computing Machinery. ISBN 0897917014. doi: 10.1145/218380.218473.
  57. Joshua B Tenenbaum. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science (80-. )., 290(5500):2319–2323, 2000. doi: 10.1126/science.290.5500.2319.
  58. Michalis Titsias. Variational learning of inducing variables in sparse gaussian processes. In David van Dyk and Max Welling (eds.), Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, volume 5 of Proceedings of Machine Learning Research, pp.  567–574, Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA, 16–18 Apr 2009. PMLR.
  59. Peter Whittle. Stochastic-processes in several dimensions. Bulletin of the International Statistical Institute, 40(2):974–994, 1963.
  60. Pathwise conditioning of gaussian processes. J. Mach. Learn. Res., 22:1–47, 2021.
  61. Interacting spiral wave patterns underlie complex brain dynamics and are related to cognitive processing. Nature Human Behaviour, pp.  1–20, 2023.
  62. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. J. Neurophysiol., 102(1):614–635, 2009. ISSN 15221598. doi: 10.1152/jn.90941.2008.
  63. Biomarkers for alzheimer’s disease—preparing for a new era of disease-modifying therapies. Molecular psychiatry, 26(1):296–308, 2021.
Citations (4)

Summary

We haven't generated a summary for this paper yet.