Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference (2407.02476v1)

Published 2 Jul 2024 in cs.LG and stat.ML

Abstract: The Multi-Output Gaussian Process is is a popular tool for modelling data from multiple sources. A typical choice to build a covariance function for a MOGP is the Linear Model of Coregionalization (LMC) which parametrically models the covariance between outputs. The Latent Variable MOGP (LV-MOGP) generalises this idea by modelling the covariance between outputs using a kernel applied to latent variables, one per output, leading to a flexible MOGP model that allows efficient generalization to new outputs with few data points. Computational complexity in LV-MOGP grows linearly with the number of outputs, which makes it unsuitable for problems with a large number of outputs. In this paper, we propose a stochastic variational inference approach for the LV-MOGP that allows mini-batches for both inputs and outputs, making computational complexity per training iteration independent of the number of outputs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
  2. Kernels for vector-valued functions: A review. Foundations and Trends® in Machine Learning, 4(3):195–266, 2012.
  3. Hans Wackernagel. Multivariate geostatistics: an introduction with applications. Springer Science & Business Media, 2003.
  4. Heterogeneous multi-output gaussian process prediction. Advances in Neural Information Processing Systems, 31, 2018.
  5. Multi-task learning for aggregated data using gaussian processes. Advances in Neural Information Processing Systems, 32, 2019.
  6. Latent variable multi-output gaussian processes for hierarchical datasets. arXiv preprint arXiv:2308.16822, 2023.
  7. Mining geostatistics. 1976.
  8. Dave Higdon. Space and space-time modeling using process convolutions. In Quantitative methods for current environmental issues, pages 37–56. Springer, 2002.
  9. Efficient modeling of latent information in supervised learning using gaussian processes. Advances in Neural Information Processing Systems, 30, 2017.
  10. Multi-task gaussian process prediction. Advances in neural information processing systems, 20, 2007.
  11. Collaborative multi-output gaussian processes. In UAI, pages 643–652, 2014.
  12. Generalised gaussian process latent variable models (gplvm) with stochastic variational inference. arXiv preprint arXiv:2202.12979, 2022.
  13. Stochastic variational inference. Journal of Machine Learning Research, 2013.
  14. Gaussian processes for big data. arXiv preprint arXiv:1309.6835, 2013.
  15. Computationally efficient convolved multiple output gaussian processes. The Journal of Machine Learning Research, 12:1459–1500, 2011.
  16. Bayesian gaussian process latent variable model. In Proceedings of the thirteenth International Conference on Artificial Intelligence and Statistics, pages 844–851. JMLR Workshop and Conference Proceedings, 2010.
  17. Michalis Titsias. Variational learning of inducing variables in sparse gaussian processes. In Artificial Intelligence and Statistics, pages 567–574. PMLR, 2009.
  18. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  19. A note on gauss—hermite quadrature. Biometrika, 81(3):624–629, 1994.
  20. Latent gaussian process with composite likelihoods and numerical quadrature. In International Conference on Artificial Intelligence and Statistics, pages 3718–3726. PMLR, 2021.
  21. Doubly stochastic variational bayes for non-conjugate inference. In International Conference on Machine Learning, pages 1971–1979. PMLR, 2014.
  22. Doubly stochastic variational inference for deep gaussian processes. Advances in Neural Information Processing Systems, 30, 2017.
  23. A unifying view of sparse approximate gaussian process regression. The Journal of Machine Learning Research, 6:1939–1959, 2005.
  24. When gaussian process meets big data: A review of scalable gps. IEEE Transactions on Neural Networks and Learning Systems, 31(11):4405–4423, 2020.
  25. Orthogonally decoupled variational gaussian processes. Advances in Neural Information Processing Systems, 31, 2018.
  26. Sparse within sparse gaussian processes using neighbor information. In International Conference on Machine Learning, pages 10369–10378. PMLR, 2021.
  27. Variational nearest neighbor gaussian process. In International Conference on Machine Learning, pages 24114–24130. PMLR, 2022.
  28. Adaptive cholesky gaussian processes. In International Conference on Artificial Intelligence and Statistics, pages 408–452. PMLR, 2023.
  29. Efficient multioutput gaussian processes through variational inducing kernels. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 25–32. JMLR Workshop and Conference Proceedings, 2010.
  30. Scalable exact inference in multi-output gaussian processes. In International Conference on Machine Learning, pages 1190–1201. PMLR, 2020.
  31. Sparse convolved gaussian processes for multi-output regression. Advances in Neural Information Processing Systems, 21, 2008.
  32. 2014–2015 crimes reported in all 5 boroughs of new york city. https://www.kaggle.com/adamschroeder/crimes-new-york-city, 2015. Accessed: 2024-05-19.
  33. Efficient inference in multi-task cox process models. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 537–546. PMLR, 2019.
  34. Spatio-temporal variational gaussian processes. Advances in Neural Information Processing Systems, 34:23621–23633, 2021.
  35. Gru-ode-bayes: Continuous modeling of sporadically-observed time series. Advances in Neural Information Processing Systems, 32, 2019.
  36. Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science, 353(6294):78–82, 2016.
  37. 10101010X Genomics. https://www.10xgenomics.com/resources/datasets/human-prostate-cancer-adenocarcinoma-with-invasive-carcinoma-ffpe-1-standard-1-3-0.
  38. Scanpy: large-scale single-cell gene expression data analysis. Genome biology, 19(1):1–5, 2018.
  39. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  40. Long-term daily climate records from stations across the contiguous united states, 2015.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets