Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adaptive Conditional Quantile Neural Processes (2305.18777v3)

Published 30 May 2023 in cs.LG and stat.ML

Abstract: Neural processes are a family of probabilistic models that inherit the flexibility of neural networks to parameterize stochastic processes. Despite providing well-calibrated predictions, especially in regression problems, and quick adaptation to new tasks, the Gaussian assumption that is commonly used to represent the predictive likelihood fails to capture more complicated distributions such as multimodal ones. To overcome this limitation, we propose Conditional Quantile Neural Processes (CQNPs), a new member of the neural processes family, which exploits the attractive properties of quantile regression in modeling the distributions irrespective of their form. By introducing an extension of quantile regression where the model learns to focus on estimating informative quantiles, we show that the sampling efficiency and prediction accuracy can be further enhanced. Our experiments with real and synthetic datasets demonstrate substantial improvements in predictive performance compared to the baselines, and better modeling of heterogeneous distributions' characteristics such as multimodality.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Modelling heterogeneous distributions with an uncountable mixture of asymmetric laplacians. Advances in neural information processing systems, 32, 2019.
  2. Deep non-crossing quantiles through the partial derivative. In International Conference on Artificial Intelligence and Statistics, pages 7902–7914. PMLR, 2022.
  3. The gaussian neural process. arXiv preprint arXiv:2101.03606, 2021.
  4. Vector quantile regression: an optimal transport approach. 2016.
  5. Vector quantile regression beyond the specified case. Journal of Multivariate Analysis, 161:96–102, 2017.
  6. Statistical inference. Cengage Learning, 2021.
  7. Nonparametric modal regression. 2016.
  8. Monge–kantorovich depth, quantiles, ranks and signs. 2017.
  9. Implicit quantile networks for distributional reinforcement learning. In International conference on machine learning, pages 1096–1105. PMLR, 2018.
  10. Monte carlo methods of inference for implicit statistical models. Journal of the Royal Statistical Society. Series B (Methodological), 46(2):193–227, 1984. ISSN 00359246.
  11. On contrastive learning for likelihood-free inference. In International conference on machine learning, pages 2771–2781. PMLR, 2020.
  12. Modelling beyond regression functions: an application of multimodal regression to speed–flow data. Journal of the Royal Statistical Society Series C: Applied Statistics, 55(4):461–475, 2006.
  13. Latent bottlenecked attentive neural processes. In The Eleventh International Conference on Learning Representations, 2023.
  14. A statistical learning approach to modal regression. The Journal of Machine Learning Research, 21(1):25–59, 2020.
  15. Meta-learning stationary stochastic process prediction with convolutional neural processes. Advances in Neural Information Processing Systems, 33:8284–8295, 2020.
  16. Conditional neural processes. In International conference on machine learning, pages 1704–1713. PMLR, 2018a.
  17. Neural processes. arXiv preprint arXiv:1807.01622, 2018b.
  18. Function contrastive learning of transferable meta-representations. In International Conference on Machine Learning, pages 3755–3765. PMLR, 2021.
  19. Convolutional conditional neural processes. arXiv preprint arXiv:1910.13556, 2019.
  20. Versatile neural processes for learning implicit neural representations. arXiv preprint arXiv:2301.08883, 2023.
  21. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 297–304. JMLR Workshop and Conference Proceedings, 2010.
  22. Equivariant learning of stochastic fields: Gaussian processes and steerable conditional neural processes. In International Conference on Machine Learning, pages 4297–4307. PMLR, 2021.
  23. hdrcde: Highest Density Regions and Conditional Density Estimation, 2022. R package version 3.4.
  24. Attentive neural processes. arXiv preprint arXiv:1901.05761, 2019.
  25. Neural processes with stochastic attention: Paying more attention to the context dataset. arXiv preprint arXiv:2204.05449, 2022.
  26. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  27. Roger Koenker. Quantile regression, volume 38. Cambridge university press, 2005.
  28. Regression quantiles. Econometrica: journal of the Econometric Society, pages 33–50, 1978.
  29. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332–1338, 2015.
  30. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  31. Bootstrapping neural processes. Advances in neural information processing systems, 33:6606–6615, 2020.
  32. Stepwise multiple quantile regression estimation using non-crossing constraints. Statistics and its Interface, 2(3):299–310, 2009.
  33. Simultaneous multiple non-crossing quantile regression estimation using kernel constraints. Journal of nonparametric statistics, 23(2):415–437, 2011.
  34. Variational implicit processes. In International Conference on Machine Learning, pages 4222–4233. PMLR, 2019.
  35. Practical conditional neural processes via tractable dependent predictions. arXiv preprint arXiv:2203.08775, 2022.
  36. On contrastive representations of stochastic processes. Advances in Neural Information Processing Systems, 34:28823–28835, 2021.
  37. Learning in implicit generative models. arXiv preprint arXiv:1610.03483, 2016.
  38. Reading digits in natural images with unsupervised feature learning. 2011.
  39. Transformer neural processes: Uncertainty-aware meta learning via sequence modeling. arXiv preprint arXiv:2207.04179, 2022.
  40. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  41. The freeway service patrol evaluation project: Database support programs, and accessibility. Transportation Research Part C: Emerging Technologies, 4(2):71–85, 1996.
  42. Function-space inference with sparse implicit processes. In International Conference on Machine Learning, pages 18723–18740. PMLR, 2022.
  43. Fast nonlinear vector quantile regression. arXiv preprint arXiv:2205.14977, 2022.
  44. Global coordination of local linear models. Advances in neural information processing systems, 14, 2001.
  45. On the connection between neural processes and gaussian processes with deep kernels. In Workshop on Bayesian Deep Learning, NeurIPS, page 14, 2018.
  46. Joint quantile regression in vector-valued rkhss. Advances in Neural Information Processing Systems, 29, 2016.
  47. Sparse gaussian processes using pseudo-inputs. Advances in neural information processing systems, 18, 2005.
  48. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  49. Bayesian context aggregation for neural processes. In International Conference on Learning Representations, 2020.
  50. Qi Wang and Herke Van Hoof. Doubly stochastic variational inference for neural processes with hierarchical latent variables. In International Conference on Machine Learning, pages 10018–10028. PMLR, 2020.
  51. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
  52. Energy-based processes for exchangeable data. In International Conference on Machine Learning, pages 10681–10692. PMLR, 2020.
  53. Bayesian quantile regression. Statistics & Probability Letters, 54(4):437–447, 2001.
  54. A three-parameter asymmetric laplace distribution and its extension. Communications in Statistics—Theory and Methods, 34(9-10):1867–1879, 2005.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now