Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Global Safe Sequential Learning via Efficient Knowledge Transfer (2402.14402v2)

Published 22 Feb 2024 in cs.LG and stat.ML

Abstract: Sequential learning methods such as active learning and Bayesian optimization select the most informative data to learn about a task. In many medical or engineering applications, the data selection is constrained by a priori unknown safety conditions. A promissing line of safe learning methods utilize Gaussian processes (GPs) to model the safety probability and perform data selection in areas with high safety confidence. However, accurate safety modeling requires prior knowledge or consumes data. In addition, the safety confidence centers around the given observations which leads to local exploration. As transferable source knowledge is often available in safety critical experiments, we propose to consider transfer safe sequential learning to accelerate the learning of safety. We further consider a pre-computation of source components to reduce the additional computational load that is introduced by incorporating source data. In this paper, we theoretically analyze the maximum explorable safe regions of conventional safe learning methods. Furthermore, we empirically demonstrate that our approach 1) learns a task with lower data consumption, 2) globally explores multiple disjoint safe regions under guidance of the source knowledge, and 3) operates with computation comparable to conventional safe learning methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Kernels for vector-valued functions: a review. arXiv, 2012.
  2. Safe controller optimization for quadrotors with gaussian processes. International Conference on Robotics and Automation, 2016.
  3. Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics. Machine Learning, 2020.
  4. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv, 2010.
  5. Bayesian experimental design: A review. Statistical Science, 1995.
  6. GoSafe: Globally Optimal Safe Robot Learning. IEEE International Conference on Robotics and Automation, 2021.
  7. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 2015.
  8. Bayesian optimization with unknown constraints. Conference on Uncertainty in Artificial Intelligence, 2014.
  9. Effect of epidural stimulation of the lumbosacral spinal cord on voluntary movement, standing, and assisted stepping after motor complete paraplegia: a case study. The Lancet, 2011.
  10. The connected-component labeling problem: A review of state-of-the-art algorithms. Pattern Recognition, 2017.
  11. Predictive entropy search for bayesian optimization with unknown constraints. International Conference on Machine Learning, 2015.
  12. A general framework for constrained bayesian optimization using information-based search. Journal of Machine Learning Research, 2016.
  13. Mining geostatistics. Academic Press London, 1976.
  14. Gaussian processes and kernel methods: A review on connections and equivalences. arXiv, 2018.
  15. Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies. Journal of Machine Learning Research, 2008.
  16. Active learning query strategies for classification, regression, and clustering: A survey. Journal of Computer Science and Technology, 2020.
  17. Posterior variance analysis of gaussian processes with application to average learning curves. arXiv, 2019.
  18. Safe active learning for multi-output gaussian processes. International Conference on Artificial Intelligence and Statistics, 2022.
  19. Multi-fidelity bayesian optimization via deep neural networks. Advances in Neural Information Processing Systems, 2020.
  20. D. V. Lindley. On a Measure of the Information Provided by an Experiment. The Annals of Mathematical Statistics, 1956.
  21. Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with bayesian optimization. IEEE International Conference on Robotics and Automation, 2017.
  22. Safe Exploration for Interactive Machine Learning. Advances in Neural Information Processing Systems, 2019.
  23. Multi-information source optimization. Advances in Neural Information Processing Systems, 2017.
  24. Gaussian processes for machine learning. MIT Press, 2006.
  25. Bayesian active learning with fully bayesian gaussian processes. Advances in Neural Information Processing Systems, 2022.
  26. Meta-Learning Priors for Safe Bayesian Optimization. 6th Annual Conference on Robot Learning, 2022.
  27. Learning with kernels: Support vector machines, regularization, optimization, and beyond. MIT Press, 2002.
  28. Safe exploration for active learning with gaussian processes. Machine Learning and Knowledge Discovery in Databases, 2015.
  29. B. W. Silverman. Spline smoothing: The equivalent variable kernel method. Annals of Statistics, 1984.
  30. Practical bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 2012.
  31. Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory, 2012.
  32. Safe exploration for optimization with gaussian processes. International Conference on Machine Learning, 2015.
  33. Multi-task bayesian optimization. Advances in Neural Information Processing Systems, 2013.
  34. Transfer learning with gaussian processes for bayesian optimization. International Conference on Artificial Intelligence and Statistics, 2022.
  35. Stagewise Safe Bayesian Optimization with Gaussian Processes. International Conference on Machine Learning, 80, 2018.
  36. Near-optimal active learning of multi-output gaussian processes. AAAI Conference on Artificial Intelligence, 2016.
  37. Safe active learning for time-series modeling with gaussian processes. Advances in Neural Information Processing Systems, 2018.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets