Deep Learning based Uncertainty Decomposition for Real-time Control (2010.02613v3)
Abstract: Data-driven control in unknown environments requires a clear understanding of the involved uncertainties for ensuring safety and efficient exploration. While aleatoric uncertainty that arises from measurement noise can often be explicitly modeled given a parametric description, it can be harder to model epistemic uncertainty, which describes the presence or absence of training data. The latter can be particularly useful for implementing exploratory control strategies when system dynamics are unknown. We propose a novel method for detecting the absence of training data using deep learning, which gives a continuous valued scalar output between $0$ (indicating low uncertainty) and $1$ (indicating high uncertainty). We utilize this detector as a proxy for epistemic uncertainty and show its advantages over existing approaches on synthetic and real-world datasets. Our approach can be directly combined with aleatoric uncertainty estimates and allows for uncertainty estimation in real-time as the inference is sample-free unlike existing approaches for uncertainty modeling. We further demonstrate the practicality of this uncertainty estimate in deploying online data-efficient control on a simulated quadcopter acted upon by an unknown disturbance model.
- Thermal soaring flight of birds and unmanned aerial vehicles. Bioinspiration & Biomimetics, 5(4), 045003.
- Stable Gaussian process based tracking control of Euler-Lagrange systems. Automatica, 23(103), 390–397.
- Data selection for multi-task learning under dynamic constraints. IEEE Control Systems Letters, 5(3), 959–964.
- Introduction to algorithms. MIT press.
- Learning and policy search in stochastic dynamical systems with Bayesian neural networks. In International Conference on Learning Representations.
- Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In International Conference on Machine Learning.
- Gal, Y. (2016). Uncertainty in Deep Learning. Ph.D. thesis, University of Cambridge.
- A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3), 362–386.
- Noise contrastive priors for functional uncertainty. In Uncertainty in Artificial Intelligence, 905–914. PMLR.
- What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (NeurIPS), 5574–5584.
- Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation.
- Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS), 6402–6413.
- Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations.
- Robust regression for safe exploration in control. In Conference on Learning for Dynamics and Control (L4DC).
- When Gaussian process meets big data: A review of scalable gps. IEEE Transactions on Neural Networks and Learning Systems, 31(11), 4405–4423.
- Boundary optimised samples training for detecting out-of-distribution images. In 2020 25th International Conference on Pattern Recognition (ICPR), 10486–10492. IEEE.
- Sampling-free epistemic uncertainty estimation using approximated variance propagation. In Proceedings of the IEEE International Conference on Computer Vision, 2931–2940.
- A unifying view of sparse approximate Gaussian process regression. The Journal of Machine Learning Research, 6, 1939–1959.
- Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, USA.
- Sparse Gaussian processes using pseudo-inputs. Advances in Neural Information Processing Systems, 18.
- Titsias, M. (2009). Variational learning of inducing variables in sparse Gaussian processes. In Artificial intelligence and statistics, 567–574. PMLR.
- Feedback linearization based on Gaussian processes with event-triggered online learning. IEEE Transactions on Automatic Control (TAC), 1–16.
- An uncertainty-based control Lyapunov approach for control-affine systems modeled by Gaussian process. IEEE Control Systems Letters, 2(3), 483–488.
- Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional space. In International Conference on Machine Learning (ICML).