2000 character limit reached
Uncertainty Regularized Evidential Regression (2401.01484v1)
Published 3 Jan 2024 in cs.LG and cs.AI
Abstract: The Evidential Regression Network (ERN) represents a novel approach that integrates deep learning with Dempster-Shafer's theory to predict a target and quantify the associated uncertainty. Guided by the underlying theory, specific activation functions must be employed to enforce non-negative values, which is a constraint that compromises model performance by limiting its ability to learn from all samples. This paper provides a theoretical analysis of this limitation and introduces an improvement to overcome it. Initially, we define the region where the models can't effectively learn from the samples. Following this, we thoroughly analyze the ERN and investigate this constraint. Leveraging the insights from our analysis, we address the limitation by introducing a novel regularization term that empowers the ERN to learn from the whole training set. Our extensive experiments substantiate our theoretical findings and demonstrate the effectiveness of the proposed solution.
- Deep Evidential Regression. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 14927–14937. Curran Associates, Inc.
- Uncertainty on asynchronous time event prediction. Advances in Neural Information Processing Systems, 32.
- Weight uncertainty in neural network. In International conference on machine learning, 1613–1622. PMLR.
- Natural Posterior Network: Deep Bayesian Predictive Uncertainty for Exponential Family Distributions. In International Conference on Learning Representations.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
- Deep evidential learning in diffusion convolutional recurrent neural network. Electronic Research Archive, 31(4): 2252–2264.
- Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, 1050–1059. PMLR.
- Unsupervised monocular depth estimation with left-right consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition, 270–279.
- On calibration of modern neural networks. In International conference on machine learning, 1321–1330. PMLR.
- Bayesian evidential deep learning with PAC regularization. arXiv preprint arXiv:1906.00816.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
- Probabilistic backpropagation for scalable learning of bayesian neural networks. In International conference on machine learning, 1861–1869. PMLR.
- The apolloscape dataset for autonomous driving. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 954–960.
- Accurate uncertainties for deep learning using calibrated regression. In International conference on machine learning, 2796–2804. PMLR.
- Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30.
- Regression prior networks. arXiv preprint arXiv:2006.11590.
- Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31.
- Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. Advances in Neural Information Processing Systems, 32.
- Ensemble Distribution Distillation. In International Conference on Learning Representations.
- Uncertainty-aware Traffic Prediction under Missing Data. The Proceedings of 2023 IEEE International Conference on Data Mining (ICDM 2023).
- Multivariate deep evidential regression. arXiv preprint arXiv:2104.06135.
- Improving evidential deep learning via multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 7895–7903.
- Learn to Accumulate Evidence from All Training Samples: Theory and Practice. In International Conference on Machine Learning, 26963–26989. PMLR.
- Uncertainty in neural networks: Approximately bayesian ensembling. In International conference on artificial intelligence and statistics, 234–244. PMLR.
- U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234–241. Springer.
- Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems, 31.
- Shafer, G. 1976. A mathematical theory of evidence, volume 42. Princeton university press.
- Indoor segmentation and support inference from rgbd images. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part V 12, 746–760. Springer.
- Attention is all you need. Advances in neural information processing systems, 30.
- Bayesian deep learning and a probabilistic perspective of generalization. Advances in neural information processing systems, 33: 4697–4708.
- Bidirectional Mapping with Contrastive Learning on Multimodal Neuroimaging Data. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 138–148. Springer.
- Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information. arXiv preprint arXiv:2305.16967.
