Gaussian Process Neural Additive Models (2402.12518v2)
Abstract: Deep neural networks have revolutionized many fields, but their black-box nature also occasionally prevents their wider adoption in fields such as healthcare and finance, where interpretable and explainable models are required. The recent development of Neural Additive Models (NAMs) is a significant step in the direction of interpretable deep learning for tabular datasets. In this paper, we propose a new subclass of NAMs that use a single-layer neural network construction of the Gaussian process via random Fourier features, which we call Gaussian Process Neural Additive Models (GP-NAM). GP-NAMs have the advantage of a convex objective function and number of trainable parameters that grows linearly with feature dimensionality. It suffers no loss in performance compared to deeper NAM approaches because GPs are well-suited for learning complex non-parametric univariate functions. We demonstrate the performance of GP-NAM on several tabular datasets, showing that it achieves comparable or better performance in both classification and regression tasks with a large reduction in the number of parameters.
- Neural Additive Models: Interpretable machine learning with neural nets. Advances in Neural Information Processing Systems.
- Laplace-Approximated Neural Additive Models: Improving Interpretability with Bayesian Inference. arXiv preprint arXiv:2305.16905.
- Node-gam: Neural generalized additive model for interpretable deep learning. arXiv preprint arXiv:2106.01613.
- XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SigKDD International Conference on Knowledge Discovery and Data Mining, 785–794.
- Kernel methods for deep learning. In Advances in Neural Information Processing Systems.
- Scalable interpretability via polynomials. Advances in Neural Information Processing Systems.
- Generalized Additive Models, volume 43. CRC Press.
- Global explanations of neural networks: Mapping the landscape of predictions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society.
- Batch Normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning.
- Deep neural networks as Gaussian processes. arXiv preprint arXiv:1711.00165.
- Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
- Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 623–631.
- A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems.
- Neal, R. M. 2012. Bayesian learning for neural networks. Springer Science & Business Media.
- Numerical Optimization, Second Edition. Springer Series in Operations Research.
- Interpretml: A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223.
- Bayesian nonparametric model averaging using scalable Gaussian process representations. In IEEE International Conference on Big Data.
- Neural oblivious decision ensembles for deep learning on tabular data. arXiv preprint arXiv:1909.06312.
- Neural basis models for interpretability. Advances in Neural Information Processing Systems.
- Random Features for Large-Scale Kernel Machines. In Advances in Neural Information Processing Systems.
- ”Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
- Rudin, C. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5): 206–215.
- Learning important features through propagating activation differences. In International Conference on Machine Learning.
- Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1): 1929–1958.
- Axiomatic attribution for deep networks. In International Conference on Machine Learning.
- Wood, S. N. 2017. Generalized Additive Models: An Introduction with R. CRC Press.
- Wei Zhang (1489 papers)
- Brian Barr (15 papers)
- John Paisley (60 papers)