Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
GPT-4o
Gemini 2.5 Pro Pro
o3 Pro
GPT-4.1 Pro
DeepSeek R1 via Azure Pro
2000 character limit reached

Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Models (2305.14585v5)

Published 23 May 2023 in cs.LG

Abstract: A recent trend in explainable AI research has focused on surrogate modeling, where neural networks are approximated as simpler ML algorithms such as kernel machines. A second trend has been to utilize kernel functions in various explain-by-example or data attribution tasks. In this work, we combine these two trends to analyze approximate empirical neural tangent kernels (eNTK) for data attribution. Approximation is critical for eNTK analysis due to the high computational cost to compute the eNTK. We define new approximate eNTK and perform novel analysis on how well the resulting kernel machine surrogate models correlate with the underlying neural network. We introduce two new random projection variants of approximate eNTK which allow users to tune the time and memory complexity of their calculation. We conclude that kernel machines using approximate neural tangent kernel as the kernel function are effective surrogate models, with the introduced trace NTK the most consistent performer. Open source software allowing users to efficiently calculate kernel functions in the PyTorch framework is available (https://github.com/pnnl/projection\_ntk).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Towards tracing knowledge in language models back to the training data. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.  2429–2446, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.findings-emnlp.180.
  2. What learning algorithm is in-context learning? investigations with linear models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=0g0X4H8yN4I.
  3. Kernels for Vector-Valued Functions: a Review. arXiv e-prints, art. arXiv:1106.6251, June 2011. doi: 10.48550/arXiv.1106.6251.
  4. Neural networks as kernel learners: The silent alignment effect. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=1NvflqAdoom.
  5. A spline theory of deep networks. In International Conference on Machine Learning, 2018.
  6. Introduction to Real Analysis (4th Edition). Wiley, 2011.
  7. An exact kernel equivalence for finite classification models, 2023.
  8. On the Opportunities and Risks of Foundation Models. arXiv e-prints, art. arXiv:2108.07258, August 2021. doi: 10.48550/arXiv.2108.07258.
  9. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019.
  10. When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=LtKcMgGOeLt.
  11. On lazy training in differentiable programming. In Neural Information Processing Systems, 2018.
  12. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp.  4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1423. URL https://aclanthology.org/N19-1423.
  13. Pedro Domingos. Every model learned by gradient descent is approximately a kernel machine. arXiv preprint arXiv:2012.00152, 2020.
  14. p𝑝pitalic_p-DkNN: Out-of-Distribution Detection Through Statistical Testing of Deep Representations. arXiv e-prints, art. arXiv:2207.12545, July 2022. doi: 10.48550/arXiv.2207.12545.
  15. Spectra of the conjugate kernel and neural tangent kernel for linear-width neural networks. Advances in neural information processing systems, 33:7710–7721, 2020.
  16. Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel. Advances in Neural Information Processing Systems, 33:5850–5861, 2020.
  17. Reproducing kernel hilbert space, mercer’s theorem, eigenfunctions, nystr\\\backslash\" om method, and use of kernels in machine learning: Tutorial and survey. arXiv preprint arXiv:2106.08443, 2021.
  18. Deep sparse rectifier neural networks. In International Conference on Artificial Intelligence and Statistics, 2011. URL https://api.semanticscholar.org/CorpusID:2239473.
  19. BadNets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 01 2019. doi: 10.1109/ACCESS.2019.2909068.
  20. Evaluation of similarity-based explanations. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=9uvhpyQwzM_.
  21. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. 2015 IEEE International Conference on Computer Vision (ICCV), pp.  1026–1034, 2015a. URL https://api.semanticscholar.org/CorpusID:13740328.
  22. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.  770–778, 2015b.
  23. David Hilbert. Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen. Number 3. BG Teubner, 1912.
  24. Kernel methods in machine learning. Annals of Statistics, 36:1171–1220, 2007.
  25. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018.
  26. Extensions of lipschitz mappings into a hilbert space. Contemporary Mathematics, 26, 1984. URL http://stanford.edu/class/cs114/readings/JL-Johnson.pdf.
  27. Understanding black-box predictions via influence functions. In International Conference on Machine Learning, pp.  1885–1894. PMLR, 2017.
  28. A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto, 2009.
  29. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471, 2021.
  30. Towards falsifiable interpretability research. arXiv e-prints, art. arXiv:2010.12016, October 2020. doi: 10.48550/arXiv.2010.12016.
  31. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. doi: 10.1109/5.726791.
  32. Deep neural networks as gaussian processes. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=B1EA-M-0Z.
  33. Wide neural networks of any depth evolve as linear models under gradient descent. Journal of Statistical Mechanics: Theory and Experiment, 2020(12):124002, December 2020. doi: 10.1088/1742-5468/abc62b.
  34. Regularizing neural networks via minimizing hyperspherical energy. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  6916–6925, 2019.
  35. Philip M Long. Properties of the after kernel. arXiv preprint arXiv:2105.10585, 2021.
  36. Evolution of neural tangent kernels under benign and adversarial training, 2022.
  37. Decoupled Weight Decay Regularization. arXiv e-prints, art. arXiv:1711.05101, November 2017. doi: 10.48550/arXiv.1711.05101.
  38. Towards deep learning models resistant to adversarial attacks. ICLR, 2019.
  39. A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernel. arXiv e-prints, art. arXiv:2206.12543, June 2022. doi: 10.48550/arXiv.2206.12543.
  40. Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning, 2010.
  41. Fast finite width neural tangent kernel. In International Conference on Machine Learning, pp.  17018–17044. PMLR, 2022.
  42. What can linearized neural networks actually say about generalization? Advances in Neural Information Processing Systems, 34:8998–9010, 2021.
  43. Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. arXiv e-prints, art. arXiv:1803.04765, March 2018.
  44. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. CoRR, 2016a.
  45. Practical black-box attacks against deep learning systems using adversarial examples. CoRR, 2016b.
  46. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences of the United States of America, 117:24652 – 24663, 2020.
  47. TRAK: Attributing Model Behavior at Scale. arXiv e-prints, art. arXiv:2303.14186, March 2023.
  48. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  49. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  50. Huy Phan. huyvnphan/pytorch_cifar10, January 2021. URL https://doi.org/10.5281/zenodo.4431043.
  51. Estimating training data influence by tracing gradient descent. In Advances in Neural Information Processing Systems, volume 33, pp.  19920–19930. Curran Associates, Inc., 2020.
  52. Efficient kernel surrogates for neural network-based regression. ArXiv, abs/2310.18612, 2023. URL https://api.semanticscholar.org/CorpusID:264590537.
  53. Feature learning in neural networks and kernel machines that recursively learn features. arXiv preprint arXiv:2212.13881, 2022.
  54. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  4510–4520, 2018.
  55. Ann-dt: an algorithm for extraction of decision trees from artificial neural networks. IEEE Transactions on Neural Networks, 10(6):1392–1401, 1999. doi: 10.1109/72.809084.
  56. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
  57. Poison forensics: Traceback of data poisoning attacks in neural networks. In 31st USENIX Security Symposium (USENIX Security 22), pp.  3575–3592, 2022.
  58. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:1958–1970, 2008.
  59. Sample based explanations via generalized representers. ArXiv, abs/2310.18526, 2023. URL https://api.semanticscholar.org/CorpusID:264590303.
  60. What can the neural tangent kernel tell us about adversarial robustness?, 2023.
  61. Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
  62. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261–272, 2020. doi: 10.1038/s41592-019-0686-2.
  63. Limitations of the NTK for Understanding Generalization in Deep Learning. arXiv e-prints, art. arXiv:2206.10012, June 2022.
  64. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=rJ4km2R5t7.
  65. Deep Adversarially-Enhanced k-Nearest Neighbors. arXiv e-prints, art. arXiv:2108.06797, August 2021. doi: 10.48550/arXiv.2108.06797.
  66. Spectral evolution and invariance in linear-width neural networks. arXiv preprint arXiv:2211.06506, 2022.
  67. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641, 2018.
  68. More than a toy: Random matrix models predict how real-world neural representations generalize. In International Conference on Machine Learning, pp.  23549–23588. PMLR, 2022.
  69. HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv e-prints, art. arXiv:1910.03771, October 2019. doi: 10.48550/arXiv.1910.03771.
  70. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. ArXiv, abs/1708.07747, 2017.
  71. How do visual explanations foster end users’ appropriate trust in machine learning? In Proceedings of the 25th International Conference on Intelligent User Interfaces, pp.  189–201, 2020.
  72. Tensor Programs IV: Feature learning in infinite-width neural networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.  11727–11737. PMLR, 18–24 Jul 2021.
  73. Representer point selection for explaining deep neural networks. Advances in neural information processing systems, 31, 2018.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets