Uncertainty-Aware Robust Learning on Noisy Graphs (2306.08210v2)
Abstract: Graph neural networks (GNNs) have excelled in various graph learning tasks, particularly node classification. However, their performance is often hampered by noisy measurements in real-world graphs, which can corrupt critical patterns in the data. To address this, we propose a novel uncertainty-aware graph learning framework inspired by distributionally robust optimization. Specifically, we use a graph neural network-based encoder to embed the node features and find the optimal node embeddings by minimizing the worst-case risk through a minimax formulation. Such an uncertainty-aware learning process leads to improved node representations and a more robust graph predictive model that effectively mitigates the impact of uncertainty arising from data noise. Our experimental results demonstrate superior predictive performance over baselines across noisy scenarios.
- Thomas N Kipf and Max Welling “Semi-Supervised Classification with Graph Convolutional Networks” In arXiv preprint arXiv:1609.02907, 2016
- “Graph Attention Networks”, 2018 arXiv:1710.10903 [stat.ML]
- “Recent Advances in Reliable Deep Graph Learning: Inherent Noise, Distribution Shift, and Adversarial Attack” In arXiv e-prints, 2022, pp. arXiv–2202
- “A survey on graph structure learning: Progress and opportunities” In arXiv e-prints, 2021, pp. arXiv–2103
- “Learning to Drop: Robust Graph Neural Network via Topological Denoising” In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM ’21 Virtual Event, Israel: Association for Computing Machinery, 2021, pp. 779–787 DOI: 10.1145/3437963.3441734
- “Toward robust graph semi-supervised learning against extreme data scarcity” In arXiv preprint arXiv:2208.12422, 2022
- John C Duchi and Hongseok Namkoong “Learning models with uniform performance via distributionally robust optimization” In The Annals of Statistics 49.3 Institute of Mathematical Statistics, 2021, pp. 1378–1406
- “Minimizing the maximal loss: How and why” In International Conference on Machine Learning, 2016, pp. 793–801 PMLR
- “Robust Hypothesis Testing Using Wasserstein Uncertainty Sets” In Advances in Neural Information Processing Systems 31 Curran Associates, Inc., 2018 URL: https://proceedings.neurips.cc/paper_files/paper/2018/file/a08e32d2f9a8b78894d964ec7fd4172e-Paper.pdf
- “Distributionally robust weighted k-nearest neighbors” In Advances in Neural Information Processing Systems 35, 2022, pp. 29088–29100
- “Robust Graph Learning Under Wasserstein Uncertainty”, 2021 arXiv:2105.04210 [cs.LG]
- “Distributionally robust graphical models” In Advances in Neural Information Processing Systems 31, 2018
- “Distributionally robust deep learning as a generalization of adversarial training” In NIPS workshop on Machine Learning and Computer Security 3, 2017, pp. 4
- Soroosh Shafieezadeh Abadeh, Peyman M Mohajerin Esfahani and Daniel Kuhn “Distributionally robust logistic regression” In Advances in Neural Information Processing Systems 28, 2015
- “Differentiable Convex Optimization Layers” In Advances in Neural Information Processing Systems, 2019
- “OptNet: Differentiable Optimization as a Layer in Neural Networks” In Proceedings of the 34th International Conference on Machine Learning 70, Proceedings of Machine Learning Research PMLR, 2017, pp. 136–145
- “Spectral Networks and Locally Connected Networks on Graphs”, 2014 arXiv:1312.6203 [cs.LG]
- Michaël Defferrard, Xavier Bresson and Pierre Vandergheynst “Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering” In Advances in Neural Information Processing Systems, 2016 URL: https://arxiv.org/abs/1606.09375
- “Deep graph infomax.” In ICLR (Poster) 2.3, 2019, pp. 4
- “Data augmentation for deep graph learning: A survey” In ACM SIGKDD Explorations Newsletter, 2022
- Yu Chen, Lingfei Wu and Mohammed Zaki “Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings” In Advances in Neural Information Processing Systems 33 Curran Associates, Inc., 2020, pp. 19314–19326 URL: https://proceedings.neurips.cc/paper_files/paper/2020/file/e05c7ba4e087beea9410929698dc41a6-Paper.pdf
- “Graph Structure Learning for Robust Graph Neural Networks” In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20 Virtual Event, CA, USA: Association for Computing Machinery, 2020, pp. 66–74 DOI: 10.1145/3394486.3403049
- “Adversarial Attacks and Defenses on Graphs” In SIGKDD Explor. Newsl. 22.2 New York, NY, USA: Association for Computing Machinery, 2021, pp. 19–34 DOI: 10.1145/3447556.3447566
- “Learning Robust Representations with Graph Denoising Policy Network” In 2019 IEEE International Conference on Data Mining (ICDM) Los Alamitos, CA, USA: IEEE Computer Society, 2019, pp. 1378–1383 DOI: 10.1109/ICDM.2019.00177
- “On the Unreasonable Effectiveness of Feature propagation in Learning on Graphs with Missing Node Features”, 2022 arXiv:2111.12128 [cs.LG]
- “Incomplete graph representation and learning via partial graph neural networks” In arXiv preprint arXiv:2003.10130, 2020
- “Distributionally Robust Semi-Supervised Learning Over Graphs”, 2021 arXiv:2110.10582 [cs.LG]
- “A review of uncertainty quantification in deep learning: Techniques, applications and challenges” In Information Fusion 76, 2021, pp. 243–297 DOI: https://doi.org/10.1016/j.inffus.2021.05.008
- Edmon Begoli, Tanmoy Bhattacharya and Dimitri Kusnezov “The need for uncertainty quantification in machine-assisted medical decision making” In Nature Machine Intelligence 1.1 Nature Publishing Group UK London, 2019, pp. 20–23
- Seongok Ryu, Yongchan Kwon and Woo Youn Kim “A Bayesian graph convolutional network for reliable prediction of molecular properties with uncertainty quantification” In Chemical science 10.36 Royal Society of Chemistry, 2019, pp. 8438–8446
- Yao Zhang “Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learning” In Chemical science 10.35 Royal Society of Chemistry, 2019, pp. 8154–8163
- Qimai Li, Zhichao Han and Xiao-ming Wu “Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning” In Proceedings of the AAAI Conference on Artificial Intelligence 32.1, 2018 DOI: 10.1609/aaai.v32i1.11604
- “Automating the construction of internet portals with machine learning” In Information Retrieval 3 Springer, 2000, pp. 127–163
- C Lee Giles, Kurt D Bollacker and Steve Lawrence “CiteSeer: An automatic citation indexing system” In Proceedings of the third ACM conference on Digital libraries, 1998, pp. 89–98
- “Collective Classification in Network Data” In AI Magazine 29.3, 2008, pp. 93 DOI: 10.1609/aimag.v29i3.2157
- Zhilin Yang, William Cohen and Ruslan Salakhudinov “Revisiting semi-supervised learning with graph embeddings” In International conference on machine learning, 2016, pp. 40–48 PMLR
- “Dp-ssl: Towards robust semi-supervised learning with a few labeled samples” In Advances in Neural Information Processing Systems 34, 2021, pp. 15895–15907
- “Few-Shot Node Classification with Extremely Weak Supervision” In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM ’23 Singapore, Singapore: Association for Computing Machinery, 2023, pp. 276–284 DOI: 10.1145/3539597.3570435
- C.E. Shannon “Communication theory of secrecy systems” In The Bell System Technical Journal 28.4, 1949, pp. 656–715 DOI: 10.1002/j.1538-7305.1949.tb00928.x
- “A review of entropy measures for uncertainty quantification of stochastic processes” In Advances in Mechanical Engineering 11.6 SAGE Publications Sage UK: London, England, 2019, pp. 1687814019857350
- Stephan Schwill “Entropy Analysis of Financial Time Series” In arXiv preprint arXiv:1807.09423, 2018
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run paper prompts using GPT-5.