Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Resurrecting Label Propagation for Graphs with Heterophily and Label Noise (2310.16560v2)

Published 25 Oct 2023 in cs.LG and cs.AI

Abstract: Label noise is a common challenge in large datasets, as it can significantly degrade the generalization ability of deep neural networks. Most existing studies focus on noisy labels in computer vision; however, graph models encompass both node features and graph topology as input, and become more susceptible to label noise through message-passing mechanisms. Recently, only a few works have been proposed to tackle the label noise on graphs. One significant limitation is that they operate under the assumption that the graph exhibits homophily and that the labels are distributed smoothly. However, real-world graphs can exhibit varying degrees of heterophily, or even be dominated by heterophily, which results in the inadequacy of the current methods. In this paper, we study graph label noise in the context of arbitrary heterophily, with the aim of rectifying noisy labels and assigning labels to previously unlabeled nodes. We begin by conducting two empirical analyses to explore the impact of graph homophily on graph label noise. Following observations, we propose a efficient algorithm, denoted as $R{2}LP$. Specifically, $R{2}LP$ is an iterative algorithm with three steps: (1) reconstruct the graph to recover the homophily property, (2) utilize label propagation to rectify the noisy labels, (3) select high-confidence labels to retain for the next iteration. By iterating these steps, we obtain a set of correct labels, ultimately achieving high accuracy in the node classification task. The theoretical analysis is also provided to demonstrate its remarkable denoising effect. Finally, we perform experiments on ten benchmark datasets with different levels of graph heterophily and various types of noise. In these experiments, we compare the performance of $R{2}LP$ against ten typical baseline methods. Our results illustrate the superior performance of the proposed $R{2}LP$.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Beyond Low-frequency Information in Graph Convolutional Networks. In AAAI. AAAI Press.
  2. Adaptive Universal Generalized PageRank Graph Neural Network. In International Conference on Learning Representations. https://openreview.net/forum?id=n6jl7fLxrP
  3. Nrgnn: Learning a label noise resistant graph neural network on sparsely and noisily labeled graphs. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 227–236.
  4. Adversarial attack on graph structured data. In International conference on machine learning. PMLR, 1115–1124.
  5. Distillation a⁢p⁢p⁢r⁢o⁢x𝑎𝑝𝑝𝑟𝑜𝑥approxitalic_a italic_p italic_p italic_r italic_o italic_x Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Network. arXiv preprint arXiv:1910.01255 (2019).
  6. Inductive representation learning on large graphs. 1024–1034.
  7. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems 31 (2018).
  8. Knowledge graphs. ACM Computing Surveys (Csur) 54, 4 (2021), 1–37.
  9. Self-adaptive training: beyond empirical risk minimization. Advances in neural information processing systems 33 (2020), 19365–19376.
  10. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  11. Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks.
  12. Predict then Propagate: Graph Neural Networks meet Personalized PageRank.
  13. Predicting positive and negative links in online social networks. In Proceedings of the 19th international conference on World wide web. 641–650.
  14. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
  15. Finding global homophily in graph neural networks when meeting heterophily. In International Conference on Machine Learning. PMLR, 13242–13256.
  16. Deep Patel and PS Sastry. 2023. Adaptive sample selection for robust learning under label noise. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3932–3942.
  17. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1944–1952.
  18. Selfie: Refurbishing unclean samples for robust deep learning. In International Conference on Machine Learning. PMLR, 5907–5915.
  19. Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems (2022).
  20. Adversarial attacks on graph neural networks via node injections: A hierarchical reinforcement learning approach. In Proceedings of the Web Conference 2020. 673–683.
  21. Graph attention networks.
  22. To smooth or not? when label smoothing meets noisy labels. Learning 1, 1 (2021), e1.
  23. M. Woodbury and M. Woodbury. 1950. Inverting modified matrices. (1950).
  24. Simplifying graph convolutional networks. PMLR, 6861–6871.
  25. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610 (2019).
  26. A topological filter for learning with label noise. Advances in neural information processing systems 33 (2020), 21382–21393.
  27. Towards robust graph neural networks against label noise. (2021).
  28. How powerful are graph neural networks?
  29. How does disagreement help generalization against label corruption?. In International Conference on Machine Learning. PMLR, 7164–7173.
  30. Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64, 3 (2021), 107–115.
  31. Learning with local and global consistency. Advances in neural information processing systems 16 (2003).
  32. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems 33 (2020), 7793–7804.
  33. Xiaojin Zhu. 2005. Semi-supervised learning with graphs. Carnegie Mellon University.
  34. Marinka Zitnik and Jure Leskovec. 2017. Predicting multicellular function through multi-layer tissue networks. Bioinformatics 33, 14 (2017), i190–i198.
  35. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2847–2856.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now