Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions

Published 18 Jan 2024 in cs.SI, cs.AI, and cs.LG | (2401.09769v4)

Abstract: Graphs are structured data that models complex relations between real-world entities. Heterophilic graphs, where linked nodes are prone to be with different labels or dissimilar features, have recently attracted significant attention and found many real-world applications. Meanwhile, increasing efforts have been made to advance learning from graphs with heterophily. Various graph heterophily measures, benchmark datasets, and learning paradigms are emerging rapidly. In this survey, we comprehensively review existing works on learning from graphs with heterophily. First, we overview over 500 publications, of which more than 340 are directly related to heterophilic graphs. After that, we survey existing metrics of graph heterophily and list recent benchmark datasets. Further, we systematically categorize existing methods based on a hierarchical taxonomy including GNN models, learning paradigms and practical applications. In addition, broader topics related to graph heterophily are also included. Finally, we discuss the primary challenges of existing studies and highlight promising avenues for future research.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In ICML, pages 21–29. PMLR, 2019.
  2. A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening. Acta metallurgica, 27(6):1085–1095, 1979.
  3. Half-hop: A graph upsampling approach for slowing down message passing. In ICML, pages 1341–1360. PMLR, 2023.
  4. Beyond low-frequency information in graph convolutional networks. In AAAI, volume 35, pages 3950–3957, 2021.
  5. Simple and deep graph convolutional networks. In ICML, pages 1725–1735. PMLR, 2020.
  6. A drug combination prediction framework based on graph convolutional network and heterogeneous information. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022.
  7. Signgt: Signed attention-based graph transformer for graph representation learning. arXiv preprint arXiv:2310.11025, 2023.
  8. Network-based prediction of drug combinations. Nature communications, 10(1):1197, 2019.
  9. Label propagation for graph label noise. arXiv preprint arXiv:2310.16560, 2023.
  10. Adaptive universal generalized pagerank graph neural network. ICLR, 2021.
  11. Enhancing graph neural networks with structure-based prompt. arXiv preprint arXiv:2310.17394, 2023.
  12. Prompt tuning for multi-view graph contrastive learning. arXiv preprint arXiv:2310.10362, 2023.
  13. Inductive representation learning on large graphs. NeurIPS, 30, 2017.
  14. Block modeling-guided graph convolutional neural networks. In AAAI, volume 36, pages 4022–4029, 2022.
  15. Contrastive learning meets homophily: two birds with one stone. In ICML, pages 12775–12789. PMLR, 2023.
  16. Universal graph convolutional networks. NeurIPS, 34:10654–10664, 2021.
  17. Dynamic relation-attentive graph neural networks for fraud detection. arXiv preprint arXiv:2310.04171, 2023.
  18. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  19. Goat: A global transformer on large-scale graphs. 2023.
  20. Towards deep attention in graph neural networks: Problems and remedies. arXiv preprint arXiv:2306.02376, 2023.
  21. Evennet: Ignoring odd-hop neighbors improves robustness of graph neural networks. NeurIPS, 35:4694–4706, 2022.
  22. Finding global homophily in graph neural networks when meeting heterophily. In ICML, pages 13242–13256. PMLR, 2022.
  23. Restructuring graph for higher homophily via adaptive spectral clustering. In AAAI, volume 37, pages 8622–8630, 2023.
  24. LD2: Scalable heterophilous graph neural network with decoupled embedding. In NeurIPS, volume 36, Dec 2023.
  25. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. NeurIPS, 34, 2021.
  26. New benchmarks for learning on non-homophilous graphs. arXiv preprint arXiv:2104.01404, 2021.
  27. Gps-net: Graph property sensing network for scene graph generation. In CVPR, pages 3746–3753, 2020.
  28. Hl-net: Heterophily learning network for scene graph generation. In CVPR, pages 19476–19485, 2022.
  29. Non-local graph neural networks. TPAMI, 44(12):10270–10276, 2021.
  30. Simga: A simple and effective heterophilous graph neural network with efficient global aggregation. arXiv preprint arXiv:2305.09958, 2023.
  31. Beyond smoothing: Unsupervised graph representation learning with edge heterophily discriminating. In AAAI, 2023.
  32. Revisiting heterophily for graph neural networks. NeurIPS, 35:1362–1375, 2022.
  33. Rethinking structural encodings: Adaptive graph transformer for node classification task. In WebConf, pages 533–544, 2023.
  34. Beyond homophily: Reconstructing structure for graph-agnostic clustering. arXiv preprint arXiv:2305.02931, 2023.
  35. Geom-gcn: Geometric graph convolutional networks. ICLR, 2020.
  36. Characterizing graph datasets for node classification: Homophily-heterophily dichotomy and beyond. In LoG, 2023.
  37. A critical look at the evaluation of gnns under heterophily: are we really making progress? arXiv preprint arXiv:2302.11640, 2023.
  38. Gradient gating for deep multi-rate learning on graphs. arXiv preprint arXiv:2210.00513, 2022.
  39. H2-fdetector: A gnn-based fraud detector with homophilic and heterophilic connections. In WebConf, 2022.
  40. Muti-scale graph neural network with signed-attention for social bot detection: A frequency perspective. arXiv preprint arXiv:2307.01968, 2023.
  41. Ordered gnn: Ordering message passing to deal with heterophily and over-smoothing. arXiv preprint arXiv:2302.01524, 2023.
  42. Breaking the limit of graph neural networks by improving the assortativity of graphs with local mixing patterns. In SIGKDD, pages 1541–1551, 2021.
  43. Graph auto-encoder via neighborhood wasserstein reconstruction. ICLR, 2022.
  44. Understanding over-squashing and bottlenecks on graphs via curvature. arXiv preprint arXiv:2111.14522, 2021.
  45. Attention is all you need. NeurIPS, 30, 2017.
  46. Graph attention networks. ICLR, 2018.
  47. Yu Wang and Tyler Derr. Tree decomposed graph neural network. In CIKM, 2021.
  48. Powerful graph convolutional networks with adaptive propagation mechanism for homophily and heterophily. In AAAI, volume 36, pages 4210–4218, 2022.
  49. Acmp: Allen-cahn message passing with attractive and repulsive forces for graph neural networks. In ICLR, 2022.
  50. Label information enhanced fraud detection against low homophily in graphs. In WebConf, pages 406–416, 2023.
  51. Nodeformer: A scalable graph structure learning transformer for node classification. NeurIPS, 35:27387–27401, 2022.
  52. Heterophily-aware social bot detection with supervised contrastive learning. arXiv preprint arXiv:2306.07478, 2023.
  53. Decoupled self-supervised learning for non-homophilous graphs. In NeurIPS, 2022.
  54. Spatial heterophily aware graph neural networks. arXiv preprint arXiv:2306.12139, 2023.
  55. Simple and asymmetric graph contrastive learning without augmentations. arXiv preprint arXiv:2310.18884, 2023.
  56. Diverse message passing for attribute with heterophily. NeurIPS, 34:4751–4763, 2021.
  57. Graph pointer neural networks. In AAAI, volume 36, pages 8832–8839, 2022.
  58. Hofa: Twitter bot detection with homophily-oriented augmentation and frequency adaptive attention. arXiv preprint arXiv:2306.12870, 2023.
  59. Muse: Multi-view contrastive learning for heterophilic graphs via information reconstruction. In CIKM, 2023.
  60. Quadratic graph attention network (q-gat) for robust construction of gene regulatory networks. arXiv preprint arXiv:2303.14193, 2023.
  61. Graph neural networks for graphs with heterophily: A survey. arXiv preprint arXiv:2202.07082, 2022.
  62. Finding the missing-half: Graph complementary learning for homophily-prone and heterophily-prone graphs. arXiv preprint arXiv:2306.07608, 2023.
  63. Link prediction on heterophilic graphs via disentangled representation learning. arXiv preprint arXiv:2208.01820, 2022.
  64. Beyond homophily in graph neural networks: Current limitations and effective designs. NeurIPS, 33:7793–7804, 2020.
  65. How does heterophily impact the robustness of graph neural networks? theoretical connections and practical implications. In SIGKDD, pages 2637–2647, 2022.
  66. Heterophily and graph neural networks: Past, present and future. IEEE Data Engineering Bulletin, 2023.
Citations (1)
View on Semantic Scholar

Summary

  • The paper offers a comprehensive taxonomy categorizing over 180 studies, emphasizing both (semi-)supervised and self-supervised learning approaches.
  • It details diverse model architectures, including decoupled message-passing networks and global attention graph transformers, to tackle the challenges of heterophily.
  • The study highlights practical applications in social and biochemical fields while suggesting future benchmarks, robustness improvements, and innovative performance metrics.

Overview of "Towards Learning from Graphs with Heterophily: Progress and Future"

The paper "Towards Learning from Graphs with Heterophily: Progress and Future" offers a thorough investigation into the burgeoning field of graph learning with heterophily, addressing the unique challenges presented when nodes in a graph tend to connect with dissimilar nodes. Unlike the common assumption of homophily, where similar nodes are typically connected, heterophilous graphs are more representative of many real-world scenarios such as social networks with bots, urban computational networks, and gene regulation networks.

Key Contributions

The authors Chenghua Gong et al. systematically analyze over 180 publications relating to heterophilous graph learning and present a hierarchical taxonomy to categorize this research landscape. The major components of this taxonomy include:

  1. Learning Strategies: This component is further divided into several methods, with significant focus on both (semi-)supervised and self-supervised learning paradigms:
    • (Semi-)Supervised Learning: Several advancements have been made, such as leveraging adaptive filters to capture high-frequency signals beyond traditional GNNs, exploring non-local graph homophily to enhance expressive power, and innovative models like GPR-GNN and MixHop.
    • Self-Supervised Learning: This includes leveraging contrastive and generative learning approaches to enhance representation learning without labeled data. Methods like MUSE and DGCN are examples of how contrastive and generative frameworks adapt to the presence of heterophily.
  2. Model Architectures: The paper delineates between message-passing neural networks (MPNNs) and graph transformers:
    • MPNNs: Researchers developed decoupled message-passing architectures, e.g., GCNII, to combat over-smoothing and better capture the richness of graph heterophily.
    • Graph Transformers: These models, like NodeFormer and SignGT, are effective due to their global attention mechanisms, albeit with high computational overheads.
  3. Practical Applications: The study highlights how heterophilous graph learning is applied in diverse fields:
    • Social Networks: Applications include fraud and automated account detection, where models like DRAG and BotSCL have been proposed to handle the graph heterophily inherent in these networks.
    • Biochemical Networks: In drug discovery and gene regulation, methods capitalize on the heterophily within networks to identify effective drug combinations and reconstruct regulatory networks.

Implications and Future Work

The nuanced analysis provided in the paper directly supports several research themes:

  • Benchmarking and Datasets: The authors suggest the need for improved, large-scale, and diverse benchmark datasets that better capture the complexity of heterophilous graphs.
  • Robustness and Explainability: Future research should prioritize developing models that are robust to attacks and noise and transparent in operations, catering to both domain experts and end-users.
  • Innovative Metrics: As current metrics for measuring graph heterophily have limitations, the development of advanced metrics is essential for accurately assessing model performance and guiding network designs.
  • Extended Applications: Further exploration into unmapped domains like weakly-supervised and few-shot learning could uncover new opportunities and leverage the potential of heterophilous graph frameworks.

Overall, this comprehensive review highlights the rich and rapidly expanding domain of learning from graphs with heterophily, underscoring its theoretical and practical significance. The paper calls for continued innovation in the methodologies, infrastructure, and application realms to fully exploit the potential heterophilous graphs hold for complex data representation.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free