Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

First-order PDES for Graph Neural Networks: Advection And Burgers Equation Models (2404.03081v1)

Published 3 Apr 2024 in cs.LG, cs.NA, and math.NA

Abstract: Graph Neural Networks (GNNs) have established themselves as the preferred methodology in a multitude of domains, ranging from computer vision to computational biology, especially in contexts where data inherently conform to graph structures. While many existing methods have endeavored to model GNNs using various techniques, a prevalent challenge they grapple with is the issue of over-smoothing. This paper presents new Graph Neural Network models that incorporate two first-order Partial Differential Equations (PDEs). These models do not increase complexity but effectively mitigate the over-smoothing problem. Our experimental findings highlight the capacity of our new PDE model to achieve comparable results with higher-order PDE models and fix the over-smoothing problem up to 64 layers. These results underscore the adaptability and versatility of GNNs, indicating that unconventional approaches can yield outcomes on par with established techniques.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (27)
  1. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5115–5124, 2017.
  2. Improving graph attention networks with large margin-based constraints. arXiv preprint arXiv:1910.11945, 2019.
  3. Inductive representation learning on large graphs. Advances in neural information processing systems, NIPS 30, 2017.
  4. Applying and improving alphafold at casp14. Wiley Online Library. Proteins: Structure, Function, and Bioinformatics, 89(12):1711–1721, 2021.
  5. Convolutional Networks on Graphs for Learning Molecular Fingerprints arXiv : 1509 . 09292v2 [ cs . LG ] 3 Nov 2015. pages 1–9.
  6. Link Prediction Based on Graph Neural Networks. NIPS, 2018.
  7. Simple and deep graph convolutional networks. In International conference on machine learning, pages 1725–1735. PMLR, 2020.
  8. Pairnorm: Tackling oversmoothing in gnns. arXiv preprint arXiv:1909.12223, 2019.
  9. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3438–3445, 2020.
  10. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861–6871. PMLR, 2019.
  11. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  12. Improving graph neural networks with learnable propagation operators. In International Conference on Machine Learning, pages 9224–9245. PMLR, 2023.
  13. PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations. Advances in Neural Information Processing Systems, 5(NeurIPS):3836–3849, 2021.
  14. Spectral Networks and Deep Locally Connected Networks on Graphs arXiv : 1312 . 6203v3 [ cs . LG ] 21 May 2014. pages 1–14.
  15. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. Advances in Neural Information Processing Systems, pages 3844–3852, jun 2016.
  16. Semi-supervised classification with graph convolutional networks. 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, pages 1–14, 2017.
  17. Grand: Graph neural diffusion. In International Conference on Machine Learning, pages 1407–1418. PMLR, 2021.
  18. Diffgcn: Graph convolutional networks via differential operators and algebraic multigrid pooling. Advances in neural information processing systems, 33:18016–18027, 2020.
  19. Combining differentiable pde solvers and graph neural networks for fluid flow prediction. In international conference on machine learning, pages 2402–2411. PMLR, 2020.
  20. Adr-gnn: Advection-diffusion-reaction graph neural networks. arXiv preprint arXiv:2307.16092, 2023.
  21. Multi-scale rotation-equivariant graph neural networks for unsteady eulerian fluid dynamics. Physics of Fluids, 34(8), 2022.
  22. Degree-biased advection–diffusion on undirected graphs/networks. Mathematical Modelling of Natural Phenomena, 17:30, 2022.
  23. Collective classification of network data. Data Classification: Algorithms and Applications, pages 399–416, 2014.
  24. 3D ShapeNets: A deep representation for volumetric shapes. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07-12-June:1912–1920, 2015.
  25. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning, pages 40–48. PMLR, 2016.
  26. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287, 2020.
  27. FAUST: Dataset and evaluation for 3D mesh registration. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3794 –3801, Columbus, Ohio, USA, June 2014.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets