2000 character limit reached
Entropy Aware Message Passing in Graph Neural Networks
Published 7 Mar 2024 in cs.LG | (2403.04636v1)
Abstract: Deep Graph Neural Networks struggle with oversmoothing. This paper introduces a novel, physics-inspired GNN model designed to mitigate this issue. Our approach integrates with existing GNN architectures, introducing an entropy-aware message passing term. This term performs gradient ascent on the entropy during node aggregation, thereby preserving a certain degree of entropy in the embeddings. We conduct a comparative analysis of our model against state-of-the-art GNNs across various common datasets.
- Grand: Graph neural diffusion. In International Conference on Machine Learning, pp. 1407–1418. PMLR, 2021.
- Simple and deep graph convolutional networks. In International conference on machine learning, pp. 1725–1735. PMLR, 2020.
- Graph neural networks for social recommendation. In The world wide web conference, pp. 417–426, 2019.
- Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, pp. 89–98, 1998.
- Simple gnn regularisation for 3d molecular property prediction & beyond. arXiv preprint arXiv:2106.07971, 2021.
- Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2018.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
- Deepgcns: Can gcns go as deep as cnns? In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9267–9276, 2019.
- Automating the construction of internet portals with machine learning. Information Retrieval, 3:127–163, 2000.
- Rankme: Reliable human ratings for natural language generation. arXiv preprint arXiv:1803.05928, 2018.
- Gradient gating for deep multi-rate learning on graphs. arXiv preprint arXiv:2210.00513, 2022.
- A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993, 2023.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Acmp: Allen-cahn message passing for graph neural networks with particle phase transition. arXiv preprint arXiv:2206.05437, 2022.
- Chemistry-intuitive explanation of graph neural networks for molecular property prediction with substructure masking. Nature Communications, 14(1):2585, May 2023. ISSN 2041-1723. doi: 10.1038/s41467-023-38192-3. URL https://doi.org/10.1038/s41467-023-38192-3.
- Pairnorm: Tackling oversmoothing in gnns. arXiv preprint arXiv:1909.12223, 2019.
- Dirichlet energy constrained learning for deep graph neural networks. Advances in Neural Information Processing Systems, 34:21834–21846, 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper.