2000 character limit reached
Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification
Published 31 Jan 2023 in cs.LG | (2301.13764v3)
Abstract: We present a deep Graph Convolutional Kernel Machine (GCKM) for semi-supervised node classification in graphs. The method is built of two main types of blocks: (i) We introduce unsupervised kernel machine layers propagating the node features in a one-hop neighborhood, using implicit node feature mappings. (ii) We specify a semi-supervised classification kernel machine through the lens of the Fenchel-Young inequality. We derive an effective initialization scheme and efficient end-to-end training algorithm in the dual variables for the full architecture. The main idea underlying GCKM is that, because of the unsupervised core, the final model can achieve higher performance in semi-supervised node classification when few labels are available for training. Experimental results demonstrate the effectiveness of the proposed framework.
- A Gentle Introduction to Deep Learning for Graphs. Neural Networks, 129: 203–221.
- To understand deep learning we need to understand kernel learning. In ICML.
- Bengio, Y. 2009. Learning Deep Architectures for AI. Foundations, 2: 1–55.
- Numerical solution of saddle point problems. Acta Numerica, 14: 1–137.
- Convex Optimization. New York: Cambridge University Press.
- Burges, C. J. 1998. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery, 2: 121–167.
- Convolutional Kernel Networks for Graph-Structured Data. In ICML.
- Adaptive Universal Generalized PageRank Graph Neural Network. In ICLR.
- Support Vector Machines. NY: Springer New York.
- Couto, J. 2005. Kernel K-Means for Categorical Data. In Advances in Intelligent Data Analysis VI, 46–56.
- Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels. In NIPS.
- KerGNNs: Interpretable Graph Neural Networks with Graph Kernels. In AAAI.
- The journey of graph kernels through two decades. Computer Science Review, 27: 88–111.
- Hamilton, W. L. 2020. Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3): 1–159.
- A Note on the Universal Approximation Capability of Support Vector Machines. Neural Processing Letters, 17: 43–53.
- BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation. In NIPS.
- Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited. In NIPS.
- Hornik, K. 1991. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2): 251–257.
- Multilayer feedforward networks are universal approximators. Neural Networks, 2(5): 359–366.
- Open graph benchmark: Datasets for machine learning on graphs. In NIPS.
- Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
- Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In ICLR.
- A survey on graph kernels. Applied Network Science, 5(1): 1–42.
- Soft Kernel Spectral Clustering. In IJCNN.
- ARPACK USERS GUIDE: Solution of Large Scale Eigenvalue Problems by Implicitly Restarted Arnoldi Methods. Philadelphia, PA: SIAM.
- Deriving Neural Architectures from Sequence and Graph Kernels. In ICML.
- Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform. In ICLR.
- Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods. In NIPS.
- Multiclass Semisupervised Learning Based Upon Kernel Spectral Clustering. IEEE Transactions on Neural Networks and Learning Systems, 26(4): 20–33.
- Mercer, J. 1909. Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, Series A, 209(441-458): 415–446.
- Kernel Graph Convolutional Neural Networks. In ICANN.
- Disentangled Representation Learning and Generation With Manifold Optimization. Neural Computation, 34(10): 2009–2036.
- Rockafellar, R. T. 1974. Conjugate Duality and Optimization. SIAM.
- Multi-Scale attributed node embedding. Journal of Complex Networks, 9(1): 1–22.
- Salakhutdinov, R. 2015. Learning Deep Generative Models. Annual Review of Statistics and Its Application, 2(1): 361–385.
- Learning with Kernels. Cambridge, MA: MIT Press.
- Collective Classification in Network Data. AI magazine, 29(3): 93–93.
- Kernels and Regularization on Graphs. In Learning Theory and Kernel Machines, 144–158.
- Suykens, J. A. K. 2017. Deep Restricted Kernel Machines Using Conjugate Feature Duality. Neural Computation, 29(8): 2123–2163.
- Least Squares Support Vector Machines. Singapore: World Scientific.
- A support vector machine formulation to PCA analysis and its Kernel version. IEEE Transactions on Neural Networks, 14(2): 447–450.
- Least Squares Support Vector Machine Classifiers. Neural Processing Letters, 9(3): 293–300.
- Tensor-based Multi-view Spectral Clustering via Shared Latent Space.
- Wasserstein Weisfeiler-Lehman Graph Kernels. In NIPS.
- Unsupervised learning of disentangled representations in deep restricted kernel machines with orthogonality constraints. Neural Networks, 142: 661–679.
- Graph Clustering with Graph Neural Networks. Journal of Machine Learning Research, 24(127): 1–21.
- Vapnik, V. 1998. Statistical Learning Theory. New York: Wiley.
- A reduction of a graph to a canonical form and an algebra arising during this reduction. Nauchno-Technicheskaya Informatsia, 2(9): 12–16.
- A Comprehensive Survey on Graph Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1): 4–24.
- How Powerful are Graph Neural Networks? In ICLR.
- Revisiting Semi-Supervised Learning with Graph Embeddings. In ICML.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.