Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TREE-G: Decision Trees Contesting Graph Neural Networks (2207.02760v5)

Published 6 Jul 2022 in cs.LG and cs.AI

Abstract: When dealing with tabular data, models based on decision trees are a popular choice due to their high accuracy on these data types, their ease of application, and explainability properties. However, when it comes to graph-structured data, it is not clear how to apply them effectively, in a way that incorporates the topological information with the tabular data available on the vertices of the graph. To address this challenge, we introduce TREE-G. TREE-G modifies standard decision trees, by introducing a novel split function that is specialized for graph data. Not only does this split function incorporate the node features and the topological information, but it also uses a novel pointer mechanism that allows split nodes to use information computed in previous splits. Therefore, the split function adapts to the predictive task and the graph at hand. We analyze the theoretical properties of TREE-G and demonstrate its benefits empirically on multiple graph and vertex prediction benchmarks. In these experiments, TREE-G consistently outperforms other tree-based models and often outperforms other graph-learning algorithms such as Graph Neural Networks (GNNs) and Graph Kernels, sometimes by large margins. Moreover, TREE-Gs models and their predictions can be explained and visualized

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Feature-based classification of networks. arXiv preprint arXiv:1610.05868.
  2. Measuring abstract reasoning in neural networks. arXiv:1807.04225.
  3. How Attentive are Graph Attention Networks? arXiv:2105.14491.
  4. A simple baseline algorithm for graph classification. arXiv preprint arXiv:1810.09155.
  5. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 34(2): 786—797.
  6. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. In Lee, D.; Sugiyama, M.; Luxburg, U.; Guyon, I.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
  7. XGraphBoost: Extracting Graph Neural Network-Based Features for a Better Prediction of Molecular Properties. Journal of Chemical Information and Modeling, 61(6): 2697–2705. PMID: 34009965.
  8. Convolutional Networks on Graphs for Learning Molecular Fingerprints. arXiv:1509.09292.
  9. Friedman, J. H. 2000. Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, 29: 1189–1232.
  10. Friedman, J. H. 2001. Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189–1232.
  11. Friedman, J. H. 2002. Stochastic Gradient Boosting. Comput. Stat. Data Anal., 38(4): 367–378.
  12. GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensembles. arXiv:2106.07802.
  13. Generalization and Representational Limits of Graph Neural Networks. arXiv:2002.06157.
  14. Neural Message Passing for Quantum Chemistry. arXiv:1704.01212.
  15. Neural Message Passing for Quantum Chemistry.
  16. A new model for learning in graph domains. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, 729–734 vol. 2.
  17. Why do tree-based models still outperform deep learning on tabular data?
  18. On Graph Kernels: Hardness Results and Efficient Alternatives. In Schölkopf, B.; and Warmuth, M. K., eds., Computational Learning Theory and Kernel Machines — Proceedings of the 16th Annual Conference on Computational Learning Theory and 7th Kernel Workshop (COLT/Kernel 2003) August 24-27, 2003, Washington, DC, USA, volume 2777 of Lecture Notes in Computer Science, 129–143. Springer, Berlin–Heidelberg, Germany.
  19. Inductive Representation Learning on Large Graphs. In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  20. Inductive Representation Learning on Large Graphs. arXiv:1706.02216.
  21. Comparison of random forest, artificial neural networks and support vector machine for intelligent diagnosis of rotating machinery. Transactions of the Institute of Measurement and Control, 40(8): 2681–2693.
  22. SimBoost: a read-across approach for predicting drug–target binding affinities using gradient boosting machines. Journal of cheminformatics, 9(1): 1–14.
  23. Heaton, J. 2016. An empirical analysis of feature engineering for predictive modeling. In SoutheastCon 2016, 1–6. IEEE.
  24. Trees with Attention for Set Prediction Tasks. In Meila, M.; and Zhang, T., eds., Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, 4250–4261. PMLR.
  25. Ho, T. K. 1995. Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition, volume 1, 278–282. IEEE.
  26. Open Graph Benchmark: Datasets for Machine Learning on Graphs.
  27. Generative Models for Graph-Based Protein Design. In Wallach, H.; Larochelle, H.; Beygelzimer, A.; d'Alché-Buc, F.; Fox, E.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  28. Boost then Convolve: Gradient Boosting Meets Graph Neural Networks. arXiv:2101.08543.
  29. Residual correlation in graph neural network regression. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 588–598.
  30. Semi-Supervised Classification with Graph Convolutional Networks. arXiv:1609.02907.
  31. Subgraph Matching Kernels for Attributed Graphs. In Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML’12, 291–298. Madison, WI, USA: Omnipress. ISBN 9781450312851.
  32. GBDTCDA: predicting circRNA-disease associations based on gradient boosting decision tree with multiple biological data fusion. International journal of biological sciences, 15(13): 2911.
  33. Link Prediction for Complex Networks via Random Forest. Journal of Physics: Conference Series, 1302: 022030.
  34. Combining Decision Trees and Neural Networks for Learning-to-Rank in Personal Search. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, 2032–2040. New York, NY, USA: Association for Computing Machinery. ISBN 9781450362016.
  35. Gated Graph Sequence Neural Networks. arXiv:1511.05493.
  36. Performance Comparison of Machine Learning Techniques in Identifying Dementia from Open Access Clinical Datasets, 79–89. ISBN 978-981-15-6047-7.
  37. TUDataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020).
  38. Graph Kernels: A Survey. Journal of Artificial Intelligence Research, 72: 943–1027.
  39. Graph Based Link Prediction between Human Phenotypes and Genes. arXiv:2105.11989.
  40. Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations.
  41. DART: Dropouts meet Multiple Additive Regression Trees. arXiv:1505.01866.
  42. IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning. In da Vitoria Lobo, N.; Kasparis, T.; Roli, F.; Kwok, J. T.; Georgiopoulos, M.; Anagnostopoulos, G. C.; and Loog, M., eds., Structural, Syntactic, and Statistical Pattern Recognition, 287–297. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-540-89689-0.
  43. Temporal Graph Networks for Deep Learning on Dynamic Graphs. arXiv:2006.10637.
  44. Weisfeiler-Lehman Graph Kernels. J. Mach. Learn. Res., 12: 2539–2561.
  45. Efficient graphlet kernels for large graph comparison. Journal of Machine Learning Research - Proceedings Track, 5: 488–495.
  46. Tabular Data: Deep Learning is Not All You Need. In 8th ICML Workshop on Automated Machine Learning (AutoML).
  47. Supervised neural networks for the classification of structures. IEEE Transactions on Neural Networks, 8(3): 714–735.
  48. A Deep Learning Approach to Antibiotic Discovery. Cell, 180(4): 688–702.e13.
  49. Graph Attention Networks. In International Conference on Learning Representations.
  50. Link Prediction Based on Random Forest in Signed Social Networks. In 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), volume 02, 251–256.
  51. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1): 4–24.
  52. How Powerful are Graph Neural Networks? In International Conference on Learning Representations.
  53. Deep Graph Kernels. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’15, 1365–1374. New York, NY, USA: Association for Computing Machinery. ISBN 9781450336642.
  54. Revisiting Semi-Supervised Learning with Graph Embeddings.
  55. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now