TUDataset: A collection of benchmark datasets for learning with graphs (2007.08663v1)

Abstract: Recently, there has been an increasing interest in (supervised) learning with graph data, especially using graph neural networks. However, the development of meaningful benchmark datasets and standardized evaluation procedures is lagging, consequently hindering advancements in this area. To address this, we introduce the TUDataset for graph classification and regression. The collection consists of over 120 datasets of varying sizes from a wide range of applications. We provide Python-based data loaders, kernel and graph neural network baseline implementations, and evaluation tools. Here, we give an overview of the datasets, standardized evaluation procedures, and provide baseline experiments. All datasets are available at www.graphlearning.io. The experiments are fully reproducible from the code available at www.github.com/chrsmrrs/tudataset.

• The paper introduces TUDataset, offering over 120 diverse benchmark datasets with standardized evaluation protocols for graph-based learning.
• It compares traditional graph kernels and modern graph neural networks, highlighting the strengths and implications of each approach.
• The resource promotes reproducible research and lays the groundwork for scalable hybrid models applicable across bioinformatics, social networks, and more.

Analysis of "TUDataset: A Collection of Benchmark Datasets for Learning with Graphs"

The paper presents the "TUDataset," a comprehensive collection of benchmark datasets aimed at advancing the field of supervised learning with graph data. The dataset comprises over 120 distinct datasets derived from diverse applications, providing a valuable resource for researchers to develop and evaluate graph-based learning models.

Problem Context and Motivation

Graph-structured data is prevalent across various domains, including bioinformatics, social networks, and computer vision. Despite the proliferation of machine learning models tailored to exploit graph structures, two major challenges persist: a lack of meaningful benchmark datasets and non-standardized evaluation procedures. These issues hinder the fair comparison of results, thereby slowing progress in the field. The "TUDataset" is introduced as a solution to these challenges, providing a wide array of datasets and standardized tools for evaluation.

Datasets and Approaches

The "TUDataset" encompasses graph datasets from numerous domains:

• Small Molecules: These datasets often involve classifying molecular structures regarding their biological or chemical properties.
• Bioinformatics: Includes datasets like DD, Enzymes, and Proteins, where graphs represent macromolecules and predict properties like enzyme class or protein function.
• Computer Vision and Image Processing: Several datasets originate from graph-based representations of visual data.
• Social Networks: These datasets model social structures to predict attributes such as gender or community type.
• Synthetic Datasets: Created to test specific features of graph learning models, such as the ability to handle continuous attributes.

The datasets are accompanied by baseline methods, including implementations of traditional graph kernels and graph neural networks (GNNs). The paper proposes standardized evaluation protocols to ensure comparability across different approaches, employing $10$-fold cross-validation for benchmarks.

Empirical Evaluation

The paper presents experimental results comparing graph kernels and GNNs across various datasets. Notably, traditional graph kernels, such as the Weisfeiler-Lehman Subtree kernel, demonstrate competitive performance despite the recent focus on GNNs. However, GNN architectures like Gin-$\varepsilon$-JK show promise, particularly on large-scale datasets, revealing the utility of neural approaches in handling complex graph structures.

Implications and Future Directions

"TUDataset" provides the community with a robust foundation for evaluating graph-based models, encouraging deeper exploration and refinement of both kernel and neural methodologies. Its wide applicability across domains enhances its utility, potentially leading to more significant advancements in graph representation learning.

The experimental findings suggest several directions for future work. The competitive performance of traditional kernels alongside GNNs highlights the need for further hybrid approaches. Additionally, the dataset's infrastructure fosters research into scalable algorithms capable of handling large, complex graph data seen in real-world applications.

Conclusion

The introduction of "TUDataset" addresses critical gaps in the graph learning domain by offering a diverse and standardized collection of datasets. This initiative is poised to facilitate empirical validation and foster innovation in developing machine learning models adept at leveraging graph-structured data. As researchers build upon this resource, potential advancements are anticipated in both the theoretical and practical aspects of graph-based learning.