Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks (1909.01315v2)

Abstract: Advancing research in the emerging field of deep graph learning requires new tools to support tensor computation over graphs. In this paper, we present the design principles and implementation of Deep Graph Library (DGL). DGL distills the computational patterns of GNNs into a few generalized sparse tensor operations suitable for extensive parallelization. By advocating graph as the central programming abstraction, DGL can perform optimizations transparently. By cautiously adopting a framework-neutral design, DGL allows users to easily port and leverage the existing components across multiple deep learning frameworks. Our evaluation shows that DGL significantly outperforms other popular GNN-oriented frameworks in both speed and memory consumption over a variety of benchmarks and has little overhead for small scale workloads.

• The paper introduces DGL, which employs a graph-centric API to simplify GNN formulation and optimize sparse computations.
• It leverages generalized sparse tensor operations and dynamic parallelization strategies to achieve up to 64x speedup on CPU tasks.
• The framework-neutral design minimizes code modifications, enabling seamless porting of models across PyTorch, TensorFlow, and MXNet.

An Expert Review of the Deep Graph Library for Graph Neural Networks

The paper presents the Deep Graph Library (DGL), a framework designed to facilitate the development and execution of Graph Neural Networks (GNNs). The library is positioned to tackle the challenges posed by the inherent structure of graph data, which does not align well with the traditional tensor-based paradigms of deep learning frameworks. DGL offers a graph-centric programming model that abstracts the complexities associated with graph computations, while maintaining high performance and adaptability to various deep learning ecosystems such as PyTorch, TensorFlow, and MXNet.

Key Contributions

1. Graph as Central Programming Abstraction: DGL introduces graphs as first-class entities in its API, simplifying the formulation of GNNs. The abstraction encapsulates both data manipulation and computational graph architecture, facilitating optimizations that are transparent to the end-user.
2. Generalized Sparse Tensor Operations: The library leverages two key computational patterns—generalized Sparse-dense Matrix Multiplication (g-SpMM) and Sampled Dense-Dense Matrix Multiplication (g-SDDMM). These operations are tailored to efficiently process the sparse nature of graph data, optimizing for both throughput and memory use.
3. Parallelization Strategies: DGL employs advanced parallelization techniques to maximize the capabilities of modern tensorized hardware like GPUs. By strategically choosing between node and edge parallelization, DGL achieves high computational efficiency across a range of graph and model configurations.
4. Framework Neutral Design: By keeping the implementation framework-independent, DGL allows models to be easily ported across various deep learning platforms, enabling researchers to leverage the unique strengths of each framework without significant re-engineering.

Numerical Results and Implications

The evaluation of DGL against other GNN libraries demonstrates significant performance improvements. For full graph training tasks, DGL shows up to a 64x speedup compared to PyTorch Geometric (PyG), especially on CPU-bound tasks, attributed to its better handling of sparse data structures. Memory consumption is also remarkably reduced, with DGL using far less memory than PyG in scenarios involving dense edge features, which is crucial for scalability to larger graphs.

The library's neutral design minimizes the code modifications required when porting models between frameworks, significantly reducing developer burden. This feature encourages experimentation and adoption across diverse environments, promoting the integration of GNNs into a wider array of applications.

Future Outlook

DGL provides a robust foundation for the continued evolution of GNNs. As the library assists in bridging the gap between graph-based data and traditional deep learning, it opens possibilities in areas like social network analysis, bioinformatics, and recommendation systems. Future developments may involve further optimizations in auto-parallelization and support for dynamically changing graphs, improving the adaptability and speed of learning systems.

The challenges addressed by DGL—such as optimizing sparse computations and ensuring framework interoperability—are central to extending deep learning capabilities into more complex data structures. Continued advancements in this space are likely to enhance the applicability of GNNs, widening the scope of real-world problems that can be practically addressed using graph-based modeling.