cuSLINK: Single-linkage Agglomerative Clustering on the GPU (2306.16354v1)

Abstract: In this paper, we propose cuSLINK, a novel and state-of-the-art reformulation of the SLINK algorithm on the GPU which requires only $O(Nk)$ space and uses a parameter $k$ to trade off space and time. We also propose a set of novel and reusable building blocks that compose cuSLINK. These building blocks include highly optimized computational patterns for $k$-NN graph construction, spanning trees, and dendrogram cluster extraction. We show how we used our primitives to implement cuSLINK end-to-end on the GPU, further enabling a wide range of real-world data mining and machine learning applications that were once intractable. In addition to being a primary computational bottleneck in the popular HDBSCAN algorithm, the impact of our end-to-end cuSLINK algorithm spans a large range of important applications, including cluster analysis in social and computer networks, natural language processing, and computer vision. Users can obtain cuSLINK at https://docs.rapids.ai/api/cuml/latest/api/#agglomerative-clustering

• The paper presents an MST-driven reformulation of the SLINK algorithm that maximizes GPU parallelism to outperform traditional CPU methods by up to 2290x.
• It employs optimized k-nearest neighbor searches and an innovative adaptation of Borůvka’s algorithm to efficiently process massive datasets.
• The approach enables scalable hierarchical clustering with practical applications in genomics, NLP, and computer vision, and is available as open-source software.

Overview of "cuSLINK: Single-linkage Agglomerative Clustering on the GPU"

The paper introduces cuSLINK, a sophisticated reformulation of the SLINK algorithm adapted for GPU architectures, offering significant advancements in hierarchical agglomerative clustering (HAC). Remarkably, it addresses both computational and space complexities, typically seen as obstacles for clustering large datasets, by innovatively utilizing a Minimum Spanning Tree (MST)-based approach.

Algorithmic Innovations and Methodology

cuSLINK leverages a combination of novel GPU-optimized algorithms, modular primitives, and a unique reformulation of the single-linkage clustering technique. It modifies the classic SLINK algorithm by introducing an MST-based approach, significantly increasing parallel CPU operations compared to purely sequential methods. Specifically, the implementation relies on a minimum spanning tree strategy, effectively handling complexity by enabling more parallelization.

Key Components:

1. Nearest Neighbor Search: The paper develops a fused approach to k-nearest neighbor (k-NN) search on GPUs, leveraging shared and register memory to perform selection and distance computations efficiently. This method results in notable speedups compared to existing implementations like FAISS.
2. Spanning Tree Construction: The construction of the MST is achieved using a novel adaptation of Borůvka’s algorithm. By avoiding explicit graph coarsening, it optimizes memory usage—making it capable of processing vast datasets that exceed a billion edges.
3. Dendrogram Construction: The algorithm builds the hierarchical clustering dendrogram separately post-MST construction to maximize exploitable parallelism. This separation allows a reduction in computational overhead.

Performance Evaluation and Results

The results presented in the paper showcase substantial improvements in clustering speeds, particularly in handling datasets considered intractable with legacy methods. The cuSLINK model is reported to be up to 2290 times faster than traditional CPU-based algorithms, as demonstrated through benchmarks on varied high-dimensional datasets.

Implications and Future Directions

1. Practical Applications: cuSLINK’s ability to handle large datasets efficiently has critical implications for disciplines like genomics, natural language processing, and computer vision, which frequently utilize hierarchical clustering.
2. Theoretical Contributions: From a theoretical standpoint, the paper contributes to our understanding of parallel processing capabilities, particularly in non-trivial algorithm reformulations for GPU architecture.
3. Open Source Availability: The availability of cuSLINK and its primitives in the open-source RAFT library increases accessibility, encouraging further exploration and enhancement by the research community.

Conclusion

cuSLINK represents a significant computational achievement in its domain, effectively marrying theoretical innovations with practical implementation. The groundwork laid by cuSLINK suggests promising directions for continued exploration, particularly in optimizing algorithms traditionally considered computationally intense, to suit modern parallel computing environments. These insights pave the way for further enhancements in algorithmic development and GPU applications across complex data processing tasks.