ICML 2023 Topological Deep Learning Challenge : Design and Results (2309.15188v4)

Abstract: This paper presents the computational challenge on topological deep learning that was hosted within the ICML 2023 Workshop on Topology and Geometry in Machine Learning. The competition asked participants to provide open-source implementations of topological neural networks from the literature by contributing to the python packages TopoNetX (data processing) and TopoModelX (deep learning). The challenge attracted twenty-eight qualifying submissions in its two-month duration. This paper describes the design of the challenge and summarizes its main findings.

• The paper presents a novel open-source challenge that unified topological neural network implementations using the TopoModelX framework.
• The study outlines precise submission criteria focused on code clarity and API adherence to promote reproducible research in complex relational data.
• The challenge outcomes pave the way for standardized benchmarks, encouraging further innovation in leveraging higher-order data structures.

Introduction to Topological Deep Learning

Graph neural networks are a cornerstone of contemporary deep learning, particularly suited for handling relational data by capturing pairwise relations between nodes in graph domains. This is a critical area of AI, where the data's intrinsic structure can be leveraged to produce insightful results in tasks such as social network analysis or molecular classification. A novel approach known as topological neural networks (TNNs) extends the potent capabilities of GNNs by accommodating higher-order relations. This enables the networks to deal with more complex structures known as topological domains, tapping into the intricate web of relationships inherent in the data. Despite showing great potential, TNN adoption has faced barriers due to limited open-source resources and the absence of consistent benchmarking standards.

Challenge Overview and Participation

The challenge at ICML 2023, centered around topological deep learning, aimed to combat these barriers. Participants were invited to contribute open-source implementations of TNNs, enhancing python packages TopoNetX, intended for data processing, and TopoModelX, designed for deep learning functionalities. Enthusiasm for the challenge was evident with twenty-eight submissions received over two months. Contributors submitted their versions of previously published TNN models and implemented these in a unifying open-source framework, underscoring a commitment to accessible and reproducible research in the field.

Submission Requirements and Evaluation

Submissions were assessed primarily on the correctness of the model implementation, with clear, legible code and adherence to TopoModelX's API as key judging criteria. The challenge did not focus on the performance or complexity of the training methods but instead aimed to endorse clean and precise model architectures to support further research in the field. The evaluators used the Condorcet method to rank submissions, allowing participating teams and reviewers to vote on the best model implementations, creating an environment that promoted community-driven development and peer review.

Toward a Unified Topological Framework

The challenge concluded with the announcement of winners for each topological domain and the submission of numerous TNN implementations to the TopoModelX package. This collaborative effort is expected to pave the way for more standardized benchmarks and spur further innovation in topological deep learning. It's a significant step toward developing deeper, more nuanced AI models that can better discern and utilize the rich, relational structures present in complex datasets. As the field continues to evolve, community engagement of this kind is likely to shape the future of topological modeling in AI.