On Valid Optimal Assignment Kernels and Applications to Graph Classification (1606.01141v3)

Abstract: The success of kernel methods has initiated the design of novel positive semidefinite functions, in particular for structured data. A leading design paradigm for this is the convolution kernel, which decomposes structured objects into their parts and sums over all pairs of parts. Assignment kernels, in contrast, are obtained from an optimal bijection between parts, which can provide a more valid notion of similarity. In general however, optimal assignments yield indefinite functions, which complicates their use in kernel methods. We characterize a class of base kernels used to compare parts that guarantees positive semidefinite optimal assignment kernels. These base kernels give rise to hierarchies from which the optimal assignment kernels are computed in linear time by histogram intersection. We apply these results by developing the Weisfeiler-Lehman optimal assignment kernel for graphs. It provides high classification accuracy on widely-used benchmark data sets improving over the original Weisfeiler-Lehman kernel.

• The paper introduces valid optimal assignment kernels for graph classification, characterizing strong base kernels needed to guarantee positive semidefinite properties.
• A hierarchical approach is presented allowing efficient, linear-time computation of these optimal assignment kernels using histogram intersections.
• An empirical evaluation shows the proposed Weisfeiler-Lehman optimal assignment kernel achieves superior classification accuracy on benchmark graph datasets like NCI1 and Reddit.

On Valid Optimal Assignment Kernels and Applications to Graph Classification

The paper by Nils M. Kriege, Pierre-Louis Giscard, and Richard C. Wilson addresses the creation of valid optimal assignment kernels for graph classification, offering a theoretical foundation and practical implementation for utilizing kernel methods on structured data. This work situates itself within the broader context of kernel methods, a cornerstone of machine learning that involves mapping input data into high-dimensional feature spaces to facilitate linear classification, regression, and clustering. While convolution kernels have been traditionally applied to measure similarity in structured data like graphs, this paper revisits the concept of optimal assignment kernels.

Key Innovations and Methodology:

1. Characterizing Strong Kernels: The authors provide a rigorous characterization of strong kernels. These are base kernels that guarantee positive semidefinite (p.s.d.) properties when used to derive optimal assignment kernels. This is pivotal because many assignment-based approaches yield indefinite functions, limiting their applicability in kernel methods.
2. Hierarchical Approach: By introducing a hierarchy-based structure to these kernels, the work demonstrates that optimal assignment kernels can be computed in linear time. This approach comprises constructing hierarchies of graph elements that facilitate efficient and p.s.d. computations of assignment kernels through histogram intersections.
3. Weisfeiler-Lehman Optimal Assignment Kernel: A significant contribution of the paper is the development of the Weisfeiler-Lehman optimal assignment kernel. It extends the traditional Weisfeiler-Lehman kernel by integrating it with the proposed optimal assignment framework, enhancing classification performance across benchmark datasets.

Empirical Evaluation:

The authors validate their theoretical assertions through extensive experiments involving structured datasets, primarily focused on graphs representing molecules and biological networks. The Weisfeiler-Lehman optimal assignment kernel, in particular, demonstrates superior classification accuracy, outperforming existing convolution kernels on several datasets such as NCI1 and Reddit, which suggests the robustness of optimal assignments in capturing graph structural similarities.

Implications and Future Directions:

The implications of this work are manifold. Practically, the research provides a more computationally efficient and semantically valid kernel methodology applicable to graph data, common in cheminformatics and bioinformatics. Theoretically, the connection established between strong kernels and hierarchies might stimulate further exploration into other structured data types and the development of kernel methods in other domains.

Future research may build upon this foundation by integrating domain-specific knowledge into the construction of hierarchies or exploring other feature spaces that may also admit strong kernel properties. Additionally, investigating the generalization capabilities of these kernels across diverse application areas could solidify their utility within machine learning pipelines.

In conclusion, this paper presents significant advancements in kernel methods for machine learning on graph data, offering a foundation for future exploration and application in various scientific and technological domains.