DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs

Abstract: In this paper, we study the problem of learning probabilistic logical rules for inductive and interpretable link prediction. Despite the importance of inductive link prediction, most previous works focused on transductive link prediction and cannot manage previously unseen entities. Moreover, they are black-box models that are not easily explainable for humans. We propose DRUM, a scalable and differentiable approach for mining first-order logical rules from knowledge graphs which resolves these problems. We motivate our method by making a connection between learning confidence scores for each rule and low-rank tensor approximation. DRUM uses bidirectional RNNs to share useful information across the tasks of learning rules for different relations. We also empirically demonstrate the efficiency of DRUM over existing rule mining methods for inductive link prediction on a variety of benchmark datasets.

• The paper introduces DRUM, a novel differentiable framework that jointly optimizes rule structures and confidence scores for inductive link prediction.
• It leverages tensor-based confidence estimation and bidirectional RNNs to efficiently mine first-order logical Horn clauses across multiple relations.
• Empirical evaluations on Kinship, UMLS, and Family datasets demonstrate DRUM's superior accuracy and enhanced interpretability over traditional methods.

DRUM: End-To-End Differentiable Rule Mining on Knowledge Graphs

The study presents DRUM, an innovative algorithm for differentiable rule mining on knowledge graphs, targeting inductive and interpretable link prediction. This research addresses the limitations of traditional methods that are mostly suited for transductive link prediction and often operate as black-box models, thereby lacking interpretability. By proposing DRUM, the researchers aim to enable the efficient mining of first-order logical rules that are both scalable and capable of managing previously unseen entities.

The key innovative stride of DRUM is its connection of rule mining with low-rank tensor approximation, allowing for efficient rule learning and confidence estimation. DRUM employs bidirectional recurrent neural networks (RNNs) to optimally share information across multiple relations in a knowledge graph, improving upon the existing rule mining techniques that are either restricted by predefined statistical heuristics or are partially non-differentiable.

Methodological Advancements

1. Differentiable Rule Mining: DRUM leverages a fully differentiable framework to explore first-order logical Horn clauses from knowledge graphs. This allows the method to jointly optimize rule structures and their respective confidence scores, a necessity for effective inductive logic programming tasks.
2. Tensor-based Confidence Estimation: The authors establish a theoretical foundation for DRUM by associating confidence score learning of rules with tensor completion problems. This novel use of tensors enhances the expressiveness of DRUM, enabling it to surpass the constraints of prior methods that usually harbor rank-one approximations.
3. Bidirectional Information Sharing: By deploying bidirectional RNNs, DRUM intelligently shares parameters across different tasks, improving rule generalization and decreasing the computational burden associated with learning rules for each individual relation separately.
4. Empirical Demonstrations: In experimental evaluations using benchmarks such as Kinship, UMLS, and Family datasets, DRUM outperforms existing systems like Neural LP, demonstrating superior predictive performance and enhanced rule interpretability.

Implications and Future Directions

The implications of DRUM for the field of AI and machine learning are significant. The approach bridges the gap between purely opaque models and those that offer interpretable, logic-based insights into predictions. Algorithms that can explain predictions and handle unseen data are vital for domains where transparency is paramount, such as healthcare and legal industries.

Future developments may explore the integration of DRUM with representation learning approaches to leverage the best of both worlds—interpretable logic-based rules and the robustness of embeddings. Additionally, future work might improve upon DRUM by incorporating negative sampling strategies to enhance rule precision further.

In conclusion, DRUM represents a substantial step forward in knowledge graph-based rule mining, providing both interpretative power and high efficacy for inductive link prediction tasks. This research lays the groundwork for further exploration into differentiable, scalable, and interpretable AI models.