KITE-DDI: A Knowledge graph Integrated Transformer Model for accurately predicting Drug-Drug Interaction Events from Drug SMILES and Biomedical Knowledge Graph (2412.05770v1)

Abstract: It is a common practice in modern medicine to prescribe multiple medications simultaneously to treat diseases. However, these medications could have adverse reactions between them, known as Drug-Drug Interactions (DDI), which have the potential to cause significant bodily injury and could even be fatal. Hence, it is essential to identify all the DDI events before prescribing multiple drugs to a patient. Most contemporary research for predicting DDI events relies on either information from Biomedical Knowledge graphs (KG) or drug SMILES, with very few managing to merge data from both to make predictions. While others use heuristic algorithms to extract features from SMILES and KGs, which are then fed into a Deep Learning framework to generate output. In this study, we propose a KG-integrated Transformer architecture to generate an end-to-end fully automated Machine Learning pipeline for predicting DDI events with high accuracy. The algorithm takes full-scale molecular SMILES sequences of a pair of drugs and a biomedical KG as input and predicts the interaction between the two drugs with high precision. The results show superior performance in two different benchmark datasets compared to existing state-of-the-art models especially when the test and training sets contain distinct sets of drug molecules. This demonstrates the strong generalization of the proposed model, indicating its potential for DDI event prediction for newly developed drugs. The model does not depend on heuristic models for generating embeddings and has a minimal number of hyperparameters, making it easy to use while demonstrating outstanding performance in low-data scenarios.

• The paper introduces a unified transformer model that combines drug SMILES and biomedical knowledge graphs to achieve superior DDI prediction accuracy.
• Methodology employs an end-to-end learning framework with pretraining and fine-tuning, outperforming benchmarks in both transductive and inductive settings.
• The research underscores potential for enhanced pharmacovigilance and drug discovery, especially in data-poor environments and novel therapeutic areas.

Overview of the KITE-DDI Model for Drug-Drug Interaction Prediction

The paper addresses the critical issue of predicting drug-drug interaction (DDI) events by proposing a novel machine learning model titled KITE-DDI. The proposed model integrates the Simplified Molecular Input Line Entry System (SMILES) of drugs with a biomedical knowledge graph (KG) to enhance the prediction accuracy of DDI events. The motivation stems from the inherent risks posed by adverse interactions between co-administered drugs, which are exacerbated by the growing number of approved pharmaceuticals.

Key Contributions

KITE-DDI stands out by uniting two significant streams of information: drug SMILES and KGs, into a cohesive predictive model. Unlike many prior approaches that depended on heuristic feature extraction or segmented models, KITE-DDI creates an end-to-end pipeline, eschewing the reliance on external knowledge or intermediate prediction steps. This model is custom-built with a transformer architecture that employs self-attention mechanisms, pretraining, and fine-tuning processes to effectively learn the underlying patterns in the drug interaction data.

The paper delineates four substantial contributions:

1. Model Efficiency and Performance: KITE-DDI is shown to outperform several state-of-the-art models in DDI prediction tasks, especially notable in scenarios with inductive settings, where drug pairs in the test sets were not present in the training data. This illustrates the model's strong generalization capabilities.
2. End-to-End Predictive Design: The model does not utilize any heuristic components reliant on domain expertise, which increases its ease of use and lowers the computational load in producing results.
3. Simple Input Requirements: The proposed model requires only two inputs (a knowledge graph and SMILES), compared to five required by the benchmark MSEDDI model, reducing dependencies on additional complex algorithms.
4. Superior Performance in Low Data Contexts: The architecture of KITE-DDI shows its strength in data-poor environments, maintaining competitive precision and recall without extensive training datasets.

Methodology

The experimental section indicates that two datasets, derived from DrugBank, were employed to evaluate the model. These datasets incorporated over 100 unique DDI classes and were prepared through a comprehensive series of preprocessing steps to ensure their suitability for the training and test phases. The KITE-DDI model leverages techniques like fivefold cross-validation and novel data splits (U1 and U2) to explore both transductive and inductive learning settings, demonstrating resilience even when encountering unseen drug pairs.

The model's architecture revolves around the advanced implementation of a transformer with multi-headed and self-attention layers. Moreover, it includes a convolutional block for feature extraction from drug embeddings. The paper further details the meticulous pretraining process wherein the model is initially exposed to SMILES data in an unsupervised manner, which enhances its foundational understanding before fine-tuning on labeled datasets.

Results

KITE-DDI surpasses existing benchmark models like DeepDDI and MSD-SA-DDI in several metrics (accuracy, F1 score, AUPR, and AUC) for both evaluated datasets. Notably, the model achieves an accuracy of 51.29% in the challenging U2 split of Dataset 1, with the performance improvements attributed to its ability to generalize from limited data.

Tables and figures within the paper highlight the different performance metrics and structural variations tested during the ablation paper. This paper establishes that the integration of the Knowledge Graph into the model's architecture contributes significantly to its predictive capability.

Implications and Future Directions

KITE-DDI’s significant implications revolve around its potential utility in pharmacovigilance and drug discovery, providing medical professionals and researchers with reliable predictive insights into possible DDI events. The paper suggests expanding to other forms of drugs, such as RNA-based therapeutics, and addressing data imbalances in training sets for rarer DDIs to improve model comprehensiveness and applicability.

This work establishes a foundation for future investigations into the efficient utilization of transformers in bioinformatics and their ability to unify diverse data types for improved predictive modelling.