MHCADDI: Multi-Head Co-Attentive DDI Encoder

• The paper introduces a novel MHCADDI architecture that integrates message passing with multi-head co-attention to enhance atom-level drug pair representations.
• It fuses early joint information from both drugs, enabling precise modeling of adverse interactions via detailed molecular graph encoding.
• Comparative analysis with models like RGDA-DDI highlights MHCADDI’s modularity and potential for improved prediction metrics such as AUC and F1-score.

A Multi-Head Co-Attentive Drug-Drug Interaction Encoder (MHCADDI) is a neural network architecture specifically designed to predict adverse effects arising from drug–drug interactions (DDIs) by operating directly on molecular graph representations of drug pairs. It leverages a novel integration of message-passing neural networks (MPNNs) with multi-head cross-drug co-attention mechanisms. The principal innovation is the incorporation of joint information from both drugs in the pair as early as possible when constructing atom-level representations, allowing for finer modeling of molecular interactions that may result in side effects (Deac et al., 2019).

1. Molecular Graph Representation

Each drug $d_x$ is modeled as an undirected molecular graph. The atomic structure is encoded as follows:

• Node (Atom) Features: Each atom $a_i^{(d_x)}$ includes:
  1. Atom type identifier (one-hot, projected to 32 dimensions)
  2. Number of bonded hydrogen atoms
  3. Formal atomic charge
  4. A learned 32-dimensional embedding per atom type

• Edge (Bond) Features: Each bond $e_{ij}^{(d_x)}$ is represented by a learned 32-dimensional embedding of bond category (single, double, etc.)

• Preprocessing Pipeline:

1. Retrieve molecular graph via PubChem ID and extract atom/bond attributes 2. Negative sampling strategy: In binary classification, generate negative triplets by corrupting one drug in each positive $(d_x, d_y, se_z)$ trio, where $se_z$ denotes the side-effect label.

2. Message Passing and Co-Attention

The MHCADDI architecture alternates between intra-drug message-passing and cross-drug co-attention for $T=3$ layers.

2.1. Intra-Drug Message Passing

Each atom feature is initialized by a projection: $${}^{(d_x)}h_i^{0} = f_i(a_i^{(d_x)}) \in \mathbb{R}^{32}$$.

At each step $t$, messages are computed as: $${}^{(d_x)}m_{ij}^t = f_e^t(e_{ij}^{(d_x)}) \odot f_v^t({}^{(d_x)}h_j^{t-1})$$ where $f_e^t$ is a two-layer LeakyReLU MLP (each $a_i^{(d_x)}$0), $a_i^{(d_x)}$1 is a single-layer projection ($a_i^{(d_x)}$2). Aggregation is by summation over neighbors.

2.2. Cross-Drug Co-Attention

For each atom $a_i^{(d_x)}$3 in $a_i^{(d_x)}$4 and $a_i^{(d_x)}$5 in $a_i^{(d_x)}$6:

Per head $a_i^{(d_x)}$7: $a_i^{(d_x)}$8 Attention weights are: $a_i^{(d_x)}$9 The attended message is: $e_{ij}^{(d_x)}$0 Outputs from the $e_{ij}^{(d_x)}$1 heads are concatenated to a $e_{ij}^{(d_x)}$2-dimensional vector and linearly projected (via $e_{ij}^{(d_x)}$3) back to $e_{ij}^{(d_x)}$4 dimensions for each atom.

An identical block is applied in the reverse direction ($e_{ij}^{(d_x)}$5).

2.3. Feature Update

The atom feature update includes normalization and residuals: $e_{ij}^{(d_x)}$6

3. Multi-Head Extension

MHCADDI employs $e_{ij}^{(d_x)}$7 independent attention heads per layer. Each head has distinct projection matrices; their output vectors ($e_{ij}^{(d_x)}$8-dim each) are concatenated into a $e_{ij}^{(d_x)}$9-dim vector and then projected back to $(d_x, d_y, se_z)$0 dimensions. This design enables modeling of multiple, potentially diverse, interaction types at the atom level, and enhances the expressiveness of the joint drug representation (Deac et al., 2019).

4. Readout and Drug-Pair Embedding

Following $(d_x, d_y, se_z)$1 interleaved message-passing/co-attention layers, final atom representations are aggregated for each drug: $(d_x, d_y, se_z)$2 where $(d_x, d_y, se_z)$3 is a single-layer LeakyReLU MLP ($(d_x, d_y, se_z)$4).

For prediction, drug pair embeddings are concatenated: $(d_x, d_y, se_z)$5 serving as input for downstream scoring modules.

5. Prediction Heads and Training Procedure

5.1. Binary Classification (Per-Side Effect Ranking)

Input triplet: $(d_x, d_y, se_z)$6, where $(d_x, d_y, se_z)$7 is a one-hot vector over 964 side effect types. The matching score is defined as: $(d_x, d_y, se_z)$8 with $(d_x, d_y, se_z)$9.

Training minimizes the margin-based ranking loss: $se_z$0 where $se_z$1.

5.2. Multi-Label Classification (All Side Effects)

Predicts a 964-dimensional vector: $se_z$2 using per-label binary cross-entropy loss.

5.3. Hyperparameters and Optimization

• Layers ($se_z$3): 3 interleaved message/co-attention
• Hidden dim: 32
• Attention heads: 8
• Dropout: 0.2 after each MLP or projection
• Optimizer: Adam, batch size 200, 30 epochs
• Learning rate: $se_z$4
• Parameter initialization: Xavier uniform

MLP specifics:

• $se_z$5: single-layer, no bias
• $se_z$6: single-layer, no bias
• $se_z$7: two-layer LeakyReLU, each $se_z$8
• $se_z$9: single-layer LeakyReLU ($T=3$0)
• $T=3$1: single-layer LeakyReLU ($T=3$2)
• $T=3$3: $T=3$4, $T=3$5

6. Relation to Recent Graph Neural Approaches

Recent architectures such as RGDA-DDI (Zhou et al., 2024) use deeper residual GAT stacks, dual-attention fusion blocks, and multi-scale hierarchical pooling to increase representational power. In contrast, MHCADDI employs parallel multi-head co-attentive message-passing, integrating pairwise context at the atom level but does not implement: (a) separate substructure/global-structure GNN stacks, (b) hierarchical (layer-wise) SAGPooling, or (c) explicit dual (drug–drug and drug–DDP) attention for fusion across multiple feature spaces. In RGDA-DDI, these enhancements yielded improvements in metrics such as AUC and F1-score on large-scale DDI datasets, suggesting that while MHCADDI introduced the key paradigm of multi-head co-attentional fusion, further architectural depth and explicit dual-attention mechanisms can further improve predictive accuracy (Zhou et al., 2024).

7. Practical Considerations and Implementation Notes

MHCADDI is suitable for end-to-end learning on large DDI datasets. All message-passing and attention operations are implemented using small feedforward networks with standard non-linearities (LeakyReLU), and attention heads scale linearly with $T=3$6. Layer normalization and residual updates mitigate representation drift due to deep architectures. Drug graphs should be preprocessed from PubChem/SMILES with atomic and bond features as described. The architectural modularity of MHCADDI allows adaptation to different molecular feature types and side-effect ontologies by modifying input encodings or adjusting the number of output labels (Deac et al., 2019).