DLGNet: Hyperedge Classification through Directed Line Graphs for Chemical Reactions (2410.06969v1)

Abstract: Graphs and hypergraphs provide powerful abstractions for modeling interactions among a set of entities of interest and have been attracting a growing interest in the literature thanks to many successful applications in several fields. In particular, they are rapidly expanding in domains such as chemistry and biology, especially in the areas of drug discovery and molecule generation. One of the areas witnessing the fasted growth is the chemical reactions field, where chemical reactions can be naturally encoded as directed hyperedges of a hypergraph. In this paper, we address the chemical reaction classification problem by introducing the notation of a Directed Line Graph (DGL) associated with a given directed hypergraph. On top of it, we build the Directed Line Graph Network (DLGNet), the first spectral-based Graph Neural Network (GNN) expressly designed to operate on a hypergraph via its DLG transformation. The foundation of DLGNet is a novel Hermitian matrix, the Directed Line Graph Laplacian, which compactly encodes the directionality of the interactions taking place within the directed hyperedges of the hypergraph thanks to the DLG representation. The Directed Line Graph Laplacian enjoys many desirable properties, including admitting an eigenvalue decomposition and being positive semidefinite, which make it well-suited for its adoption within a spectral-based GNN. Through extensive experiments on chemical reaction datasets, we show that DGLNet significantly outperforms the existing approaches, achieving on a collection of real-world datasets an average relative-percentage-difference improvement of 33.01%, with a maximum improvement of 37.71%.

• The paper introduces DLGNet, a spectral Graph Neural Network designed for hyperedge classification using a novel Directed Line Graph representation.
• DLGNet utilizes complex-valued edge weights within its Directed Line Graph to capture and leverage the directional information present in directed hypergraphs.
• Experimental results demonstrate that DLGNet significantly outperforms existing methods across chemical reaction datasets, highlighting the value of modeling directionality.

The paper introduces Directed Line Graph Network (DLGNet), a spectral-based Graph Neural Network (GNN) designed for hyperedge classification in directed hypergraphs, with a specific application to chemical reaction classification.

The authors define the concept of a Directed Line Graph (DLG) associated with a directed hypergraph $\vec H$. In this DLG(%%%%1%%%%), vertices represent the hyperedges of $\vec H$, and edges connect vertices if their corresponding hyperedges in $\vec H$ share at least one vertex. Complex-valued edge weights in DLG($\vec H$) encode the directionality of interactions within $\vec H$.

Key contributions include:

• A formal definition of a directed line graph associated with a directed hypergraph $\vec H$, denoted as DLG($\vec H$).
• The Directed Line Graph Laplacian $\mathbb{\vec L}_N$, a Hermitian matrix capturing both directed and undirected relationships between hyperedges in a directed hypergraph via its DLG. The paper proves that $\mathbb{\vec L}_N$ possesses spectral properties such as being positive semidefinite.
• DLGNet, a spectral-based GNN designed to operate on directed line graphs, convolving hyperedge features.

The paper defines an undirected hypergraph as an ordered pair $H = (V, E)$, with $n := |V|$ and $m := |E|$, where $V$ is the set of vertices and $E \subseteq 2^{V} \setminus \{\}$ is the set of hyperedges. The hyperedges' weights are stored in the diagonal matrix $W \in \mathbb{R}^{m \times m}$, where $w_e$ is the weight of hyperedge $e \in E$. The vertex degree $d_u$ and hyperedge degree $\delta_e$ are defined as $d_u := \sum_{e \in E: u \in e} |w_e|$ for $u \in V$, and $\delta_e := |e|$ for $e \in E$, stored in diagonal matrices $D_v \in \mathbb{R}^{n \times n}$ and $D_e \in \mathbb{R}^{m \times m}$. For 2-uniform hypergraphs, the adjacency matrix $A \in \mathbb{R}^{n \times n}$ is defined such that $A_{uv} = w_e$ for each $e=\{u,v\} \in E$ and $A_{uv} = 0$ otherwise. Directed hypergraph $\vec H$ is defined as a hypergraph where each hyperedge $e \in E$ is partitioned in a head set $H(e)$ and a tail set $T(e)$.

The relationship between vertices and hyperedges in a undirected hypergraph ${H}$ is classically represented via an incidence matrix $B$ of size $|V| \times |E|$, where

$${B_{ve} = \begin{cases} 1 & \text{if } v \in e \ 0 & \text{otherwise} \end{cases} \qquad v \in V, e \in E.}$$From the incidence matrix$B$, one can derive the Signless Laplacian Matrix$Q$as well as its normalized version$Q_N$:$Q := B W B^{\top$and$$Q_N} := {D_v} B W D_e^{-1} B^\top {D_v}$$,

where $W, D_e, D_v$ are the diagonal matrices defined above.

The Laplacian for a general undirected hypergraph is defined as:

$\Delta := I-Q_N$.

Given a Laplacian matrix $\mathcal{L}$ of a hypergraph $H$ that admits an eigenvalue decomposition $\mathcal{L} = U \Lambda U^*$, where $U \in \mathbb{C}^{n \times n}$ represents the eigenvectors, $U^*$ is its conjugate transpose, and $\Lambda \in \mathbb{R}^{n \times n}$ is the diagonal matrix containing the eigenvalues, the convolution $y \circledast x$ between $x$ and another graph signal $y \in \mathbb{C}^n$ is defined in the frequency space as $y \circledast x = U \text{diag}(U^* y) U^* x$.

The adjacency matrix of $L(H)$ is defined as:

$A(L(H)) := \mathbb{Q} - W D_e$,

where $\mathbb{Q} := B^\top B$ is the Signless Laplacian of $L(H)$. The normalized Signless Laplacian $\mathbb{Q}_{N}$ and the normalized Laplacian $\mathbb{L}_N$ are defined as:

$\mathbb{Q} := {W}B^\top B {W}$, ${\mathbb{Q}_{N} := {D}_e} {W} {B}^\top {D}_v^{-1}{B} {W} {D}_e}$, and $\mathbb{L}_N := I - \mathbb{Q}_{N}$.

The complex-valued incidence matrix $\vec{B}$ preserves the directionality of $\vec{H}$:

$$\vec{B}_{ve} := \begin{cases} 1 & \text{if } v \in H(e), \ -i & \text{if } v \in T(e), \ 0 & \text{otherwise}. \end{cases} \qquad v \in V, e \in E.$$The adjacency matrix is computed as:$$A(DLG(\vec{H})) = {W}\vec{B}^* \vec{B}{W} - W D_e$$.

The normalized Signless Laplacian $\mathbb{\vec{Q}_{N}$ and the normalized Laplacian $\mathbb{\vec{L}_N$ of $DLG$ are:

$\mathbb{\vec{Q}_{N} := {\vec{D}_e} {W} \vec{B}^* \vec{D}_v^{-1} \vec{B} {W} {\vec{D}_e}$ and $\mathbb{\vec{L}_N := I - \mathbb{\vec{Q}_{N}}$.

The scalar form of $\mathbb{\vec L}_N(ij)$ for a pair of hyperedges $i, j \in E$ is:

$\mathbb{\vec L}<em>N(ij)= \left{ \begin{array}{lr} \displaystyle 1 - \sum</em>{ u \in i}\frac{w_i}{d_u\delta_i}&amp; i = j\ \displaystyle \left(-\hspace{-.4cm}\sum_{\substack{%i,j \in E: \ u \in H(i) \cap H(j) \ \vee u \in T(i) \cap T(j)} \hspace{-.5cm}\frac{w_i} {w_j}{d_u} - i \left(\sum_{\substack{%i, j \in E:\ u \in H(i) \cap T(j)}%\ \wedge u \in T(j)} \hspace{-.3cm}\frac{w_i} {w_j}{d_u} - \sum_{\substack{%i,j \in E:\ u \in T(i) \cap H(j)} %\ \wedge u \in H(j)} \hspace{-.3cm}\frac{w_i} {w_j}{d_u}\right)\right) \frac{1}{\delta_i} \frac{1}{\delta_j} &amp; i \neq j \end{array} \right.$The Euclidean norm induced by$\mathbb{\vec L}<em>N$of a complex-valued signal$x = a + i b \in \mathbb{C}^{{n}$reads:$$\frac{1}{2}} \sum</em>{u \in V} \frac{1}{d(u)} \sum_{i, j \in E} {w(i)} \Bigg( \left(\left(\frac{a_i}{\delta(i)} - \frac{a_j}{\delta(j)} \right)² + \left(\frac{b_i}{\delta(i)} - \frac{b_j}{\delta(j)}\right)^2\right) \mathbf{1}<em>{ u \in H(i) \cap H(j) \vee u \in T(i) \cap T(j)}\ + \left(\left(\frac{a_i}{\delta(i)} - \frac{b_j}{\delta(j)} \right)² + \left(\frac{a_j}{\delta(j)} + \frac{b_i}{\delta(i)}\right)^2\right) \mathbf{1}</em>{u\in H(i) \cap T(j)} \ + \left( \left(\frac{a_i}{\delta(i)} + \frac{b_j}{\delta(j)} \right)² + \left(\frac{a_j}{\delta(j)} - \frac{b_i}{\delta(i)}\right)^2\right) \mathbf{1}_{u\in T(i) \cap H(j)} \Bigg) {w(j)}$$.

The convolution operator is defined as $$\hat{Y} x = \theta_0 I + \theta_1 \mathbb{\vec L}_N$$.

Given $X \in \mathbb{C}^{m \times c_0}$ as a $c_0$-dimensional graph signal, the feature matrix for the vertices of $DGL(\vec H)$ is defined as $X = \vec{B}^* X'$, where $X' \in \mathbb{C}^{n \times c_0}$ is the feature matrix of the nodes of $\vec H$.

The convolution is computed as:

$$Z (X) = \phi\left(IX\Theta_0 + \mathbb{\vec L}_N X\Theta_1\right)$$,

where $\phi$ is a complex ReLU activation function, and $\Theta_0, \Theta_1 \in \mathbb{C}^{c_0 \times c}$ are learnable parameters.

The paper presents experiments conducted on three real-world chemical reaction datasets: {\tt Dataset-1} (50K reactions from USPTO granted patents), {\tt Dataset-2} (5300 reactions from five different sources), and {\tt Dataset-3} (649 competitive reactions extracted from \cite{von2020thousands}). Node features are based on Morgan Fingerprints (MFs).

The results demonstrate that DLGNet outperforms existing methods, achieving an average relative percentage difference improvement of 33.01\% over the second-best method across three real-world datasets. Specifically, DLGNet achieves the best improvement on {\tt Dataset-3}, with an average RPD improvement of approximately 37.71\% and an average additive improvement of 31.65 percentage points.

An ablation paper demonstrates the importance of directionality, showing that DLGNet consistently outperforms its undirected counterpart.