Structure-aware HyperGraph Transformer

• The paper introduces SHGT, a framework for next-visit diagnosis prediction that models EHRs as hypergraphs with medical codes as nodes and visits as hyperedges.
• It employs an incidence-derived structural encoder fused with code and visit embeddings via a Transformer to capture both local and global interactions.
• Experimental results on MIMIC-III and MIMIC-IV datasets show notable improvements in F1, Recall@10, and Recall@20 compared to baseline methods.

Structure-aware HyperGraph Transformer (SHGT) is a framework for next-visit diagnosis prediction from electronic health records (EHR) that represents the entire corpus as a hypergraph in which medical codes are nodes and visits are hyperedges. Its defining combination is an incidence-derived structural encoder for local higher-order code–visit relations, a Transformer over concatenated code and visit embeddings for global interaction modeling, and a hypergraph reconstruction objective that preserves the original topology during training (Wang et al., 28 Aug 2025).

1. Problem formulation and conceptual scope

SHGT addresses next-visit diagnosis prediction. For each patient, the EHR is modeled as an ordered sequence of inpatient visits, and each visit contains sets of diagnosis codes, medication codes, and procedure codes. Given the visit history up to time $T$, the target is the diagnosis set at time $T+1$, represented as a multi-hot vector $y^{T+1} \in \{0,1\}^{|D|}$. The task is therefore explicitly multi-label, since the output is a set of diagnoses rather than a single class (Wang et al., 28 Aug 2025).

The motivation for SHGT is that medical codes induce higher-order relations. A visit is not naturally a collection of independent pairwise code–code links; it is a set-valued clinical event in which diagnoses, medications, and procedures co-occur jointly. The framework therefore rejects pairwise graph construction as a sufficient representation and instead models each visit as a hyperedge connecting all codes appearing in that visit. The work also argues that ordinary graph neural networks remain limited by localized message passing, so even if a pairwise graph is constructed, relevant dependencies that span distant visits or globally distributed patterns may remain weakly modeled (Wang et al., 28 Aug 2025).

This places SHGT within the broader family of structure-aware hypergraph methods, but with a specific operational meaning of “structure-aware”: local structure is derived from the hypergraph incidence pattern, global dependencies are handled by Transformer self-attention over code and visit tokens, and the learned representation is regularized by reconstructing the original incidence matrix. A frequent misconception is that the name implies a fully specified hypergraph-attention Transformer layer in the formal HGNN sense. The actual design is narrower: the Transformer itself is standard self-attention over concatenated code and visit embeddings, while the hypergraph-specific component is the incidence-based structural encoder and reconstruction loss (Wang et al., 28 Aug 2025).

2. Hypergraph representation of electronic health records

The framework defines the full medical-code vocabulary as

$C= \{c_1, c_2, \ldots, c_m\},$

with diagnosis vocabulary

$D = \{d_1, d_2, \ldots, d_{|D|}\}.$

Patients are denoted

$P = \{p_\mathit{u} \mid u \in U\},$

and a patient $p_u$ has visit history

$p_u = \{v_u^1, v_u^2, \ldots, v_u^T\}.$

Across the dataset, the total number of visits is $n$ (Wang et al., 28 Aug 2025).

The EHR hypergraph is

$G = (V, E, \mathbf{H}),$

where $T+1$0 is the set of medical-code nodes with $T+1$1, $T+1$2 is the set of visit hyperedges with $T+1$3, and $T+1$4 is the incidence matrix. Its entries satisfy:

• $T+1$5 if code $T+1$6 appears in visit $T+1$7,
• $T+1$8 otherwise.

Rows therefore index codes and columns index visits. This representation preserves three structural facts simultaneously: code identity, visit identity, and explicit code–visit membership (Wang et al., 28 Aug 2025).

The initial code embedding matrix is

$T+1$9

and visit embeddings

$y^{T+1} \in \{0,1\}^{|D|}$0

are obtained by mean pooling over the embeddings of codes contained in each visit. The work does not introduce separate patient nodes or patient hyperedges. Patient-level representations are instead derived later by pooling visit embeddings. This suggests that SHGT models patients indirectly through their visit trajectories rather than as first-class entities in the hypergraph (Wang et al., 28 Aug 2025).

3. Structural encoder and Transformer backbone

The computational pipeline has four stages: hypergraph construction from raw EHR, local structural encoding from the incidence matrix, Transformer-based global interaction modeling over codes and visits, and joint optimization with diagnosis prediction plus hypergraph reconstruction (Wang et al., 28 Aug 2025).

The hypergraph structural encoder introduces local structure-aware embeddings for codes and visits. The work gives

$y^{T+1} \in \{0,1\}^{|D|}$1

and

$y^{T+1} \in \{0,1\}^{|D|}$2

as the code-side and visit-side structural embeddings. The typeset formulas are partially corrupted and dimensionally underspecified in the available text, but their intended role is clear: both sides receive incidence-derived structural signals. These structural embeddings are fused with the original semantic embeddings by residual-style addition: $y^{T+1} \in \{0,1\}^{|D|}$3

$y^{T+1} \in \{0,1\}^{|D|}$4

The Transformer input is the concatenation of code and visit representations: $y^{T+1} \in \{0,1\}^{|D|}$5 Consequently, the token set contains both code nodes and visit hyperedges, and the self-attention layer operates jointly over all $y^{T+1} \in \{0,1\}^{|D|}$6 entities. The attention matrix at layer $y^{T+1} \in \{0,1\}^{|D|}$7 is written as

$y^{T+1} \in \{0,1\}^{|D|}$8

followed by the update

$y^{T+1} \in \{0,1\}^{|D|}$9

This gives direct code-to-code, code-to-visit, and visit-to-visit interactions in one layer (Wang et al., 28 Aug 2025).

A second misconception is that SHGT specifies a full hypergraph attention operator with explicit node-degree matrices, hyperedge-degree matrices, or node-to-hyperedge / hyperedge-to-node propagation equations. It does not. Despite the name “hypergraph structural encoder,” the method section does not present a standard degree-normalized hypergraph convolution; the Transformer remains standard self-attention, and structure enters through the incidence-derived embeddings and the auxiliary reconstruction loss. The text also does not provide explicit multi-head equations, layer normalization equations, or feed-forward equations for this model, even though the implementation clearly uses a Transformer backbone (Wang et al., 28 Aug 2025).

4. Reconstruction objective and diagnosis head

After the final Transformer layer, the output $C= \{c_1, c_2, \ldots, c_m\},$0 is split back into final code embeddings $C= \{c_1, c_2, \ldots, c_m\},$1 and final visit embeddings $C= \{c_1, c_2, \ldots, c_m\},$2. These are used to reconstruct the incidence matrix: $C= \{c_1, c_2, \ldots, c_m\},$3 Each entry $C= \{c_1, c_2, \ldots, c_m\},$4 estimates the probability that code $C= \{c_1, c_2, \ldots, c_m\},$5 belongs to visit $C= \{c_1, c_2, \ldots, c_m\},$6. This acts as a topology-preserving regularizer, counterbalancing the tendency of global self-attention to drift away from the original code–visit structure (Wang et al., 28 Aug 2025).

Because the true incidence matrix is highly sparse, reconstruction is trained with negative sampling. Let $C= \{c_1, c_2, \ldots, c_m\},$7 be positive samples with $C= \{c_1, c_2, \ldots, c_m\},$8 and $C= \{c_1, c_2, \ldots, c_m\},$9 sampled negatives with $D = \{d_1, d_2, \ldots, d_{|D|}\}.$0. The structural loss is binary cross-entropy: $D = \{d_1, d_2, \ldots, d_{|D|}\}.$1

For prediction, patient-level embeddings are obtained by mean pooling over visit embeddings. A dense layer with sigmoid activation then produces diagnosis probabilities, and the classification loss is

$D = \{d_1, d_2, \ldots, d_{|D|}\}.$2

The printed upper limit $D = \{d_1, d_2, \ldots, d_{|D|}\}.$3 is a notation inconsistency in the source, since the target space is diagnosis codes; semantically, $D = \{d_1, d_2, \ldots, d_{|D|}\}.$4 is the relevant output dimension. The total objective is

$D = \{d_1, d_2, \ldots, d_{|D|}\}.$5

The reported hyperparameters are: learning rate $D = \{d_1, d_2, \ldots, d_{|D|}\}.$6, dropout $D = \{d_1, d_2, \ldots, d_{|D|}\}.$7, embedding dimension $D = \{d_1, d_2, \ldots, d_{|D|}\}.$8, and $D = \{d_1, d_2, \ldots, d_{|D|}\}.$9 for MIMIC-III and $P = \{p_\mathit{u} \mid u \in U\},$0 for MIMIC-IV. The source does not specify optimizer type, batch size, or number of epochs. This suggests that the paper’s main emphasis is architectural rather than optimization-heavy (Wang et al., 28 Aug 2025).

5. Experimental evidence

Evaluation uses two real-world ICU EHR datasets. MIMIC-III contains 5,442 patients, 14,124 visits, 1,956 diagnoses, 300 medications, and 1,399 procedures. MIMIC-IV contains 10,000 patients, 29,303 visits, 1,134 diagnoses, 281 medications, and 899 procedures. Patients with only one visit are excluded, the data split is 70% training, 10% validation, and 20% test, and each experiment is repeated five times with different random seeds. Metrics are weighted $P = \{p_\mathit{u} \mid u \in U\},$1, Recall@10, and Recall@20 (Wang et al., 28 Aug 2025).

SHGT is compared against three sequence-based methods—RETAIN, Transformer, and StageNet—and four graph-based methods—CGL, Chet, Sherbet, and ADRL. On MIMIC-III, SHGT reports 25.60 weighted $P = \{p_\mathit{u} \mid u \in U\},$2, 27.16 Recall@10, and 37.48 Recall@20, outperforming the strongest baseline ADRL, which reports 23.88, 25.64, and 35.30. On MIMIC-IV, SHGT reports 34.29 weighted $P = \{p_\mathit{u} \mid u \in U\},$3, 36.25 Recall@10, and 46.77 Recall@20, compared with ADRL at 32.02, 34.72, and 44.91. The reported gains over ADRL are therefore +1.72 / +1.52 / +2.18 on MIMIC-III and +2.27 / +1.53 / +1.86 on MIMIC-IV (Wang et al., 28 Aug 2025).

The ablation study evaluates three removed components:

• SHGT-w/o-S: without hypergraph structural encoding,
• SHGT-w/o-T: without Transformer,
• SHGT-w/o-L: without reconstruction loss.

The text gives the qualitative conclusion but not the exact numbers: removing structural encoding significantly degrades performance, removing the Transformer also harms performance, and removing reconstruction loss reduces performance as well. Sensitivity analysis shows that the best number of Transformer layers is $P = \{p_\mathit{u} \mid u \in U\},$4 on MIMIC-III and $P = \{p_\mathit{u} \mid u \in U\},$5 on MIMIC-IV; deeper settings reduce performance, which the source attributes to overfitting. Varying $P = \{p_\mathit{u} \mid u \in U\},$6 shows that moderate structural regularization is beneficial, but excessive reconstruction weight degrades semantic prediction quality. The work does not provide runtime or memory analysis, interpretability results, or case studies (Wang et al., 28 Aug 2025).

6. Position in the literature, limitations, and development paths

SHGT belongs to a wider research line in which hypergraph structure and Transformer computation are combined, but different works operationalize “structure awareness” in substantially different ways. HyperGT processes a joint token set of nodes and hyperedges, adds incidence-based positional encodings

$P = \{p_\mathit{u} \mid u \in U\},$7

and regularizes attention with the star-expansion transition matrix, making it a direct hypergraph Transformer for semi-supervised node classification (Liu et al., 2023). THTN constructs hypergraphs from overlapping graph communities, augments node inputs with

$P = \{p_\mathit{u} \mid u \in U\},$8

and injects topology-derived importance terms into node-to-hyperedge and hyperedge-to-node attention, making topology guidance explicit inside attention itself (Saifuddin et al., 2023). HyperBERT, by contrast, is highly relevant but “not a true hypergraph transformer”: it combines BERT with HGNN structural propagation and additive fusion, so structure is encoded through HGNN convolution rather than hypergraph self-attention (Bazaga et al., 2024). HGFormer shows a parallel development in vision, where semantically sampled hyperedges and node–hyperedge–node topology-aware attention improve structured perception (Wang et al., 3 Apr 2025).

Model	Structural mechanism	Relation to SHGT
HyperGT	Incidence-based PE and structure regularization	Joint node/hyperedge Transformer on hypergraphs
THTN	Structural/spatial encodings plus topology-biased attention	Explicit topology-guided hypergraph attention
HyperBERT	HGNN structural block fused with BERT	Hypergraph-aware, but not a true hypergraph Transformer
HGFormer	Node–hyperedge–node topology-aware hypergraph attention	Vision analogue of topology-aware hypergraph reasoning

Adjacent transformer models can be close architecturally while not being hypergraph models. HypHGT, for example, uses relation-specific hyperbolic attention and a local heterogeneous GNN branch, but it operates on heterogeneous graphs with typed pairwise edges rather than hyperedges, so its relevance to SHGT is methodological rather than direct (Park et al., 13 Jan 2026).

Within this broader landscape, SHGT is best characterized as a structure-aware hypergraph representation plus Transformer reasoning with incidence reconstruction, rather than as a fully formal hypergraph-attention Transformer layer. Its limitations are explicit in the source: the structural encoder is mathematically lightweight, several equations in the method section are corrupted or notation-inconsistent, temporal order is used only indirectly through the patient-history setting, patient entities are not part of the hypergraph itself, scalability is not analyzed, and no interpretability study is presented (Wang et al., 28 Aug 2025). A future direction explicitly stated for the model is to integrate hypergraph neural networks with LLMs to generate more informative code embeddings and further improve diagnostic accuracy. A plausible implication is that subsequent SHGT-like systems may tighten the coupling between structured hypergraph reasoning and richer semantic priors while preserving explicit topology (Wang et al., 28 Aug 2025).