HSA-Net: Hierarchical and Structure-Aware Networks

• HSA-Net is a graph representation approach that leverages hierarchical tree encoding and structure-aware aggregation to preserve multi-hop path correlations.
• It integrates an Integrated Hierarchical Aggregation Module with GRU for sequential neighborhood processing and a Relational Metric Learning Module for type-specific similarity measures.
• Empirical results show T-GNN, an instance of HSA-Net, outperforms traditional GNNs in clustering, classification, and link prediction tasks on heterogeneous data.

Hierarchical and Structure-Aware Network (HSA-Net) denotes, in the present exposition, a class of graph representation models exemplified by T-GNN, “Tree Structure-Aware Graph Representation Learning via Integrated Hierarchical Aggregation and Relational Metric Learning,” for heterogeneous graphs with many-to-one schema and hierarchical tree structure (Qiao et al., 2020). In this formulation, an HSA-Net is characterized by multi-level tree encoding, path-by-path aggregation, and heterogeneity-aware similarity learning. T-GNN instantiates this design through two tightly coupled modules: the Integrated Hierarchical Aggregation Module (IHAM), which preserves hierarchical and sequential neighborhood information on schema-induced trees, and the Relational Metric Learning Module (RMLM), which preserves heterogeneity by embedding each node type into a type-specific space with distinct distribution based on similarity metrics (Qiao et al., 2020).

1. Problem formulation and motivating limitations

The motivating setting is heterogeneous graph representation learning. The underlying problem is that standard GNNs aggregate neighbors’ information regardless of node types, which is effective on homogeneous graphs but problematic on heterogeneous graphs (Qiao et al., 2020). Prior work is described as exploiting relations or meta-path to sample neighbors with distinct categories and then using attention mechanism to learn different importance for different categories. The exposition identifies two limitations in that line of work.

First, the learned representations for different types of nodes should own different feature spaces, whereas the approaches under critique still project node representations into one feature space (Qiao et al., 2020). Second, after exploring massive heterogeneous graphs, the work identifies that multiple nodes with the same type always connect to a node with another type, which reveals the many-to-one schema, a.k.a. the hierarchical tree structure. Under ordinary aggregation, the exact multi-hop path correlation from neighbors to the target node is erased through aggregation (Qiao et al., 2020).

This framing is central to the HSA-Net concept. An HSA-Net is not merely relation-aware in the sense of distinguishing edge or node categories; it is also structure-aware in the stronger sense of preserving directed multi-hop path organization induced by hierarchical schemas. A common misconception is that relation-aware neighbor sampling or meta-path attention alone preserves higher-order structure. The T-GNN formulation argues that it does not, because indiscriminate aggregation collapses path-specific correlations even when the sampled neighborhoods are type-aware (Qiao et al., 2020).

2. Architectural composition of T-GNN as an HSA-Net

T-GNN is organized around two modules: IHAM and RMLM (Qiao et al., 2020). IHAM decomposes each node’s multi-hop neighborhood into a set of directed trees according to a predefined collection of hierarchical tree schemas, described as directed meta-paths ending at the target node type. On each tree, it performs a relation-aware GNN aggregation at each level combined with a Gated Recurrent Unit (GRU), thereby preserving sequential, path-by-path information flow from leaves up to the root. For each schema $s$, the model yields a schema-specific embedding $\mathbf{z}_o^{s}$ for node $o$, and then integrates all schema-specific embeddings together with the node’s own transformed feature through a self-attention mechanism into a final embedding $\mathbf{u}_o$ (Qiao et al., 2020).

RMLM assigns each node type $t$ its own embedding space $\mathbb{R}^{d'}_t$ and mapping $f_t\colon N_t\to\mathbb{R}^{d'}_t$ (Qiao et al., 2020). Similarity is then computed in a relation-aware fashion. If two nodes have the same type, the model uses a dot product in the same space; if the node types differ, it uses a relation-type-specific metric, for example a bilinear or additive MLP form. The two modules interact end-to-end: IHAM produces node embeddings $\{\mathbf{u}_o\}$, and RMLM defines a loss that pulls true neighbors closer under the relation-specific metric and pushes non-neighbors apart (Qiao et al., 2020).

These design choices define the HSA-Net principles explicitly stated in the exposition. The model is hierarchical because it operates on multi-level trees; structure-aware because it encodes path-by-path correlations rather than collapsing all multi-hop evidence into a single neighborhood pool; and heterogeneity-aware because it uses type-specific projections and relation-specific metrics (Qiao et al., 2020).

3. Integrated hierarchical aggregation

The hierarchical component begins with a set of schemas for a node of type $t_m$: $$\mathcal{S}_t=\{s_1,\dots,s_K\}, \quad s = t_0 \xrightarrow{r_1} t_1 \xrightarrow{r_2} \cdots \xrightarrow{r_m} t_m.$$ For each schema $\mathbf{z}_o^{s}$0, the model extracts a directed tree whose leaves are at level $\mathbf{z}_o^{s}$1 and root is at level $\mathbf{z}_o^{s}$2 (Qiao et al., 2020).

Let $\mathbf{z}_o^{s}$3 denote the hidden state after processing level $\mathbf{z}_o^{s}$4 for node $\mathbf{z}_o^{s}$5 of type $\mathbf{z}_o^{s}$6, and let $\mathbf{z}_o^{s}$7 be its raw feature. Initialization at the leaf level is

$\mathbf{z}_o^{s}$8

For $\mathbf{z}_o^{s}$9, the recurrence is

$o$0

$o$1

A common choice for the aggregator is the weighted mean

$o$2

where $o$3 is a trainable matrix for relation $o$4 and $o$5 is a normalization or attention weight. After level $o$6, the root’s schema-specific output is

$o$7

This recurrence is the mechanism by which the model preserves the tree structure by combining GNN with GRU to integrate hierarchical and sequential neighborhood information on the tree structure into node representations (Qiao et al., 2020).

The significance of this design lies in the explicit preservation of ordering along the path. The exposition states that the level-by-level GRU+GNN aggregation never “forgets” the ordering along the path, unlike simple layer stacking. This suggests that the HSA-Net formulation treats depth not merely as repeated message passing but as an ordered latent process defined by schema-constrained tree traversal (Qiao et al., 2020).

4. Schema integration and type-aware embedding spaces

After obtaining schema-specific outputs, T-GNN introduces a schema-level attention mechanism. Let $o$8 denote the “null” schema, with output

$o$9

Given $\mathbf{u}_o$0, the attention weights are

$\mathbf{u}_o$1

where $\mathbf{u}_o$2 is a parameter vector and $\mathbf{u}_o$3 denotes concatenation. The final embedding is

$\mathbf{u}_o$4

This produces an attention-weighted mixture of path-specific representations rather than a single undifferentiated neighborhood summary (Qiao et al., 2020).

The heterogeneity-preserving component is equally explicit. Each node type $\mathbf{u}_o$5 has its own projection $\mathbf{u}_o$6 for raw features and its own embedding space $\mathbf{u}_o$7, while cross-type similarities are measured in relation-specific metric spaces (Qiao et al., 2020). The exposition states that this eliminates distributional conflicts that arise when all types share one “global” space. A plausible implication is that HSA-Net, in this instantiation, treats type heterogeneity as a geometric issue as much as a sampling or attention issue: the representational manifold itself is allowed to vary by node type.

The schema-level attention and the type-specific spaces jointly define the model’s notion of structure awareness. Schemas preserve exact multi-hop path correlations by separate processing; attention controls how those path-specific embeddings are integrated; and type-specific projections prevent heterogeneous semantics from being collapsed into a uniform latent geometry (Qiao et al., 2020).

5. Relational metric learning and optimization objective

The relational similarity function depends on whether the compared nodes share a type. If $\mathbf{u}_o$8, then

$\mathbf{u}_o$9

If $t$0 are linked by a relation of type $t$1, with $t$2 and $t$3, then the exposition provides two alternatives (Qiao et al., 2020). The bilinear metric is

$t$4

The perceptron, or additive, metric is

$t$5

where $t$6 and $t$7.

Training uses a skip-gram-style graph context loss with negative sampling (Qiao et al., 2020). For node $t$8 and context node $t$9 under relation $\mathbb{R}^{d'}_t$0, the model maximizes

$\mathbb{R}^{d'}_t$1

where $\mathbb{R}^{d'}_t$2 is the set of all nodes of type $\mathbb{R}^{d'}_t$3. This is approximated via negative sampling $\mathbb{R}^{d'}_t$4: $\mathbb{R}^{d'}_t$5 Aggregated over random-walk contexts $\mathbb{R}^{d'}_t$6 and window $\mathbb{R}^{d'}_t$7, with $\mathbb{R}^{d'}_t$8 regularization, the loss is

$\mathbb{R}^{d'}_t$9

This optimization objective is integral to the HSA-Net formulation. IHAM supplies embeddings that preserve tree-structured multi-hop context; RMLM determines how proximity should be measured across homogeneous and heterogeneous node pairs. The resulting training criterion is therefore not only contrastive but relation-aware at the metric level (Qiao et al., 2020).

6. Training procedure, empirical behavior, and limitations

The high-level training procedure begins by initializing all parameters $f_t\colon N_t\to\mathbb{R}^{d'}_t$0, precomputing random-walk contexts $f_t\colon N_t\to\mathbb{R}^{d'}_t$1, and then iterating mini-batch optimization until convergence (Qiao et al., 2020). In each forward pass, for each unique node in the batch and for each schema in $f_t\colon N_t\to\mathbb{R}^{d'}_t$2, the model extracts the corresponding level-$f_t\colon N_t\to\mathbb{R}^{d'}_t$3 to level-$f_t\colon N_t\to\mathbb{R}^{d'}_t$4 tree, runs the recurrence, obtains $f_t\colon N_t\to\mathbb{R}^{d'}_t$5, adds $f_t\colon N_t\to\mathbb{R}^{d'}_t$6, computes attention weights $f_t\colon N_t\to\mathbb{R}^{d'}_t$7, and forms

$f_t\colon N_t\to\mathbb{R}^{d'}_t$8

It then computes similarities for positive and negative pairs, forms the loss, backpropagates, and updates $f_t\colon N_t\to\mathbb{R}^{d'}_t$9 via Adam (Qiao et al., 2020). Inference for a new node $\{\mathbf{u}_o\}$0 follows the same IHAM pipeline by extracting its trees for each schema and re-running the forward pass.

Empirically, the exposition reports that across node clustering, node classification, inductive representation tasks and link prediction, T-GNN outperforms strong homogeneous GNNs (GCN, GraphSAGE, GAT) and heterogeneous baselines (Metapath2Vec, HAN, HetGNN, RHINE), with gains of 2–10% in NMI/ARI and F1 (Qiao et al., 2020). Inductive tests with up to 20% of nodes held out at training time confirm that the GRU-based hierarchical aggregator generalizes to unseen nodes by re-extracting their schemas and re-running the forward pass. Ablations on the relational metric—dot product, bilinear, and perceptron—show that the additive MLP (perceptron) metric often yields the best performance, presumably due to greater expressiveness in aligning cross-type embeddings (Qiao et al., 2020).

The same exposition identifies several limitations. Enumerating hierarchical schemas $\{\mathbf{u}_o\}$1 may require domain expertise, although in practice one can restrict to 2–3 hops. Computation and memory cost grow with the number of schemas and tree depth, which implies the need for careful sampling and batching strategies at scale. In addition, the negative-sampling loss is unsupervised; semi-supervised and fully supervised tasks require adding classification or regression heads (Qiao et al., 2020).

Taken together, these observations define the substantive content of HSA-Net in this setting. Its distinctive claim is not simply that heterogeneity should be modeled, but that heterogeneity, many-to-one schema, and exact multi-hop path correlations must be handled jointly. In the T-GNN formulation, that joint handling is achieved by coupling schema-wise GRU+GNN aggregation with relation-aware metric learning in a single end-to-end graph representation framework (Qiao et al., 2020).