Meta-path-aware Masking in Heterogeneous Graphs

• The paper introduces meta-path-aware masking which strategically obfuscates node attributes and connectivity along meta-paths to boost self-supervised representation learning.
• It employs tailored masking strategies—random, degree-based, and attention-based—that ensure both endpoints and intermediate nodes contribute to rich semantic encoding.
• Empirical results demonstrate that joint reconstruction with intra-meta-path augmentation significantly improves performance metrics such as Micro-F1, Macro-F1, and AUC across datasets.

Meta-path-aware masking refers to the strategic obfuscation of both node attributes and connectivity patterns along specified meta-paths within heterogeneous graphs. The aim is to enhance the self-supervised representation learning signal, especially by ensuring that internal nodes along meta-paths—rather than only endpoints—contribute to the structural and semantic information leveraged during training. This methodology is exemplified in the IMPA-HGAE (Intra-Meta-Path Augmented Heterogeneous Graph Autoencoder) framework, where meta-path-induced masking enables richer semantic encoding and more robust generative self-supervised learning on complex, multi-typed graph structures (Lin et al., 7 Jun 2025).

1. Meta-Path-Aware Masking Schemes

IMPA-HGAE implements both edge-level and node-level masking strategies, operating explicitly along meta-paths $$\phi = (v_1 \rightarrow v_2 \rightarrow ... \rightarrow v_{l+1})$$ within the heterogeneous graph.

Edge-masking schemes (applied to meta-path adjacency matrices $A^\phi$):

• Random Edge-Masking: Uniformly samples and removes a fixed fraction (mask_rate) of edges in $A^\phi$.
• Degree-Based Edge-Masking: Prioritizes edges for removal according to a degree weight $$DW[e] = \mathrm{Scale}\left(\frac{\deg_{\text{out}}(u) + \deg_{\text{in}}(v)}{2}\right)$$ for edge $e=(u \rightarrow v)$. Edges are sampled without replacement in proportion to their degree weights.
• Attention-Score-Based Edge-Masking: Utilizes node-level GAT attention scores $p_i$ collected during message passing. Pre-softmax logit scores $z_i = \ln(p_i) + C$ are used to set sampling probabilities for edge masking.

Attribute-masking schemes (applied to node features $X$):

• Full-Node Masking: Uniformly selects a subset of nodes and zeroes out their entire feature vectors.
• Partial-Attribute Masking: On the remaining nodes, randomly zeros out a fraction of feature dimensions.

These masking operations occur for every meta-path and affect not only endpoints but also all internal nodes and edges, ensuring full exploitation of heterogeneous node information along each path.

2. Formal Masking Definitions

The principal masking formulations are as follows:

• Random Edge-Masking:

$$S \sim \mathrm{Uniform}\left\{ \text{subsets of }E\text{ of size }k \right\},\qquad k = \text{mask\_rate}\cdot|E|$$

• Degree-Based Edge-Masking:

$$DW[e] = \mathrm{Scale}\left(\tfrac{\deg_{\mathrm{out}}(u)+\deg_{\mathrm{in}}(v)}{2}\right)$$

Sampling without replacement is achieved through conditional multinomials based on $DW$.

• Attention-Score-Based Edge-Masking:

$z_i = \ln(p_i) + C$

as above, sampling via exponentiated logits.

• Node/Attribute Masking:

For node $v$, mask vector $M_\text{node}(v)\in\{0,1\}$ with $$\Pr[M_\text{node}(v)=0]=\text{mask\_rate}_\text{node}$$; for features $f$, $M_\text{attr}(v,f)\in\{0,1\}$ with $$\Pr[M_\text{attr}(v,f)=0]=\text{mask\_rate}_\text{attr}$$. The masked feature is:

$$\widetilde X_v = M_\text{node}(v) \left(M_\text{attr}(v)\odot X_v\right)$$

3. Generative Self-Supervised Objectives

IMPA-HGAE integrates masking into a generative SSL paradigm with dual objectives:

• Feature Reconstruction Loss:

$$\mathcal{L}_{\mathrm{feat}} = \frac{1}{|\mathcal{V}|} \sum_{v \in \mathcal{V}} \left(1 - \cos\left(X_v,\,H_{D,v}\right)\right)^{\gamma_1}$$

• Meta-Path Adjacency Reconstruction Loss:

For meta-path $\phi$,

$$\mathcal{L}^{\phi} = \frac{1}{|A^\phi|} \sum_{v \in \mathcal{V}} \left(1 - \cos\left(A_v^\phi,~A_v^{\phi'}\right)\right)^{\gamma_2}$$

with $A^{\phi'}$ the reconstructed meta-path adjacency and $A^{\phi}$ the original. Meta-path attention weights from HAN ($\alpha^\phi$) combine losses:

$$\mathcal{L}_{\mathrm{mp}} = \sum_{\phi \in \Phi} \alpha^\phi\,\mathcal{L}^{\phi}$$

• Total Loss:

$$\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{feat}} + \lambda\,\mathcal{L}_{\mathrm{mp}} + \text{(optional regularization)}$$

This joint loss structure forces the autoencoder to model both node attribute integrity and higher-order meta-path-based connectivity.

4. IMPA-HGAE Architecture and Training Workflow

IMPA-HGAE is composed of four core modules:

1. Preprocessing/Masking: Inputs are node features $X$ and meta-path adjacencies $\{A^\phi\}$. Node/attribute and meta-path edge masks are applied, yielding $\widetilde X$ and $\{\widetilde A^\phi\}$.
2. Encoder $f_E$: Adopts a HAN-style multi-relation GNN, ingesting the masked data to produce initial $H_\phi^\text{tar}$ for the target type and raw $X^\text{other}$ for other types.
3. Intra-Meta-Path Augmentation: Non-target features are projected to the shared space. For each meta-path, $K$ GATConv layers propagate signals along paths, enabling embeddings to absorb semantics from all nodes in $\phi$.
4. Decoder $f_D$ & Reconstruction: Accepts enhanced embeddings, reconstructing both features ($H_D^{\text{tar}}$) and meta-path adjacencies ($A^{\phi'}$) for the dual losses above.

Training alternates between masking, encoding, intra-meta-path propagation, decoding, loss calculation, and parameter updates. Masking is not used during inference (Lin et al., 7 Jun 2025).

5. Semantic Effects and Interpretability

Meta-path-aware masking has crucial implications for semantic alignment and interpretability:

• Forcing Semantic Discovery: Masking internal edges and features compels the encoder/propagation to recover the semantics of every intermediate node type rather than simply encoding start-end node pairs.
• Degree-Based Masking: Preferentially removes edges from highly connected regions, resulting in more uniform and fragmented graphs. Empirically, this increases the number of connected components and strengthens the self-supervision signal.
• Attention-Based Masking: By perturbing learned node-node attentions, this scheme highlights which meta-path hops are pivotal for downstream reconstruction.
• Partial-Attribute Masking: Induces intra-type noise, pushing the model toward leveraging structural (topological) signals over single-feature reliance.
• Empirical Visualization: T-SNE plots show decoder outputs more closely clustered around true meta-path structures when proper masking is applied, compared to encoder-only embeddings.

6. Empirical Performance and Comparative Analysis

Experimental evaluation on DBLP, Freebase, ACM, and AMiner demonstrates:

• IMPA-HGAE achieves top scores on Micro-F1, Macro-F1, and AUC across nearly all splits, outperforming HetGNN, HAN, DMGI, HeCo, GraphMAE, HGMAE, and DiffGraph.
• Degree-based edge-masking exhibits the most robust and consistent gains versus random and attention-based masking.
• Joint reconstruction (features and meta-paths) with subgraph enhancement outperforms single-task variants.
• Ablation studies confirm that:
  • Masking provides an improvement of approximately 1–2 percentage points over no masking;
  • Dual reconstruction confers an additional 0.5–1 point;
  • Intra-meta-path propagation gives a further 0.5–1 point advantage.
• Mask-rate parameter sweeps reveal a U-shaped relationship with performance—degree-based masking is most resilient to parameter variation.

The following table summarizes masking schemes and their notable empirical effects:

Scheme	Mask Target	Empirical Outcome
Degree-based masking	Edges (A^φ)	Most robust performance, increased uniformity
Attention-based masking	Edges (A^φ)	Reveals crucial meta-path hops
Full-node masking	Node features (X)	Forces structural reliance

A plausible implication is that optimally parametrized, degree-guided masking directly addresses over-reliance on dense substructures, leading to better generalization and semantic disentanglement.

7. Implications and Future Directions

Meta-path-aware masking, as instantiated in IMPA-HGAE, demonstrates how architectural innovations in generative SSL for heterogeneous graphs can be paired with principled masking for enhanced representation learning. By leveraging the full internal structure of meta-paths, these strategies deliver not only improved empirical performance but also greater control over the alignment of learned embeddings with human-interpretable semantic structure. Further exploration may yield additional interpretability tools and robustness benchmarks for masking-based generative models (Lin et al., 7 Jun 2025).