HyperGCL: Hypergraph Contrastive Learning

• HyperGCL is a framework for hypergraph contrastive learning that integrates both fabricated and generative augmentations to improve robustness and generalizability.
• It constructs multi-modal views—attribute, local, and global—to harness complementary information and optimize contrastive objectives across hypergraph representations.
• Empirical evaluations demonstrate that HyperGCL achieves state-of-the-art performance in node classification, fairness tasks, and resistance to structure-poisoning attacks.

HyperGCL is a class of methods for hypergraph contrastive learning that leverage hypergraph structure and data-driven augmentations to enhance the generalizability, robustness, and fairness of hypergraph neural networks in supervised and self-supervised learning settings (Wei et al., 2022, Saifuddin et al., 18 Feb 2025). The approach merges hypergraph-based data modeling with contrastive objectives, employing both handcrafted (fabricated) and learnable (generative or adaptive) augmentations. Two principal frameworks—one centered on augmentations for low-label regimes (Wei et al., 2022), and one focused on multi-modal view construction and adaptive topology (Saifuddin et al., 18 Feb 2025)—constitute the state of the art under the HyperGCL paradigm.

1. HyperGCL Framework and Motivations

HyperGCL provides a solution to overfitting and low generalizability in hypergraph neural networks (HyperGNNs) when label supervision is limited. Given a hypergraph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$—where $\mathcal{V}$ is a set of vertices and $\mathcal{E}$ denotes hyperedges—HyperGCL aims to learn node (and optionally hyperedge) representations using contrastive self-supervision. This process involves constructing multiple augmented "views" via perturbations or generative modeling, encoding each view with HyperGNNs, projecting the resultant embeddings, and optimizing a contrastive loss (e.g., InfoNCE). The contrastive objective pulls representations of the same node or hyperedge across views together and pushes all others apart (Wei et al., 2022, Saifuddin et al., 18 Feb 2025).

Traditional graph contrastive methods employ handcrafted augmentations (node/edge dropping, feature masking) and typically treat edges and attributes as a single modality, potentially destroying task-relevant information. HyperGCL addresses these limitations by: (i) leveraging hypergraphs to inherently model higher-order relationships; (ii) constructing multi-modal views that better exploit attribute and structure complementarities; and (iii) enabling learnable augmentations that respect underlying data distribution (Saifuddin et al., 18 Feb 2025).

2. Hypergraph View Construction

HyperGCL systems construct distinct hypergraph views from input graph data, each encoding different information granularity:

• Attribute-driven view $\mathcal{H}^a$: Derived from node features. For each node $v_i$, form a hyperedge $\bar e_i$ by selecting $v_i$ and its $k$ nearest neighbors in feature space. Optionally, $k$-means clustering is used, and each cluster forms a hyperedge. The resultant incidence matrix $H^{(a)}$ encodes both local (similarity) and cluster-level feature relations.
• Local structure-driven view $\mathcal{H}^l$: Based on the original graph's adjacency. Each 1-hop ego-network (node and its immediate neighbors) is treated as a hyperedge, yielding $H^{(l)}$.
• Global structure-driven view $\mathcal{H}^g$: Communities in the graph, detected via overlapping community detection algorithms (e.g., Chen et al. 2010), form hyperedges. A small set of high-centrality nodes can be added for global connectivity. The corresponding incidence matrix $H^{(g)}$ encodes long-range structural dependencies (Saifuddin et al., 18 Feb 2025).

This multi-view construction allows HyperGCL to disentangle and leverage different modalities and levels of graph semantics.

3. Augmentation Schemes: Fabricated and Generative

HyperGCL encompasses two classes of augmentations for producing effective contrastive views:

3.1. Fabricated Augmentations

Five handcrafted augmentations are defined (Wei et al., 2022):

Augmentation	Operation Type	Description
A1	Hyperedge dropping	Each hyperedge $e$ kept with rate $1-r_e$
A2	Bipartite-edge perturb	Each incidence $(v,e)$ dropped with $r_{ve}$
A3	Vertex dropping	Each node $v$ removed with probability $r_v$
A4	Attribute masking	Feature dims of $x_v$ masked with $r_m$
A5	Subgraph sampling	Uniformly sample a fraction $s$ of nodes

Empirically, perturbing the hypergraph at the hyperedge- or incidence-level (A2), specifically via bipartite-edge perturbation, yields the largest performance gains, highlighting the importance of higher-order relations over vertex-level or attribute perturbations. In low-label regimes (10% labeled), A2 reaches up to +9% improvement over standard SetGNN on multiple benchmarks (Wei et al., 2022).

3.2. Generative (Learnable) Augmentations

HyperGCL introduces a generative augmentation view (A6) that is learned in a data-driven fashion:

• Variational Hypergraph Autoencoder (VHGAE): Encodes the input hypergraph with two HyperGNNs, parameterizes variational distributions for vertices and hyperedges, and reconstructs the incidence matrix via Bernoulli likelihoods of $$p(\tilde H_{v,e}=1|z_v,z_e)=\text{Sigmoid}(z_v^\top z_e)$$. Training optimizes a variational ELBO (reconstruction + KL divergences).
• Differentiable Hyperedge Sampling: To generate augmented views that are "hard" samples but allow for end-to-end gradients, Gumbel-Softmax trick is applied on the decoder outputs. Each incidence $(v,e)$ is sampled with:

$$T_{v,e} = \mathrm{Sigmoid}\Bigg(\frac{w_{v,e} + \log\delta - \log(1-\delta)}{\tau}\Bigg),\ \delta\sim U[0,1],\ w_{v,e}=z_v^\top z_e$$

With $\tau\to0$, $T_{v,e}$ approximates a binary gate.

• Joint Optimization: The training combines the VHGAE loss and InfoNCE contrastive losses (for both generative and fixed fabricated views) in a unified multitask objective (Wei et al., 2022).

In the multi-modal HyperGCL framework (Saifuddin et al., 18 Feb 2025), adaptive topology augmentation is implemented per-view via Gumbel-Softmax masked incidences, where learnable logits $\Phi$ are updated end-to-end to optimally filter out noise while preserving task-relevant associations.

4. Encoders and Contrastive Objectives

View-specific encoders are deployed for processing each hypergraph view:

• HyGAN: For attribute-driven views, utilizes two-level attention (node-to-hyperedge and hyperedge-to-node) with learnable coefficients, incorporating element-wise and cross-level attention for expressivity.
• Structure-aware HyGAN (SHyGAN): For structure-driven views, node features are enriched with local connectivity embeddings, centrality measures, and distinctiveness scores. Attention coefficients are further biased using structural features such as clustering coefficient (local) and hyperedge density (global).

Contrastive objectives are computed between each pair of views using an InfoNCE loss. Positive pairs are defined by topological proximity (shared hyperedge or neighbor) and feature similarity, negatives are adaptively sampled by graph distance or low cosine similarity. The loss is:

$$\mathcal{L}_{a-l} = -\frac{1}{n}\sum_{v_i\in V} \log \frac{\sum_{v_j\in\mathrm{PosS}_{v_i}}\exp(\mathrm{sim}(z_i^a, z_j^l)/\eta)} {\sum_{v_j\in\mathrm{PosS}_{v_i}\cup\mathrm{NegS}_{v_i}}\exp(\mathrm{sim}(z_i^a, z_j^l)/\eta)}$$

The final loss is the sum of all cross-view contrastive losses plus supervised classification terms on labeled nodes (Saifuddin et al., 18 Feb 2025).

5. Empirical Results and Insights

On node classification benchmarks (Cora, Citeseer, Pubmed, Wiki, Twitch-PT, LastFMAsia), HyperGCL achieves state-of-the-art accuracy, e.g., 85.88% on Cora (distance-based negatives) versus 83.80% for GraphMAE and 72.22% for best hypergraph baseline DHGNN. Removing any of the three views noticeably degrades accuracy (global view omission yields the largest drops), and ablations confirm the necessity of adaptive augmentation and topology-aware contrastive sampling (Saifuddin et al., 18 Feb 2025).

For fabricated augmentations, bipartite-edge perturbation (A2) consistently outperforms others except for the generative approach. Generative augmentation (A6) further pushes average rank to 1.0 on 13 tested datasets and achieves relative error-rate reductions of 15–20% (Wei et al., 2022).

HyperGCL shows increased robustness under structure-poisoning attacks. On multiple datasets and attacks (Random, NetAttack, MinMax), HyperGCL-A6 achieves 2–3% higher accuracy than unaugmented SetGNN. In fairness-sensitive tasks (German Credit, Recidivism, Credit Defaulter), generative HyperGCL reduces both statistical parity and equalized odds gaps while maintaining AUROC and F1 (Wei et al., 2022).

6. Implementation and Practical Considerations

Key implementation details include:

• For fabricated augmentations, recommended keep-rates are $r_e \approx 0.2$–$0.4$ for A1/A2, $r_v\approx0.2$ for A3, $r_m\approx0.3$ for A4, and $s\approx0.8$ for A5.
• Temperature $\tau = 0.5$ is used in contrastive loss.
• Joint training uses Adam optimizer (learning rate $10^{-3}$, weight decay $10^{-5}$) for 200 epochs with early stopping (Wei et al., 2022).
• All modules (data splits, augmentations, SetGNN, InfoNCE, Gumbel-Softmax sampling) are released in the HyperGCL codebase (https://github.com/weitianxin/HyperGCL).

The multi-modal HyperGCL pipelines are trained by alternately or jointly optimizing view-specific encoders and mask logits, following the algorithmic routine outlined above (Saifuddin et al., 18 Feb 2025).

7. Theoretical and Practical Implications

Empirical findings and ablations indicate that (1) augmentations respecting higher-order structure in hypergraphs yield greater boost in few-shot settings; (2) learnable or generative augmentation strategies outperform static ones, as they better preserve complex topology and semantic relations; (3) topology-aware contrastive sampling avoids the pitfalls of vision-inspired negative selection by incorporating graph connectivity; (4) robust and fair representations can be obtained through contrastive mechanisms optimizing over both structure and attributes (Wei et al., 2022, Saifuddin et al., 18 Feb 2025).

A plausible implication is that hypergraph-based contrastive learning, when coupled with adaptive view generation, provides a general paradigm for robust, scalable, and fair representation learning beyond purely dyadic graphs. The formalization of positives/negatives and the balancing of local/global hypergraph semantics are critical to this success.