TriCL Hypergraph Representation Learning

• The paper introduces a tri-directional contrastive framework using node-to-node, group-to-group, and group-to-membership contrasts to capture diverse hypergraph structures.
• TriCL generates high-quality node and hyperedge embeddings, enabling state-of-the-art performance in tasks such as node classification and clustering.
• Its minimal augmentation strategy and lightweight negative sampling design ensure robust, scalable performance across various datasets.

Hypergraph representation learning generalizes graph learning to domains where relationships among entities are naturally modeled as sets of arbitrary cardinality (hyperedges) rather than binary pairs. The TriCL framework introduces a principled, unsupervised approach to hypergraph representation learning specifically tailored to capture both microscopic (node-level) and mesoscopic (group-level) structures. TriCL leverages tri-directional contrastive learning—node-to-node, group-to-group, and group-to-membership—to generate high-quality node and hyperedge embeddings suitable for downstream tasks such as node classification and clustering (Lee et al., 2022).

1. Formal Problem Definition

Given a hypergraph $G = (V, E)$ with vertex set $V = \{v_1, ..., v_{|V|}\}$ and hyperedge set $E = \{e_1, ..., e_{|E|}\}$, where each $e_j \subseteq V$, each node $v_i$ is associated with an $F$-dimensional feature vector $x_i$, aggregated in a matrix $X \in \mathbb{R}^{|V| \times F}$. The incidence matrix $H \in \{0,1\}^{|V| \times |E|}$ encodes memberships, i.e., $H[i,j]=1$ iff $v_i \in e_j$.

The objective is to learn an encoder $f_\theta:(X,H) \mapsto (P,Q)$ that produces node embeddings $P \in \mathbb{R}^{|V| \times F'}$ and hyperedge embeddings $Q \in \mathbb{R}^{|E| \times F''}$ in an unsupervised (contrastive) setting. These embeddings must encode both local and higher-order hypergraph structure, supporting tasks such as node classification and clustering (Lee et al., 2022).

2. Tri-directional Contrastive Learning Objectives

TriCL implements three coupled contrastive objectives using a two-view augmentation framework:

a) Node–Node Contrast:

For each node $i$, embeddings from two augmentations ($z_{1,i}$, $z_{2,i}$) are forced to match via InfoNCE loss. Negatives are all other nodes in the opposing view.

b) Group–Group Contrast:

For each hyperedge $j$, the objective enforces alignment between $y_{1,j}$, $y_{2,j}$, with negatives from other hyperedges.

c) Group–Member Contrast:

For each true membership pair $(i, j)$ where $H[i, j]=1$, the loss aligns node-hyperedge embedding pairs across augmented views, with negatives sampled from non-membership pairs.

The aggregate loss is: $L = L_n + \lambda_g L_g + \lambda_m L_m$ where $L_n, L_g, L_m$ are node-node, group-group, and group-member contrastive losses respectively, and $\lambda_g, \lambda_m$ are scalar weights (Lee et al., 2022).

3. Data Augmentation and Negative Sampling

TriCL employs minimal augmentations proven effective for hypergraphs:

• Node-feature masking: Each feature dimension is independently masked (Bernoulli, probability $p_f$) in each view.
• Membership masking: A random subset of membership incidences is masked from $H$ in each view (Bernoulli, probability $p_m$).

Negative sampling strategies in TriCL are lightweight—only one negative per positive example in membership contrast, and random (often subsampled) negatives for node-node and group-group InfoNCE. Subsampling as few as $k=2$ negatives per anchor incurs negligible loss in accuracy (<1%) (Lee et al., 2022).

4. Algorithmic Workflow and Implementation

TriCL’s training loop for a single epoch comprises:

1. Generation of two random augmented hypergraph views $(X_1,H_1)$, $(X_2,H_2)$.
2. Encoding via a shared hypergraph encoder $f_\theta$, resulting in node and hyperedge representations.
3. Projection into normalized contrastive space using MLP heads.
4. Computation of all three contrastive losses ($L_n, L_g, L_m$) using the structured negative sampling described.
5. Joint backpropagation and parameter update with optimizer (e.g., AdamW).

Essential hyperparameters include InfoNCE temperatures ($\tau_n,\tau_g,\tau_m$), augmentation rates ($p_f, p_m$), loss weights ($\lambda_g, \lambda_m$), and embedding dimensions. The procedure is scalable to large hypergraphs and does not require heavy augmentations (Lee et al., 2022).

5. Experimental Evaluation

TriCL was benchmarked on 10 datasets spanning citation/co-authorship graphs (Cora-C, Citeseer, Pubmed, DBLP), vision/3D (NTU2012, ModelNet40), and UCI categorical data, with vertex counts ranging from 101 to 41,000 and hyperedges from 43 to 22,000. Features range from 16 to 3,700 dimensions.

Baselines include supervised hypergraph models (HGNN, HyperConv, HNHN, HyperGCN, HyperSAGE, UniGCN, AllSetTransformer), classical graph learning on clique expansions (GCN*, GAT*, Node2vec*, DGI*, GRACE*), and recent unsupervised hypergraph approaches.

In node classification (linear evaluation), TriCL achieves best or tied-best performance on 9/10 datasets, ranking highest in average across all tested settings. In clustering (k-means NMI, pairwise F1), TriCL also ranks first by average rank, indicating robust embedding quality across application domains (Lee et al., 2022).

Method	Node Classification Accuracy (Cora-C)	Node Classification Accuracy (Citeseer)	Avg-rank
Supervised	77.5	67.8	6.1
Unsupervised	79.1	68.8	6.8
TriCL-NG	81.4	71.4	2.0
TriCL	81.6	72.0	1.5

6. Ablation Studies and Insights

TriCL’s ablations systematically demonstrate:

• Using all three contrastive directions is strictly superior; ablations confirm joint node–node, group–group, and membership-level contrasts yield the strongest separation and representation quality.
• Variants with only membership-level contrast exhibit embedding collapse (degenerate representations).
• Sampling just a few negatives is sufficient; large negative sets do not yield incremental improvements.
• Feature masking plus membership masking forms the most effective augmentation regime.
• Adding self-loop hyperedges for each node yields up to 2 point gains in classification.
• TriCL is robust to moderate choices of temperature and loss weights (visualized via heatmaps).
• t-SNE visualizations reveal that TriCL embeddings exhibit the most pronounced cluster separation, as measured by Silhouette scores (Lee et al., 2022).

7. Context and Implications

Contrastive learning in hypergraph domains has lagged relative to its graph and vision analogs. TriCL addresses the challenge by leveraging the full incidence structure and three levels of structural granularity. Its simplicity in augmentations and negative sampling, empirical robustness, and state-of-the-art performance on node-level and group-level tasks provide a reference blueprint for unsupervised hypergraph representation learning.

By maximizing tri-directional agreement under random augmentations, TriCL captures both fine-grained and set-level dependencies not accessible to classical two-way contrastive or expansion-based methods. The methodological design, especially the group–member contrast component, is critical for avoiding collapse and maintaining high-quality embeddings (Lee et al., 2022).

A plausible implication is that the tri-directional contrast framework may serve as a general template for unsupervised learning in data modalities admitting set-structured relations and variable granularity.

---

References:

TriCL framework and results: "I'm Me, We're Us, and I'm Us: Tri-directional Contrastive Learning on Hypergraphs" (Lee et al., 2022).