---
title: Graph Invariant and Variant Embedding (GIVE)
url: https://www.emergentmind.com/topics/graph-invariant-and-variant-embedding-give
type: topic
---

# Graph Invariant and Variant Embedding (GIVE)

Graph Invariant and Variant Embedding (GIVE) is a framework for disentangling and jointly modeling graph substructures that are invariants—statistically or causally stable under given distributional shifts—and those that are variant—domain-specific, spurious, or transient. GIVE is central to recent advances in out-of-distribution (OOD) generalization, expressive graph learning, and interpretable representation schemes for graph data.

## 1. Formal Foundations: Graph Invariant and Variant Functions

Let $G = (V, E, A, X)$ denote a graph with nodes $V$, edges $E$ (adjacency matrix $A$), and optional node features $X$. In the GIVE conceptualization, the goal is to create two complementary representations:
- **Invariant embedding**: Captures substructures and features whose relation to downstream labels is consistent across environments $E$ or under interventions; formally, $P(Y\,|\,C, E)$ is invariant in $E$, for $C = f_\mathrm{inv}(G)$.
- **Variant embedding**: Captures substructures whose statistical properties vary across conditions or environments and may contribute to spurious correlations.

The mathematical backbone of invariant embeddings is permutation invariance with respect to node relabeling; i.e., for a permutation $\sigma \in S_n$, $f_\mathrm{inv}(\sigma \cdot G) = f_\mathrm{inv}(G)$. This is realized at the graph level (for tasks like classification or regression) or at the node/edge level via permutation equivariant functions $g$ such that $g(\sigma \cdot G) = \sigma \cdot g(G)$. Universality results establish that suitable architectures (e.g., higher-order Folklore GNNs) can approximate every continuous invariant or equivariant function, up to the distinguishing power of Weisfeiler-Lehman graph isomorphism tests [2006.15646].

## 2. Deep Learning Architectures for GIVE

### 2.1 GOODFormer: Entropy-Guided GIVE

GOODFormer operationalizes the GIVE principle through three key modules [2508.00304]:
- **Entropy-Guided Invariant Subgraph Disentangler:** For each graph, computes attention maps via hybrid Transformer/MPNN layers. Two paths are created:
  - Invariant ($Z_c$): Softmax$(E)$ attention; edges/nodes with high attention are considered invariant.
  - Variant ($Z_s$): Softmax$(-E)$ attention; the complement, capturing variant content.
  Soft masks $M = \sigma(E)$ partition the adjacency $A$ into $A_c = M \odot A$ (invariant) and $A_s = (1-M) \odot A$ (variant), guiding MPNN propagation. An entropy penalty $L_E = \mathbb{E}_E [H(\mathrm{Softmax}(E))]$ enforces attention sharpness, supplemented at inference by per-graph temperature tuning to maintain sharpness under shifted test distributions.
- **Evolving Subgraph Positional/Structural Encoding (PSE):** Standard graph positional encodings are replaced with a trainable MPNN $h_\mathrm{PSE}$ that efficiently generates a PSE for each subgraph, with an auxiliary MLP $w_\mathrm{PSE}$ reconstructing a global hand-crafted PE to regulate information flow and prevent shortcut leakage between invariant/variant streams.
- **Invariant Learning with Interventional Loss:** Final merged representations from invariant and variant streams are pooled and classified separately; a causal-intervention-inspired loss (with variance-penalized risk under do($G_S$)) is minimized to force $G_S$'s (variant subgraph) information to be uninformative, given $G_C$ (invariant subgraph). The final prediction uses only $\hat{y}_c$, "blocking" spurious effects at inference. Training jointly optimizes disentanglement, PSE reconstruction, and invariant objectives.

These modules jointly yield a robust graph embedding generalizing under substantial distribution shifts, as evidenced in benchmarks [2508.00304].

### 2.2 SNIGL: Probability of Necessity and Sufficiency–Based GIVE

SNIGL [2407.15273] refines the GIVE paradigm by requiring that the extracted invariant subgraph $C$ is both necessary and sufficient for the label $Y$. The "probability of necessity and sufficiency" (PNS) framework rigorously quantifies this core invariance:
\[
\mathrm{PNS}(c,y) = P(Y_{do(C=c)}=y,\, Y_{do(C\neq c)} \neq y)
\]
The SNIGL architecture comprises two parallel rationale-extracting GNNs: $f_{\theta^c}$ for $C$ (invariant) and $f_{\theta^s}$ for $S$ (variant), with subsequent independent classifiers and a calibrated, ensemble COMBINE prediction rule at inference. The training objective explicitly minimizes PNS-based risk, auxiliary invariance risk, joint cross-entropy, and independence penalties, addressing OOD generalization loss due to insufficient or unnecessary invariants. Empirical evaluation on OOD benchmarks shows SNIGL achieving state-of-the-art OOD robustness [2407.15273].

### 2.3 Sinkhorn-Based GIVE Extensions

GSINA [2402.07191] proposes Graph Sinkhorn Attention, leveraging entropic optimal transport to achieve sparse, soft, and fully differentiable attention masks for subgraph extraction. The GIVE extension computes two attention-based embeddings: an invariant stream using the first row of the Sinkhorn transport plan, and a variant stream using the complement. Fused representations allow for simultaneous training of invariant and variant channels, with optional orthogonality or contrastive penalties to enforce information separation [2402.07191].

### 2.4 Longitudinal Dynamics: Brain Graphs

In longitudinal brain connectome modeling, GIVE facilitates temporal and spatial decomposition by using EvolveGCN-style time-dynamic node embeddings as invariants, and hypergraph-based edge embeddings (spatial and cross-time) as variants. These multi-type embeddings allow interpretable, tokenized input to Transformer-based architectures for diagnosis/prognosis tasks [2307.00858].

## 3. Theoretical Guarantees and Expressiveness

GIVE unifies the function classes of permutation-invariant and partial permutation-invariant maps. Theorem 6 in [1909.12903] establishes that any continuous (partial) permutation-invariant function on neighborhoods (or graphs) can be represented as
\[
f(X_1,\ldots,X_K) = \rho\biggl(\sum_{n=1}^{N_1} \phi_1(x_{1,n}), \ldots, \sum_{n=1}^{N_K} \phi_K(x_{K,n}) \biggr)
\]
with (shared) MLPs $\phi_k,\, \rho$, thus enabling universal approximation in (typed) graphs. This mathematical form underlies practical invariance-oriented architectures such as PINE, guaranteeing that all node or graph representations are robust to neighbor ordering and identity [1909.12903].

More generally, [2006.15646] shows that for any $k$, $k$-FGNNs achieve the full expressiveness of $(k+1)$-WL, with universality as $k\to n$ (number of nodes). Invariant layers capture graph-level tasks; equivariant layers are required for node- or edge-level tasks.

## 4. Practical Methodologies: Masking, Attention, and Efficient Embedding

In practice, various design patterns are established:
- **Soft/hard subgraph masking:** Masks (soft via sigmoid or Sinkhorn, hard via sampling) assign edge/node membership to invariant/variant subgraphs.
- **Hybrid GNN–Transformer blocks:** Enable simultaneous long-range/global and local processing, with fine-grained attention used to distinguish invariants from variants.
- **Masked batched tensors:** Padding and binary masks (as in FGNNs [2006.15646]) enable efficient batching and training over variable-sized graphs.
- **Bi-Lipschitz and kernel-based permutation-invariant embeddings:** Sorting-based and polynomial-based mechanisms (see [2203.07546]) yield embeddings that are robust, injective, and theoretically suited for universal function approximation on graphs, enabling permutation-invariant downstream learning.

## 5. Applications and Empirical Results

GIVE has demonstrated significant utility in:
- **OOD generalization in molecular property prediction:** Both GOODFormer and SNIGL outperform prior baselines on synthetic and real-world datasets, including motif-perturbed and scaffold-split benchmarks. For instance, on GOOD-HIV and OGBG-Molsider, SNIGL shows ROC-AUC improvements of 2–4% over the second-best [2407.15273, 2508.00304].
- **Brain connectome analysis:** GIVE-based longitudinal representations yield notable gains in distinguishing Alzheimer’s stages and progression on fMRI data [2307.00858].
- **Universal node/graph embeddings:** PINE outperforms DeepWalk, node2vec, GCN, and GAT on both homogeneous and heterogeneous node classification, with 2–5 points improvement across evaluation splits [1909.12903].

A summary table of key GIVE paradigms and their architectural signatures:

| Model         | Invariant Subgraph Extraction | Variant Channel | Fusion/Downstream           |
|:--------------|:-----------------------------|:---------------|:----------------------------|
| GOODFormer    | Entropy-guided softmask + hybrid attention | Complementary softmax; MPNN/MLP | Interventional loss, block variant at test |
| SNIGL         | PNS-optimized masking (Gumbel-Softmax)    | Parallel rationale GNN, domain-specific | Logit-space calibrated ensemble           |
| GSINA         | Sinkhorn OT-based edge selection           | Residual attention            | Concatenation or contrastive head         |
| PINE          | Partial permutation-invariant sum + MLP    | N/A (nodewise emb.)           | Direct node embedding                     |
| BrainTokenGT  | GCN+GRU node evolution                    | Dual (spatial/temporal) edge token | Token-level Transformer readout      |

## 6. Open Problems and Future Directions

Current challenges and future research avenues for GIVE include:
- **Relaxing independence assumptions:** Many algorithms assume conditional independence $C \perp S \mid Y$ for clean decomposition; relaxing this may broaden the framework's applicability [2407.15273].
- **Scaling high-order invariance and efficient embedding:** While higher-order FGNNs yield maximal expressivity, their $O(n^{k+1})$ complexity for k-WL separation remains a bottleneck [2006.15646].
- **Automated subgraph selection:** Strategies for subgraph size/higher-order motif discovery in $C/S$ remain mostly hand-tuned or fixed [2508.00304, 2407.15273].
- **Generalization beyond binary/multiclass labels:** Extending GIVE to multi-label or regression settings, and for dynamic or heterogeneous graphs, is ongoing.
- **Theory-guided learning objectives:** Incorporating recent advances in causal risk minimization and information bottleneck regularization to further enforce invariance [2407.15273, 2508.00304].

GIVE synthesizes foundational invariance theories, deep graph representation learning techniques, and modern causal-verification principles to deliver robust, interpretable, and powerful graph embeddings for a wide array of challenging tasks.

Source: https://www.emergentmind.com/topics/graph-invariant-and-variant-embedding-give