Sparse Signed Message Passing Networks

• Sparse Signed Message Passing Networks are GNN architectures that integrate Bayesian modeling with sparse coding to enable robust semi-supervised node classification.
• They employ Monte Carlo sampling over latent signed adjacency matrices to capture edge uncertainty and mitigate adverse effects of heterophily.
• Empirical evaluations on heterophilic graphs demonstrate enhanced accuracy and scalability compared to traditional baselines under structural noise.

Sparse Signed Message Passing Networks (SpaM) are a class of graph neural network (GNN) architectures that address the challenge of semi-supervised node classification under structural uncertainty and heterophily in real-world graphs. Instead of operating on fixed, unsigned graph structures, these networks explicitly infer a posterior distribution over signed adjacency matrices where each edge can be positive, negative, or absent. The methodology is grounded in Bayesian principles and leverages sparse coding techniques for selective signed message aggregation, yielding notable robustness to both edge noise and label-disassortative (heterophilic) structures (Choi et al., 3 Jan 2026).

1. Bayesian Modeling of Signed Graph Structure

SpaM formalizes the true signed adjacency matrix $Z \in \{-1,0,+1\}^{n \times n}$ as a latent random variable. Observed graph edges are treated as noisy manifestations of the underlying $Z$. A factorized prior is imposed: $$p(Z) = \prod_{(i,j)\in\mathcal{E}_\text{obs}} p(z_{ij}), \quad \text{with} \quad p(z_{ij}=s) = \pi^0_s, \; s \in \{-1,0,+1\}$$ and $z_{ij}=0$ for $(i,j)\notin\mathcal{E}_{\rm obs}$.

Given node features $X$ and a candidate signed adjacency $Z$, the likelihood for semi-supervised classification is defined through a GNN parameterized by $\theta$: $$p_\theta(y_i\mid X,Z) = \text{softmax}(W_c h_i^{(L)} + c)$$ and

$$p_\theta(Y_\mathcal{L}\mid X,Z) = \prod_{i\in\mathcal{L}} p_\theta(y_i\mid X,Z)$$

forming the basis for Bayesian inference over both graph structure and node labels.

The posterior over signed structure, up to normalization, is

$$p(Z \mid A_\text{obs}, X, Y_\mathcal{L}) \propto p(Z) \; p(A_\text{obs} \mid Z) \; p_\theta(Y_\mathcal{L} \mid X, Z)$$

with variational inference employed for tractability. The chosen variational family is a factorized categorical distribution per observed edge, parameterized by a VGAE-style encoder.

2. Posterior Marginalization and Edge Uncertainty

Rather than relying on a single graph realization, SpaM marginalizes over possible signed structures, providing Bayes-optimal prediction for an unlabeled node $i$: $$p^*(y_i \mid A_\text{obs}, X, Y_\mathcal{L}) = \mathbb{E}_{Z \sim p(Z \mid A_\text{obs}, X, Y_\mathcal{L})}\left[ p_\theta(y_i \mid X, Z) \right]$$ This is approximated by Monte Carlo sampling from the variational posterior: $$\hat p_\theta(y_i \mid A_\text{obs}, X) \approx \frac{1}{K} \sum_{k=1}^K p_\theta(y_i \mid X, Z^{(k)})$$ with $$Z^{(k)} \sim q_\phi(Z \mid A_\text{obs}, X, Y_\mathcal{L})$$.

Edge-wise uncertainty is retained via the learned probabilities $\{\pi^+_{ij}, \pi^0_{ij}, \pi^-_{ij}\}$, providing direct quantification of structural ambiguity.

3. Sparse Signed Message Passing and Aggregation

For each Monte Carlo sample of $Z$, the message passing layer integrates three key steps:

1. Value Projection: Node states $H^{(\ell)}$ are projected via $V = H^{(\ell)} W_v$.
2. Local Sparse Coding (LASSO): For node $i$, a neighborhood dictionary $V_i$ is formed from signed neighbors. The coding problem seeks sparse coefficients $\alpha_i^\star$ by minimizing

$\| t_i - V_i \alpha \|_2^2 + \lambda \|\alpha\|_1$

with target $t_i = W_t h_i^{(\ell)}$. This enforces selective aggregation.

1. Signed Aggregation: The coding coefficients $\alpha_i^\star$ are decomposed into positive and negative contributions:

$$h_i^{(\ell+1)} = \sigma \left( W_o \left[ \sum_{j \in \mathcal{N}_i^+} \alpha_{ij}^+ v_j - \gamma \sum_{j \in \mathcal{N}_i^-} |\alpha_{ij}^-| v_j \right] + b \right)$$

where positive neighbors contribute additively while negative neighbors exert subtractive influence, scaled by $\gamma$. Sparsity arises through the $\ell_1$-penalized LASSO coding.

4. Training Procedure and Objective

SpaM training interleaves variational structure inference and message passing, with Algorithm 1 outlining one training epoch. Key steps include:

• Encoding structure using a two-layer GCN followed by MLP to parameterize edge-wise categorical distributions for $q_\phi(Z)$.
• Monte Carlo sampling of signed graph structures.
• Forward propagation through $L$ layers of sparse signed aggregation.
• Accumulation of classification loss ($\mathcal{L}_{\rm cls}$), sparsity penalty ($\mathcal{L}_{\rm sp}$), and a structural regularization term combining KL divergence and graph likelihood ($\mathcal{L}_{\rm struct}$):

$$\mathcal{L}_{\rm total} = \frac{1}{|\mathcal{L}|}\mathcal{L}_{\rm cls} + \lambda_{\rm sp}\mathcal{L}_{\rm sp} + \lambda_{\rm st}\mathcal{L}_{\rm struct}$$

• Optimization is performed jointly over GNN and variational parameters using Adam.
• At inference, MC samples of $Z$ are drawn, predictions aggregated, and edge uncertainty remains explicit.

5. Computational Complexity and Scalability

Let $n$ be the number of nodes, $m$ observable edges, average degree $\bar d = m/n$, and $d_{\rm val}$ the message dimension. Each layer costs $O(n\, d_{\rm val}\, \bar d)$ (LASSO coding), plus $O(m\, d_{\rm val})$ (signed aggregation), and $O(Km)$ for MC samples. Thus, for $L$ layers, per-epoch cost is: $$O\bigl( K L ( n\, d_{\rm val}\, \bar d + m\, d_{\rm val} ) \bigr)$$ which is linear in both $n$ and $m$, and in practice, $K=5\text{–}10$ suffices for reliable marginalization. This scaling enables practical training on large graphs (Choi et al., 3 Jan 2026).

6. Theoretical Guarantees: Robustness to Heterophily and Structural Noise

SpaM provides explicit robustness guarantees:

• Risk Decomposition: Theorem 3.1 bounds excess risk due to variational inference by the $\ell_1$ distance between approximate and true structure posteriors, linking structural inference quality directly to node classification performance.
• Heterophily Handling: Under the Contextual Stochastic Block Model (CSBM), signed aggregation complemented by sparsity approximates Bayes-optimal updates. Theorem 3.4 demonstrates that, when inter-cluster edge probability exceeds intra-cluster ($p_{\rm out} > p_{\rm in}$), negative edges enhance inter-cluster margin.
• Sparsity Recovery: Theorem 3.5 establishes that the LASSO-based sparse aggregation preferentially selects informative neighbors, actively suppressing contributions from noisy or uncorrelated nodes.

This framework is consequently robust to edge deletions, adversarial edge additions, and feature noise.

7. Empirical Evaluation and Comparative Results

Datasets and Noise Injections

• Benchmarks: Nine heterophilic graphs (e.g., RomanEmpire, Chameleon, Cornell, Texas), homophilic graphs (Cora, Citeseer, Pubmed), and larger heterophilic graphs (Penn94, arXiv-year, snap-patents).
• Perturbations Evaluated: Random edge deletions, additive Gaussian feature noise, targeted adversarial edge perturbations (bounded-budget attacks).

Results Summary

Dataset Type	SpaM Accuracy	Best Baseline Accuracy
Cornell (heterophilic)	70.8%	57.9%
Texas (heterophilic)	83.8%	76.7%
Penn94 (large heterophilic)	83.7%	81.3% (GCN)

• On all nine key heterophilic graphs, SpaM achieves the highest accuracy, outperforming state-of-the-art baselines.
• On homophilic benchmarks, SpaM matches or marginally exceeds GCN/GAT.
• Ablation studies reveal that full posterior marginalization yields a 7% gain over NoPosterior and 4% over HardSign variants.
• Removal of sparsity or sign-aware updates leads to accelerated oversmoothing.
• Marginalization over $K=4\text{–}8$ posterior samples achieves significant accuracy improvements.
• Empirical robustness curves show SpaM maintains superior accuracy under increasing structural noise or adversarial perturbations.

These results collectively establish SpaM as a principled, scalable, and robust framework for semi-supervised node classification with structural uncertainty and heterophily (Choi et al., 3 Jan 2026).