Beyond-diagonal Gaussian neighborhood surrogates

Develop more expressive neighborhood-distribution surrogates, such as mixture-of-Gaussian or full-covariance models, to represent teacher–student alignment costs for multimodal or strongly heterophilic graph neighborhoods beyond the diagonal Gaussian approximation.

Background

The G²MLP curvature-adaptive Wasserstein objective approximates each neighborhood’s hidden-feature distribution with a diagonal Gaussian obtained by moment matching. The paper notes that this approximation can collapse multiple modes into a single mean and coordinate-wise variance when neighborhoods contain mixed communities or lie between separated label groups.

The authors explicitly identify mixture-of-Gaussian and full-covariance extensions as unresolved. Although full-covariance Wasserstein computations are computationally expensive, the paper suggests that low-rank covariance approximations may offer a more practical direction.

References

Empirically the diagonal approximation is sufficient on every benchmark we evaluate (Tables~\ref{tab:transductive},~\ref{tab:inductive}), but extensions to mixture-of-Gaussian or full-covariance surrogates remain open.

— Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs  (2610.10520 - Chen et al., 7 Oct 2026) in Appendix, Section "Limitations and Future Directions," paragraph "Diagonal Gaussian neighborhood surrogate"