Full-Spectrum Graph Neural Network: Expressive and Scalable

Abstract: It is well established that spectral graph neural networks (GNNs) can universally approximate node signals; however, their expressive power remains bounded by the 1-dimensional Weisfeiler-Lehman test, which is mirrored in their lack of universality for higher-order signals. To go beyond this bound, we propose the Full-Spectrum GNN (FSpecGNN), a second-order generalization of classical spectral GNNs. FSpecGNN advances spectral filtering in two perspectives: (1) it lifts the signal from the node domain to the node-pair domain; and (2) it extends the univariate spectral filter over eigenvalues to a bivariate filter over eigenvalue pairs. We show that classical spectral GNNs arise as a diagonal special case of FSpecGNN, and prove that FSpecGNN can be at most as expressive as Local 2-GNN while universally approximating node-pair signals, the latter being particularly beneficial for heterophilic graph learning. Moreover, FSpecGNN admits scalable implementations that avoid explicit node-pair-level computations; combined with a low-rank approximation that reduces full-spectrum convolution to a combination of polynomial spectral filters, it enables learning on large graphs. Empirically, FSpecGNN validates the predicted expressivity and delivers strong performance on heterophilic benchmarks.

• The paper introduces FSPECGNN, extending Laplacian filters to the node-pair domain to achieve second-order spectral expressivity.
• It employs structured tensor decompositions and low-rank approximations to efficiently overcome the computational bottlenecks of traditional methods.
• Empirical studies demonstrate that FSPECGNN outperforms classical spectral models, especially in handling heterophily and substructure-sensitive tasks.

Full-Spectrum Graph Neural Network: A Second-Order Spectral Paradigm

Motivation and Theoretical Advancement

The work introduces the Full-Spectrum Graph Neural Network (FSPECGNN), a novel spectral GNN architecture that systematically extends the expressive power of Laplacian-based graph filters from the classical node domain to the node-pair domain. This is accomplished via two pivotal generalizations: lifting the input signal to the space of node pairs (i.e., $V \times V$), and generalizing univariate Laplacian filters to bivariate spectral filters acting over pairs of eigenvalues. The construction unifies the spectrum of Laplacians across both nodes in a pair, enabling expressive transformations that operate beyond the diagonal entries of classical spectral filtering.

Theoretically, the architecture is established as the proper second-order analog of classical spectral GNNs, strictly subsuming standard spectral models as a special (diagonal) case. The authors rigorously prove that FSPECGNN universally approximates node-pair signals and, under mild conditions (simple spectrum, full Fourier support), matches or exceeds the expressivity of Local 2-GNNs in distinguishing non-isomorphic graphs. This surpasses the expressivity barrier inherent to classical spectral GNNs, which are upper-bounded by the 1-WL isomorphism test. FSPECGNN thus fills a critical gap in the spectral GNN literature—providing a tractable, rigorous route to higher-order expressivity without resorting to intractable spatial message passing in the full $\mathcal{O}(n^2)$ node-pair domain.

Scalable Implementation Strategies

Second-order lifting typically presents severe computational bottlenecks. A salient contribution of the paper is a suite of scalable implementation schemes that circumvent the explicit construction and manipulation of $n^2 \times n^2$ objects. The key is interpreting polynomial bivariate filters as structured Kronecker and tensor products of univariate filters, with computation efficiently reduced to linear-algebraic primitives on $n \times n$ matrices via Lemma 2.3 and Proposition 3.9. The authors demonstrate that low-rank tensor approximations of the filter coefficients enable practical implementations with runtime and memory complexity comparable to previous polynomial spectral GNNs, even for moderate filter sizes. For small graphs or analysis, the framework allows explicit eigenspace learning of the spectral filter, with guarantees of exact universality.

Expressivity, Heterophily, and Spectral Off-Diagonality

The FSPECGNN is shown to model off-diagonal spectral interactions, which the paper demonstrates as essential for high-performance under strong heterophily. Mathematically, the optimal graph convolution operator for maximal inter-class separation must annihilate inter-class edges—an operator that cannot be realized by classical (diagonal) spectral filters, as proved in Theorem 4.2. The FSPECGNN can directly express such block structures via its bivariate filter, providing both a conceptual and practical resolution to the representational limitations of previously-known spectral GNNs. The prioritization of spectral off-diagonals is empirically validated: as heterophily increases, learned filters concentrate energy further off the diagonal, aligning with the inability of diagonal-only architectures to fit such tasks.

Experimental Validation

Empirical analyses on both canonical and synthetic tasks underpin the theoretical advances. On heterophilic node classification benchmarks (e.g., Texas, Chameleon, Squirrel) and structure counting (homomorphisms/cycles), FSPECGNN consistently outperforms state-of-the-art spectral GNNs (ChebNet, BernNet, JacobiConv, GPRGNN, ChebNetII), and closely matches or exceeds the performance of second-order spatial GNNs, but with markedly greater efficiency. For example, FSPECGNN achieves up to $5\times$ faster runtime and significantly lower peak GPU memory consumption compared to its spatial counterparts, as demonstrated in detailed ablation and runtime studies.

Ablation studies confirm that the model's off-diagonal spectral modeling and in-filter components are essential, producing substantial drops in accuracy if omitted—a result precisely consistent with the theoretical framework. Notably, the lowest MAE is consistently achieved on substructure counting tasks by either FSPECGNN or Local 2-GNN, validating the predicted expressivity.

Implications and Future Directions

FSPECGNN's construction sets forth a natural blueprint for developing higher-order spectral GNNs, generalizing beyond pairs to $k$-tuples, which would bridge to higher-WL expressivity in spectral models. The approach is theoretically extensible to Laplace operators on other combinatorial/topological domains, e.g., cell/simplicial complexes, yielding clear routes toward multi-scale, multi-order spectral learning. The tractability of spectral filtering via tensor decompositions invites the integration of adaptive spectral parameterizations and efficient basis selection, pointing toward a family of faster, more expressive GNNs for graphs with complex, irregular, or heterophilic patterns.

Of note, the universality results obtained herein are contingent on full spectral support in the node-pair domain and require modification for multi-channel and deep nonlinear settings. The canonical implementation choices (GAT-based initialization, low-rank spectral decompositions) afford wide latitude for future architectural augmentation, including deeper exploration of initialization, filter parameterization and stacking strategies.

Conclusion

The Full-Spectrum GNN (FSPECGNN) provides the first scalable, expressive, and principled second-order extension of spectral GNNs, systematically overcoming both the 1-WL expressivity barrier and the diagonal limitation of Laplacian filtering. By unifying efficient computation with spectral generalization, the framework significantly strengthens the utility of spectral GNNs, with compelling empirical and theoretical support for heterophilic and substructure-sensitive learning tasks. This establishes FSPECGNN as a technically robust and versatile foundation for the next generation of expressive and scalable spectral graph representation models.