Geometric Scattering Attention Networks (GSAN)

• The paper introduces GSAN, which fuses low-pass graph convolutions with geometric scattering-derived band-pass filters to overcome oversmoothing in node classification.
• The method leverages node-wise multi-head attention to automatically balance filter outputs, enabling adaptive spectral-channel selection.
• GSAN builds on diffusion wavelets to capture multiscale and higher-frequency structural details in weighted graphs for enhanced representation learning.

Searching arXiv for GSAN and related geometric scattering papers to ground the article in the relevant literature. Geometric Scattering Attention Networks (GSAN) are hybrid graph neural networks that combine low-pass graph convolution channels with band-pass channels derived from geometric scattering, and fuse these responses through node-wise multi-head attention to produce adaptive task-driven node representations (Min et al., 2020). The architecture was introduced for semi-supervised node classification on weighted graphs, with the explicit aim of mitigating oversmoothing in graph convolutional networks while preserving access to higher-frequency and multiscale structural information (Min et al., 2020). Within the broader development of graph scattering methods, GSAN occupies an intermediate position between fixed wavelet-based representations and fully learned message-passing systems: it retains the geometric scattering machinery built from diffusion wavelets, but replaces handcrafted channel balancing with learned attention over filter outputs (Min et al., 2020). Subsequent work on hybrid scattering networks and learnable geometric scattering modules further clarified GSAN’s relationship to oversmoothing, underreaching, and adaptive multiscale filtering (Wenkel et al., 2022, Tong et al., 2020).

1. Origins and problem setting

GSAN was proposed in the context of semi-supervised node classification on a weighted graph

$G=(V,E,w),$

with nodes $V=\{v_1,\dots,v_n\}$, weighted edges $w:E\to(0,\infty)$, adjacency matrix $\mathbf W$, degree matrix $\mathbf D$, and node features

$\mathbf X\in\mathbb R^{n\times d_0}.$

A graph neural network then produces layer-wise node embeddings $\mathbf H^{(\ell)}\in\mathbb R^{n\times d_\ell}$, with $\mathbf H^{(0)}=\mathbf X$ (Min et al., 2020).

The immediate motivation was the observation that standard graph convolutional networks perform local smoothing through updates of the form

$$\mathbf H^{(\ell)}=\sigma\!\left(\mathbf A\,\mathbf H^{(\ell-1)}\mathbf\Theta^{(\ell)}\right),$$

where

$$\mathbf A=(\mathbf D+\mathbf I_n)^{-1/2}(\mathbf W+\mathbf I_n)(\mathbf D+\mathbf I_n)^{-1/2}.$$

This operation is effectively low-pass: repeated propagation pushes features toward smooth, low-frequency components, which can cause neighboring node representations to become increasingly similar. GSAN was therefore designed to preserve both smooth structural content and non-smooth, higher-order variation by combining GCN-type channels with scattering channels (Min et al., 2020).

The model also emerged from the limitations of an earlier hybrid architecture, Sc-GCN, which already combined GCN and scattering filters but required manual selection of the balance between low-pass and band-pass components (Min et al., 2020). GSAN replaced this fixed balance with node-wise attention. A later formulation framed the same issue in graph signal processing terms: shallow GCN-style networks avoid severe oversmoothing but then underreach, because information can only travel a few hops; geometric scattering adds band-pass filters that preserve higher-frequency structural information and capture longer-range interactions, while attention allows the network to decide locally which spectral content to emphasize (Wenkel et al., 2022).

This suggests that GSAN should be understood less as a variant of neighbor-attention models such as GAT, and more as a spectral-channel selection architecture: the central adaptive choice is not which neighbors to emphasize, but which filtered representations matter at each node.

2. Geometric scattering foundations

The scattering component in GSAN is built from the lazy random walk matrix

$V=\{v_1,\dots,v_n\}$0

which defines a diffusion process on the graph (Min et al., 2020). From $V=\{v_1,\dots,v_n\}$1, the model constructs diffusion wavelets

$V=\{v_1,\dots,v_n\}$2

so that the first filter captures immediate local contrast and larger $V=\{v_1,\dots,v_n\}$3 represent progressively coarser diffusion scales (Min et al., 2020).

Scattering features are generated by cascading these wavelet