Beyond Low-frequency Information in Graph Convolutional Networks (2101.00797v1)

Abstract: Graph neural networks (GNNs) have been proven to be effective in various network-related tasks. Most existing GNNs usually exploit the low-frequency signals of node features, which gives rise to one fundamental question: is the low-frequency information all we need in the real world applications? In this paper, we first present an experimental investigation assessing the roles of low-frequency and high-frequency signals, where the results clearly show that exploring low-frequency signal only is distant from learning an effective node representation in different scenarios. How can we adaptively learn more information beyond low-frequency information in GNNs? A well-informed answer can help GNNs enhance the adaptability. We tackle this challenge and propose a novel Frequency Adaptation Graph Convolutional Networks (FAGCN) with a self-gating mechanism, which can adaptively integrate different signals in the process of message passing. For a deeper understanding, we theoretically analyze the roles of low-frequency signals and high-frequency signals on learning node representations, which further explains why FAGCN can perform well on different types of networks. Extensive experiments on six real-world networks validate that FAGCN not only alleviates the over-smoothing problem, but also has advantages over the state-of-the-arts.

• The paper presents FAGCN, a technique that adaptively combines low and high-frequency signals to improve node representation.
• It employs an enhanced filtering and self-gating mechanism to dissect and flexibly aggregate node features without prior structural assumptions.
• Empirical results demonstrate that FAGCN outperforms state-of-the-art methods, effectively mitigating over-smoothing in disassortative networks.

Introduction to Frequency Adapting Mechanism in GNNs

Graph Neural Networks (GNNs), a significant breakthrough in AI, have shown great success in tasks involving networked data. The typical GNNs operate under the assumption that the similarity in data attributes or features propagates through interconnected nodes—an idea that promotes reliance on low-frequency information. However, the sufficiency and exclusivity of low-frequency information in practical applications remain questionable, leading to the main inquiry of this paper: do GNNs fully leverage the information embedded in node features for learning effective node representation?

The Role of Signal Frequencies in GNNs

Through a series of controlled experiments, authors dissect the roles of both low-frequency and high-frequency signals necessary for learning node representations. They find that both types of signals contribute to node classification in different scenarios. Particularly, in real-world networks that often exhibit non-assortative behavior (nodes from distinct classes are interlinked), the paper reveals that high-frequency information, capturing node differences, holds significant value. This sets the scene for the development of a new method that can adaptively harness information from different frequency signals during node feature aggregation, dubbed as the Frequency Adapting Graph Convolutional Network (FAGCN).

FAGCN: A Novel Approach

FAGCN proposes a robust solution to adaptively integrate signals of varying frequencies in the message passing process of GNNs. It utilizes an enhanced filter to dissect low and high-frequency signals from raw node features, integrating them without the prior knowledge of network structure or assortativity. Furthermore, a self-gating mechanism is designed for the flexible aggregation of different signals from neighboring nodes. The model is tested across six real-world networks and is theoretically analyzed to underscore its capability to avoid over-smoothing—a common pitfall that leads to the loss of discriminative power in node representations.

Performance and Implications

Empirical results confirm that FAGCN not only addresses the over-smoothing issue but also showcases superior performance over state-of-the-art methods on various networks, particularly in disassortative settings. A visual examination of the edge coefficients employed by FAGCN further substantiates its adaptability in different network contexts. This advancement promises to enhance GNNs' applicability and accuracy in scenarios where both similarity and dissimilarity among data points need equal consideration within network structures. The paper suggests that future studies might delve into leveraging signals with multiple frequency ranges to further enrich the representational capacity of GNNs.

In conclusion, this paper stands out in the field of graph neural networks by interrogating the dominant assumptions of signal processing and providing an innovative framework to utilize both low and high-frequency information effectively. The findings and the proposed FAGCN model mark a significant step toward more adaptable and robust GNNs that can deal with the complexity of real-world data.