Gegenbauer Graph Neural Networks for Time-varying Signal Reconstruction (2403.19800v2)
Abstract: Reconstructing time-varying graph signals (or graph time-series imputation) is a critical problem in machine learning and signal processing with broad applications, ranging from missing data imputation in sensor networks to time-series forecasting. Accurately capturing the spatio-temporal information inherent in these signals is crucial for effectively addressing these tasks. However, existing approaches relying on smoothness assumptions of temporal differences and simple convex optimization techniques have inherent limitations. To address these challenges, we propose a novel approach that incorporates a learning module to enhance the accuracy of the downstream task. To this end, we introduce the Gegenbauer-based graph convolutional (GegenConv) operator, which is a generalization of the conventional Chebyshev graph convolution by leveraging the theory of Gegenbauer polynomials. By deviating from traditional convex problems, we expand the complexity of the model and offer a more accurate solution for recovering time-varying graph signals. Building upon GegenConv, we design the Gegenbauer-based time Graph Neural Network (GegenGNN) architecture, which adopts an encoder-decoder structure. Likewise, our approach also utilizes a dedicated loss function that incorporates a mean squared error component alongside Sobolev smoothness regularization. This combination enables GegenGNN to capture both the fidelity to ground truth and the underlying smoothness properties of the signals, enhancing the reconstruction performance. We conduct extensive experiments on real datasets to evaluate the effectiveness of our proposed approach. The experimental results demonstrate that GegenGNN outperforms state-of-the-art methods, showcasing its superior capability in recovering time-varying graph signals.
- A. Ortega, P. Frossard, J. Kovačević, J. M. Moura, and P. Vandergheynst, “Graph signal processing: Overview, challenges, and applications,” Proceedings of the IEEE, vol. 106, no. 5, pp. 808–828, 2018.
- M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” in NeurIPS, 2016.
- T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR, 2017.
- Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4–24, 2020.
- M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, “Geometric deep learning: going beyond euclidean data,” IEEE Signal Processing Magazine, vol. 34, no. 4, pp. 18–42, 2017.
- X. Fan, M. Gong, Z. Tang, and Y. Wu, “Deep neural message passing with hierarchical layer aggregation and neighbor normalization,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 12, pp. 7172–7184, 2022.
- L. Wu, H. Lin, B. Hu, C. Tan, Z. Gao, Z. Liu, and S. Z. Li, “Beyond homophily and homogeneity assumption: Relation-based frequency adaptive graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–13, 2023.
- Z. Zeng, Q. Peng, X. Mou, Y. Wang, and R. Li, “Graph neural networks with high-order polynomial spectral filters,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–14, 2023.
- X. Chen, R. Cai, Y. Fang, M. Wu, Z. Li, and Z. Hao, “Motif graph neural network,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2023.
- A. Duval and F. Malliaros, “Higher-order clustering and pooling for graph neural networks,” in CIKM, 2022.
- G. Li, M. Muller, A. Thabet, and B. Ghanem, “DeepGCNs: Can GCNs go as deep as CNNs?” in IEEE ICCV, 2019.
- J. H. Giraldo, S. Javed, and T. Bouwmans, “Graph moving object segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2485–2503, 2022.
- X. Zhang, C. Xu, X. Tian, and D. Tao, “Graph edge convolutional neural networks for skeleton-based action recognition,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 8, pp. 3047–3060, 2019.
- W. Uwents, G. Monfardini, H. Blockeel, M. Gori, and F. Scarselli, “Neural networks for relational learning: An experimental comparison,” Machine Learning, vol. 82, no. 3, pp. 315–349, 2011.
- L. Wu, J. Li, P. Sun, R. Hong, Y. Ge, and M. Wang, “DiffNet++: A neural influence and interest diffusion network for social recommendation,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 10, pp. 4753–4766, 2022.
- G. Panagopoulos, N. Tziortziotis, M. Vazirgiannis, and F. D. Malliaros, “Maximizing influence with graph neural networks,” in IEEE/ACM ASONAM, 2023.
- A. Benamira, B. Devillers, E. Lesot, A. K. Ray, M. Saadi, and F. D. Malliaros, “Semi-supervised learning and graph neural networks for fake news detection,” in IEEE/ACM ASONAM, 2019.
- A. Duval, V. Schmidt, A. Hernández-García, S. Miret, F. D. Malliaros, Y. Bengio, and D. Rolnick, “FAENet: Frame averaging equivariant GNN for materials modeling,” in ICML, 2023.
- P. Gainza, F. Sverrisson, F. Monti, E. Rodola, D. Boscaini, M. Bronstein, and B. Correia, “Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning,” Nature Methods, vol. 17, no. 2, pp. 184–192, 2020.
- A. G. Marques, S. Segarra, G. Leus, and A. Ribeiro, “Sampling of graph signals with successive local aggregations,” IEEE Transactions on Signal Processing, vol. 64, no. 7, pp. 1832–1843, 2015.
- D. Romero, M. Ma, and G. B. Giannakis, “Kernel-based reconstruction of graph signals,” IEEE Transactions on Signal Processing, vol. 65, no. 3, pp. 764–778, 2016.
- A. Parada-Mayorga, D. L. Lau, J. H. Giraldo, and G. R. Arce, “Blue-noise sampling on graphs,” IEEE Transactions on Signal and Information Processing over Networks, vol. 5, no. 3, pp. 554–569, 2019.
- J. H. Giraldo, A. Mahmood, B. Garcia-Garcia, D. Thanou, and T. Bouwmans, “Reconstruction of time-varying graph signals via Sobolev smoothness,” IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 201–214, 2022.
- J. A. Castro-Correa, J. H. Giraldo, A. Mondal, M. Badiey, T. Bouwmans, and F. D. Malliaros, “Time-varying signals recovery via graph neural networks,” in IEEE ICASSP, 2023.
- J. Lu, Z. Lai, H. Wang, Y. Chen, J. Zhou, and L. Shen, “Generalized embedding regression: A framework for supervised feature extraction,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 1, pp. 185–199, 2020.
- H. Zhang, J. Xia, G. Zhang, and M. Xu, “Learning graph representations through learning and propagating edge features,” IEEE Transactions on Neural Networks and Learning Systems, 2023.
- B. Girault, “Stationary graph signals using an isometric graph translation,” in EUSIPCO, 2015.
- J. H. Giraldo and T. Bouwmans, “On the minimization of Sobolev norms of time-varying graph signals: Estimation of new Coronavirus disease 2019 cases,” in IEEE MLSP, 2020.
- S. Chen and Y. C. Eldar, “Time-varying graph signal inpainting via unrolling networks,” in IEEE ICASSP, 2021.
- K. Qiu, X. Mao, X. Shen, X. Wang, T. Li, and Y. Gu, “Time-varying graph signal reconstruction,” IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 6, pp. 870–883, 2017.
- A. Anis, A. El Gamal, A. S. Avestimehr, and A. Ortega, “A sampling theory perspective of graph-based semi-supervised learning,” IEEE Transactions on Information Theory, vol. 65, no. 4, pp. 2322–2342, 2018.
- M. Badiey, L. Wan, and A. Song, “Three-dimensional mapping of evolving internal waves during the Shallow Water 2006 experiment,” The Journal of the Acoustical Society of America, vol. 134, no. 1, pp. EL7–EL13, 2013.
- S. Chen, R. Varma, A. Sandryhaila, and J. Kovačević, “Discrete signal processing on graphs: Sampling theory,” IEEE Transactions on Signal Processing, vol. 63, no. 24, pp. 6510–6523, 2015.
- P. Di Lorenzo, S. Barbarossa, P. Banelli, and S. Sardellitti, “Adaptive least mean squares estimation of graph signals,” IEEE Transactions on Signal and Information Processing over Networks, vol. 2, no. 4, pp. 555–568, 2016.
- A. Anis, A. Gadde, and A. Ortega, “Efficient sampling set selection for bandlimited graph signals using graph spectral proxies,” IEEE Transactions on Signal Processing, vol. 64, no. 14, pp. 3775–3789, 2016.
- S. P. Chepuri and G. Leus, “Graph sampling for covariance estimation,” IEEE Transactions on Signal and Information Processing over Networks, vol. 3, no. 3, pp. 451–466, 2017.
- D. Valsesia, G. Fracastoro, and E. Magli, “Sampling of graph signals via randomized local aggregations,” IEEE Transactions on Signal and Information Processing over Networks, vol. 5, no. 2, pp. 348–359, 2018.
- A. Venkitaraman, S. Chatterjee, and P. Händel, “Predicting graph signals using kernel regression where the input signal is agnostic to a graph,” IEEE Transactions on Signal and Information Processing over Networks, vol. 5, no. 4, pp. 698–710, 2019.
- A. Cini, I. Marisca, and C. Alippi, “Filling the G_ap_s: Multivariate time series imputation by graph neural networks,” in ICLR, 2022.
- Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” arXiv preprint arXiv:1906.00121, 2019.
- I. Pesenson, “Sampling in Paley-Wiener spaces on combinatorial graphs,” Transactions of the American Mathematical Society, vol. 360, no. 10, pp. 5603–5627, 2008.
- M. Belkin, I. Matveeva, and P. Niyogi, “Regularization and semi-supervised learning on large graphs,” in COLT, 2004.
- S. K. Narang, A. Gadde, E. Sanou, and A. Ortega, “Localized iterative methods for interpolation in graph structured data,” in IEEE GlobalSIP, 2013.
- S. Chen, R. Varma, A. Singh, and J. Kovačević, “Signal recovery on graphs: Fundamental limits of sampling strategies,” IEEE Transactions on Signal and Information Processing over Networks, vol. 2, no. 4, pp. 539–554, 2016.
- S. Chen, A. Sandryhaila, J. M. F. Moura, and J. Kovačević, “Signal recovery on graphs: Variation minimization,” IEEE Transactions on Signal Processing, vol. 63, no. 17, pp. 4609–4624, 2015.
- N. Perraudin and P. Vandergheynst, “Stationary signal processing on graphs,” IEEE Transactions on Signal Processing, vol. 65, no. 13, pp. 3462–3477, 2017.
- A. Loukas and N. Perraudin, “Stationary time-vertex signal processing,” EURASIP Journal on Advances in Signal Processing, vol. 2019, no. 1, pp. 1–19, 2019.
- F. Grassi, A. Loukas, N. Perraudin, and B. Ricaud, “A time-vertex signal processing framework: Scalable processing and meaningful representations for time-series on graphs,” IEEE Transactions on Signal Processing, vol. 66, no. 3, pp. 817–829, 2017.
- X. Wang, M. Wang, and Y. Gu, “A distributed tracking algorithm for reconstruction of graph signals,” IEEE Journal of Selected Topics in Signal Processing, vol. 9, no. 4, pp. 728–740, 2015.
- B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting,” arXiv preprint arXiv:1709.04875, 2017.
- D. Cao, Y. Wang, J. Duan, C. Zhang, X. Zhu, C. Huang, Y. Tong, B. Xu, J. Bai, J. Tong et al., “Spectral temporal graph neural network for multivariate time-series forecasting,” NeurIPS, 2020.
- M. Jin, Y. Zheng, Y.-F. Li, S. Chen, B. Yang, and S. Pan, “Multivariate time series forecasting with dynamic graph neural odes,” IEEE Transactions on Knowledge and Data Engineering, 2022.
- D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Processing Magazine, vol. 30, no. 3, pp. 83–98, 2013.
- V. Kalofolias, “How to learn a graph from smooth signals,” in AISTATS, 2016.
- W. Jin, Y. Ma, X. Liu, X. Tang, S. Wang, and J. Tang, “Graph structure learning for robust graph neural networks,” in ACM SIGKDD, 2020.
- X. Dong, D. Thanou, M. Rabbat, and P. Frossard, “Learning graphs from data: A signal representation perspective,” IEEE Signal Processing Magazine, vol. 36, no. 3, pp. 44–63, 2019.
- Q. Zhang, J. Chang, G. Meng, S. Xu, S. Xiang, and C. Pan, “Learning graph structure via graph convolutional networks,” Pattern Recognition, vol. 95, pp. 308–318, 2019.
- H. E. Egilmez, E. Pavez, and A. Ortega, “Graph learning from filtered signals: Graph system and diffusion kernel identification,” IEEE Transactions on Signal and Information Processing over Networks, vol. 5, no. 2, pp. 360–374, 2018.
- U. von Luxburg, “A tutorial on spectral clustering,” Statistics and Computing, vol. 17, no. 4, pp. 395–416, Dec 2007.
- A. Marco and J. J. Martinez, “Polynomial least squares fitting in the bernstein basis,” Linear Algebra and its Applications, vol. 433, no. 7, pp. 1254–1264, 2010.
- M. He, Z. Wei, Z. Huang, and H. Xu, “BernNet: Learning arbitrary graph spectral filters via bernstein approximation,” NeurIPS, 2021.
- X. Wang and M. Zhang, “How powerful are spectral graph neural networks,” in ICML, 2022.
- D. S. Kim, T. Kim, and S.-H. Rim, “Some identities involving gegenbauer polynomials,” Advances in Difference Equations, vol. 2012, pp. 1–11, 2012.
- J. A. Castro-Correa, S. A. Arnett, T. B. Neilsen, L. Wan, and M. Badiey, “Supervised classification of sound speed profiles via dictionary learning,” Journal of Atmospheric and Oceanic Technology, 2022.
- K. Kiani and K. Saleem, “K-nearest temperature trends: A method for weather temperature data imputation,” in ICISDM, 2017.
- P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” in ICLR, 2018.
- Y. Shi, Z. Huang, S. Feng, H. Zhong, W. Wang, and Y. Sun, “Masked label prediction: Unified message passing model for semi-supervised classification,” in IJCAI, 2021.
- L. Pasa, N. Navarin, and A. Sperduti, “Multiresolution reservoir graph neural network,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 6, pp. 2642–2653, 2021.
- M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” in ICLR-W, 2019.
- M. Badiey, L. Wan, and J. F. Lynch, “Statistics of nonlinear internal waves during the shallow water 2006 experiment,” Journal of Atmospheric and Oceanic Technology, vol. 33, no. 4, pp. 839 – 846, 2016.