Papers
Topics
Authors
Recent
2000 character limit reached

FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference (2410.20105v1)

Published 26 Oct 2024 in cs.LG and cs.CR

Abstract: Personalized Federated Graph Learning (pFGL) facilitates the decentralized training of Graph Neural Networks (GNNs) without compromising privacy while accommodating personalized requirements for non-IID participants. In cross-domain scenarios, structural heterogeneity poses significant challenges for pFGL. Nevertheless, previous pFGL methods incorrectly share non-generic knowledge globally and fail to tailor personalized solutions locally under domain structural shift. We innovatively reveal that the spectral nature of graphs can well reflect inherent domain structural shifts. Correspondingly, our method overcomes it by sharing generic spectral knowledge. Moreover, we indicate the biased message-passing schemes for graph structures and propose the personalized preference module. Combining both strategies, we propose our pFGL framework FedSSP which Shares generic Spectral knowledge while satisfying graph Preferences. Furthermore, We perform extensive experiments on cross-dataset and cross-domain settings to demonstrate the superiority of our framework. The code is available at https://github.com/OakleyTan/FedSSP.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. Personalized subgraph federated learning. In ICML, pages 1396–1415, 2023.
  2. Graph contrastive learning with stable and scalable spectral encoding. In NeurIPS, volume 36, 2023.
  3. Specformer: Spectral graph neural networks meet transformers. arXiv preprint arXiv:2303.01028, 2023.
  4. A survey on spectral graph neural networks. arXiv preprint arXiv:2302.05631, 2023.
  5. Rethinking graph neural architecture search from message-passing. In CVPR, pages 6657–6666, 2021.
  6. On bridging generic and personalized federated learning for image classification. arXiv preprint arXiv:2107.00778, 2021.
  7. Nagphormer: A tokenized graph transformer for node classification in large graphs. In ICLR, 2023.
  8. Ntformer: A composite node tokenized graph transformer for node classification. CoRR, abs/2406.19249, 2024.
  9. PAMT: A novel propagation-based approach via adaptive similarity mask for node classification. IEEE Trans. Comput. Soc. Syst., 11(5):5973–5983, 2024.
  10. Leveraging contrastive learning for enhanced node representations in tokenized graph transformers. In NeurIPS, 2024.
  11. Fede: Embedding knowledge graphs in federated setting. In IJCKG, pages 80–88, 2021.
  12. Bridging the gap between spatial and spectral domains: A unified framework for graph neural networks. ACM Computing Surveys, pages 1–42, 2023.
  13. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In NeurIPS, pages 3557–3568, 2020.
  14. Robust federated learning with noisy and heterogeneous clients. In CVPR, pages 10072–10081, 2022.
  15. Robust heterogeneous federated learning under data corruption. In ICCV, pages 5020–5030, 2023.
  16. How powerful are k-hop message passing graph neural networks. In NeurIPS, pages 4776–4790, 2022.
  17. Miroslav Fiedler. Algebraic connectivity of graphs. Czechoslovak mathematical journal, 1973.
  18. Federated graph machine learning: A survey of concepts, techniques, and applications. arXiv preprint arXiv:2207.11812, 2022.
  19. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997, 2018.
  20. Fedgraphnn: A federated learning system and benchmark for graph neural networks. In ICLR, 2021.
  21. Convolutional neural networks on graphs with chebyshev approximation, revisited. In NeurIPS, pages 7264–7276, 2022.
  22. Bernnet: Learning arbitrary graph spectral filters via bernstein approximation. In NeurIPS, pages 14239–14251, 2021.
  23. Fedmut: Generalized federated learning via stochastic mutation. In AAAI, pages 12528–12537, 2024.
  24. Fedcross: Towards accurate federated learning via multi-model cross-aggregation. In ICDE, pages 2137–2150, 2024.
  25. Federated graph semantic and structural learning. In IJCAI, pages 3830–3838, 2023.
  26. Few-shot model agnostic federated learning. In ACM MM, pages 7309–7316, 2022.
  27. Federated learning for generalization, robustness, fairness: A survey and benchmark. TPAMI, 2024.
  28. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977, 2019.
  29. Scaffold: Stochastic controlled averaging for on-device federated learning. In ICML, pages 5132–5143, 2020.
  30. A method for stochastic optimization. In ICLR, 2015.
  31. Rethinking graph transformers with spectral attention. In NeruIPS, pages 21618–21629, 2021.
  32. Model-contrastive federated learning. In CVPR, pages 10713–10722, 2021.
  33. A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE TKDE, pages 3347–3366, 2021.
  34. Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127, 2020.
  35. Fedphp: Federated personalization with inherited private models. In ECML, 2021.
  36. Fedgta: Topology-aware averaging for federated graph learning. In VLDB, 2024.
  37. A survey of knowledge graph reasoning on graph types: Static, dynamic, and multi-modal. TPAMI, 2024.
  38. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01523, 2020.
  39. Lanczosnet: Multi-scale deep graph convolutional networks. In ICLR, 2019.
  40. Federated graph neural networks: Overview, techniques and challenges. arXiv preprint arXiv:2202.07256, 2022.
  41. Exploiting variational domain-invariant user embedding for partially overlapped cross domain recommendation. In SIGIR, pages 312–321, 2022.
  42. Graph condensation via eigenbasis matching. arXiv preprint arXiv:2310.09202, 2023.
  43. Reinforcement graph clustering with unknown cluster number. In ACMMM, pages 3528–3537, 2023.
  44. Simple contrastive graph clustering. TNNLS, 2023.
  45. Hard sample aware network for contrastive deep graph clustering. In AAAI, pages 8914–8922, 2023.
  46. Epilearn: A python library for machine learning in epidemic modeling. arXiv preprint arXiv:2406.06016, 2024.
  47. A review of graph neural networks in epidemic modeling. In ACM SIGKDD, pages 6577–6587, 2024.
  48. Building powerful and equivariant graph neural networks with structural message-passing. NeurIPS, 2020.
  49. Adapt to adaptation: Learning personalization for cross-silo federated learning. In IJCAI, pages 2166–2173, 2022.
  50. Communication-efficient learning of deep networks from decentralized data. In AISTATS, pages 1273–1282, 2017.
  51. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663, 2020.
  52. The pagerank citation ranking: Bring order to the web. Technical report, 1998.
  53. Recipe for a general, powerful, scalable graph transformer. NeurIPS, pages 14501–14515, 2022.
  54. Self-supervised graph transformer on large-scale molecular data. In NeurIPS, pages 12559–12571, 2024.
  55. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868, 2018.
  56. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing. Mag., 2013.
  57. Federated multi-task learning. In NeurIPS, 2017.
  58. Does gnn pretraining help molecular representation? In NeurIPS, pages 12096–12109, 2022.
  59. Personalized federated learning with moreau envelopes. In NeurIPS, pages 21394–21405, 2020.
  60. Federated learning on non-iid graphs via structural knowledge sharing. In AAAI, pages 9953–9961, 2023.
  61. Building powerful and equivariant graph neural networks with structural message-passing. In NeurIPS, pages 14143–14155, 2020.
  62. Epidemiology-aware neural ode with continuous disease transmission graph. arXiv preprint arXiv:2410.00049, 2024.
  63. S3GCL: Spectral, swift, spatial graph contrastive learning. In ICML, 2024.
  64. Iterative structural inference of directed graphs. NeurIPS, 35:8717–8730, 2022.
  65. Active learning based structural inference. In ICML, pages 36224–36245, 2023.
  66. Structural inference with dynamics encoding and partial correlation coefficients. In ICLR, 2024.
  67. Effective and efficient structural inference with reservoir computing. In ICML, pages 36391–36410, 2023.
  68. Stone: A spatio-temporal ood learning framework kills both spatial and temporal shifts. In SIGKDD, pages 2948–2959, 2024.
  69. Towards dynamic spatial-temporal graph learning: A decoupled perspective. In AAAI, pages 9089–9097, 2024.
  70. Condition-guided urban traffic co-prediction with multiple sparse surveillance data. TVT, 2024.
  71. Knowledge expansion and consolidation for continual traffic prediction with expanding graphs. IEEE TITS, 2023.
  72. Pattern expansion and consolidation on evolving graphs for continual traffic prediction. In SIGKDD, pages 2223–2232, 2023.
  73. How powerful are spectral graph neural networks. In ICML, pages 23341–23362, 2023.
  74. Unifying unsupervised graph-level anomaly detection and out-of-distribution detection: A benchmark. arXiv preprint arXiv:2406.15523, 2024.
  75. A comprehensive survey on graph neural networks. IEEE TNNLS, pages 4–24, 2020.
  76. Federated graph classification over non-iid graphs. NeurIPS, 34:18839–18852, 2021.
  77. Tilp: Differentiable learning of temporal logical rules on knowledge graphs. In ICLR, 2023.
  78. Teilp: Time prediction over knowledge graphs via logical reasoning. In AAAI, volume 38, pages 16112–16119, 2024.
  79. Infogcl: Information-aware graph contrastive learning. In NeurIPS, pages 30414–30425, 2021.
  80. Does graph distillation see like vision dataset counterpart? In NeurIPS, volume 36, 2023.
  81. Heterogeneous federated learning: State-of-the-art and research challenges. ACM Computing Surveys, 2023.
  82. Vertical federated learning for effectiveness, security, applicability: A survey. arXiv preprint arXiv:2405.17495, 2024.
  83. Do transformers really perform badly for graph representation?, 2021.
  84. Cut the crap: An economical communication pipeline for llm-based multi-agent systems. arXiv preprint arXiv:2410.02506, 2024.
  85. G-designer: Architecting multi-agent communication topologies via graph neural networks. arXiv preprint arXiv:2410.11782, 2024.
  86. Bayesian graph local extrema convolution with long-tail strategy for misinformation detection. In SIGKDD, pages 1–21, 2024.
  87. Federated graph learning – a position paper. arXiv preprint arXiv:2105.11099, 2021.
  88. Eliminating domain bias for federated learning in representation space. In NeurIPS, volume 36, 2023.
  89. Fedala: Adaptive local aggregation for personalized federated learning. In AAAI, pages 11237–11244, 2023.
  90. Fedcp: Separating feature information for personalized federated learning via conditional policy. In SIGKDD, pages 3249–3261, 2023.
  91. Subgraph federated learning with missing neighbor generation. In NeurIPS, volume 34, pages 6671–6682, 2021.
  92. Robust self-supervised structural graph neural network for social network prediction. In WWW, pages 1352–1361, 2022.
  93. Minimizing polarization and disagreement in social networks via link recommendation. In NeurIPS, volume 34, pages 2072–2084, 2021.
  94. Interpreting and unifying graph neural networks with an optimization framework. In WWW, pages 1215–1226, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up