Papers
Topics
Authors
Recent
Search
2000 character limit reached

Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning

Published 25 Oct 2024 in cs.GT and cs.LG | (2410.19321v3)

Abstract: Federated learning (FL) is a machine learning paradigm that allows multiple FL participants (FL-PTs) to collaborate on training models without sharing private data. Due to data heterogeneity, negative transfer may occur in the FL training process. This necessitates FL-PT selection based on their data complementarity. In cross-silo FL, organizations that engage in business activities are key sources of FL-PTs. The resulting FL ecosystem has two features: (i) self-interest, and (ii) competition among FL-PTs. This requires the desirable FL-PT selection strategy to simultaneously mitigate the problems of free riders and conflicts of interest among competitors. To this end, we propose an optimal FL collaboration formation strategy -- FedEgoists -- which ensures that: (1) a FL-PT can benefit from FL if and only if it benefits the FL ecosystem, and (2) a FL-PT will not contribute to its competitors or their supporters. It provides an efficient clustering solution to group FL-PTs into coalitions, ensuring that within each coalition, FL-PTs share the same interest. We theoretically prove that the FL-PT coalitions formed are optimal since no coalitions can collaborate together to improve the utility of any of their members. Extensive experiments on widely adopted benchmark datasets demonstrate the effectiveness of FedEgoists compared to nine state-of-the-art baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL-PTs that engage in business activities.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Hedonic games. In Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D.Editors Procaccia, editors, Handbook of Computational Social Choice, page 356–376. Cambridge University Press, 2016. doi: 10.1017/CBO9781107446984.016.
  2. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  3. Coopetition: a systematic review, synthesis, and future research directions. Review of managerial science, 9:577–601, 2015.
  4. Fairness in federated learning via core-stability. In Advances in Neural Information Processing Systems (NeurIPS’22), volume 35, pages 5738–5750, 2022.
  5. Collaboration equilibrium in federated learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22), page 241–251, 2022a.
  6. Collaboration equilibrium in federated learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 241–251, 2022b.
  7. Dual transfer learning with generative filtering model for multiobjective multitasking optimization. Memetic Computing, 15(1):3–29, 2023.
  8. Collaborative learning by detecting collaboration partners. In Advances in Neural Information Processing Systems (NeurIPS’22), volume 35, pages 15629–15641, 2022.
  9. NetworkX Documentation. all_simple_paths. https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.simple_paths.all_simple_paths.html. Accessed: 2024-10-24.
  10. Model-sharing games: Analyzing federated learning under voluntary participation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(6):5303–5311, May 2021.
  11. Trustworthy Federated Learning. Springer, Cham, 2023.
  12. Coopetition against an amazon. Journal of Artificial Intelligence Research, 76:1077–1116, 2023.
  13. Byzantine-resilient decentralized stochastic gradient descent. IEEE Transactions on Circuits and Systems for Video Technology (TCVT), 32(6):4096–4106, 2021.
  14. Enhancing causal discovery in federated settings with limited local samples. In International Workshop on Federated Foundation Models in Conjunction with NeurIPS 2024, 2024.
  15. Learning conjoint attentions for graph neural nets. Advances in Neural Information Processing Systems, 34:2641–2653, 2021.
  16. Polarized message-passing in graph neural networks. Artificial Intelligence, 331:104129, 2024.
  17. Duopoly business competition in cross-silo federated learning. IEEE Transactions on Network Science and Engineering, 11(1):340–351, 2024a.
  18. Promoting collaboration in cross-silo federated learning: Challenges and opportunities. IEEE Communications Magazine, 62(4):82–88, 2024b.
  19. Towards the practical utility of federated learning in the medical domain. In Conference on Health, Inference, and Learning, pages 163–181. PMLR, 2023.
  20. Donald B Johnson. Finding all the elementary circuits of a directed graph. SIAM Journal on Computing, 4(1):77–84, 1975.
  21. Advances and open problems in federated learning. Found. Trends Mach. Learn., 14(1–2):1–210, jun 2021. ISSN 1935-8237. doi: 10.1561/2200000083.
  22. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pages 5132–5143. PMLR, 2020.
  23. Mechanisms that incentivize data sharing in federated learning. In the Workshop on Federated Learning: Recent Advances and New Challenges, in Conjunction with NeurIPS, 2022.
  24. Learning multiple layers of features from tiny images. 2009.
  25. Federated learning on non-iid data silos: An experimental study. In Proceedings of the IEEE 38th International Conference on Data Engineering (ICDE’22), pages 965–978, 2022.
  26. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429–450, 2020.
  27. Benchmarking data heterogeneity evaluation approaches for personalized federated learning. In International Workshop on Federated Foundation Models in Conjunction with NeurIPS 2024, pages 1–9, 2024.
  28. On privacy and personalization in cross-silo federated learning. Advances in Neural Information Processing Systems, 35:5925–5940, 2022a.
  29. Intelligent edge-enabled efficient multi-source data fusion for autonomous surface vehicles in maritime internet of things. IEEE Transactions on Green Communications and Networking (TGCN), 6(3):1574–1587, 2022b.
  30. Federated learning for open banking. In Federated Learning, pages 240–254. Springer, 2020.
  31. Joint filter and channel pruning of convolutional neural networks as a bi-level optimization problem. Memetic Computing, 16(1):71–90, 2024.
  32. Towards fair and privacy-preserving federated deep models. IEEE Transactions on Parallel and Distributed Systems, 31(11):2524–2541, 2020.
  33. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS’17), pages 1273–1282, 2017.
  34. Learning the pareto front with hypernetworks. In International Conference on Learning Representations (ICLR’21), 2021.
  35. Event-driven spiking neural networks with spike-based learning. Memetic Computing, 15(2):205–217, 2023.
  36. Industry-scale orchestrated federated learning for drug discovery. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 15576–15584, 2023.
  37. The eicu collaborative research database, a freely available multi-center database for critical care research. Scientific data, 5(1):1–13, 2018.
  38. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8):3710–3722, 2020.
  39. Personalized federated learning using hypernetworks. In International Conference on Machine Learning, pages 9489–9502. PMLR, 2021.
  40. Federated learning in competitive ev charging market. In 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE), pages 1–5, 2023.
  41. Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, 33:21394–21405, 2020.
  42. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, pages 1–17, 2022. doi: 10.1109/TNNLS.2022.3160699.
  43. Fedcompetitors: Harmonious collaboration in federated learning with competing participants. Proceedings of the AAAI Conference on Artificial Intelligence, 38(14):15231–15239, Mar. 2024.
  44. Robert Tarjan. Depth-first search and linear graph algorithms. SIAM journal on computing, 1(2):146–160, 1972.
  45. The worst-case time complexity for generating all maximal cliques and computational experiments. Theoretical computer science, 363(1):28–42, 2006.
  46. Strategic data sharing between competitors. Advances in Neural Information Processing Systems, 36, 2024.
  47. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  48. Fed-ltd: Towards cross-platform ride hailing via federated learning to dispatch. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22), pages 4079–4089, 2022.
  49. Personalized federated learning with parameter propagation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2594–2605, 2023a.
  50. Algorithms for scheduling deadline-sensitive malleable tasks. In Operations Research Forum, volume 5, page 30. Springer, 2024.
  51. Mars-fl: Enabling competitors to collaborate in federated learning. IEEE Transactions on Big Data, pages 1–11, 2022.
  52. A framework for allocating server time to spot and on-demand services in cloud computing. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), 4(4):1–31, 2019.
  53. Delay and price differentiation in cloud computing: A service model, supporting architectures, and performance. ACM Transactions on Modeling and Performance Evaluation of Computing Systems, 8(3):1–40, 2023b.
  54. Federated recommendation systems. In Federated Learning: Privacy and Incentive, pages 225–239. Springer, 2020a.
  55. Federated machine learning: concept and applications. ACM Transactions on Intelligent Systems and Technology, 10(2):12:1–12:19, 2019.
  56. Federated Learning: Privacy and Incentive. Springer, Cham, 2020b.
  57. Federated Learning. Springer, Cham, 2020c.
  58. Personalized federated learning with inferred collaboration graphs. In International Conference on Machine Learning, pages 39801–39817. PMLR, 2023a.
  59. Feddisco: Federated learning with discrepancy-aware collaboration. In International Conference on Machine Learning, pages 39879–39902. PMLR, 2023b.
  60. Reputation-aware task allocation for human trustees. In The 13th International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’14), pages 357–364, 2014.
  61. Mitigating herding in hierarchical crowdsourcing networks. Scientific Reports, 6(4), 2016.
  62. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11237–11244, 2023.
  63. Data-free knowledge distillation for heterogeneous federated learning. In International conference on machine learning, pages 12878–12889. PMLR, 2021.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free