Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Distributed Continual Learning (2405.17466v1)

Published 23 May 2024 in cs.LG and cs.MA

Abstract: This work studies the intersection of continual and federated learning, in which independent agents face unique tasks in their environments and incrementally develop and share knowledge. We introduce a mathematical framework capturing the essential aspects of distributed continual learning, including agent model and statistical heterogeneity, continual distribution shift, network topology, and communication constraints. Operating on the thesis that distributed continual learning enhances individual agent performance over single-agent learning, we identify three modes of information exchange: data instances, full model parameters, and modular (partial) model parameters. We develop algorithms for each sharing mode and conduct extensive empirical investigations across various datasets, topology structures, and communication limits. Our findings reveal three key insights: sharing parameters is more efficient than sharing data as tasks become more complex; modular parameter sharing yields the best performance while minimizing communication costs; and combining sharing modes can cumulatively improve performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Exploiting task relatedness for multiple task learning. In Bernhard Schölkopf and Manfred K. Warmuth, editors, Learning Theory and Kernel Machines, pages 567–580, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg.
  2. On the Control of Multi-Agent Systems: A Survey. 2019.
  3. Lifelong Machine Learning. Morgan & Claypool Publishers, 2nd edition, 2018.
  4. Palm: Scaling language modeling with pathways, 2022.
  5. Open x-embodiment: Robotic learning datasets and rt-x models, 2024.
  6. G. Csibra and G. Gergely. Natural pedagogy as evolutionary adaptation. Philos Trans R Soc Lond B Biol Sci, 366(1567):1149–1157, Apr 2011.
  7. I. Csiszar. I𝐼Iitalic_I-Divergence Geometry of Probability Distributions and Minimization Problems. The Annals of Probability, 3(1):146 – 158, 1975.
  8. Introduction to planning in multiagent systems. Multiagent and Grid Systems, 5:345–355, 2009. 4.
  9. Multi-agent systems: A survey. IEEE Access, 6:28573–28593, 2018.
  10. Edmund H. Durfee. Distributed Problem Solving and Planning, pages 118–149. Springer Berlin Heidelberg, Berlin, Heidelberg, 2001.
  11. P Erdös and A Rényi. On random graphs i. Publicationes Mathematicae Debrecen, 6:290–297, 1959.
  12. Melinda Fagan. Collective scientific knowledge. Philosophy Compass, 7(12):821–831, 2012.
  13. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022.
  14. Counterfactual multi-agent policy gradients, 2017.
  15. Lightweight learner for shared knowledge lifelong learning. Transactions on Machine Learning Research, 2023.
  16. Decentralized sharing and valuation of fleet robotic data. In Aleksandra Faust, David Hsu, and Gerhard Neumann, editors, Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 1795–1800. PMLR, 08–11 Nov 2022.
  17. Selective experience replay for lifelong learning. In AAAI, 2018.
  18. Adaptive group sparse regularization for continual learning. NeurIPS, 2020.
  19. Detecting change in data streams. pages 180–191, 04 2004.
  20. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, mar 2017.
  21. Modular networks: Learning to decompose neural computation, 2018.
  22. Multi-agent hierarchical reinforcement learning in urban and search rescue. Robotics Institute Summer Scholar’ Working Papers Journals, 9:157–164, 2021.
  23. Learning shared knowledge for deep lifelong learning using deconvolutional networks. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 2837–2844. International Joint Conferences on Artificial Intelligence Organization, 7 2019.
  24. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection, 2016.
  25. Sequential theory of mind modeling in team search and rescue tasks. In Nikolos Gurney and Gita Sukthankar, editors, Computational Theory of Mind for Human-Machine Teams, pages 158–172, Cham, 2022. Springer Nature Switzerland.
  26. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50–60, may 2020.
  27. Federated optimization in heterogeneous networks, 2020.
  28. Fedbn: Federated learning on non-iid features via local batch normalization, 2021.
  29. Learning without forgetting, 2017.
  30. Modular federated learning. In 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, jul 2022.
  31. Gradient episodic memory for continual learning, 2022.
  32. Towards heterogeneous clients with elastic federated learning, 2021.
  33. Domain adaptation: Learning bounds and algorithms, 2023.
  34. Catastrophic interference in connectionist networks: The sequential learning problem. volume 24 of Psychology of Learning and Motivation, pages 109–165. Academic Press, 1989.
  35. Communication-efficient learning of deep networks from decentralized data, 2023.
  36. Lifelong learning of compositional structures, 2021.
  37. How to reuse and compose knowledge for a lifetime of tasks: A survey on continual learning and functional composition, 2023.
  38. Composuite: A compositional reinforcement learning benchmark, 2022.
  39. Modular lifelong reinforcement learning via neural composition, 2022.
  40. Beyond shared hierarchies: Deep multitask learning through soft layer ordering, 2018.
  41. Collaborative learning through shared collective knowledge and local expertise. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6, 2019.
  42. Sharing lifelong reinforcement learning knowledge via modulating masks, 2023.
  43. Leep: A new measure to evaluate transferability of learned representations, 2020.
  44. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019.
  45. Benedetto Piccoli. Control of multi-agent systems: results, open problems, and applications, 2023.
  46. Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123–146, 1995.
  47. Multi-agent distributed lifelong learning for collective knowledge acquisition, 2018.
  48. Fetchsgd: Communication-efficient federated learning with sketching, 2020.
  49. Progressive neural networks, 2022.
  50. ELLA: An efficient lifelong learning algorithm. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 507–515, Atlanta, Georgia, USA, 17–19 Jun 2013. PMLR.
  51. Progress & compress: A scalable framework for continual learning, 2018.
  52. E. Semsar-Kazerooni and K. Khorasani. A game theory approach to multi-agent team cooperation. In 2009 American Control Conference, pages 4512–4518, 2009.
  53. Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison, 2009.
  54. Asynchronous federated continual learning, 2023.
  55. Overcoming forgetting in federated learning on non-iid data, 2019.
  56. A collective ai via lifelong learning and sharing at the edge. Nature Machine Intelligence, 6(3):251–264, Mar 2024.
  57. Sebastian U. Stich. Local sgd converges fast and communicates little, 2019.
  58. Multiagent systems: A survey from a machine learning perspective. Autonomous Robots, 8(3):345–383, Jun 2000.
  59. Decentralized federated averaging, 2021.
  60. Otce: A transferability metric for cross-domain cross-task representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15779–15788, June 2021.
  61. Gido M. van de Ven and Andreas S. Tolias. Three scenarios for continual learning. CoRR, abs/1904.07734, 2019.
  62. Powergossip: Practical low-rank communication compression in decentralized deep learning, 2020.
  63. Personalized federated learning via heterogeneous modular networks, 2022.
  64. Federated continual learning with weighted inter-client transfer, 2021.
  65. Scalable and order-robust continual learning with additive parameter decomposition, 2020.
  66. Lifelong learning with dynamically expandable networks, 2018.
  67. Continual learning through synaptic intelligence, 2017.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now