Papers
Topics
Authors
Recent
Search
2000 character limit reached

Model CBOR Serialization for Federated Learning

Published 25 Jan 2024 in cs.NI | (2401.14056v1)

Abstract: The typical federated learning workflow requires communication between a central server and a large set of clients synchronizing model parameters between each other. The current frameworks use communication protocols not suitable for resource-constrained devices and are either hard to deploy or require high-throughput links not available on these devices. In this paper, we present a generic message framework using CBOR for communication with existing federated learning frameworks optimised for use with resource-constrained devices and low power and lossy network links. We evaluate the resulting message sizes against JSON serialized messages where compare both with model parameters resulting in optimal and worst case serialization length, and with a real-world LeNet-5 model. Our benchmarks show that with our approach, messages are up to 75 % smaller in size when compared to the JSON alternative.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. S. S. Saha, S. S. Sandha, and M. Srivastava, “Machine learning for microcontroller-class hardware-a review,” IEEE Sensors Journal, 2022.
  2. “TensorFlow Federated.” [Online]. Available: https://www.tensorflow.org/federated
  3. D. J. Beutel et al., “Flower: A friendly federated learning framework,” 2022.
  4. E. International, “Ecma-404—the json data interchange format,” 2013.
  5. “gRPC.” [Online]. Available: https://grpc.io/
  6. “Protocol buffers.” [Online]. Available: https://protobuf.dev/
  7. I. Fette and A. Melnikov, “The websocket protocol,” RFC 6455, 2011.
  8. U. Hunkeler et al., “Mqtt-s - a publish/subscribe protocol for wireless sensor networks.” in COMSWARE, S. Choi, J. Kurose, and K. Ramamritham, Eds.   IEEE, 2008, pp. 791–798.
  9. A. Stanford-Clark and H. L. Truong, “Mqtt for sensor networks (mqtt-sn) protocol specification,” IBM Corporation version, vol. 1, no. 2, pp. 1–28, 2013.
  10. Z. Shelby, K. Hartke, and C. Bormann, “The Constrained Application Protocol (CoAP),” RFC 7252, Jun. 2014. [Online]. Available: https://www.rfc-editor.org/info/rfc7252
  11. C. Bormann and P. E. Hoffman, “Concise Binary Object Representation (CBOR),” RFC 8949, Dec. 2020. [Online]. Available: https://www.rfc-editor.org/info/rfc8949
  12. M. M. Grau et al., “On-device training of machine learning models on microcontrollers with a look at federated learning,” in Proceedings ACM GoodIT, 2021, p. 198–203.
  13. K. Kopparapu and E. Lin, “Tinyfedtl: Federated transfer learning on tiny devices,” arXiv preprint arXiv:2110.01107, 2021.
  14. P. Pinyoanuntapong et al., “Edgeml: Towards network-accelerated federated learning over wireless edge,” Computer Networks, vol. 219, p. 109396, 2022.
  15. B. McMahan et al., “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  16. J. Konečnỳ et al., “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016.
  17. P. Kairouz et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
  18. “Ieee standard for low-rate wireless networks,” IEEE Std 802.15.4-2020, pp. 1–800, 2020.
  19. B. Moran et al., “A Firmware Update Architecture for Internet of Things,” RFC 9019, Apr. 2021. [Online]. Available: https://www.rfc-editor.org/info/rfc9019
  20. C. F. Jennings et al., “Sensor Measurement Lists (SenML),” RFC 8428, Aug. 2018. [Online]. Available: https://www.rfc-editor.org/info/rfc8428
  21. H. Birkholz, C. Vigano, and C. Bormann, “Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures,” RFC 8610, Jun. 2019. [Online]. Available: https://www.rfc-editor.org/info/rfc8610
  22. P. Thubert, “An Architecture for IPv6 over the Time-Slotted Channel Hopping Mode of IEEE 802.15.4 (6TiSCH),” RFC 9030, May 2021. [Online]. Available: https://www.rfc-editor.org/info/rfc9030
  23. M. Veillette et al., “CoAP Management Interface (CORECONF),” Internet Engineering Task Force, Internet-Draft draft-ietf-core-comi-12, Mar. 2023. [Online]. Available: https://datatracker.ietf.org/doc/draft-ietf-core-comi/12/
  24. C. Bormann, “Concise Binary Object Representation (CBOR) Sequences,” RFC 8742, Feb. 2020. [Online]. Available: https://www.rfc-editor.org/info/rfc8742
  25. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
Citations (1)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.