Papers
Topics
Authors
Recent
Search
2000 character limit reached

Computation and Communication Efficient Lightweighting Vertical Federated Learning for Smart Building IoT

Published 30 Mar 2024 in cs.LG and cs.DC | (2404.00466v2)

Abstract: With the increasing number and enhanced capabilities of IoT devices in smart buildings, these devices are evolving beyond basic data collection and control to actively participate in deep learning tasks. Federated Learning (FL), as a decentralized learning paradigm, is well-suited for such scenarios. However, the limited computational and communication resources of IoT devices present significant challenges. While existing research has extensively explored efficiency improvements in Horizontal FL, these techniques cannot be directly applied to Vertical FL due to fundamental differences in data partitioning and model structure. To address this gap, we propose a Lightweight Vertical Federated Learning (LVFL) framework that jointly optimizes computational and communication efficiency. Our approach introduces two distinct lightweighting strategies: one for reducing the complexity of the feature model to improve local computation, and another for compressing feature embeddings to reduce communication overhead. Furthermore, we derive a convergence bound for the proposed LVFL algorithm that explicitly incorporates both computation and communication lightweighting ratios. Experimental results on an image classification task demonstrate that LVFL effectively mitigates resource demands while maintaining competitive learning performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. Y. Liu, T. Fan, T. Chen, Q. Xu, and Q. Yang, “Fate: An industrial grade platform for collaborative learning with data protection,” Journal of Machine Learning Research, vol. 22, no. 226, pp. 1–6, 2021.
  2. T. J. Castiglia, A. Das, S. Wang, and S. Patterson, “Compressed-vfl: Communication-efficient learning with vertically partitioned data,” in International Conference on Machine Learning.   PMLR, 2022, pp. 2738–2766.
  3. H. Wang and J. Xu, “Online vertical federated learning for cooperative spectrum sensing,” arXiv preprint arXiv:2312.11363, 2023.
  4. S. Hardy, W. Henecka, H. Ivey-Law, R. Nock, G. Patrini, G. Smith, and B. Thorne, “Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption,” arXiv preprint arXiv:1711.10677, 2017.
  5. L. Yang, D. Chai, J. Zhang, Y. Jin, L. Wang, H. Liu, H. Tian, Q. Xu, and K. Chen, “A survey on vertical federated learning: From a layered perspective,” arXiv preprint arXiv:2304.01829, 2023.
  6. K. Wei, J. Li, C. Ma, M. Ding, S. Wei, F. Wu, G. Chen, and T. Ranbaduge, “Vertical federated learning: Challenges, methodologies and experiments,” arXiv preprint arXiv:2202.04309, 2022.
  7. Y. Liu, Y. Kang, T. Zou, Y. Pu, Y. He, X. Ye, Y. Ouyang, Y.-Q. Zhang, and Q. Yang, “Vertical federated learning,” arXiv preprint arXiv:2211.12814, 2022.
  8. S. Feng, “Vertical federated learning-based feature selection with non-overlapping sample utilization,” Expert Systems with Applications, vol. 208, p. 118097, 2022.
  9. Y. Kang, Y. Liu, and X. Liang, “Fedcvt: Semi-supervised vertical federated learning with cross-view training,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 4, pp. 1–16, 2022.
  10. J. Sun, Y. Yao, W. Gao, J. Xie, and C. Wang, “Defending against reconstruction attack in vertical federated learning,” arXiv preprint arXiv:2107.09898, 2021.
  11. Y. Liu, X. Zhang, Y. Kang, L. Li, T. Chen, M. Hong, and Q. Yang, “Fedbcd: A communication-efficient collaborative learning framework for distributed features,” IEEE Transactions on Signal Processing, vol. 70, pp. 4277–4290, 2022.
  12. T. Castiglia, S. Wang, and S. Patterson, “Flexible vertical federated learning with heterogeneous parties,” arXiv preprint arXiv:2208.12672, 2022.
  13. M. Li, Y. Chen, Y. Wang, and Y. Pan, “Efficient asynchronous vertical federated learning via gradient prediction and double-end sparse compression,” in 2020 16th international conference on control, automation, robotics and vision (ICARCV).   IEEE, 2020, pp. 291–296.
  14. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  15. Z. Wei, Q. Pei, N. Zhang, X. Liu, C. Wu, and A. Taherkordi, “Lightweight federated learning for large-scale iot devices with privacy guarantee,” IEEE Internet of Things Journal, vol. 10, no. 4, pp. 3179–3191, 2021.
  16. X. Dong, S. Chen, and S. Pan, “Learning to prune deep neural networks via layer-wise optimal brain surgeon,” Advances in neural information processing systems, vol. 30, 2017.
  17. V. Sanh, T. Wolf, and A. Rush, “Movement pruning: Adaptive sparsity by fine-tuning,” Advances in Neural Information Processing Systems, vol. 33, pp. 20 378–20 389, 2020.
  18. X. Ding, G. Ding, Y. Guo, and J. Han, “Centripetal sgd for pruning very deep convolutional networks with complicated structure,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4943–4953.
  19. H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” arXiv preprint arXiv:1608.08710, 2016.
  20. Z. You, K. Yan, J. Ye, M. Ma, and P. Wang, “Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks,” Advances in neural information processing systems, vol. 32, 2019.
  21. S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” Advances in neural information processing systems, vol. 28, 2015.
  22. S. Han, X. Liu, H. Mao, J. Pu, A. Pedram, M. A. Horowitz, and W. J. Dally, “Eie: Efficient inference engine on compressed deep neural network,” ACM SIGARCH Computer Architecture News, vol. 44, no. 3, pp. 243–254, 2016.
  23. Y. Jiang, S. Wang, V. Valls, B. J. Ko, W.-H. Lee, K. K. Leung, and L. Tassiulas, “Model pruning enables efficient federated learning on edge devices,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  24. Z. Jiang, Y. Xu, H. Xu, Z. Wang, J. Liu, Q. Chen, and C. Qiao, “Computation and communication efficient federated learning with adaptive model pruning,” IEEE Transactions on Mobile Computing, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Tweets

Sign up for free to view the 2 tweets with 1 like about this paper.

Sign Up for Free