Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SplitMAC: Wireless Split Learning over Multiple Access Channels (2311.02405v2)

Published 4 Nov 2023 in cs.IT, eess.SP, and math.IT

Abstract: This paper presents a novel split learning (SL) framework, referred to as SplitMAC, which reduces the latency of SL by leveraging simultaneous uplink transmission over multiple access channels. The key strategy is to divide devices into multiple groups and allow the devices within the same group to simultaneously transmit their smashed data and device-side models over the multiple access channels. The optimization problem of device grouping to minimize SL latency is formulated, and the benefit of device grouping in reducing the uplink latency of SL is theoretically derived. By examining a two-device grouping case, two asymptotically-optimal algorithms are devised for device grouping in low and high signal-to-noise ratio (SNR) scenarios, respectively, while providing proofs of their optimality. By merging these algorithms, a near-optimal device grouping algorithm is proposed to cover a wide range of SNR. Our SL framework is also extended to consider practical fading channels and to support a general group size. Simulation results demonstrate that our SL framework with the proposed device grouping algorithm is superior to existing SL frameworks in reducing SL latency.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. J. Konečnỳ, B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacente,” 2015, arXiv:1511.03575.
  2. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. Int. Conf. Artificial Intell. Statist. (AISTATS), Fort Lauderdale, FL, USA, Apr. 2017, pp. 1273–1282.
  3. S. Niknam, H. S. Dhillon, and J. H. Reed, “Federated learning for wireless communications: Motivation, opportunities, and challenges,” IEEE Commun. Mag., vol. 58, no. 6, pp. 46–51, Jun. 2020.
  4. Y.-S. Jeon, M. M. Amiri, J. Li, and H. V. Poor, “A compressive sensing approach for federated learning over massive MIMO communication systems,” IEEE Trans. Wireless Commun., vol. 20, no. 3, pp. 1990–2004, Mar. 2021.
  5. Y. Oh, N. Lee, Y.-S. Jeon, and H. V. Poor, “Communication-efficient federated learning via quantized compressed sensing,” IEEE Trans. Wireless Commun., vol. 22, no. 2, pp. 1087–1100, Feb. 2023.
  6. O. Gupta and R. Raskar, “Distributed learning of deep neural network over multiple agents,” J. Netw. Comput. Appl., vol. 116, pp. 1–8, Aug. 2018.
  7. P. Vepakomma, O. Gupta, T. Swedish, and R. Raskar, “Split learning for health: Distributed deep learning without sharing raw patient data,” in Proc. Int. Conf. Learn. Represent. (ICLR) Workshop AI Social Good, New Orleans, LA, USA, May 2019, pp. 1–7.
  8. K. Wei, J. Li, C. Ma, M. Ding, S. Wei, F. Wu, G. Chen, and T. Ranbaduge, “Vertical federated learning: Challenges, methodologies and experiments,” 2022, arXiv:2202.04309.
  9. K. B. Letaief, Y. Shi, J. Lu, and J. Lu, “Edge artificial intelligence for 6g: Vision, enabling technologies, and applications,” IEEE J. Sel. Areas Commun., vol. 40, no. 1, pp. 5–36, Jan. 2022.
  10. N.-P. Tran, N.-N. Dao, T.-V. Nguyen, and S. Cho, “Privacy-preserving learning models for communication: A tutorial on advanced split learning,” in Proc. Int. Conf. Inf. Commun. Technol. Convergence (ICTC), Jeju Island, Republic of Korea, Oct. 2022, pp. 1059–1064.
  11. A. Singh, P. Vepakomma, O. Gupta, and R. Raskar, “Detailed comparison of communication efficiency of split learning and federated learning,” 2019, arXiv:1909.09145.
  12. Y. Li and X. Lyu, “Convergence analysis of sequential federated learning on heterogeneous data,” in Adv. Neural Inf. Process. Syst. (NeurIPS), New Orleans, LA, USA, Dec. 2023, pp. 56 700–56 755.
  13. C. Shiranthika, P. Saeedi, and I. V. Bajić, “Decentralized learning in healthcare: A review of emerging techniques,” IEEE Access, vol. 11, pp. 54,188–54,209, Jun. 2023.
  14. B. Yuan, S. Ge, and W. Xing, “A federated learning framework for healthcare IoT devices,” 2020, arXiv:2005.05083.
  15. J. Wang, H. Qi, A. S. Rawat, S. Reddi, S. Waghmare, F. X. Yu, and G. Joshi, “FedLite: A scalable approach for federated learning on resource-constrained clients,” 2022, arXiv:2201.11865.
  16. Y. Oh, J. Lee, C. G. Brinton, and Y.-S. Jeon, “Communication-efficient split learning via adaptive feature-wise compression,” 2023, arXiv:2307.10805.
  17. C. Thapa, P. C. M. Arachchige, S. Camtepe, and L. Sun, “SplitFed: When federated learning meets split learning,” in Proc. AAAI Conf. Artif. Intell., Jun. 2022, pp. 8485–8493.
  18. W. Wu, M. Li, K. Qu, C. Zhou, X. Shen, W. Zhuang, X. Li, and W. Shi, “Split learning over wireless networks: Parallel design and resource management,” IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 1051–1066, Apr. 2023.
  19. Z. Lin, G. Zhu, Y. Deng, X. Chen, Y. Gao, K. Huang, and Y. Fang, “Efficient parallel split learning over resource-constrained wireless edge networks,” 2023, arXiv:2303.15991.
  20. X. Chen, Z. Zhang, C. Zhong, and D. W. K. Ng, “Exploiting multiple-antenna techniques for non-orthogonal multiple access,” IEEE J. Sel. Areas Commun., vol. 35, no. 10, pp. 2207–2220, Jul. 2017.
  21. M. Vaezi, G. A. A. Baduge, Y. Liu, A. Arafa, F. Fang, and Z. Ding, “Interplay between NOMA and other emerging technologies: A survey,” IEEE Trans. Cogn. Commun. Netw., vol. 5, no. 4, pp. 900–919, Aug. 2019.
  22. Z. Wei, L. Yang, D. W. K. Ng, J. Yuan, and L. Hanzo, “On the performance gain of NOMA over OMA in uplink communication systems,” IEEE Trans. Wireless Commun., vol. 68, no. 1, pp. 536–568, Oct. 2019.
  23. Z. Chen, Z. Ding, X. Dai, and R. Zhang, “An optimization perspective of the superiority of NOMA compared to conventional OMA,” IEEE Trans. Signal Process., vol. 65, no. 17, pp. 5191–5202, 2017.
  24. Z. Song, Y. Liu, and X. Sun, “Joint task offloading and resource allocation for NOMA-enabled multi-access mobile edge computing,” IEEE Trans. Commun., vol. 69, no. 3, pp. 1548–1564, 2021.
  25. V. D. Tuong, W. Noh, and S. Cho, “Delay minimization for NOMA-enabled mobile edge computing in industrial internet of things,” IEEE Trans. Ind. Informat., vol. 18, no. 10, pp. 7321–7331, Oct. 2021.
  26. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
  27. A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” M.S. thesis, Univ. Toronto, Toronto, ON, Canada, 2009.
  28. P. Xiao, S. Cheng, V. Stankovic, and D. Vukobratovic, “Averaging is probably not the optimum way of aggregating parameters in federated learning,” Entropy, vol. 22, no. 3, p. 314, 2020.
  29. Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” 2018, arXiv:1806.00582.
  30. T. Takeda and K. Higuchi, “Enhanced user fairness using non-orthogonal access with sic in cellular uplink,” in Proc. IEEE Veh. Technol. Conf. (VTC), San Francisco, CA, USA, Sep. 2011, pp. 1–5.
  31. V. Felbab, P. Kiss, and T. Horváth, “Optimization in federated learning,” in Proc. 19th Conf. Inf. Technol.–Appl. Theory (ITAT), Donovaly, Slovakia, Sep 2019, pp. 58–65.
  32. M. A. Sedaghat and R. R. Müller, “On user pairing in uplink NOMA,” IEEE Trans. Wireless Commun., vol. 17, no. 5, pp. 3474–3486, Mar. 2018.
  33. T. David and P. Viswanath, “Fundamentals of Wireless Communication,” Univ. Cambridge, Cambridge, Press, U.K., 2005.
  34. J. Zhang, M. You, G. Zheng, I. Krikidis, and L. Zhao, “Model-driven learning for generic MIMO downlink beamforming with uplink channel information,” IEEE Trans. Wireless Commun., vol. 21, no. 4, pp. 2368–2382, Apr. 2022.
  35. X. Hu, L. Wang, K.-K. Wong, M. Tao, Y. Zhang, and Z. Zheng, “Edge and central cloud computing: A perfect pairing for high energy efficiency and low-latency,” IEEE Trans. Wireless Commun., vol. 19, no. 2, pp. 1070–1083, Feb. 2019.
  36. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov 1998.
  37. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, May 2015, pp. 1–14.
  38. Q. Li, Y. Diao, Q. Chen, and B. He, “Federated learning on non-iid data silos: An experimental study,” in Proc. IEEE Int. Conf. Data Eng. (ICDE), Kuala Lumpur, Malaysia, May 2022, pp. 965–978.
  39. D. Wen, P. Liu, G. Zhu, Y. Shi, J. Xu, Y. C. Eldar, and S. Cui, “Task-oriented sensing, computation, and communication integration for multi-device edge AI,” IEEE Trans. Wireless Commun., early access, Aug. 14, 2023, doi: 10.1109/TWC.2023.3303232.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets