DISTINQT: A Distributed Privacy Aware Learning Framework for QoS Prediction for Future Mobile and Wireless Networks (2401.10158v3)
Abstract: Beyond 5G and 6G networks are expected to support new and challenging use cases and applications that depend on a certain level of Quality of Service (QoS) to operate smoothly. Predicting the QoS in a timely manner is of high importance, especially for safety-critical applications as in the case of vehicular communications. Although until recent years the QoS prediction has been carried out by centralized AI solutions, a number of privacy, computational, and operational concerns have emerged. Alternative solutions have surfaced (e.g. Split Learning, Federated Learning), distributing AI tasks of reduced complexity across nodes, while preserving the privacy of the data. However, new challenges rise when it comes to scalable distributed learning approaches, taking into account the heterogeneous nature of future wireless networks. The current work proposes DISTINQT, a novel multi-headed input privacy-aware distributed learning framework for QoS prediction. Our framework supports multiple heterogeneous nodes, in terms of data types and model architectures, by sharing computations across them. This enables the incorporation of diverse knowledge into a sole learning process that will enhance the robustness and generalization capabilities of the final QoS prediction model. DISTINQT also contributes to data privacy preservation by encoding any raw input data into highly complex, compressed, and irreversible latent representations before any transmission. Evaluation results showcase that DISTINQT achieves a statistically identical performance compared to its centralized version, while also proving the validity of the privacy preserving claims. DISTINQT manages to achieve a reduction in prediction error of up to 65% on average against six state-of-the-art centralized baseline solutions presented in the Tele-Operated Driving use case.
- W. Jiang, B. Han, M. A. Habibi, and H. D. Schotten, “The road towards 6g: A comprehensive survey,” IEEE Open Journal of the Communications Society, vol. 2, pp. 334–366, 2021.
- M. Moussaoui, E. Bertin, and N. Crespi, “5g shortcomings and beyond-5g/6g requirements,” in 2022 1st International Conference on 6G Networking (6GNet), 2022, pp. 1–8.
- one6G, “6g vertical use cases,” Tech. Rep., June 2022. [Online]. Available: https://one6g.org/download/2027/
- 5GAA, “Making 5G Proactive and Predictive for the Automotive Industry,” White Paper, November 2019.
- A. Kaloxylos, A. Gavras, D. Camps Mur, M. Ghoraishi, and H. Hrasnica, “Ai and ml – enablers for beyond 5g networks,” Dec. 2020. [Online]. Available: https://doi.org/10.5281/zenodo.4299895
- R. Zhohov, A. Palaios, and P. Geuer, “One step further: Tunable and explainable throughput prediction based on large-scale commercial networks,” in 2021 IEEE 4th 5G World Forum (5GWF), 2021, pp. 430–435.
- D. Minovski, N. Ogren, C. Ahlund, and K. Mitra, “Throughput prediction using machine learning in lte and 5g networks,” IEEE Transactions on Mobile Computing, pp. 1–1, 2021.
- B. Sliwa, H. Schippers, and C. Wietfeld, “Machine learning-enabled data rate prediction for 5g nsa vehicle-to-cloud communications,” in 2021 IEEE 4th 5G World Forum (5GWF), 2021, pp. 299–304.
- M. A. Gutierrez-Estevez, Z. Utkovski, A. Kousaridas, and C. Zhou, “A statistical learning framework for qos prediction in v2x,” in 2021 IEEE 4th 5G World Forum (5GWF), 2021, pp. 441–446.
- A. Kousaridas, R. P. Manjunath, J. Perdomo, C. Zhou, E. Zielinski, S. Schmitz, and A. Pfadler, “Qos prediction for 5g connected and automated driving,” IEEE Communications Magazine, vol. 59, no. 9, pp. 58–64, 2021.
- A. Palaios, C. Vielhaus, D. F. Külzer, P. Geuer, R. Sattiraju, J. Fink, M. Kasparick, C. Watermann, G. Fettweis, F. H. P. Fitzek, H. D. Schotten, and S. Stańczak, “Effect of spatial, temporal and network features on uplink and downlink throughput prediction,” in 2021 IEEE 4th 5G World Forum (5GWF), 2021, pp. 418–423.
- one6G, “6g technology overview,” Tech. Rep. Third Edition, October 2023. [Online]. Available: https://one6g.org/download/3921/?tmstv=1697099550
- O. Nassef, W. Sun, H. Purmehdi, M. Tatipamula, and T. Mahmoodi, “A survey: Distributed machine learning for 5g and beyond,” Computer Networks, vol. 207, p. 108820, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1389128622000421
- 3GPP, “3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Study of Enablers for Network Automation for the 5G System (5GS); Phase 3 (Release 18),” Technical Report 3GPP TR 23.700-81, May 2022.
- ETSI, “5G; Architecture enhancements for 5G System (5GS) to support network data analytics services (3GPP TS 23.288 version 17.4.0 Release 17),” Technical Specification ETSI TS 123 288 V17.4.0, May 2022.
- 3GPP, “3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Study on AI/ML Model Transfer Phase 2 (Release 19),” Technical Report 3GPP TR 22.876 V19.1.0, September 2023.
- Test case definition and test site description part 1. [Online]. Available: https://5gcroco.eu/images/templates/rsvario/images/5GCroCo_D2_1.pdf
- Y. Zhang, X. Zhang, and X. Li, “Towards efficient and privacy-preserving service qos prediction with federated learning,” in Collaborative Computing: Networking, Applications and Worksharing, H. Gao, X. Wang, M. Iqbal, Y. Yin, J. Yin, and N. Gu, Eds. Cham: Springer International Publishing, 2021, pp. 37–57.
- J. Xu, J. Lin, Y. Li, and Z. Xu, “Multifed: A fast converging federated learning framework for services qos prediction via cloud–edge collaboration mechanism,” Knowledge-Based Systems, vol. 268, p. 110463, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950705123002137
- X. Li, S. Li, Y. Li, Y. Zhou, C. Chen, and Z. Zheng, “A personalized federated tensor factorization framework for distributed iot services qos prediction from heterogeneous data,” IEEE Internet of Things Journal, vol. 9, no. 24, pp. 25 460–25 473, 2022.
- I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, Eds., vol. 27. Curran Associates, Inc., 2014.
- L. Magoula, N. Koursioumpas, S. Barmpounakis, P. Kontopoulos, M. A. Gutierrez-Estevez, R. Khalili, and A. Kousaridas, “A deep learning approach for distributed qos prediction in beyond 5g networks,” in 2022 IEEE 33rd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 2022, pp. 1407–1412.
- Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” vol. 10, no. 2, jan 2019. [Online]. Available: https://doi.org/10.1145/3298981
- F. Z. Errounda and Y. Liu, “A mobility forecasting framework with vertical federated learning,” in 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), 2022, pp. 301–310.
- J. Ryu, D. Won, and Y. Lee, “A study of split learning model,” in 2022 16th International Conference on Ubiquitous Information Management and Communication (IMCOM), 2022, pp. 1–4.
- J. Jeon and J. Kim, “Privacy-sensitive parallel split learning,” in 2020 International Conference on Information Networking (ICOIN), 2020, pp. 7–9.
- J. Kim, S. Park, S. Jung, and S. Yoo, “Spatio-temporal split learning,” 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S), pp. 11–12, 2021.
- Q. Sun, J. Xu, X. Ma, A. Zhou, C.-H. Hsu, and S. Wang, “Edge-enabled distributed deep learning for 5g privacy protection,” IEEE Network, vol. 35, no. 4, pp. 213–219, 2021.
- D. Saguil and A. Azim, “A layer-partitioning approach for faster execution of neural network-based embedded applications in edge networks,” IEEE Access, vol. 8, pp. 59 456–59 469, 2020.
- T. Mohammed, C. Joe-Wong, R. Babbar, and M. D. Francesco, “Distributed inference acceleration with adaptive dnn partitioning and offloading,” in IEEE INFOCOM 2020 - IEEE Conference on Computer Communications, 2020, pp. 854–863.
- Z. Xu, L. Zhao, W. Liang, O. F. Rana, P. Zhou, Q. Xia, W. Xu, and G. Wu, “Energy-aware inference offloading for dnn-driven applications in mobile edge clouds,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 4, pp. 799–814, 2021.
- L. Zeng, X. Chen, Z. Zhou, L. Yang, and J. Zhang, “Coedge: Cooperative dnn inference with adaptive workload partitioning over heterogeneous edge devices,” IEEE/ACM Transactions on Networking, vol. 29, no. 2, pp. 595–608, 2021.
- J. Du, X. Zhu, M. Shen, Y. Du, Y. Lu, N. Xiao, and X. Liao, “Model parallelism optimization for distributed inference via decoupled cnn structure,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 7, pp. 1665–1676, 2021.
- Z. Hu, A. B. Tarakji, V. Raheja, C. Phillips, T. Wang, and I. Mohomed, “Deephome: Distributed inference with heterogeneous devices in the edge,” in The 3rd International Workshop on Deep Learning for Mobile Systems and Applications, ser. EMDL ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 13–18. [Online]. Available: https://doi.org/10.1145/3325413.3329787
- J. Zhai, S. Zhang, J. Chen, and Q. He, “Autoencoder and its various variants,” in 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2018, pp. 415–419.
- N. Phan, Y. Wang, X. Wu, and D. Dou, “Differential privacy preservation for deep auto-encoders: An application of human behavior prediction,” in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, ser. AAAI’16. AAAI Press, 2016, p. 1309–1316.
- H. C. Tanuwidjaja, R. Choi, S. Baek, and K. Kim, “Privacy-preserving deep learning on machine learning as a service—a comprehensive survey,” IEEE Access, vol. 8, pp. 167 425–167 447, 2020.
- R. M. Alguliyev, R. M. Aliguliyev, and F. J. Abdullayeva, “Privacy-preserving deep learning algorithm for big personal data analysis,” Journal of Industrial Information Integration, vol. 15, pp. 1–14, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2452414X18301456
- K. Bayoudh, R. Knani, F. Hamdaoui, and A. Mtibaa, “A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets,” The Visual Computer, vol. 38, no. 8, pp. 2939–2970, Aug 2022. [Online]. Available: https://doi.org/10.1007/s00371-021-02166-7
- “Deep neural networks compression: A comparative survey and choice recommendations,” Neurocomputing, vol. 520, pp. 152–170, 2023.
- 5GAA, “C-V2X Use Cases, Methodology, Examples and Service Level Requirements,” White Paper, July 2019.
- Official ns-3 website. [Online]. Available: https://www.nsnam.org/
- 3rd Generation Partnership Project (3GPP), “Study on Scenarios and Requirements for Next Generation Access Technologies (Rel. 16),” Technical Report (TR) 38.913, July 2020.
- 3rd Generation Partnership Project (3GPP), “Study on evaluation methodology of new Vehicle-to-Everything (V2X) use cases for LTE and NR (Rel. 15),” Technical Report (TR) 37.885, June 2019.
- G. A. A. (5GAA), “Tele-operated Driving Use Cases, System Architecture and Business Considerations,” White Paper, December 2021.
- Simulation of urban mobility. [Online]. Available: https://eclipse.dev/sumo/
- S. Barmpounakis, L. Magoula, N. Koursioumpas, R. Khalili, J. M. Perdomo, and R. P. Manjunath, “Lstm-based qos prediction for 5g-enabled connected and automated mobility applications,” in 2021 IEEE 4th 5G World Forum (5GWF), 2021, pp. 436–440.
- R. Madan and P. S. Mangipudi, “Predicting computer network traffic: A time series forecasting approach using dwt, arima and rnn,” in 2018 Eleventh International Conference on Contemporary Computing (IC3), 2018, pp. 1–5.
- S. Barmpounakis, N. Maroulis, N. Koursioumpas, A. Kousaridas, A. Kalamari, P. Kontopoulos, and N. Alonistioti, “Ai-driven, qos prediction for v2x communications in beyond 5g systems,” Computer Networks, vol. 217, p. 109341, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1389128622003759