Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FlagVNE: A Flexible and Generalizable Reinforcement Learning Framework for Network Resource Allocation (2404.12633v4)

Published 19 Apr 2024 in cs.AI and cs.NI

Abstract: Virtual network embedding (VNE) is an essential resource allocation task in network virtualization, aiming to map virtual network requests (VNRs) onto physical infrastructure. Reinforcement learning (RL) has recently emerged as a promising solution to this problem. However, existing RL-based VNE methods are limited by the unidirectional action design and one-size-fits-all training strategy, resulting in restricted searchability and generalizability. In this paper, we propose a FLexible And Generalizable RL framework for VNE, named FlagVNE. Specifically, we design a bidirectional action-based Markov decision process model that enables the joint selection of virtual and physical nodes, thus improving the exploration flexibility of solution space. To tackle the expansive and dynamic action space, we design a hierarchical decoder to generate adaptive action probability distributions and ensure high training efficiency. Furthermore, to overcome the generalization issue for varying VNR sizes, we propose a meta-RL-based training method with a curriculum scheduling strategy, facilitating specialized policy training for each VNR size. Finally, extensive experimental results show the effectiveness of FlagVNE across multiple key metrics. Our code is available at GitHub (https://github.com/GeminiLight/flag-vne).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. learn2learn: A library for meta-learning research. arXiv preprint arXiv:2008.12284, 2020.
  2. NeuroViNE: A neural preprocessor for your virtual network embedding algorithm. In IEEE International Conference on Computer Communications, pages 405–413, 2018.
  3. Efficient join order selection learning with graph-based representation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 97–107, 2022.
  4. DYVINE: Fitness-based dynamic virtual network embedding in cloud computing. IEEE Journal on Selected Areas in Communications, 37(5):1029–1045, 2019.
  5. Node essentiality assessment and distributed collaborative virtual network embedding in datacenters. IEEE Transactions on Parallel and Distributed Systems, 34(4):1265–1280, 2023.
  6. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1126–1135, 2017.
  7. Virtual network embedding: A survey. IEEE Communications Surveys Tutorials, 15(4):1888–1906, 2013.
  8. Gal-vne: Solving the vne problem with global reinforcement learning and local one-shot neural prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, page 531–543, 2023.
  9. Virtual network embedding via Monte Carlo tree search. IEEE Transactions on Cybernetics, 48(2):510–521, 2017.
  10. Leveraging deep reinforcement learning with attention mechanism for virtual network function placement and routing. IEEE Transactions on Parallel and Distributed Systems, 34(4):1186–1201, 2023.
  11. The 37 implementation details of proximal policy optimization. In ICLR Blog Track, 2022.
  12. VNE-HPSO Virtual Network Embedding Algorithm Based on Hybrid Particle Swarm Optimization, pages 129–152. Springer Singapore, Singapore, 2021.
  13. Latency-aware VNF chain deployment with efficient resource reuse at network edge. In IEEE International Conference on Computer Communications, pages 267–276, 2020.
  14. Learning the travelling salesperson problem requires rethinking generalization. Constraints, 27(1-2):70–98, 2022.
  15. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  16. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
  17. Impact-aware maneuver decision with enhanced perception for autonomous vehicle. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 3255–3268, 2023.
  18. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, pages 1928–1937, 2016.
  19. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
  20. On the hardness and inapproximability of virtual network embeddings. IEEE/ACM Transactions on Networking, 28(2):791–803, 2020.
  21. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  22. Virtual network survivability through joint spare capacity allocation and embedding. IEEE Journal on Selected Areas in Communications, 36(3):502–518, 2018.
  23. Energy-aware virtual network embedding. IEEE/ACM Transactions on Networking, 22(5):1607–1620, 2014.
  24. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018.
  25. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 12, 1999.
  26. Variable interval time sequence modeling for career trajectory prediction: Deep collaborative perspective. In Proceedings of the Web Conference 2021, pages 612–623, 2021.
  27. Personalized and explainable employee training course recommendations: A bayesian variational approach. ACM Transactions on Information Systems (TOIS), 40(4):1–32, 2021.
  28. DRL-SFCP: Adaptive service function chains placement with deep reinforcement learning. In IEEE International Conference on Communications, pages 1–6, 2021.
  29. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):4555–4576, 2021.
  30. Setrank: A setwise bayesian approach for collaborative ranking in recommender system. ACM Transactions on Information Systems, 42(2):1–32, 2023.
  31. Joint admission control and resource allocation of virtual network embedding via hierarchical deep reinforcement learning. IEEE Transactions on Services Computing, pages 1–14, 2023.
  32. B.M. Waxman. Routing of multipoint connections. IEEE Journal on Selected Areas in Communications, 6(9):1617–1622, 1988.
  33. Nfvdeep: Adaptive online service function chain deployment with deep reinforcement learning. In IEEE/ACM International Symposium on Quality of Service, 2019.
  34. Automatic virtual network embedding: A deep reinforcement learning approach with graph convolutional networks. IEEE Journal on Selected Areas in Communications, 38(6):1040–1057, 2020.
  35. Recent advances of resource allocation in network function virtualization. IEEE Transactions on Parallel and Distributed Systems, 32(2):295–314, 2021.
  36. Constrained update projection approach to safe policy optimization. Advances in Neural Information Processing Systems, 35:9111–9124, 2022.
  37. Policy optimization with stochastic mirror descent. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8823–8831, 2022.
  38. A continuous-decision virtual network embedding scheme relying on reinforcement learning. IEEE Transactions on Network and Service Management, 17(2):864–875, 2020.
  39. Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet of Things Journal, 5(5):3298–3304, 2018.
  40. Reinforcement learning assisted bandwidth aware virtual network resource allocation. IEEE Transactions on Network and Service Management, 19(4):4111–4123, 2022.
  41. Dapper: Deploying service function chains in the programmable data plane via deep reinforcement learning. IEEE Transactions on Services Computing, pages 1–14, 2023.
  42. Towards omni-generalizable neural methods for vehicle routing problems. In the 40th International Conference on Machine Learning, 2023.
  43. SDN/NFV-empowered future IoV with enhanced communication, computing, and caching. Proceedings of the IEEE, 108(2):274–291, 2020.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets