Papers
Topics
Authors
Recent
2000 character limit reached

Towards Constraint-aware Learning for Resource Allocation in NFV-enabled Networks (2410.22999v1)

Published 30 Oct 2024 in cs.NI

Abstract: Virtual Network Embedding (VNE) is a challenging combinatorial optimization problem that refers to resource allocation associated with hard and multifaceted constraints in network function virtualization (NFV). Existing works for VNE struggle to handle such complex constraints, leading to compromised system performance and stability. In this paper, we propose a \textbf{CON}straint-\textbf{A}ware \textbf{L}earning framework for VNE, named \textbf{CONAL}, to achieve efficient constraint management. Concretely, we formulate the VNE problem as a constrained Markov decision process with violation tolerance. This modeling approach aims to improve both resource utilization and solution feasibility by precisely evaluating solution quality and the degree of constraint violation. We also propose a reachability-guided optimization with an adaptive reachability budget method that dynamically assigns budget values. This method achieves persistent zero violation to guarantee the feasibility of VNE solutions and more stable policy optimization by handling instances without any feasible solution. Furthermore, we propose a constraint-aware graph representation method to efficiently learn cross-graph relations and constrained path connectivity in VNE. Finally, extensive experimental results demonstrate the superiority of our proposed method over state-of-the-art baselines. Our code is available at https://github.com/GeminiLight/conal-vne.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Constrained policy optimization. In International conference on machine learning, pp.  22–31. PMLR, 2017.
  2. Eitan Altman. Constrained Markov decision processes. Routledge, 2021.
  3. Hamilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp.  2242–2253. IEEE, 2017.
  4. Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 290(2):405–421, 2021.
  5. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp.  1597–1607, 2020.
  6. Virtual network embedding with coordinated node and link mapping. In IEEE International Conference on Computer Communications, pp.  783–791, 2009.
  7. DYVINE: Fitness-based dynamic virtual network embedding in cloud computing. IEEE Journal on Selected Areas in Communications, 37(5):1029–1045, 2019.
  8. Node essentiality assessment and distributed collaborative virtual network embedding in datacenters. IEEE Transactions on Parallel and Distributed Systems, 34(4):1265–1280, 2023.
  9. Virtual network embedding: A survey. IEEE Communications Surveys Tutorials, 15(4):1888–1906, 2013.
  10. Gal-vne: Solving the vne problem with global reinforcement learning and local one-shot neural prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  531–543, 2023.
  11. Toward profit-seeking virtual network embedding algorithm via global resource capacity. In IEEE International Conference on Computer Communications, pp.  1–9, 2014.
  12. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271–21284, 2020.
  13. A review of safe reinforcement learning: Methods, theory and applications. arXiv preprint arXiv:2205.10330, 2022.
  14. Virtual network embedding via Monte Carlo tree search. IEEE Transactions on Cybernetics, 48(2):510–521, 2017.
  15. Leveraging deep reinforcement learning with attention mechanism for virtual network function placement and routing. IEEE Transactions on Parallel and Distributed Systems, 34(4):1186–1201, 2023a.
  16. Autocost: Evolving intrinsic cost for zero-violation reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.  14847–14855, 2023b.
  17. VNE-HPSO Virtual Network Embedding Algorithm Based on Hybrid Particle Swarm Optimization, pp.  129–152. Springer Singapore, Singapore, 2021.
  18. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  19. Policy learning with constraints in model-free reinforcement learning: A survey. In The 30th International Joint Conference on Artificial Intelligence (IJCAI), 2021.
  20. Feasible actor-critic: Constrained reinforcement learning for ensuring statewise safety. arXiv preprint arXiv:2105.10682, 2021.
  21. Graph convolutional network aided virtual network embedding for internet of thing. IEEE Transactions on Network Science and Engineering, 10(1):265–274, 2023.
  22. Reinforcement learning for combinatorial optimization: A survey. Computers & Operations Research, 134:105400, 2021.
  23. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  24. Adjustable robust reinforcement learning for online 3d bin packing. In Advances in Neural Information Processing Systems, volume 36, pp.  51926–51954, 2023.
  25. Benchmarking safe exploration in deep reinforcement learning. arXiv preprint arXiv:1910.01708, 7(1):2, 2019.
  26. On the hardness and inapproximability of virtual network embeddings. IEEE/ACM Transactions on Networking, 28(2):791–803, 2020.
  27. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  28. Virtual network survivability through joint spare capacity allocation and embedding. IEEE Journal on Selected Areas in Communications, 36(3):502–518, 2018.
  29. Graph attention networks. In International Conference on Learning Representations, 2018.
  30. DRL-SFCP: Adaptive service function chains placement with deep reinforcement learning. In IEEE International Conference on Communications, pp.  1–6, 2021.
  31. Joint admission control and resource allocation of virtual network embedding via hierarchical deep reinforcement learning. IEEE Transactions on Services Computing, pp.  1–14, 2023.
  32. B.M. Waxman. Routing of multipoint connections. IEEE Journal on Selected Areas in Communications, 6(9):1617–1622, 1988.
  33. Ai-empowered virtual network embedding:a comprehensive survey. IEEE Communications Surveys & Tutorials, 2024.
  34. Automatic virtual network embedding: A deep reinforcement learning approach with graph convolutional networks. IEEE Journal on Selected Areas in Communications, 38(6):1040–1057, 2020.
  35. Constrained update projection approach to safe policy optimization. Advances in Neural Information Processing Systems, 35:9111–9124, 2022.
  36. A continuous-decision virtual network embedding scheme relying on reinforcement learning. IEEE Transactions on Network and Service Management, 17(2):864–875, 2020.
  37. Deepaco: Neural-enhanced ant systems for combinatorial optimization. Advances in Neural Information Processing Systems, 36, 2023.
  38. A comprehensive survey of network function virtualization. Computer Networks, 133:212–262, 2018.
  39. Reachability constrained reinforcement learning. In International Conference on Machine Learning, pp.  25636–25655. PMLR, 2022.
  40. Barlow twins: Self-supervised learning via redundancy reduction. In International conference on machine learning, pp.  12310–12320. PMLR, 2021.
  41. Learning to dispatch for job shop scheduling via deep reinforcement learning. Advances in neural information processing systems, 33:1621–1632, 2020.
  42. Robust scheduling with gflownets. In International Conference on Learning Representations, 2023a.
  43. Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet of Things Journal, 5(5):3298–3304, 2018.
  44. Virtual network embedding based on modified genetic algorithm. Peer-to-Peer Networking and Applications, 12(2):481–492, 2019.
  45. Reinforcement learning assisted bandwidth aware virtual network resource allocation. IEEE Transactions on Network and Service Management, 19(4):4111–4123, 2022.
  46. Dapper: Deploying service function chains in the programmable data plane via deep reinforcement learning. IEEE Transactions on Services Computing, pp.  1–14, 2023b.
  47. Online 3d bin packing with constrained deep reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  741–749, 2021.
  48. Model-free safe control for zero-violation reinforcement learning. In Proceedings of the 5th Conference on Robot Learning, volume 164, pp.  784–793. PMLR, 2022.
  49. State-wise safe reinforcement learning: A survey. In International Joint Conference on Artificial Intelligence, 2023.
  50. Towards omni-generalizable neural methods for vehicle routing problems. In International Conference on Machine Learning, pp.  42769–42789. PMLR, 2023.
  51. SDN/NFV-empowered future IoV with enhanced communication, computing, and caching. Proceedings of the IEEE, 108(2):274–291, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - GeminiLight/conal-vne (1 star)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free