Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes (2401.08669v3)

Published 8 Jan 2024 in cs.LG and cs.AI

Abstract: Deep reinforcement learning (RL) has been shown to be effective in producing approximate solutions to some vehicle routing problems (VRPs), especially when using policies generated by encoder-decoder attention mechanisms. While these techniques have been quite successful for relatively simple problem instances, there are still under-researched and highly complex VRP variants for which no effective RL method has been demonstrated. In this work we focus on one such VRP variant, which contains multiple trucks and multi-leg routing requirements. In these problems, demand is required to move along sequences of nodes, instead of just from a start node to an end node. With the goal of making deep RL a viable strategy for real-world industrial-scale supply chain logistics, we develop new extensions to existing encoder-decoder attention models which allow them to handle multiple trucks and multi-leg routing requirements. Our models have the advantage that they can be trained for a small number of trucks and nodes, and then embedded into a large supply chain to yield solutions for larger numbers of trucks and nodes. We test our approach on a real supply chain environment arising in the operations of Japanese automotive parts manufacturer Aisin Corporation, and find that our algorithm outperforms Aisin's previous best solution.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. The truck dispatching problem. Management science, 6(1):80–91, 1959.
  2. Vehicle routing: problems, methods, and applications. SIAM, 2014.
  3. A survey for vehicle routing problems and its derivatives. IOP Conference Series: Materials Science and Engineering, 452(4):042024, dec 2018. doi: 10.1088/1757-899X/452/4/042024. URL https://dx.doi.org/10.1088/1757-899X/452/4/042024.
  4. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940, 2016.
  5. Pointer networks. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https://proceedings.neurips.cc/paper/2015/file/29921001f2f04bd3baee84a12e98098f-Paper.pdf.
  6. Reinforcement learning for solving the vehicle routing problem. arXiv preprint arXiv:1802.04240, 2018.
  7. Deep reinforcement learning for solving the heterogeneous capacitated vehicle routing problem. IEEE Transactions on Cybernetics, 52(12):13572–13585, December 2022. ISSN 2168-2275. doi: 10.1109/tcyb.2021.3111082. URL http://dx.doi.org/10.1109/TCYB.2021.3111082.
  8. Attention, learn to solve routing problems! arXiv preprint arXiv:1803.08475, 2018.
  9. Online vehicle routing with neural combinatorial optimization and deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 20(10):3806–3817, 2019. doi: 10.1109/TITS.2019.2909109.
  10. Rl solver pro: Reinforcement learning for solving vehicle routing problem. In 2019 1st International Conference on Artificial Intelligence and Data Sciences (AiDAS), pages 94–99, 2019. doi: 10.1109/AiDAS47888.2019.8970890.
  11. Johan Oxenstierna. Warehouse vehicle routing using deep reinforcement learning. Master’s thesis, Uppsala University, Department of Information Technology, 2019.
  12. A learning-based iterative method for solving vehicle routing problems. In International conference on learning representations, 2019.
  13. Combining reinforcement learning with lin-kernighan-helsgaun algorithm for the traveling salesman problem. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 12445–12452, 2021.
  14. Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning. In Asian conference on machine learning, pages 465–480. PMLR, 2020.
  15. Learning 2-opt local search from heuristics as expert demonstrations. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2021. doi: 10.1109/IJCNN52387.2021.9533697.
  16. Learning improvement heuristics for solving routing problems. IEEE Transactions on Neural Networks and Learning Systems, 33(9):5057–5069, 2022. doi: 10.1109/TNNLS.2021.3068828.
  17. Jakub Nalepa. Chapter 7 - where machine learning meets smart delivery systems. In Jakub Nalepa, editor, Smart Delivery Systems, Intelligent Data-Centric Systems, pages 203–226. Elsevier, 2020. ISBN 978-0-12-815715-2. doi: https://doi.org/10.1016/B978-0-12-815715-2.00013-0. URL https://www.sciencedirect.com/science/article/pii/B9780128157152000130.
  18. Deep reinforcement learning for the electric vehicle routing problem with time windows. IEEE Transactions on Intelligent Transportation Systems, 23(8):11528–11538, 2022. doi: 10.1109/TITS.2021.3105232.
  19. A hybrid of deep reinforcement learning and local search for the vehicle routing problems. IEEE Transactions on Intelligent Transportation Systems, 22(11):7208–7218, 2021. doi: 10.1109/TITS.2020.3003163.
  20. A deep reinforcement learning algorithm using dynamic attention model for vehicle routing problems. In Kangshun Li, Wei Li, Hui Wang, and Yong Liu, editors, Artificial Intelligence Algorithms and Applications, pages 636–650, Singapore, 2020. Springer Singapore. ISBN 978-981-15-5577-0.
  21. A Deep Reinforcement Learning Approach for Global Routing. Journal of Mechanical Design, 142(6):061701, 11 2019. ISSN 1050-0472. doi: 10.1115/1.4045044. URL https://doi.org/10.1115/1.4045044.
  22. Deep reinforcement learning approach to solve dynamic vehicle routing problem with stochastic customers. Proceedings of the International Conference on Automated Planning and Scheduling, 30(1):394–402, Jun. 2020. doi: 10.1609/icaps.v30i1.6685. URL https://ojs.aaai.org/index.php/ICAPS/article/view/6685.
  23. Deep reinforcement learning based dynamic route planning for minimizing travel time. In 2021 IEEE International Conference on Communications Workshops (ICC Workshops), pages 1–6. IEEE, 2021.
  24. Deep reinforcement learning for the dynamic and uncertain vehicle routing problem. Applied Intelligence, 53(1):405–422, 2023.
  25. A multi-agent deep reinforcement learning approach for solving the multi-depot vehicle routing problem. Journal of Management Analytics, 10(3):493–515, 2023.
  26. A hybrid reinforcement learning-based model for the vehicle routing problem in transportation logistics. IEEE Access, 9:163325–163347, 2021.
  27. Reinforcement learning-based approach for dynamic vehicle routing problem with stochastic demand. Computers & Industrial Engineering, 182:109443, 2023.
  28. Fair collaborative vehicle routing: A deep multi-agent reinforcement learning approach. Transportation Research Part C: Emerging Technologies, 157:104376, 2023.
  29. Deep reinforcement learning for the capacitated pickup and delivery problem with time windows. Pattern Recognition and Image Analysis, 33(2):169–178, 2023.
  30. Vehicle routing problem using reinforcement learning: Recent advancements. In Deepak Gupta, Koj Sambyo, Mukesh Prasad, and Sonali Agarwal, editors, Advanced Machine Intelligence and Signal Processing, pages 269–280, Singapore, 2022. Springer Nature Singapore. ISBN 978-981-19-0840-8.
  31. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017.
  32. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018. doi: 10.1126/science.aar6404. URL https://www.science.org/doi/abs/10.1126/science.aar6404.
  33. Reinforcement learning for multi-truck vehicle routing problems. arXiv preprint https://arxiv.org/abs/2211.17078, 2022.
  34. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679, 2015.
  35. Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3):229–256, 1992. doi: 10.1007/BF00992696. URL https://doi.org/10.1007/BF00992696.
  36. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. PMLR, 2015.
  37. Reinforcement Learning: An Introduction. The MIT Press, second edition, 2018. URL http://incompleteideas.net/book/the-book-2nd.html.
  38. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6980.
  39. Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets