Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cross-Problem Learning for Solving Vehicle Routing Problems (2404.11677v2)

Published 17 Apr 2024 in cs.AI

Abstract: Existing neural heuristics often train a deep architecture from scratch for each specific vehicle routing problem (VRP), ignoring the transferable knowledge across different VRP variants. This paper proposes the cross-problem learning to assist heuristics training for different downstream VRP variants. Particularly, we modularize neural architectures for complex VRPs into 1) the backbone Transformer for tackling the travelling salesman problem (TSP), and 2) the additional lightweight modules for processing problem-specific features in complex VRPs. Accordingly, we propose to pre-train the backbone Transformer for TSP, and then apply it in the process of fine-tuning the Transformer models for each target VRP variant. On the one hand, we fully fine-tune the trained backbone Transformer and problem-specific modules simultaneously. On the other hand, we only fine-tune small adapter networks along with the modules, keeping the backbone Transformer still. Extensive experiments on typical VRPs substantiate that 1) the full fine-tuning achieves significantly better performance than the one trained from scratch, and 2) the adapter-based fine-tuning also delivers comparable performance while being notably parameter-efficient. Furthermore, we empirically demonstrate the favorable effect of our method in terms of cross-distribution application and versatility.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Beit: Bert pre-training of image transformers. In ICLR, 2022.
  2. Neural combinatorial optimization with reinforcement learning. In ICLR, 2017.
  3. Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 290(2):405–421, 2021.
  4. Learning generalizable models for vehicle routing problems via knowledge distillation. In NeurIPS, 2022.
  5. A simple framework for contrastive learning of visual representations. In ICML, pages 1597–1607, 2020.
  6. Simulation-guided beam search for neural combinatorial optimization. In NeurIPS, 2022.
  7. Learning combinatorial optimization algorithms over graphs. In NeurIPS, 2017.
  8. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  9. Unified language model pre-training for natural language understanding and generation. In NeurIPS, pages 13063–13075, 2019.
  10. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
  11. Generalize a small pre-trained model to arbitrarily large tsp instances. In AAAI, pages 7474–7482, 2021.
  12. Learn to design the heuristics for vehicle routing problem. arXiv preprint arXiv:2002.08539, 2020.
  13. Generalization of neural combinatorial solvers through the lens of adversarial robustness. In ICLR, 2022.
  14. Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2022.
  15. Momentum contrast for unsupervised visual representation learning. In CVPR, pages 9729–9738, 2020.
  16. Neural large neighborhood search for the capacitated vehicle routing problem. In ECAI, pages 443–450, 2020.
  17. Efficient active search for combinatorial optimization problems. In ICLR, 2022.
  18. Parameter-efficient transfer learning for nlp. In ICML, pages 2790–2799, 2019.
  19. Strategies for pre-training graph neural networks. In ICLR, 2020.
  20. Lora: Low-rank adaptation of large language models. In ICLR, 2021.
  21. Graph neural network guided local search for the traveling salesperson problem. In ICLR, 2022.
  22. Learning collaborative policies to solve np-hard routing problems. In NeurIPS, pages 10418–10430, 2021.
  23. Learning to cross exchange to solve min-max vehicle routing problems. In ICLR, 2022.
  24. Sym-NCO: Leveraging symmetricity for neural combinatorial optimization. In NeurIPS, 2022.
  25. Reformer: The efficient transformer. In ICLR, 2020.
  26. An efficient evolutionary algorithm for the orienteering problem. Computers & Operations Research, 90:42–59, 2018.
  27. Vehicle routing problem and related algorithms for logistics distribution: A literature review and classification. Operational research, pages 1–30, 2022.
  28. Attention, learn to solve routing problems! In ICLR, 2018.
  29. Pomo: Policy optimization with multiple optima for reinforcement learning. In NeurIPS, pages 21188–21198, 2020.
  30. Matrix encoding networks for neural combinatorial optimization. In NeurIPS, pages 5138–5149, 2021.
  31. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In ACL, pages 7871–7880, 2020.
  32. Learning to delegate for large-scale vehicle routing. In NeurIPS, pages 26198–26211, 2021.
  33. Neural combinatorial optimization with heavy decoder: Toward large scale generalization. In NeurIPS, 2023.
  34. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. In NeurIPS, pages 11096–11107, 2021.
  35. Reinforcement learning for solving the vehicle routing problem. In NeurIPS, page 9861–9871, 2018.
  36. Adapterfusion: Non-destructive task composition for transfer learning. In EACL, pages 487–503, 2021.
  37. Gcc: Graph contrastive coding for graph neural network pre-training. In KDD, pages 1150–1160, 2020.
  38. DIMES: A differentiable meta solver for combinatorial optimization problems. In NeurIPS, 2022.
  39. Beyond accuracy: Behavioral testing of nlp models with checklist. In ACL, pages 4902–4912, 2020.
  40. Neural networks for local search and crossover in vehicle routing: A possible overkill? arXiv preprint arXiv:2210.12075, 2022.
  41. Lst: Ladder side-tuning for parameter and memory efficient transfer learning. In NeurIPS, pages 12991–13005, 2022.
  42. Reinforcement learning: An introduction. MIT press, 2018.
  43. Training data-efficient image transformers & distillation through attention. In ICML, pages 10347–10357, 2021.
  44. Theodore Tsiligirides. Heuristic methods applied to orienteering. Journal of the Operational Research Society, 35:797–809, 1984.
  45. Attention is all you need. In NeurIPS, page 6000–6010, 2017.
  46. Vehicle routing problem with drones. Transportation research part B: methodological, 122:350–364, 2019.
  47. A bi-level framework for learning to solve combinatorial optimization on graphs. In NeurIPS, pages 21453–21466, 2021.
  48. Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256, 1992.
  49. Learning improvement heuristics for solving routing problems. IEEE transactions on neural networks and learning systems, 2021.
  50. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In AAAI, pages 12042–12049, 2021.
  51. Neurolkh: Combining deep learning model with lin-kernighan-helsgaun heuristic for solving the traveling salesman problem. In NeurIPS, pages 7472–7483, 2021.
  52. Graph contrastive learning with augmentations. In NeurIPS, pages 5812–5823, 2020.
  53. Graph transformer networks. In NeurIPS, pages 11983–11993, 2019.
  54. Side-tuning: a baseline for network adaptation via additive side networks. In ECCV, pages 698–714. Springer, 2020.
  55. Masking as an efficient alternative to finetuning for pretrained language models. In EMNLP, pages 2226–2241, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Zhuoyi Lin (4 papers)
  2. Yaoxin Wu (26 papers)
  3. Bangjian Zhou (5 papers)
  4. Zhiguang Cao (48 papers)
  5. Wen Song (24 papers)
  6. Yingqian Zhang (30 papers)
  7. Senthilnath Jayavelu (13 papers)
Citations (6)
View on Semantic Scholar