Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimization (2402.14048v1)

Published 21 Feb 2024 in cs.LG and cs.AI

Abstract: Reinforcement learning-based methods for constructing solutions to combinatorial optimization problems are rapidly approaching the performance of human-designed algorithms. To further narrow the gap, learning-based approaches must efficiently explore the solution space during the search process. Recent approaches artificially increase exploration by enforcing diverse solution generation through handcrafted rules, however, these rules can impair solution quality and are difficult to design for more complex problems. In this paper, we introduce PolyNet, an approach for improving exploration of the solution space by learning complementary solution strategies. In contrast to other works, PolyNet uses only a single-decoder and a training schema that does not enforce diverse solution generation through handcrafted rules. We evaluate PolyNet on four combinatorial optimization problems and observe that the implicit diversity mechanism allows PolyNet to find better solutions than approaches the explicitly enforce diverse solution generation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Concorde TSP solver, 2006.
  2. Neural Combinatorial Optimization with Reinforcement Learning. ArXiv, abs/1611.0, 2016.
  3. ASlib: A benchmark library for algorithm selection. Artificial Intelligence, 2016.
  4. Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Computing Surveys, 2003.
  5. Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis. International Conference on Learning Representations, 2018.
  6. Combinatorial Optimization with Policy Adaptation using Latent Space Search. Advances in Neural Information Processing Systems, 2023.
  7. Learning to Perform Local Rewriting for Combinatorial Optimization. Advances in Neural Information Processing Systems, 2019.
  8. Simulation-guided Beam Search for Neural Combinatorial Optimization. Advances in Neural Information Processing Systems, 2022.
  9. CPLEX-Optimization-Studio. V20.1: User’s Manual for CPLEX, 2020.
  10. BQ-NCO: Bisimulation Quotienting for Generalizable Neural Combinatorial Optimization. ArXiv, abs/2301.03313, 2023.
  11. Diversity is All You Need: Learning Skills without a Reward Function. International Conference on Learning Representations, 2019.
  12. Learning to Solve Vehicle Routing Problems with Time Windows through Joint Attention. arXiv preprint arXiv:2006.09100, 2020.
  13. Generalize a Small Pre-trained Model to Arbitrarily Large TSP Instances. AAAI Conference on Artificial Intelligence, 2021.
  14. Equivalence notions and model minimization in Markov decision processes. Artificial Intelligence, 2003.
  15. Winner Takes It All: Training Performant RL Populations for Combinatorial Optimization. Advances in Neural Information Processing Systems, 2023.
  16. Keld Helsgaun. An extension of the Lin-Kernighan-Helsgaun TSP solver for constrained traveling salesman and vehicle routing problems. Roskilde: Roskilde University, 2017.
  17. Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problem. European Conference on Artificial Intelligence, 2020.
  18. Learning a Latent Search Space for Routing Problems using Variational Autoencoders. International Conference on Learning Representations, 2021.
  19. Efficient Active Search for Combinatorial Optimization Problems. International Conference on Learning Representations, 2022.
  20. An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem. arXiv preprint arXiv:1906.01227, 2019.
  21. Learning Collaborative Policies to Solve NP-hard Routing Problems. Advances in Neural Information Processing Systems, 2021.
  22. Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimization. Advances in Neural Information Processing Systems, 2022.
  23. Attention, Learn to Solve Routing Problems! International Conference on Learning Representations, 2019.
  24. The EURO Meets NeurIPS 2022 Vehicle Routing Competition. NeurIPS 2022 Competitions Track, 2022a.
  25. Deep Policy Dynamic Programming for Vehicle Routing Problems. Integration of Constraint Programming, Artificial Intelligence, and Operations Research, 2022b.
  26. POMO: Policy Optimization with Multiple Optima for Reinforcement Learning. Advances in Neural Information Processing Systems, 2020.
  27. Matrix encoding networks for neural combinatorial optimization. Advances in Neural Information Processing Systems, 2021.
  28. Celebrating Diversity in Shared Multi-Agent Reinforcement Learning. Advances in Neural Information Processing Systems, 2021.
  29. Learning Feature Embedding Refiner for Solving Vehicle Routing Problems. IEEE Transactions on Neural Networks and Learning Systems, 2023.
  30. Learning to Iteratively Solve Routing Problems with Dual-Aspect Collaborative Transformer. Advances in Neural Information Processing Systems, 2021.
  31. Reinforcement learning for solving the vehicle routing problem. Advances in Neural Information Processing Systems, 2018.
  32. Evolving Populations of Diverse RL Agents with MAP-Elites. International Conference on Learning Representations, 2023.
  33. Christian Prins. Two memetic algorithms for heterogeneous fleet vehicle routing problems. Engineering Applications of Artificial Intelligence, 2009.
  34. 10,000 optimal CVRP solutions for testing machine learning based heuristics. AAAI-22 Workshop on Machine Learning for Operations Research (ML4OR), 2022.
  35. Dynamics-Aware Unsupervised Discovery of Skills. International Conference on Learning Representations, 2019.
  36. Marius M Solomon. Algorithms for the Vehicle Routing and Scheduling Problems with Time Window Constraints. Operations Research, 1987.
  37. Attention is All you Need. Advances in Neural Information Processing Systems, 2017.
  38. Thibaut Vidal. Hybrid genetic search for the CVRP: Open-source implementation and SWAP* Neighborhood. Computers & Operations Research, 2022.
  39. A Hybrid Genetic Algorithm for Multidepot and Periodic Vehicle Routing Problems. Operations Research, 2012.
  40. Pointer Networks. Advances in Neural Information Processing Systems, 2015.
  41. PyVRP: A high-performance VRP solver package. INFORMS Journal on Computing, 2024.
  42. Quality-Similar Diversity via Population Based Reinforcement Learning. International Conference on Learning Representations, 2023.
  43. Multi-Decoder Attention Model with Embedding Glimpse for Solving Vehicle Routing Problems. AAAI Conference on Artificial Intelligence, 2021.
  44. Learning Novel Policies For Tasks. International Conference on Machine Learning, 2019.
Citations (7)

Summary

We haven't generated a summary for this paper yet.