Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fast and Continual Learning for Hybrid Control Policies using Generalized Benders Decomposition (2401.00917v2)

Published 1 Jan 2024 in cs.RO

Abstract: Hybrid model predictive control with both continuous and discrete variables is widely applicable to robotic control tasks, especially those involving contact with the environment. Due to the combinatorial complexity, the solving speed of hybrid MPC can be insufficient for real-time applications. In this paper, we proposed a hybrid MPC solver based on Generalized Benders Decomposition (GBD). The algorithm enumerates and stores cutting planes online inside a finite buffer. After a short cold-start phase, the stored cuts provide warm-starts for the new problem instances to enhance the solving speed. Despite the disturbance and randomly changing environment, the solving speed maintains. Leveraging on the sparsity of feasibility cuts, we also propose a fast algorithm for Benders master problems. Our solver is validated through controlling a cart-pole system with randomly moving soft contact walls, and a free-flying robot navigating around obstacles. The results show that with significantly less data than previous works, the solver reaches competitive speeds to the off-the-shelf solver Gurobi despite the Python overhead.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion, and R. Tedrake, “Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot,” Autonomous robots, vol. 40, pp. 429–455, 2016.
  2. X. Lin, J. Zhang, J. Shen, G. Fernandez, and D. W. Hong, “Optimization based motion planning for multi-limbed vertical climbing robots,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2019, pp. 1918–1925.
  3. F. R. Hogan and A. Rodriguez, “Reactive planar non-prehensile manipulation with hybrid model predictive control,” The International Journal of Robotics Research, vol. 39, no. 7, pp. 755–773, 2020.
  4. J. Zhang, X. Lin, and D. W. Hong, “Transition motion planning for multi-limbed vertical climbing robots using complementarity constraints,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 2033–2039.
  5. A. Bemporad and M. Morari, “Control of systems integrating logic, dynamics, and constraints,” Automatica, vol. 35, no. 3, pp. 407–427, 1999.
  6. W. Heemels, J. M. Schumacher, and S. Weiland, “Linear complementarity systems,” SIAM journal on applied mathematics, vol. 60, no. 4, pp. 1234–1269, 2000.
  7. E. Sontag, “Nonlinear regulation: The piecewise linear approach,” IEEE Transactions on automatic control, vol. 26, no. 2, pp. 346–358, 1981.
  8. W. P. Heemels, B. De Schutter, and A. Bemporad, “Equivalence of hybrid dynamical models,” Automatica, vol. 37, no. 7, pp. 1085–1091, 2001.
  9. A. Aydinoglu, A. Wei, and M. Posa, “Consensus complementarity control for multi-contact mpc,” arXiv preprint arXiv:2304.11259, 2023.
  10. S. L. Cleac’h, T. Howell, M. Schwager, and Z. Manchester, “Fast contact-implicit model-predictive control,” arXiv preprint arXiv:2107.05616, 2021.
  11. T. Marcucci and R. Tedrake, “Warm start of mixed-integer programs for model predictive control of hybrid systems,” IEEE Transactions on Automatic Control, vol. 66, no. 6, pp. 2433–2448, 2020.
  12. A. M. Geoffrion, “Generalized benders decomposition,” Journal of optimization theory and applications, vol. 10, pp. 237–260, 1972.
  13. A. Cauligi, P. Culbertson, E. Schmerling, M. Schwager, B. Stellato, and M. Pavone, “Coco: Online mixed-integer control via supervised learning,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1447–1454, 2021.
  14. R. Deits, T. Koolen, and R. Tedrake, “Lvis: Learning from value function intervals for contact-aware robot controllers,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 7762–7768.
  15. A. Cauligi, P. Culbertson, B. Stellato, D. Bertsimas, M. Schwager, and M. Pavone, “Learning mixed-integer convex optimization strategies for robot planning and control,” in 2020 59th IEEE Conference on Decision and Control (CDC).   IEEE, 2020, pp. 1698–1705.
  16. L. Xuan, “Generalized benders decomposition with continual learning for hybrid model predictive control in dynamic environment,” arXiv preprint arXiv:2310.03344, 2023.
  17. J. Tobajas, F. Garcia-Torres, P. Roncero-Sánchez, J. Vázquez, L. Bellatreche, and E. Nieto, “Resilience-oriented schedule of microgrids with hybrid energy storage system using model predictive control,” Applied Energy, vol. 306, p. 118092, 2022.
  18. Y. Cao, Z. Zhang, F. Cheng, and S. Su, “Trajectory optimization for high-speed trains via a mixed integer linear programming approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10, pp. 17 666–17 676, 2022.
  19. M. Klaučo, J. Drgoňa, M. Kvasnica, and S. Di Cairano, “Building temperature control by simple mpc-like feedback laws learned from closed-loop data,” IFAC Proceedings Volumes, vol. 47, no. 3, pp. 581–586, 2014.
  20. T. Marcucci, R. Deits, M. Gabiccini, A. Bicchi, and R. Tedrake, “Approximate hybrid model predictive control for multi-contact push recovery in complex environments,” in 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).   IEEE, 2017, pp. 31–38.
  21. A. M. Geoffrion, “Lagrangian relaxation for integer programming,” 50 Years of Integer Programming 1958-2008: From the Early Years to the State-of-the-Art, pp. 243–281, 2010.
  22. M. Anitescu, “On using the elastic mode in nonlinear programming approaches to mathematical programs with complementarity constraints,” SIAM Journal on Optimization, vol. 15, no. 4, pp. 1203–1236, 2005.
  23. R. Lazimy, “Mixed-integer quadratic programming,” Mathematical Programming, vol. 22, pp. 332–349, 1982.
  24. A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos, “The explicit linear quadratic regulator for constrained systems,” Automatica, vol. 38, no. 1, pp. 3–20, 2002.
  25. J.-J. Zhu and G. Martius, “Fast non-parametric learning to accelerate mixed-integer programming for hybrid model predictive control,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 5239–5245, 2020.
  26. X. Lin, G. I. Fernandez, and D. W. Hong, “Reduce: Reformulation of mixed integer programs using data from unsupervised clusters for learning efficient strategies,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 4459–4465.
  27. X. Lin, F. Xu, A. Schperberg, and D. Hong, “Learning near-global-optimal strategies for hybrid non-convex model predictive control of single rigid body locomotion,” arXiv preprint arXiv:2207.07846, 2022.
  28. P. Hespanhol, R. Quirynen, and S. Di Cairano, “A structure exploiting branch-and-bound algorithm for mixed-integer model predictive control,” in 2019 18th European Control Conference (ECC).   IEEE, 2019, pp. 2763–2768.
  29. B. Stellato, G. Banjac, P. Goulart, A. Bemporad, and S. Boyd, “Osqp: An operator splitting solver for quadratic programs,” Mathematical Programming Computation, vol. 12, no. 4, pp. 637–672, 2020.
  30. Y. Shirai, X. Lin, A. Schperberg, Y. Tanaka, H. Kato, V. Vichathorn, and D. Hong, “Simultaneous contact-rich grasping and locomotion via distributed optimization enabling free-climbing for multi-limbed robots,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 13 563–13 570.
  31. X. Lin, G. I. Fernandez, Y. Liu, T. Zhu, Y. Shirai, and D. Hong, “Multi-modal multi-agent optimization for limms, a modular robotics approach to delivery automation,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 12 674–12 681.
  32. B. Wu and B. Ghanem, “ℓpsubscriptℓ𝑝\ell_{p}roman_ℓ start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT-box admm: A versatile framework for integer programming,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 7, pp. 1695–1708, 2018.
  33. J. Benders, “Partitioning procedures for solving mixed-variables programming problems ‘,” Numerische mathematik, vol. 4, no. 1, pp. 238–252, 1962.
  34. G. B. Dantzig and P. Wolfe, “Decomposition principle for linear programs,” Operations research, vol. 8, no. 1, pp. 101–111, 1960.
  35. L. J. Watters, “Reduction of integer polynomial programming problems to zero-one linear programming problems,” Operations Research, vol. 15, no. 6, pp. 1171–1174, 1967.
  36. F. Glover, “Improved linear integer programming formulations of nonlinear integer problems,” Management science, vol. 22, no. 4, pp. 455–460, 1975.
  37. R. McBride and J. Yormark, “An implicit enumeration algorithm for quadratic integer programming,” Management Science, vol. 26, no. 3, pp. 282–296, 1980.
  38. J. N. Hooker and H. Yan, “Logic circuit verification by benders decomposition,” Principles and practice of constraint programming: the newport papers, pp. 267–288, 1995.
  39. J. N. Hooker and G. Ottosson, “Logic-based benders decomposition,” Mathematical Programming, vol. 96, no. 1, pp. 33–60, 2003.
  40. G. Codato and M. Fischetti, “Combinatorial benders’ cuts for mixed-integer linear programming,” Operations Research, vol. 54, no. 4, pp. 756–766, 2006.
  41. P.-D. Moroşan, R. Bourdais, D. Dumur, and J. Buisson, “A distributed mpc strategy based on benders’ decomposition applied to multi-source multi-zone temperature regulation,” Journal of Process Control, vol. 21, no. 5, pp. 729–737, 2011.
  42. R. Rahmaniani, T. G. Crainic, M. Gendreau, and W. Rei, “The benders decomposition algorithm: A literature review,” European Journal of Operational Research, vol. 259, no. 3, pp. 801–817, 2017.
  43. W. Rei, J.-F. Cordeau, M. Gendreau, and P. Soriano, “Accelerating benders decomposition by local branching,” INFORMS Journal on Computing, vol. 21, no. 2, pp. 333–345, 2009.
  44. A. M. Costa, J.-F. Cordeau, B. Gendron, and G. Laporte, “Accelerating benders decomposition with heuristicmaster problem solutions,” Pesquisa Operacional, vol. 32, pp. 03–20, 2012.
  45. T. L. Magnanti and R. T. Wong, “Accelerating benders decomposition: Algorithmic enhancement and model selection criteria,” Operations research, vol. 29, no. 3, pp. 464–484, 1981.
  46. J.-F. Cordeau, G. Stojković, F. Soumis, and J. Desrosiers, “Benders decomposition for simultaneous aircraft routing and crew scheduling,” Transportation science, vol. 35, no. 4, pp. 375–388, 2001.
  47. J. A. Rodríguez, M. F. Anjos, P. Côté, and G. Desaulniers, “Accelerating benders decomposition for short-term hydropower maintenance scheduling,” European Journal of Operational Research, vol. 289, no. 1, pp. 240–253, 2021.
  48. A. Grothey, S. Leyffer, and K. McKinnon, “A note on feasibility in benders decomposition,” Numerical Analysis Report NA/188, Dundee University, 1999.
  49. M. V. Pereira and L. M. Pinto, “Stochastic optimization of a multireservoir hydroelectric system: A decomposition approach,” Water resources research, vol. 21, no. 6, pp. 779–792, 1985.
  50. ——, “Multi-stage stochastic optimization applied to energy planning,” Mathematical programming, vol. 52, pp. 359–375, 1991.
  51. J. Warrington, P. N. Beuchat, and J. Lygeros, “Generalized dual dynamic programming for infinite horizon problems in continuous state and action spaces,” IEEE Transactions on Automatic Control, vol. 64, no. 12, pp. 5012–5023, 2019.
  52. S. Menta, J. Warrington, J. Lygeros, and M. Morari, “Learning q-function approximations for hybrid control problems,” IEEE Control Systems Letters, vol. 6, pp. 1364–1369, 2021.
  53. L. Wu and M. Shahidehpour, “Accelerating the benders decomposition for network-constrained unit commitment problems,” Energy Systems, vol. 1, no. 3, pp. 339–376, 2010.
  54. N. Beheshti Asl and S. MirHassani, “Accelerating benders decomposition: multiple cuts via multiple solutions,” Journal of Combinatorial Optimization, vol. 37, pp. 806–826, 2019.
  55. G. Optimization. (2023) Mipgap. [Online]. Available: https://www.gurobi.com/documentation/9.5/refman/mipgap2.html
  56. K. Hauser, “Learning the problem-optimum map: Analysis and application to global optimization in robotics,” IEEE Transactions on Robotics, vol. 33, no. 1, pp. 141–152, 2016.
  57. V. Dua and E. N. Pistikopoulos, “An algorithm for the solution of multiparametric mixed integer linear programming problems,” Annals of operations research, vol. 99, pp. 123–139, 2000.
  58. H. J. Ferreau, H. G. Bock, and M. Diehl, “An online active set strategy to overcome the limitations of explicit mpc,” International Journal of Robust and Nonlinear Control: IFAC-Affiliated Journal, vol. 18, no. 8, pp. 816–830, 2008.
  59. M. Walter, “Sparsity of lift-and-project cutting planes,” in Operations Research Proceedings 2012: Selected Papers of the International Annual Conference of the German Operations Research Society (GOR), Leibniz University of Hannover, Germany, September 5-7, 2012.   Springer, 2013, pp. 9–14.
  60. L. Alfandari, I. Ljubić, and M. D. M. da Silva, “A tailored benders decomposition approach for last-mile delivery with autonomous robots,” European Journal of Operational Research, vol. 299, no. 2, pp. 510–525, 2022.
  61. S. S. Dey, M. Molinaro, and Q. Wang, “Analysis of sparse cutting planes for sparse milps with applications to stochastic milps,” Mathematics of Operations Research, vol. 43, no. 1, pp. 304–332, 2018.
  62. T. Marcucci, M. Petersen, D. von Wrangel, and R. Tedrake, “Motion planning around obstacles with convex optimization,” arXiv preprint arXiv:2205.04422, 2022.
  63. R. Quirynen and S. Di Cairano, “Tailored presolve techniques in branch-and-bound method for fast mixed-integer optimal control applications,” Optimal Control Applications and Methods, vol. 44, no. 6, pp. 3139–3167, 2023.
  64. S. Caron, D. Arnström, S. Bonagiri, A. Dechaume, N. Flowers, A. Heins, T. Ishikawa, D. Kenefake, G. Mazzamuto, D. Meoli, B. O’Donoghue, A. A. Oppenheimer, A. Pandala, J. J. Quiroz Omaña, N. Rontsis, P. Shah, S. St-Jean, N. Vitucci, S. Wolfers, @bdelhaisse, @MeindertHH, @rimaddo, @urob, and @shaoanlu, “qpsolvers: Quadratic Programming Solvers in Python,” Dec. 2023. [Online]. Available: https://github.com/qpsolvers/qpsolvers
  65. E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” 2016.
  66. T. Marcucci, J. Umenberger, P. A. Parrilo, and R. Tedrake, “Shortest paths in graphs of convex sets,” arXiv preprint arXiv:2101.11565, 2021.
  67. D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. Scokaert, “Constrained model predictive control: Stability and optimality,” Automatica, vol. 36, no. 6, pp. 789–814, 2000.
  68. P. O. Scokaert, D. Q. Mayne, and J. B. Rawlings, “Suboptimal model predictive control (feasibility implies stability),” IEEE Transactions on Automatic Control, vol. 44, no. 3, pp. 648–654, 1999.
  69. C. N. Jones and M. Morari, “Approximate explicit mpc using bilevel optimization,” in 2009 European control conference (ECC).   IEEE, 2009, pp. 2396–2401.
  70. M. N. Zeilinger, C. N. Jones, and M. Morari, “Real-time suboptimal model predictive control using a combination of explicit mpc and online optimization,” IEEE Transactions on Automatic Control, vol. 56, no. 7, pp. 1524–1534, 2011.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now