Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Parametric-Task MAP-Elites (2402.01275v2)

Published 2 Feb 2024 in cs.NE and cs.LG

Abstract: Optimizing a set of functions simultaneously by leveraging their similarity is called multi-task optimization. Current black-box multi-task algorithms only solve a finite set of tasks, even when the tasks originate from a continuous space. In this paper, we introduce Parametric-Task MAP-Elites (PT-ME), a new black-box algorithm for continuous multi-task optimization problems. This algorithm (1) solves a new task at each iteration, effectively covering the continuous space, and (2) exploits a new variation operator based on local linear regression. The resulting dataset of solutions makes it possible to create a function that maps any task parameter to its optimal solution. We show that PT-ME outperforms all baselines, including the deep reinforcement learning algorithm PPO on two parametric-task toy problems and a robotic problem in simulation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Simulated Binary Crossover for Continuous Search Space. Complex Systems 9 (06 2000).
  2. Timothée Anne and Jean-Baptiste Mouret. 2023. Multi-Task Multi-Behavior MAP-Elites. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (Lisbon, Portugal) (GECCO ’23 Companion). Association for Computing Machinery, New York, NY, USA, 111–114. https://doi.org/10.1145/3583133.3590730
  3. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine 34, 6 (2017), 26–38. https://doi.org/10.1109/MSP.2017.2743240
  4. Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning 47 (05 2002), 235–256. https://doi.org/10.1023/A:1013689704352
  5. Cognizant Multitasking in Multiobjective Multifactorial Evolution: MO-MFEA-II. IEEE Transactions on Cybernetics 51, 4 (2021), 1784–1796. https://doi.org/10.1109/TCYB.2020.2981733
  6. Multifactorial Evolutionary Algorithm With Online Transfer Parameter Estimation: MFEA-II. IEEE Transactions on Evolutionary Computation 24, 1 (2020), 69–83. https://doi.org/10.1109/TEVC.2019.2906927
  7. Adrien Baranes and Pierre-Yves Oudeyer. 2013. Active learning of inverse models with intrinsically motivated goal exploration in robots. Robotics and Autonomous Systems 61, 1 (2013), 49–73.
  8. S Barnett. 1968. A simple class of parametric linear programming problems. Operations Research 16, 6 (1968), 1160–1165.
  9. A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning. CoRR abs/1012.2599 (2010). arXiv:1012.2599 http://arxiv.org/abs/1012.2599
  10. Proximal policy optimization guidance algorithm for intercepting near-space maneuvering targets. Aerospace Science and Technology 132 (2023), 108031.
  11. Robots that can adapt like animals. Nature 521, 7553 (2015), 503–507. https://doi.org/10.1038/nature14422
  12. System-in-package design using multi-task memetic learning and optimization. Memetic Computing 14 (03 2022), 1–15. https://doi.org/10.1007/s12293-021-00346-5
  13. Whole-body teleoperation of the Talos humanoid robot: preliminary results. In ICRA 2021 - 5th Workshop on Teleoperation of Dynamic Legged Robots in Real Scenarios. Xi’an / Virtual, China. https://hal.inria.fr/hal-03245005
  14. B. N. Delaunay. 1928. Sur la sphere vide. In Proceedings of the Mathematics, Toronto. Toronto, 695–700. 11-16 August 1924.
  15. Centroidal Voronoi Tessellations: Applications and Algorithms. SIAM Rev. 41, 4 (1999), 637–676. https://doi.org/10.1137/S0036144599352836 arXiv:https://doi.org/10.1137/S0036144599352836
  16. MPC on a chip—Recent advances on the application of multi-parametric model-based control. Computers and Chemical Engineering 32, 4 (2008), 754–765. https://doi.org/10.1016/j.compchemeng.2007.03.008 Festschrift devoted to Rex Reklaitis on his 65th Birthday.
  17. Explicit Evolutionary Multitasking for Combinatorial Optimization: A Case Study on Capacitated Vehicle Routing Problem. IEEE Transactions on Cybernetics 51, 6 (2021), 3143–3156. https://doi.org/10.1109/TCYB.2019.2962865
  18. Anthony V. Fiacco. 1976. Sensitivity Analysis for Nonlinear Programming Using Penalty Methods. Math. Program. 10, 1 (dec 1976), 287–311. https://doi.org/10.1007/BF01580677
  19. Roger Fletcher. 1987. Practical Methods of Optimization (second ed.). John Wiley & Sons, New York, NY, USA.
  20. Matthew Fontaine and Stefanos Nikolaidis. 2021. Differentiable Quality Diversity. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.), Vol. 34. Curran Associates, Inc., 10040–10052. https://proceedings.neurips.cc/paper_files/paper/2021/file/532923f11ac97d3e7cb0130315b067dc-Paper.pdf
  21. Matthew Fontaine and Stefanos Nikolaidis. 2023. Covariance Matrix Adaptation MAP-Annealing. In Proceedings of the Genetic and Evolutionary Computation Conference (Lisbon, Portugal) (GECCO ’23). Association for Computing Machinery, New York, NY, USA, 456–465. https://doi.org/10.1145/3583131.3590389
  22. Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference (Cancún, Mexico) (GECCO ’20). Association for Computing Machinery, New York, NY, USA, 94–102. https://doi.org/10.1145/3377930.3390232
  23. An Introduction to Deep Reinforcement Learning. Foundations and Trends® in Machine Learning 11, 3-4 (2018), 219–354. https://doi.org/10.1561/2200000071
  24. Tomas Gal and Josef Nedoma. 1972. Multiparametric linear programming. Management Science 18, 7 (1972), 406–422.
  25. Procedural Content Generation through Quality Diversity. 2019 IEEE Conference on Games (CoG) (2019), 1–8. https://api.semanticscholar.org/CorpusID:195848208
  26. Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization. IEEE Transactions on Vehicular Technology 69, 11 (2020), 12597–12608. https://doi.org/10.1109/TVT.2020.3026111
  27. Abhishek Gupta and Yew Ong. 2018. Memetic Computation: The Mainspring of Knowledge Transfer in a Data-Driven Optimization Era.
  28. Multifactorial Evolution: Toward Evolutionary Multitasking. IEEE Transactions on Evolutionary Computation 20, 3 (June 2016), 343–357. https://doi.org/10.1109/TEVC.2015.2458037 Conference Name: IEEE Transactions on Evolutionary Computation.
  29. Half a Dozen Real-World Applications of Evolutionary Multitasking, and More. IEEE Computational Intelligence Magazine 17 (05 2022), 49–66. https://doi.org/10.1109/MCI.2022.3155332
  30. Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES). Evolutionary computation 11 (02 2003), 1–18. https://doi.org/10.1162/106365603321828970
  31. A Unified Framework of Graph-Based Evolutionary Multitasking Hyper-Heuristic. IEEE Transactions on Evolutionary Computation 25, 1 (2021), 35–47. https://doi.org/10.1109/TEVC.2020.2991717
  32. Surrogate-Assisted Evolutionary Framework with Adaptive Knowledge Transfer for Multi-Task Optimization. IEEE Transactions on Emerging Topics in Computing 9, 4 (2021), 1930–1944. https://doi.org/10.1109/TETC.2019.2945775
  33. Ensemble Multifactorial Evolution With Biased Skill-Factor Inheritance for Many-Task Optimization. IEEE Transactions on Evolutionary Computation 27, 6 (2023), 1735–1749. https://doi.org/10.1109/TEVC.2022.3227120
  34. Ensemble Multifactorial Evolution With Biased Skill-Factor Inheritance for Many-Task Optimization. IEEE Transactions on Evolutionary Computation 27, 6 (Dec. 2023), 1735–1749. https://doi.org/10.1109/TEVC.2022.3227120 Conference Name: IEEE Transactions on Evolutionary Computation.
  35. Jeppe Theiss Kristensen and Paolo Burelli. 2020. Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games. In Proceedings of the 15th International Conference on the Foundations of Digital Games (Bugibba, Malta) (FDG ’20). Association for Computing Machinery, New York, NY, USA, Article 2, 10 pages. https://doi.org/10.1145/3402942.3402944
  36. Joel Lehman and Kenneth Stanley. 2011. Abandoning Objectives: Evolution Through the Search for Novelty Alone. Evolutionary computation 19 (06 2011), 189–223. https://doi.org/10.1162/EVCO_a_00025
  37. Evolutionary multi-task optimization for parameters extraction of photovoltaic models. Energy Conversion and Management 207 (03 2020), 112509. https://doi.org/10.1016/j.enconman.2020.112509
  38. Evolutionary Many-Task Optimization Based on Multisource Knowledge Transfer. IEEE Transactions on Evolutionary Computation 26, 2 (2022), 319–333. https://doi.org/10.1109/TEVC.2021.3101697
  39. Rung-Tzuo Liaw and Chuan-Kang Ting. 2017. Evolutionary many-tasking based on biocoenosis through symbiosis: A framework and benchmark problems. In 2017 IEEE Congress on Evolutionary Computation (CEC). 2266–2273. https://doi.org/10.1109/CEC.2017.7969579
  40. Rung-Tzuo Liaw and Chuan-Kang Ting. 2019. Evolutionary Manytasking Optimization Based on Symbiosis in Biocoenosis. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (July 2019), 4295–4303. https://doi.org/10.1609/aaai.v33i01.33014295 Number: 01.
  41. Siyu Lin and Peter A Beling. 2021. An end-to-end optimal trade execution framework based on proximal policy optimization. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 4548–4554.
  42. A Multitasking Electric Power Dispatch Approach With Multi-Objective Multifactorial Optimization Algorithm. IEEE Access 8 (2020), 155902–155911. https://doi.org/10.1109/ACCESS.2020.3018484
  43. Songrit Maneewongvatana and David M Mount. 1999. Analysis of approximate nearest neighbor searching with clustered point sets. arXiv preprint cs/9901013 (1999).
  44. Multiproblem Surrogates: Transfer Evolutionary Multiobjective Optimization of Computationally Expensive Problems. IEEE Transactions on Evolutionary Computation 23, 1 (2019), 15–28. https://doi.org/10.1109/TEVC.2017.2783441
  45. Jean-Baptiste Mouret. 2023. Fast generation of centroids for MAP-Elites. In Companion Proceedings of the Conference on Genetic and Evolutionary Computation, GECCO 2023, Companion Volume, Lisbon, Portugal, July 15-19, 2023, Sara Silva and Luís Paquete (Eds.). ACM, 155–158. https://doi.org/10.1145/3583133.3590726
  46. Jean-Baptiste Mouret and Jeff Clune. 2015. Illuminating search spaces by mapping elites. https://doi.org/10.48550/ARXIV.1504.04909
  47. Jean-Baptiste Mouret and Glenn Maguire. 2020. Quality Diversity for Multi-Task Optimization. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference (Cancún, Mexico) (GECCO ’20). Association for Computing Machinery, New York, NY, USA, 121–129. https://doi.org/10.1145/3377930.3390203
  48. Olle Nilsson and Antoine Cully. 2021. Policy gradient assisted MAP-Elites. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, Lille France, 866–875. https://doi.org/10.1145/3449639.3459304
  49. Multiparametric Programming in Process Systems Engineering: Recent Developments and Path Forward. Frontiers in Chemical Engineering 2 (2021). https://doi.org/10.3389/fceng.2020.620168
  50. Michael Pearce and Juergen Branke. 2018. Continuous multi-task Bayesian Optimisation with correlation. European Journal of Operational Research 270, 3 (2018), 1074–1085. https://doi.org/10.1016/j.ejor.2018.03.017
  51. Deep Reinforcement Learning Based on Proximal Policy Optimization for the Maintenance of a Wind Farm with Multiple Crews. Energies 14, 20 (2021). https://doi.org/10.3390/en14206743
  52. On-line optimization via off-line parametric optimization tools. Computers & Chemical Engineering 24, 2 (July 2000), 183–188. https://doi.org/10.1016/S0098-1354(00)00510-X
  53. On-line optimization via off-line parametric optimization tools. Computers and Chemical Engineering 26, 2 (2002), 175–185. https://doi.org/10.1016/S0098-1354(01)00739-6
  54. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research 22, 268 (2021), 1–8. http://jmlr.org/papers/v22/20-1364.html
  55. Ramon Sagarna and Yew-Soon Ong. 2016. Concurrently searching branches in software tests generation through multitask evolution. In 2016 IEEE Symposium Series on Computational Intelligence (SSCI). 1–8. https://doi.org/10.1109/SSCI.2016.7850040
  56. Proximal Policy Optimization Algorithms. CoRR abs/1707.06347 (2017).
  57. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE 104, 1 (2016), 148–175. https://doi.org/10.1109/JPROC.2015.2494218
  58. Bridging Locomotion and Manipulation Using Reconfigurable Robotic Limbs via Reinforcement Learning. Biomimetics 8, 4 (2023). https://doi.org/10.3390/biomimetics8040364
  59. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17 (2020), 261–272. https://doi.org/10.1038/s41592-019-0686-2
  60. Solving Multitask Optimization Problems With Adaptive Knowledge Transfer via Anomaly Detection. IEEE Transactions on Evolutionary Computation 26, 2 (2022), 304–318. https://doi.org/10.1109/TEVC.2021.3068157
  61. Multifactorial Evolutionary Algorithm Enhanced with Cross-task Search Direction. In 2019 IEEE Congress on Evolutionary Computation (CEC). 2244–2251. https://doi.org/10.1109/CEC.2019.8789959
  62. Multifactorial optimization using Artificial Bee Colony and its application to Car Structure Design Optimization. In 2019 IEEE Congress on Evolutionary Computation (CEC). 3404–3409. https://doi.org/10.1109/CEC.2019.8789940
  63. Dynamic Scheduling Method for Job-Shop Manufacturing Systems by Deep Reinforcement Learning with Proximal Policy Optimization. Sustainability 14, 9 (2022). https://doi.org/10.3390/su14095177
  64. What makes evolutionary multi-task optimization better: A comprehensive survey. Applied Soft Computing 145 (2023), 110545. https://doi.org/10.1016/j.asoc.2023.110545
  65. Multifactorial Genetic Programming for Symbolic Regression Problems. IEEE Transactions on Systems, Man, and Cybernetics: Systems 50, 11 (2020), 4492–4505. https://doi.org/10.1109/TSMC.2018.2853719
  66. Workstation Suitability Maps: Generating Ergonomic Behaviors on a Population of Virtual Humans With Multi-Task Optimization. IEEE Robotics Autom. Lett. 8, 11 (2023), 7384–7391. https://doi.org/10.1109/LRA.2023.3318191

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets