Papers
Topics
Authors
Recent
Search
2000 character limit reached

Kernel Expansions for High-Dimensional Mean-Field Control with Non-local Interactions

Published 17 May 2024 in math.OC | (2405.10922v1)

Abstract: Mean-field control (MFC) problems aim to find the optimal policy to control massive populations of interacting agents. These problems are crucial in areas such as economics, physics, and biology. We consider the non-local setting, where the interactions between agents are governed by a suitable kernel. For $N$ agents, the interaction cost has $\mathcal{O}(N2)$ complexity, which can be prohibitively slow to evaluate and differentiate when $N$ is large. To this end, we propose an efficient primal-dual algorithm that utilizes basis expansions of the kernels. The basis expansions reduce the cost of computing the interactions, while the primal-dual methodology decouples the agents at the expense of solving for a moderate number of dual variables. We also demonstrate that our approach can further be structured in a multi-resolution manner, where we estimate optimal dual variables using a moderate $N$ and solve decoupled trajectory optimization problems for large $N$. We illustrate the effectiveness of our method on an optimal control of 5000 interacting quadrotors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Random features for high-dimensional nonlocal mean-field games. J. Comput. Phys., 459:111136, 2022.
  2. Optimal incentives to mitigate epidemics: a Stackelberg mean field game approach. SIAM Journal on Control and Optimization, 60(2):S294–S322, 2022.
  3. J. Barreiro-Gomez and H. Tembine. Mean-field-type model predictive control: An application to water distribution networks. IEEE Access, 7:135332–135339, 2019.
  4. J. Barreiro-Gomez and H. Tembine. Mean-field-type Games for Engineers. CRC Press, 2021.
  5. Multi-Task Allocation and Path Planning for Cooperating UAVs, pages 23–41. Springer US, Boston, MA, 2003.
  6. Mean Field Games and Mean Field Type Control Theory, volume 101. Springer, 2013.
  7. Multiagent control of self-reconfigurable robots. Artificial Intelligence, 142(2):99–120, 2002. International Conference on MultiAgent Systems 2000.
  8. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning, 3(1):1–122, 2011.
  9. A mean-field model of optimal investment. arXiv preprint arXiv:2404.02871, 2024.
  10. An overview of recent progress in the study of distributed multi-agent coordination. IEEE Transactions on Industrial Informatics, 9, 07 2012.
  11. P. Cardaliaguet and C.-A. Lehalle. Mean field game of controls and an application to trade crowding. Mathematics and Financial Economics, 12:335–363, 2018.
  12. Quad Rotorcraft Control: Vision-Based Hovering and Navigation. Springer Science & Business Media, 2012.
  13. Covid-19 pandemic control: balancing detection policy and lockdown intervention under icu sustainability. Mathematical Modelling of Natural Phenomena, 15:57, 2020.
  14. A numerical algorithm for inverse problem from partial boundary measurement arising from mean field game problem. Inverse Problems, 39(1):014001, 2022.
  15. Electrical vehicles in the smart grid: A mean field game analysis. IEEE Journal on Selected Areas in Communications, 30(6):1086–1096, 2012.
  16. Event-triggered control for multi-agent systems. In Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, pages 7131–7136, 2009.
  17. Event-triggered cooperative control. In 2009 European Control Conference (ECC), pages 3015–3020, 2009.
  18. A mean field game inverse problem. Journal of Scientific Computing, 92(1):7, 2022.
  19. A stochastic maximum principle for risk-sensitive mean-field type control. IEEE Transactions on Automatic Control, 60(10):2640–2649, 2015.
  20. K. Elamvazhuthi and S. Berman. Optimal control of stochastic coverage strategies for robotic swarms. In 2015 ieee international conference on robotics and automation (icra), pages 1822–1829. IEEE, 2015.
  21. K. Elamvazhuthi and S. Berman. Mean-field models in swarm robotics: A survey. Bioinspiration & Biomimetics, 15(1):015001, 2019.
  22. Mean-field stabilization of robotic swarms to probability distributions with disconnected supports. In 2018 Annual American Control Conference (ACC), pages 885–892. IEEE, 2018.
  23. Mean-field controllability and decentralized stabilization of markov chains. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 3131–3137. IEEE, 2017.
  24. Age of information minimization using multi-agent uavs based on ai-enhanced mean field resource allocation. IEEE Transactions on Vehicular Technology, pages 1–14, 2024.
  25. Discrete approximations to optimal trajectories using direct transcription and nonlinear programming. Journal of Guidance, Control, and Dynamics, 15(4):994–1002, 1992.
  26. Kernel-Based Approximation Methods using Matlab, volume 19. World Scientific Publishing Company, 2015.
  27. How to train your neural ode: the world of jacobian and kinetic regularization. In International conference on machine learning, pages 3154–3164. PMLR, 2020.
  28. Controlled Markov Processes and Viscosity Solutions, volume 25. Springer Science & Business Media, 2006.
  29. S. W. Fung and L. Ruthotto. An uncertainty-weighted asynchronous ADMM method for parallel pde parameter estimation. SIAM Journal on Scientific Computing, 41(5):S129–S148, 2019.
  30. ADMM-softmax: An ADMM approach for multinomial logistic regression. Electronic Transactions on Numerical Analysis, 52:214–229, 2020.
  31. V. Gazi and B. Fidan. Coordination and control of multi-agent dynamic systems: Models and approaches. Swarm Robotics, 4433:71–102, 07 2007.
  32. Pseudospectral optimal control for military and industrial applications. In 2007 46th IEEE Conference on Decision and Control, pages 4128–4142, 2007.
  33. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  34. J. Han and E. Weinan. Deep learning approximation for stochastic control problems. ArXiv, abs/1611.07422, 2016.
  35. Trajectory planning for quadrotor swarms. IEEE Transactions on Robotics, 34(4):856–869, 2018.
  36. Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Transactions on Automatic Control, 48(6):988–1001, 2003.
  37. Control of variable-speed wind turbines: standard and adaptive techniques for maximizing energy capture. IEEE Control Systems Magazine, 26(3):70–81, 2006.
  38. D. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
  39. A hamilton–jacobi formulation for optimal coordination of heterogeneous multiple vehicle systems. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11623–11630, 2020.
  40. Overview: Collective control of multiagent systems. IEEE Transactions on Control of Network Systems, 3(4):334–347, 2016.
  41. Mean field control problems for vaccine distribution. Research in the Mathematical Sciences, 9(3):51, 2022.
  42. Controlling propagation of epidemics via mean-field control. SIAM Journal on Applied Mathematics, 81(1):190–207, 2021.
  43. Cooperative Control of Multi-Agent Systems: Optimal and Adaptive Design Approaches. Springer US, 01 2014.
  44. Cooperative Control of Distributed Multi‐Agent Systems, pages i–xv. John Wiley and Sons, Ltd, 2007.
  45. A neural network approach for stochastic optimal control. arXiv preprint arXiv:2209.13104, 2022.
  46. Alternating the population and control neural networks to solve high-dimensional stochastic mean-field games. Proceedings of the National Academy of Sciences, 118(31):e2024713118, 2021.
  47. D. C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical programming, 45(1):503–528, 1989.
  48. Inverse problems for mean field games. Inverse Problems, 39(8):085003, 2023.
  49. Computational methods for first-order nonlocal mean field games with applications. SIAM Journal on Numerical Analysis, 59(5):2639–2668, 2021.
  50. S. Liu and L. Nurbekyan. Splitting methods for a class of non-potential mean field games. Journal of Dynamics and Games, 8(4):467–486, 2021.
  51. Sympocnet: Solving optimal control problems with applications to high-dimensional multiagent path planning problems. SIAM Journal on Scientific Computing, 44(6):B1341–B1368, 2022.
  52. J. Mercer. Functions of positive and negative type, and their connection the theory of integral equations. Philosophical transactions of the royal society of London. Series A, 209(441-458):415–446, 1909.
  53. Foundations of Machine Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA, 2 edition, 2018.
  54. R. M. Murray. Recent Research in Cooperative Control of Multivehicle Systems. Journal of Dynamic Systems, Measurement, and Control, 129(5):571–583, 05 2007.
  55. L. Nurbekyan. One-dimensional, non-local, first-order stationary mean-field games with congestion: A Fourier approach. Discrete and Continuous Dynamical Systems - S, 11(5):963–990, 2018.
  56. L. Nurbekyan et al. Fourier approximation methods for first-order nonlocal mean-field games. Portugaliae Mathematica, 75(3):367–396, 2019.
  57. OT-Flow: fast and accurate continuous normalizing flows via optimal transport. In AAAI Conference on Artificial Intelligence, 2020.
  58. A neural network approach applied to multi-agent optimal control. In 2021 European Control Conference (ECC), pages 1036–1041. IEEE, 2021.
  59. A neural network approach for high-dimensional optimal control applied to multiagent path finding. IEEE Transactions on Control Systems Technology, 31(1):235–251, 2023.
  60. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007.
  61. Mean field type control with species dependent dynamics via structured tensor optimization. IEEE Control Systems Letters, 2023.
  62. L. Ruthotto. Differential equations for continuous-time deep learning. arXiv preprint arXiv:2401.03965, 2024.
  63. L. Ruthotto and E. Haber. An introduction to deep generative modeling. GAMM-Mitteilungen, 44(2):e202100008, 2021.
  64. A machine learning framework for solving high-dimensional mean field game and mean field control problems. Proceedings of the National Academy of Sciences, 117(17):9183–9193, 2020.
  65. Risk-sensitive mean-field games. IEEE Transactions on Automatic Control, 59(4):835–850, 2013.
  66. Neural network approaches for parameterized optimal control. arXiv preprint arXiv:2402.10033, 2024.
  67. Novel type of phase transition in a system of self-driven particles. Phys. Rev. Lett., 75:1226–1229, Aug 1995.
  68. Taming hyperparameter tuning in continuous normalizing flows using the jko scheme. Scientific reports, 13(1):4501, March 2023.
  69. J. Wang and X. Hu. Distributed consensus in multi-vehicle cooperative control: Theory and applications (ren, w. and beard, r.w.; 2008) [book shelf]. IEEE Control Systems Magazine, 30(3):85–86, 2010.
  70. Global convergence of policy gradient for linear-quadratic mean-field control/game in continuous time. In International Conference on Machine Learning, pages 10772–10782. PMLR, 2021.
  71. A survey on mean-field game for dynamic management and control in space-air-ground network. IEEE Communications Surveys & Tutorials, pages 1–1, 2024.
  72. Efficient neural network approaches for conditional optimal transport with applications in bayesian inference. arXiv preprint arXiv:2310.16975, 2023.
  73. Finite-time formation control for multi-agent systems. Automatica, 45(11):2605–2611, 2009.
  74. Multi-agent coordination by decentralized estimation and control. IEEE Transactions on Automatic Control, 53(11):2480–2496, 2008.
  75. Adaptive uncertainty-weighted ADMM for distributed optimization. Journal of Applied & Numerical Optimization, 4(2), 2022.
  76. R. Zhang and J. Kwok. Asynchronous distributed ADMM for consensus optimization. In International conference on machine learning, pages 1701–1709. PMLR, 2014.
  77. Y. Zhao and J. Han. Offline supervised learning vs online direct policy optimization: A comparative study and a unified training paradigm for neural network-based optimal feedback control. arXiv preprint arXiv:2211.15930, 2022.
  78. Transporting robotic swarms via mean-field feedback control. IEEE Transactions on Automatic Control, 67(8):4170–4177, 2021.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

Sign Up for Free