Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Implicit Regularization and Momentum Algorithms in Nonlinearly Parameterized Adaptive Control and Prediction (1912.13154v7)

Published 31 Dec 2019 in math.OC and cs.LG

Abstract: Stable concurrent learning and control of dynamical systems is the subject of adaptive control. Despite being an established field with many practical applications and a rich theory, much of the development in adaptive control for nonlinear systems revolves around a few key algorithms. By exploiting strong connections between classical adaptive nonlinear control techniques and recent progress in optimization and machine learning, we show that there exists considerable untapped potential in algorithm development for both adaptive nonlinear control and adaptive dynamics prediction. We begin by introducing first-order adaptation laws inspired by natural gradient descent and mirror descent. We prove that when there are multiple dynamics consistent with the data, these non-Euclidean adaptation laws implicitly regularize the learned model. Local geometry imposed during learning thus may be used to select parameter vectors -- out of the many that will achieve perfect tracking or prediction -- for desired properties such as sparsity. We apply this result to regularized dynamics predictor and observer design, and as concrete examples, we consider Hamiltonian systems, Lagrangian systems, and recurrent neural networks. We subsequently develop a variational formalism based on the Bregman Lagrangian. We show that its Euler Lagrange equations lead to natural gradient and mirror descent-like adaptation laws with momentum, and we recover their first-order analogues in the infinite friction limit. We illustrate our analyses with simulations demonstrating our theoretical results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. Robust Adaptive Control. Dover Publications, 2012.
  2. Stable Adaptive Systems. Dover Publications, 2005.
  3. Applied Nonlinear Control. Prentice Hall, 1991.
  4. On the adaptive control of robot manipulators. The International Journal of Robotics Research, 6(3):49–59, 1987. doi: https://doi.org/10.1177/027836498700600303.
  5. Gaussian networks for direct adaptive control. IEEE Transactions on Neural Networks, 3(6):837–863, 1992. ISSN 1941-0093. doi: 10.1109/72.165588.
  6. B. T. Polyak. Gradient methods for minimizing functionals (in russian). U.S.S.R. Comput. Math. Math. Phys., 3:643–653, 1963.
  7. Adaptation and parameter estimation in systems with unstable target dynamics and nonlinear parametrization. IEEE Transactions on Automatic Control, 52(9):1543–1559, 2007. ISSN 2334-3303. doi: 10.1109/TAC.2007.904448.
  8. A. L. Fradkov. Speed-gradient scheme and its application in adaptive control problems. Automation and Remote Control, 40:1333–1342, 1980.
  9. Efficient learning of generalized linear and single index models with isotonic regression. arXiv:1104.2018, 2011.
  10. A natural adaptive control law for robot manipulators. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1–9, 2018. doi: 10.1109/IROS.2018.8593727.
  11. Beyond convexity – contraction and global convergence of gradient descent. arXiv:1806.06655, 2018.
  12. A variational perspective on accelerated methods in optimization. Proceedings of the National Academy of Sciences, 113(47):E7351–E7358, 2016. ISSN 0027-8424. doi: 10.1073/pnas.1614734113.
  13. On symplectic optimization. arXiv:1802.03653, 2018.
  14. A lyapunov analysis of momentum methods in optimization. arXiv:1611.02635, 2016.
  15. Nonlinear and Adaptive Control of Complex Systems. Springer Netherlands, 1999.
  16. A. L. Fradkov. Integrodifferentiating velocity gradient algorithms. Sov. Phys. Dokl., 31:97–98, 1986.
  17. Velocity-gradient algorithms in control and adaptation problems. Automation and Remote Control, 49:1533–1564, 1988.
  18. A. S. Morse. High-Order Parameter Tuners for the Adaptive Control of Linear and Nonlinear Systems, pages 339–364. Birkhäuser Boston, Boston, MA, 1992. ISBN 978-1-4757-2204-8. doi: https://doi.org/10.1007/978-1-4757-2204-8_23.
  19. Generalized momentum-based methods: A hamiltonian perspective. arXiv:1906.00436, 2019.
  20. Hamiltonian descent methods. arXiv:1809.05042, 2018.
  21. A dynamical systems perspective on nesterov acceleration. arXiv:1905.07436, 2019.
  22. Optimization with momentum: Dynamical, control-theoretic, and symplectic perspectives. arXiv:2002.12493, 2020.
  23. A continuous-time analysis of distributed stochastic gradient. Neural Computation, 32(1):36–96, 2020. doi: https://doi.org/10.1162/neco_a_01248.
  24. A differential equation for modeling nesterov’s accelerated gradient method: Theory and insights. Journal of Machine Learning Research, 17(153):1–43, 2016.
  25. Yurii Nesterov. A Method for Solving a Convex Programming Problem with Convergence Rate O⁢(1/k2)𝑂1superscript𝑘2O(1/k^{2})italic_O ( 1 / italic_k start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ). Soviet Mathematics Doklady, 26:367–372, 1983.
  26. B.T. Polyak. Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics, 4(5):1 – 17, 1964. ISSN 0041-5553. doi: https://doi.org/10.1016/0041-5553(64)90137-5.
  27. Provably correct learning algorithms in the presence of time-varying features using a variational perspective. arXiv:1903.04666, 2019.
  28. Adaptive control of continuous time systems with convex/concave parametrization. Automatica, 34(1):33 – 49, 1998. ISSN 0005-1098. doi: https://doi.org/10.1016/S0005-1098(97)00159-3.
  29. Adaptation in the presence of a general nonlinear parameterization: an error model approach. IEEE Transactions on Automatic Control, 44(9):1634–1652, 1999. ISSN 2334-3303. doi: 10.1109/9.788531.
  30. Adaptive control of nonlinearly parameterized systems with a triangular structure. Automatica, 38(1):115 – 123, 2002. ISSN 0005-1098. doi: https://doi.org/10.1016/S0005-1098(01)00173-X.
  31. A. Astolfi and R. Ortega. Immersion and invariance: a new tool for stabilization and adaptive control of nonlinear systems. IEEE Transactions on Automatic Control, 48(4):590–606, 2003.
  32. Immersion and invariance adaptive control of nonlinearly parameterized nonlinear systems . IEEE Transactions on Automatic Control, 55(9):2209–2214, 2010.
  33. Parameter estimation of nonlinearly parameterized regressions without overparameterization nor persistent excitation: Application to system identification and adaptive control. arXiv:1910.08016, 2019.
  34. Learning neural networks with two nonlinear layers in polynomial time. arXiv:1709.06010, 2017.
  35. Learning one convolutional layer with overlapping patches. arXiv:1802.02547, 2018.
  36. The implicit bias of gradient descent on separable data. J. Mach. Learn. Res., 19(1):2822–2878, 2018. ISSN 1532-4435.
  37. Implicit bias of gradient descent on linear convolutional networks. In Advances in Neural Information Processing Systems 31, pages 9461–9471. Curran Associates, Inc., 2018a.
  38. Characterizing implicit bias in terms of optimization geometry. arXiv:1802.08246, 2018b.
  39. Stochastic mirror descent on overparameterized nonlinear models: Convergence, implicit regularization, and generalization. arXiv:1906.03830, 2019.
  40. Stochastic gradient/mirror descent: Minimax optimality and implicit regularization. In International Conference on Learning Representations, 2019.
  41. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849–15854, 2019. ISSN 0027-8424. doi: 10.1073/pnas.1903070116.
  42. David G. Luenberger. Introduction to Dynamic Systems. John Wiley & Sons, 1979.
  43. Adaptive sliding controller synthesis for non-linear systems. International Journal of Control, 43(6):1631–1651, 1986. doi: https://doi.org/10.1080/00207178608933564.
  44. Ivan Tyukin. Adaptation in Dynamical Systems. Cambridge University Press, 2011.
  45. I. Yu. Tyukin. Adaptation algorithms in finite form for nonlinear dynamic objects. Automation and Remote Control, 64(6):951–974, 2003. ISSN 1608-3032. doi: https://doi.org/10.1023/A:1024141700331.
  46. Contraction metrics in adaptive nonlinear control, 2019.
  47. Learning stability certificates from data. Conference on Robot Learning (CoRL), 2020.
  48. L.M. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7(3):200 – 217, 1967. ISSN 0041-5553. doi: https://doi.org/10.1016/0041-5553(67)90040-7.
  49. Shun-ichi Amari. Natural gradient works efficiently in learning. Neural Computation, 10(2):251–276, 1998. doi: https://doi.org/10.1162/089976698300017746.
  50. Geometric robot dynamic identification: A convex programming approach. IEEE Transactions on Robotics, pages 1–18, 2019. ISSN 1941-0468. doi: 10.1109/TRO.2019.2926491.
  51. Linear matrix inequalities for physically consistent inertial parameter identification: A statistical perspective on the mass distribution. IEEE Robotics and Automation Letters, 3(1):60–67, 2018. ISSN 2377-3774. doi: 10.1109/LRA.2017.2729659.
  52. Understanding deep learning requires rethinking generalization. arXiv:1611.03530, 2016.
  53. Benign overfitting in linear regression. arXiv:1906.11300, 2019.
  54. Harmless interpolation of noisy data in regression. In 2019 IEEE International Symposium on Information Theory (ISIT), pages 2299–2303, 2019. doi: 10.1109/ISIT.2019.8849614.
  55. Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996. doi: 10.1111/j.2517-6161.1996.tb02080.x.
  56. D. Sussillo and L. F. Abbott. Generating coherent patterns of activity from chaotic neural networks. Neuron, 63(4):544–557, 2009.
  57. Learning nonlinear dynamics in efficient, balanced spiking networks using local plasticity rules. AAAI Conference on Artificial Intelligence, 2018.
  58. Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network. eLife, 6:e28295, 2017. ISSN 2050-084X.
  59. On contraction analysis for non-linear systems. Automatica, 34(6):683–696, 1998. ISSN 0005-1098. doi: http://dx.doi.org/10.1016/S0005-1098(98)00019-3.
  60. Jean-Jacques E. Slotine. Modular stability tools for distributed computation and control. International Journal of Adaptive Control and Signal Processing, 17(6):397–416, 2003. doi: 10.1002/acs.754.
  61. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45):22445–22451, 2019. ISSN 0027-8424. doi: 10.1073/pnas.1906995116. URL https://www.pnas.org/content/116/45/22445.
  62. Stable adaptive control of robot manipulators using “neural” networks. Neural Computation, 7(4):753–790, 1995. doi: 10.1162/neco.1995.7.4.753. URL https://doi.org/10.1162/neco.1995.7.4.753.
  63. Symplectic recurrent neural networks, 2019.
  64. Learning nonlinear dynamical systems from a single trajectory. arXiv:2004.14681, 2020.
  65. A class of high order tuners for adaptive systems. IEEE Control Systems Letters, 5(2):391–396, 2021.
  66. Conformal symplectic and relativistic optimization. arXiv:1903.04100, 2019.
  67. Acceleration via symplectic discretization of high-resolution differential equations. arXiv:1902.03694, 2019.
  68. Dynamic scaling and observer design with application to adaptive control. Automatica, 45(12):2883 – 2889, 2009. ISSN 0005-1098. doi: https://doi.org/10.1016/j.automatica.2009.09.013.
  69. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4):838–855, 1992a. doi: ttps://doi.org/10.1137/0330046.
  70. David Ruppert. Efficient estimations from a slowly convergent robbins–monro process. Cornell University Operations Research and Industrial Engineering Technical Report, 1988.
  71. B. Polyak and A. Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4):838–855, 1992b. doi: 10.1137/0330046.
  72. Iterate averaging as regularization for stochastic gradient descent. In Sébastien Bubeck, Vianney Perchet, and Philippe Rigollet, editors, Proceedings of the 31st Conference On Learning Theory, volume 75 of Proceedings of Machine Learning Research, pages 3222–3242. PMLR, 06–09 Jul 2018.
  73. Deep learning with elastic averaging sgd. arXiv:1412.6651, 2014.
  74. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31(3):167 – 175, 2003. ISSN 0167-6377. doi: https://doi.org/10.1016/S0167-6377(02)00231-6.
  75. Problem Complexity and Method Efficiency in Optimization. Wiley, 1983. ISBN 9780471103455.
  76. Accelerated mirror descent in continuous and discrete time. In Advances in Neural Information Processing Systems 28, pages 2845–2853. Curran Associates, Inc., 2015.
  77. Elad Hazan. Introduction to online convex optimization. Foundations and Trends in Optimization, 2(3-4):157–325, 2016. ISSN 2167-3888. doi: 10.1561/2400000013.
  78. Observability of complex systems. Proceedings of the National Academy of Sciences, 110(7):2460–2465, 2013. ISSN 0027-8424. doi: 10.1073/pnas.1215508110. URL https://www.pnas.org/content/110/7/2460.
  79. Claudio Gentile. The robustness of the p-norm algorithms. Machine Learning, 53(3):265–299, 2003. ISSN 1573-0565. doi: https://doi.org/10.1023/A:1026319107706.
Citations (34)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now