A Passivity-Based Method for Accelerated Convex Optimisation
Abstract: This study presents a constructive methodology for designing accelerated convex optimisation algorithms in continuous-time domain. The two key enablers are the classical concept of passivity in control theory and the time-dependent change of variables that maps the output of the internal dynamic system to the optimisation variables. The Lyapunov function associated with the optimisation dynamics is obtained as a natural consequence of specifying the internal dynamics that drives the state evolution as a passive linear time-invariant system. The passivity-based methodology provides a general framework that has the flexibility to generate convex optimisation algorithms with the guarantee of different convergence rate bounds on the objective function value. The same principle applies to the design of online parameter update algorithms for adaptive control by re-defining the output of internal dynamics to allow for the feedback interconnection with tracking error dynamics.
- Su, W., Boyd, S., and Candès, E. J., “A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights,” Journal of Machine Learning Research, Vol. 17, No. 153, 2016, pp. 1–43. http://jmlr.org/papers/v17/15-084.html.
- Gaudio, J. E., Annaswamy, A. M., Bolender, M. A., Lavretsky, E., and Gibson, T. E., “A Class of High Order Tuners for Adaptive Systems,” IEEE Control Systems Letters, Vol. 5, No. 2, 2021, pp. 391–396. doi:10.1109/LCSYS.2020.3002513.
- Suh, J. J., Roh, G., and Ryu, E. K., “Continuous-Time Analysis of Accelerated Gradient Methods via Conservation Laws in Dilated Coordinate Systems,” Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162, edited by K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, PMLR, 2022, pp. 20640–20667. https://proceedings.mlr.press/v162/suh22a.html.
- Hu, B., and Lessard, L., “Control Interpretations for First-Order Optimization Methods,” American Control Conference, Seattle, WA, USA, 2017a, pp. 3114–3119. doi:10.23919/ACC.2017.7963426.
- Hu, B., and Lessard, L., “Dissipativity Theory for Nesterov’s Accelerated Method,” Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 70, edited by D. Precup and Y. W. Teh, PMLR, 2017b, pp. 1549–1557. https://proceedings.mlr.press/v70/hu17a.html.
- doi:10.1109/MCS.2022.3157115.
- Kottenstette, N., McCourt, M. J., Xia, M., Gupta, V., and Antsaklis, P. J., “On Relationships Among Passivity, Positive Realness, and Dissipativity in Linear Systems,” Automatica, Vol. 50, No. 4, 2014, pp. 1003–1016. doi:10.1016/j.automatica.2014.02.013.
- Madeira, D. d. S., and Adamy, J., “On the Equivalence Between Strict Positive Realness and Strict Passivity of Linear Systems,” IEEE Transactions on Automatic Control, Vol. 61, No. 10, 2016, pp. 3091–3095. doi:10.1109/TAC.2015.2506993.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.