Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Output-feedback Synthesis Orbit Geometry: Quotient Manifolds and LQG Direct Policy Optimization (2403.17157v3)

Published 25 Mar 2024 in math.OC, cs.SY, and eess.SY

Abstract: We consider direct policy optimization for the linear-quadratic Gaussian (LQG) setting. Over the past few years, it has been recognized that the landscape of dynamic output-feedback controllers of relevance to LQG has an intricate geometry, particularly pertaining to the existence of degenerate stationary points, that hinders gradient methods. In order to address these challenges, in this paper, we adopt a system-theoretic coordinate-invariant Riemannian metric for the space of dynamic output-feedback controllers and develop a Riemannian gradient descent for direct LQG policy optimization. We then proceed to prove that the orbit space of such controllers, modulo the coordinate transformation, admits a Riemannian quotient manifold structure. This geometric structure--that is of independent interest--provides an effective approach to derive direct policy optimization algorithms for LQG with a local linear rate convergence guarantee. Subsequently, we show that the proposed approach exhibits significantly faster and more robust numerical performance as compared with ordinary gradient descent.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (22)
  1. B. Hu, K. Zhang, N. Li, M. Mesbahi, M. Fazel, and T. Başar, “Toward a theoretical foundation of policy optimization for learning control policies,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 6, pp. 123–158, 2023.
  2. B. Hu and Y. Zheng, “Connectivity of the feasible and sublevel sets of dynamic output feedback control with robustness constraints,” IEEE Control Systems Letters, vol. 7, pp. 442–447, 2022.
  3. J. Bu, A. Mesbahi, and M. Mesbahi, “On topological and metrical properties of stabilizing feedback gains: the mimo case,” arXiv preprint arXiv:1904.02737, 2019.
  4. J. Bu, A. Mesbahi, and M. Mesbahi, “On topological properties of the set of stabilizing feedback gains,” IEEE Transactions on Automatic Control, vol. 66, no. 2, pp. 730–744, 2020.
  5. Y. Tang, Y. Zheng, and N. Li, “Analysis of the optimization landscape of Linear Quadratic Gaussian (LQG) control,” in Learning for Dynamics and Control, pp. 599–610, PMLR, 2021.
  6. J. C. Doyle, “Guaranteed margins for LQG regulators,” IEEE Transactions on automatic Control, vol. 23, no. 4, pp. 756–757, 1978.
  7. S.-I. Amari, “Natural gradient works efficiently in learning,” Neural computation, vol. 10, no. 2, pp. 251–276, 1998.
  8. B. Mishra and R. Sepulchre, “Riemannian preconditioning,” SIAM Journal on Optimization, vol. 26, no. 1, pp. 635–660, 2016.
  9. S. Talebi and M. Mesbahi, “Policy optimization over submanifolds for constrained feedback synthesis,” arXiv preprint arXiv:2201.11157, 2022.
  10. R. Kalman, “Algebraic geometric description of the class of linear systems of constant dimension,” in 8th Annual Princeton Conference on Information Sciences and Systems, vol. 3, Princeton University Princeton, NJ, 1974.
  11. M. Hazewinkel, “Moduli and canonical forms for linear dynamical systems ii: The topological case,” Mathematical Systems Theory, vol. 10, no. 1, pp. 363–385, 1976.
  12. B. Afsari and R. Vidal, “Bundle reduction and the alignment distance on spaces of state-space lti systems,” IEEE Transactions on Automatic Control, vol. 62, no. 8, pp. 3804–3819, 2017.
  13. M. Hazewinkel, “(fine) moduli (spaces) for linear systems: what are they and what are they good for?,” in Geometrical Methods for the Theory of Linear Systems: Proceedings of a NATO Advanced Study Institute and AMS Summer Seminar in Applied Mathematics held at Harvard University, Cambridge, Mass., June 18–29, 1979, pp. 125–193, Springer, 1980.
  14. Springer, 2006.
  15. N. Boumal, An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023.
  16. Princeton University Press, 2008.
  17. J. M. Lee, Smooth Manifolds. Springer, 2012.
  18. Springer, 2018.
  19. John Wiley & Sons, 1996.
  20. N. Boumal, P.-A. Absil, and C. Cartis, “Global rates of convergence for nonconvex optimization on manifolds,” IMA Journal of Numerical Analysis, vol. 39, no. 1, pp. 1–33, 2019.
  21. P. S. Krishnaprasad and C. F. Martin, “On families of systems and deformations,” International Journal of Control, vol. 38, no. 5, pp. 1055–1079, 1983.
  22. P. S. Krishnaprasad, Geometry of Minimal Systems and the Identification Problem. Harvard University Press, 1977.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now