Composing MPC with LQR and Neural Network for Amortized Efficiency and Stable Control (2112.07238v3)

Abstract: Model predictive control (MPC) is a powerful control method that handles dynamical systems with constraints. However, solving MPC iteratively in real time, i.e., implicit MPC, remains a computational challenge. To address this, common solutions include explicit MPC and function approximation. Both methods, whenever applicable, may improve the computational efficiency of the implicit MPC by several orders of magnitude. Nevertheless, explicit MPC often requires expensive pre-computation and does not easily apply to higher-dimensional problems. Meanwhile, function approximation, although scales better with dimension, still requires pre-training on a large dataset and generally cannot guarantee to find an accurate surrogate policy, the failure of which often leads to closed-loop instability. To address these issues, we propose a triple-mode hybrid control scheme, named Memory-Augmented MPC, by combining a linear quadratic regulator, a neural network, and an MPC. From its standard form, we further derive two variants of such hybrid control scheme: one customized for chaotic systems and the other for slow systems. The proposed scheme does not require pre-computation and can improve the amortized running time of the composed MPC with a well-trained neural network. In addition, the scheme maintains closed-loop stability with any neural networks of proper input and output dimensions, alleviating the need for certifying optimality of the neural network in safety-critical applications.

• The paper introduces Memory-Augmented MPC (MAMPC) that integrates MPC, LQR, and NN to significantly reduce computational cost while ensuring closed-loop stability.
• It demonstrates theoretical proofs and proposes variants such as alternating-authority and way-point MAMPC to address chaotic dynamics and slow system behavior.
• Extensive numerical experiments on systems like pendulums, bicopters, and quadcopters validate the method’s efficiency gains in real-time industrial control applications.

Composing MPC with LQR and Neural Networks for Amortized Efficiency and Stable Control

The paper Composing MPC with LQR and Neural Networks for Amortized Efficiency and Stable Control addresses a significant challenge in Model Predictive Control (MPC): the high computational cost associated with solving MPC problems iteratively in real-time. To mitigate this issue, the authors propose a hybrid control scheme named Memory-Augmented MPC (MAMPC), which combines the strengths of an implicit MPC, a Linear Quadratic Regulator (LQR), and a Neural Network (NN).

The MAMPC method aims at achieving both safety and computational efficiency, making it particularly suitable for industrial robotics applications where long-term operational cost and safety are critical. Common examples include factory robotic arm manipulation and fixed-route quadcopter payload transport.

Hybrid Control Scheme

MAMPC operates by utilizing three distinct control methods dynamically:

1. MPC: This is the default mode and is used whenever stability cannot be guaranteed by the NN or LQR. The implicit MPC is computationally expensive but provides closed-loop stability and constraint satisfaction.
2. LQR: Applied when the system's state is within the region of attraction of the LQR. LQR offers computational efficiency due to its nature as a linear state feedback controller, ensuring local asymptotic stability.
3. NN: The NN, trained via supervised learning on the MPC policy data, serves as an efficient surrogate model to approximate the MPC. However, the NN’s lack of guarantees regarding convergence and bounded approximation errors is mitigated by performing forward verification before invocation.

Theoretical Contributions

The authors provide rigorous proofs to demonstrate the stability properties of the proposed control scheme. Key theoretical contributions include:

• Stability of Standard MAMPC: The standard form of MAMPC incorporates an LQR, an NN, and an MPC, ensuring that the system remains in a stable operating region by switching among these controllers based on verification of safety and performance criteria.
• Variants of MAMPC:
  • Alternating-Authority MAMPC: This variant specifically addresses chaotic systems, which are sensitive to initial conditions. By periodically switching control authority between NN and MPC, it limits the accumulation of errors.
  • Way-Point MAMPC: Designed for slow systems, this variant employs a way-point set to ensure that the trajectory remains within a safe operating region before applying the NN controller.

Numerical Validation

The authors conduct extensive numerical experiments demonstrating the efficacy of MAMPC on various robotic systems:

1. Pendulum: This experiment illustrates initial inefficiency with NN mode gradually achieving improved performance through learning.
2. Triple Pendulum: By applying the alternating-authority variant, the control scheme effectively handles the chaotic nature of the triple pendulum system.
3. Bicopter: The experiment validates the application of MAMPC in stabilizing a bicopter, showcasing the gradual reduction in computational overhead.
4. Quadcopter: The way-point variant ensures efficient and safe control of a more complex aerial vehicle, maintaining stability and reducing computational load over time.

Results and Performance

Summarizing the results, MAMPC shows a significant reduction in computational time in its converged state when compared to the implicit MPC. The amortized efficiency gains make it highly advantageous in long-term applications. Specifically, per-step running times for MAMPC modes indicate that reliance on the LQR and NN controllers can considerably reduce computation, with results summarized in detailed tables.

Practical and Theoretical Implications

The practical implications of this research are profound for industrial applications that demand real-time control combined with safety guarantees. Theoretically, the hybrid nature of MAMPC opens avenues for further exploration in control theory, particularly in the context of leveraging machine learning models (NNs) as efficient, albeit uncertain, surrogates for traditional control policies.

Future Directions

Future research directions could explore enhancing the robustness of MAMPC through advanced machine learning techniques like model-based reinforcement learning or safe kernel-based methods for even better performance under uncertainties. Moreover, automating the design of set regions and exploring higher-dimensional systems could further extend the applicability and efficiency of MAMPC.

In conclusion, by intelligently blending MPC, LQR, and Neural Networks, the proposed MAMPC not only offers a viable solution to the computational challenges in real-time control systems but also ensures stability and safety, making it a valuable contribution to the field of automation and control engineering.