---
title: Deep Koopman-Based EMPC
url: https://www.emergentmind.com/topics/deep-koopman-based-economic-model-predictive-control-empc
type: topic
---

# Deep Koopman-Based EMPC

Deep Koopman-Based Economic Model Predictive Control (EMPC) is an advanced control paradigm that synthesizes deep learning-based approximations of the Koopman operator with convex economic model predictive control, often refined through reinforcement learning. The methodology replaces nonlinear process models with lifted linear surrogates that retain the expressive power necessary to capture complex system behaviors while enabling tractable optimization via real-time quadratic programming. This framework is designed to provide high-fidelity economic optimization with strong closed-loop constraint satisfaction, even for highly nonlinear, high-dimensional, or partially observable processes.

## 1. Koopman Operator Theory and Deep Lifting for Control

The Koopman operator provides a linear (though infinite-dimensional) reformulation of nonlinear dynamical systems by describing the evolution of observables rather than states. In practice, deep Koopman-based approaches construct a finite-dimensional linear approximation by learning a nonlinear encoder $\phi_\theta$ that lifts system states $x_t \in \mathbb{R}^n$ into a higher-dimensional latent space $z_t = \phi_\theta(x_t)\in\mathbb{R}^N$. The system dynamics in the latent (Koopman) space are modeled as
\[
z_{t+1} = A_\theta z_t + B_\theta u_t
\]
where $(A_\theta, B_\theta)$ are trainable Koopman matrices and $u_t\in\mathbb{R}^m$ is the control input. For practical output prediction, a decoder $\psi_\theta$ reconstructs system states via
\[
\hat{x}_{t+1} = \psi_\theta(z_{t+1})
\]
Both encoder and decoder are typically parameterized as multi-layer perceptrons (MLPs) with tanh or ELU activations; latent dimension $N$ is selected to maximize model expressiveness while maintaining real-time feasibility, with values ranging from $N=8$ for CSTRs [2403.14425, 2503.18787] to $N=26\text{–}60$ for industrial systems [2405.12478, 2511.04522, 2511.04437].

The full deep Koopman surrogate identification loss function balances reconstruction, lifting, and state-prediction terms:
\[
L_\text{ID} = \lambda_1 L_\text{rec} + \lambda_2 L_\text{lift} + \lambda_3 L_\text{state}
\]
where
- $L_\text{rec} = \|\psi_\theta(\phi_\theta(x_t)) - x_t\|_2^2$
- $L_\text{lift} = \|\phi_\theta(x_{t+1}) - (A_\theta \phi_\theta(x_t) + B_\theta u_t)\|_2^2$
- $L_\text{state} = \|\psi_\theta(A_\theta \phi_\theta(x_t) + B_\theta u_t) - x_{t+1}\|_2^2$

Advanced variants incorporate input-output encoders, time-varying operator parameters via deep networks, and history-dependent lifting via LSTMs to address high-dimensional and partially observed systems [2504.06818].

## 2. Economic Model Predictive Control Formulation in Lifted Space

EMPC seeks to minimize economic costs over a receding horizon, accommodating nonlinear process constraints and real-world operational objectives. In the deep Koopman-based context, the receding-horizon OCP is formulated with lifted dynamics:
\[
\begin{aligned}
&\min_{\{u_k\}_{k=0}^{N-1}} J(\{u_k\}) = \sum_{k=0}^{N-1} \ell_e(x_k, u_k) + V_f(x_N)\\
\text{subject to} \quad 
& z_0 = \phi_\theta(x_0), \\
& z_{k+1} = A_\theta z_k + B_\theta u_k, \\
& x_k = \psi_\theta(z_k), \\
& x_k \in \mathcal{X}, \quad u_k \in \mathcal{U}
\end{aligned}
\]
with $\ell_e$ denoting an economic stage cost (e.g., energy, material loss), and $V_f(x_N)$ a terminal cost such as a quadratic penalty on deviation from steady state.

For partially observed or output-based systems, the entire EMPC OCP—including input, output, and economic cost quadratic decoders—remains a convex quadratic program in the lifted variables, supporting high-dimensional problems in real time [2405.12478, 2504.06818].

Slack variables are systematically incorporated to soften hard constraints and ensure feasibility under disturbances or modeling error:
\[
\ell_e(z_k, u_k; \varepsilon_k) = c_\text{energy} u_{k,3} + c_\text{material} (\varepsilon_{k,1} + \varepsilon_{k,3}) + ...
\]
with corresponding bounds on control, output, and slack variables.

## 3. End-to-End RL Refinement and Differentiable Optimization

A defining feature of recent deep Koopman-EMPC frameworks is the end-to-end refinement of surrogate and control policy via reinforcement learning algorithms such as Proximal Policy Optimization (PPO) or Short-Horizon Actor-Critic (SHAC) [2505.08122, 2403.14425, 2503.18787, 2511.04522]. The controller is reinterpreted as a differentiable policy,
\[
\pi_\theta(x_t) = u_t^*,
\]
where $u_t^*$ is the first control of the EMPC solution, and all model and policy parameters are co-optimized with respect to the closed-loop reward signal (e.g., economic savings minus constraint violation penalties). Differentiable convex programming tools such as cvxpylayers or OptNet are utilized to allow gradient backpropagation through the EMPC layer, which is critical for joint model-controller learning.

PPO updates are based on the clipped surrogate loss,
\[
L^\text{PPO}(\theta) = \mathbb{E}[ \min \{ r_t(\theta) \hat{A}_t, \text{clip}(r_t(\theta), 1-\epsilon, 1+\epsilon) \hat{A}_t\} ],
\]
where $r_t(\theta)$ is the policy probability ratio, $\hat{A}_t$ the estimated advantage, and $\epsilon$ typically $0.1$–$0.2$.

Hybrid model-based RL schemes (Editor’s term) employ batch policy optimization in both real and simulated (physics-informed) environments to further improve sample efficiency and accelerate closed-loop performance [2503.18787].

## 4. Benchmark Case Studies and Quantitative Performance

Deep Koopman-EMPC has been benchmarked on a broad class of nonlinear processes including continuous stirred tank reactors (CSTRs) [2505.08122, 2403.14425, 2503.18787], large-scale air separation units (ASU) [2511.04522], pasteurization units [2511.04437], water treatment plants [2405.12478], and shipboard carbon capture systems [2504.06818]. Case studies consistently demonstrate the following:

- **Superior constraint handling:** RL-refined Koopman controllers eliminate or dramatically reduce constraint violations versus MLP and system ID-only surrogates. For example, Koopman-RL achieves constraint violation rates of $0.22–0.39\%$ compared to $6–36\%$ for neural or system ID policies [2505.08122, 2403.14425, 2511.04522].
- **Improved economic cost:** EMPC with RL-tuned Koopman surrogates yields lower or comparable operating cost; for instance, a $32\%$ cost reduction and $10.2\%$ steady-state energy savings in a pasteurization unit vs. subspace identification [2511.04437], and $1$–$7\%$ cost improvement for shipboard PCC process vs. PI and RL policies [2504.06818].
- **Sample efficiency and convergence:** Physics-informed Koopman-EMPC converges in $200$–$500$ real steps, compared to $2000+$ for pure data-driven MLP controllers [2503.18787].
- **Real-time feasibility:** With latent dimensions $N=8$–$60$ and convex quadratic programs, solve times are $\sim$ tens to hundreds of milliseconds per step, even for large state spaces [2405.12478, 2504.06818, 2511.04522].

| Controller                 | Economic cost | Constraint viol. [%] | Avg. compute [s] |
|----------------------------|---------------|----------------------|------------------|
| Deep Koopman-EMPC (RL)     | 0.90–0.94×    | $\leq$0.4            | 0.03–0.7         |
| System ID Koopman-EMPC     | 0.90–0.92×    | 8.8–36.2             | 0.03–0.7         |
| Black-box MLP (PPO)        | 0.88–0.96×    | 6–16                 | 0.03–0.7         |

*Table: Representative closed-loop benchmark results from CSTR, WWTP, ASU, and shipboard PCC studies [2403.14425, 2505.08122, 2405.12478, 2503.18787, 2511.04522, 2504.06818].*

## 5. Practical Implementation and Design Guidelines

**Model architecture:**
- Encoders and decoders: 3–5 layer MLPs, 64–256 units, tanh or ELU.
- Latent dimension: $N=8$–$60$ (trade-off: accuracy vs. speed).
- Koopman matrices $(A,B)$: learned directly, sometimes structured for controllability.

**Optimization and learning:**
- SI pretraining: Collect random and OCP-generated data, minimize $L_\text{ID}$.
- RL refinement: PPO or SHAC, value loss weight $0.5$, entropy $0.01$.

**EMPC hyperparameters:**
- Horizon: $N=12$–$36$ (3–9 h typical).
- Cost coefficients: Empirically set based on economics and violation priorities.
- Slack penalties: $M=10^4$ or larger to discourage infeasibility.
- QP solvers: OSQP, GUROBI, or cvxpylayers for differentiability.

**Deployment:**
- Closed-loop: At each step, encode current state, solve MPC/QP, apply optimal $u_0$.
- Real-time constraints are satisfied with sub-second computation for dimensions up to $h=40$ [2504.06818].

**Robustness:**
- Slack variables guarantee feasibility under disturbances.
- Kalman filtering or moving-horizon estimation can be used to address partial observability.

## 6. Extensions, Scalability, and Limitations

Deep Koopman-EMPC is extensible to:
- Large-scale systems and partially observed processes via output-based lifting and history-dependent encoders [2405.12478, 2504.06818, 2511.04522].
- Uncertainty and robustness: Integrating robust or stochastic Koopman-model predictive control.
- Physics-informed modeling: Using PINNs or hybrid models to constrain surrogates and increase data efficiency [2503.18787].
- Online adaptation: Continual fine-tuning via differentiable solvers.

Limitations include the need for careful model selection to avoid overfitting to simulator dynamics when deploying on real-world plants, especially when measurements are limited or dynamic regimes shift rapidly [2511.04522]. Future directions cited include real-plant validation and model-based RL integration.

## 7. Comparative Analysis and Synthesis

Across multiple studies, deep Koopman-based EMPC demonstrates a recurring theme: embedding nonlinear system dynamics into a well-structured lifted space, optimized both for predictive accuracy and for the economic control task. This yields convex OCPs amenable to real-time solutions and enables end-to-end RL refinement that outperforms both classical subspace-identification and generic neural approaches in constraint satisfaction and operational cost. Differentiable optimization layers (e.g., cvxpylayers) are now standard for enabling joint training of encoders, Koopman dynamics, cost decoders, and MPC policies.

Practical guidelines stress two-stage architectures (nonlinear decoder for identification, linear decoder for control), high-fidelity step-response training data, explicit economic cost encoding, and slack-augmented feasibility. The resulting controllers have been deployed and benchmarked in challenging domains such as wastewater treatment [2405.12478], pasteurization [2511.04437], air separation [2511.04522], and energy-intensive chemical reactors [2503.18787], achieving consistently strong performance metrics in both economic and constraint objectives.

Source: https://www.emergentmind.com/topics/deep-koopman-based-economic-model-predictive-control-empc