Learning Predictive Control with Deep Koopman Operators for Autonomous Vehicle Motion Planning

Abstract: Model Predictive Control (MPC) is widely used for autonomous-vehicle (AV) motion planning, but its real-time applicability is often limited by the need for accurate models and online solution of nonlinear, nonconvex optimization problems in dynamic road environments. Actor-critic reinforcement learning offers a promising alternative for online policy generation, yet its policy-learning process often lacks explicit control-theoretic structure. This article proposes a learning predictive control (LPC) framework with deep Koopman operators for efficient real-time motion planning under nonconvex constraints. To address nonlinear and uncertain vehicle dynamics, a deep-Koopman-based predictor is used to lift the system into an interpretable linear observable space in a data-driven manner. Unlike traditional MPC, which computes open-loop control sequences, the proposed LPC framework yields a closed-loop state-feedback policy within each prediction interval through receding-horizon actor-critic learning. To ensure safety under nonconvex environmental constraints, LPC constructs convex local surrogate representations of obstacles and defines corresponding potential-field functions. These functions and their gradients are directly embedded into the actor-critic structure, enabling efficient, safety-aware policy learning. Extensive simulations and real-world experiments on the HongQi-EHS3 platform demonstrate favorable performance in diverse obstacle-avoidance scenarios in terms of safety, computational efficiency, and driving comfort, compared with benchmark methods such as CBF-MPC and LMPCC.

• The paper presents a novel predictive control framework integrating deep Koopman models to linearize nonlinear vehicle dynamics and enable real-time planning.
• It leverages receding-horizon actor–critic reinforcement learning with convex obstacle surrogates to ensure safety under nonconvex constraints.
• Experimental results show improved safety margins, computational efficiency (5–8 ms per cycle), and smoother driving performance compared to traditional MPC.

Learning Predictive Control with Deep Koopman Operators for Autonomous Vehicle Motion Planning

Motivation and Background

Autonomous vehicle (AV) motion planning in dynamic, obstacle-rich environments requires real-time control policies that are both safe and computationally efficient. Traditional Model Predictive Control (MPC) methods, though widely used for trajectory planning, are hampered by the need for accurate models and the computational intractability of online nonlinear, nonconvex optimization—especially under changing road and traffic conditions. Actor–critic reinforcement learning (RL) offers adaptable online policy generation but historically lacks the explicit incorporation of safety constraints and struggles with interpretability and sample efficiency.

Recent progress in Koopman operator theory provides an avenue for data-driven modeling of nonlinear systems by lifting them to higher-dimensional spaces where their evolution is approximately linear. This property can potentially mitigate the computational challenges of nonlinear control. Building on this, the paper proposes a Learning Predictive Control (LPC) framework leveraging deep Koopman operators for AV motion planning under nonconvex safety constraints, integrating the strengths of both MPC and RL paradigms.

Framework Architecture

The proposed LPC framework unifies deep Koopman-based vehicle modeling with a receding-horizon actor–critic learning mechanism to achieve fast, safety-aware closed-loop control policy synthesis.

Deep Koopman Vehicle Dynamics Model

Central to the method is a deep neural network-based Koopman model which lifts the nonlinear, uncertain vehicle dynamics into a latent observable space where they can be represented by linear operators $(A, B)$. An encoder network maps vehicle states into the Koopman space, linear dynamics are applied, and a decoder network predicts back to the original state space. The network is trained using a multi-step prediction loss to achieve accuracy over extended horizons, supporting the quality of receding-horizon planning and online policy updates.

Unified Safety-Aware Actor–Critic Learning

Obstacle avoidance and road-boundary compliance are handled by constructing convex local surrogate representations of obstacles (Convex Feasible Set, CFS approach), from which potential-field functions are derived. These potential fields, together with their gradients, are embedded directly as basis functions in both actor and critic networks, allowing safety shaping to be reflected throughout the RL policy learning and evaluation process.

Figure 2: Illustration of convex representation for obstacle constraints.

Figure 1: LPC framework integration of deep Koopman models and actor–critic learning, merging obstacle perception, prediction, and online policy refinement.

Algorithmic Details

The LPC policy synthesis proceeds over finite-time receding horizons. At each time step:

• The Koopman model predicts the future observable states given a sequence of candidate actions.
• The stage and terminal costs incorporate state deviation, control effort, and the soft safety penalties from road/obstacle potential fields.
• Actor–critic iteration updates both the value function and policy, with the actor producing actions to minimize expected cumulative cost (inclusive of safety).
• Only the first action of each optimized sequence is applied; the process repeats at the next time step with updated perception.

Strong claims include polynomial per-cycle computational complexity and closed-loop stability within each planning interval (subject to Koopman model accuracy and properly chosen potential field gains).

Quantitative Evaluations

The proposed method was subjected to extensive simulation and real-world validation, benchmarking against Control Barrier Function-based MPC (CBF-MPC) and Local Model Predictive Contour Control (LMPCC) across four representative scenarios: static and dynamic obstacle avoidance (straight roads), as well as overtaking on straight and curved roads.

Figure 3: Obstacle avoidance paths generated by the proposed method, CBF-MPC, and LMPCC in Scenario I (stationary obstacles on a straight road).

Figure 6: Lateral displacement and heading angle errors showcasing tracking behavior during obstacle avoidance in Scenario I.

Figure 4: Vehicle obstacle avoidance under dynamic obstacles on a straight road, illustrating anticipatory behavior and early avoidance maneuvers using the proposed method.

Figure 5: Lateral displacement and heading angle tracking errors for dynamic obstacle scenario (Scenario II).

Figure 7: Overtaking paths in Scenario III (straight road), showing how the proposed method preserves safety margins by prolonging stay in the overtaking lane.

Figure 8: Lateral error and heading deviation during overtaking for Scenario III.

Figure 9: Vehicle overtaking trajectories on a curved road (Scenario IV), with the proposed method achieving safe, smooth lane changes under challenging geometry.

Numerical Highlights and Comparative Results

• Safety: Across all scenarios, the proposed method maintains higher minimum safety margins ($I_s$ index) compared to CBF-MPC and LMPCC, especially notable in dynamic and curved-road settings where late avoidance or premature lane return by baselines is problematic.
• Computational Efficiency: LPC achieves per-cycle computation latencies of $\approx$5–8 ms (well under real-time thresholds), whereas CBF-MPC and LMPCC require hundreds to thousands of milliseconds—missing real-time requirements by an order of magnitude.
• Driving Comfort: Owing to earlier and smoother avoidance/return maneuvers, the average trajectory curvature (comfort metric $I_\rho$) is consistently lower by 7–27% compared to benchmarks.
• Real-World Tests: On the HongQi-EHS3 platform, the method demonstrated safe, smooth, and real-time performance not just on structured roads but also during challenging maneuvers on curved, mixed-surface roads.

Theoretical and Practical Implications

From a theoretical lens, the work reinforces the viability of Koopman-based lifted linear modeling as a surrogate for nonlinear vehicle dynamics when combined with actor–critic learning. The use of convex obstacle surrogates and embedded safety gradients directly in the costates and actions unifies safety and optimality within the learning process, rather than relegating safety to external filters or post-hoc corrections.

Practically, the real-time feasibility and empirical robustness to sim-to-real gaps (demonstrated even through surface changes) suggest strong potential for deployment in production AV systems. The method’s explicit, closed-loop state-feedback structure affords greater reactivity to disturbance and sensing errors compared to open-loop MPC optimization.

Limitations and Future Directions

Several open challenges remain. The framework's robustness is fundamentally bounded by the Koopman model's accuracy, and its current safety properties are enforced through soft penalties rather than formal certificates. Broader operating-domain validation (e.g., wider friction variability, payloads, or unseen traffic scenarios) is warranted. Future work may focus on:

• Adaptive online tuning of safety parameters in response to changing conditions.
• Hybrid architectures combining hard constraint enforcement with learned safety shaping.
• Broader scaling to multi-agent or interactive scenarios via distributed Koopman modeling and policy composition.

Conclusion

This work presents a computationally efficient, safety-aware learning predictive control framework for AV motion planning that synergistically combines deep Koopman models for vehicle dynamics with receding-horizon actor–critic RL, directly embedding safety shaping into policy learning. Both simulated and real-world evidence establishes superior safety margins, comfort, and real-time feasibility compared to dominant MPC-based benchmarks. The approach advances both theoretical methodology for safe learning-based control and practical capabilities for autonomous driving in dynamic, uncertain environments.

Reference:

"Learning Predictive Control with Deep Koopman Operators for Autonomous Vehicle Motion Planning" (2606.08136)