Sim-to-real transfer for reinforcement learning driving planners

Demonstrate successful sim-to-real transfer of reinforcement learning-based autonomous driving planning policies trained in simulation (e.g., CARLA and nuPlan) to real-world vehicles to establish practical relevance.

Background

The work establishes strong performance in simulation for reinforcement learning planners using a simple reward formulation and large-scale PPO training on CARLA and nuPlan. However, the authors acknowledge that real-world applicability hinges on demonstrating transfer from simulation to real cars.

They explicitly defer this step to future work, indicating that establishing effective sim-to-real transfer for RL planners remains unresolved.

References

For RL to become relevant for real cars, Sim2Real transfer \citep{Miki2022SR, Kaufmann2023Nature, Zeng2024CORL, Lin2025ARXIV} needs to be demonstrated, which we leave for future work.

— CaRL: Learning Scalable Planning Policies with Simple Rewards  (2504.17838 - Jaeger et al., 24 Apr 2025) in Limitations

Future work should focus on three open problems: establishing standardized cross-benchmark evaluation protocols that fairly compare modular and end-to-end systems; developing hybrid architectures with formally verifiable safety properties; and reducing the sim-to-real gap for end-to-end learned policies through domain randomization and adaptive transfer techniques.

— End-to-End Learning vs. Modular Architectures: Comparative Insights into Autonomous Driving Systems  (2610.01746 - Kapse, 1 Oct 2026) in Section IV, Conclusion

However, simulation-to-real transfer remains a critical challenge because differences between simulated and real-world environments can lead to discrepancies in model performance, especially when dealing with rare events.

— A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform  (2610.00926 - Xu et al., 1 Oct 2026) in Section 5.2.1, “Simulation-Based Testing and Closed-Loop Validation”