Self driving algorithm for an active four wheel drive racecar (2506.06077v1)

Abstract: Controlling autonomous vehicles at their handling limits is a significant challenge, particularly for electric vehicles with active four wheel drive (A4WD) systems offering independent wheel torque control. While traditional Vehicle Dynamics Control (VDC) methods use complex physics-based models, this study explores Deep Reinforcement Learning (DRL) to develop a unified, high-performance controller. We employ the Proximal Policy Optimization (PPO) algorithm to train an agent for optimal lap times in a simulated racecar (TORCS) at the tire grip limit. Critically, the agent learns an end-to-end policy that directly maps vehicle states, like velocities, accelerations, and yaw rate, to a steering angle command and independent torque commands for each of the four wheels. This formulation bypasses conventional pedal inputs and explicit torque vectoring algorithms, allowing the agent to implicitly learn the A4WD control logic needed for maximizing performance and stability. Simulation results demonstrate the RL agent learns sophisticated strategies, dynamically optimizing wheel torque distribution corner-by-corner to enhance handling and mitigate the vehicle's inherent understeer. The learned behaviors mimic and, in aspects of grip utilization, potentially surpass traditional physics-based A4WD controllers while achieving competitive lap times. This research underscores DRL's potential to create adaptive control systems for complex vehicle dynamics, suggesting RL is a potent alternative for advancing autonomous driving in demanding, grip-limited scenarios for racing and road safety.

• The paper demonstrates a DRL-based end-to-end control strategy using PPO to optimize lap times and enhance stability at the tire grip limit.
• It maps vehicle states to steering and independent torque commands, bypassing traditional physics-based control models.
• The study shows that RL can mitigate understeer and boost performance, offering promising insights for advanced autonomous systems.

Self-Driving Algorithm for an Active Four Wheel Drive Racecar

The exploration of efficacious control strategies for autonomous vehicles at their handling limits constitutes a salient challenge. Particularly, electric vehicles equipped with active four-wheel drive (A4WD) systems, which offer independent wheel torque control, present unique opportunities and complexities. This paper investigates the potential of Deep Reinforcement Learning (DRL) as an alternative to conventional Vehicle Dynamics Control (VDC) methodologies, which traditionally rely on intricate physics-based models and coordination strategies. Specifically, the research employs the Proximal Policy Optimization (PPO) algorithm to train an autonomous agent within the TORCS racing simulator environment, aiming for optimal lap times while operating at the tire grip limit.

The PPO algorithm facilitates the development of an end-to-end policy that directly maps vehicle-friendly states, such as velocities, accelerations, and yaw rate, to control outputs including steering angle and independent torque commands for each of the four wheels. This architecture bypasses conventional pedal inputs and explicit torque vectoring algorithms, thereby allowing the agent to implicitly learn the A4WD control logic required to enhance performance and stability. Simulation results indicate that the RL agent successfully acquires sophisticated control strategies that dynamically optimize wheel torque distribution corner-by-corner, thereby mitigating the inherent understeer characteristics of the base vehicle. The learned behaviors not only mimic, but in certain aspects related to grip utilization, potentially surpass traditional physics-based A4WD controllers, achieving competitive lap times.

The implications of this research are manifold:

1. Theoretical Implications:
   • DRL provides a viable alternative to model-based control approaches, offering a framework that automatically discovers optimal control strategies without relying on explicit vehicle models.
   • The paper contributes to the ongoing discourse on the effectiveness and applicability of end-to-end RL approaches in nonlinear, high-dimensional vehicular dynamics systems.
2. Practical Implications:
   • The demonstrated capability of RL-based controllers to enhance vehicle stability and performance, particularly in autonomous racing scenarios, suggests potential applications in the development of advanced safety and performance systems for everyday vehicles.
3. Future Developments:
   • Evaluating the generalization capabilities of the trained agent across various racetracks and differing environmental conditions remains a critical area for future investigation.
   • Extending the control architecture to encompass additional vehicle actuators, such as Active Suspension (AS), could further enhance the adaptivity and efficacy of RL-based vehicle controllers.

In summary, the paper showcases the substantial potential of DRL in creating adaptive, high-performance control systems for complex vehicle dynamics. It underscores the promise of RL methodologies as potent alternatives for advancing autonomous driving capabilities, particularly in scenarios demanding operation near or at the limits of tire adhesion. As autonomous driving systems continue to evolve, the insights gained from optimizing control strategies in demanding environments like racetracks may ultimately contribute to the broader goals of improving vehicle safety, performance, and efficiency on public roads.