Mixed Traffic Control and Coordination from Pixels

Abstract: Traffic congestion is a persistent problem in our society. Previous methods for traffic control have proven futile in alleviating current congestion levels leading researchers to explore ideas with robot vehicles given the increased emergence of vehicles with different levels of autonomy on our roads. This gives rise to mixed traffic control, where robot vehicles regulate human-driven vehicles through reinforcement learning (RL). However, most existing studies use precise observations that require domain expertise and hand engineering for each road network's observation space. Additionally, precise observations use global information, such as environment outflow, and local information, i.e., vehicle positions and velocities. Obtaining this information requires updating existing road infrastructure with vast sensor environments and communication to potentially unwilling human drivers. We consider image observations, a modality that has not been extensively explored for mixed traffic control via RL, as the alternative: 1) images do not require a complete re-imagination of the observation space from environment to environment; 2) images are ubiquitous through satellite imagery, in-car camera systems, and traffic monitoring systems; and 3) images only require communication to equipment. In this work, we show robot vehicles using image observations can achieve competitive performance to using precise information on environments, including ring, figure eight, intersection, merge, and bottleneck. In certain scenarios, our approach even outperforms using precision observations, e.g., up to 8% increase in average vehicle velocity in the merge environment, despite only using local traffic information as opposed to global traffic information.

• The paper demonstrates that reinforcement learning using local image data achieves comparable or superior traffic control to precise sensor methods.
• The methodology models mixed traffic as a POMDP using bird’s-eye view inputs, resulting in up to an 8% increase in vehicle velocity during merges.
• The results imply that leveraging ubiquitous image data can effectively mitigate congestion while reducing the cost and complexity of traditional traffic management systems.

Mixed Traffic Control and Coordination from Pixels

Introduction

The paper "Mixed Traffic Control and Coordination from Pixels" (2302.09167) addresses a novel approach to traffic management in mixed autonomy environments, introducing robot vehicles (RVs) that utilize image data as a modality for reinforcement learning (RL) based control policies. Traditional traffic management strategies have been inadequate for resolving increasing congestion on modern roads. Therefore, leveraging the varying autonomy levels of vehicles presents a promising alternative. Existing methods for RV control often rely on precise, global observations involving extensive sensor networks and communication with human-driven vehicles (HVs), perceived as cost-intensive and technically demanding. This paper proposes the use of locally captured, ubiquitous image data as a more realistic and practical alternative for effective mixed traffic coordination, achieving performance on par with, and sometimes superior to, methods using precise observations.

Methodology

The authors model the mixed traffic coordination problem as a Partially Observable Markov Decision Process (POMDP), utilizing bird's-eye view images as input to the RL policies. These images are less cumbersome to acquire, as they require minimal changes to existing road sensor infrastructure, and avoid the need for vehicle-to-everything (V2X) communications. This approach is tested across five typical road environments: ring, figure eight, intersection, merge, and bottleneck, using experimental scenarios designed to reflect common real-world driving situations.

Road Environments and Observations

Each environment introduces unique challenges. For instance, the ring environment aims to prevent stop-and-go traffic waves caused by disturbances among HVs. Here, RVs reduce these phenomena effectively by using a local-view image instead of relying on exact velocities and positions from precise observations.

Similarly, in the figure eight environment, which simulates intersection behavior with potential queue formations, the RVs' goal is to optimize the velocity of all vehicles. The approach of using static, localized images scaled to capture relevant vehicular data proves effective, delivering performances on par with more complex observation methods.

Figure 1: We experiment on five mixed traffic control environments (bottleneck shown in Fig~\ref{fig:hetero_bn}).

For intersections characterized by challenging east-west traffic flow problems, the RVs succeed in minimizing queue lengths and improving traffic flow using image observations focused on the center of intersections.

In the merge scenario, a significant increase in average vehicle velocities (up to 8%) was observed compared to precise observation methodologies, highlighting the potential for image-based RV systems to enhance road performance without requiring full traffic information.

Figure 2: Bottleneck environment with heterogeneous human-driven traffic. We add motorcycles, public buses, semi-trucks, and delivery trucks alongside regular passenger vehicles.

Finally, in the heterogeneous traffic conditions of a bottleneck environment, the authors introduce varied vehicle types, showing that even without direct access to average velocity and positional data across the environment, RVs maintain effectiveness in improving throughput and mitigating capacity drops.

Results and Analysis

The empirical evaluation suggests that RVs leveraging image-based observations attain traffic control efficacy comparable to, or better than, RVs using detailed, highly instrumented observation feeds. By broadening the training conditions with varied road configurations, the findings indicate an adaptability of the image-based RL approach to different congestion levels and traffic scenarios.

Figure 3: LEFT: An RV using image observations prevents stop-and-go waves. RIGHT: Mixed traffic control performance using image observations.

Moreover, the experiments demonstrate the potential to achieve substantial enhancements in traffic conditions without the intensive infrastructural overhead associated with precise observation systems. The study of using only position information (Section \ref{sec:onlypos}) corroborates the feasibility of lightweight, local sensory inputs in producing significant traffic management outcomes.

Conclusion

The study presents compelling evidence that end-to-end trained RVs, guided by image observations, offer a viable solution to current traffic congestion challenges. The results challenge the traditional reliance on precise, global traffic metrics and emphasize the utility of harnessing existing visual data streams from road infrastructure and vehicles themselves.

Future work may explore the extension of the current approach to more complex and dynamically changing environments, further integration of predictive models into observation spaces, and enhancements to the resilience against data perturbations or adversarial attacks. As traffic situations become more intricate with the proliferation of autonomous technologies, the foundational insights from this study pave the way for scalable, efficient urban traffic systems.