I see you: A Vehicle-Pedestrian Interaction Dataset from Traffic Surveillance Cameras

Abstract: The development of autonomous vehicles arises new challenges in urban traffic scenarios where vehicle-pedestrian interactions are frequent e.g. vehicle yields to pedestrians, pedestrian slows down due approaching to the vehicle. Over the last years, several datasets have been developed to model these interactions. However, available datasets do not cover near-accident scenarios that our dataset covers. We introduce I see you, a new vehicle-pedestrian interaction dataset that tackles the lack of trajectory data in near-accident scenarios using YOLOv5 and camera calibration methods. I see you consist of 170 near-accident occurrences in seven intersections in Cusco-Peru. This new dataset and pipeline code are available on Github.