TartanAviation: Image, Speech, and ADS-B Trajectory Datasets for Terminal Airspace Operations (2403.03372v1)

Abstract: We introduce TartanAviation, an open-source multi-modal dataset focused on terminal-area airspace operations. TartanAviation provides a holistic view of the airport environment by concurrently collecting image, speech, and ADS-B trajectory data using setups installed inside airport boundaries. The datasets were collected at both towered and non-towered airfields across multiple months to capture diversity in aircraft operations, seasons, aircraft types, and weather conditions. In total, TartanAviation provides 3.1M images, 3374 hours of Air Traffic Control speech data, and 661 days of ADS-B trajectory data. The data was filtered, processed, and validated to create a curated dataset. In addition to the dataset, we also open-source the code-base used to collect and pre-process the dataset, further enhancing accessibility and usability. We believe this dataset has many potential use cases and would be particularly vital in allowing AI and machine learning technologies to be integrated into air traffic control systems and advance the adoption of autonomous aircraft in the airspace.

• The paper introduces a novel open-source multimodal dataset integrating image, speech, and ADS-B trajectory data for enhanced terminal airspace operations.
• It details rigorous data collection and preprocessing methodologies across diverse airport environments and weather conditions.
• The dataset enables advancements in computer vision, time-series analysis, and speech-to-text systems for improved air traffic control communications.

TartanAviation: Comprehensive Multimodal Dataset for Enhancing Terminal Airspace Operations through AI

Introduction

The ever-increasing demand for air travel and the imminent integration of Advanced Aerial Mobility (AAM) into the National Airspace System underscores a critical need for advancements in air traffic control systems. Addressing this need, TartanAviation emerges as a novel open-source multimodal dataset aimed at fostering innovations in terminal airspace operations. This dataset offers an unparalleled perspective of the airport environment by incorporating concurrent collections of image, speech, and ADS-B trajectory data within airport boundaries. TartanAviation’s encompassing approach not only facilitates the development of AI-driven technologies for air traffic management but also aligns with the broader objective of integrating autonomous aircraft into the airspace.

Dataset Overview

Collected across towered and non-towered airfields within the US, TartanAviation provides a rich tapestry of data reflecting diverse aircraft operations, seasons, aircraft types, and weather conditions. The dataset encompasses 3.1M images, 3374 hours of air traffic control speech data, and 661 days of ADS-B trajectory data. It's a holistic resource created with rigorous filtering, processing, and validation methodologies. Moreover, the open-sourcing of the collection and preprocessing code-base significantly enhances the dataset’s accessibility and usability.

Multimodality at its Core

Vision Data

TartanAviation’s vision data, collected using an array of Sony IMX 264 cameras, portrays a wide array of scenarios including adverse weather conditions, providing over 700k aircraft labels. This real-world large-scale dataset is essential for developing robust computer vision techniques aimed at long-range object detection, crucial for aviation safety through visual detect-and-avoid (DAA) systems.

Trajectory Data

The trajectory component of TartanAviation is an extensive collection of time-series information depicting aircraft movements within terminal airspaces. It extends prior work by offering 661 days of data from both towered and non-towered airports, enabling research not only in aviation but also in broader areas such as time-series forecasting, and anomaly detection.

Speech Data

Unique to TartanAviation is its inclusion of air traffic control speech data from smaller airports, offering both towered and non-towered fields. This first-of-its-kind speech data, complemented by concurrent trajectory information, opens avenues for multi-modal speech-to-text translation and intent prediction research, tailored to the context of air traffic control communications.

Implications and Future Directions

TartanAviation stands as a testament to the growing intersection of AI and aviation, particularly in the domain of air traffic management. By offering a dataset that conjoins images, speech, and trajectory data collected in the complex environment of terminal airspace, it sets the stage for the development of AI solutions capable of enhancing traffic management efficiency and safety. The dataset supports a broad spectrum of applications from vision-based object detection and trajectory prediction to speech understanding and intent prediction in air traffic control communications. TartanAviation not only challenges existing methodologies but also encourages the exploration of multi-modal data utilisation in aviation, paving the way for advancements in both theoretical and practical aspects of AI in aviation.

Accessibility and Utilization

TartanAviation’s structured dataset, accompanied by the deployment of dataloaders and preprocessing utilities, ensures seamless integration with existing technology stacks. The provision of data in common formats encourages immediate adoption within the researcher community, facilitating rapid experimentation and iteration. This accessibility, combined with the dataset’s comprehensive nature, promises significant contributions to enhancing terminal airspace operations through AI-driven solutions.

Concluding Remarks

TartanAviation represents a significant stride towards realizing the potential of AI in revolutionizing terminal airspace operations. By providing a rich, multimodal dataset, the research community is equipped with the tools necessary to drive innovations that can shape the future of air travel and air traffic management. As the dataset continues to evolve, it will likely become an invaluable resource for researchers and practitioners alike, fostering a new era of AI-enabled advancements in aviation.