Synthetic data augmentation for robotic mobility aids to support blind and low vision people (2409.11164v1)

Abstract: Robotic mobility aids for blind and low-vision (BLV) individuals rely heavily on deep learning-based vision models specialized for various navigational tasks. However, the performance of these models is often constrained by the availability and diversity of real-world datasets, which are challenging to collect in sufficient quantities for different tasks. In this study, we investigate the effectiveness of synthetic data, generated using Unreal Engine 4, for training robust vision models for this safety-critical application. Our findings demonstrate that synthetic data can enhance model performance across multiple tasks, showcasing both its potential and its limitations when compared to real-world data. We offer valuable insights into optimizing synthetic data generation for developing robotic mobility aids. Additionally, we publicly release our generated synthetic dataset to support ongoing research in assistive technologies for BLV individuals, available at https://hchlhwang.github.io/SToP.

• The paper develops a synthetic data generation pipeline using Unreal Engine 4 and NVIDIA tools to augment datasets for enhancing robotic mobility aids.
• It introduces task-specific datasets such as SToP for tactile paving and a Synthetic Street Crossing set to improve object recognition and scene description.
• Experimental results with models like YOLOv8 and Florence-2 demonstrate enhanced performance, underscoring synthetic data's role in robust model training.

Synthetic Data Augmentation for Robotic Mobility Aids for Blind and Low Vision Individuals

The paper titled "Synthetic data augmentation for robotic mobility aids to support blind and low vision people" explores the constraints and opportunities inherent in deploying synthetic data to bolster the efficacy of deep learning-based vision models used in robotic mobility aids. These aids play a critical role in enhancing the mobility and autonomy of blind and low-vision (BLV) individuals. This paper is authored by Hochul Hwang, Krisha Adhikari, Satya Shodhaka, and Donghyun Kim from the University of Massachusetts Amherst.

Background and Motivation

As the global population of people with visual impairments is projected to escalate, there is an intensified need for advanced robotic assistance devices capable of navigating complex environments. Traditional aids like guide dogs and canes, albeit useful, have limitations concerning their range of interaction and the cognitive load on users. Robotic aids, by contrast, possess the potential for broader applicability and adaptability, albeit contingent on the quality and quantity of data underpinning their vision models.

Research Focus and Methodology

The paper primarily investigates the viability and effectiveness of synthetic data created via Unreal Engine 4 in training these robotic vision models. The challenge for these systems is the scarcity of diverse, annotated datasets, pivotal for tasks like object recognition and scene understanding. Synthetic data, known for its capacity to generate extensive, varied, and controlled datasets, offers a potential remedy to these data acquisition challenges.

Key Contributions

The paper makes several noteworthy contributions:

1. Synthetic Data Generation Pipeline: A comprehensive pipeline using Unreal Engine 4 and NVIDIA Deep Learning Dataset Synthesizer is proposed for generating photorealistic and annotated datasets. This approach includes generating environments that mirror urban and park settings to accommodate various scenarios encountered by BLV users.
2. Specific Task-Oriented Datasets: The generation of two main datasets—the Synthetic Tactile-on-Paving (SToP) for tactile paving detection and a Synthetic Street Crossing Dataset for scene description—reflects a tailored approach to prevalent navigational tasks. This specialization aids in the enhancement of model robustness and task-specific performance.
3. Public Dataset Sharing: A particular focus is given to ensuring that these datasets are available for broader research applications, promoting further advancements in assistive robotic technologies. The availability of such open datasets can generate notable downstream impacts in developing more effective aids.

Experimental Results

The evaluation underscores the tangible benefits of utilizing synthetic data. Models such as YOLOv8 and Florence-2, when trained or fine-tuned with synthetic data, exhibited improved performance in tactile paving detection and scene description tasks, critical for safety navigation in street-crossing scenarios. Comparison with real-world performance also revealed that while real-world data can yield superior precision, the contribution of synthetic data remains significant and complementary.

Implications and Future Directions

This research underscores synthetic data's pivotal role in overcoming real-world data constraints, providing a scalable, diverse, and flexible foundation for training robotic mobility aids. Notably, synthetic data facilitates the simulation of varied and nuanced environments unforeseen in conventional datasets, fostering the development of more generalized and robust models.

Moreover, it encourages future explorations into the balance between synthetic and real-world data integrations, optimizing model performance and extending applicability beyond controlled environments. Future research could explore enhancing realism in synthetic datasets and advancing domain adaptation methods to close any residual performance gaps.

Conclusion

This paper advances the dialogue on how synthetic data can be strategically deployed to evolve the landscape of assistive technology for BLV individuals. By addressing both the technical and practical aspects of dataset generation and model training, it paves the way for further innovations in developing autonomous systems that are not only technically proficient but also markedly safer and more reliable for end-user engagement.