The Mobile AR Sensor Logger for Android and iOS Devices

Abstract: In recent years, commodity mobile devices equipped with cameras and inertial measurement units (IMUs) have attracted much research and design effort for augmented reality (AR) and robotics applications. Based on such sensors, many commercial AR toolkits and public benchmark datasets have been made available to accelerate hatching and validating new ideas. To lower the difficulty and enhance the flexibility in accessing the rich raw data of typical AR sensors on mobile devices, this paper present the mobile AR sensor (MARS) logger for two of the most popular mobile operating systems, Android and iOS. The logger highlights the best possible synchronization between the camera and the IMU allowed by a mobile device, and efficient saving of images at about 30Hz, and recording the metadata relevant to AR applications. This logger has been tested on a relatively large spectrum of mobile devices, and the collected data has been used for analyzing the sensor characteristics. We see that this application will facilitate research and development related to AR and robotics, so it has been open sourced at https://github.com/OSUPCVLab/mobile-ar-sensor-logger.

• The paper demonstrates that the MARS Logger efficiently synchronizes visual and inertial data to support robust SLAM and AR applications.
• It leverages native APIs on Android (camera2) and iOS (AVFoundation) to capture data streams at around 30Hz for visuals and up to 100Hz for inertial sensors.
• Empirical trials across multiple devices validate its performance, offering a practical and open-source solution for advancing AR and robotics research.

An Analysis of "The Mobile AR Sensor Logger for Android and iOS Devices"

The paper presents a detailed exposition of the Mobile AR Sensor (MARS) Logger, developed to facilitate synchronized visual and inertial data acquisition on Android and iOS devices. The initiative addresses existing complexities in accessing comprehensive Augmented Reality (AR) sensor data from mobile devices, particularly in educational and exploratory contexts within AR and robotics research.

The MARS logger capitalizes on the proliferation and affordability of mobile devices, leveraging their systemic capabilities—cameras and Inertial Measurement Units (IMUs)—to provide an accessible platform for Simultaneous Localization and Mapping (SLAM) applications. With the growth of commercially available AR technologies, such as ARCore and ARKit, this logger is poised to catalyze developments in applications that demand high integrity and synchronization of data.

Technical Overview

The MARS logger is designed utilizing standard mobile OS APIs. On Android, it efficiently implements the camera2 API for capturing high-quality visual and metadata information at approximately 30 Hz while recording inertial data through a SensorEventListener. The logger reconciles discrepancies in timebases between camera and IMU data acquisition by establishing a common time source or facilitating temporal corrections during offline processing.

For iOS devices, the logger employs the AVFoundation framework to capture frames and inertial data, synchronizing timestamps accurately by utilizing system-provided time reference conversion functionalities. Handling these processes efficiently ensures that visual and inertial data are time-congruent, which is critical in optimizing SLAM performance.

Experimental Results

The paper includes substantial empirical data underscoring the logger’s technical robustness. Through trials conducted on five different mobile devices, the MARS logger demonstrated its ability to capture synchronized visual-inertial data streams efficiently. Numerical analysis revealed an average sensor operation frequency: visual data logging consistently at 30Hz and IMU readings at rates ranging from 50Hz to 100Hz. Despite minor variations, the logger's synchronization accuracy was validated using state-of-the-art SLAM algorithms—MSCKF and VINS-Mono—further illustrating the logger's capacity to support demanding AR applications.

Implications and Future Directions

The MARS logger presents a practical solution not only for capturing large-scale, diverse datasets but also for enabling real-time SLAM/AR applications on commodity devices. Given its open-source availability, future research might explore enhancements in data fidelity and operational scalability. Additionally, its deployment could inspire novel applications within academic, industrial, and consumer markets by decreasing development costs and resource requirements.

Further work could potentially involve expanding compatibility across a broader range of mobile devices and operating systems. Furthermore, enhanced data storage and processing methods might be explored to address limitations associated with mobile device hardware, especially when dealing with ever-increasing data rates and quantities in AR applications.

In conclusion, the MARS logger signifies an important step towards democratizing access to AR and robotics data for research and development, providing a pivotal resource for accelerating advancements within these fields. As mobile devices continue to evolve, similarly adaptable tools will be critical to leveraging their growing computational prowess.