DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing (1611.01942v2)

Abstract: Mobile sensing applications usually require time-series inputs from sensors. Some applications, such as tracking, can use sensed acceleration and rate of rotation to calculate displacement based on physical system models. Other applications, such as activity recognition, extract manually designed features from sensor inputs for classification. Such applications face two challenges. On one hand, on-device sensor measurements are noisy. For many mobile applications, it is hard to find a distribution that exactly describes the noise in practice. Unfortunately, calculating target quantities based on physical system and noise models is only as accurate as the noise assumptions. Similarly, in classification applications, although manually designed features have proven to be effective, it is not always straightforward to find the most robust features to accommodate diverse sensor noise patterns and user behaviors. To this end, we propose DeepSense, a deep learning framework that directly addresses the aforementioned noise and feature customization challenges in a unified manner. DeepSense integrates convolutional and recurrent neural networks to exploit local interactions among similar mobile sensors, merge local interactions of different sensory modalities into global interactions, and extract temporal relationships to model signal dynamics. DeepSense thus provides a general signal estimation and classification framework that accommodates a wide range of applications. We demonstrate the effectiveness of DeepSense using three representative and challenging tasks: car tracking with motion sensors, heterogeneous human activity recognition, and user identification with biometric motion analysis. DeepSense significantly outperforms the state-of-the-art methods for all three tasks. In addition, DeepSense is feasible to implement on smartphones due to its moderate energy consumption and low latency

• The paper introduces DeepSense, a unified framework combining CNNs and RNNs to effectively process noisy time-series mobile sensor data.
• It leverages frequency transformation and sensor-specific convolutional subnets to extract both local and temporal features, reducing error in tasks like car tracking.
• Its versatile design achieves superior regression and classification performance, enabling energy-efficient deployment on mobile and embedded devices.

An Overview of DeepSense: A Unified Deep Learning Framework for Mobile Sensing

The paper "DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing" presents an integrative model designed to address the inherent challenges in mobile sensing applications. Sensing devices often collect noisy data from various sensors like accelerometers and gyroscopes. Traditional models rely on either physical system models or manually designed features, both of which face limitations due to noise and variability. This paper introduces DeepSense, a deep learning framework that circumvents these issues by leveraging convolutional and recurrent neural networks.

Framework Architecture

DeepSense is architecturally innovative as it merges CNNs and RNNs. The CNNs are employed to capture local interactions within the sensor data, while RNNs model temporal dependencies across different time intervals. This dual-layer approach allows DeepSense to simultaneously learn local sensor interactions and overarching temporal patterns, making it adaptable to a variety of tasks, including both regression and classification.

The framework processes input data by first converting time-series measurements into frequency representations, a pre-processing step that captures relevant patterns within the data. Individual convolutional subnets address data from each sensor type, which are then merged to capture interactions across different sensors at a higher abstraction.

Empirical Validation

The efficacy of DeepSense is demonstrated through three challenging tasks:

1. Car Tracking with Motion Sensors: By using accelerometers, gyroscopes, and magnetometers, DeepSense outperforms state-of-the-art methods in tracking car trajectories despite the significant sensitivity of the task to noisy data. The framework achieves a mean absolute error significantly lower than traditional sensor fusion methods.
2. Heterogeneous Human Activity Recognition (HHAR): In the context of activity recognition, DeepSense extracts robust features capable of generalizing across diverse user profiles. It achieves superior performance compared to conventional feature-based methods like random forests and SVMs, as well as deep learning models such as RBMs, as evidenced by higher accuracy and F1 scores.
3. User Identification via Biometric Motion Analysis: For identification tasks based on motion signature, DeepSense demonstrates its capability to extract distinct user-specific features, surpassing alternative approaches like template-based methods and dedicated CNN architectures. The model achieves high accuracy across different activities such as biking and climbing stairs.

Implications and Future Directions

DeepSense's robust performance on both regression and classification tasks underscores its versatility and potential impact on mobile applications. With moderate energy consumption and low computational latency, the framework is feasible for deployment on mobile and embedded devices, suggesting substantial practical utility.

Future work might focus on optimizing DeepSense for even lower energy consumption or tailoring components for application-specific needs. Further exploration into handling drastic changes in the physical environment or applying DeepSense to different sensor types could expand its applicability.

Conclusion

By introducing a novel architecture that deftly integrates CNNs and RNNs, DeepSense provides a comprehensive solution to the challenges of mobile sensing. Its ability to outperform existing methods across varied tasks illustrates the potential of deep learning frameworks in processing noisy, heterogeneous sensor data effectively. As mobile sensing continues to grow, the methodologies proposed in this paper could serve as a cornerstone for future advancements.