Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition (2306.01108v1)

Published 1 Jun 2023 in cs.LG and eess.SP

Abstract: Human activity recognition (HAR) in wearable computing is typically based on direct processing of sensor data. Sensor readings are translated into representations, either derived through dedicated preprocessing, or integrated into end-to-end learning. Independent of their origin, for the vast majority of contemporary HAR, those representations are typically continuous in nature. That has not always been the case. In the early days of HAR, discretization approaches have been explored - primarily motivated by the desire to minimize computational requirements, but also with a view on applications beyond mere recognition, such as, activity discovery, fingerprinting, or large-scale search. Those traditional discretization approaches, however, suffer from substantial loss in precision and resolution in the resulting representations with detrimental effects on downstream tasks. Times have changed and in this paper we propose a return to discretized representations. We adopt and apply recent advancements in Vector Quantization (VQ) to wearables applications, which enables us to directly learn a mapping between short spans of sensor data and a codebook of vectors, resulting in recognition performance that is generally on par with their contemporary, continuous counterparts - sometimes surpassing them. Therefore, this work presents a proof-of-concept for demonstrating how effective discrete representations can be derived, enabling applications beyond mere activity classification but also opening up the field to advanced tools for the analysis of symbolic sequences, as they are known, for example, from domains such as natural language processing. Based on an extensive experimental evaluation on a suite of wearables-based benchmark HAR tasks, we demonstrate the potential of our learned discretization scheme and discuss how discretized sensor data analysis can lead to substantial changes in HAR.

Authors (3)

Harish Haresamudram (12 papers)
Irfan Essa (91 papers)
Thomas Ploetz (28 papers)

Citations (6)

View on Semantic Scholar

Summary

Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition

The paper "Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition" advocates a paradigm shift in feature representation for human activity recognition (HAR), proposing a return to discretized, symbolic representations rather than the prevalent continuous features. The authors present a novel method that leverages advancements in Vector Quantization (VQ) to autonomously learn discrete symbols that represent short spans of sensor data collected from wearables, such as wrist-worn accelerometers.

Methodological Advancements

At the core of the paper is the integration of VQ within a self-supervised learning framework, specifically Contrastive Predictive Coding (CPC). This framework, known as VQ-CPC, enables the derivation of discrete vector representations through an unsupervised process. The authors modify the standard CPC setup by incorporating a VQ module, which learns a codebook of vector identifiers. Each sensor segment is matched to its nearest codebook vector, thereby creating a symbolic representation, referred to as codewords. This process is designed to capture the salient features of movement while overcoming the limitations of traditional discretization methods like Symbolic Aggregate approXimation (SAX).

Empirical Evaluation

The evaluation, conducted across several benchmark datasets at various sensor placements (wrist, waist, and leg), demonstrates that discrete representations maintain robust recognition performance. Discrete representations generated using VQ-CPC provide comparable, if not superior, results to continuous representations produced by state-of-the-art self-supervised methods such as SimCLR and multi-task learning approaches. Notable improvements were observed for datasets involving common activities such as walking and other locomotion-related tasks, asserting the viability of discrete representations in diverse real-world scenarios.

Implications and Future Directions

Learning discrete representations opens up several intriguing possibilities for HAR. First, the obtained symbolic sequences translate seamlessly to applications like motif discovery and routine pattern detection, areas historically hindered by the lack of efficient, meaningful symbolic representations. Moreover, the reduced dimensionality inherent in discrete representations can facilitate more efficient data transmission and storage, critical in ubiquitous computing environments where bandwidth and storage may be constrained.

The paper also underscores the potential for leveraging established NLP techniques to enhance HAR systems. For instance, the paper explores using RoBERTa, a transformer-based model pre-trained on symbolic sequences derived from wearable data, to improve downstream activity classification. This novel cross-pollination between NLP pre-training strategies and HAR offers pathways to more powerful and generalizable models.

Conclusion

This investigation into discrete representations for HAR suggests that with the appropriate methodological approaches, the benefits of symbolic representations can be effectively harnessed. The paper reveals promising results, particularly for common activity datasets, suggesting a potential reorientation in how sensor-derived movement data is conceptualized. Future research could refine these models further, emphasizing discriminatory power for fine-grained activities and incorporating techniques to distill large models like RoBERTa for more resource-efficient deployments. This work represents a significant step forward in marrying symbolic sequence analysis with continuous sensor data depiction, proposing a practical and efficient alternative for HAR applications that merits further exploration in subsequent research endeavors.

PDF Markdown

Related Papers

Find Related Papers

YouTube

Show All Videos