Learning Behavioral Representations of Routines From Large-scale Unlabeled Wearable Time-series Data Streams using Hawkes Point Process (2307.04445v1)
Abstract: Continuously-worn wearable sensors enable researchers to collect copious amounts of rich bio-behavioral time series recordings of real-life activities of daily living, offering unprecedented opportunities to infer novel human behavior patterns during daily routines. Existing approaches to routine discovery through bio-behavioral data rely either on pre-defined notions of activities or use additional non-behavioral measurements as contexts, such as GPS location or localization within the home, presenting risks to user privacy. In this work, we propose a novel wearable time-series mining framework, Hawkes point process On Time series clusters for ROutine Discovery (HOT-ROD), for uncovering behavioral routines from completely unlabeled wearable recordings. We utilize a covariance-based method to generate time-series clusters and discover routines via the Hawkes point process learning algorithm. We empirically validate our approach for extracting routine behaviors using a completely unlabeled time-series collected continuously from over 100 individuals both in and outside of the workplace during a period of ten weeks. Furthermore, we demonstrate this approach intuitively captures daily transitional relationships between physical activity states without using prior knowledge. We also show that the learned behavioral patterns can assist in illuminating an individual's personality and affect.
- A. Roebuck and V. Monasterio and E. Gederi and M. Osipov and J. Behar, A. Malhotra and T. Penzel and G. D. Clifford. 2014. A review of signals used in sleep analysis. Physiological Measurement 35, 1 (2014), R1.
- A. Savitzky and M. Golay. 1964. Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Analytical chemistry 36 (July 1964), 1627–1639.
- J. Allen. 2007. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 28 (Jan. 2007).
- Personality and Sedentary Behavior: A Systematic Review and Meta-Analysis. Health Psychology 36 (10 2016).
- Multimodal Estimation of Change Points of Physiological Arousal in Drivers. arXiv preprint arXiv:2210.15826 (2022).
- J.D. Baldwin. 1988. Habit, Emotion, and Self-Conscious Action. Sociological Perspectives 31, 1 (1988), 35–57.
- Toward robust interpretable human movement pattern analysis in a workplace setting. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 7630–7634.
- Multimodal human and environmental sensing for longitudinal behavioral studies in naturalistic settings: Framework for sensor selection, deployment, and management. Journal of medical Internet research 21, 8 (2019), e12832.
- G. Box and G. Jenkins. 1990. Time Series Analysis, Forecasting and Control. Holden-Day, Inc.
- Recurrent neural networks for multivariate time series with missing values. Scientific reports 8, 1 (2018), 6085.
- Modeling patterns of activities using activity curves. Pervasive and Mobile Computing 28 (2016), 51 – 68.
- V. Didelez. 2008. Graphical models for marked point processes based on local independence. Journal of the Royal Statistical Society Series B 70 (February 2008), 245–264.
- Y. Dong and J. Peng. 2013. Principled missing data methods for researchers. SpringerPlus 2 (December 2013), 222.
- TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting. arXiv preprint arXiv:2306.09364 (2023).
- Classifying spatial trajectories using representation learning. International Journal of Data Science and Analytics 2, 3 (01 Dec 2016), 107–117.
- Long term analysis of daily activities in smart home. In ESANN.
- A multimodal analysis of physical activity, sleep, and work shift in nurses with wearable sensor data. Scientific reports 11, 1 (2021), 8693.
- Tiles audio recorder: an unobtrusive wearable solution to track audio activity. In Proceedings of the 4th ACM Workshop on Wearable Systems and Applications. 33–38.
- Tiantian Feng and Shrikanth Narayanan. 2019a. Imputing missing data in large-scale multivariate biomedical wearable recordings using bidirectional recurrent neural networks with temporal activation regularization. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2529–2534.
- Tiantian Feng and Shrikanth S Narayanan. 2019b. Discovering optimal variable-length time series motifs in large-scale wearable recordings of human bio-behavioral signals. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 7615–7619.
- [Glossary of chronobiology (author’s transl)]. Chronobiologia 4 Suppl 1 (1977), 1—189.
- Toeplitz inverse covariance-based clustering of multivariate time series data. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 215–223.
- A. Hawkes. 1971. Spectra of Some Self-Exciting and Mutually Exciting Point Processes. Biometrika 58 (April 1971), 83.
- Z. Hira and D.F. Gillies. 2015. A Review of Feature Selection and Feature Extraction Methods Applied on Microarray Data. 2015 (July 2015), 1–13.
- Fitbit Inc. [n. d.]. Fitbit Charge 2222. https://www.fitbit.com/us/charge2
- Temporal dynamics of workplace acoustic scenes: Egocentric analysis and prediction. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), 756–769.
- K. Hui, and R. Sherratt. 2018. Coverage of Emotion Recognition for Common Wearable Biosensors. Biosensors 8, 2 (2018).
- Accurate Activity Recognition in a Home Setting. In Proceedings of the 10th International Conference on Ubiquitous Computing (Seoul, Korea) (UbiComp ’08). ACM, 1–9.
- M. Lawton and E.M. Brody. 1969. Assessment of Older People: Self-Maintaining and Instrumental Activities of Daily Living1. The Gerontologist 9 (10 1969), 179–186.
- Location-based intelligence-modeling behavior in humans using GPS. In 2006 IEEE International Symposium on Technology and Society. IEEE, 1–8.
- Y. Mohammad and T. Nishida. 2010. Using physiological signals to detect natural interactive behavior. Applied Intelligence 33, 1 (Aug. 2010), 79–92.
- TILES-2018, a longitudinal physiologic and behavioral data set of hospital workers. Scientific Data 7, 1 (2020), 354.
- OMSignal. [n. d.]. OMSignal. https://omsignal.com/
- John Paparrizos and Luis Gravano. 2015. k-shape: Efficient and accurate clustering of time series. In Proceedings of the 2015 ACM SIGMOD international conference on management of data. 1855–1870.
- John Paparrizos and Luis Gravano. 2017. Fast and accurate time-series clustering. ACM Transactions on Database Systems (TODS) 42, 2 (2017), 1–49.
- A review of wearable sensors and systems with application in rehabilitation. Journal of NeuroEngineering and Rehabilitation 9, 1 (20 Apr 2012), 21.
- Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study. Journal of medical Internet research 17, 7 (2015), e4273.
- A. Sano and R. W. Picard. 2013. Stress Recognition Using Wearable Sensors and Mobile Phones. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. 671–676.
- C. Soto and O.P. John. 2017. The Next Big Five Inventory (BFI-2): Developing and Assessing a Hierarchical Model With 15 Facets to Enhance Bandwidth, Fidelity, and Predictive Power. Journal of Personality and Social Psychology 113 (July 2017), 117–143.
- The five-factor model of personality and physical inactivity: A meta-analysis of 16 samples. Journal of Research in Personality 63 (2016), 22 – 28.
- Gabriel Vivó-Truyols and Peter J Schoenmakers. 2006. Automatic selection of optimal Savitzky- Golay smoothing. Analytical chemistry 78, 13 (2006), 4598–4608.
- Mild Cognitive Impairment and Everyday Function: Evidence of Reduced Speed in Performing Instrumental Activities of Daily Living. The American Journal of Geriatric Psychiatry 16, 5 (2008), 416 – 424.
- D. Watson and L. Clark. 1999. The PANAS-X: Manual for the positive and negative affect schedule-expanded form. (January 1999).
- Learning Granger causality for Hawkes processes. In International conference on machine learning. PMLR, 1717–1726.
- Matrix profile I: all pairs similarity joins for time series: a unifying view that includes motifs, discords and shapelets. In 2016 IEEE 16th international conference on data mining (ICDM). Ieee, 1317–1322.
- Han Yu and Akane Sano. 2023. Semi-Supervised Learning for Wearable-based Momentary Stress Detection in the Wild. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1–23.
- Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8980–8987.
- Decomposing activities of daily living to discover routine clusters. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 28.
- Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects. arXiv preprint arXiv:2306.10125 (2023).
- Q. Zhang and Z. Chen. 2014. A weighted kernel possibilistic c-means algorithm based on cloud computing for clustering big data. International Journal of Communication Systems 27, 9 (2014), 1378–1391.