Papers
Topics
Authors
Recent
Search
2000 character limit reached

Elevator, Escalator or Neither? Classifying Pedestrian Conveyor State Using Inertial Navigation System

Published 6 May 2024 in cs.CV | (2405.03218v2)

Abstract: Knowing a pedestrian's conveyor state of "elevator," "escalator," or "neither" is fundamental in many applications such as indoor navigation and people flow management. We study, for the first time, classifying the conveyor state of a pedestrian, given the multimodal INS (inertial navigation system) readings of accelerometer, gyroscope and magnetometer sampled from the pedestrian phone. This problem is challenging because the INS signals of the conveyor state are entangled with unpredictable independent pedestrian motions, confusing the classification process. We propose ELESON, a novel, effective and lightweight INS-based deep learning approach to classify whether a pedestrian is in an elevator, escalator or neither. ELESON utilizes a causal feature extractor to disentangle the conveyor state from pedestrian motion, and a magnetic feature extractor to capture the unique magnetic characteristics of moving elevators and escalators. Given the results of the extractors, it then employs an evidential state classifier to estimate the confidence of the conveyor states. Based on extensive experiments conducted on real pedestrian data, we demonstrate that ELESON outperforms significantly previous INS-based classification approaches, achieving 14% improvement in F1 score, strong confidence discriminability of 0.81 in AUROC (Area Under the Receiver Operating Characteristics), and low computational and memory requirements for smartphone deployment.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. H. et al., “MM-Tap: Adaptive and Scalable Tap Localization on Ubiquitous Surfaces with mm-level Accuracy,” IEEE IoTJ, 2023.
  2. ——, “Self-Supervised Association of Wi-Fi Probe Requests Under MAC Address Randomization,” IEEE Transactions on Mobile Computing, 2022.
  3. H. Ma, Y. He, M. Li, N. Patwari, and S. Sigg, “Introduction to the Special Issue on Wireless Sensing for IoT,” pp. 1–4, 2023.
  4. Z. et al., “FIS-ONE: Floor Identification System with One Label for Crowdsourced RF Signals,” in ICDCS 2023, 2023, pp. 418–428.
  5. J. Tan, H. Wu, K.-H. Chow, and S.-H. G. Chan, “Implicit Multimodal Crowdsourcing for Joint RF and Geomagnetic Fingerprinting,” IEEE Transactions on Mobile Computing, vol. 22, no. 2, pp. 935–950, 2023.
  6. R. Vrskova, P. Kamencay, R. Hudec, and P. Sykora, “A New Deep-learning Method for Human Activity Recognition,” Sensors, vol. 23, no. 5, p. 2816, 2023.
  7. W. Lu, J. Wang, X. Sun, Y. Chen, and X. Xie, “Out-of-Distribution Representation Learning for Time Series Classification,” 2023.
  8. J. e. a. Lu, “Robust Single Accelerometer-based Activity Recognition Using Modified Recurrence Plot,” IEEE Sensors Journal, vol. 19, no. 15, pp. 6317–6324, 2019.
  9. A.-A. et al., “Human Activity Recognition Using Temporal Convolutional Neural Network Architecture,” Expert Systems with Applications, vol. 191, p. 116287, 2022.
  10. H. Xu, P. Zhou, R. Tan, and M. Li, “Practically Adopting Human Activity Recognition,” in Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, 2023, pp. 1–15.
  11. H. Li, S. J. Pan, S. Wang, and A. C. Kot, “Domain Generalization With Adversarial Feature Learning,” in CVPR, 2018, pp. 5400–5409.
  12. L. Chen, Y. Zhang, Y. Song, A. Van Den Hengel, and L. Liu, “Domain generalization via rationale invariance,” in CVPR, 2023, pp. 1751–1760.
  13. M. et al., “Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition,” IMWUT, vol. 7, no. 4, pp. 1–25, 2024.
  14. K. et al., “Human Activity Recognition from Multiple Sensors Data Using Deep CNNs,” Multimedia Tools and Applications, vol. 83, no. 4, pp. 10 815–10 838, 2024.
  15. Z. Yang, C. Wu, Z. Zhou, X. Zhang, X. Wang, and Y. Liu, “Mobility Increases Localizability: A Survey on Wireless Indoor Localization Using Inertial Sensors,” ACM Computing Surveys (Csur), vol. 47, no. 3, pp. 1–34, 2015.
  16. E. Bulbul, A. Cetin, and I. A. Dogru, “Human Activity Recognition Using Smartphones,” in ISMSIT.   IEEE, 2018, pp. 1–6.
  17. A. Brajdic and R. Harle, “Walk Detection and Step Counting on Unconstrained Smartphones,” in UbiComp, 2013, pp. 225–234.
  18. X. Kang, B. Huang, and G. Qi, “A Novel Walking Detection and Step Counting Algorithm Using Unconstrained Smartphones,” Sensors, vol. 18, no. 1, p. 297, 2018.
  19. F. Demrozi, G. Pravadelli, A. Bihorac, and P. Rashidi, “Human activity recognition using inertial, physiological and environmental sensors: A comprehensive survey,” IEEE access, vol. 8, pp. 210 816–210 836, 2020.
  20. M. et al., “CNN-based Sensor Fusion Techniques for Multimodal Human Activity Recognition,” in ACM ISWC, 2017, pp. 158–165.
  21. C. et al., “Comparing CNN and Human Crafted Features for Human Activity Recognition,” in UIC-ATC.   IEEE, 2019, pp. 960–967.
  22. K. et al., “Deep CNN-LSTM with Self-attention Model for Human Activity Recognition Using Wearable Sensor,” IEEE Journal of Translational Engineering in Health and Medicine, vol. 10, pp. 1–16, 2022.
  23. O. Nafea, W. Abdul, and G. Muhammad, “Multi-sensor Human Activity Recognition Using CNN and GRU,” International Journal of Multimedia Information Retrieval, vol. 11, no. 2, pp. 135–147, 2022.
  24. G. Saleem, U. I. Bajwa, and R. H. Raza, “Toward Human Activity Recognition: A Survey,” Neural Computing and Applications, vol. 35, no. 5, pp. 4145–4182, 2023.
  25. B. et al., “Comparing Handcrafted Features and Deep Neural Representations for Domain Generalization in Human Activity Recognition,” Sensors, vol. 22, no. 19, p. 7324, 2022.
  26. T. Shen, I. Di Giulio, and M. Howard, “A Probabilistic Model of Human Activity Recognition with Loose Clothing,” Sensors, vol. 23, no. 10, p. 4669, 2023.
  27. R. Hu, L. Chen, S. Miao, and X. Tang, “SWL-Adapt: An Unsupervised Domain Adaptation Model with Sample Weight Learning for Cross-user Wearable Human Activity Recognition,” in AAAI, vol. 37, no. 5, 2023, pp. 6012–6020.
  28. J. Yang, Y. Xu, H. Cao, H. Zou, and L. Xie, “Deep Learning and Transfer Learning for Device-free Human Activity Recognition: A Survey,” Journal of Automation and Intelligence, vol. 1, no. 1, p. 100007, 2022.
  29. B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio, “Toward Causal Representation Learning,” Proceedings of the IEEE, vol. 109, no. 5, pp. 612–634, 2021.
  30. G. et al., “A Survey of Uncertainty in Deep Neural Networks,” Artificial Intelligence Review, vol. 56, no. Suppl 1, pp. 1513–1589, 2023.
  31. Y. Gal, “Uncertainty in Deep Learning,” University of Cambridge, 2016.
  32. L. et al., “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles,” NIPS, vol. 30, 2017.
  33. Y. Wen, D. Tran, and J. Ba, “Batchensemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning,” arXiv preprint arXiv:2002.06715, 2020.
  34. Y. Li and Y. Gal, “Dropout Inference in Bayesian Neural Networks with Alpha-divergences,” in ICML.   PMLR, 2017, pp. 2052–2061.
  35. P. Thiagarajan, P. Khairnar, and S. Ghosh, “Explanation and Use of Uncertainty Quantified by Bayesian Neural Network Classifiers for Breast Histopathology Images,” IEEE transactions on medical imaging, vol. 41, no. 4, pp. 815–825, 2021.
  36. J. Lee, M. Humt, J. Feng, and R. Triebel, “Estimating Model Uncertainty of Neural Networks in Sparse Information Form,” in ICML.   PMLR, 2020, pp. 5702–5713.
  37. N. et al., “Towards Maximizing the Representation Gap between In-domain & Out-of-distribution Examples,” NIPS, vol. 33, pp. 9239–9250, 2020.
  38. A. Malinin and M. Gales, “Predictive Uncertainty Estimation via Prior Networks,” NIPS, vol. 31, 2018.
  39. T. Ramalho and M. Miranda, “Density Estimation in Representation Space to Predict Model Uncertainty,” in EDSMLS.   Springer, 2020, pp. 84–96.
  40. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On Calibration of Modern Neural Networks,” in ICML.   PMLR, 2017, pp. 1321–1330.
  41. A. Karandikar, N. Cain, D. Tran, B. Lakshminarayanan, J. Shlens, M. C. Mozer, and B. Roelofs, “Soft calibration objectives for neural networks,” Advances in Neural Information Processing Systems, vol. 34, pp. 29 768–29 779, 2021.
  42. M. Sensoy, L. Kaplan, and M. Kandemir, “Evidential Deep Learning to Quantify Classification Uncertainty,” in NIPS, vol. 31, 2018.
  43. J. Liu, Z. Lin, S. Padhy, D. Tran, T. Bedrax Weiss, and B. Lakshminarayanan, “Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness,” NIPS, vol. 33, pp. 7498–7512, 2020.
  44. S. Hochreiter and J. Schmidhuber, “Long Short-term Memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  45. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet Classification with Deep Convolutional Neural Networks,” NIPS, vol. 25, 2012.
  46. W. et al., “A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone,” IEEE Sensors Journal, vol. 16, no. 11, 2016.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up