Papers
Topics
Authors
Recent
Search
2000 character limit reached

Unsupervised Work Behavior Pattern Extraction Based on Hierarchical Probabilistic Model

Published 16 May 2024 in cs.LG | (2405.09838v1)

Abstract: Evolving consumer demands and market trends have led to businesses increasingly embracing a production approach that prioritizes flexibility and customization. Consequently, factory workers must engage in tasks that are more complex than before. Thus, productivity depends on each worker's skills in assembling products. Therefore, analyzing the behavior of a worker is crucial for work improvement. However, manual analysis is time consuming and does not provide quick and accurate feedback. Machine learning have been attempted to automate the analyses; however, most of these methods need several labels for training. To this end, we extend the Gaussian process hidden semi-Markov model (GP-HSMM), to enable the rapid and automated analysis of worker behavior without pre-training. The model does not require labeled data and can automatically and accurately segment continuous motions into motion classes. The proposed model is a probabilistic model that hierarchically connects GP-HSMM and HSMM, enabling the extraction of behavioral patterns with different granularities. Furthermore, it mutually infers the parameters between the GP-HSMM and HSMM, resulting in accurate motion pattern extraction. We applied the proposed method to motion data in which workers assembled products at an actual production site. The accuracy of behavior pattern extraction was evaluated using normalized Levenshtein distance (NLD). The smaller the value of NLD, the more accurate is the pattern extraction. The NLD of motion patterns captured by GP-HSMM and HSMM layers in our proposed method was 0.50 and 0.33, respectively, which are the smallest compared to that of the baseline methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. I. Budiman, A. C. Sembiring, J. Tampubolon, D. Wahyuni, and A. Dharmala, “Improving effectiveness and efficiency of assembly line with a stopwatch time study and balancing activity elements,” Journal of Physics: Conference Series, vol. 1230, no. 1, 2019.
  2. N. Yoshimura, T. Maekawa, T. Hara, A. Wada, and Y. Namioka, “Acceleration-based activity recognition of repetitive works with lightweight ordered-work segmentation network,” vol. 6, no. 2, jul 2022. [Online]. Available: https://doi.org/10.1145/3534572
  3. F. Moya Rueda, R. Grzeszick, G. A. Fink, S. Feldhorst, and M. Ten Hompel, “Convolutional neural networks for human activity recognition using body-worn sensors,” Informatics, vol. 5, no. 2, 2018. [Online]. Available: https://www.mdpi.com/2227-9709/5/2/26
  4. S. Feldhorst, M. Masoudenijad, M. Hompel, and G. Fink, “Motion classification for analyzing the order picking process using mobile sensors - general concepts, case studies and empirical evaluation,” 01 2016, pp. 706–713.
  5. M. Aehnelt, E. Gutzeit, and B. Urban, “Using activity recognition for the tracking of assembly processes: Challenges and requirements,” 03 2014.
  6. T. Nakamura, T. Nagai, D. Mochihashi, I. Kobayashi, H. Asoh, and M. Kaneko, “Segmenting continuous motions with hidden semi-markov models and gaussian processes,” Frontiers in neurorobotics, vol. 11, p. 67, 2017.
  7. E. B. Fox, E. B. Sudderth, M. I. Jordan, and A. S. Willsky, “Joint modeling of multiple related time series via the beta process,” 2011.
  8. Y. Matsubara, Y. Sakurai, and C. Faloutsos, “Autoplait: Automatic mining of co-evolving time sequences,” in Proceedings of the 2014 ACM SIGMOD international conference on Management of data, 2014, pp. 193–204.
  9. C. Lea, M. D. Flynn, R. Vidal, A. Reiter, and G. D. Hager, “Temporal convolutional networks for action segmentation and detection,” in proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 156–165.
  10. T. Kobayashi, Y. Aoki, S. Shimizu, K. Kusano, and S. Okumura, “Fine-grained action recognition in assembly work scenes by drawing attention to the hands,” in 2019 15th International Conference on Signal-Image Technology Internet-Based Systems (SITIS), 2019, pp. 440–446.
  11. S. Yeung, O. Russakovsky, G. Mori, and L. Fei-Fei, “End-to-end learning of action detection from frame glimpses in videos,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2678–2687.
  12. A. Diba, M. Fayyaz, V. Sharma, M. M. Arzani, R. Yousefzadeh, J. Gall, and L. Van Gool, “Spatio-temporal channel correlation networks for action classification,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 284–299.
  13. P. Bojanowski, R. Lajugie, F. Bach, I. Laptev, J. Ponce, C. Schmid, and J. Sivic, “Weakly supervised action labeling in videos under ordering constraints,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13.   Springer, 2014, pp. 628–643.
  14. D.-A. Huang, L. Fei-Fei, and J. C. Niebles, “Connectionist temporal modeling for weakly supervised action labeling,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14.   Springer, 2016, pp. 137–153.
  15. A. Richard, H. Kuehne, and J. Gall, “Weakly supervised action learning with rnn based fine-to-coarse modeling,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 754–763.
  16. F. Sener and A. Yao, “Unsupervised learning and segmentation of complex activities from video,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8368–8376.
  17. S.-Z. Yu, “Hidden semi-markov models,” Artificial intelligence, vol. 174, no. 2, pp. 215–243, 2010.
  18. M. Wächter and T. Asfour, “Hierarchical segmentation of manipulation actions based on object relations and motion characteristics,” in 2015 International Conference on Advanced Robotics (ICAR), 2015, pp. 549–556.
  19. T. Taniguchi and S. Nagasaka, “Double articulation analyzer for unsegmented human motion using pitman-yor language model and infinite hidden markov model,” 12 2011.
  20. T. Taniguchi, K. Hamahata, and N. Iwahashi, “Unsupervised segmentation of human motion data using a sticky hierarchical dirichlet process-hidden markov model and minimal description length-based chunking method for imitation learning,” Advanced Robotics, vol. 25, no. 17, pp. 2143–2172, 2011.
  21. G. Reddy, L. Desban, H. Tanaka, J. Roussel, O. Mirat, and C. Wyart, “A lexical approach for identifying behavioural action sequences,” PLOS Computational Biology, vol. 18, pp. 1–29, 01 2022. [Online]. Available: https://doi.org/10.1371/journal.pcbi.1009672
  22. S. Goldwater, “Nonparametric bayesian models of lexical acquisition,” 01 2006.
  23. D. Mochihashi, T. Yamada, and N. Ueda, “Bayesian unsupervised word segmentation with nested Pitman-Yor language modeling,” in Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.   Suntec, Singapore: Association for Computational Linguistics, Aug. 2009, pp. 100–108. [Online]. Available: https://aclanthology.org/P09-1012
  24. K. Uchiumi, H. Tsukahara, and D. Mochihashi, “Inducing word and part-of-speech with Pitman-Yor hidden semi-Markov models,” in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).   Beijing, China: Association for Computational Linguistics, Jul. 2015, pp. 1774–1782. [Online]. Available: https://aclanthology.org/P15-1171
  25. N. O. Khanfar, H. I. Ashqar, M. Elhenawy, Q. Hussain, A. Hasasneh, and W. K. Alhajyaseen, “Application of unsupervised machine learning classification for the analysis of driver behavior in work zones in the state of qatar,” Sustainability, vol. 14, no. 22, p. 15184, 2022.
  26. G. Wang, X. Zhang, S. Tang, H. Zheng, and B. Y. Zhao, “Unsupervised clickstream clustering for user behavior analysis,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ser. CHI ’16.   New York, NY, USA: Association for Computing Machinery, 2016, pp. 225–236. [Online]. Available: https://doi.org/10.1145/2858036.2858107
  27. A. Ball, D. Rye, F. Ramos, and M. Velonaki, “Unsupervised clustering of people from ’skeleton’ data,” in Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction, ser. HRI ’12.   New York, NY, USA: Association for Computing Machinery, 2012, pp. 225–226. [Online]. Available: https://doi.org/10.1145/2157689.2157767
  28. K. Su, X. Liu, and E. Shlizerman, “Predict & cluster: Unsupervised skeleton based action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9631–9640.
  29. L. Li, M. Wang, B. Ni, H. Wang, J. Yang, and W. Zhang, “3d human action representation learning via cross-view consistency pursuit,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4741–4750.
  30. T. Nakamura, T. Nagai, and T. Taniguchi, “Serket: An architecture for connecting stochastic models to realize a large-scale cognitive model,” Frontiers in Neurorobotics, vol. 12, pp. 1–16, 2018.
  31. T. Taniguchi, T. Nakamura, M. Suzuki, R. Kuniyasu, K. Hayashi, A. Taniguchi, T. Horii, and T. Nagai, “Neuro-serket: development of integrative cognitive system through the composition of deep probabilistic generative models,” New Generation Computing, pp. 1–26, 2020.
  32. M. Nagano, T. Nakamura, T. Nagai, D. Mochihashi, I. Kobayashi, and W. Takano, “Hvgh: Unsupervised segmentation for high-dimensional time series using deep neural compression and statistical generative model,” Frontiers in Robotics and AI, vol. 6, 2019. [Online]. Available: https://www.frontiersin.org/articles/10.3389/frobt.2019.00115
  33. V. I. Levenshtein et al., “Binary codes capable of correcting deletions, insertions, and reversals,” in Soviet physics doklady, vol. 10, no. 8.   Soviet Union, 1966, pp. 707–710.
  34. M. Nagano, T. Nakamura, T. Nagai, D. Mochihashi, I. Kobayashi, and M. Kaneko, “Sequence pattern extraction by segmenting time series data using gp-hsmm with hierarchical dirichlet process,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2018, pp. 4067–4074.
  35. D. Nguyen-Tuong, J. Peters, and M. Seeger, “Local gaussian process regression for real time online model learning,” Advances in neural information processing systems, vol. 21, 2008.
  36. Y. Okadome, K. Urai, Y. Nakamura, T. Yomo, and H. Ishiguro, “Adaptive lsh based on the particle swarm method with the attractor selection model for fast approximation of gaussian process regression,” Artificial Life and Robotics, vol. 19, pp. 220–226, 2014.
  37. J. Gardner, G. Pleiss, K. Q. Weinberger, D. Bindel, and A. G. Wilson, “Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration,” Advances in neural information processing systems, vol. 31, 2018.
  38. R. M. Neal, “Slice sampling,” The Annals of Statistics, vol. 31, no. 3, pp. 705 – 767, 2003. [Online]. Available: https://doi.org/10.1214/aos/1056562461

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 2 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free