Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
120 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SecurePose: Automated Face Blurring and Human Movement Kinematics Extraction from Videos Recorded in Clinical Settings (2402.14143v1)

Published 21 Feb 2024 in cs.CV and cs.AI

Abstract: Movement disorders are typically diagnosed by consensus-based expert evaluation of clinically acquired patient videos. However, such broad sharing of patient videos poses risks to patient privacy. Face blurring can be used to de-identify videos, but this process is often manual and time-consuming. Available automated face blurring techniques are subject to either excessive, inconsistent, or insufficient facial blurring - all of which can be disastrous for video assessment and patient privacy. Furthermore, assessing movement disorders in these videos is often subjective. The extraction of quantifiable kinematic features can help inform movement disorder assessment in these videos, but existing methods to do this are prone to errors if using pre-blurred videos. We have developed an open-source software called SecurePose that can both achieve reliable face blurring and automated kinematic extraction in patient videos recorded in a clinic setting using an iPad. SecurePose, extracts kinematics using a pose estimation method (OpenPose), tracks and uniquely identifies all individuals in the video, identifies the patient, and performs face blurring. The software was validated on gait videos recorded in outpatient clinic visits of 116 children with cerebral palsy. The validation involved assessing intermediate steps of kinematics extraction and face blurring with manual blurring (ground truth). Moreover, when SecurePose was compared with six selected existing methods, it outperformed other methods in automated face detection and achieved ceiling accuracy in 91.08% less time than a robust manual face blurring method. Furthermore, ten experienced researchers found SecurePose easy to learn and use, as evidenced by the System Usability Scale. The results of this work validated the performance and usability of SecurePose on clinically recorded gait videos for face blurring and kinematics extraction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. C. Einspieler and H. F. Prechtl, “Prechtl’s assessment of general movements: a diagnostic tool for the functional assessment of the young nervous system,” Mental retardation and developmental disabilities research reviews, vol. 11, no. 1, pp. 61–67, 2005.
  2. C. L. Comella, S. Leurgans, J. Wuu, G. T. Stebbins, T. Chmura, and D. S. Group, “Rating scales for dystonia: a multicenter assessment,” Movement disorders, vol. 18, no. 3, pp. 303–312, 2003.
  3. T. Brown and A. Lalor, “The movement assessment battery for children—second edition (mabc-2): a review and critique,” Physical & occupational therapy in pediatrics, vol. 29, no. 1, pp. 86–103, 2009.
  4. R. Bajpai and D. Joshi, “A-gas: A probabilistic approach for generating automated gait assessment score for cerebral palsy children,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 2530–2539, 2021.
  5. A. Act, “Health insurance portability and accountability act of 1996,” Public law, vol. 104, p. 191, 1996.
  6. P. Regulation, “General data protection regulation,” Intouch, vol. 25, pp. 1–5, 2018.
  7. D. Prasad M and S. Menon C, “The personal data protection bill, 2018: India’s regulatory journey towards a comprehensive data protection law,” International Journal of Law and Information Technology, vol. 28, no. 1, pp. 1–19, 2020.
  8. Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7291–7299.
  9. [Online]. Available: https://www.tensorflow.org/hub/tutorials/movenet/
  10. [Online]. Available: https://github.com/google-coral/project-posenet/
  11. Z. Liu, H. Chen, R. Feng, S. Wu, S. Ji, B. Yang, and X. Wang, “Deep dual consecutive network for human pose estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 525–534.
  12. R. A. Güler, N. Neverova, and I. Kokkinos, “Densepose: Dense human pose estimation in the wild,” 2018.
  13. J. Jiang, W. Skalli, A. Siadat, and L. Gajny, “Effect of face blurring on human pose estimation: Ensuring subject privacy for medical and occupational health applications,” Sensors, vol. 22, no. 23, p. 9376, 2022.
  14. A. Ali-Gombe, E. Elyan, and J. Zwiegelaar, “Towards a reliable face recognition system,” in Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference: Proceedings of the EANN 2020 21.   Springer, 2020, pp. 304–316.
  15. S. P. Otta, S. Kolipara, S. Panda, and C. Hota, “User identification with face recognition: A systematic analysis,” in 2022 3rd International Conference for Emerging Technology (INCET).   IEEE, 2022, pp. 1–6.
  16. M. K. Hasan, M. S. Ahsan, S. S. Newaz, and G. M. Lee, “Human face detection techniques: A comprehensive review and future research directions,” Electronics, vol. 10, no. 19, p. 2354, 2021.
  17. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour models,” International journal of computer vision, vol. 1, no. 4, pp. 321–331, 1988.
  18. A. Nikolaidis and I. Pitas, “Facial feature extraction and pose determination,” Pattern Recognition, vol. 33, no. 11, pp. 1783–1791, 2000.
  19. C.-L. Huang and C.-W. Chen, “Human facial feature extraction for face interpretation and recognition,” Pattern recognition, vol. 25, no. 12, pp. 1435–1444, 1992.
  20. A. L. Yuille, P. W. Hallinan, and D. S. Cohen, “Feature extraction from faces using deformable templates,” International journal of computer vision, vol. 8, pp. 99–111, 1992.
  21. A. Lanitis, A. Hill, T. F. Cootes, and C. J. Taylor, “Locating facial features using genetic algorithms,” in International Conference on Digital Signal Processing, 1995, pp. 520–525.
  22. J. L. Crowley and F. Berard, “Multi-modal tracking of faces for video communications,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.   IEEE, 1997, pp. 640–645.
  23. S. McKenna, S. Gong, and H. Liddell, “Real-time tracking for an integrated face recognition system,” in Second European Workshop on Parallel Modelling of Neural Operators, vol. 11, 1995.
  24. H. P. Graf, E. Cosatto, D. Gibbon, M. Kocheisen, and E. Petajan, “Multi-modal system for locating heads and faces,” in Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.   IEEE, 1996, pp. 88–93.
  25. G. Guo, S. Z. Li, and K. L. Chan, “Support vector machines for face recognition,” Image and Vision computing, vol. 19, no. 9-10, pp. 631–638, 2001.
  26. N. Ahmed, T. Natarajan, and K. R. Rao, “Discrete cosine transform,” IEEE transactions on Computers, vol. 100, no. 1, pp. 90–93, 1974.
  27. A. R. Chadha, P. P. Vaidya, and M. M. Roja, “Face recognition using discrete cosine transform for global and local features,” in 2011 International Conference On Recent Advancements In Electrical, Electronics And Control Engineering.   IEEE, 2011, pp. 502–505.
  28. X. He and P. Niyogi, “Locality preserving projections,” Advances in neural information processing systems, vol. 16, 2003.
  29. J. Hérault, “Réseaux de neurones à synapses modifiables: Décodage de messages sensoriels composites par une apprentissage non supervisé et permanent,” CR Acad. Sci. Paris, pp. 525–528, 1984.
  30. P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, vol. 1.   Ieee, 2001, pp. I–I.
  31. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol. 1.   Ieee, 2005, pp. 886–893.
  32. A. Devaux, N. Paparoditis, F. Precioso, and B. Cannelle, “Face blurring for privacy in street-level geoviewers combining face, body and skin detectors.” in MVA, 2009, pp. 86–89.
  33. K. Brkić, T. Hrkać, Z. Kalafatić, and I. Sikirić, “Face, hairstyle and clothing colour de-identification in video sequences,” IET Signal Processing, vol. 11, no. 9, pp. 1062–1068, 2017.
  34. Z. Liu, M. Hao, and Y. Hu, “Visual anonymity: Automated human face blurring for privacy-preserving digital videos.”
  35. F. S. Al-Mukhtar, “Tracking and blurring the face in a video file,” Al-Nahrain Journal of Science, no. 1, pp. 202–207, 2018.
  36. Y. Pang, Y. Yuan, X. Li, and J. Pan, “Efficient hog human detection,” Signal processing, vol. 91, no. 4, pp. 773–781, 2011.
  37. A. Adouani, W. M. B. Henia, and Z. Lachiri, “Comparison of haar-like, hog and lbp approaches for face detection in video sequences,” in 2019 16th International Multi-Conference on Systems, Signals & Devices (SSD).   IEEE, 2019, pp. 266–271.
  38. C. Rahmad, R. A. Asmara, D. Putra, I. Dharma, H. Darmono, and I. Muhiqqin, “Comparison of viola-jones haar cascade classifier and histogram of oriented gradients (hog) for face detection,” in IOP conference series: materials science and engineering, vol. 732, no. 1.   IOP Publishing, 2020, p. 012038.
  39. H. A. Rowley, S. Baluja, and T. Kanade, “Neural network-based face detection,” IEEE Transactions on pattern analysis and machine intelligence, vol. 20, no. 1, pp. 23–38, 1998.
  40. K. Steinbuch and U. A. Piske, “Learning matrices and their applications,” IEEE Transactions on Electronic Computers, no. 6, pp. 846–862, 1963.
  41. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” nature, vol. 323, no. 6088, pp. 533–536, 1986.
  42. M. J. Orr et al., “Introduction to radial basis function networks,” 1996.
  43. S.-Y. Kung, S.-H. Lin, and M. Fang, “A neural network approach to face/palm recognition,” in Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.   IEEE, 1995, pp. 323–332.
  44. M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of cognitive neuroscience, vol. 3, no. 1, pp. 71–86, 1991.
  45. L. Sirovich and M. Kirby, “Low-dimensional procedure for the characterization of human faces,” Josa a, vol. 4, no. 3, pp. 519–524, 1987.
  46. M. A. Turk and A. P. Pentland, “Face recognition using eigenfaces,” in Proceedings. 1991 IEEE computer society conference on computer vision and pattern recognition.   IEEE Computer Society, 1991, pp. 586–587.
  47. B. Moghaddam and A. Pentland, “Probabilistic visual learning for object detection,” in Proceedings of IEEE international conference on computer vision.   IEEE, 1995, pp. 786–793.
  48. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,” IEEE Transactions on pattern analysis and machine intelligence, vol. 19, no. 7, pp. 711–720, 1997.
  49. K. Pearson, “Liii. on lines and planes of closest fit to systems of points in space,” The London, Edinburgh, and Dublin philosophical magazine and journal of science, vol. 2, no. 11, pp. 559–572, 1901.
  50. H. Hotelling, “Analysis of a complex of statistical variables into principal components.” Journal of educational psychology, vol. 24, no. 6, p. 417, 1933.
  51. V. N. Vapnik, “Pattern recognition using generalized portrait method,” Automation and remote control, vol. 24, no. 6, pp. 774–780, 1963.
  52. D. E. King, “Max-margin object detection,” arXiv preprint arXiv:1502.00046, 2015.
  53. K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016.
  54. X. Zhu and D. Ramanan, “Face detection, pose estimation, and landmark localization in the wild,” in 2012 IEEE conference on computer vision and pattern recognition.   IEEE, 2012, pp. 2879–2886.
  55. J. Prinosil and O. Maly, “Detecting faces with face masks,” in 2021 44th International Conference on Telecommunications and Signal Processing (TSP).   IEEE, 2021, pp. 259–262.
  56. T. Edirisooriya and E. Jayatunga, “Comparative study of face detection methods for robust face recognition systems,” in 2021 5th SLAAI International Conference on Artificial Intelligence (SLAAI-ICAI).   IEEE, 2021, pp. 1–6.
  57. J. Zhou, C.-M. Pun, and Y. Wang, “Pixelation is not done in videos yet,” arXiv preprint arXiv:1903.10836, 2019.
  58. T. Park, N. P. Phu, and H. Kim, “Cnn based face tracking and re-identification for privacy protection in video contents,” Journal of the Korea Information and Communications Society, vol. 25, no. 1, pp. 63–68, 2021.
  59. Y. Su and X. Liu, “Front view gait (fvg-b) database,” 2022.
  60. X. Wei and Q. Yin, “Comparison and application of two face detection algorithms,” in Proceedings of the 5th International Conference on Signal Processing and Information Communications.   Springer, 2022, pp. 53–62.
  61. S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems, vol. 28, 2015.
  62. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14.   Springer, 2016, pp. 21–37.
  63. J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv, 2018.
  64. W. Chen, H. Huang, S. Peng, C. Zhou, and C. Zhang, “Yolo-face: a real-time face detector,” The Visual Computer, vol. 37, pp. 805–813, 2021.
  65. K. Kim, Z. Yang, I. Masi, R. Nevatia, and G. Medioni, “Face and body association for video-based face recognition,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).   IEEE, 2018, pp. 39–48.
  66. S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li, “S3fd: Single shot scale-invariant face detector,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 192–201.
  67. A. Meethal, M. Pedersoli, S. Belharbi, and E. Granger, “Convolutional stn for weakly supervised object localization,” in 2020 25th International Conference on Pattern Recognition (ICPR).   IEEE, 2021, pp. 10 157–10 164.
  68. P. Hu and D. Ramanan, “Finding tiny faces,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 951–959.
  69. S. Yang, P. Luo, C. C. Loy, and X. Tang, “Faceness-net: Face detection through deep facial part responses,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 8, pp. 1845–1859, 2017.
  70. R. Ranjan, V. M. Patel, and R. Chellappa, “A deep pyramid deformable part model for face detection,” in 2015 IEEE 7th international conference on biometrics theory, applications and systems (BTAS).   IEEE, 2015, pp. 1–8.
  71. Z. Yang and R. Nevatia, “A multi-scale cascade fully convolutional network face detector,” in 2016 23rd International Conference on Pattern Recognition (ICPR).   IEEE, 2016, pp. 633–638.
  72. M. Mathias, R. Benenson, M. Pedersoli, and L. Van Gool, “Face detection without bells and whistles,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13.   Springer, 2014, pp. 720–735.
  73. S. Mroz, N. Baddour, C. McGuirk, P. Juneau, A. Tu, K. Cheung, and E. Lemaire, “Comparing the quality of human pose estimation with blazepose or openpose,” in 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART).   IEEE, 2021, pp. 1–4.
  74. J. Stenum, C. Rossi, and R. T. Roemmich, “Two-dimensional video-based analysis of human gait using pose estimation,” PLoS computational biology, vol. 17, no. 4, p. e1008935, 2021.
  75. B. Zhu, H. Fang, Y. Sui, and L. Li, “Deepfakes for medical video de-identification: Privacy protection and diagnostic information preservation,” in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 2020, pp. 414–420.
  76. A. Kumar, A. Kaur, and M. Kumar, “Face detection techniques: a review,” Artificial Intelligence Review, vol. 52, pp. 927–948, 2019.
  77. [Online]. Available: https://sourceforge.net/projects/securepose/
  78. [Online]. Available: https://www.youtube.com/playlist?list=PLO4_jCYO5Ib23MoBpn-Wpj1_b6DAYlDwk
  79. P. Rosenbaum, N. Paneth, A. Leviton, M. Goldstein, M. Bax, D. Damiano, B. Dan, B. Jacobsson et al., “A report: the definition and classification of cerebral palsy april 2006,” Dev Med Child Neurol Suppl, vol. 109, no. suppl 109, pp. 8–14, 2007.
  80. R. Palisano, P. Rosenbaum, S. Walter, D. Russell, E. Wood, and B. Galuppi, “Development and reliability of a system to classify gross motor function in children with cerebral palsy,” Developmental medicine & child neurology, vol. 39, no. 4, pp. 214–223, 1997.
  81. F. Lundh, “An introduction to tkinter,” URL: www. pythonware. com/library/tkinter/introduction/index. htm, 1999.
  82. C. M. O’Connor, S. K. Thorpe, M. J. O’Malley, and C. L. Vaughan, “Automatic detection of gait events using kinematic data,” Gait & posture, vol. 25, no. 3, pp. 469–474, 2007.
  83. R. Bajpai and D. Joshi, “Movenet: A deep neural network for joint profile prediction across variable walking speeds and slopes,” IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1–11, 2021.
  84. L. K. Osterkamp, “Current perspective on assessment of human body proportions of relevance to amputees,” Journal of the American Dietetic Association, vol. 95, no. 2, pp. 215–218, 1995.
  85. [Online]. Available: https://github.com/opencv/opencv/blob/master/data/haarcascades/haarcascade_frontalface_default.xml
  86. [Online]. Available: https://pypi.org/project/dlib/
  87. [Online]. Available: https://github.com/davisking/dlib-models/blob/master/mmod_human_face_detector.dat.bz2
  88. [Online]. Available: https://pypi.org/project/mtcnn/
  89. [Online]. Available: https://github.com/derronqi/yolov8-face
  90. [Online]. Available: https://github.com/sfzhang15/SFD
  91. [Online]. Available: https://www.shotcut.org/
  92. R. Padilla, S. L. Netto, and E. A. Da Silva, “A survey on performance metrics for object-detection algorithms,” in 2020 international conference on systems, signals and image processing (IWSSIP).   IEEE, 2020, pp. 237–242.
  93. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision, vol. 88, pp. 303–338, 2010.
  94. J. Brooke, “Sus: a “quick and dirty’usability,” Usability evaluation in industry, vol. 189, no. 3, pp. 189–194, 1996.
Citations (1)
View on Semantic Scholar

Summary

  • The paper introduces SecurePose, which automates face blurring and extracts kinematic data with 0.992 precision and 0.990 recall in face detection.
  • It utilizes OpenPose for 2D skeletal tracking and human body proportions to achieve more accurate face localization than traditional HOG and MTCNN detectors.
  • High usability ratings from clinical researchers demonstrate that SecurePose is practical for routine anonymization and movement assessment in clinical settings.

SecurePose: Efficient Automated De-Identification and Movement Analysis in Clinical Settings

The paper "SecurePose: Automated Face Blurring and Human Movement Kinematics Extraction from Videos Recorded in Clinical Settings" introduces SecurePose, an innovative software tool designed to address critical challenges in the de-identification of patient video data. The development focuses on enabling precise face blurring and reliable kinematic feature extraction from clinically acquired videos — a task essential for protecting patient privacy and enhancing the quantification of movement disorders.

Significance of De-Identification and Kinematic Extraction

In clinical environments, videos play a pivotal role in diagnosing and assessing movement disorders. Traditional methods of manual face blurring inherently pose risks due to their time-intensive nature, potential for error, and susceptibility to inconsistencies. Existing automated approaches typically struggle with performance variability, especially in videos with dynamic lighting and motion, common in real-world clinical scenarios.

SecurePose distinguishes itself by combining high-fidelity de-identification with robust kinematic extraction. It operates using OpenPose for 2D skeletal tracking, ensuring that patient identification is both fast and dependable. Moreover, SecurePose takes advantage of human body proportions, facilitating accurate face localization and blurring — a marked improvement over many existing face detection frameworks.

Performance Evaluation

The authors validate SecurePose by testing it on a dataset of 116 videos from cerebral palsy patients, providing compelling evidence of its efficacy in clinical contexts. SecurePose achieved impressive metrics, including a precision of 0.992 and a recall of 0.990 in face detection, substantially outperforming other contemporary methods. These high values underline the software's potential for clinically unacceptable oversight and excessive blurring that can hinder video analysis.

Performance comparisons are made with other prevalent methodologies such as HOG-based detectors and MTCNN, establishing SecurePose's superiority in both efficiency and accuracy in identifying and anonymizing faces across video frames. Specifically, it mitigates common issues like incorrect individual tracking and misalignment of facial features — problems that frequently plague attempts to accurately de-identify video data in clinical environments.

Usability and Practical Applications

An additional strength of SecurePose lies in its high system usability score by ten experienced clinical researchers, which attests to its practicality and ease of adoption in clinical and research settings. The software facilitates the anonymization and sharing of medical videos, promoting collaborative research efforts while adhering to privacy guidelines dictated by standards such as HIPAA and GDPR.

Implications and Future Directions

The paper illustrates how structured, automatic video processing aligns with the requirements of modern AI-driven healthcare research. With SecurePose, clinicians are empowered to share de-identified data confidently, fostering data-driven advances in diagnosing and treating movement disorders. Furthermore, the paper establishes a critical precedent for future face blurring developments that integrate seamlessly with kinematic data extraction methods.

The authors also acknowledge areas for further enhancement, such as extending validation to other movement studies and streamlining computation through algorithmic optimization and parallel processing. Additionally, future iterations could explore cloud-based implementations to facilitate broader accessibility across various clinical setups without necessitating high-specification local servers.

In summary, SecurePose represents a promising step forward in safeguarding patient privacy while harnessing quantifiable data from clinical video footage. This advancement could substantially augment the existing methodologies of automated anonymization, simultaneously laying groundwork for expanding the compatible applications of pose estimation technologies in quantitatively assessing motor disorders.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube