Papers
Topics
Authors
Recent
Search
2000 character limit reached

VSViG: Real-time Video-based Seizure Detection via Skeleton-based Spatiotemporal ViG

Published 24 Nov 2023 in cs.CV | (2311.14775v2)

Abstract: An accurate and efficient epileptic seizure onset detection can significantly benefit patients. Traditional diagnostic methods, primarily relying on electroencephalograms (EEGs), often result in cumbersome and non-portable solutions, making continuous patient monitoring challenging. The video-based seizure detection system is expected to free patients from the constraints of scalp or implanted EEG devices and enable remote monitoring in residential settings. Previous video-based methods neither enable all-day monitoring nor provide short detection latency due to insufficient resources and ineffective patient action recognition techniques. Additionally, skeleton-based action recognition approaches remain limitations in identifying subtle seizure-related actions. To address these challenges, we propose a novel Video-based Seizure detection model via a skeleton-based spatiotemporal Vision Graph neural network (VSViG) for its efficient, accurate and timely purpose in real-time scenarios. Our experimental results indicate VSViG outperforms previous state-of-the-art action recognition models on our collected patients' video data with higher accuracy (5.9% error), lower FLOPs (0.4G), and smaller model size (1.4M). Furthermore, by integrating a decision-making rule that combines output probabilities and an accumulative function, we achieve a 5.1 s detection latency after EEG onset, a 13.1 s detection advance before clinical onset, and a zero false detection rate. The project homepage is available at: https://github.com/xuyankun/VSViG/

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. 2d human pose estimation: New benchmark and state of the art analysis. In Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pages 3686–3693, 2014.
  2. Blazepose: On-device real-time body pose tracking. arXiv preprint arXiv:2006.10204, 2020.
  3. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, 2017.
  4. Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE TPAMI, 2019.
  5. Channel-wise topology refinement graph convolution for skeleton-based action recognition. In ICCV, pages 13359–13368, 2021.
  6. Using spatio-temporal interest points (stip) for myoclonic jerk detection in nocturnal video. In IEEE EMBC, pages 4454–4457, 2012.
  7. Msr-gcn: Multi-scale residual graph convolution networks for human motion prediction. In ICCV, pages 11467–11476, 2021.
  8. Dg-stgcn: dynamic spatial-temporal modeling for skeleton-based action recognition. arXiv preprint arXiv:2210.05895, 2022a.
  9. Revisiting skeleton-based action recognition. In CVPR, pages 2969–2978, 2022b.
  10. Alphapose: Whole-body regional multi-person pose estimation and tracking in real-time. IEEE TPAMI, 2022.
  11. Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition. In CVPR, pages 203–213, 2020.
  12. Slowfast networks for video recognition. In CVPR, pages 6202–6211, 2019.
  13. Automated video-based detection of nocturnal convulsive seizures in a residential care setting. Epilepsia, 59:53–60, 2018.
  14. Vision gnn: An image is worth graph of nodes. NeurIPS, 35:8291–8303, 2022.
  15. Vision hgnn: An image is more than a graph of nodes. In ICCV, pages 19878–19888, 2023.
  16. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  17. Epileptic seizure detection: A deep learning approach. arXiv preprint arXiv:1803.09848, 2018.
  18. 3d convolutional neural networks for human action recognition. IEEE TPAMI, 35(1):221–231, 2012.
  19. Automatic segmentation of episodes containing epileptic clonic seizures in video sequences. IEEE TBME, 59(12):3379–3385, 2012.
  20. Novel 3d video action recognition deep learning approach for near real time epileptic seizure classification. Scientific Reports, 12(1):19571, 2022.
  21. Human action recognition and prediction: A survey. International Journal of Computer Vision, 130(5):1366–1401, 2022.
  22. Deepgcns: Can gcns go as deep as cnns? In CVPR, pages 9267–9276, 2019a.
  23. Actional-structural graph convolutional networks for skeleton-based action recognition. In CVPR, pages 3595–3603, 2019b.
  24. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  25. Disentangling and unifying graph convolutions for skeleton-based action recognition. In CVPR, pages 143–152, 2020.
  26. Deep dual consecutive network for human pose estimation. In CVPR, pages 525–534, 2021.
  27. Epilepsy: new advances. The Lancet, 385(9971):884–898, 2015.
  28. Mobilevig: Graph-based sparse attention for mobile vision applications. In CVPR, pages 2210–2218, 2023.
  29. Daniil Osokin. Real-time 2d multi-person pose estimation on cpu: Lightweight openpose. In arXiv preprint arXiv:1811.12004, 2018.
  30. Two-stream adaptive graph convolutional networks for skeleton-based action recognition. In CVPR, pages 12026–12035, 2019.
  31. Skeleton-based action recognition with multi-stream adaptive graph convolutional networks. IEEE TIP, 29:9532–9545, 2020.
  32. Application of machine learning to epileptic seizure detection. In ICML, pages 975–982, 2010.
  33. Stronger, faster and more explainable: A graph convolutional baseline for skeleton-based action recognition. In ACM MM, pages 1625–1633, 2020.
  34. Self-supervised graph neural networks for improved electroencephalographic seizure analysis. arXiv preprint arXiv:2104.08336, 2021.
  35. Epilepsy in adults. The Lancet, 393(10172):689–701, 2019.
  36. Learning robust features using deep learning for automatic seizure detection. In Proceedings of the 1st Machine Learning for Healthcare Conference, pages 178–190. PMLR, 2016.
  37. Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1653–1660, 2014.
  38. Learning spatiotemporal features with 3d convolutional networks. In ICCV, pages 4489–4497, 2015.
  39. A closer look at spatiotemporal convolutions for action recognition. In CVPR, pages 6450–6459, 2018.
  40. Automated epileptic seizure detection methods: a review study. Epilepsy–Histological, Electroencephalographic and Psychological Aspects, pages 2027–2036, 2012.
  41. Value of video monitoring for nocturnal seizure detection in a residential setting. Epilepsia, 2016.
  42. Automated video-based detection of nocturnal motor seizures in children. Epilepsia, 61:S36–S40, 2020.
  43. Pvg: Progressive vision graph for vision recognition. In ACM MM, pages 2477–2486, 2023.
  44. Shorter latency of real-time epileptic seizure detection via probabilistic prediction. Expert Systems with Applications, 236, 2024.
  45. Spatial temporal graph convolutional networks for skeleton-based action recognition. In AAAI, 2018.
  46. Video-based detection of generalized tonic-clonic seizures using deep learning. IEEE JBHI, 2021.
  47. Action recognition based on rgb and skeleton data sets: A survey. Neurocomputing, 2022.
  48. Visualizing and understanding convolutional networks, 2013.
  49. Factorized omnidirectional representation based vision gnn for anisotropic 3d multimodal mr image segmentation. In ACM MM, pages 1607–1615, 2023.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up