Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

MMASD: A Multimodal Dataset for Autism Intervention Analysis (2306.08243v3)

Published 14 Jun 2023 in cs.CV and cs.LG

Abstract: Autism spectrum disorder (ASD) is a developmental disorder characterized by significant social communication impairments and difficulties perceiving and presenting communication cues. Machine learning techniques have been broadly adopted to facilitate autism studies and assessments. However, computational models are primarily concentrated on specific analysis and validated on private datasets in the autism community, which limits comparisons across models due to privacy-preserving data sharing complications. This work presents a novel privacy-preserving open-source dataset, MMASD as a MultiModal ASD benchmark dataset, collected from play therapy interventions of children with Autism. MMASD includes data from 32 children with ASD, and 1,315 data samples segmented from over 100 hours of intervention recordings. To promote public access, each data sample consists of four privacy-preserving modalities of data; some of which are derived from original videos: (1) optical flow, (2) 2D skeleton, (3) 3D skeleton, and (4) clinician ASD evaluation scores of children, e.g., ADOS scores. MMASD aims to assist researchers and therapists in understanding children's cognitive status, monitoring their progress during therapy, and customizing the treatment plan accordingly. It also has inspiration for downstream tasks such as action quality assessment and interpersonal synchrony estimation. MMASD dataset can be easily accessed at https://github.com/Li-Jicheng/MMASD-A-Multimodal-Dataset-for-Autism-Intervention-Analysis.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR Surveillance Summaries 67, 6 (2018), 1.
  2. Automatic classification of autistic child vocalisations: A novel database and results. (2017).
  3. Constrained local neural fields for robust facial landmark detection in the wild. In Proceedings of the IEEE international conference on computer vision workshops. 354–361.
  4. The DREAM Dataset: Supporting a data-driven study of autism spectrum disorder and robot enhanced therapy. PloS one 15, 8 (2020), e0236939.
  5. OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. IEEE transactions on pattern analysis and machine intelligence 43, 1 (2019), 172–186.
  6. Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6299–6308.
  7. Shi Chen and Qi Zhao. 2019. Attention-Based Autism Spectrum Disorder Screening With Privileged Modality. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
  8. Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments. arXiv preprint arXiv:2306.17278 (2023).
  9. Moreno I Coco and Rick Dale. 2014. Cross-recurrence quantification analysis of categorical and continuous time series: an R package. Frontiers in psychology 5 (2014), 510.
  10. Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder. Scientific reports 8, 1 (2018), 17008.
  11. Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019. Translational psychiatry 10, 1 (2020), 333.
  12. Study of mechanisms of social interaction stimulation in autism spectrum disorder by assisted humanoid robot. IEEE Transactions on Cognitive and Developmental Systems 10, 4 (2017), 993–1004.
  13. A dataset of eye movements for the children with autism spectrum disorder. In Proceedings of the 10th ACM Multimedia Systems Conference. 255–260.
  14. Convolutional Two-Stream Network Fusion for Video Action Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1933–1941. https://doi.org/10.1109/CVPR.2016.213
  15. An asymmetric modeling for action assessment. In European Conference on Computer Vision. Springer, 222–238.
  16. Standardizing ADOS scores for a measure of severity in autism spectrum disorders. Journal of autism and developmental disorders 39 (2009), 693–705.
  17. Social Visual Behavior Analytics for Autism Therapy of Children Based on Automated Mutual Gaze Detection. In Proceedings of the International Conference on Cooperative and Human Aspects of Software Engineering (Orlando, Florida) (CHASE ’23).
  18. Array programming with NumPy. Nature 585, 7825 (Sept. 2020), 357–362. https://doi.org/10.1038/s41586-020-2649-2
  19. Aggregating local image descriptors into compact codes. IEEE transactions on pattern analysis and machine intelligence 34, 9 (2011), 1704–1716.
  20. Dense 3D face alignment from 2D videos in real-time. In 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), Vol. 1. IEEE, 1–8.
  21. Intraface. In IEEE International Conference on Face and Gesture Recognition.
  22. Ivan Laptev. 2005. On space-time interest points. International journal of computer vision 64 (2005), 107–123.
  23. Advancements in Face Alignment Evaluation for Contact-less Vital Sign Detection. In 2023 IEEE-EMBS International Conference on Body Sensor Networks: Sensor and Systems for Digital Health (BSN).
  24. Improving the Movement Synchrony Estimation with Action Quality Assessment in Children Play Therapy. In Proceedings of the International Conference on Multimodal Interaction (Montréal, QC, Canada) (ICMI ’21). 397–406.
  25. A Two-stage Multi-modal Affect Analysis Framework for Children with Autism Spectrum Disorder. In Proceedings of the AAAI-21 Workshop on Affective Content Analysis (New York, USA). 1–8. http://ceur-ws.org/Vol-2897/AffconAAAI-21_paper1.pdf
  26. Dyadic Movement Synchrony Estimation Under Privacy-preserving Conditions. In 2022 26th International Conference on Pattern Recognition (ICPR). IEEE, 762–769.
  27. Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer. In Proceedings of the International Conference on Multimodal Interaction (Bengaluru, India) (ICMI ’22). 73–82.
  28. The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of autism and developmental disorders 30, 3 (2000), 205–223.
  29. Bruce D Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision. In IJCAI’81: 7th international joint conference on Artificial intelligence, Vol. 2. 674–679.
  30. 3d human sensing, action and emotion recognition in robot assisted therapy of children with autism. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2158–2167.
  31. Objective measurement of head movement differences in children with and without autism spectrum disorder. Molecular autism 9 (2018), 1–10.
  32. Guided Weak Supervision for Action Recognition with Scarce Data to Assess Skills of Children with Autism. Proceedings of the AAAI Conference on Artificial Intelligence 34, 01 (Apr. 2020), 463–470. https://doi.org/10.1609/aaai.v34i01.5383
  33. Self-stimulatory behaviours in the wild for autism diagnosis. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 755–761.
  34. Decoding children’s social behavior. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3414–3421.
  35. DE-ENIGMA: Multimodal Human-Robot Interaction for Teaching and Expanding Social Imagination in Autistic Children. Cyberpsychology, behavior and social networking 23, 11 (2020), 806–807.
  36. Sara S Sparrow and Domenic V Cicchetti. 1989. The Vineland adaptive behavior scales. Allyn & Bacon.
  37. The effects of embodied rhythm and robotic interventions on the spontaneous and responsive verbal communication skills of children with Autism Spectrum Disorder (ASD): A further outcome of a pilot randomized controlled trial. Research in autism spectrum disorders 27 (2016), 73–87.
  38. The effects of rhythm and robotic interventions on the imitation/praxis, interpersonal synchrony, and motor performance of children with autism spectrum disorder (ASD): a pilot randomized controlled trial. Autism research and treatment 2015 (2015).
  39. A comparison of the effects of rhythm and robotic interventions on repetitive behaviors and affective states of children with Autism Spectrum Disorder (ASD). Research in autism spectrum disorders 18 (2015), 51–63.
  40. Monocular, One-stage, Regression of Multiple 3D People. In ICCV.
  41. Paul Viola and Michael J Jones. 2004. Robust real-time face detection. International journal of computer vision 57 (2004), 137–154.
  42. Use of machine learning to shorten observation-based screening and diagnosis of autism. Translational psychiatry 2, 4 (2012), e100–e100.
  43. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Thirty-second AAAI conference on artificial intelligence.
  44. 3D Human Pose Estimation with Spatial and Temporal Transformers. Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2021).
  45. Video gesture analysis for autism spectrum disorder detection. In Proc. of International conference on pattern recognition (ICPR). IEEE, 3421–3426.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Jicheng Li (19 papers)
  2. Vuthea Chheang (8 papers)
  3. Pinar Kullu (6 papers)
  4. Eli Brignac (1 paper)
  5. Zhang Guo (4 papers)
  6. Kenneth E. Barner (14 papers)
  7. Anjana Bhat (6 papers)
  8. Roghayeh Leila Barmaki (14 papers)
Citations (9)
View on Semantic Scholar