Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey (2207.10574v2)

Published 20 Jul 2022 in cs.HC, cs.AI, cs.CV, cs.LG, and cs.MM

Abstract: Automated co-located human-human interaction analysis has been addressed by the use of nonverbal communication as measurable evidence of social and psychological phenomena. We survey the computing studies (since 2010) detecting phenomena related to social traits (e.g., leadership, dominance, personality traits), social roles/relations, and interaction dynamics (e.g., group cohesion, engagement, rapport). Our target is to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings (free-standing conversations, meetings, dyads, and crowds). We also present a comprehensive summary of the related datasets and outline future research directions which are regarding the implementation of artificial intelligence, dataset curation, and privacy-preserving interaction analysis. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively; multimodal features are prominently performing better; deep learning architectures showed improved performance in overall, but there exist many phenomena whose detection has never been implemented through deep models. We also identified several limitations such as the lack of scalable benchmarks, annotation reliability tests, cross-dataset experiments, and explainability analysis.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (210)
  1. With whom do I interact? Detecting social interactions in egocentric photo-streams. In Proc. of IEEE ICPR. 2959–2964.
  2. Social Relation Recognition in Egocentric Photostreams. In Proc. of IEEE ICIP. 3227–3231.
  3. Analyzing Free-Standing Conversational Groups: A Multimodal Approach. In Proc. of ACMMM. 5–14.
  4. Understanding social relationships in egocentric vision. Pattern Recognition 48, 12 (2015), 4082 – 4096.
  5. From Ego to Nos-Vision: Detecting Social Relationships in First-Person Views. In Proc. of IEEE CVPRw. 594–599.
  6. Thin Slices of Depression: Improving Depression Detection Performance Through Data Segmentation. In Proc. of ICASSP. 6257–6261.
  7. Oya Aran and Daniel Gatica-Perez. 2010. Fusing Audio-Visual Nonverbal Cues to Detect Dominant People in Small Group Conversations. In Proc. of ICPR. 3687–3690.
  8. Oya Aran and Daniel Gatica-Perez. 2013a. Cross-domain personality prediction: from video blogs to small group meetings. In Proc. of ACM ICMI. 127–130.
  9. Oya Aran and Daniel Gatica-Perez. 2013b. One of a kind: inferring personality impressions in meetings. In Proc. of ACM ICMI. 9–13.
  10. Umut Avci and Oya Aran. 2014. Effect of nonverbal behavioral patterns on the performance of small groups. In Workshop on Understanding Modeling Multiparty Multimodal Interactions. 9–14.
  11. Umut Avci and Oya Aran. 2016. Predicting the Performance in Decision-Making Tasks: From Individual Cues to Group Interaction. IEEE Trans. Multimedia 18, 4 (2016), 643–658.
  12. Predicting dominance in multi-person videos. In Proc. of IJCAI. 4643–4650.
  13. OpenFace: An open source facial behavior analysis toolkit.. In Proc. of IEEE WACV. 1–10.
  14. Multimodal Egocentric Analysis of Focused Interactions. IEEE Access 6 (2018), 37493–37505.
  15. Finding Time Together: Detection and Classification of Focused Interaction in Egocentric Video. In Proc. of IEEE CVPR workshops. 2322–2330.
  16. Gaze and conversational engagement in multiparty video conversation: an annotation scheme and classification of high and low levels of engagement. In Proc. Workshop on eye gaze in intelligent human machine interaction. 1–6.
  17. Analysis of Face-Touching Behavior in Large Scale Social Interaction Dataset.
  18. Identification of Emergent Leaders in a Meeting Scenario Using Multiple Kernel Learning. In Proc. of ACM ICMI-ASSP4MI. 3–10.
  19. Prediction of the Leadership Style of an Emergent Leader Using Audio and Visual Nonverbal Features. IEEE Trans. Multimedia 20, 2 (2018), 441–456.
  20. Detecting Emergent Leader in a Meeting Environment Using Nonverbal Visual Features Only. In Proc. of ACM ICMI. 317–324.
  21. Moving as a Leader: Detecting Emergent Leadership in Small Groups using Body Pose. In Proc. of ACMMM. 1425–1433.
  22. A Sequential Data Analysis Approach to Detect Emergent Leaders in Small Groups. IEEE Trans. on Multimedia 21, 8 (2019), 2107–2116.
  23. Investigation of Small Group Social Interactions Using Deep Visual Activity-Based Nonverbal Features. In Proc. of ACMMM. 311–319.
  24. Personality Traits Classification Using Deep Visual Activity-Based Nonverbal Features of Key-Dynamic Images. IEEE Trans. on Affective Comput. 12, 4 (2021), 1084–1099.
  25. The Unobtrusive Group Interaction (UGI) Corpus. In Proc. of ACM MMSys. 249–254.
  26. A Multimodal-Sensor-Enabled Room for Unobtrusive Group Meeting Analysis. In Proc. of ACM ICMI. 347–355.
  27. The WoNoWa Dataset: Investigating the Transactive Memory System in Small Group Interactions. In Proc. of ACM ICMI. 528–537.
  28. The MAHNOB Mimicry Database: A database of naturalistic human interactions. Pattern Recognition Letters 66 (2015), 52–61.
  29. Dynamic Image Networks for Action Recognition. In Proc. of IEEE CVPR. 3034–3042.
  30. McKenzie Braley and Gabriel Murray. 2018. The Group Affect and Performance (GAP) Corpus. In Proc. of ACM GIFT.
  31. A Tutorial on Human Activity Recognition Using Body-Worn Inertial Sensors. ACM Comput. Surv. 46, 3 (2014).
  32. The MatchNMingle dataset: a novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates. IEEE Trans. on Affective Comput. (2018).
  33. Estimating Self-Assessed Personality from Body Movements and Proximity in Crowded Mingling Scenarios. In Proc. of ACM ICMI. New York, NY, USA, 238–242.
  34. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Trans. PAMI (2019).
  35. Guillaume Chanel and Christian Mühl. 2015. Connecting brains and bodies: applying physiological computing to support social interaction. Interacting with Computers 27, 5 (2015), 534–550.
  36. Convokit: A toolkit for the analysis of conversations. arXiv preprint arXiv:2005.04246 (2020).
  37. Quantifying EDA synchrony through joint sparse representation: a case-study of couples’ interactions. In Proc. of IEEE ICASSP. 817–821.
  38. Exploring sparse representation measures of physiological synchrony for romantic couples. In Proc. of ACII. 267–272.
  39. Discovering social interactions in real work environments. In Proc. of IEEE FG. 933–938.
  40. Social interaction discovery by statistical analysis of F-formations. In Proc. of BMVC. 23.1–23.12.
  41. Towards Computational Proxemics: Inferring Social Relations from Interpersonal Distances. In Proc. of PASSAT/SocialComp. 290–297.
  42. Human behavior analysis in video surveillance: A Social Signal Processing perspective. Neurocomputing 100 (2013), 86 – 97.
  43. Modeling functional roles dynamics in small group interactions. IEEE Trans. Multimedia 15, 1 (2013), 83–95.
  44. Temporal Triplet Mining for Personality Recognition. In Proc. of IEEE FG. 171–178.
  45. Being the Center of Attention: A Person-Context CNN Framework for Personality Recognition. 10, 3 (2020).
  46. MUMBAI: multi-person, multimodal board game affect and interaction analysis dataset. Journal on Multimodal User Interfaces 15, 4 (2021), 373–391.
  47. Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data. In Proc. of CHI. 1–16.
  48. Yun Fu (Ed.). 2016. Human Activity Recognition and Prediction. Springer.
  49. Automatic detection of dominance and expected interest. In EURASIP Journal on Advances in Signal Processing. 1–12.
  50. The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing. IEEE Trans. on Affective Comput. 7, 2 (2016), 190–202.
  51. Recent Developments in OpenSMILE, the Munich Open-Source Multimedia Feature Extractor. In Proc. of ACMMM. 835–838.
  52. Bernard Ghanem Fabian Caba Heilbron, Victor Escorcia and Juan Carlos Niebles. 2015. ActivityNet: A Large-Scale Video Benchmark for Human Activity Understanding. In Proc. of CVPR. 961–970.
  53. Personality classification and behaviour interpretation: an approach based on feature categories. In Proc. of ACM ICMI. 225–232.
  54. Social interactions: A first-person perspective. In Proc. of IEEE CVPR. 1226–1233.
  55. Detecting personality and emotion traits in crowds from video sequences. Mach. Vis. Appl. 30 (2019).
  56. Detecting Posture Mirroring in Social Interactions with Wearable Sensors. Proc. of Int. Symp. on Wearable Computers, 119–120.
  57. Quantifying Behavioral Mimicry by Automatic Detection of Nonverbal Cues from Body Motion. In Proc. of IEEE SocialCom/PASSAT. 520–525.
  58. Discriminating Individually Considerate and Authoritarian Leaders by Speech Activity Cues. In Proc. of IEEE SocialCom/PASSAT. 1460–1465.
  59. Combined Speaker Clustering and Role Recognition in Conversational Speech. In Proc. of INTERSPEECH. 1378–1382.
  60. MobileSSI: Asynchronous Fusion for Social Signal Interpretation in the Wild. In Proc. of ACM ICMI. 266–273.
  61. Automatic recognition of social roles using long term role transitions in small group interactions. In Proc. of INTERSPEECH. 2065–2069.
  62. Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss. In Proc. of ICRP.
  63. Temporal Encoded F-Formation System for Social Interaction Detection. In Proc. of ACMMM. 937–946.
  64. Daniel Gatica-Perez. 2009. Automatic Nonverbal Analysis of Social Interaction in Small Groups: A Review. Image and Vision Computing 27, 12 (2009), 1775–1787.
  65. Ekin Gedik and Hayley Hung. 2018. Detecting Conversing Groups Using Social Dynamics from Wearable Acceleration: Group Size Awareness. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4 (2018).
  66. Reduced nonverbal interpersonal synchrony in autism spectrum disorder independent of partner diagnosis: a motion energy study. Molecular autism 11, 1 (2020), 1–14.
  67. Behavioral signal processing for understanding (distressed) dyadic interactions: some recent developments. In ACM workshop on Human gesture and behavior understanding. 7–12.
  68. Predicting group cohesiveness in images. In Proc. of IJCNN. 1–8.
  69. Theodoros Giannakopoulos. 2015. pyAudioAnalysis: An Open-Source Python Library for Audio Signal Analysis. PloS one 10, 12 (2015).
  70. Sensing social interactions through BLE beacons and commercial mobile devices. Pervasive and Mobile Computing 67 (2020), 101198.
  71. Detecting Social Situations from Interaction Geometry. In IEEE Int. Conf. on Social Computing. 1–8.
  72. Speech, movement, and gaze behaviours during dyadic conversation in noise. Scientific reports 9, 1 (2019), 1–8.
  73. Predicting Levels of Rapport in Dyadic Interactions through Automatic Detection of Posture and Posture Congruence. In Proc. of IEEE SocialCom/PASSAT. 613–616.
  74. Fasih Haider and Saturnino Luz. 2019. A System for Real-Time Privacy Preserving Data Collection for Ambient Assisted Living. In Proc. of Interspeech. 2374–2375.
  75. Speaker Dependency Analysis, Audiovisual Fusion Cues and a Multimodal BLSTM for Conversational Engagement Recognition. In INTERSPEECH. 3359–3363.
  76. Hayley Hung and Gokul Chittaranjan. 2010. The Idiap Wolf Corpus: Exploring Group Behaviour in a Competitive Role-Playing Game. In Proc. of ACMMM. 879–882.
  77. Hayley Hung and Daniel Gatica-Perez. 2010. Estimating Cohesion in Small Groups Using Audio-Visual Nonverbal Behavior. IEEE Trans. on Multimedia 12 (2010), 563 – 575.
  78. Estimating Dominance in Multi-Party Meetings Using Speaker Diarization. IEEE Trans. Audio, Speech, Language Process. 19, 4 (2011), 847–860.
  79. Workshop on Interdisciplinary Insights into Group and Team Dynamics. In Proc. of ACM ICMI. 876–877.
  80. Shoichi Inaba and Yoshimitsu Aoki. 2016. Conversational Group Detection Based on Social Context Using Graph Clustering Algorithm. In Proc. of SITIS. 526–531.
  81. Analysis of Respiration for Prediction of "Who Will Be Next Speaker and When?" In Multi-Party Meetings. In ACM ICMI. 18–25.
  82. Privacy-sensitive recognition of group conversational context with sociometers. Springer Multimedia Systems 18 (2012), 3–14.
  83. Dinesh Babu Jayagopi and Daniel Gatica-Perez. 2010. Mining group nonverbal conversational patterns using probabilistic topic models. IEEE Trans. Multimedia 12, 8 (2010), 790–802.
  84. Cao Jie and Pan Peng. 2010. Recognize the most dominant person in multi-party meetings using nontraditional features. In Proc. of ICIS. 312–316.
  85. David Johnson and Gabriel Murray. 2021. Clustering and Multimodal Analysis of Participants in Task-Based Discussions. In Companion Publication of ACM ICMI. 273–277.
  86. Panoptic Studio: A Massively Multiview System for Social Interaction Capture. IEEE Trans. on PAMI 41, 1 (2019), 190–204.
  87. Modeling dominance effects on nonverbal behaviors using granger causality. In Proc. of ACM ICMI. 23–26.
  88. Automatic Modeling of Dominance Effects Using Granger Causality. Human Behavior Understanding, Lecture Notes in Computer Science, A. Salah, Lepri, B., Eds., Springer, Berlin/Heidelberg 7065 (2011), 124–133.
  89. Causal-modelling of personality traits: extraversion and locus of control. In 2nd Inter. Workshop on Social Signal Processing.
  90. F. Kamper. 2017. An Empirical Study of Gaussian Belief Propagation and Application in the Detection of F-Formations. In Proc. of ACMMM. 1–6.
  91. Estimating Romantic, Social, and Sexual Attraction by Quantifying Bodily Coordination using Wearable Sensors. In Proc. of ACII. 154–160.
  92. Detecting Group Formations Using IBeacon Technology. In Proc. of ACM UbiComp. 742–752.
  93. Finding Dory in the Crowd: Detecting Social Interactions Using Multi-Modal Mobile Sensing. In Proc. of SenSys-ML. 37–42.
  94. Estimation of interest and comprehension level of audience through multi-modal behaviors in poster conversations. In Proc. of INTERSPEECH. 1882–1885.
  95. Vyaktitv: A multimodal peer-to-peer hindi conversations based dataset for personality assessment. In IEEE BigMM. 103–111.
  96. Human Activity Recognition and Pattern Discovery. IEEE Pervasive Comput. 9, 1 (2010), 48–53.
  97. Multi-domain and multi-task prediction of extraversion and leadership from meeting videos. EURASIP JIVP 77 (2017), 1–14.
  98. Mikkel Baun Kjærgaard and Henrik Blunck. 2014. Tool Support for Detection and Analysis of Following and Leadership Behavior of Pedestrians from Mobile Sensing Data. Pervasive Mob. Comput. 10 (2014), 104–117.
  99. Automatic recognition of personality profiles using EEG functional connectivity during emotional processing. Brain sciences 10, 5 (2020), 278.
  100. Panagiotis Koromilas and Theodoros Giannakopoulos. 2021. Unsupervised Multimodal Language Representations using Convolutional Autoencoders. arXiv preprint arXiv:2110.03007 (2021).
  101. M. Koutsombogera and C. Vogel. 2018. Modeling Collaborative Multimodal Behavior in Group Dialogues: The MULTISIMO Corpus. In LREC.
  102. Analyzing Verbal and Nonverbal Features for Predicting Group Performance. In Proc. of INTERSPEECH. 1896–1900.
  103. Automatic gaze analysis in multiparty conversations based on Collective First-Person Vision. In Proc. of IEEE FG, Vol. 05. 1–8.
  104. Unobtrusive tracking of interpersonal orienting and distance predicts the subjective quality of social interactions. Royal Society Open Science 7 (2020).
  105. Catherine Lai and Gabriel Murray. 2018. Predicting Group Satisfaction in Meeting Discussions. In Proc. of the Workshop on Modeling Cognitive Processes from Multimodal Data.
  106. Quantification of prosodic entrainment in affective spontaneous spoken interactions of married couples. In Proc. of ISCA.
  107. Using measures of vocal entrainment to inform outcome-related behaviors in marital conflicts. In Proc. of APSIPA. 1–5.
  108. Honest Signals and Their Contribution to the Automatic Analysis of Personality Traits A Comparative Study. Human Behavior Understanding, ser. Lecture Notes in Computer Science, A. Salah, T. Gevers, N. Sebe, and A. Vinciarelli, Eds. Springer, Berlin / Heidelberg 6219 (2010), 140–150.
  109. The SocioMetric Badges Corpus: A Multilevel Behavioral Dataset for Social Behavior in Complex Organizations. In Proc. of IEEE SocialCom/PASSAT. 623–628.
  110. Connecting Meeting Behavior with Extraversion-A Systematic Study. IEEE Trans. on Affective Comput. 3 (2012), 443–455.
  111. Employing social gaze and speaking activity for automatic determination of the Extraversion trait. In Proc. of ACM ICMI-MLMI.
  112. Think before you speak: An investigation of eye activity patterns during conversations using eyewear. Int. Journal of Human-Computer Studies 143 (2020), 102468.
  113. Yun-Shao Lin and Chi-Chun Lee. 2016. Using Interlocutor-Modulated Attention BLSTM to Predict Personality Traits in Small Group Interaction. In Proc. of ACM ICMI. 163–169.
  114. Yun-Shao Lin and Chi-Chun Lee. 2020. Predicting Performance Outcome with a Conversational Graph Convolutional Network for Small Group Interactions. In Proc. of IEEE ICASSP. 8044–8048.
  115. Social Relation Recognition From Videos via Multi-Scale Spatial-Temporal Reasoning. In Proc. of CVPR.
  116. Nichola Lubold and Heather Pon-Barry. 2014. Acoustic-prosodic entrainment and rapport in collaborative learning dialogues. In Proc. ACM Multimodal Learning Analytics Workshop and Grand Challenge. 5–12.
  117. Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30.
  118. Multi-stream Fusion Model for Social Relation Recognition from Videos. In MultiMedia Modeling. Cham, 355–368.
  119. Jinna Lv and Bin Wu. 2019. Spatio-Temporal Attention Model Based on Multi-view for Social Relation Understanding. In MultiMedia Modeling. Cham, 390–401.
  120. GAME-ON: A Multimodal Dataset for Cohesion and Group Analysis. IEEE Access (2020).
  121. Cross-Device Interaction via Micro-Mobility and f-Formations. In Proc. of ACM UIST. 13–22.
  122. Mobile Monitoring of Formal and Informal Social Interactions at Workplace. In Proc. of ACM UbiComp. 1035–1044.
  123. Recent trends in deep learning based personality detection. Artificial Intelligence Review 53 (2020), 2313–2339.
  124. Chreston Miller and Christa Miller. 2019. Timing is Everything: Identifying Diverse Interaction Dynamics in Scenario and Non-Scenario Meetings. In International Conference on eScience. 203–212.
  125. AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups. IEEE Trans. on Affective Comput. (2018).
  126. Go Miura and Shogo Okada. 2019. Task-Independent Multimodal Prediction of Group Performance Based on Product Dimensions. In Proc. of ACM ICMI. 264–273.
  127. Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Trans. on Affective Comput. 10, 1 (2017), 18–31.
  128. Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour. In Proc. of Inter. Conf. on Intelligent User Interfaces. 153–164.
  129. Philipp Matthias Muller and Andreas Bulling. 2019. Emergent Leadership Detection Across Datasets.
  130. A Tale of Two Interactions: Inferring Performance in Hospitality Encounters from Cross-Situation Social Sensing. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3 (2018).
  131. Gabriel Murray and Catharine Oertel. 2018. Predicting Group Performance in Task-Based Interaction. In Proc. of ACM ICMI.
  132. Automated prediction and analysis of job interview performance: The role of what you say and how you say it. In Proc. of IEEE FG, Vol. 1. 1–6.
  133. Estimating Verbal Expressions of Task and Social Cohesion in Meetings by Quantifying Paralinguistic Mimicry. In Proc. of ACM ICMI. 206–215.
  134. Hire me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior. IEEE Trans. Multimedia 16, 4 (2014), 1018–1031.
  135. Towards a Model of Nonverbal Leadership in Unstructured Joint Physical Activity. In Proc. of MOCO.
  136. Predicting Influential Statements in Group Discussions Using Speech and Head Motion Information. In Proc. of ACM ICMI. 136–143.
  137. D64: a corpus of richly recorded conversational interaction. Journal on Multimodal User Interfaces 7, 1-2 (2012), 19–28.
  138. Deciphering the Silent Participant: On the Use of Audio-Visual Cues for the Classification of Listener Categories in Group Discussions. In Proc. of ACM ICMI. 107–114.
  139. Who Will Get the Grant? A Multimodal Corpus for the Analysis of Conversational Behaviours in Group Interviews. 27–32.
  140. Catharine Oertel and Giampiero Salvi. 2013. A Gaze-Based Method for Relating Group Involvement to Individual Engagement in Multimodal Multiparty Dialogue. In Proc. of ACM ICMI. 99–106.
  141. The KTH games corpora: How to catch a werewolf. In In Multimodal-Corpora:Beyond Audio and Gaze, 2013.
  142. Toward Interest Estimation from Head Motion Using Wearable Sensors: A Case Study in Story Time for Children. In Human-Computer Interaction. Interaction Contexts. 353–363.
  143. Personality Trait Classification via Co-Occurrent Multiparty Multimodal Event Discovery. In Proc. of ACM ICMI. 15–22.
  144. Modeling Dyadic and Group Impressions with Intermodal and Interperson Features. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019).
  145. Analyzing Nonverbal Behaviors along with Praising. In Proc. of ACM ICMI. 609–613.
  146. Predicting Group Contribution Behaviour in a Public Goods Game from Face-to-Face Communication. Sensors 19, 12 (2019), 2786.
  147. A Survey on Mobile SSP. ACM Comput. Surv. 48, 4 (2016).
  148. Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset. In Proc. of IEEE WACVw. 1–12.
  149. Subspace Clustering for Action Recognition with Covariance Representations and Temporal Pruning. (2021), 6035–6042.
  150. Unsupervised Human Action Recognition with Skeletal Graph Laplacian and Self-Supervised Viewpoints Invariance. In Proc. of BMVC.
  151. I already know your answer: Using nonverbal behaviors to predict immediate outcomes in a dyadic negotiation. In Proc. ACM ICMI. 19–22.
  152. Mutual behaviors during dyadic negotiation: Automatic prediction of respondent reactions. In Proc. of ACII. 423–428.
  153. Inferring Neuronal Network Connectivity from Spike Data: A Temporal Data Mining Approach. Sci. Program. 16, 1 (2008), 49–77.
  154. Alex Pentland. 2007. Social Signal Processing. IEEE Signal Processing Magazine 24, 4 (2007), 108–111.
  155. Social Role Discovery in Human Events. In Proc. of IEEE CVPR. 2475–2482.
  156. Sowmya Rasipuram and Dinesh Babu Jayagopi. 2020. Automatic multimodal assessment of soft skills in social interactions: a review. Multim. Tools Appl. 79, 19-20 (2020), 13037–13060.
  157. Uncovering Interactions and Interactors: Joint Estimation of Head, Body Orientation and F-Formations from Surveillance Videos. In Proc. of IEEE ICCV. 4660–4668.
  158. From Proxemics Theory to Socially-Aware Navigation: A Survey. Int. Journal of Social Robotics (2015).
  159. Detecting F-formations Roles in Crowded Social Scenes with Wearables: Combining Proxemics Dynamics using LSTMs. In Proc. of ACII. 147–153.
  160. J. B. Rotter. 1966. Generalized expectancies for internal versus external control of reinforcement. Psychological monographs 80 1 (1966), 1–28.
  161. A new multichannel multi modal dyadic interaction database.. In Proc. of INTERSPEECH. 1982–1985.
  162. ImageNet Large Scale Visual Recognition Challenge. IJCV 115, 3 (2015), 211–252.
  163. Sina Sajadmanesh and Daniel Gatica-Perez. 2021. Locally Private Graph Neural Networks. In Proc. of ACM CCS.
  164. Emergent Leaders through Looking and Speaking: from Audio-Visual Data to Multimodal Recognition. Journal on Multimodal User Interfaces 7, 1-2 (2012), 39–53.
  165. Identifying emergent leadership in small groups using nonverbal communicative cues. In Proc. of ACM ICMI-MLMI. 8–10.
  166. A Nonverbal Behavior Approach to Identify Emergent Leaders in Small Groups. IEEE Trans. Multimedia 14, 3 (2012), 816–832.
  167. Ashtosh Sapru and Herve Bourlard. 2013. Automatic Social Role Recognition In Professional Meetings Using Conditional Random Fields. In Proc. of INTERSPEECH. 1530–1534.
  168. Ashtosh Sapru and Herve Bourlard. 2015. Automatic recognition of emergent social roles in small group interactions. IEEE Trans. Multimedia 17, 5 (2015), 746–760.
  169. Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization. In Proc. of IEEE ICCV.
  170. Multi-scale f-formation discovery for group detection. In Proc. of IEEE ICIP. 3547–3551.
  171. S-VVAD: Visual Voice Activity Detection by Motion Segmentation. In Proc. of IEEE WACV.
  172. Automatic Group Level Affect and Cohesion Prediction in Videos. In Proc. of ACII. 161–167.
  173. First Impressions: A Survey on Vision-based Apparent Personality Trait Analysis. IEEE Trans.on Affective Comput. (2019), 1–1.
  174. Learning to identify leaders in crowd. In IEEE CVPRw. 43–48.
  175. Multimodal human behavior analysis: learning correlation and interaction across modalities. In Proc. ACM ICMI. 27–30.
  176. Don’t Ask Me What i’m like, Just Watch and Listen. In Proc. of ACMMM. New York, NY, USA, 329–338.
  177. Contextual Modeling of Personality States? Dynamics in Face-to-Face Interactions. In Proc. of PASSAT/SocialCom. 898–899.
  178. Automatic modeling of personality states in small group interactions. In Proc. of ACMMM. 989–992.
  179. Alexandros Stergiou and Ronald Poppe. 2019. Analyzing human–human interactions: A survey. CVIU 188 (2019).
  180. Jointly Estimating Interactions and Head, Body Pose of Interactors from Distant Social Scenes. In Proc. of ACMMM. 835–838.
  181. On the relationship between head pose social attention and personality prediction for unstructured and dynamic group interactions. In Proc. of ACM ICMI. 3–10.
  182. Detection of Division of Labor in Multiparty Collaboration. In Human Interface and the Management of Information. Information and Interaction for Learning, Culture, Collaboration and Business,. 362–371.
  183. What Your Face Vlogs About: Expressions of Emotion and Big-Five Traits Impressions in YouTube. IEEE Trans. on Affective Computing 6, 2 (2015), 193–205.
  184. Head-gestures mirroring detection in dyadic social interactions with computer vision-based wearable devices. Neurocomputing 175 (2016), 866 – 876.
  185. Predicting Gaze from Egocentric Social Interaction Videos and IMU Data. In ACM ICMI. 717–722.
  186. Group Dynamics and Multimodal Interaction Modeling Using a Smart Digital Signage. In Proc. of ECCVw. 362–371.
  187. Multiparty Interaction Understanding Using Smart Multimodal Digital Signage. IEEE Trans.s on Human-Machine Systems 44, 5 (2014), 625–637.
  188. Annotation and recognition of personality traits in spoken conversations from the AMI meetings corpus. In Proc. of INTERSPEECH. 1183–1186.
  189. Joint Estimation of Human Pose and Conversational Groups from Social Scenes. Proc. of ICCV 126 (2018), 410–429.
  190. Arno Veenstra and Hayley Hung. 2011. Do they like me? Using video cues to predict desires during speed-dates. In IEEE ICCV Workshops. 838–845.
  191. Alessandro Vinciarelli and Gelareh Mohammadi. 2014. A Survey of Personality Computing. IEEE Trans. on Affective Comput. 5, 3 (2014), 273–291.
  192. Social signal processing: Survey of an emerging domain. Image and Vision Computing 27, 12 (2009), 1743 – 1759.
  193. Understanding Social Signals in Multi-party Conversations: Automatic Recognition of Socio-Emotional Roles in the AMI Meeting Corpus. In Proc. of IEEE SMC. 374–379.
  194. Temporal segment networks: Towards good practices for deep action recognition. In Proc. of ECCV. Springer, 20–36.
  195. Automatic detection of speaker state: Lexical, prosodic, and phonetic approaches to level-of-interest and intoxication classification. Computer Speech & Language 27, 1 (2013), 168–189.
  196. Generic Representation Learning for Dynamic Social Interaction. In Proc. of ACM SIGKDD Int. Conf. on KDDM MLG Workshop.
  197. Modeling therapist empathy through prosody in drug addiction counseling. In Proc. of ISCA.
  198. Multimodal detection of salient behaviors of approach-avoidance in dyadic interactions. In Proc. of ACM ICMI. 141–144.
  199. Modeling therapist empathy and vocal entrainment in drug addiction counseling. In Proc. of INTERSPEECH. 2861–2865.
  200. Analyzing speech rate entrainment and its relation to therapist empathy in drug addiction counseling. In Proc. of ISCA.
  201. Analysis Environment of Conversational Structure with Nonverbal Multimodal Data. In Proc. of ACM ICMI-MLMI.
  202. A. Zadeh and P. Pu. 2018. Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. In Proc. of ACL.
  203. Space Speaks: Towards Socially and Personality Aware Visual Surveillance. In Proc. of ACM MPVA. 37–42.
  204. Multiparty Visual Co-Occurrences for Estimating Personality Traits in Group Meetings. In Proc. of IEEE WACV.
  205. Improved Visual Focus of Attention Estimation and Prosodic Features for Analyzing Group Interactions. In Proc. of ACM ICMI. 385–394.
  206. Lingyu Zhang and Richard J. Radke. 2020. A Multi-Stream Recurrent Neural Network for Social Role Detection in Multiparty Interactions. IEEE Journal of Selected Topics in Signal Processing 14, 3 (2020), 554–567.
  207. The I in Team: Mining Personal Social Interaction Routine with Topic Models from Long-Term Team Data. In Proc. of IUI. 421–426.
  208. TeamSense: Assessing Personal Affect and Group Cohesion in Small Teams through Dyadic Interaction and Behavior Analysis with Wearable Sensors. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3 (2018).
  209. Formation Period Matters: Towards Socially Consistent Group Detection via Dense Subgraph Seeking. In Proc. of ACM ICMR. 475–478.
  210. Predicting Group Performances Using a Personality Composite-Network Architecture During Collaborative Task. In Proc. of INTERSPEECH. 1676–1680.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now