2000 character limit reached
Fine-Grained Analysis of Team Collaborative Dialogue (2312.05471v1)
Published 9 Dec 2023 in cs.CL and cs.AI
Abstract: Natural language analysis of human collaborative chat dialogues is an understudied domain with many unique challenges: a large number of dialogue act labels, underspecified and dynamic tasks, interleaved topics, and long-range contextual dependence. While prior work has studied broad metrics of team dialogue and associated performance using methods such as LSA, there has been little effort in generating fine-grained descriptions of team dynamics and individual performance from dialogue. We describe initial work towards developing an explainable analytics tool in the software development domain using Slack chats mined from our organization, including generation of a novel, hierarchical labeling scheme; design of descriptive metrics based on the frequency of occurrence of dialogue acts; and initial results using a transformer + CRF architecture to incorporate long-range context.
- Tatiana Anikina and Ivana Kruijff-Korbayova. 2019. Dialogue act classification in team communication for robot assisted disaster response. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 399–410, Stockholm, Sweden. Association for Computational Linguistics.
- John Langshaw Austin. 1962. How to do things with words. William James Lectures. Oxford University Press.
- Teamwork as an Essential Component of High-Reliability Organizations. Health Services Research, 41(4p2):1576–1598.
- Longformer: The long-document transformer. CoRR, abs/2004.05150.
- Analyzing communication sequences for team training needs assessment. Human Factors, 40(4):672–679.
- Defining competencies and establishing team training requirements. Team effectiveness and decision making in organizations, pages 333 – 380.
- Dialogue act recognition via crf-attentive structured network. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, page 225–234, New York, NY, USA. Association for Computing Machinery.
- Pretrained language models for sequential sentence classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3693–3699, Hong Kong, China. Association for Computational Linguistics.
- BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
- A Document Analysis Method for Characterizing Design Team Performance . Journal of Mechanical Design, 126(3):378–385.
- Evaluation of latent semantic analysis-based measures of team communications content. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 47(3):424–428.
- Towards Computational Tools for Supporting the Reflective Team, pages 305–325. Springer Netherlands, Dordrecht.
- spaCy: Industrial-strength Natural Language Processing in Python.
- Bag of tricks for efficient text classification.
- Automating measurement of team cognition through analysis of communication data.
- Visualizing real-time language-based feedback on teamwork behavior in computer-mediated groups. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’09, page 537–546, New York, NY, USA. Association for Computing Machinery.
- Using context information for dialog act classification in DNN framework. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2170–2178, Copenhagen, Denmark. Association for Computational Linguistics.
- Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
- Melanie J. Martin and Peter W. Foltz. 2004. Automated team discourse annotation and performance prediction using lsa. In Proceedings of the Human Language Technology and North American Association for Computational.
- Michael F. McTear. 2002. Spoken dialogue technology: Enabling the conversational user interface. ACM Comput. Surv., 34(1):90–169.
- The relationship between task-induced stress, vocal changes, and physiological state during a dyadic team task. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, ICMI ’17, page 426–432, New York, NY, USA. Association for Computing Machinery.
- Is there a “Big Five” in Teamwork? Small Group Research, 36(5):555–599.
- Assessing the language of chat for teamwork dialogue. Educational Technology and Society, 20:224–237.
- Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–374.
- Stefan Trausan-Matu. 2010. Automatic support for the analysis of online collaborative learning chat conversations. In Hybrid Learning, pages 383–394, Berlin, Heidelberg. Springer Berlin Heidelberg.