Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ECHO: An Automated Contextual Inquiry Framework for Anonymous Qualitative Studies using Conversational Assistants (2312.07576v1)

Published 10 Dec 2023 in cs.HC

Abstract: Qualitative research studies often employ a contextual inquiry, or a field study that involves in-depth observation and interviews of a small sample of study participants, in-situ, to gain a robust understanding of the reasons and circumstances that led to the participant's thoughts, actions, and experiences regarding the domain of interest. Contextual inquiry, especially in sensitive data studies, can be a challenging task due to reasons such as participant privacy, as well as physical constraints such as in-person presence and manual analysis of the qualitative data gathered. In this work, we discuss Enqu^ete Contextuelle Habile Ordinateur (ECHO); a virtual-assistant framework to automate the erstwhile manual process of conducting contextual inquiries and analysing the respondents' subjective qualitative data. ECHO automates the contextual inquiry pipeline, while not compromising on privacy preservation or response integrity. Its adaptive conversational interface enables respondents to provide unstructured or semi-structured responses in free-form natural language, allowing researchers to explore larger narratives in participant response data. It supports response-driven exploratory questions and automates coding methodologies for qualitative data, thus enabling the inquirer to dive deeper into correlated questions and to do better cause-effect analysis. It focuses on addressing the limitations of manual annotation, bringing standardisation to free-form text, and eliminating perspective bias amongst different reviewers of subjective responses. A participatory mental health study was conducted on 167 young adults bifurcated into two focus groups; one of which was administered a conventional contextual inquiry, and the other via ECHO, virtually. ECHO outperformed on participant transparency, response detail and median time required for end-to-end inquiry completion, per participant.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Charu C Aggarwal and ChengXiang Zhai. 2012. A survey of text classification algorithms. In Mining text data. Springer, 163–222.
  2. Content analysis in studying interpersonal interaction. In Studying interpersonal interaction. Guilford Press, New York, 239–254.
  3. Product innovation processes in small firms: Combining entrepreneurial effectuation and managerial causation. Journal of Product Innovation Management 31, 3 (2014), 616–635.
  4. Karen L Bouchard. 2016. Anonymity as a double-edge sword: Reflecting on the implications of online qualitative research in studying sensitive topics. The Qualitative Report 21, 1 (2016), 59–67.
  5. Adam H Bourne and Maggie A Robson. 2015. Participants’ reflections on being interviewed about risk and sexual behaviour: implications for collection of qualitative data on sensitive topics. International journal of social research methodology 18, 1 (2015), 105–116.
  6. Jonathan Bragg and Daniel S Weld. 2018. Sprout: Crowd-powered task design for crowdsourcing. In Proceedings of the 31st annual acm symposium on user interface software and technology. 165–176.
  7. Ashley Castleberry and Amanda Nolen. 2018. Thematic analysis of qualitative research data: Is it as easy as it sounds? Currents in pharmacy teaching and learning 10, 6 (2018), 807–815.
  8. Idyll studio: A structured editor for authoring interactive & data-driven articles. In The 34th Annual ACM Symposium on User Interface Software and Technology. 1–12.
  9. Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–13.
  10. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In Proceedings of the 12th international conference on World Wide Web. 519–528.
  11. Jayati Dev and L Jean Camp. 2020. User engagement with chatbots: a discursive psychology approach. In Proceedings of the 2nd Conference on Conversational User Interfaces. 1–4.
  12. Doing sensitive research: what challenges do qualitative researchers face? Qualitative research 7, 3 (2007), 327–353.
  13. Risk to researchers in qualitative research on sensitive topics: Issues and strategies. Qualitative Health Research 18, 1 (2008), 133–144.
  14. Effects of anonymity, gender, and erotophilia on the quality of data obtained from self-reports of socially sensitive behaviors. Journal of behavioral medicine 25, 5 (2002), 439–467.
  15. The exploratory labeling assistant: Mixed-initiative label curation with large document collections. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 153–164.
  16. Geoparsing the historical gazetteers of scotland: Accurately computing location in mass digitised texts. In Proceedings of the 8th Workshop on Challenges in the Management of Large Corpora. 24–30.
  17. Marcelle: composing interactive machine learning workflows and interfaces. In The 34th Annual ACM Symposium on User Interface Software and Technology. 39–53.
  18. Barney G Glaser and Anselm L Strauss. 2017. The discovery of grounded theory: Strategies for qualitative research. Routledge.
  19. Recent named entity recognition and classification techniques: a systematic review. Computer Science Review 29 (2018), 21–43.
  20. Carol Grbich. 2012. Qualitative data analysis: An introduction. sage.
  21. Measuring mental health of the Dutch population: a comparison of the GHQ-12 and the MHI-5. Health and quality of life outcomes 2, 1 (2004), 1–6.
  22. Sungsoo Hwang. 2008. Utilizing qualitative data analysis software: A review of Atlas. ti. Social Science Computer Review 26, 4 (2008), 519–527.
  23. Text classification using machine learning techniques. WSEAS transactions on computers 4, 8 (2005), 966–974.
  24. Ethnography and human development: Context and meaning in social inquiry. University of Chicago Press.
  25. Katharina Kaiser and Silvia Miksch. 2005. Information extraction. A survey, Institute of Software Technology & Interactive Systems, Vienna University of Technology (2005).
  26. Soo-Min Kim and Eduard Hovy. 2004. Determining the sentiment of opinions. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics. 1367–1373.
  27. Tae Kyun Kim. 2017. Understanding one-way ANOVA using conceptual figures. Korean journal of anesthesiology 70, 1 (2017), 22–26.
  28. Can I Talk to You about Your Social Needs? Understanding Preference for Conversational User Interface in Health. In Proceedings of the 3rd Conference on Conversational User Interfaces. 1–10.
  29. The PHQ-9: validity of a brief depression severity measure. Journal of general internal medicine 16, 9 (2001), 606–613.
  30. Ted Kwartler. 2021. Text Analytics and Natural Language Processing. In The Machine Age of Customer Insight. Emerald Publishing Limited.
  31. The parable of Google Flu: traps in big data analysis. Science 343, 6176 (2014), 1203–1205.
  32. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies 5, 1 (2012), 1–167.
  33. Bing Liu et al. 2010. Sentiment analysis and subjectivity. Handbook of natural language processing 2, 2010 (2010), 627–666.
  34. Creative Research Question Generation for Human-Computer Interaction Research. (2023).
  35. Sharon Mallon and Iris Elliott. 2019. The emotional risks of turning stories into data: An exploration of the experiences of qualitative researchers working on sensitive topics. Societies 9, 3 (2019), 62.
  36. S Manikandan. 2011a. Frequency distribution. Journal of pharmacology & pharmacotherapeutics 2, 1 (2011), 54.
  37. S Manikandan. 2011b. Measures of central tendency: The mean. Journal of Pharmacology and Pharmacotherapeutics 2, 2 (2011), 140.
  38. Erika McCallister. 2010. Guide to protecting the confidentiality of personally identifiable information. Vol. 800. Diane Publishing.
  39. Andrew McCallum and Wei Li. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. (2003).
  40. Craig RM McKenzie. 2004. Hypothesis testing and evaluation. Blackwell handbook of judgment and decision making (2004), 200–219.
  41. Patrick E McKight and Julius Najab. 2010. Kruskal-wallis test. The corsini encyclopedia of psychology (2010), 1–1.
  42. Coming of age in Samoa. Penguin New York.
  43. Alyona Medelyan. 2020. Coding qualitative data: How to code qualitative research. Insights Thematic (2020).
  44. Matthew B Miles and A Michael Huberman. 1994. Qualitative data analysis: An expanded sourcebook. sage.
  45. Behrang Mohit. 2014. Named entity recognition. In Natural language processing of semitic languages. Springer, 221–245.
  46. Sue Newell and Marco Marabelli. 2015. Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’. The Journal of Strategic Information Systems 24, 1 (2015), 3–14.
  47. Anthony D Ong and David J Weiss. 2000. The impact of anonymity on responses to sensitive questions 1. Journal of Applied Social Psychology 30, 8 (2000), 1691–1708.
  48. Mapping Saldana’s Coding Methods onto the Literature Review Process. Journal of Educational Issues 2, 1 (2016), 130–150.
  49. Evaluating the accuracy of cloud NLP services using ground-truth experiments. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, 341–350.
  50. Fundamentals of sentiment analysis: concepts and methodology. In Sentiment analysis and ontology engineering. Springer, 25–48.
  51. Claire M Renzetti and Raymond M Lee. 1993. Researching sensitive topics. (1993).
  52. Rahime Belen Sağlam and Jason RC Nurse. 2020. Is your chatbot GDPR compliant? Open issues in agent design. In Proceedings of the 2nd Conference on Conversational User Interfaces. 1–3.
  53. Elif Kuş Saillard et al. 2011. Systematic versus interpretive analysis with two CAQDAS packages: NVivo and MAXQDA. In Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, Vol. 12.
  54. Kim Salazar. 2020. Contextual inquiry: Inspire design by observing and interviewing users in their context. https://www.nngroup.com/articles/contextual-inquiry/
  55. Ethical challenges of researchers in qualitative studies: The necessity to develop a specific guideline. Journal of medical ethics and history of medicine 7 (2014).
  56. Anonymising interview data: Challenges and compromise in practice. Qualitative research 15, 5 (2015), 616–632.
  57. A replicable comparison study of NER software: StanfordNLP, NLTK, OpenNLP, SpaCy, Gate. In 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS). IEEE, 338–343.
  58. Satoshi Sekine. 2004. Named entity: History and future. Project notes, New York University (2004), 4.
  59. Mads Soegaard and Rikke Friis Dam. 2012. The encyclopedia of human-computer interaction. The encyclopedia of human-computer interaction (2012).
  60. Anselm Strauss and Juliet Corbin. 1990. Basics of qualitative research. Sage publications.
  61. Jon Swain. 2018. A hybrid approach to thematic analysis in qualitative research: Using a practical example. SAGE Publications Ltd.
  62. The WHO-5 Well-Being Index: a systematic review of the literature. Psychotherapy and psychosomatics 84, 3 (2015), 167–176.
  63. Roger Tourangeau and Ting Yan. 2007. Sensitive questions in surveys. Psychological bulletin 133, 5 (2007), 859.
  64. The validity of sentiment analysis: Comparing manual annotation, crowd-coding, dictionary approaches, and machine learning algorithms. Communication Methods and Measures 15, 2 (2021), 121–140.
  65. Manipulating and evaluating levels of personality perceptions of voice assistants through enactment-based dialogue design. In Proceedings of the 3rd Conference on Conversational User Interfaces. 1–12.
  66. Screen2words: Automatic mobile UI summarization with multimodal learning. In The 34th Annual ACM Symposium on User Interface Software and Technology. 498–510.
  67. Terrence E White and Manjeet Rege. 2020. Sentiment Analysis on Google Cloud Platform. Issues Inf. Syst 21 (2020), 221–228.
  68. User preference and categories for error responses in conversational user interfaces. In Proceedings of the 2nd Conference on Conversational User Interfaces. 1–8.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Rishika Dwaraghanath (1 paper)
  2. Rahul Majethia (2 papers)
  3. Sanjana Gautam (11 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now