Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Qualitative software engineering research -- reflections and guidelines (1712.08341v3)

Published 22 Dec 2017 in cs.SE and cs.CY

Abstract: Researchers are increasingly recognizing the importance of human aspects in software development. Since qualitative methods are used to explore human behavior in-depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social sciences. The purpose of this study was to extend the software engineering community's current body of knowledge regarding available qualitative methods and their quality assurance frameworks, and to provide recommendations and guidelines for their use. With the support of an epistemological argument and a survey of the literature, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria. We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (141)
  1. Weinberg GM. The psychology of computer programming. New York: Van Nostrand Reinhold . 1971.
  2. Highsmith J. Agile software development ecosystems. Addison-Wesley Longman Publishing Co., Inc. . 2002.
  3. Stebbins RA. Exploratory research in the social sciences. 48. Sage . 2001.
  4. Brown RB. Doing your dissertation in business and management: the reality of researching and writing. Sage . 2006.
  5. Seaman CB. Qualitative methods in empirical studies of software engineering. IEEE Transactions on software engineering 1999; 25(4): 557–572.
  6. Langley A. Strategies for theorizing from process data. Academy of Management review 1999; 24(4): 691–710.
  7. Runeson P, Höst M. Guidelines for conducting and reporting case study research in software engineering. Empirical software engineering 2009; 14(2): 131.
  8. Stol KJ, Fitzgerald B. The ABC of software engineering research. ACM Transactions on Software Engineering and Methodology (TOSEM) 2018; 27(3): 1–51.
  9. Hoda R. Socio-technical grounded theory for software engineering. IEEE Transactions on Software Engineering 2021.
  10. Willig C. Introducing qualitative research in psychology. Berkshire, UK: McGraw-Hill Education . 2013.
  11. Ponterotto JG. Qualitative research in counseling psychology: A primer on research paradigms and philosophy of science.. Journal of counseling psychology 2005; 52(2): 126–136.
  12. LeGreco M, Tracy SJ. Discourse tracing as qualitative practice. Qualitative Inquiry 2009.
  13. Tracy SJ. Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative inquiry 2010; 16(10): 837–851.
  14. Seale C. Quality in qualitative research. Qualitative inquiry 1999; 5(4): 465–478.
  15. Johnson RB, Onwuegbuzie AJ. Mixed methods research: A research paradigm whose time has come. Educational researcher 2004; 33(7): 14–26.
  16. SAGE. sixth edition ed. 2023.
  17. Runkel PJ, McGrath JE. Research on human behavior: A systematic guide to method. Holt, Rinehart and Winston . 1972.
  18. Mcgrath JE. Methodology Matters: Doing Research in the Behavioral and Social Sciences. In: Elsevier. 1995 (pp. 152–169)
  19. doi: 10.1007/s10664-020-09858-z
  20. doi: 10.1145/3469888
  21. Corbin JM, Strauss AL. Basics of qualitative research: techniques and procedures for developing grounded theory. SAGE. 4th ed. 2015.
  22. Hoepfl MC. Choosing Qualitative Research: A Primer for Technology Education Researchers. Journal of Technology Education 1997; 9(1): 47.
  23. Patton MQ. Qualitative evaluation and research methods. SAGE Publications, inc . 1990.
  24. Research Triangle Park North Carolina Family Health International [FHI] 2005. . 2005.
  25. Fossey EM, Harvey CA. A conceptual review of functioning: implications for the development of consumer outcome measures. Australian and New Zealand Journal of Psychiatry 2001; 35(1): 91–98.
  26. Sofaer S. Qualitative methods: what are they and why use them?. Health services research 1999; 34(5 Pt 2): 1101.
  27. Rolfe G. Validity, trustworthiness and rigour: quality and the idea of qualitative research. Journal of advanced nursing 2006; 53(3): 304–310.
  28. Lincoln YS, Guba EG. Naturalistic inquiry. 75. Sage . 1985.
  29. LeCompte MD, Goetz JP. Problems of reliability and validity in ethnographic research. Review of educational research 1982; 52(1): 31–60.
  30. Maxwell J. Understanding and validity in qualitative research. Harvard educational review 1992; 62(3): 279–301.
  31. Sandelowski M. The problem of rigor in qualitative research.. Advances in nursing science 1986; 8(3): 27–37.
  32. Sandelowski M. Rigor or rigor mortis: the problem of rigor in qualitative research revisited.. Advances in nursing science 1993; 16(2): 1–8.
  33. Hammersley M. Reading ethnographic research: A critical guide. Longman . 1990.
  34. Myers MD, others . Qualitative research in information systems. Management Information Systems Quarterly 1997; 21(2): 241–242.
  35. Glaser BG, Strauss AL. The discovery of grounded theory: Strategies for qualitative research. Transaction publishers . 2009.
  36. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative research in psychology 2006; 3(2): 77–101.
  37. DeFranco JF, Laplante PA. A content analysis process for qualitative software engineering research. Innovations in Systems and Software Engineering 2017: 1–13.
  38. Harper D. The visual ethnographic narrative. Visual Anthropology 1987; 1(1): 1–19.
  39. Willig C. Reflections on the use of a phenomenological method. Qualitative Research in Psychology 2007; 4(3): 209–225.
  40. Torraco RJ. Writing Integrative Literature Reviews: Guidelines and Examples. Human Resource Development Review 2005; 4(3): 356–367. doi: 10.1177/1534484305278283
  41. Snyder H. Literature review as a research methodology: An overview and guidelines. Journal of Business Research 2019; 104: 333–339. doi: 10.1016/j.jbusres.2019.07.039
  42. Patton MQ. Qualitative research. Wiley Online Library . 2005.
  43. doi: 10.1007/s11192-015-1798-9
  44. Mongeon P, Paul-Hus A. The Journal Coverage of Web of Science and Scopus: A Comparative Analysis. Scientometrics 2016; 106(1): 213–228. doi: 10.1007/s11192-015-1765-5
  45. doi: 10.1108/EL-12-2011-0174
  46. doi: 10.1007/s11192-015-1608-4
  47. Smith JA. Qualitative psychology: A practical guide to research methods. Sage . 2007.
  48. American Psychological Association . 2003.
  49. Chia R. Discourse analysis organizational analysis. Organization 2000; 7(3): 513–518.
  50. Czarniawska B. A narrative approach to organization studies. 43. Sage Publications . 1997.
  51. Weick KE. Sensemaking in organizations. 3. Sage . 1995.
  52. Gill MJ. The possibilities of phenomenology for organizational research. Organizational Research Methods 2014; 17(2): 118–137.
  53. Tomkins L, Eatough V. The feel of experience: Phenomenological ideas for organizational research. Qualitative Research in Organizations and Management: An International Journal 2013; 8(3): 258–275.
  54. Gough B. Reflexivity in qualitative psychological research. Journal of Positive Psychology 2016: 1–2.
  55. doi: 10.46743/2160-3715/2017.2552
  56. Gough B, Madill A. Subjectivity in psychological science: From problem to prospect.. Psychological methods 2012; 17(3): 374.
  57. Watt D. On becoming a qualitative researcher: The value of reflexivity. The Qualitative Report 2007; 12(1): 82–101.
  58. Morrow SL. Quality and trustworthiness in qualitative research in counseling psychology.. Journal of counseling psychology 2005; 52(2): 250.
  59. Guillemin M, Gillam L. Ethics, reflexivity, and “ethically important moments” in research. Qualitative inquiry 2004; 10(2): 261–280.
  60. Bradbury-Jones C. Enhancing rigour in qualitative health research: exploring subjectivity through Peshkin’s I’s. Journal of Advanced Nursing 2007; 59(3): 290–298.
  61. Berger R. Now I see it, now I don’t: Researcher’s position and reflexivity in qualitative research. Qualitative Research 2015; 15(2): 219–234.
  62. Smith S. Encouraging the use of reflexivity in the writing up of qualitative research. International Journal of Therapy and Rehabilitation 2006; 13(5): 209–215. doi: 10.12968/ijtr.2006.13.5.21377
  63. Mays N, Pope C. Assessing quality in qualitative research. BMJ: British Medical Journal 2000; 320(7226): 50.
  64. Russell G, Kelly N. Research as Interacting Dialogic Processes: Implications for Reflexivity. Forum : Qualitative Social Research 2002; 3.
  65. Buckner S. Taking the debate on reflexivity further: Psychodynamic team analysis of a BNIM interview. Journal of Social Work Practice 2005; 19(1): 59–72.
  66. Macbeth D. On “reflexivity” in qualitative research: Two readings, and a third. qualitative inquiry 2001; 7(1): 35–68.
  67. Frisina A. Back-talk Focus Groups as a Follow-Up Tool in Qualitative Migration Research: The Missing Link?. Forum : Qualitative Social Research 2006; 7(3).
  68. Josselson R. The ethical attitude in narrative research: Principles and practicalities. Handbook of narrative inquiry: Mapping a methodology 2007; 21: 545.
  69. Padgett DK. Qualitative methods in social work research. 36. Sage Publications . 2016.
  70. doi: 10.1177/104973299129121677
  71. Smith JA. Reflecting on the development of interpretative phenomenological analysis and its contribution to qualitative research in psychology. Qualitative research in psychology 2004; 1(1): 39–54.
  72. Pedler M, Abbott C. Facilitating action learning: a practitioner’s guide. Maidenhead: Open University Press . 2013.
  73. Probst B, Berenson L. The double arrow: How qualitative social work researchers use reflexivity. Qualitative Social Work 2014; 13(6): 813–827. doi: 10.1177/1473325013506248
  74. Millward LJ. The transition to motherhood in an organizational context: An interpretative phenomenological analysis. Journal of Occupational and Organizational Psychology 2006; 79(3): 315–333.
  75. Gill MJ. Elite identity and status anxiety: An interpretative phenomenological analysis of management consultants. Organization 2015; 22(3): 306–325.
  76. Malley-Keighran MP, Coleman M. ‘I am not a tragedy. I am full of hope’: communication impairment narratives in newspapers. International journal of language & communication disorders 2014; 49(2): 174–188.
  77. Tsui A. Complexities of identity formation: A narrative inquiry of an EFL teacher. Tesol Quarterly 2007; 41(4): 657–680.
  78. Aritz J, Walker RC. Cognitive organization and identity maintenance in multicultural teams: A discourse analysis of decision-making meetings. The Journal of Business Communication (1973) 2010; 47(1): 20–41.
  79. Jian G. Articulating circumstance, identity and practice: toward a discursive framework of organizational changing. Organization 2011; 18(1): 45–64.
  80. Smith JA. Evaluating the contribution of interpretative phenomenological analysis. Health psychology review 2011; 5(1): 9–27.
  81. Smith JA. Qualitative psychology: A practical guide to research methods. Sage . 2015.
  82. Pietkiewicz I, Smith JA. A practical guide to using interpretative phenomenological analysis in qualitative research psychology. Psychological Journal 2014; 20(1): 7–14.
  83. Brocki JM, Wearden AJ. A critical evaluation of the use of interpretative phenomenological analysis (IPA) in health psychology. Psychology and health 2006; 21(1): 87–108.
  84. Sage . 2009.
  85. Pietkiewicz I, Smith JA. Praktyczny przewodnik interpretacyjnej analizy fenomenologicznej w badaniach jakościowych w psychologii. Czasopismo Psychologiczne 2012; 18(2): 361–369.
  86. Makkreel RA. Husserl, Dilthey and the Relation of the Life-World to History. Research in Phenomenology 1982; 12: 39–58.
  87. Moran D. Husserl’s Crisis of the European sciences and transcendental phenomenology: an introduction. Cambridge University Press . 2012.
  88. Larkin M, Thompson A. Interpretative phenomenological analysis. Qualitative research methods in mental health and psychotherapy: A guide for students and practitioners 2012: 99–116.
  89. Weise P. Frictions in Software Development : An Interpretive Phenomenological Analysis. Universität Stuttgart 2021. doi: 10.18419/OPUS-12079
  90. Singh K, Strobel J. Exploring Lived Experiences of Agile Developers with Daily Stand-up Meetings: A Phenomenological Study. Behaviour & Information Technology 2023; 42(4): 403–423. doi: 10.1080/0144929X.2021.2023636
  91. Heider F, Simmel M. An experimental study of apparent behavior. The American journal of psychology 1944; 57(2): 243–259.
  92. Murray M. Narrative psychology. Qualitative psychology: A practical guide to research methods 2003: 111–131.
  93. Kugelmann R. Introducing narrative psychology: Self, trauma and the construction of meaning. Journal of health psychology 2001; 6(5): 604–606.
  94. Ricoeur P. Time and narrative. 3. University of Chicago Press . 2010.
  95. Riessman CK. Narrative analysis. Narrative, memory & everyday life 2005: 1–7.
  96. Kohler-Riessman C. Analysis of personal narratives. Qualitative research in social work 2000: 168–191.
  97. Sarbin TR. Narrative psychology: The storied nature of human conduct. Praeger Publishers/Greenwood Publishing Group . 1986.
  98. Crossley M. Introducing narrative psychology. McGraw-Hill Education (UK) . 2000.
  99. Burck C. Comparing qualitative research methodologies for systemic research: The use of grounded theory, discourse analysis and narrative analysis. Journal of family therapy 2005; 27(3): 237–262.
  100. Flick U. An introduction to qualitative research. Sage . 2014.
  101. Overcash JA. Narrative research: a review of methodology and relevance to clinical practice. Critical reviews in oncology/hematology 2003; 48(2): 179–184.
  102. Shkedi A. Multiple case narrative: A qualitative approach to studying multiple populations. 7. John Benjamins Publishing . 2005.
  103. Connelly FM, Clandinin DJ. Narrative inquiry: Experience and story in qualitative research. Educational Researcher 2000; 6: 94–118.
  104. Koller V. How to Analyse Collective Identity in Discourse-Textual and Contextual Parameters.. Critical Approaches to Discourse Analysis across Disciplines 2012; 5(2).
  105. Morgan A. Discourse analysis: An overview for the neophyte researcher. Journal of Health and Social Care Improvement 2010; 1(May): 1–7.
  106. Crowe M. The power of the word: some post-structural considerations of qualitative approaches in nursing research. Journal of advanced nursing 1998; 28(2): 339–344.
  107. Gee JP. An introduction to discourse analysis: Theory and method. Routledge . 2014.
  108. Lyons J. Introduction to theoretical linguistics. Cambridge university press . 1968.
  109. Chandler D. Semiotics: the basics. Routledge . 2007.
  110. Georgaca E, Avdi E. Discourse analysis. Qualitative research methods in mental health and psychotherapy: A guide for students and practitioners 2012: 147–162.
  111. Parker I. Critical discursive psychology. Springer . 2015.
  112. McLeod J. Qualitative approaches to research in counselling and psychotherapy: Issues and challenges. British Journal of Guidance and Counselling 1996; 24(3): 309–316.
  113. Cutcliffe JR, Ramcharan P. Leveling the playing field? Exploring the merits of the ethics-as-process approach for judging qualitative research proposals. Qualitative Health Research 2002; 12(7): 1000–1010.
  114. Ginger Hofman N. Toward critical research ethics: transforming ethical conduct in qualitative health care research. Health Care for Women International 2004; 25(7): 647–662.
  115. Liamputtong P. Qualitative research methods. Australia: Oxford University Press . 2013.
  116. Stuhlmiller CM. Narrative methods in qualitative research: Potential for therapeutic transformation. The emotional nature of qualitative research 2001: 63–80.
  117. Hunter SV. Analysing and representing narrative data: The long and winding road. Current Narratives 2010; 1(2): 44–54.
  118. Kapoulas A, Mitic M. Understanding challenges of qualitative research: Rhetorical issues and reality traps. Qualitative Market Research: An International Journal 2012; 15(4): 354–368.
  119. Atieno OP. An analysis of the strengths and limitation of qualitative and quantitative research paradigms. Problems of Education in the 21st Century 2009; 13(1): 13–38.
  120. Maxwell JA. Qualitative research design: An interactive approach. 41. Sage publications . 2012.
  121. Foster E. Adolescents’ Experience of’Adjustment’to Life with Diabetes: An Interpretative Phenomenological Analysis. PhD thesis. University of Hertfordshire, University of Hertfordshire; 2010.
  122. Clancy M. Is reflexivity the key to minimising problems of interpretation in phenomenological research?. Nurse researcher 2013; 20(6): 12–16.
  123. Polkinghorne DE. Validity issues in narrative research. Qualitative inquiry 2007; 13(4): 471–486.
  124. Sas D, Avgeriou P. Quality attribute trade-offs in the embedded systems industry: an exploratory case study. Software Quality Journal 2019: 1–30.
  125. Ghobadi S, Mathiassen L. A Generational Perspective on the Software Workforce: Precocious Users of Social Networking in Software Development. Journal of Management Information Systems 2020; 37(1): 96–128.
  126. Lyon LA, Green E. Women in coding boot camps: an alternative pathway to computing jobs. Computer Science Education 2020; 30(1): 102–123.
  127. Dennehy D, Conboy K. Breaking the flow: a study of contradictions in information systems development (ISD). Information Technology & People 2019.
  128. Fannoun S, Kerins J. Towards organisational learning enhancement: assessing software engineering practice. The Learning Organization 2019.
  129. Kurtanović Z, Maalej W. On user rationale in software engineering. Requirements Engineering 2018; 23(3): 357–379.
  130. Ebad SA. The influencing causes of software unavailability: A case study from industry. Software: Practice and Experience 2018; 48(5): 1056–1076.
  131. Senapathi M, Drury-Grogan ML. Refining a model for sustained usage of agile methodologies. Journal of Systems and Software 2017; 132: 298–316.
  132. Siddoo V, Wongsai N. Factors Influencing the Adoption of ISO/IEC 29110 in Thai Government Projects: A Case Study. International Journal of Information Technologies and Systems Approach (IJITSA) 2017; 10(1): 22–44.
  133. Bass JM. Artefacts and agile method tailoring in large-scale offshore software development programmes. Information and Software Technology 2016; 75: 1–16.
  134. Hoda R, Murugesan LK. Multi-level agile project management challenges: A self-organizing team perspective. Journal of Systems and Software 2016; 117: 245–257.
  135. Mishra GTSK. Professional identity construction among software engineering students: A study in India. Information Technology & People 2016; 29(1): 146–172.
  136. Zayour I, Hamdar A. A qualitative study on debugging under an enterprise IDE. Information and Software Technology 2016; 70: 130–139.
  137. Taylor KJ. Adopting Agile software development: the project manager experience. Information Technology & People 2016.
  138. Gandomani TJ, Nafchi MZ. Agile transition and adoption human-related challenges and issues: A Grounded Theory approach. Computers in Human Behavior 2016; 62: 257–266.
  139. Gandomani TJ, Nafchi MZ. An empirically-developed framework for Agile transition and adoption: A Grounded Theory approach. Journal of Systems and Software 2015; 107: 204–219.
  140. Giuffrida R, Dittrich Y. A conceptual framework to study the role of communication through social software for coordination in globally-distributed software teams. Information and Software Technology 2015; 63: 11–30.
  141. doi: 10.5539/ijbm.v13n12p140
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Per Lenberg (9 papers)
  2. Robert Feldt (80 papers)
  3. Lucas Gren (45 papers)
  4. Lars Göran Wallgren Tengberg (3 papers)
  5. Inga Tidefors (1 paper)
  6. Daniel Graziotin (52 papers)
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now