Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Seeking Enlightenment: Incorporating Evidence-Based Practice Techniques in a Research Software Engineering Team (2403.16827v1)

Published 25 Mar 2024 in cs.SE

Abstract: Evidence-based practice (EBP) in software engineering aims to improve decision-making in software development by complementing practitioners' professional judgment with high-quality evidence from research. We believe the use of EBP techniques may be helpful for research software engineers (RSEs) in their work to bring software engineering best practices to scientific software development. In this study, we present an experience report on the use of a particular EBP technique, rapid reviews, within an RSE team at Sandia National Laboratories, and present practical recommendations for how to address barriers to EBP adoption within the RSE community.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. R. Baxter, N. C. Hong, D. Gorissen, J. Hetherington, and I. Todorov, “The research software engineer,” in Digital Research Conference, Oxford, 2012, pp. 1–3.
  2. T. Dyba, B. A. Kitchenham, and M. Jorgensen, “Evidence-based software engineering for practitioners,” IEEE software, vol. 22, no. 1, pp. 58–65, 2005.
  3. R. Milewicz, J. Willenbring, and D. Vigil, “Research, develop, deploy: Building a full spectrum software engineering and research department,” 2020.
  4. J. M. Willenbring and R. Milewicz, “Moving forward together: How a software engineering department can impact developer productivity in a research organization.” The 2020 Collegeville Workshop on Scientific Software, 6 2020. [Online]. Available: https://www.osti.gov/biblio/1806263
  5. B. Cartaxo, G. Pinto, and S. Soares, “The role of rapid reviews in supporting decision-making in software engineering practice,” in Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018, 2018, pp. 24–34.
  6. D. S. Katz, K. McHenry, C. Reinking, and R. Haines, “Research software development & management in universities: case studies from manchester’s rsds group, illinois’ ncsa, and notre dame’s crc,” in 2019 IEEE/ACM 14th International Workshop on Software Engineering for Science (SE4Science).   IEEE, 2019, pp. 17–24.
  7. I. A. Cosden, “The princeton university rse group model: Operational and organizational approaches,” Computing in Science & Engineering, 2023.
  8. A. Malviya-Thakur, D. E. Bernholdt, W. F. Godoy, G. R. Watson, M. Doucet, M. A. Coletti, D. M. Rogers, M. McDonnell, J. J. Billings, and B. Maccabe, “Research software engineering at oak ridge national laboratory,” Computing in Science & Engineering, 2023.
  9. D. S. Katz, L. C. McInnes, D. E. Bernholdt, A. C. Mayes, N. P. C. Hong, J. Duckles, S. Gesing, M. A. Heroux, S. Hettrick, R. C. Jimenez et al., “Community organizations: Changing the culture in which research software is developed and sustained,” Computing in Science & Engineering, vol. 21, no. 2, pp. 8–24, 2018.
  10. T. Hacker, P. Smith, D. Brunson, L. Arafune, T. Cheatham, and E. Deelman, “Building the research innovation workforce: Challenges and recommendations from a virtual workshop to advance the research computing community,” in Practice and Experience in Advanced Research Computing, 2022, pp. 1–7.
  11. D. E. Bernholdt, J. Cary, M. Heroux, and L. C. McInnes, “The science of scientific software development and use,” 5 2022. [Online]. Available: https://www.osti.gov/biblio/1846008
  12. M. Mundt, K. Beattie, J. Bisila, C. Ferenbaugh, W. Godoy, R. Gupta, J. Guyer, M. Kiran, A. Malviya-Thakur, R. Milewicz et al., “For the public good: Connecting, retaining, and recognizing current and future rses at national organizations,” Computing in Science & Engineering, 2023.
  13. S. Hettrick, R. Bast, S. Crouch, C. Wyatt, O. Philippe, A. Botzki, J. Carver, I. Cosden, F. D’Andrea, A. Dasgupta, W. Godoy, A. Gonzalez-Beltran, U. Hamster, S. Henwood, P. Holmvall, S. Janosch, T. Lestang, N. May, J. Philips, N. Poonawala-Lohani, P. Richmond, M. Sinha, F. Thiery, B. Werkhoven, and Q. Zhang, “International rse survey 2022,” Aug. 2022, If you use this dataset, please cite it using the metadata from this file. [Online]. Available: https://doi.org/10.5281/zenodo.7015772
  14. A.-L. Lamprecht, C. Martinez-Ortiz, M. Barker, S. L. Bartholomew, J. Barton, N. C. Hong, J. Cohen, S. Druskat, J. Forest, J.-N. Grad et al., “What do we (not) know about research software engineering?” Journal of Open Research Software, vol. 10, 2022.
  15. K. Yoshii, “What does the post-moore era mean for research software engineering?” Research Software Engineers in HPC (RSE-HPC-2021), 2021.
  16. B. van Werkhoven, W. J. Palenstijn, and A. Sclocco, “Lessons learned in a decade of research software engineering gpu applications,” in Computational Science–ICCS 2020: 20th International Conference, Amsterdam, The Netherlands, June 3–5, 2020, Proceedings, Part VII 20.   Springer, 2020, pp. 399–412.
  17. M. Meinel and M. Stoffers, “Revisiting secure software engineering for research software,” in RSE Conference 2022, September 2022. [Online]. Available: https://elib.dlr.de/188479/
  18. R. Milewicz, J. Carver, S. Grayson, and T. Atkison, “A secure future for open-source computational science and engineering,” Computing in Science & Engineering, vol. 24, no. 4, pp. 65–69, 2022.
  19. L. Morris, “Understanding software sustainability in the field of research software engineering,” Ph.D. dissertation, University of Huddersfield, 2021.
  20. G. Orlando, “Assessing chatgpt for coding finite element methods,” no. MOX-Report No. 31/2023, 2023.
  21. S. R. Piccolo, P. Denny, A. Luxton-Reilly, S. Payne, and P. G. Ridge, “Many bioinformatics programming tasks can be automated with chatgpt,” arXiv preprint arXiv:2303.13528, 2023.
  22. J. Cohen, D. S. Katz, M. Barker, N. C. Hong, R. Haines, and C. Jay, “The four pillars of research software engineering,” IEEE Software, vol. 38, no. 1, pp. 97–105, 2020.
  23. I. A. Cosden, K. McHenry, and D. S. Katz, “Research software engineers: Career entry points and training gaps,” Computing in Science & Engineering, 2023.
  24. R. Milewicz and M. Mundt, “An exploration of the mentorship needs of research software engineers,” Research Software Engineers in HPC (RSE-HPC-2021), 2021.
  25. J. C. Carver, N. Weber, K. Ram, S. Gesing, and D. S. Katz, “A survey of the state of the practice for research software in the united states,” PeerJ Computer Science, vol. 8, p. e963, 2022.
  26. D. Trumbo and R. Milewicz, “Poster: Towards a culture of continuous learning and improvement within rse teams.”   2021 Collegeville Workshop on Scientific Software, 2021.
  27. J. M. Satterfield, B. Spring, R. C. Brownson, E. J. Mullen, R. P. Newhouse, B. B. Walker, and E. P. Whitlock, “Toward a transdisciplinary model of evidence-based practice,” The Milbank Quarterly, vol. 87, no. 2, pp. 368–390, 2009.
  28. B. A. Kitchenham, T. Dyba, and M. Jorgensen, “Evidence-based software engineering,” in Proceedings. 26th International Conference on Software Engineering.   IEEE, 2004, pp. 273–281.
  29. B. Kitchenham, O. P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, “Systematic literature reviews in software engineering–a systematic literature review,” Information and software technology, vol. 51, no. 1, pp. 7–15, 2009.
  30. S. Pizard, D. Vallespir, and B. Kitchenham, “A longitudinal case study on the effects of an evidence-based software engineering training,” in Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training, 2022, pp. 1–13.
  31. P. Devanbu, T. Zimmermann, and C. Bird, “Belief & evidence in empirical software engineering,” in Proceedings of the 38th international conference on software engineering, 2016, pp. 108–119.
  32. C. Le Goues, C. Jaspan, I. Ozkaya, M. Shaw, and K. T. Stolee, “Bridging the gap: From research to practical advice,” IEEE Software, vol. 35, no. 5, pp. 50–57, 2018.
  33. M. A. Heroux, “Research software science: Expanding the impact of research software engineering,” Computing in Science & Engineering, 2023.
  34. B. H. Sims, “Research software engineering: Professionalization, roles, and identity,” Los Alamos National Lab.(LANL), Los Alamos, NM (United States), Tech. Rep., 2022.
  35. S. Sufi, “The rise of a new digital third space professional in higher education: Recognising research software engineering,” 2021.
  36. M. Mundt and R. Milewicz, “Working in harmony: Towards integrating rses into multi-disciplinary cse teams,” Workshop on the Science of Scientific-Software Development and Use, sponsored by the U.S. Department of Energy, Office of Advanced Scientific Computing Research, Dec 2021.
  37. R. Milewicz and M. Mundt, “Building bridges: Establishing a dialogue between software engineering research and computational science,” Workshop on the Science of Scientific-Software Development and Use, sponsored by the U.S. Department of Energy, Office of Advanced Scientific Computing Research, Dec 2021.
  38. S. Khangura, K. Konnyu, R. Cushman, J. Grimshaw, and D. Moher, “Evidence summaries: the evolution of a rapid review approach,” Systematic reviews, vol. 1, no. 1, pp. 1–9, 2012.
  39. B. Cartaxo, G. Pinto, B. Fonseca, M. Ribeiro, P. Pinheiro, M. T. Baldassarre, and S. Soares, “Software engineering research community viewpoints on rapid reviews,” in 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).   IEEE, 2019, pp. 1–12.
  40. B. Cartaxo, G. Pinto, and S. Soares, “Rapid reviews in software engineering,” Contemporary Empirical Methods in Software Engineering, pp. 357–384, 2020.
  41. M. Niazi, “Do systematic literature reviews outperform informal literature reviews in the software engineering domain? an initial case study,” Arabian Journal for Science and Engineering, vol. 40, pp. 845–855, 2015.
  42. L. Scurlock-Evans, P. Upton, and D. Upton, “Evidence-based practice in physiotherapy: a systematic review of barriers, enablers and interventions,” Physiotherapy, vol. 100, no. 3, pp. 208–219, 2014.
  43. A. Kasoju, K. Petersen, and M. V. Mäntylä, “Analyzing an automotive testing process with evidence-based software engineering,” Information and Software Technology, vol. 55, no. 7, pp. 1237–1259, 2013.
  44. S. Pizard, F. Acerenza, D. Vallespir, and B. Kitchenham, “Assessing attitudes towards evidence-based software engineering in a government agency,” Information and Software Technology, vol. 154, p. 107101, 2023.
  45. J. A. Holton, “The coding process and its challenges,” The Sage handbook of grounded theory, vol. 3, pp. 265–289, 2007.
  46. R. M. Milewicz, J. Bisila, M. Mundt, and J. Teves, “Rapid Review Dataset for Seeking Enlightenment: Incorporating Evidence-Based Practice Techniques in a Research Software Engineering Team,” Jul. 2023. [Online]. Available: https://doi.org/10.5281/zenodo.8169610
  47. L. Freund, “Contextualizing the information-seeking behavior of software engineers,” Journal of the Association for Information Science and Technology, vol. 66, no. 8, pp. 1594–1605, 2015.
  48. L. Hartling, J.-M. Guise, E. Kato, J. Anderson, S. Belinson, E. Berliner, D. M. Dryden, R. Featherstone, M. D. Mitchell, M. Motu’Apuaka et al., “A taxonomy of rapid reviews links report types and methods to specific decision-making contexts,” Journal of Clinical Epidemiology, vol. 68, no. 12, pp. 1451–1462, 2015.
  49. R. Borah, A. W. Brown, P. L. Capers, and K. A. Kaiser, “Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the prospero registry,” BMJ open, vol. 7, no. 2, p. e012545, 2017.
  50. D. M. Rousseau and B. C. Gunia, “Evidence-based practice: The psychology of ebp implementation,” Annual review of psychology, vol. 67, pp. 667–692, 2016.
  51. B. M. Melnyk, E. Fineout-Overholt, N. Fischbeck Feinstein, H. Li, L. Small, L. Wilcox, and R. Kraus, “Nurses’ perceived knowledge, beliefs, skills, and needs regarding evidence-based practice: Implications for accelerating the paradigm shift,” Worldviews on Evidence-Based Nursing, vol. 1, no. 3, pp. 185–193, 2004.
  52. T. L. Kiss, M. O’Malley, and T. J. Hendrix, “Self-efficacy-based training for research literature appraisal: a competency for evidence-based practice,” Journal for Nurses in Professional Development, vol. 26, no. 4, pp. 170–177, 2010.
  53. J. Leeman, L. Calancie, M. C. Kegler, C. T. Escoffery, A. K. Herrmann, E. Thatcher, M. A. Hartman, and M. E. Fernandez, “Developing theory to guide building practitioners’ capacity to implement evidence-based interventions,” Health Education & Behavior, vol. 44, no. 1, pp. 59–69, 2017.
  54. C. V. Belden, J. Leafman, G. Nehrenz, and P. Miller, “The effect of evidence based practice on workplace empowerment of rural registered nurses,” Online Journal of Rural Nursing and Health Care, vol. 12, no. 2, pp. 64–76, 2012.
  55. A. Dalheim, S. Harthug, R. M. Nilsen, and M. W. Nortvedt, “Factors influencing the development of evidence-based practice among nurses: a self-report survey,” BMC health services research, vol. 12, pp. 1–10, 2012.
  56. B. M. Melnyk, E. Fineout-Overholt, L. Gallagher-Ford, and L. Kaplan, “The state of evidence-based practice in us nurses: critical implications for nurse leaders and educators,” JONA: The Journal of Nursing Administration, vol. 42, no. 9, pp. 410–417, 2012.
  57. C. Wohlin and A. Rainer, “Challenges and recommendations to publishing and using credible evidence in software engineering,” Information and software technology, vol. 134, p. 106555, 2021.
  58. M. L. Callaham, “Expert opinion: supplementing the gaps in evidence-based medicine,” Annals of Emergency Medicine, vol. 65, no. 1, pp. 61–62, 2015.
  59. D. E. Klein, D. D. Woods, G. Klein, and S. J. Perry, “Can we trust best practices? six cognitive challenges of evidence-based approaches,” Journal of Cognitive Engineering and Decision Making, vol. 10, no. 3, pp. 244–254, 2016.
  60. K. M. Goldstein, D. Vogt, A. Hamilton, S. M. Frayne, J. Gierisch, J. Blakeney, A. Sadler, B. M. Bean-Mayberry, D. Carney, B. DiLeone et al., “Practice-based research networks add value to evidence-based quality improvement,” in Healthcare, vol. 6, no. 2.   Elsevier, 2018, pp. 128–134.
  61. E. Neilson, K. C. Smith, D. Steinwachs, D. F. Phelan-Emrick, R. S. Lawrence, J. V. Bowie, and B. Cohen, “Moving research into practice: the diffusion of evidence-based recommendations through professional societies,” in Implementation Science, vol. 10, no. 1.   BioMed Central, 2015, pp. 1–2.
  62. M. A. Rodwin, “Commentary: The politics of evidence-based medicine,” Journal of Health Politics, Policy and Law, vol. 26, no. 2, pp. 439–446, 2001.
  63. E. Gambrill, “Evidence-based practice: An alternative to authority-based practice (revisiting our heritage),” Families in Society, vol. 99, no. 3, pp. 283–294, 2018.
  64. J. P. Jacobs, G. Wernovsky, D. S. Cooper, and T. R. Karl, “Principles of shared decision-making within teams,” Cardiology in the Young, vol. 25, no. 8, pp. 1631–1636, 2015.
  65. K. Kutash, M. Acri, M. Pollock, K. Armusewicz, S.-c. Serene Olin, and K. E. Hoagwood, “Quality indicators for multidisciplinary team functioning in community-based children’s mental health services,” Administration and Policy in Mental Health and Mental Health Services Research, vol. 41, pp. 55–68, 2014.
  66. A. Mathieson, G. Grande, and K. Luker, “Strategies, facilitators and barriers to implementation of evidence-based practice in community nursing: a systematic mixed-studies review and qualitative synthesis,” Primary health care research & development, vol. 20, p. e6, 2019.
  67. D. F. Kelly, “A software chasm: Software engineering and scientific computing,” IEEE software, vol. 24, no. 6, pp. 120–119, 2007.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets