Papers
Topics
Authors
Recent
Search
2000 character limit reached

Practitioners' Challenges and Perceptions of CI Build Failure Predictions at Atlassian

Published 15 Feb 2024 in cs.SE and cs.LG | (2402.09651v2)

Abstract: Continuous Integration (CI) build failures could significantly impact the software development process and teams, such as delaying the release of new features and reducing developers' productivity. In this work, we report on an empirical study that investigates CI build failures throughout product development at Atlassian. Our quantitative analysis found that the repository dimension is the key factor influencing CI build failures. In addition, our qualitative survey revealed that Atlassian developers perceive CI build failures as challenging issues in practice. Furthermore, we found that the CI build prediction can not only provide proactive insight into CI build failures but also facilitate the team's decision-making. Our study sheds light on the challenges and expectations involved in integrating CI build prediction tools into the Bitbucket environment, providing valuable insights for enhancing CI processes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Keeping master green at scale. In Proceedings of the Fourteenth EuroSys Conference 2019. 1–15.
  2. Sebastian Baltes and Stephan Diehl. 2016. Worse than spam: Issues in sampling software developers. In Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement. 1–6.
  3. Oops, my tests broke the build: An explorative analysis of travis ci with github. In 2017 IEEE/ACM 14th International conference on mining software repositories (MSR). IEEE, 356–367.
  4. Travistorrent: Synthesizing travis ci and github for full-stack research on continuous integration. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 447–450.
  5. The silent helper: the impact of continuous integration on code reviews. In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 423–434.
  6. BUILDFAST: History-aware build outcome prediction for fast feedback and reduced cost in continuous integration. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. 42–53.
  7. Bradley Efron. 1986. How biased is the apparent error rate of a prediction rule? Journal of the American statistical Association 81, 394 (1986), 461–470.
  8. Data stream mining for predicting software build outcomes using source code metrics. Information and Software Technology 56, 2 (2014), 183–198.
  9. Foyzul Hassan and Xiaoyin Wang. 2017. Change-aware build prediction model for stall avoidance in continuous integration. In 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, 157–162.
  10. Trade-offs in continuous integration: assurance, security, and flexibility. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 197–207.
  11. Usage, costs, and benefits of continuous integration in open-source projects. In Proceedings of the 31st IEEE/ACM international conference on automated software engineering. 426–437.
  12. Analysis of covariance. Applied Statistics for the Behavioral Sciences. 4th ed. Boston, MA: Houghton Mifflin Company (1998), 516–547.
  13. An empirical study of model-agnostic techniques for defect prediction models. IEEE Transactions on Software Engineering 48, 1 (2020), 166–185.
  14. Practitioners’ perceptions of the goals and visual explanations of defect prediction models. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). IEEE, 432–443.
  15. Marcin Kawalerowicz and Lech Madeyski. 2023. Continuous build outcome prediction: an experimental evaluation and acceptance modelling. Applied Intelligence 53, 8 (2023), 8673–8692.
  16. Why do automated builds break? an empirical study. In 2014 IEEE International Conference on Software Maintenance and Evolution. IEEE, 41–50.
  17. Code review quality: How developers see it. In Proceedings of the 38th international conference on software engineering. 1028–1038.
  18. Studying pull request merges: A case study of shopify’s active merchant. In Proceedings of the 40th international conference on software engineering: software engineering in practice. 124–133.
  19. An empirical study of the impact of modern code review practices on software quality. Empirical Software Engineering 21 (2016), 2146–2189.
  20. Ansong Ni and Ming Li. 2017. Cost-effective build outcome prediction using cascaded classifiers. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 455–458.
  21. On the differences between unit and integration testing in the travistorrent dataset. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 451–454.
  22. Mohammad Masudur Rahman and Chanchal K Roy. 2017. Impact of continuous integration on code reviews. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 499–502.
  23. An empirical analysis of build failures in the continuous integration workflows of java-based open-source software. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 345–355.
  24. The impact of human factors on the participation decision of reviewers in modern code review. Empirical Software Engineering 24 (2019), 973–1016.
  25. Predicting continuous integration build failures using evolutionary search. Information and Software Technology 128 (2020), 106392.
  26. Improving the prediction of continuous integration build failures using deep learning. Automated Software Engineering 29, 1 (2022), 21.
  27. Investigating the impact of continuous integration practices on the productivity and quality of open-source projects. In Proceedings of the 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 137–147.
  28. Ray Schneider. 2008. Continuous integration: improving software quality and reducing risk. Software Quality Professional 10, 4 (2008), 51.
  29. Adrian Schröter. 2010. Predicting build outcome with developer interaction in jazz. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 2. 511–512.
  30. Programmers’ build errors: a case study (at google). In Proceedings of the 36th International Conference on Software Engineering. 724–734.
  31. Chakkrit Tantithamthavorn and Ahmed E Hassan. 2018. An experience report on defect modelling in practice: Pitfalls and challenges. In Proceedings of the 40th International conference on software engineering: Software engineering in practice. 286–295.
  32. Chakkrit Kla Tantithamthavorn and Jirayus Jiarpakdee. 2021. Explainable ai for software engineering. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1–2.
  33. Review participation in modern code review: An empirical study of the android, Qt, and OpenStack projects. Empirical Software Engineering 22 (2017), 768–817.
  34. Quality and productivity outcomes relating to continuous integration in GitHub. In Proceedings of the 2015 10th joint meeting on foundations of software engineering. 805–816.
  35. A tale of CI build failures: An open source and a financial organization perspective. In 2017 IEEE international conference on software maintenance and evolution (ICSME). IEEE, 183–193.
  36. Predicting build failures using social network analysis on developer communication. In 2009 IEEE 31st international conference on software engineering. IEEE, 1–11.
  37. Sarle WS. 1990. The VARCLUS Procedure. AppliThe VARCLUS Procedure, 4th edn. SAS Institute, Inc. (1990).
  38. Jing Xia and Yanhui Li. 2017. Could we predict the result of a continuous integration build? An empirical study. In 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE, 311–315.
  39. A study on the interplay between pull request review and continuous integration builds. In 2019 IEEE 26th international conference on software analysis, evolution and reengineering (SANER). IEEE, 38–48.
  40. The impact of continuous integration on other software development practices: a large-scale empirical study. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 60–71.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free