Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Large Scale Survey of Motivation in Software Development and Analysis of its Validity

Published 12 Apr 2024 in cs.SE and cs.LG | (2404.08303v1)

Abstract: Context: Motivation is known to improve performance. In software development in particular, there has been considerable interest in the motivation of contributors to open source. Objective: We identify 11 motivators from the literature (enjoying programming, ownership of code, learning, self use, etc.), and evaluate their relative effect on motivation. Since motivation is an internal subjective feeling, we also analyze the validity of the answers. Method: We conducted a survey with 66 questions on motivation which was completed by 521 developers. Most of the questions used an 11 point scale. We evaluated the validity of the answers validity by comparing related questions, comparing to actual behavior on GitHub, and comparison with the same developer in a follow up survey. Results: Validity problems include moderate correlations between answers to related questions, as well as self promotion and mistakes in the answers. Despite these problems, predictive analysis, investigating how diverse motivators influence the probability of high motivation, provided valuable insights. The correlations between the different motivators are low, implying their independence. High values in all 11 motivators predict increased probability of high motivation. In addition, improvement analysis shows that an increase in most motivators predicts an increase in general motivation.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (102)
  1. Github user search showing 37,446,292 available users. URL https://github.com/search?q=type:user&type=Users. Accessed: 2019-05-23.
  2. Sampling: Why and how of it. Indian Journal of Medical Specialties, 4(2):330–333, 2013. doi:10.7713/ijms.2013.0032.
  3. J. S. Adams. Toward an understanding of inequity. Journal of Abnormal Psychology, 67(5):422–436, 1963. doi:10.1037/h0040968.
  4. The work preference inventory: Assessing intrinsic and extrinsic motivational orientations. Journal of Personality and Social Psychology, 66(5):950–967, 1994. doi:10.1037/0022-3514.66.5.950.
  5. I. Amit. End to end software engineering research. 2021, arXiv:2112.11858 [cs.SE].
  6. Follow your nose – which code smells are worth chasing? 2021, arXiv:2103.01861 [cs.SE].
  7. I. Amit and D. G. Feitelson. Which refactoring reduces bug rate? In Proceedings of the 15th International Conference on Predictive Models and Data Analytics in Software Engineering, PROMISE’19, pages 12–15, New York, NY, USA, 2019. Association for Computing Machinery. doi:10.1145/3345629.3345631.
  8. I. Amit and D. G. Feitelson. Corrective commit probability: A measure of the effort invested in bug fixing. Software Quality Journal, 29(4):817–861, Aug 2021. doi:10.1007/s11219-021-09564-z.
  9. I. Amit and D. G. Feitelson. Replication package https://github.com/evidencebp/motivation-survey, Dec 2023.
  10. Framework for semi-supervised learning when no labeled data is given. U.S. patent #US11468358B2, 2017. URL https://patents.google.com/patent/US11468358B2.
  11. M. Argyle. Do happy workers work harder? the effect of job satisfaction on job performance. In R. Veenhoven, editor, How harmful is happiness? Consequences of enjoying life or not, pages 94–105. Universitaire Pers, Rotterdam, The Netherlands, 1989.
  12. Man’s search for meaning: The case of legos. Journal of Economic Behavior & Organization, 67(3-4):671–677, 2008. doi:10.1016/j.jebo.2008.01.004.
  13. A closer look at memorization in deep networks. In 34th International Conference on Machine Learning, pages 233–242. PMLR, 2017.
  14. G. Asproni. Motivation, teamwork, and agile development. Agile Times, 4, Mar 2004.
  15. N. Baddoo and T. Hall. Motivators of software process improvement: an analysis of practitioners’ views. Journal of Systems and Software, 62(2):85–96, 2002. doi:10.1016/S0164-1212(01)00125-X.
  16. Managing information systems personnel: A review of the literature and managerial implications. MIS Quarterly, 6(4):49–70, Dec 1982. URL http://dl.acm.org/citation.cfm?id=2025521.2025525.
  17. N. Bassett‐Jones and G. C. Lloyd. Does herzberg’s motivation theory have staying power? Journal of Management Development, 24:929–943, Dec 2005. doi:10.1108/02621710510627064.
  18. An empirical investigation of perceptions of ratee behavior frequency and ratee behavior change using behavioral expectation scales (BES). Personnel Psychology, 30(4):647–658, 1977. doi:10.1111/j.1744-6570.1977.tb02333.x.
  19. Working overtime hours: relations with fatigue, work motivation, and the quality of work. Journal of Occupational and Environmental Medicine, pages 1282–1289, 2004.
  20. Motivation in software engineering: A systematic literature review. Information & Software Technology, 50(9-10):860–878, 2008. doi:10.1016/j.infsof.2007.09.004.
  21. J. K. Beggan. On the social nature of nonsocial perception: The mere ownership effect. Journal of Personality and Social Psychology, 62(2):229, 1992. doi:10.1037/0022-3514.62.2.229.
  22. S. Belenzon and M. Schankerman. Motivation and Sorting in Open Source Software Innovation. CEP Discussion Papers dp0893, Centre for Economic Performance, LSE, Oct 2008. URL https://ideas.repec.org/p/cep/cepdps/dp0893.html.
  23. R. Bénabou and J. Tirole. Intrinsic and extrinsic motivation. The Review of Economic Studies, 70(3):489–520, 2003.
  24. D. J. Campbell and C. Lee. Self-appraisal in performance evaluation: Development versus evaluation. The Academy of Management Review, 13(2):302–314, 1988. doi:10.2307/258579.
  25. A theory of performance. In N. Schmitt, W. C. Borman, and Associates, editors, Personnel Selection in Organizations, pages 35–70. Jossey-Bass Pub., 1993.
  26. Incivility in the workplace: Incidence and impact. Journal of Occupational Health Psychology, 6:64–80, Feb 2001. doi:10.1037/1076-8998.6.1.64.
  27. J. D. Couger. Motivators vs. demotivators in the is environment. Journal of Systems Management, 39(6):36, 1988.
  28. A. B. Cristina and C. Rossi. Altruistic individuals, selfish firms? the structure of motivation in open source software. First Monday, 9:9, 2004. doi:10.5210/fm.v9i1.1113.
  29. T. DeMarco and T. Lister. Peopleware: Productive Projects and Teams. Dorset House, 1987.
  30. Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1):1–4, 2016. doi:10.11648/j.ajtas.20160501.11.
  31. Measuring programming experience. In 20th IEEE International Conference on Program Comprehension, pages 73–82, 2012. doi:10.1109/ICPC.2012.6240511.
  32. Turnover in open-source projects: The case of core developers. In Proceedings of the 34th Brazilian Symposium on Software Engineering, pages 447–456, 2020. doi:10.1145/3422392.3422433.
  33. S. A. Frangos. Motivated humans for reliable software products. In D. Gritzalis, editor, Reliability, Quality and Safety of Software-Intensive Systems: IFIP TC5 WG5.4 3rd International Conference on Reliability, Quality and Safety of Software-Intensive Systems, pages 83–91. Springer US, Boston, MA, May 1997. doi:10.1007/978-0-387-35097-4_7.
  34. Motivation and satisfaction of software engineers. IEEE Transactions on Software Engineering, 46(2):118–140, Feb 2020. doi:10.1109/TSE.2018.2842201.
  35. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, 1997. doi:10.1006/jcss.1997.1504.
  36. The shifting sands of motivation: Revisiting what drives contributors in open source. In Proceedings of the 43rd International Conference on Software Engineering, pages 1046–1058, 2021. doi:10.1109/ICSE43902.2021.00098.
  37. The impact of motivator and demotivator factors on agile software development. International Journal of Advanced Computer Science and Applications, 9(7), 2018. doi:10.14569/IJACSA.2018.090712.
  38. A. Grant. The significance of task significance: Job performance effects, relational mechanisms, and boundary conditions. The Journal of Applied Psychology, 93:108–24, Feb 2008. doi:10.1037/0021-9010.93.1.108.
  39. What happens when software developers are (un)happy. Journal of Systems and Software, 140:32–47, 2018. doi:10.1016/j.jss.2018.02.041.
  40. Motivation Through the Design of Work: Test of a Theory. Academic Press, New York, 1976.
  41. A fine-grained data set and analysis of tangling in bug fixing commits. Empirical Software Engineering, 27(6):1–49, 2022. doi:10.1007/s10664-021-10083-5.
  42. Motivation of software developers in open source projects: an internet-based survey of contributors to the linux kernel. Research Policy, 32(7):1159–1177, 2003. doi:10.1016/S0048-7333(03)00047-7. Open Source Software Development.
  43. F. Herzberg. One more time: How do you motivate employees? New York: The Leader Manager, pages 433–448, 1986.
  44. Motivation to Work. Wiley, New York, 1959.
  45. Validation of a job satisfaction scale in the australian clinical medical workforce. Evaluation & the Health Professions, 35:47–76, Mar 2011. doi:10.1177/0163278710397339.
  46. Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4):396, 2015. doi:10.9734/BJAST/2015/14975.
  47. The job satisfaction-job performance relationship: a qualitative and quantitative review. Psychological Bulletin, 127:376–407, Jan 2001. doi:10.1037/0033-2909.127.3.376.
  48. Dispositional effects on job and life satisfaction: The role of core evaluations. Journal of Applied Psychology, 83:17–34, 1998. doi:10.1037/0021-9010.83.1.17.
  49. A. Kluger and O. Bouskila‐Yam. Facilitating listening scale (FLS). In D. L. Worthington and G. D. Bodie, editors, The Sourcebook of Listening Research: Methodology and Measures, pages 272–280. Wiley Online Library, Aug 2017. doi:10.1002/9781119102991.ch25.
  50. J. Kruger and D. Dunning. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77:1121–1134, 1999. doi:10.1037/0022-3514.77.6.1121.
  51. Flow, intrinsic motivation, and developer experience in software engineering. In H. Sharp and T. Hall, editors, Agile Processes in Software Engineering and Extreme Programming, pages 104–117. Springer International Publishing, 2016. doi:10.1007/978-3-319-33515-5_9.
  52. Self-determined work motivation predicts job outcomes, but what predicts self-determined work motivation? Journal of Research in Personality, 42(4):1109–1115, 2008. doi:10.1016/j.jrp.2008.02.002.
  53. F. J. Landy. Psychology of work behavior. Thomson Brooks/Cole Publishing Co, Belmont, CA, US, 4st edition, 1989.
  54. Deep learning. Nature, 521(7553):436–444, 2015. doi:10.1038/nature14539.
  55. Behavioral software engineering: A definition and systematic literature review. Journal of Systems and Software, 107:15–37, 2015. doi:10.1016/j.jss.2015.04.084.
  56. J. Lerner and J. Tirole. Some simple economics of open-source. Journal of Industrial Economics, 50:197–234, Feb 2002. URL http://www.people.hbs.edu/jlerner/simple.pdf.
  57. Motivating open source software developers: Influence of transformational and transactional leaderships. In Proceedings of the 2006 ACM SIGMIS CPR Conference on Computer Personnel Research: Forty Four Years of Computer Personnel Research: Achievements, Challenges & the Future, pages 34–43, Jan 2006. doi:10.1145/1125170.1125182.
  58. R. Likert. A technique for the measurement of attitudes. Archives of Psychology, 22:55, 1932.
  59. E. A. Locke. Toward a theory of task motivation and incentives. Organizational Behavior and Human Performance, 3(2):157–189, 1968. doi:10.1016/0030-5073(68)90004-4.
  60. Maslach burnout inventory third edition. In C. P. Zalaquett and R. J. Wood, editors, Evaluating Stress: A Book of Resources, pages 191–218. Scarecrow Education, 1997.
  61. A. H. Maslow. A theory of human motivation. Psychological Review, 50:370–396, 1943.
  62. Seventy-five years of motivation measures (1930–2005): A descriptive analysis. Motivation and Emotion, 31:83–103, 2007.
  63. D. C. McClelland. The Achieving Society. Van Nostrand, Princeton, NJ, 1961.
  64. Today was a good day: The daily life of software developers. IEEE Transactions on Software Engineering, 47(5):863–880, 2021. doi:10.1109/TSE.2019.2904957.
  65. What predicts software developers’ productivity? IEEE Transactions on Software Engineering, 47(3):582–594, 2021. doi:10.1109/TSE.2019.2900308.
  66. F. Murtagh and P. Contreras. Algorithms for hierarchical clustering: an overview. WIREs Data Mining and Knowledge Discovery, 2(1):86–97, 2012. doi:10.1002/widm.53.
  67. Code and commit metrics of developer productivity: a study on team leaders perceptions. Empirical Software Engineering, 25(4):2519–2549, Jul 2020. doi:10.1007/s10664-020-09820-z.
  68. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011. URL http://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf.
  69. Common method biases in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5):879, 2003. doi:10.1037/0021-9010.88.5.879.
  70. J. R. Quinlan. C4. 5: programs for machine learning. Elsevier, 2014.
  71. S. Ramachandran and S. V. Rao. An effort towards identifying occupational culture among information systems professionals. In Proceedings of the 2006 ACM SIGMIS CPR Conference on Computer Personnel Research: Forty Four Years of Computer Personnel Research: Achievements, Challenges & the Future, pages 198–204, 2006. doi:10.1145/1125170.1125221.
  72. Stress and burnout in open source: Toward finding, understanding, and mitigating unhealthy interactions. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results, pages 57–60, 2020. doi:10.1145/3377816.3381732.
  73. Data programming: Creating large training sets, quickly. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 3567–3575. Curran Associates, Inc., 2016. URL http://papers.nips.cc/paper/6523-data-programming-creating-large-training-sets-quickly.pdf.
  74. E. S. Raymond. The Cathedral and the Bazaar. O’Reilly & Associates, Inc., Sebastopol, CA, USA, 1st edition, 1999.
  75. D. Riehle. The economic motivation of open source software: Stakeholder perspectives. Computer, 40(4):25–32, 2007. doi:10.1109/MC.2007.147.
  76. Understanding the motivations, participation, and performance of open source software developers: A longitudinal study of the apache projects. Manage. Sci., 52(7):984–999, Jul 2006. doi:10.1287/mnsc.1060.0554.
  77. Floss 2013: A survey dataset about free software contributors: Challenges for curating, sharing, and combining. In Proceedings of the 11th Working Conference on Mining Software Repositories, pages 396–399, 2014. doi:10.1145/2597073.2597129.
  78. Management and the worker. Harvard Univ. Press, Oxford, England, 1939.
  79. A. S. Tsui. A role set analysis of managerial reputation. Academy of Management Proceedings, 1982:265–269, Aug 1982. doi:10.5465/AMBPP.1982.4976642.
  80. Employee surveys are still one of the best ways to measure engagement. Harvard Business Review, March 2018. URL https://hbr.org/2018/03/employee-surveys-are-still-one-of-the-best-ways-to-measure-engagement.
  81. Understanding the latent structure of job performance ratings. The Journal of Applied Psychology, 85:956–70, Jan 2001. doi:10.1037//0021-9010.85.6.956.
  82. Models of motivation in software engineering. Information & Software Technology, 51(1):219–233, 2009. doi:10.1016/j.infsof.2008.05.009.
  83. H. Sharp and T. Hall. An initial investigation of software practitioners’ motivation. In Proceedings of the ICSE Workshop on Cooperative and Human Aspects on Software Engineering, pages 84–91, 2009. doi:10.1109/CHASE.2009.5071418.
  84. H. Shavit and R. Shouval. Self-esteem and cognitive consistency effects on self-other evaluation. Journal of Experimental Social Psychology, 16(5):417–425, 1980. doi:10.1016/0022-1031(80)90048-7.
  85. J. A. Shepperd. Productivity loss in performance groups: A motivation analysis. Psychological Bulletin, 113(1):67, 1993. doi:10.1037/0033-2909.113.1.67.
  86. B. F. Skinner. The Behavior of Organisms: An Experimental Analysis. New York: Appleton-Century-Crofts, 1938.
  87. Stackoverflow. 2021 annual developer survey, 2021. URL https://insights.stackoverflow.com/survey/2021.
  88. R. Stallman. Why software should be free. 2007. URL https://www.gnu.org/philosophy/shouldbefree.en.html.
  89. F. R. Tanner. On motivating engineers. In IEMC ’03 Proceedings. Managing Technologically Driven Organizations: The Human Side of Innovation and Change, pages 214–218, Nov 2003. doi:10.1109/IEMC.2003.1252263.
  90. M. Teghtsoonian and R. Teghtsoonian. How repeatable are stevens’s power law exponents for individual subjects? Perception & Psychophysics, 10(3):147–149, 1971. doi:10.3758/BF03205774.
  91. Gender differences and bias in open source: Pull request acceptance of women versus men. PeerJ Computer Science, 3:e111, 2017. doi:10.7717/peerj-cs.111.
  92. V. Vapnik. The Nature of Statistical Learning Theory. Springer science & business media, 2013.
  93. V. Vapnik and A. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability & Its Applications, 16(2):264–280, 1971. doi:10.1137/1116025.
  94. P. F. Verhulst. Resherches mathematiques sur la loi d’accroissement de la population. Nouveaux Memoires de l’Academie Royale des Sciences, 18:1–41, 1845.
  95. Comparative analysis of reliability of job performance ratings. Journal of Applied Psychology, 81:557–574, Oct 1996. doi:10.1037/0021-9010.81.5.557.
  96. Carrots and rainbows: Motivation and social practice in open source software development. MIS Quarterly, 36(2):649–676, 2012. URL http://www.jstor.org/stable/41703471.
  97. V. H. Vroom. Work and Motivation. Wiley, New York, 1964.
  98. Systematic literature review of machine learning based software development effort estimation models. Information and Software Technology, 54(1):41–59, 2012. doi:10.1016/j.infsof.2011.09.002.
  99. B. Wigert and J. Harter. Re-engineering performance management. Gallup.com, 2019. URL http://news.gallup.com/reports/208811/re-engineering-performance-management.aspx.
  100. T. A. Wright and R. Cropanzano. Psychological well-being and job satisfaction as predictors of job performance. Journal of Occupational Health Psychology, 5:84–94, 2000. doi:10.1037/1076-8998.5.1.84.
  101. Y. Ye and K. Kishida. Toward an understanding of the motivation open source software developers. In Proceedings of the 25th International Conference on Software Engineering, pages 419–429, 2003. doi:10.1109/ICSE.2003.1201220.
  102. B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 694–699, 2002. doi:10.1145/775047.775151.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free