Agile Effort Estimation: Comparing the Accuracy and Efficiency of Planning Poker, Bucket System, and Affinity Estimation methods

Abstract: Published studies on agile effort estimation predominantly focus on comparisons of the accuracy of different estimation methods, while efficiency comparisons, i.e. how much time the estimation methods consume was not in the forefront. However, for practical use in software development, the time required can be a very important cost factor for enterprises, especially when the accuracy of different agile effort estimations is similar. In this study, we thus try to advance the current standard accuracy comparison between methods by introducing efficiency i.e. time it takes to use a method as an additional dimension of comparison. We conduct this comparison between three agile effort estimation methods that were not yet compared in the literature, namely Planning Poker, Bucket System and Affinity Estimation. For the comparison, we used eight student teams with 29 students that had to use all the effort estimation methods during the course where they had to finish a programming project in 3 weeks. The results indicate that after the students get used to using the different methods the accuracy between them is not statistically significantly different, however, the efficiency is. On average Bucket System and Affinity Estimation methods take half as much time as Planning Poker.