Exploring the Impact of Source Code Linearity on the Programmers Comprehension of API Code Examples (2404.02377v1)
Abstract: Context: Application Programming Interface (API) code examples are an essential knowledge resource for learning APIs. However, a few user studies have explored how the structural characteristics of the source code in code examples impact their comprehensibility and reusability. Objectives: We investigated whether the (a) linearity and (b) length of the source code in API code examples affect users performance in terms of correctness and time spent. We also collected subjective ratings. Methods: We conducted an online controlled code comprehension experiment with 61 Java developers. As a case study, we used the API code examples from the Joda-Time Java library. We had participants perform code comprehension and reuse tasks on variants of the example with different lengths and degrees of linearity. Findings: Participants demonstrated faster reaction times when exposed to linear code examples. However, no substantial differences in correctness or subjective ratings were observed. Implications: Our findings suggest that the linear presentation of a source code may enhance initial example understanding and reusability. This, in turn, may provide API developers with some insights into the effective structuring of their API code examples. However, we highlight the need for further investigation.
- 2023. Gorilla Experiment Builder. https://gorilla.sc
- Syntax, predicates, idioms — what really affects code complexity? Empirical Software Engineering 24, 1 (2 2019), 287–328. https://doi.org/10.1007/s10664-018-9628-3
- Synthesising Linear API Usage Examples for API Documentation. In 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME). 607–611.
- Eran Avidan and Dror G. Feitelson. 2017. Effects of Variable Names on Comprehension: An Empirical Study. In IEEE 25th International Conference on Program Comprehension. IEEE Computer Society, 55–65. https://doi.org/10.1109/ICPC.2017.27
- The impact of identifier style on effort and comprehension. Empirical Software Engineering 18, 2 (4 2013), 219–276. https://doi.org/10.1007/s10664-012-9201-4
- Does Code Structure Affect Comprehension? On Using and Naming Intermediate Variables. In IEEE International Conference on Program Comprehension, Vol. 2021-May. IEEE Computer Society, 118–126. https://doi.org/10.1109/ICPC52881.2021.00020
- Do you really code? Designing and evaluating screening questions for online surveys with programmers. In Proceedings - International Conference on Software Engineering. IEEE Computer Society, 537–548.
- Measuring Programming Experience. In 2012 20th IEEE International Conference on Program Comprehension. IEEE, 73–82. https://doi.org/10.1109/ICPC.2012.6240511
- Dror G. Feitelson. 2021. Considerations and Pitfalls in Controlled Experiments on Code Comprehension. In IEEE/ACM 29th International Conference on Program Comprehension (ICPC). 106–117. https://doi.org/10.1109/ICPC52881.2021.00019
- Exploring Programmers’ API Learning Processes: Collecting Web Resources as External Memory. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–10.
- Yorai Geffen and Shahar Maoz. 2016. On method ordering. In IEEE International Conference on Program Comprehension, Vol. 2016-July. IEEE Computer Society, 1–10. https://doi.org/10.1109/ICPC.2016.7503711
- Shorter identifier names take longer to comprehend. Empirical Software Engineering 24, 1 (2 2019), 417–443. https://doi.org/10.1007/s10664-018-9621-x
- Levene Howard. 1960. Robust tests for equality of variances. Contributions to probability and statistics (1960), 278–292.
- Ahmad Jbara and Dror G. Feitelson. 2014. On the effect of code regularity on comprehension. In 22nd International Conference on Program Comprehension (ICPC 2014). Association for Computing Machinery, 189–200. https://doi.org/10.1145/2597008.2597140
- Mental models and software maintenance. Journal of Systems and Software 7, 4 (1987), 341–355. https://www.sciencedirect.com/science/article/pii/0164121287900331
- Application Programming Interface Documentation: What Do Software Developers Want? Journal of Technical Writing and Communication 48, 3 (2018), 295–330. https://doi.org/10.1177/0047281617721853
- What drives the reading order of programmers? an eye tracking study. In 28th International Conference on Program Comprehension (ICPC ’20). ACM, 342–353. https://doi.org/10.1145/3387904.3389279
- Software Engineering User Study Recruitment on Prolific: An Experience Report. In International Workshop on Recruiting Participants for Empirical Software Engineering (RoPES’22), Vol. 5. Association for Computing Machinery.
- Daniel Russo. 2022. Recruiting Software Engineers on Prolific. In the1st International Workshop on Recruiting Participants for Empirical Software Engineering (RoPES 2022), Vol. 5. Association for Computing Machinery.
- Samuel Shapiro and Martin Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3/4 (1965), 591–611.
- Bonita Sharif and Jonathan I. Maletic. 2010. An eye tracking study on camelcase and under-score identifier styles. In IEEE 18th International Conference on Program Comprehension. 196–205. https://doi.org/10.1109/ICPC.2010.41
- Measuring and modeling programming experience. Empirical Software Engineering 19, 5 (2014), 1299–1334. https://doi.org/10.1007/s10664-013-9286-4
- Sören Sparman and Carsten Schulte. 2023. Analysing the API learning process through the use of eye tracking. In the 2023 Symposium on Eye Tracking Research and Applications (ETRA). 1–6.
- Anneliese Von Mayrhauser and A. Marie Vans. 1998. Program Understanding Behavior During Adaptation of Large Scale Software. In the 6th International Workshop on Program Comprehension (IWPC’98). 164–172.