Cognition in Software Engineering: A Taxonomy and Survey of a Half-Century of Research

Abstract: Cognition plays a fundamental role in most software engineering activities. This article provides a taxonomy of cognitive concepts and a survey of the literature since the beginning of the Software Engineering discipline. The taxonomy comprises the top-level concepts of perception, attention, memory, cognitive load, reasoning, cognitive biases, knowledge, social cognition, cognitive control, and errors, and procedures to assess them both qualitatively and quantitatively. The taxonomy provides a useful tool to filter existing studies, classify new studies, and support researchers in getting familiar with a (sub) area. In the literature survey, we systematically collected and analysed 311 scientific papers spanning five decades and classified them using the cognitive concepts from the taxonomy. Our analysis shows that the most developed areas of research correspond to the four life-cycle stages, software requirements, design, construction, and maintenance. Most research is quantitative and focuses on knowledge, cognitive load, memory, and reasoning. Overall, the state of the art appears fragmented when viewed from the perspective of cognition. There is a lack of use of cognitive concepts that would represent a coherent picture of the cognitive processes active in specific tasks. Accordingly, we discuss the research gap in each cognitive concept and provide recommendations for future research.

• The paper presents a taxonomy categorizing key cognitive concepts and assessment methods in software engineering, reviewing 311 publications from 1973 to 2020.
• It finds that research increasingly focuses on cognitive load and biases, with quantitative methods predominating while revealing gaps in qualitative approaches.
• The study recommends expanding investigation in understudied areas like testing and quality and integrating cognitive science theories for deeper insights.

Cognition in Software Engineering: A Taxonomy and Survey

This essay outlines the paper "Cognition in Software Engineering: A Taxonomy and Survey of a Half-Century of Research" (2201.05551), which presents a comprehensive review of cognition-related research within software engineering. The paper develops a taxonomy of cognitive concepts, analyzes literature spanning five decades using this taxonomy, and discusses research gaps and future directions.

Taxonomy of Cognitive Concepts and Assessment Procedures

The taxonomy introduced in the paper categorizes cognitive concepts relevant to software engineering and enumerates methods for assessing these concepts.

Cognitive Concepts

1. Perception: Processes organizing sensory information to form an understanding of the environment.
2. Attention: Focused awareness on a subset of perceptual information, sub-categorized into selective, divided, and sustained attention.
3. Memory: Systems of acquiring and retaining information, covering working, short-term, and long-term memory.
4. Cognitive Load: The informational processing demands on attentional and working memory resources.
5. Reasoning: Goal-oriented thinking processes, including problem-solving and decision-making.
6. Cognitive Biases: Systematic errors stemming from heuristic simplifications of complex judgment tasks.
7. Knowledge: Framework of experiences and information guiding new evaluations, split into explicit and tacit knowledge.
8. Social Cognition: Cognitive activity mediating social behavior and acquisition of information from the social context.
9. Cognitive Control: Ensemble of cognitive processes governing behavior, including attentional control, cognitive inhibition, and cognitive flexibility.

10. Errors: Irregularities in rational results arising from perceptual distortions and inaccurate judgments.

Assessment Procedures

1. Qualitative Methods: Fieldwork, interviews, task-based observations, and open observations.
2. Quantitative Methods: Task performance, physiological measures, subjective ratings, and behavioral measures.

Literature Survey and Findings

The survey systematically analyzes 311 scientific publications from 1973 to 2020, using the taxonomy to classify and characterize the research landscape.

Trends and Context

• The research has continuously evolved, with increased focus on cognitive load and biases since the early 2000s.
• Most studies focus on software construction, design, requirements, and maintenance phases.
• Research context is fairly balanced between academic (student) and industrial settings.

  Figure 1: Distribution of publications per year. The data ranges from 1973 to May 2020.

SWEBOK Areas

• Software Construction and Design are most addressed; Testing and Quality are relatively understudied.
• Knowledge and Cognitive Load are frequently examined across lifecycle areas.

  Figure 2: Number of publications according to SWEBOK areas.

Assessment Methods

• Predominantly quantitative approaches; interviews and task-based observations serve as primary qualitative methods.
• Use of physiological and behavioral data offers potential for more nuanced insights.

  Figure 3: Research methods in SWEBOK areas.

Theoretical Implications

• Few papers explicitly use psychological theories; the conceptual basis is often ad-hoc or inspired by general cognitive ideas.
• Greater integration with cognitive science can enhance both theoretical grounding and practical application.

Discussion and Future Directions

The state of cognition-related research in software engineering exhibits fragmentation, reflecting a diverse array of cognitive aspects studied in isolation. Gaps identified in areas such as perception and long-term memory present avenues for future research. Leveraging mixed-method approaches and advanced assessment techniques, including physiological measures, can deepen insights into cognitive processes in software engineering.

Recommendations

• Expand research in understudied areas like software testing and quality using the provided taxonomy to bring structured clarity to cognitive processes.
• Enhance interdisciplinary collaboration to integrate established cognitive science theories more robustly.
• Employ unobtrusive and physiological measurements to enrich understanding of concepts like attention and cognitive load.

Conclusion

The paper provides a valuable contribution to organizing and understanding cognition in software engineering, offering a taxonomy that serves as a blueprint for future research efforts. By positioning cognitive concepts within operational contexts, it opens pathways for enhancing both theoretical understanding and practical decision-making in software engineering.