Software Engineering for AI-Based Systems: A Survey (2105.01984v2)

Abstract: AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.

Software Engineering for AI-Based Systems: A Survey

Software systems incorporating AI components, referred to as AI-based systems, have gained substantial traction in recent years. This paper titled "Software Engineering for AI-Based Systems: A Survey" addresses the nuances and complexities involved in engineering such systems, marking a significant intersection between software engineering (SE) and AI. The authors conducted a systematic mapping paper, analyzing a vast body of literature to identify key themes, challenges, and opportunities in software engineering for AI.

Key Findings

Bibliometrics and Study Characteristics

The paper observes a dramatic growth in publications addressing SE for AI-based systems, especially post-2015. Over the period from January 2010 to March 2020, the authors reviewed 248 studies, noting a substantial increase in academic and industry collaboration. Predominantly published conference papers and a notable presence on pre-print platforms like arXiv highlight the ongoing dynamic research pursuits in this field.

In terms of geographical distribution, the United States emerges as the leading contributor, followed by countries like Germany, China, and Japan, underscoring a strong North American and European presence. Research institutions such as IBM, Carnegie Mellon University, and Google are among key contributors, demonstrating significant industry engagement.

Scope and Characteristics of AI-Based Systems

AI-based systems cover a broad spectrum, from individual components to entire systems implementing AI technologies. There is a notable emphasis on deep learning models across various domains, with autonomous vehicles being the most prominent application area. While the survey finds that most research focuses on entire AI systems, addressing software components offers substantial opportunities for targeted improvements in their implementation.

Quality Attributes and Software Engineering Areas

Dependability, safety, and accuracy were identified as the most studied quality attributes for AI-based systems. This aligns with the prevalence of research focusing on software testing and quality assurance. Software testing, in particular, accounts for nearly half of the papers reviewed, illustrating a focus on methods and frameworks that ensure the robustness of AI-based systems against adversarial conditions and intrinsic uncertainties.

Notably, areas such as software maintenance and configuration management are underrepresented, signaling an opportunity for further exploration and investment. The authors advocate for more holistic research spanning various aspects of SE, aiming for integrated and comprehensive solutions that address lifecycle challenges.

Identified Challenges and Research Gaps

A recurrent theme throughout the survey is the unique challenges posed by AI-specific aspects, notably data-related issues in requirements specification, modeling, and testing. Effective integration of AI within SE processes calls for advancements in standard practices, particularly concerning standards and certifications like ISO 25000 and ISO 26262. The survey emphasizes the need for multidisciplinary collaboration to create new paradigms that elevate the engineering of AI systems beyond traditional software design and maintenance.

Implications and Future Directions

The evolving SE for AI-based systems landscape presents multiple directions for theoretical and practical development. The authors highlight several implications:

• There is a pressing need for updated frameworks and best practices that address the peculiarities of AI components, especially concerning safety standards.
• Educational initiatives should aim to bridge SE and AI domains, equipping engineers with necessary interdisciplinary skills for a seamless integration.
• As AI systems grow in complexity and deployment scale, ethical considerations and stakeholder interactions deserve rigorous focus, ensuring alignment with social norms and governance.

Ultimately, the paper provides a detailed, categorized overview of the current research trends and challenges within SE for AI-based systems, contributing significantly to foundational understanding and future research endeavors in this influential intersection of fields.