Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices (1703.07019v1)

Abstract: Context: Continuous practices, i.e., continuous integration, delivery, and deployment, are the software development industry practices that enable organizations to frequently and reliably release new features and products. With the increasing interest in and literature on continuous practices, it is important to systematically review and synthesize the approaches, tools, challenges, and practices reported for adopting and implementing continuous practices. Objective: This research aimed at systematically reviewing the state of the art of continuous practices to classify approaches and tools, identify challenges and practices in this regard, and identify the gaps for future research. Method: We used systematic literature review (SLR) method for reviewing the peer-reviewed papers on continuous practices published between 2004 and 1st June 2016. We applied thematic analysis method for analysing the data extracted from reviewing 69 papers selected using predefined criteria. Results: We have identified thirty approaches and associated tools, which facilitate the implementation of continuous practices in the following ways: (1) "reducing build and test time in continuous integration (CI)"; (2) "increasing visibility and awareness on build and test results in CI"; (3) "supporting (semi-) automated continuous testing"; (4) "detecting violations, flaws and faults in CI"; (5) "addressing security and scalability issues in deployment pipeline", and (6) "improving dependability and reliability of deployment process". We have also determined a list of critical factors such as "testing (effort and time)", "team awareness and transparency", "good design principles", "customer", "highly skilled and motivated team", "application domain", and "appropriate infrastructure" that should be carefully considered when introducing continuous practices in a given organization.

• The paper employs a systematic literature review of 69 studies to reveal 30 principal approaches and tools for continuous integration, delivery, and deployment.
• The paper identifies key challenges such as communication gaps, architectural constraints, and resistance to change in adopting these continuous practices.
• The paper outlines effective practices like incremental changes and automation that enhance software delivery speed and reliability.

Systematic Review of Continuous Integration, Delivery and Deployment: Approaches, Tools, Challenges, and Practices

This paper provides a systematic literature review of continuous integration (CI), delivery (CDE), and deployment (CD) practices within the software engineering field. Managed by professionals such as Mojtaba Shahin, Muhammad Ali Babar, and Liming Zhu, this comprehensive paper offers insight into the methodologies employed by software development practitioners to implement these practices effectively. With an interest in transforming software delivery paradigms by improving frequency and reliability, the paper identifies key areas in the current literature, offering useful guidance for future developments.

Methodology and Goals

Utilizing a systematic literature review (SLR) method, the researchers scrutinized peer-reviewed papers from 2004 to mid-2016, employing thematic analysis to delve into 69 selected studies. This methodical approach served to identify prevailing approaches, tools, and challenges associated with continuous practices and highlighted existing gaps. The authors sought to classify and synthesize the findings, providing a structured overview of the continuous practices landscape while aiming to derive pivotal questions and directions for upcoming studies.

Key Findings

1. Approaches and Tools: The paper outlines thirty primary approaches and their associated tools which assist in implementing continuous integration, delivery, and deployment. These tools are instrumental in tasks like reducing build and test time, increasing project visibility, supporting automated testing, and managing security.
2. Challenges: It identifies several challenges that software organizations face, emphasizing common obstacles like lack of team awareness, communication troubles, insufficient investment in tools and skills, and resistance to change. Of particular significance are the architectural and infrastructural needs that can complicate adoption efforts amidst rapidly changing technological landscapes.
3. Practices: A notable portion of the synthesis includes practices that have proven beneficial in deploying continuous practices. These include refining team structures, establishing effective communication protocols, and executing incremental changes while taking advantage of automation and feedback loops.

Implications and Contributions

The paper's findings underscore CI, CDE, and CD as critical components in modern software engineering, facilitating quicker releases and promoting enhanced customer satisfaction. Importantly, it addresses both the technical and organizational elements necessary for successful implementation, recommending a cohesive approach combining technological tools and human factors like skills training and process transparency.

Moreover, this systematic review contributes to software engineering literature by mapping out the current state of continuous practices, thereby serving as a valuable reference for both researchers and practitioners looking to navigate the complex landscape of software delivery and deployment. The paper also points toward specific areas needing further exploration — such as securing deployment pipelines and optimizing architecture for better deployability — urging future research to bridge these gaps.

Future Directions

Future research could focus on refining deployment pipeline security, considering it a relatively underexplored aspect in the surveyed literature. In addition, studying the interplay between different architectural patterns and their impact on continuous practices could yield valuable insights. Addressing these areas would enhance understanding and capability within continuous integration, delivery, and deployment, further advancing the efficacy and agility of software development methodologies.

In conclusion, the systematic review by Shahin et al. presents a thorough and insightful examination of continuous practices in software engineering. It outlines existing capabilities while simultaneously providing a roadmap for overcoming current challenges and paving the way for further advancements in the field.